Image processing device and method

Abstract

To make it possible to suppress increases in memory capacity required for an orthogonal transformation and an inverse orthogonal transformation.SOLUTION: A first transformation matrix is used to derive a second transformation matrix, the derived second transformation matrix is used to perform an orthogonal transformation of a predicted residual of an image, coefficient data obtained by performing an orthogonal transformation of the predicted residual is encoded, and a bit stream is generated. This disclosure can be applied to, for example, an image processing device, an image encoding device, an image decoding device, or the like.SELECTED DRAWING: Figure 10

Description

[Technical field]

[0001] The present disclosure relates to an image processing device and method, and more particularly to an image processing device and method capable of suppressing an increase in memory capacity required for orthogonal transform and inverse orthogonal transform. [Background technology]

[0002] Conventionally, for luminance, adaptive primary transforms (AMT: Adaptive Multiple Core Transforms) have been disclosed that adaptively select a primary transform from a plurality of different orthogonal transforms for each horizontal primary transform PThor (also referred to as a primary horizontal transform) and each vertical primary transform PTver (also referred to as a primary vertical transform) for each TU (Transform Unit) (see, for example, Non-Patent Document 1).

[0003] In Non-Patent Document 1, five one-dimensional orthogonal transforms, DCT-II, DST-VII, DCT-VIII, DST-I, and DCT-VI, are candidates for the primary transform. In addition, it is proposed to add two more one-dimensional orthogonal transforms, DST-IV and IDT (Identity Transform: one-dimensional transform skip), making a total of seven one-dimensional orthogonal transforms candidates for the primary transform (see, for example, Non-Patent Document 2). [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Jianle Chen, Elena Alshina, Gary J. Sullivan, Jens-Rainer, Jill Boyce, "Algorithm Description of Joint Exploration Test Model 4", JVET-G1001_v1, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 7th Meeting: Torino, IT, 13-21 July 2017 [Non-Patent Document 2] V. Lorcy, P. Philippe, "Proposed improvements to the Adaptive multiple Core transform", JVET-C0022, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 3rd Meeting: Geneva, CH, 26 May - 1 June 2016 Summary of the Invention [Problem to be solved by the invention]

[0005] However, these methods have the risk of increasing the size of the LUT (Look Up Table) required to store all the transformation matrices of the primary transformation, i.e., when considering hardware implementation of the primary transformation, there is a risk of increasing the memory size required to store the coefficients of the transformation matrices.

[0006] The present disclosure has been made in consideration of such circumstances, and makes it possible to suppress an increase in memory capacity required for orthogonal transform and inverse orthogonal transform. [Means for solving the problem]

[0007] An image processing device according to one aspect of the present technology is an image processing device including: a derivation unit that derives a second transformation matrix using a first transformation matrix; an orthogonal transformation unit that orthogonally transforms a prediction residual of an image using the second transformation matrix derived by the derivation unit; and an encoding unit that encodes coefficient data obtained by orthogonally transforming the prediction residual by the orthogonal transformation unit, and generates a bitstream.

[0008] An image processing method according to one aspect of the present technology includes deriving a second transformation matrix using a first transformation matrix, orthogonally transforming a prediction residual of an image using the derived second transformation matrix, encoding coefficient data obtained by orthogonally transforming the prediction residual, and generating a bitstream.

[0009] An image processing device according to another aspect of the present technology is an image processing device including a decoding unit that decodes a bitstream and obtains coefficient data obtained by orthogonal transforming a prediction residual of an image, a derivation unit that derives a second transform matrix using a first transform matrix, and an inverse orthogonal transform unit that performs an inverse orthogonal transform on the coefficient data obtained by the decoding unit using the second transform matrix derived by the derivation unit.

[0010] An image processing method according to another aspect of the present technology is an image processing method that decodes a bitstream, obtains coefficient data obtained by orthogonally transforming a prediction residual of an image, derives a second transform matrix using a first transform matrix, and performs an inverse orthogonal transform on the obtained coefficient data using the derived second transform matrix.

[0011] According to yet another aspect of the present technology, an image processing device includes an operation unit that performs a substitution operation on a prediction residual of an image, an orthogonal transformation unit that performs an orthogonal transform on the prediction residual that has been replaced by the operation unit using a base transformation matrix, and an encoding unit that encodes coefficient data obtained by orthogonally transforming the prediction residual by the orthogonal transformation unit, and generates a bit stream.

[0012] An image processing method according to yet another aspect of the present technology is an image processing method that performs a substitution operation on a prediction residual of an image, orthogonally transforms the prediction residual that has been substituted using a base transformation matrix, encodes coefficient data obtained by orthogonally transforming the prediction residual, and generates a bitstream.

[0013] An image processing device according to yet another aspect of the present technology is an image processing device including a decoding unit that decodes a bit stream and obtains coefficient data obtained by orthogonal transform of a prediction residual of an image, an inverse orthogonal transform unit that performs an inverse orthogonal transform on the coefficient data obtained by the decoding unit, and an operation unit that performs a substitution operation on a result of the inverse orthogonal transform of the coefficient data obtained by the inverse orthogonal transform unit.

[0014] An image processing method according to yet another aspect of the present technology is an image processing method that decodes a bitstream, obtains coefficient data obtained by orthogonal transforming a prediction residual of an image, performs an inverse orthogonal transform on the obtained coefficient data, and performs a substitution operation on a result of the inverse orthogonal transform of the obtained coefficient data.

[0015] In an image processing device and method according to one aspect of the present technology, a first transformation matrix is ​​used to derive a second transformation matrix, and the derived second transformation matrix is ​​used to orthogonally transform a prediction residual of an image, and coefficient data obtained by orthogonally transforming the prediction residual is encoded to generate a bitstream.

[0016] In an image processing device and method according to another aspect of the present technology, a bitstream is decoded, coefficient data is obtained by orthogonally transforming a prediction residual of an image, a first transform matrix is ​​used to derive a second transform matrix, and the derived second transform matrix is ​​used to inversely orthogonally transform the obtained coefficient data.

[0017] In an image processing device and method according to still another aspect of the present technology, a prediction residual of an image is subjected to a substitution operation, a base transformation matrix is ​​used to perform an orthogonal transformation on the prediction residual after the substitution operation, and coefficient data obtained by orthogonal transformation of the prediction residual is encoded to generate a bitstream.

[0018] In an image processing device and method according to still another aspect of the present technology, a bitstream is decoded, coefficient data is obtained by orthogonally transforming a prediction residual of an image, the obtained coefficient data is inversely orthogonally transformed, and a substitution operation is performed on the result of the inverse orthogonal transform of the obtained coefficient data. Effect of the Invention

[0019] According to the present disclosure, it is possible to process images, and in particular to suppress an increase in memory capacity required for orthogonal transformation and inverse orthogonal transformation. [Brief description of the drawings]

[0020] [Figure 1] FIG. 13 is a diagram showing a correspondence relationship between a transform set and a selected orthogonal transform. [Diagram 2] FIG. 11 is a diagram showing the correspondence between types of orthogonal transformation and functions used. [Diagram 3] FIG. 13 is a diagram showing a correspondence relationship between a transform set and a prediction mode. [Figure 4] FIG. 13 is a diagram illustrating an example of a type of orthogonal transform stored in the LUT. [Diagram 5] FIG. 1 is a diagram showing an example of an LUT size required to hold a transformation matrix in HEVC. [Figure 6] 11A and 11B are diagrams illustrating an example of an LUT size required to hold a transformation matrix. [Figure 7] 1A and 1B are diagrams illustrating examples of similarity between transformation matrices. [Figure 8] FIG. 13 is a diagram illustrating examples of transformation types that can be replaced by a flip. [Figure 9] FIG. 13 is a diagram illustrating an example of a transformation type that can be replaced by a transposition. [Figure 10] FIG. 11 is a diagram showing a list of main concrete examples of alternative transformation matrices. [Figure 11] FIG. 1 is a block diagram showing an example of the main configuration of an image encoding device. [Figure 12] 11 is a block diagram showing an example of the main configuration of an orthogonal transformation unit. FIG. [Figure 13]11 is a flowchart illustrating an example of the flow of an image encoding process. [Figure 14] 11 is a flowchart illustrating an example of the flow of an orthogonal transform process. [Figure 15] FIG. 2 is a block diagram showing an example of the main configuration of an image decoding device. [Figure 16] 11 is a block diagram showing an example of the main configuration of an inverse orthogonal transform unit. FIG. [Figure 17] 13 is a flowchart illustrating an example of the flow of an image decoding process. [Figure 18] 13 is a flowchart illustrating an example of the flow of an inverse orthogonal transform process. [Figure 19] FIG. 13 illustrates an example of conversion type derivation. [Figure 20] FIG. 13 is a diagram showing a specific example of conversion type derivation. [Figure 21] 11A and 11B are diagrams illustrating an example of an LUT size required to hold a transformation matrix. [Figure 22] 4 is a block diagram showing an example of a main configuration of a primary conversion unit; FIG. [Diagram 23] 11 is a block diagram showing an example of the main configuration of a primary horizontal conversion unit. FIG. [Figure 24] 13 is a block diagram showing an example of the main configuration of a transformation matrix derivation unit. FIG. [Diagram 25] 13 is a block diagram showing an example of the main configuration of a primary vertical conversion unit. FIG. [Figure 26] 13 is a block diagram showing an example of the main configuration of a transformation matrix derivation unit. FIG. [Figure 27] 13 is a flowchart illustrating an example of the flow of a primary conversion process. [Figure 28] 13 is a flowchart illustrating an example of the flow of a primary horizontal transformation process. [Figure 29] FIG. 13 is a diagram illustrating an example of an arithmetic expression for each element. [Diagram 30] 13 is a flowchart illustrating an example of the flow of a transformation matrix derivation process. [Diagram 31] FIG. 13 illustrates an example of the assignment of conversion types to conversion type identifiers. [Diagram 32]13 is a flowchart illustrating an example of the flow of a primary vertical transformation process. [Diagram 33] FIG. 13 is a diagram illustrating an example of an arithmetic expression for each element. [Diagram 34] 11 is a block diagram showing an example of a main configuration of an inverse primary conversion unit. FIG. [Diagram 35] 13 is a block diagram showing an example of the main configuration of an inverse primary vertical transform unit. FIG. [Diagram 36] 13 is a block diagram showing an example of the main configuration of a transformation matrix derivation unit. FIG. [Figure 37] 13 is a block diagram showing an example of the main configuration of an inverse primary horizontal transform unit. FIG. [Figure 38] 13 is a block diagram showing an example of the main configuration of a transformation matrix derivation unit. FIG. [Figure 39] 13 is a flowchart illustrating an example of the flow of a reverse primary transformation process. [Diagram 40] 13 is a flowchart illustrating an example of the flow of an inverse primary transformation selection process. [Diagram 41] 13 is a flowchart illustrating an example of the flow of an inverse primary vertical transform process. [Diagram 42] 13 is a flowchart illustrating an example of the flow of an inverse primary horizontal transform process. [Diagram 43] FIG. 13 illustrates an example of conversion type derivation. [Diagram 44] FIG. 13 is a diagram showing a specific example of conversion type derivation. [Diagram 45] 11A and 11B are diagrams illustrating an example of an LUT size required to hold a transformation matrix. [Figure 46] 13 is a block diagram showing an example of the main configuration of a transformation matrix derivation unit. FIG. [Figure 47] 13 is a block diagram showing an example of the main configuration of a transformation matrix derivation unit. FIG. [Figure 48] 13 is a block diagram showing an example of the main configuration of a transformation matrix derivation unit. FIG. [Figure 49] FIG. 13 illustrates an example of the assignment of conversion types to conversion type identifiers. [Figure 50] 13 is a block diagram showing an example of the main configuration of a transformation matrix derivation unit. FIG. [Figure 51] 13 is a block diagram showing an example of the main configuration of a transformation matrix derivation unit. FIG. [Figure 52] FIG. 13 illustrates an example of conversion type derivation. [Figure 53] 11A and 11B are diagrams illustrating an example of an LUT size required to hold a transformation matrix. [Figure 54] 13 is a flowchart illustrating an example of the flow of a transformation matrix derivation process. [Figure 55] FIG. 13 illustrates an example of the assignment of conversion types to conversion type identifiers. [Figure 56] FIG. 13 illustrates an example of conversion type derivation. [Figure 57] 11A and 11B are diagrams illustrating an example of an LUT size required to hold a transformation matrix. [Figure 58] FIG. 13 illustrates an example of the assignment of conversion types to conversion type identifiers. [Figure 59] FIG. 13 illustrates an example of conversion type derivation. [Figure 60] 11A and 11B are diagrams illustrating an example of an LUT size required to hold a transformation matrix. [Figure 61] 13 is a flowchart illustrating an example of the flow of a transformation matrix derivation process. [Figure 62] FIG. 13 illustrates an example of the assignment of conversion types to conversion type identifiers. [Figure 63] FIG. 1 is a diagram illustrating spatial symmetry of a two-dimensional orthogonal transformation. [Figure 64] FIG. 13 is a diagram illustrating symmetry in the horizontal direction. [Figure 65] FIG. 13 is a diagram illustrating symmetry in the vertical direction. [Figure 66] FIG. 1 is a diagram illustrating symmetry in the horizontal and vertical directions. [Figure 67] FIG. 11 is a diagram showing a list of main concrete examples of alternative transformation matrices involving transformation of prediction residuals. [Figure 68] 11A and 11B are diagrams illustrating an example of an LUT size required to hold a transformation matrix. [Figure 69] 4 is a block diagram showing an example of a main configuration of a primary conversion unit; FIG. [Figure 70]13 is a flowchart illustrating an example of the flow of a primary conversion process. [Figure 71] 13 is a flowchart illustrating an example of the flow of a prediction residual replacement operation process. [Figure 72] FIG. 13 illustrates an example of the assignment of conversion types to conversion type identifiers. [Figure 73] 11 is a block diagram showing an example of a main configuration of an inverse primary conversion unit. FIG. [Figure 74] 13 is a flowchart illustrating an example of the flow of a reverse primary transformation process. [Figure 75] 11A and 11B are diagrams illustrating an example of an LUT size required to hold a transformation matrix. [Figure 76] 13 is a block diagram showing an example of the main configuration of a transformation matrix derivation unit. FIG. [Figure 77] 13 is a block diagram showing an example of the main configuration of a transformation matrix derivation unit. FIG. [Figure 78] 13 is a flowchart illustrating an example of the flow of a transformation matrix derivation process. [Figure 79] FIG. 13 illustrates an example of the assignment of conversion types to conversion type identifiers. [Figure 80] 13 is a block diagram showing an example of the main configuration of a transformation matrix derivation unit. FIG. [Figure 81] 13 is a block diagram showing an example of the main configuration of a transformation matrix derivation unit. FIG. [Figure 82] FIG. 13 illustrates an example of the assignment of conversion types to conversion type identifiers. [Figure 83] FIG. 13 illustrates an example of the assignment of conversion types to conversion type identifiers. [Figure 84] 11A and 11B are diagrams illustrating derivation of a transformation matrix using a submatrix. [Figure 85] FIG. 11 is a diagram showing a list of main concrete examples of transformation matrices derived from sub-matrices. [Figure 86] FIG. 11 is a diagram illustrating an example of how a transformation matrix is ​​derived. [Figure 87] FIG. 11 is a diagram illustrating an example of how a transformation matrix is ​​derived. [Figure 88] FIG. 11 is a diagram illustrating an example of how a transformation matrix is ​​derived. [Figure 89] FIG. 11 is a diagram illustrating an example of how a transformation matrix is ​​derived. [Figure 90] FIG. 11 is a diagram illustrating an example of how a transformation matrix is ​​derived. [Figure 91] FIG. 11 is a diagram illustrating an example of how a transformation matrix is ​​derived. [Figure 92] 13 is a block diagram showing an example of the main configuration of a transformation matrix derivation unit. FIG. [Figure 93] 13 is a block diagram showing an example of the main configuration of a transformation matrix derivation unit. FIG. [Figure 94] 13 is a flowchart illustrating an example of the flow of a transformation matrix derivation process. [Figure 95] FIG. 13 illustrates an example of the assignment of conversion types to conversion type identifiers. [Figure 96] 13 is a block diagram showing an example of the main configuration of a transformation matrix derivation unit. FIG. [Figure 97] 13 is a block diagram showing an example of the main configuration of a transformation matrix derivation unit. FIG. [Figure 98] 11A and 11B are diagrams illustrating an example of an LUT size required to hold a transformation matrix. [Figure 99] 13 is a block diagram showing an example of the main configuration of a transformation matrix derivation unit. FIG. [Figure 100] 13 is a flowchart illustrating an example of the flow of a transformation matrix derivation process. [Figure 101] FIG. 13 illustrates an example of the assignment of conversion types to conversion type identifiers. [Figure 102] FIG. 1 is a block diagram showing an example of a main configuration of a computer. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] Hereinafter, modes for carrying out the present disclosure (hereinafter referred to as embodiments) will be described in the following order. 1. Adaptive Primary Transformation

[0022] 2. First embodiment (deriving a transformation matrix from a transformation matrix) 2-1. Common Concept 2-2. Example 1-1 2-3. Example 1-2 2-4. Examples 1-3 2-5. Examples 1-4 2-6. Examples 1-5 3. Second embodiment (prediction residual conversion)

[0567] 3-1. Common Concept 3-2. Example 2-1 3-3. Example 2-2 3-4. Example 2-3 3-5. Example 2-4 4. Third embodiment (deriving a transformation matrix from a submatrix)

[0740] 4-1. Common Concept 5. Fourth embodiment (combination of embodiments)

[0846] 5-1. Common Concept 6. Notes

[0022] <1. Adaptive Primary Conversion> <References supporting technical content and technical terms> The scope of the disclosure of the present technology includes not only the contents described in the examples but also the contents described in the following non-patent documents that were publicly known at the time of filing the application.

[0023] Non-patent document 1: (mentioned above) Non-patent document 3: TELECOMMUNICATION STANDARDIZATION SECTOR OF ITU (International Telecommunication Union), "Advanced video coding for generic audiovisual services", H.264, 04 / 2017 Non-patent document 4: TELECOMMUNICATION STANDARDIZATION SECTOR OF ITU (International Telecommunication Union), "High efficiency video coding", H.265, 12 / 2016

[0024] In other words, the contents described in the above non-patent documents are also the basis for determining the support requirements. For example, even if the Quad-Tree Block Structure described in Non-Patent Document 4 and the QTBT (Quad Tree Plus Binary Tree) Block Structure described in Non-Patent Document 1 are not directly described in the examples, they are within the disclosure scope of the present technology and meet the support requirements of the claims. Similarly, even if technical terms such as parsing, syntax, and semantics are not directly described in the examples, they are within the disclosure scope of the present technology and meet the support requirements of the claims.

[0025] In addition, in this specification, a "block" (not a block indicating a processing unit) used in the description as a partial region or processing unit of an image (picture) indicates any partial region in a picture, and its size, shape, characteristics, etc. are not limited, unless otherwise specified. For example, a "block" includes any partial region (processing unit) such as a TB (Transform Block), TU (Transform Unit), PB (Prediction Block), PU (Prediction Unit), SCU (Smallest Coding Unit), CU (Coding Unit), LCU (Largest Coding Unit), CTB (Coding Tree Block), CTU (Coding Tree Unit), transform block, sub-block, macroblock, tile, or slice described in the above-mentioned Non-Patent Document 1, Non-Patent Document 3, and Non-Patent Document 4.

[0026] In addition, when specifying the size of such a block, the block size may be specified indirectly in addition to directly. For example, the block size may be specified using identification information for identifying the size. For example, the block size may be specified by a ratio or difference with respect to the size of a reference block (for example, LCU or SCU). For example, when transmitting information specifying a block size as a syntax element or the like, the information may be used to indirectly specify the size as described above. In this way, the amount of information can be reduced, and coding efficiency may be improved. The specification of the block size also includes specification of a range of block sizes (for example, specification of a range of allowable block sizes, etc.).

[0027] <Adaptive Primary Transformation> In the test model (JEM4 (Joint Exploration Test Model 4)) described in non-patent document 1, an adaptive primary transform (AMT (Adaptive Multiple Core Transforms)) is disclosed that adaptively selects a primary transform from multiple different one-dimensional orthogonal transforms for each horizontal primary transform PThor (also called primary horizontal transform) and each vertical primary transform PTver (also called primary vertical transform) for a luminance transform block.

[0028] Specifically, when the adaptive primary transformation flag apt_flag indicating whether or not to perform adaptive primary transformation for a luminance transformation block is 0 (false), the primary transformation is uniquely determined by the mode information as DCT (Discrete Cosine Transform)-II or DST (Discrete Sine Transform)-VII (TrSetIdx = 4), for example, as shown in the table (LUT_TrSetToTrTypIdx) in Figure 1.

[0029] When the adaptive primary transform flag apt_flag is 1 (true) and the current CU (Coding Unit) including the luminance transform block to be processed is an intra CU, a transform set TrSet including orthogonal transforms that are candidates for the primary transform in each of the horizontal direction (x direction) and vertical direction (y direction) is selected from the three transform sets TrSet (TrSetIdx = 0, 1, 2) shown in Fig. 1, as shown in the table in Fig. 1. Note that DST-VII, DCT-VIII, etc. shown in Fig. 1 indicate the type of orthogonal transform, and functions as shown in the table in Fig. 2 are used for each.

[0030] This transform set TrSet is uniquely determined based on (the intra-prediction mode information of) the correspondence table between mode information and transform sets shown in Fig. 3. For example, as shown in the following formulas (1) and (2), a transform set identifier TrSetIdx that specifies the corresponding transform set TrSet is set for each transform set TrSetH, TrSetV.

[0031]

number

[0032] Here, TrSetH indicates a transform set of the primary horizontal transform PThor, TrSetV indicates a transform set of the primary vertical transform PTver, and the lookup table LUT_IntraModeToTrSet is the correspondence table in Fig. 3. The first array of the lookup table LUT_IntraModeToTrSet[][] takes the intra prediction mode IntraMode as an argument, and the second array takes {H=0, V=1} as an argument.

[0033] For example, in the case of intra-prediction mode number 19 (IntraMode == 19), the transform set with transform set identifier TrSetIdx = 0 shown in the table of Figure 1 is selected as the transform set TrSetH (also referred to as the primary horizontal transform set) of the primary horizontal transform PThor, and the transform set with transform set identifier TrSetIdx = 2 shown in the table of Figure 1 is selected as the transform set TrSetV (also referred to as the primary vertical transform set) of the primary vertical transform PTver.

[0034] In addition, when the adaptive primary transform flag apt_flag is 1 (true) and the current CU containing the luminance transform block to be processed is an inter CU, the transform set TrSetH for the primary horizontal transform and the transform set TrSetV for the primary vertical transform are assigned the transform set InterTrSet (TrSetIdx = 3) dedicated to inter CUs.

[0035] Next, for each of the horizontal and vertical directions, which orthogonal transform to apply from the selected transform set TrSet is selected based on the corresponding one of the primary horizontal transform specification flag pt_hor_flag and the primary vertical transform specification flag pt_ver_flag.

[0036] For example, as shown in the following equations (3) and (4), the primary {horizontal, vertical} transformation set TrSet{H, V} and the primary {horizontal, vertical} transformation specification flag pt_{hor, ver}_flag are used as arguments to derive from the transformation set definition table (LUT_TrSetToTrTypeIdx) shown in Figure 1.

[0037]

number

[0038] For example, when the intra prediction mode number is 34 (IntraMode == 34) (i.e., the primary horizontal transform set TraSetH is 0) and the primary horizontal transform specification flag pt_hor_flag is 0, the value of the transform type identifier TrTypeIdxH in equation (3) is 4 from the transform set definition table (LUT_TrSetToTrTypeIdx) in FIG. 1, and the transform type TrTypeH corresponding to the value of the transform type identifier TrTypeIdxH is DST-VII with reference to FIG. 2. That is, DST-VII of the transform set whose transform set identifier TrSetIdx is 0 is selected as the transform type of the primary horizontal transform PThor. Also, when the primary horizontal transform specification flag pt_hor_flag is 1, DCT-VIII is selected as the transform type. Note that selecting the transform type TrType includes selecting the transform type specified by the transform type identifier TrTypeIdx via the transform type identifier TrTypeIdx.

[0039] The primary transform identifier pt_idx is derived from the primary horizontal transform specification flag pt_hor_flag and the primary vertical transform specification flag pt_ver_flag based on the following formula (5). That is, the most significant bit of the primary transform identifier pt_idx corresponds to the value of the primary vertical transform specification flag, and the least significant bit corresponds to the value of the primary horizontal transform specification flag.

[0040]

number

[0041] The encoding is performed by applying arithmetic coding to the bin string of the derived primary transform identifier pt_idx to generate a bit string. Note that the adaptive primary transform flag apt_flag and the primary transform identifier pt_idx are signaled in the luma transform block.

[0042] As described above, Non-Patent Document 1 proposes five one-dimensional orthogonal transforms, DCT-II (DCT2), DST-VII (DST7), DCT-VIII (DCT8), DST-I (DST1), and DCT-V (DCT5), as candidates for the primary transform. In addition, Non-Patent Document 2 proposes adding two more one-dimensional orthogonal transforms, DST-IV (DST4) and IDT (Identity Transform: one-dimensional transform skip), to the above, making a total of seven one-dimensional orthogonal transforms candidates for the primary transform.

[0043] That is, in the case of Non-Patent Document 1, one-dimensional orthogonal transforms are stored in the LUT as candidates for primary transforms, as shown in Fig. 4. In addition, in the case of Non-Patent Document 2, in addition to those, DST-IV (DST4) and IDT are also stored in the LUT (see Fig. 4).

[0044] In the case of HEVC (High Efficiency Video Coding), the size of the LUT (Look Up Table) required to hold the transformation matrix is ​​as shown in the table in FIG. 5. That is, the size of the LUT is about 1.3 KB in total. In contrast, in the case of the method described in Non-Patent Document 1, for example, DCT2 needs to hold the transformation matrix for each size of 2 / 4 / 8 / 16 / 32 / 64 / 128 points in the LUT. In addition, other one-dimensional transformations (DST7 / DST1 / DCT8) need to hold the transformation matrix for each size of 4 / 8 / 16 / 32 / 64 points in the LUT. In this case, assuming that the bit precision of each coefficient of the transformation matrix is ​​10 bits, the size of the LUT required to hold all transformation matrices of the primary transformation is as shown in A of FIG. 6. That is, the size of the LUT in this case is about 53 KB in total. That is, the size of the LUT in this case increases by about 50 times compared to the case of HEVC.

[0045] Similarly, in the case of the method described in Non-Patent Document 2, the size of the LUT required to hold all the transformation matrices of the primary transformation is as shown in the table in FIG. 6B. That is, the size of the LUT in this case is about 67 KB in total. That is, the size of the LUT in this case is about 60 times larger than that in the case of HEVC.

[0046] When considering hardware implementation of primary transformation, the size of the LUT is reflected in the storage capacity (memory capacity). In other words, in the case of the methods described in Non-Patent Document 1 and Non-Patent Document 2, the circuit scale (memory capacity required to hold coefficients of the transformation matrix) may increase by about 50 to 60 times compared to the case of HEVC.

[0047] <2. First embodiment> <2-1. Common Concept> <Deriving the transformation matrix> Therefore, a second transformation matrix is ​​derived using the first transformation matrix, and the prediction residual of the image is orthogonally transformed using the derived second transformation matrix. The coefficient data obtained by orthogonally transforming the prediction residual is then coded to generate a bitstream.

[0048] For example, an image processing device may include a derivation unit that derives a second transformation matrix using a first transformation matrix, an orthogonal transformation unit that performs an orthogonal transformation on a prediction residual of an image using the second transformation matrix derived by the derivation unit, and an encoding unit that encodes coefficient data obtained by orthogonally transforming the prediction residual by the orthogonal transformation unit to generate a bit stream.

[0049] By doing this, transformation matrices can be derived from other transformation matrices, which makes it possible to suppress an increase in the number of transformation matrices prepared for orthogonal transformation and to suppress an increase in the memory capacity required for orthogonal transformation.

[0050] In addition, the bit stream is decoded to obtain coefficient data obtained by orthogonally transforming the prediction residual of the image, a second transform matrix is ​​derived using the first transform matrix, and the obtained coefficient data is inversely orthogonally transformed using the derived second transform matrix.

[0051] For example, an image processing device may include a decoding unit that decodes a bit stream to obtain coefficient data obtained by orthogonally transforming the prediction residual of an image, a derivation unit that derives a second transform matrix using a first transform matrix, and an inverse orthogonal transform unit that performs an inverse orthogonal transform on the coefficient data obtained by the decoding unit using the second transform matrix derived by the derivation unit.

[0052] By doing this, it is possible to derive a transformation matrix from another transformation matrix, thereby suppressing an increase in the number of transformation matrices prepared for the inverse orthogonal transformation and suppressing an increase in the memory capacity required for the inverse orthogonal transformation.

[0053] <Characteristics of transformation matrices> One of the main roles of a transformation matrix is ​​to bias low-order (especially 0th order) frequency component signals toward the DC component, and how these frequency components are collected is an important characteristic. How the frequency components are biased depends on the waveform components of the low-order (especially 0th order) basis vectors (row vectors). In other words, transformation matrices with similar tendencies in the waveform components of these basis vectors can be expected to have similar performance in orthogonal transforms and inverse orthogonal transforms (the way they bias frequency components is similar).

[0054] Therefore, we focus on the waveforms of the low-order (especially 0th order) basis vectors (row vectors) of the transformation matrix. For example, in the transformation matrix 30 of Fig. 7, the waveforms (tendency of the values ​​of each element) of the low-order (especially 0th order) row vectors in the frame 31 are shown as in graph 32.

[0055] Graph 32 shows (tendency of) lower frequency element values ​​toward the left side of the figure, and higher frequency element values ​​toward the right side of the figure. Graph 32 also shows larger values ​​toward the top of the figure, and smaller values ​​toward the bottom of the figure. The center of graph 32 in the vertical direction indicates 0, the area above the center indicates positive values, and the area below the center indicates negative values.

[0056] Waveform 32A of graph 32 shows the waveform of the 0th-order row vector of transformation matrix 30. As shown in waveform 32A, in this case, the 0th-order row vector of transformation matrix 30 has a tendency to have larger values ​​from low-frequency components to high-frequency components.

[0057] Also, for example, in the transformation matrix 30 of Fig. 7, the waveform (tendency of the value of each element) of the low-order (particularly 0th order) column vector within frame 33 is shown as in graph 34. Graph 34 indicates (tendency of) element values ​​of lower frequency components toward the upper side in the figure, and indicates (tendency of) element values ​​of higher frequency components toward the lower side in the figure. Graph 34 also indicates that the values ​​are larger toward the left side in the figure, and smaller toward the right side in the figure. Note that the center of graph 34 in the horizontal direction indicates 0, the values ​​to the left of the center indicate positive values, and the values ​​to the right of the center indicate negative values.

[0058] Waveform 34A of graph 34 shows the waveform of the 0th-order column vector of transformation matrix 30. As shown in waveform 34A, in this case, the 0th-order column vector of transformation matrix 30 has a peak at the intermediate frequency component (that is, the value tends to become smaller toward the lower frequency component on the low frequency side, and the value tends to become smaller toward the higher frequency component on the high frequency side).

[0059] In this specification, the waveform of the zeroth-order column vector of the transformation matrix 30 may be expressed in a transposed state as in graph 35. The structure of graph 35 is similar to that of graph 32. Waveform 35A is equivalent to waveform 34A.

[0060] As described above, transformation matrices with similar waveforms of low-order (especially 0th order) basis vectors (row vectors) have similar performance. In other words, a transformation matrix can be replaced by another transformation matrix with similar waveforms of low-order (especially 0th order) basis vectors (row vectors). Therefore, by utilizing this, it is possible to suppress an increase in the number of transformation matrices stored in the LUT.

[0061] Here, when focusing on the transformation types described in Non-Patent Document 1 and Non-Patent Document 2, the waveforms of the 0th-order row vectors and 0th-order column vectors of the transformation matrices of these transformation types can be classified into four types. An example is shown in Figure 8.

[0062] The first type is a flat type. This flat type is a type of waveform in which the values ​​are approximately uniform for each frequency component. The second type is an increasing type. This increasing type is a type of waveform in which the values ​​tend to increase from low frequency components to high frequency components. The third type is a decreasing type. This decreasing type is a type of waveform in which the values ​​tend to decrease from low frequency components to high frequency components. The fourth type is a mountain type. This mountain type is a type of waveform that tends to have a peak (maximum value) along the way. In other words, in the case of this mountain type, the waveform tends to decrease in value toward the lower frequency components on the low frequency component side, and decrease in value toward the higher frequency components on the high frequency component side.

[0063] Note that these types indicate approximate waveform shapes, and do not need to be completely identical. For example, in the case of an increasing type, the waveform only needs to have a tendency to increase in value from the low frequency side to the high frequency side as a whole, and does not need to strictly increase monotonically from the low frequency side to the high frequency side.

[0064] Similarly, in the case of a decreasing type, the waveform need only tend to have smaller values ​​overall from the low frequency side to the high frequency side, and does not have to strictly decrease monotonically from the low frequency side to the high frequency side.

[0065] Similarly, in the case of a mountain-shaped waveform, it is sufficient that the waveform as a whole has a peak (maximum value) near the center, and that on both sides of the waveform the values ​​tend to decrease in the direction away from the peak, but it is not necessary that the values ​​on both sides of the peak decrease monotonically in the direction away from the peak. Also, the peak does not have to be formed by one component, and may be such that, for example, the approximate position and value of the peak can be identified from multiple components. Also, the position of the peak does not have to be strictly in the center.

[0066] Similarly, in the case of a flat type, the waveform does not have to be strictly flat as long as the values ​​are approximately uniform overall. In other words, some variation in values ​​is acceptable. In other words, anything that cannot be classified into the other three types may be classified as a flat type.

[0067] The above classification of waveforms is an example, and is not limited to the above example. In other words, the waveforms may be classified into types other than those described above, and the number of types to be classified is arbitrary, and is not limited to the above four types. Note that this classification is performed for the convenience of explaining the present technology, and is not performed as an actual process.

[0068] According to this classification, as shown in FIG. 8, the waveform of the 0th-order row vector of the transformation matrix of DCT2 is classified as a flat type, and the waveform of the 0th-order column vector is classified as a decreasing type. In addition, the waveform of the 0th-order row vector of the transformation matrix of DST7 is classified as an increasing type, and the waveform of the 0th-order column vector is classified as a mountain type. In addition, the waveform of the 0th-order row vector of the transformation matrix of DCT8 is classified as a decreasing type, and the waveform of the 0th-order column vector is classified as a decreasing type. In addition, the waveform of the 0th-order row vector of the transformation matrix of DCT5 is classified as a flat type, and the waveform of the 0th-order column vector is classified as a flat type. In addition, the waveform of the 0th-order row vector of the transformation matrix of DST4 is classified as an increasing type, and the waveform of the 0th-order column vector is classified as an increasing type.

[0069] As mentioned above, a transformation matrix can be replaced by another transformation matrix that has a similar waveform of the 0th-order row vector. In other words, transformation types that have the same waveform of the 0th-order row vector can replace each other's transformation matrices.

[0070] In other words, when the derivation unit derives the second transformation matrix using the above-mentioned first transformation matrix, the derivation unit may derive a second transformation matrix in which the lowest-order row vector has a desired type of waveform. In this way, it is possible to suppress an increase in the number of transformation matrices prepared for the orthogonal transformation and the inverse orthogonal transformation, and to suppress an increase in the memory capacity required for the orthogonal transformation and the inverse orthogonal transformation.

[0071] For example, the derivation unit may use the first transformation matrix to derive a second transformation matrix in which the lowest-order row vector has a flat-type waveform. In this way, the derived second transformation matrix can replace the transformation matrix in which the lowest-order row vector has a flat-type waveform. Also, for example, the derivation unit may use the first transformation matrix to derive a second transformation matrix in which the lowest-order row vector has an increasing-type waveform. In this way, the derived second transformation matrix can replace the transformation matrix in which the lowest-order row vector has an increasing-type waveform.

[0072] Also, for example, the derivation unit may use the first transformation matrix to derive a second transformation matrix in which the lowest-order row vector has a decreasing waveform. In this way, the derived second transformation matrix can replace the transformation matrix in which the waveform of the lowest-order row vector is decreasing. Also, for example, the derivation unit may use the first transformation matrix to derive a second transformation matrix in which the lowest-order row vector has a mountain-shaped waveform. In this way, the derived second transformation matrix can replace the transformation matrix in which the waveform of the lowest-order row vector is mountain-shaped.

[0073] For example, in Fig. 8, DST7, DST4, DST8, and DST3 all have increasing waveforms of the 0th order row vectors, so they can be substituted for each other. In other words, even if the transformation matrices are of different transformation types, they can be substituted for each other.

[0074] That is, when the derivation unit derives the second transformation matrix using the above-mentioned first transformation matrix, the derivation unit may derive the second transformation matrix of a transformation type different from the first transformation matrix. In this way, it is possible to suppress an increase in the number of transformation types prepared for the orthogonal transformation and the inverse orthogonal transformation, and it is also possible to suppress an increase in the memory capacity required for the orthogonal transformation and the inverse orthogonal transformation.

[0075] In addition, when deriving the second transformation matrix, the derivation unit may derive a second transformation matrix having the same number of rows and columns as the first transformation matrix. When the number of rows and columns is changed, the type of waveform may change unintentionally. Therefore, by making the number of rows and columns the same as the first transformation matrix, the possibility of an unintentional change in the type of waveform can be suppressed, and the derivation of the second transformation matrix can be more easily performed.

[0076] Incidentally, a matrix can easily perform operations on its elements. Operations on matrix elements include, for example, rearranging the elements. More specifically, for example, a matrix can flip (reverse) the order of its elements in a predetermined direction, or transpose the elements to interchange rows and columns. Incidentally, transposition is equivalent to a flip (reversal) about the diagonal line connecting the upper left end and lower right end of the matrix. In other words, transposition can be said to be a part of flipping. It is also easy to invert the sign of each element (from positive to negative, or from negative to positive).

[0077] By using such operations, it is possible to intentionally change the waveform (type) of the 0th-order row vector. For example, if a matrix in which the waveform of the 0th-order row vector is increasing is flipped row-wise, the waveform of the 0th-order row vector changes to decreasing type. Conversely, if a matrix in which the waveform of the 0th-order row vector is decreasing is flipped row-wise, the waveform of the 0th-order row vector changes to increasing type.

[0078] In other words, as shown in Figure 8, by flipping a transformation matrix of a transformation type in which the waveform of the 0th order row vector is an increasing type (e.g., DST7, DST4, DST8, DST3) in the row direction, it is possible to obtain a transformation matrix that can replace a transformation matrix of a transformation type in which the waveform of the 0th order row vector is a decreasing type (e.g., DCT7, DCT4, DCT8, DCT3).

[0079] For example, when a matrix is ​​transposed, the waveform type of the 0th row vector and the waveform type of the 0th column vector are swapped. In other words, the transposition causes the waveform of the 0th row vector of the matrix to become the same type as the waveform of the 0th column vector of the matrix before the transposition.

[0080] For example, as shown in Fig. 9, by transposing the transformation matrix of DCT2 (DCT6) in which the waveform of the 0th-order column vector is decreasing, a transformation matrix capable of substituting for the transformation matrix of the transformation type (DCT3, DCT7, DCT4, DCT8) in which the waveform of the 0th-order row vector is decreasing can be obtained. Also, for example, by transposing the transformation matrix of DST7 (DST3) in which the waveform of the 0th-order column vector is mountain-shaped, a transformation matrix capable of substituting for the transformation matrix of the transformation type (DST2, DST6, DST1, DST5) in which the waveform of the 0th-order row vector is mountain-shaped can be obtained.

[0081] That is, the derivation unit may derive the second transformation matrix by performing an operation on the elements of the first transformation matrix. The operation on the elements may include rearrangement of the elements (changing the order of the elements) as described above. In this manner, the type of waveform can be changed intentionally, so that a wider variety of second transformation matrices can be derived from the first transformation matrix. Therefore, an increase in the number of transformation types prepared for the orthogonal transformation and the inverse orthogonal transformation can be suppressed, and an increase in the memory capacity required for the orthogonal transformation and the inverse orthogonal transformation can be suppressed.

[0082] Of course, the derivation unit may perform such an operation multiple times to derive the second transformation matrix. For example, operations such as flip and transpose can be arbitrarily combined. Also, the same operation may be repeated multiple times. In this way, more diverse second transformation matrices can be derived from the first transformation matrix.

[0083] As described above, the transformation matrix used for the orthogonal transformation and the inverse orthogonal transformation is stored in the LUT. Therefore, the derivation unit may derive the second transformation matrix using the first transformation matrix stored in this lookup table (LUT). In this way, it is possible to suppress an increase in the size of the LUT. Therefore, it is possible to suppress an increase in the memory capacity required for the orthogonal transformation and the inverse orthogonal transformation.

[0084] <Derivation example> A list of examples of derivation of transformation matrices involving the above-mentioned operations is shown in FIG. 10. Note that the transformation matrix (first transformation matrix) used for derivation is the base transformation matrix T base The transformation type of the base transformation matrix is ​​also referred to as the base transformation type or the first transformation type. Also, the transformation type of the derived transformation matrix (second transformation matrix) is also referred to as the second transformation type.

[0085] In the table shown in Figure 10, excluding the row of item names in the top row, the example in the first row from the top is derived by focusing on the similarity between the waveform of the lowest-order row vector of the first transformation type and the waveform of the lowest-order row vector of the transformation type of the alternative transformation matrix.

[0086] In this case, the derivation unit flips the first transformation matrix to derive the second transformation matrix. More specifically, the derivation unit flips the transformation matrix of DST7 to the base transformation matrix T baseThen, it is flipped in the row direction to derive the FlipDST7 transformation matrix. Since the waveform of the 0th-order row vector of the DST7 transformation matrix is ​​increasing, the waveform of the 0th-order row vector of the derived FlipDST7 transformation matrix is ​​decreasing. Therefore, this FlipDST7 transformation matrix can replace the DCT8 transformation matrix in which the waveform of the 0th-order row vector is decreasing.

[0087] By applying such derivation, it is not necessary to prepare a DCT8 transformation matrix as a candidate for the transformation matrix used in the orthogonal transformation and inverse orthogonal transformation. In other words, the inherent transformation type can be reduced. In other words, an increase in the LUT size can be suppressed. Furthermore, by performing the orthogonal transformation and inverse orthogonal transformation using the derived transformation matrix of the second transformation type (FlipDST7), it is possible to obtain coding efficiency equivalent to that when the DCT8 transformation matrix is ​​used for the orthogonal transformation and inverse orthogonal transformation. Furthermore, in this case, the second transformation matrix (reduced alternative transformation matrix) can be derived in a single operation.

[0088] Furthermore, the example one row below (second from the top) is derived by focusing on the similarity between the waveform of the lowest-order column vector of the first transformation type and the waveform of the lowest-order row vector of the transformation type of the alternative transformation matrix.

[0089] In this case, the derivation unit transposes the first transformation matrix to derive the second transformation matrix. More specifically, the derivation unit transposes the transformation matrix of DST7 to the base transformation matrix T base and then transpose it to derive the transformation matrix for TrDST7. Since the waveform of the 0th-order column vector of the transformation matrix for DST7 is mountain-shaped, the waveform of the 0th-order row vector of the derived transformation matrix for TrDST7 is also mountain-shaped. Therefore, this transformation matrix for TrDST7 can replace the transformation matrix for DST1, in which the waveform of the 0th-order row vector is mountain-shaped.

[0090] By applying such a derivation, it is no longer necessary to prepare a DST1 transform matrix as a candidate for the transform matrix used in the orthogonal transform and inverse orthogonal transform. In other words, the inherent transform type can be reduced. In other words, an increase in the LUT size can be suppressed. Furthermore, by performing the orthogonal transform and inverse orthogonal transform using the derived transform matrix of the second transform type (TrDST7), it is possible to obtain coding efficiency equivalent to that when the DST1 transform matrix is ​​used for the orthogonal transform and inverse orthogonal transform. Furthermore, in this case, the second transform matrix (alternative transform matrix with a mountain shape) can be derived in a single operation.

[0091] The derivation of the example one level below (third level from the top) focuses on the characteristics between paired DCT / DSTs. More specifically, between paired DCT / DSTs (e.g. DST7 and DCT8), even-numbered row vectors are axially symmetric, and odd-numbered row vectors are point symmetric.

[0092] In this case, the derivation unit flips the first transformation matrix, and inverts the signs of odd-numbered row vectors in the first transformation matrix after the flip to derive the second transformation matrix. More specifically, the derivation unit derives the second transformation matrix by converting the transformation matrix of DST7 into a base transformation matrix T base Then, it is flipped in the row direction, and the signs of the odd-order row vectors are inverted to derive the DCT8 transformation matrix. Note that the sign of a row vector can be inverted by converting the most significant bit of each element of the row vector. Naturally, the derived DCT8 transformation matrix can replace the DST8 transformation matrix in which the waveform of the 0th-order row vector is reduced.

[0093] By applying such derivation, it becomes unnecessary to prepare a DCT8 transform matrix as a candidate for the transform matrix used in the orthogonal transform and inverse orthogonal transform. In other words, it is possible to reduce the number of eigentransformation types. In other words, it is possible to suppress an increase in the LUT size. Naturally, by performing the orthogonal transform and inverse orthogonal transform using the derived transform matrix of the second transform type (DCT8), it is possible to obtain the same coding efficiency as when the transform matrix of DCT8 is used for the orthogonal transform and inverse orthogonal transform. In this case, it is also possible to derive the second transform matrix (the counterpart alternative transform matrix) in two operations.

[0094] Furthermore, the example one row below (fourth row from the top) is derived, as in the case of the first row from the top, by focusing on the similarity between the waveform of the lowest row vector of the first transformation type and the waveform of the lowest row vector of the transformation type of the alternative transformation matrix.

[0095] In this case, the derivation unit flips the first transformation matrix to derive the second transformation matrix. More specifically, the derivation unit flips the transformation matrix of the DCT8 to the base transformation matrix T base Then, it is flipped in the row direction to derive the FlipDCT8 transformation matrix. Since the waveform of the 0th-order row vector of the DCT8 transformation matrix is ​​decreasing, the waveform of the 0th-order row vector of the derived FlipDCT8 transformation matrix is ​​increasing. Therefore, this FlipDCT8 transformation matrix can replace the DST7 transformation matrix in which the waveform of the 0th-order row vector is increasing.

[0096] By applying such derivation, it is no longer necessary to prepare a DST7 transform matrix as a candidate for the transform matrix used in the orthogonal transform and inverse orthogonal transform. In other words, the inherent transform type can be reduced. In other words, an increase in the LUT size can be suppressed. Furthermore, by performing the orthogonal transform and inverse orthogonal transform using the derived transform matrix of the second transform type (FlipDCT8), it is possible to obtain coding efficiency equivalent to that when the DST7 transform matrix is ​​used for the orthogonal transform and inverse orthogonal transform. Furthermore, in this case, the second transform matrix (reduced alternative transform matrix) can be derived in a single operation.

[0097] Furthermore, the example one row below (fifth row from the top) is derived by focusing on the similarity between the waveform of the highest-order column vector of the first transformation type and the waveform of the lowest-order row vector of the transformation type of the alternative transformation matrix.

[0098] In this case, the derivation unit flips the first transformation matrix and transposes the first transformation matrix after the flip to derive the second transformation matrix. More specifically, the derivation unit converts the transformation matrix of the DCT8 into a base transformation matrix T base Then, it is flipped in the row direction and further transposed to derive the transformation matrix of TrFlipDCT8. Since the waveform of the highest-order column vector of the transformation matrix of DCT8 is mountain-shaped, the waveform of the 0th-order row vector of the derived transformation matrix of TrFlipDCT8 is mountain-shaped. Therefore, this transformation matrix of TrFlipDCT8 can replace the transformation matrix of DST1, in which the waveform of the 0th-order row vector is mountain-shaped.

[0099] By applying such derivation, it is no longer necessary to prepare a DST1 transform matrix as a candidate for the transform matrix used in the orthogonal transform and inverse orthogonal transform. In other words, the inherent transform type can be reduced. In other words, an increase in the LUT size can be suppressed. Furthermore, by performing the orthogonal transform and inverse orthogonal transform using the derived transform matrix of the second transform type (TrFlipDCT8), it is possible to obtain coding efficiency equivalent to that when the DST1 transform matrix is ​​used for the orthogonal transform and inverse orthogonal transform. Furthermore, in this case, the second transform matrix (alternative transform matrix with a mountain shape) can be derived in two operations.

[0100] The derivation of the example one row below (sixth from the top) focuses on the characteristics between paired DCT / DSTs, just like the third row from the top. More specifically, it focuses on the fact that between paired DCT / DSTs (e.g. DCT8 and DST7), even-numbered row vectors are axially symmetric and odd-numbered row vectors are point symmetric.

[0101] In this case, the derivation unit flips the first transformation matrix, and inverts the signs of odd-numbered row vectors in the first transformation matrix after the flip to derive the second transformation matrix. More specifically, the derivation unit derives the second transformation matrix by converting the transformation matrix of the DCT8 into a base transformation matrix T base Then, it is flipped in the row direction, and the signs of the odd-order row vectors are inverted to derive the DST7 transformation matrix. Naturally, the derived DST7 transformation matrix can be used to replace the DST7 transformation matrix in which the waveform of the 0th-order row vector increases.

[0102] By applying such a derivation, it becomes unnecessary to prepare a DST7 transform matrix as a candidate for the transform matrix used in the orthogonal transform and inverse orthogonal transform. In other words, it is possible to reduce the number of eigentransform types. In other words, it is possible to suppress an increase in the LUT size. Naturally, by performing the orthogonal transform and inverse orthogonal transform using the derived transform matrix of the second transform type (DST7), it is possible to obtain the same coding efficiency as when the DST7 transform matrix is ​​used for the orthogonal transform and inverse orthogonal transform. In this case, it is also possible to derive the second transform matrix (the counterpart alternative transform matrix) in two operations.

[0103] Furthermore, the example one row below (seventh row from the top) is derived by focusing on the similarity between the waveform of the lowest-order column vector of the first transformation type and the waveform of the lowest-order row vector of the transformation type of the alternative transformation matrix.

[0104] In this case, the derivation unit transposes the first transformation matrix to derive the second transformation matrix. More specifically, the derivation unit transposes the transformation matrix of DCT2 to the base transformation matrix T base Then, the DCT3 transformation matrix is ​​derived by transposing it. Since the waveform of the 0th order column vector of the DCT2 transformation matrix is ​​a decreasing type, the waveform of the 0th order row vector of the derived DCT3 transformation matrix is ​​also a decreasing type. Therefore, this DCT3 transformation matrix can replace the DCT8 transformation matrix, which has a decreasing type waveform for the 0th order row vector.

[0105] By applying such derivation, it is not necessary to prepare a DCT8 transform matrix as a candidate for the transform matrix used in the orthogonal transform and inverse orthogonal transform. In other words, the inherent transform type can be reduced. In other words, an increase in the LUT size can be suppressed. Furthermore, by performing the orthogonal transform and inverse orthogonal transform using the derived transform matrix of the second transform type (DCT3), it is possible to obtain coding efficiency equivalent to that in the case where the DCT8 transform matrix is ​​used for the orthogonal transform and inverse orthogonal transform. Furthermore, in this case, the second transform matrix (reduced alternative transform matrix) can be derived in a single operation.

[0106] Furthermore, the example one row below (the eighth row from the top) is derived by focusing on the similarity between the waveform of the highest-order column vector of the first transformation type and the waveform of the lowest-order row vector of the transformation type of the alternative transformation matrix.

[0107] In this case, the derivation unit transposes the first transformation matrix and flips the transposed first transformation matrix to derive the second transformation matrix. More specifically, the derivation unit derives the transformation matrix of DCT2 as a base transformation matrix T base Then, it is transposed and further flipped in the row direction to derive the FlipDCT3 transformation matrix. Since the waveform of the highest-order column vector of the DCT2 transformation matrix is ​​increasing, the waveform of the 0th-order row vector of the derived FlipDCT3 transformation matrix is ​​increasing. Therefore, this FlipDCT3 transformation matrix can replace the DST7 transformation matrix, in which the waveform of the 0th-order row vector is increasing.

[0108] By applying such derivation, it is not necessary to prepare a DST7 transform matrix as a candidate for the transform matrix used in the orthogonal transform and inverse orthogonal transform. In other words, the inherent transform type can be reduced. In other words, an increase in the LUT size can be suppressed. Furthermore, by performing the orthogonal transform and inverse orthogonal transform using the derived transform matrix of the second transform type (FlipDCT3), it is possible to obtain coding efficiency equivalent to that when the DST7 transform matrix is ​​used for the orthogonal transform and inverse orthogonal transform. Furthermore, in this case, the second transform matrix (an augmented alternative transform matrix) can be derived in two operations.

[0109] Each of the above-described derivation examples may be implemented independently, or a plurality of derivation examples may be implemented in combination.

[0110] <Image encoding device> Next, a configuration for deriving the above-mentioned transformation matrix will be described. Fig. 11 is a block diagram showing an example of the configuration of an image encoding device, which is one aspect of an image processing device to which the present technology is applied. The image encoding device 100 shown in Fig. 11 is a device for encoding image data of a moving image. For example, the image encoding device 100 implements the technology described in Non-Patent Document 1, Non-Patent Document 3, or Non-Patent Document 4, and encodes image data of a moving image in a method conforming to a standard described in any of these documents.

[0111] Note that Fig. 11 shows the main processing units, data flows, etc., and is not limited to all that are shown in Fig. 11. In other words, in the image encoding device 100, there may be processing units that are not shown as blocks in Fig. 11, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 11. This is the same in other figures that explain processing units, etc. in the image encoding device 100.

[0112] As shown in FIG. 11, the image encoding device 100 includes a control unit 101, a rearrangement buffer 111, a calculation unit 112, an orthogonal transformation unit 113, a quantization unit 114, an encoding unit 115, an accumulation buffer 116, an inverse quantization unit 117, an inverse orthogonal transformation unit 118, a calculation unit 119, an in-loop filter unit 120, a frame memory 121, a prediction unit 122, and a rate control unit 123.

[0113] <Control Unit> The control unit 101 divides the video data held by the rearrangement buffer 111 into blocks (CU, PU, ​​transform block, etc.) of processing units based on an external or pre-specified block size of processing units. The control unit 101 also determines the coding parameters (header information Hinfo, prediction mode information Pinfo, transform information Tinfo, filter information Finfo, etc.) to be supplied to each block based on, for example, RDO (Rate-Distortion Optimization).

[0114] These coding parameters will be described in detail later. After determining the coding parameters as described above, the control unit 101 supplies them to each block. More specifically, the process is as follows.

[0115] Header information Hinfo is provided for each block. The prediction mode information Pinfo is supplied to the encoding unit 115 and the prediction unit 122 . The transformation information Tinfo is supplied to the encoding unit 115 , the orthogonal transformation unit 113 , the quantization unit 114 , the inverse quantization unit 117 , and the inverse orthogonal transformation unit 118 . The filter information Finfo is supplied to the in-loop filter unit 120 .

[0116] <Sorting Buffer> To the image coding device 100, each field (input image) of video image data is input in its playback order (display order). The reordering buffer 111 acquires and holds (stores) each input image in its playback order (display order). Under the control of the control unit 101, the reordering buffer 111 reorders the input images in coding order (decoding order) and divides them into blocks of processing units. The reordering buffer 111 supplies each processed input image to the calculation unit 112. The reordering buffer 111 also supplies each input image (original image) to the prediction unit 122 and the in-loop filter unit 120.

[0117] <Arithmetic section> The calculation unit 112 receives as input an image I corresponding to a block of a processing unit and a predicted image P supplied from the prediction unit 122, subtracts the predicted image P from the image I as shown in the following equation (6) to derive a prediction residual D, and supplies it to the orthogonal transformation unit 113.

[0118]

number

[0119] <Orthogonal transformation section> The orthogonal transform unit 113 receives the prediction residual D supplied from the calculation unit 112 and the transformation information Tinfo supplied from the control unit 101, and performs an orthogonal transform on the prediction residual D based on the transformation information Tinfo to derive the transformation coefficient Coeff. The orthogonal transform unit 113 supplies the obtained transformation coefficient Coeff to the quantization unit 114.

[0120] <Quantization section> The quantization unit 114 receives the transform coefficient Coeff supplied from the orthogonal transformation unit 113 and the transform information Tinfo supplied from the control unit 101 as input, and scales (quantizes) the transform coefficient Coeff based on the transform information Tinfo. The quantization rate is controlled by the rate control unit 123. The quantization unit 114 supplies the quantized transform coefficient obtained by such quantization, i.e., the quantized transform coefficient level, to the encoding unit 115 and the inverse quantization unit 117.

[0121] <Encoding part> The encoding unit 115 receives as input the quantized transform coefficient level LEVEL supplied from the quantization unit 114, various encoding parameters (header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, filter information Finfo, etc.) supplied from the control unit 101, information on filters such as filter coefficients supplied from the in-loop filter unit 120, and information on an optimal prediction mode supplied from the prediction unit 122. The encoding unit 115 performs variable-length coding (for example, arithmetic coding) on ​​the quantized transform coefficient level LEVEL to generate a bit string (coded data).

[0122] Furthermore, the encoding unit 115 derives residual information Rinfo from the quantized transform coefficient level level, encodes the residual information Rinfo, and generates a bit string.

[0123] Furthermore, the encoding unit 115 includes information on the filter supplied from the in-loop filter unit 120 in the filter information Finfo, and includes information on the optimal prediction mode supplied from the prediction unit 122 in the prediction mode information Pinfo. Then, the encoding unit 115 encodes the various encoding parameters (header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, filter information Finfo, etc.) described above to generate a bit string.

[0124] The encoding unit 115 multiplexes the bit strings of the various pieces of information generated as described above to generate encoded data, and supplies the encoded data to the accumulation buffer 116.

[0125] <Accumulation buffer> The accumulation buffer 116 temporarily holds the encoded data obtained in the encoding unit 115. The accumulation buffer 116 outputs the held encoded data, for example as a bit stream, to the outside of the image encoding device 100 at a predetermined timing. For example, this encoded data is transmitted to the decoding side via an arbitrary recording medium, an arbitrary transmission medium, an arbitrary information processing device, etc. In other words, the accumulation buffer 116 is also a transmission unit that transmits the encoded data (bit stream).

[0126] <Inverse quantization section> The inverse quantization unit 117 performs processing related to inverse quantization. For example, the inverse quantization unit 117 receives the quantized transformation coefficient level level supplied from the quantization unit 114 and the transformation information Tinfo supplied from the control unit 101 as input, and scales (inverse quantizes) the value of the quantized transformation coefficient level level based on the transformation information Tinfo. Note that this inverse quantization is the inverse process of the quantization performed in the quantization unit 114. The inverse quantization unit 117 supplies the transformation coefficient Coeff_IQ obtained by such inverse quantization to the inverse orthogonal transformation unit 118.

[0127] <Inverse orthogonal transform section> The inverse orthogonal transform unit 118 performs processing related to inverse orthogonal transform. For example, the inverse orthogonal transform unit 118 receives the transform coefficient Coeff_IQ supplied from the inverse quantization unit 117 and the transform information Tinfo supplied from the control unit 101, and performs inverse orthogonal transform on the transform coefficient Coeff_IQ based on the transform information Tinfo to derive a prediction residual D'. Note that this inverse orthogonal transform is an inverse process of the orthogonal transform performed in the orthogonal transform unit 113. The inverse orthogonal transform unit 118 supplies the prediction residual D' obtained by such inverse orthogonal transform to the calculation unit 119. Note that the inverse orthogonal transform unit 118 is the same as the inverse orthogonal transform unit on the decoding side (described later), and therefore the description on the decoding side (described later) can be applied to the inverse orthogonal transform unit 118.

[0128] <Arithmetic section> The calculation unit 119 receives as input the prediction residual D' supplied from the inverse orthogonal transform unit 118 and the predicted image P supplied from the prediction unit 122. The calculation unit 119 adds the prediction residual D' to the predicted image P corresponding to the prediction residual D' to generate a locally decoded image R local The calculation unit 119 derives the derived locally decoded image R local is supplied to an in-loop filter unit 120 and a frame memory 121.

[0129] <In-loop filter section> The in-loop filter unit 120 performs processing related to in-loop filtering. For example, the in-loop filter unit 120 uses the locally decoded image R local The in-loop filter unit 120 receives as input filter information Finfo supplied from the control unit 101 and an input image (original image) supplied from the rearrangement buffer 111. Any information is input to the in-loop filter unit 120, and information other than the above information may be input. For example, a prediction mode, motion information, a code amount target value, a quantization parameter QP, a picture type, information on a block (CU, CTU, etc.), etc. may be input to the in-loop filter unit 120 as necessary.

[0130] Based on the filter information Finfo, the in-loop filter unit 120 generates a locally decoded image R local The in-loop filter unit 120 also uses an input image (original image) and other input information for the filter process as necessary.

[0131] For example, the in-loop filter unit 120 applies four in-loop filters, a bilateral filter, a deblocking filter (DBF (DeBlocking Filter)), an adaptive offset filter (SAO (Sample Adaptive Offset)), and an adaptive loop filter (ALF (Adaptive Loop Filter)), in this order, as described in Non-Patent Document 1. Note that which filters are applied and in what order they are applied are arbitrary and can be appropriately selected.

[0132] Of course, the filtering process performed by the in-loop filter unit 120 is arbitrary and is not limited to the above example. For example, the in-loop filter unit 120 may apply a Wiener filter or the like.

[0133] The in-loop filter unit 120 outputs the filtered locally decoded image R localto the frame memory 121. Note that, for example, when information related to a filter such as a filter coefficient is transmitted to the decoding side, the in-loop filter unit 120 supplies the information related to the filter to the encoding unit 115.

[0134] <Frame memory> The frame memory 121 performs processing related to storage of image-related data. For example, the frame memory 121 stores the locally decoded image R local and the filtered local decoded image R local The frame memory 121 receives the locally decoded image R local The frame memory 121 reconstructs and holds a decoded image R for each picture (stores it in a buffer in the frame memory 121). The frame memory 121 supplies the decoded image R (or a part thereof) to the prediction unit 122 in response to a request from the prediction unit 122.

[0135] <Prediction Department> The prediction unit 122 performs processing related to the generation of a predicted image. For example, the prediction unit 122 receives the prediction mode information Pinfo supplied from the control unit 101, the input image (original image) supplied from the rearrangement buffer 111, and the decoded image R (or a part thereof) read from the frame memory 121 as input. The prediction unit 122 performs prediction processing such as inter prediction or intra prediction using the prediction mode information Pinfo and the input image (original image), performs prediction by referring to the decoded image R as a reference image, performs motion compensation processing based on the prediction result, and generates a predicted image P. The prediction unit 122 supplies the generated predicted image P to the calculation unit 112 and the calculation unit 119. The prediction unit 122 also supplies information on the prediction mode selected by the above processing, that is, the optimal prediction mode, to the encoding unit 115 as necessary.

[0136] <Rate control section> The rate control unit 123 performs processing related to rate control. For example, the rate control unit 123 controls the rate of the quantization operation of the quantization unit 114 based on the code amount of the encoded data accumulated in the accumulation buffer 116 so as to prevent overflow or underflow from occurring.

[0137] In the image coding device 100 configured as above, the orthogonal transform unit 113 performs the above-mentioned processing to which the present technology is applied as a derivation unit and an orthogonal transform unit. Moreover, the coding unit 115 performs the above-mentioned processing to which the present technology is applied as a coding unit. Furthermore, the inverse orthogonal transform unit 118 performs the above-mentioned processing to which the present technology is applied as an inverse orthogonal transform unit and a derivation unit. Therefore, the image coding device 100 can suppress an increase in memory capacity required for orthogonal transform and inverse orthogonal transform.

[0138] <Details of the orthogonal transformation section> Fig. 12 is a block diagram showing an example of the main configuration of orthogonal transform unit 113 in Fig. 11. As shown in Fig. 12, orthogonal transform unit 113 has switch 151, primary transform unit 152, and secondary transform unit 153.

[0139] The switch 151 receives the prediction residual D and a transform skip flag ts_flag[compID] corresponding to the component identifier compID as input, and when the value of the transform skip flag ts_flag[compID] is NO_TS (=0) (when transform skip is not applied), supplies the prediction residual D to the primary transform unit 152. When the value of the transform skip flag ts_flag[compID] is 2D_TS (=1) (when indicating that a two-dimensional transform skip is applied), the switch 151 skips the primary transform unit 152 and the secondary transform unit 153, and outputs the prediction residual D as a transform coefficient Coeff to the outside of the orthogonal transform unit 113 (supplies it to the quantization unit 114).

[0140] The primary transform unit 152 performs processing related to primary transform, which is a predetermined transform processing such as orthogonal transform. For example, the primary transform unit 152 receives as input a component identifier compID, an adaptive primary transform flag apt_flag[compID] of the component identifier compID, a primary transform identifier pt_idx[compID] of the component identifier compID, prediction mode information PInfo, a size of a transform block (logarithmic value of width log2TBWSize, logarithmic value of height log2TBHSize), and a prediction residual D. Note that the width TBWSize of the transform block is also referred to as TBWidth, and its logarithmic value is also referred to as log2TBWidth. Similarly, the height TBHSize of the transform block is also referred to as TBHeight, and its logarithmic value is also referred to as log2TBHeight.

[0141] The primary conversion unit 152 refers to the prediction mode information PInfo, the component identifier compID, the adaptive primary conversion flag apt_flag[compID] for the component identifier compID, and the primary conversion identifier pt_idx[compID] for the component identifier compID, and selects a conversion type TrTypeH of the primary horizontal conversion corresponding to the component identifier compID (and a primary horizontal conversion type identifier TrTypeIdxH indicating said conversion type), and a conversion type TrTypeV of the primary vertical conversion (and a primary vertical conversion type identifier TrTypeIdxV indicating said conversion type).

[0142] Furthermore, the primary transform unit 152 performs a primary horizontal transform on the prediction residual D, which is determined by the primary horizontal transform type identifier TrTypeIdxH (or the primary horizontal transform type TrTypeH) and the horizontal width log2TBWSize of the transform block, and a primary vertical transform determined by the primary vertical transform type identifier TrTypeIdxV (or the primary vertical transform type TrTypeV) and the vertical width log2TBHSize of the transform block, to derive transform coefficients Coeff_P after the primary transform. The primary horizontal transform is a one-dimensional orthogonal transform in the horizontal direction, and the primary vertical transform is a one-dimensional orthogonal transform in the vertical direction.

[0143] The primary conversion unit 152 supplies the derived conversion coefficient Coeff_P to the secondary conversion unit 153 .

[0144] The secondary transform unit 153 performs processing related to secondary transform, which is a predetermined transform processing such as orthogonal transform. For example, the secondary transform unit 153 receives as input a secondary transform identifier st_idx, a scan identifier scanIdx indicating a method of scanning transform coefficients, and a transform coefficient Coeff_P. The secondary transform unit 153 performs secondary transform on the transform coefficient Coeff_P based on the secondary transform identifier st_idx and the scan identifier scanIdx, and derives a transform coefficient Coeff_S after the secondary transform.

[0145] More specifically, when the secondary transformation identifier st_idx indicates that a secondary transformation is to be applied (st_idx>0), the secondary transformation unit 153 performs a secondary transformation process corresponding to the secondary transformation identifier st_idx on the transformation coefficient Coeff_P, and derives the transformation coefficient Coeff_S after the secondary transformation.

[0146] The secondary transform unit 153 outputs the secondary transform coefficient Coeff_S to the outside of the orthogonal transform unit 113 as the transform coefficient Coeff (supplies it to the quantization unit 114).

[0147] In addition, if the secondary transform identifier st_idx indicates that the secondary transform is not to be applied (st_idx==0), the secondary transform unit 153 skips the secondary transform and outputs the transform coefficient Coeff_P after the primary transform as the transform coefficient Coeff (the transform coefficient Coeff_S after the secondary transform) outside the orthogonal transform unit 113 (supplies it to the quantization unit 114).

[0148] In the orthogonal transform unit 113 configured as above, the primary transform unit 152 performs the above-mentioned processing to which the present technology is applied as a derivation unit and an orthogonal transform unit. That is, the derivation unit derives a second transform matrix using a first transform matrix, and the orthogonal transform unit performs a primary transform on the prediction residual using the second transform matrix derived by the derivation unit. Therefore, it is possible to suppress an increase in memory capacity required for the primary transform.

[0149] As described above, the primary transform unit 152 performs a primary horizontal transform and a primary vertical transform as the primary transform. That is, the derivation unit derives a second transform matrix for one-dimensional orthogonal transform in the horizontal direction and a second transform matrix for one-dimensional orthogonal transform in the vertical direction, and the orthogonal transform unit performs a one-dimensional orthogonal transform in the horizontal direction as the primary transform using the second transform matrix for one-dimensional orthogonal transform in the horizontal direction derived by the derivation unit, and further performs a one-dimensional orthogonal transform in the vertical direction using the second transform matrix for one-dimensional orthogonal transform in the vertical direction derived by the derivation unit. Therefore, it is possible to suppress an increase in memory capacity required for such a primary transform in which one-dimensional orthogonal transform in the horizontal direction and one-dimensional orthogonal transform in the vertical direction are performed.

[0150] <Image encoding process flow> Next, a description will be given of the flow of each process executed by the above-described image encoding device 100. First, an example of the flow of image encoding process will be described with reference to the flowchart in FIG.

[0151] When the image encoding process starts, in step S101, the reordering buffer 111 is controlled by the control unit 101 to reorder the frames of the input video data from the display order to the encoding order.

[0152] In step S102, the control unit 101 sets a processing unit for the input image held by the sorting buffer 111 (performs block division).

[0153] In step S103, the control unit 101 determines (sets) coding parameters for the input image held by the reordering buffer 111.

[0154] In step S104, the prediction unit 122 performs a prediction process to generate a predicted image, etc., of an optimal prediction mode. For example, in this prediction process, the prediction unit 122 performs intra prediction to generate a predicted image, etc., of an optimal intra prediction mode, performs inter prediction to generate a predicted image, etc., of an optimal inter prediction mode, and selects an optimal prediction mode from among them based on a cost function value, etc.

[0155] In step S105, the calculation unit 112 calculates the difference between the input image and the prediction image of the optimal mode selected by the prediction process in step S104. That is, the calculation unit 112 generates a prediction residual D between the input image and the prediction image. The prediction residual D thus obtained has a reduced data amount compared to the original image data. Therefore, the data amount can be compressed compared to the case where the image is encoded as it is.

[0156] In step S106, the orthogonal transform unit 113 performs an orthogonal transform process on the prediction residual D generated in the process of step S105, and derives a transform coefficient Coeff.

[0157] In step S107, the quantization unit 114 quantizes the transform coefficient Coeff obtained by the process of step S106 by using the quantization parameter calculated by the control unit 101, etc., to derive the quantized transform coefficient level level.

[0158] In step S108, the inverse quantization unit 117 inverse quantizes the quantized transform coefficient level generated by the process of step S107 with characteristics corresponding to the quantization characteristics of step S107, to derive the transform coefficient Coeff_IQ.

[0159] In step S109, the inverse orthogonal transform unit 118 performs inverse orthogonal transform on the transform coefficient Coeff_IQ obtained by the process of step S108 by a method corresponding to the orthogonal transform process of step S106, and derives a prediction residual D'. Note that this inverse orthogonal transform process is similar to the inverse orthogonal transform process (described later) performed on the decoding side, and therefore the explanation (described later) on the decoding side can be applied to the inverse orthogonal transform process of step S109.

[0160] In step S110, the calculation unit 119 generates a locally decoded image by adding the prediction image obtained by the prediction process in step S104 to the prediction residual D' derived in the process in step S109.

[0161] In step S111, the in-loop filter unit 120 performs an in-loop filter process on the locally decoded image derived in the process of step S110.

[0162] In step S112, the frame memory 121 stores the locally decoded image derived by the process of step S110 and the locally decoded image filtered in step S112.

[0163] In step S113, the encoding unit 115 encodes the quantized transform coefficient level obtained by the process of step S107. For example, the encoding unit 115 encodes the quantized transform coefficient level, which is information related to an image, by arithmetic encoding or the like to generate encoded data. At this time, the encoding unit 115 also encodes various encoding parameters (header information Hinfo, prediction mode information Pinfo, and transformation information Tinfo). Furthermore, the encoding unit 115 derives residual information RInfo from the quantized transform coefficient level, and encodes the residual information RInfo.

[0164] In step S114, the accumulation buffer 116 accumulates the encoded data thus obtained and outputs it, for example, as a bit stream, to the outside of the image encoding device 100. This bit stream is transmitted to the decoding side, for example, via a transmission path or a recording medium. In addition, the rate control unit 123 performs rate control as necessary.

[0165] When the process of step S114 ends, the image encoding process ends.

[0166] In the image coding process having the above flow, the process to which the present technology is applied is performed as the process of step S106. The process to which the present technology is applied is performed as the process of step S109. The process to which the present technology is applied is performed as the process of step S113. Therefore, by performing this image coding process, it is possible to suppress an increase in memory capacity required for orthogonal transform and inverse orthogonal transform.

[0167] <Flow of orthogonal transform processing> Next, an example of the flow of the orthogonal transform process executed in step S106 in FIG. 13 will be described with reference to the flowchart in FIG.

[0168] When the orthogonal transform process is started, the switch 151 determines in step S131 whether the transform skip flag ts_flag is 2D_TS (indicating a two-dimensional transform skip) (for example, 1 (true)) or the transform quantization bypass flag transquant_bypass_flag is 1 (true). If it is determined that the transform skip flag ts_flag is 2D_TS (for example, 1 (true)) or the transform quantization bypass flag is 1 (true), the orthogonal transform process ends and the process returns to FIG. 13. In this case, the orthogonal transform process (primary transform or secondary transform) is omitted, and the input prediction residual D is set as the transform coefficient Coeff.

[0169] 14, if it is determined that the transform skip flag ts_flag is not 2D_TS (not a two-dimensional transform skip) (e.g., 0 (false)) and the transform quantization bypass flag transquant_bypass_flag is 0 (false), the process proceeds to step S132. In this case, the primary transform process and the secondary transform process are performed.

[0170] In step S132, the primary transform unit 152 performs primary transform processing on the input prediction residuals D based on the adaptive primary transform information specified by the component identifier compID, and derives transform coefficients Coeff_P after primary transform.

[0171] In step S133, the secondary transform unit 153 performs a secondary transform process on the transform coefficient Coeff_P to derive a transform coefficient Coeff_S (transform coefficient Coeff) after the secondary transform.

[0172] When the process of step S133 ends, the orthogonal transform process ends.

[0173] In the orthogonal transform process described above, the process to which the present technology is applied is performed as the process of step S132. Therefore, by performing this orthogonal transform process, it is possible to suppress an increase in memory capacity required for the primary transform.

[0174] <Image Decoding Device> FIG. 15 is a block diagram showing an example of the configuration of an image decoding device, which is one aspect of an image processing device to which the present technology is applied. The image decoding device 200 shown in FIG. 15 is a device that decodes encoded data in which a prediction residual between an image and its predicted image is encoded, such as AVC or HEVC. For example, the image decoding device 200 implements the technology described in Non-Patent Document 1, Non-Patent Document 3, or Non-Patent Document 4, and decodes encoded data in which image data of a moving image is encoded by a method that complies with a standard described in any of these documents. For example, the image decoding device 200 decodes encoded data (bit stream) generated by the above-mentioned image encoding device 100.

[0175] Note that Fig. 15 shows the main processing units, data flows, and the like, and is not limited to all that is shown in Fig. 15. In other words, in the image decoding device 200, there may be processing units that are not shown as blocks in Fig. 15, and there may be processing and data flows that are not shown as arrows or the like in Fig. 15. This is the same in other figures that explain processing units and the like in the image decoding device 200.

[0176] 15, the image decoding device 200 includes an accumulation buffer 211, a decoding unit 212, an inverse quantization unit 213, an inverse orthogonal transformation unit 214, a calculation unit 215, an in-loop filter unit 216, a rearrangement buffer 217, a frame memory 218, and a prediction unit 219. The prediction unit 219 includes an intra prediction unit and an inter prediction unit, not shown. The image decoding device 200 is a device for generating moving image data by decoding encoded data (bit stream).

[0177] <Accumulation buffer> The accumulation buffer 211 acquires and holds (stores) the bitstream input to the image decoding device 200. The accumulation buffer 211 supplies the accumulated bitstream to the decoding unit 212 at a predetermined timing or when a predetermined condition is met.

[0178] <Decryption section> The decoding unit 212 performs processing related to image decoding. For example, the decoding unit 212 receives a bit stream supplied from the accumulation buffer 211 as input, and performs variable-length decoding of the syntax values ​​of each syntax element from the bit string in accordance with the definition of the syntax table, to derive parameters.

[0179] The parameters derived from the syntax elements and the syntax values ​​of the syntax elements include, for example, header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, residual information Rinfo, filter information Finfo, and other information. That is, the decoding unit 212 parses (analyzes and obtains) these pieces of information from the bitstream. These pieces of information will be described below.

[0180] <Header information Hinfo> The header information Hinfo includes, for example, header information such as a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), and a slice header (SH). The header information Hinfo includes, for example, information that specifies an image size (horizontal width PicWidth, vertical width PicHeight), bit depth (luminance bitDepthY, chrominance bitDepthC), a chrominance array type ChromaArrayType, a maximum value MaxCUSize / minimum value MinCUSize of CU size, a maximum depth MaxQTDepth / minimum depth MinQTDepth of quad-tree partitioning, a maximum depth MaxBTDepth / minimum depth MinBTDepth of binary-tree partitioning, a maximum value MaxTSSize of transform skip blocks (also called maximum transform skip block size), an on / off flag (also called valid flag) of each encoding tool, and the like.

[0181] For example, the on / off flag of the encoding tool included in the header information Hinfo includes on / off flags related to the conversion and quantization processes shown below. The on / off flag of the encoding tool can also be interpreted as a flag indicating whether or not syntax related to the encoding tool is present in the encoded data. When the value of the on / off flag is 1 (true), it indicates that the encoding tool is usable, and when the value of the on / off flag is 0 (false), it indicates that the encoding tool is not usable. The interpretation of the flag value may be reversed.

[0182] Inter-component prediction enabled flag (ccp_enabled_flag): Flag information indicating whether inter-component prediction (CCP (Cross-Component Prediction), also called CC prediction) is available. For example, when this flag information is "1" (true), it indicates that it is available, and when it is "0" (false), it indicates that it is not available.

[0183] This CCP is also called component-to-component linear prediction (CCLM or CCLMP).

[0184] <Prediction mode information Pinfo> The prediction mode information Pinfo includes, for example, size information PBSize (prediction block size) of the processing target PB (prediction block), intra prediction mode information IPinfo, motion prediction information MVinfo, and other information.

[0185] The intra prediction mode information IPinfo includes, for example, prev_intra_luma_pred_flag, mpm_idx, and rem_intra_pred_mode in JCTVC-W1005, 7.3.8.5 Coding Unit syntax, and a luma intra prediction mode IntraPredModeY derived from the syntax.

[0186] In addition, the intra-prediction mode information IPinfo includes, for example, an inter-component prediction flag (ccp_flag (cclmp_flag)), a multi-class linear prediction mode flag (mclm_flag), a chrominance sample position type identifier (chroma_sample_loc_type_idx), a chrominance MPM identifier (chroma_mpm_idx), and a luma intra-prediction mode (IntraPredModeC) derived from these syntaxes.

[0187] The inter-component prediction flag (ccp_flag (cclmp_flag)) is flag information indicating whether or not inter-component linear prediction is applied. For example, when ccp_flag==1, it indicates that inter-component prediction is applied, and when ccp_flag==0, it indicates that inter-component prediction is not applied.

[0188] The multi-class linear prediction mode flag (mclm_flag) is information on the linear prediction mode (linear prediction mode information). More specifically, the multi-class linear prediction mode flag (mclm_flag) is flag information indicating whether or not to use the multi-class linear prediction mode. For example, "0" indicates a one-class mode (single class mode) (e.g., CCLMP), and "1" indicates a two-class mode (multi-class mode) (e.g., MCLMP).

[0189] The chroma sample location type identifier (chroma_sample_loc_type_idx) is an identifier for identifying the type of pixel location of the chroma component (also called the chroma sample location type). For example, when the chroma array type (ChromaArrayType), which is information on the color format, indicates the 420 format, the chroma sample location type identifier is assigned as shown in the following formula (7).

[0190]

number

[0191] Note that this chrominance sample location type identifier (chroma_sample_loc_type_idx) is transmitted (stored in) information about the pixel location of the chrominance component (chroma_sample_loc_info()).

[0192] The chroma MPM identifier (chroma_mpm_idx) is an identifier indicating which prediction mode candidate in the chroma intra prediction mode candidate list (intraPredModeCandListC) is to be specified as the chroma intra prediction mode.

[0193] The motion prediction information MVinfo includes information such as merge_idx, merge_flag, inter_pred_idc, ref_idx_LX, mvp_lX_flag, X={0,1}, mvd, etc. (see, for example, JCTVC-W1005, 7.3.8.6 Prediction Unit Syntax).

[0194] Of course, the prediction mode information Pinfo may include any information, and may include information other than the above information.

[0195] <Conversion information Tinfo> The conversion information Tinfo includes, for example, the following information: Of course, the information included in the conversion information Tinfo is arbitrary, and information other than the above information may be included.

[0196] The width size TBWSize and height TBHSize of the transform block to be processed (or the logarithmic values ​​log2TBWSize and log2TBHSize of TBWSize and TBHSize with the base 2 as the logarithms). Transform skip flag (ts_flag): A flag indicating whether or not to skip the (inverse) primary transform and the (inverse) secondary transform. Scan Identifier (scanIdx) Quantization parameter (qp) Quantization matrix (scaling_matrix (e.g., JCTVC-W1005, 7.3.4 Scaling list data syntax))

[0197] <Residual information Rinfo> The residual information Rinfo (see, for example, 7.3.8.11 Residual Coding syntax in JCTVC-W1005) includes, for example, the following syntax:

[0198] cbf(coded_block_flag): Residual data presence flag last_sig_coeff_x_pos: last non-zero coefficient X coordinate last_sig_coeff_y_pos: Y coordinate of the last non-zero coefficient coded_sub_block_flag: Sub-block non-zero coefficient presence / absence flag sig_coeff_flag: Non-zero coefficient presence / absence flag gr1_flag: Flag indicating whether the level of the non-zero coefficient is greater than 1 (also called the GR1 flag) gr2_flag: Flag indicating whether the level of the non-zero coefficient is greater than 2 (also called the GR2 flag) sign_flag: The sign indicating whether a non-zero coefficient is positive or negative (also called the sign sign) coeff_abs_level_remaining: The remaining level of non-zero coefficients (also called the non-zero coefficient residual level) etc.

[0199] Of course, the information included in the residual information Rinfo is arbitrary, and information other than the above information may be included.

[0200] <Filter information Finfo> The filter information Finfo includes, for example, control information related to each of the filter processes shown below.

[0201] Control information for the deblocking filter (DBF) Control information for pixel adaptive offset (SAO) Control information for the adaptive loop filter (ALF) Other control information for linear and nonlinear filters

[0202] More specifically, for example, information specifying a picture to which each filter is applied, an area within a picture, filter On / Off control information for each CU, filter On / Off control information related to slice and tile boundaries, etc. Of course, any information is included in the filter information Finfo, and information other than these pieces of information may be included.

[0203] Returning to the description of the decoding unit 212, the decoding unit 212 derives the quantized transform coefficient level level of each coefficient position in each transform block by referring to the residual information Rinfo. The decoding unit 212 supplies the quantized transform coefficient level level to the inverse quantization unit 213.

[0204] Furthermore, the decoding unit 212 supplies the parsed header information Hinfo, prediction mode information Pinfo, quantization transformation coefficient level level, transformation information Tinfo, and filter information Finfo to each block.

[0205] The header information Hinfo is supplied to the inverse quantization unit 213 , the inverse orthogonal transformation unit 214 , the prediction unit 219 , and the in-loop filter unit 216 . The prediction mode information Pinfo is supplied to the inverse quantization unit 213 and the prediction unit 219 . The transformation information Tinfo is supplied to the inverse quantization unit 213 and the inverse orthogonal transformation unit 214 . The filter information Finfo is supplied to the in-loop filter unit 216 .

[0206] Of course, the above example is merely an example, and the present invention is not limited to this example. For example, each encoding parameter may be supplied to an arbitrary processing unit. In addition, other information may be supplied to an arbitrary processing unit.

[0207] <Inverse quantization section> The inverse quantization unit 213 performs processing related to inverse quantization. For example, the inverse quantization unit 213 receives the transformation information Tinfo and the quantized transformation coefficient level level supplied from the decoding unit 212 as input, scales (inverse quantizes) the value of the quantized transformation coefficient level level based on the transformation information Tinfo, and derives the transformation coefficient Coeff_IQ after inverse quantization.

[0208] Note that this inverse quantization is performed as an inverse process of the quantization performed by the quantization unit 114. Also, this inverse quantization is the same process as the inverse quantization performed by the inverse quantization unit 117. That is, the inverse quantization unit 117 performs the same process (inverse quantization) as the inverse quantization unit 213.

[0209] The inverse quantization unit 213 supplies the derived transform coefficient Coeff_IQ to the inverse orthogonal transformation unit 214 .

[0210] <Inverse orthogonal transform section> The inverse orthogonal transform unit 214 performs processing related to inverse orthogonal transform. For example, the inverse orthogonal transform unit 214 receives the transform coefficient Coeff_IQ supplied from the inverse quantization unit 213 and the transform information Tinfo supplied from the decoding unit 212 as input, and performs inverse orthogonal transform processing on the transform coefficient Coeff_IQ based on the transform information Tinfo to derive a prediction residual D'.

[0211] This inverse orthogonal transform is performed as an inverse process of the orthogonal transform performed by the orthogonal transform unit 113. This inverse orthogonal transform is the same process as the inverse orthogonal transform performed by the inverse orthogonal transform unit 118. That is, the inverse orthogonal transform unit 118 performs the same process (inverse orthogonal transform) as the inverse orthogonal transform unit 214.

[0212] The inverse orthogonal transform unit 214 supplies the derived prediction residual D′ to the calculation unit 215.

[0213] <Arithmetic section> The calculation unit 215 performs processing related to addition of information related to an image. For example, the calculation unit 215 receives as input a prediction residual D' supplied from the inverse orthogonal transform unit 214 and a predicted image P supplied from the prediction unit 219. As shown in the following equation (8), the calculation unit 215 adds the prediction residual D' and a predicted image P (prediction signal) corresponding to the prediction residual D' to generate a locally decoded image R local Derive.

[0214]

number

[0215] The calculation unit 215 calculates the derived locally decoded image R local is supplied to the in-loop filter unit 216 and the frame memory 218.

[0216] <In-loop filter section> The in-loop filter unit 216 performs processing related to in-loop filtering. For example, the in-loop filter unit 216 uses the locally decoded image R local and filter information Finfo supplied from the decoding unit 212. Note that any information is input to the in-loop filter unit 216, and information other than the above information may also be input.

[0217] The in-loop filter unit 216 performs appropriate filtering on the locally decoded image Rlocal based on the filter information Finfo.

[0218] For example, the in-loop filter unit 216 applies four in-loop filters, a bilateral filter, a deblocking filter (DBF (DeBlocking Filter)), an adaptive offset filter (SAO (Sample Adaptive Offset)), and an adaptive loop filter (ALF (Adaptive Loop Filter)), in this order, as described in Non-Patent Document 1. Note that which filters are applied and in what order they are applied are arbitrary and can be appropriately selected.

[0219] The in-loop filter unit 216 performs a filter process corresponding to the filter process performed by the encoding side (for example, the in-loop filter unit 120 of the image encoding device 100). Of course, the filter process performed by the in-loop filter unit 216 is arbitrary and is not limited to the above example. For example, the in-loop filter unit 216 may apply a Wiener filter or the like.

[0220] The in-loop filter unit 216 outputs the filtered locally decoded image R local are supplied to a reordering buffer 217 and a frame memory 218.

[0221] <Sorting Buffer> The reordering buffer 217 receives the locally decoded image R local The reordering buffer 217 stores the locally decoded image R local The reordering buffer 217 reconstructs the decoded images R for each picture using the reordering buffer 217 and holds them (stores them in a buffer). The reordering buffer 217 reorders the obtained decoded images R from the decoding order to the playback order. The reordering buffer 217 outputs the reordered decoded images R to the outside of the image decoding device 200 as video data.

[0222] <Frame memory> The frame memory 218 performs processing related to storage of image-related data. For example, the frame memory 218 stores the locally decoded image R local The decoded image R is reconstructed for each picture and stored in a buffer in the frame memory 218.

[0223] The frame memory 218 also stores the in-loop filtered locally decoded image R localThe prediction unit 219 receives input, reconstructs a decoded image R for each picture, and stores the reconstructed image in a buffer in the frame memory 218. The frame memory 218 appropriately supplies the stored decoded image R (or a part thereof) to the prediction unit 219 as a reference image.

[0224] The frame memory 218 may store header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, filter information Finfo, and the like, which are related to the generation of a decoded image.

[0225] <Prediction Department> The prediction unit 219 performs processing related to the generation of a predicted image. For example, the prediction unit 219 receives prediction mode information Pinfo supplied from the decoding unit 212 as an input, performs prediction by a prediction method specified by the prediction mode information Pinfo, and derives a predicted image P. When deriving the predicted image P, the prediction unit 219 uses a pre-filter or post-filter decoded image R (or a part thereof) stored in the frame memory 218, which is specified by the prediction mode information Pinfo, as a reference image. The prediction unit 219 supplies the derived predicted image P to the calculation unit 215.

[0226] In the image decoding device 200 configured as above, the inverse orthogonal transform unit 214 performs the above-mentioned processing to which the present technology is applied as a derivation unit and an inverse orthogonal transform unit. Also, the decoding unit 212 performs the above-mentioned processing to which the present technology is applied as a decoding unit. Therefore, the image decoding device 200 can suppress an increase in memory capacity required for the inverse orthogonal transform.

[0227] <Details of the inverse orthogonal transform section> Fig. 16 is a block diagram showing an example of the main configuration of the inverse orthogonal transform unit 214 in Fig. 15. As shown in Fig. 16, the inverse orthogonal transform unit 214 has a switch 251, an inverse secondary transform unit 252, and an inverse primary transform unit 253.

[0228] The switch 251 receives the transform coefficient Coeff_IQ and the transform skip flag ts_flag[compID] as input. When the value of the transform skip flag ts_flag[compID] is NO_TS (=0), that is, when the transform skip is not applied, the switch 251 supplies the transform coefficient Coeff_IQ to the inverse secondary transform unit 252. When the value of the transform skip flag ts_flag[compID] is 2D_TS (=1), that is, when the switch 251 indicates that a two-dimensional transform skip is applied, the switch 251 skips the inverse secondary transform unit 252 and the inverse primary transform unit 253, and outputs the transform coefficient Coeff_IQ as a prediction residual D' to the outside of the inverse orthogonal transform unit 214 (supplies the transform coefficient Coeff_IQ to the calculation unit 215).

[0229] The inverse secondary transform unit 252 performs processing related to inverse secondary transform, which is the inverse processing of the secondary transform performed on the encoding side (for example, the secondary transform unit 153 of the image encoding device 100). For example, the inverse secondary transform unit 252 receives as input a secondary transform identifier st_idx, a scan identifier scanIdx indicating a scan method for transform coefficients, and a transform coefficient Coeff_IQ supplied from the switch 251.

[0230] The inverse secondary transform unit 252 performs an inverse secondary transform on the transform coefficient Coeff_IQ based on the secondary transform identifier st_idx and the scan identifier scanIdx, and derives the transform coefficient Coeff_IS after the inverse secondary transform.

[0231] More specifically, when the secondary transform identifier st_idx indicates that an inverse secondary transform is to be applied (st_idx>0), the inverse secondary transform unit 252 performs an inverse secondary transform process corresponding to the secondary transform identifier st_idx on the transform coefficient Coeff_IQ to derive the transform coefficient Coeff_IS after the inverse secondary transform. The inverse secondary transform unit 252 supplies the transform coefficient Coeff_IS after the inverse secondary transform to the inverse primary transform unit 253.

[0232] In addition, if the secondary transform identifier st_idx indicates that the inverse secondary transform is not to be applied (st_idx==0), the inverse secondary transform unit 252 skips the inverse secondary transform and supplies the transform coefficient Coeff_IQ to the inverse primary transform unit 253 as the transform coefficient Coeff_IS after the inverse secondary transform.

[0233] The inverse primary transform unit 253 performs processing related to inverse primary transform, which is the inverse processing of the primary transform performed on the encoding side (for example, the primary transform unit 152 of the image encoding device 100). For example, the inverse primary transform unit 253 receives as input a component identifier compID, an adaptive primary transform flag apt_flag[compID] of the component identifier compID, a primary transform identifier pt_idx[compID] of the component identifier compID, prediction mode information PInfo, a size of the transform block (logarithmic value of width log2TBWSize, logarithmic value of height log2TBHSize), and a transform coefficient Coeff_IS after the inverse secondary transform.

[0234] The inverse primary transform unit 253 refers to the prediction mode information PInfo, the component identifier compID, the adaptive primary transform flag apt_flag[compID] for the component identifier compID, and the primary transform identifier pt_idx[compID] for the component identifier compID, and selects a transform type TrTypeH of the inverse primary horizontal transform corresponding to the component identifier compID (and an inverse primary horizontal transform type identifier TrTypeIdxH indicating said transform type), and a transform type TrTypeV of the inverse primary vertical transform (and an inverse primary vertical transform type identifier TrTypeIdxV indicating said transform type).

[0235] Moreover, the inverse primary transform unit 253 performs an inverse primary vertical transform on the transform coefficient Coeff_IS after the inverse secondary transform, which is determined by the inverse primary vertical transform type identifier TrTypeIdxV (or the inverse primary vertical transform type TrTypeV) and the vertical width log2TBHSize of the transform block, and an inverse primary horizontal transform on the transform coefficient Coeff_IS after the inverse secondary transform, which is determined by the inverse primary horizontal transform type identifier TrTypeIdxH (or the inverse primary horizontal transform type TrTypeH) and the horizontal width log2TBWSize of the transform block, to derive the transform coefficient Coeff_IP after the inverse primary transform. The inverse primary vertical transform is an inverse one-dimensional orthogonal transform in the vertical direction, and the inverse primary horizontal transform is an inverse one-dimensional orthogonal transform in the horizontal direction.

[0236] The inverse primary transform unit 253 outputs the transform coefficients Coeff_IP after the inverse primary transform as prediction residuals D' to the outside of the inverse orthogonal transform unit 214 (to the calculation unit 215).

[0237] In the inverse orthogonal transform unit 214 configured as above, the inverse primary transform unit 253 performs the above-mentioned processing to which the present technology is applied as a derivation unit and an inverse orthogonal transform unit. That is, the derivation unit derives a second transform matrix using a first transform matrix, and the inverse orthogonal transform unit performs an inverse primary transform on the inverse secondary transform result using the second transform matrix derived by the derivation unit. Therefore, it is possible to suppress an increase in memory capacity required for such an inverse primary transform.

[0238] As described above, the inverse primary transform unit 253 performs an inverse primary vertical transform and an inverse primary horizontal transform as the inverse primary transform. That is, the derivation unit derives a second transform matrix for the inverse one-dimensional orthogonal transform in the vertical direction and a second transform matrix for the inverse one-dimensional orthogonal transform in the horizontal direction, and the inverse orthogonal transform unit performs an inverse one-dimensional orthogonal transform in the vertical direction using the second transform matrix for the inverse one-dimensional orthogonal transform in the vertical direction derived by the derivation unit as the inverse primary transform, and further performs an inverse one-dimensional orthogonal transform in the horizontal direction using the second transform matrix for the inverse one-dimensional orthogonal transform in the horizontal direction derived by the derivation unit. Therefore, it is possible to suppress an increase in memory capacity required for such a primary transform in which an inverse one-dimensional orthogonal transform in the vertical direction and an inverse one-dimensional orthogonal transform in the horizontal direction are performed.

[0239] <Flow of image decoding process> Next, a description will be given of the flow of each process executed by the above-described image decoding device 200. First, an example of the flow of an image decoding process will be described with reference to the flowchart in FIG.

[0240] When the image decoding process starts, in step S201, the accumulation buffer 211 acquires and holds (accumulates) encoded data (bit stream) supplied from outside the image decoding device 200.

[0241] In step S202, the decoding unit 212 decodes the coded data (bit stream) to obtain the quantized transform coefficient level level. Furthermore, the decoding unit 212 parses (analyzes and obtains) various coding parameters from the coded data (bit stream) through this decoding.

[0242] In step S203, the inverse quantization unit 213 performs inverse quantization, which is the inverse process of the quantization performed on the encoding side, on the quantized transform coefficient level obtained by the process in step S202 to obtain a transform coefficient Coeff_IQ.

[0243] In step S204, the inverse orthogonal transform unit 214 performs inverse orthogonal transform processing, which is the inverse processing of the orthogonal transform processing performed on the encoding side, on the transform coefficient Coeff_IQ obtained by the processing in step S203, to obtain a prediction residual D'.

[0244] In step S205, the prediction unit 219 performs a prediction process using a prediction method specified by the encoding side based on the information parsed in step S202, and generates a predicted image P by, for example, referring to a reference image stored in the frame memory 218.

[0245] In step S206, the calculation unit 215 adds the prediction residual D′ obtained in the process of step S204 and the predicted image P obtained in the process of step S205 to obtain a locally decoded image R local Derive.

[0246] In step S207, the in-loop filter unit 216 filters the locally decoded image R obtained by the process in step S206. local In-loop filtering is performed on

[0247] In step S208, the sorting buffer 217 sorts the filtered locally decoded image R obtained by the process in step S207. local The decoded images R are derived using the above formula, and the order of the decoded images R is rearranged from the decoding order to the playback order. The decoded images R rearranged in the playback order are output to the outside of the image decoding device 200 as a moving image.

[0248] In step S209, the frame memory 218 stores the locally decoded image R obtained in step S206. local , and the locally decoded image R obtained by the process of step S207 after the filtering process. local At least one of the above is stored.

[0249] When the process of step S209 ends, the image decoding process ends.

[0250] In the image decoding process of the above flow, the process to which the present technology is applied is performed as the process of step S202. Also, the process to which the present technology is applied is performed as the process of step S204. Therefore, by performing this image decoding process, it is possible to suppress an increase in memory capacity required for inverse orthogonal transform.

[0251] <Processing flow of inverse orthogonal transform> Next, an example of the flow of the inverse orthogonal transform process executed in step S204 of Fig. 17 will be described with reference to the flowchart of Fig. 18. When the inverse orthogonal transform process is started, the switch 251 determines in step S231 whether the transform skip flag ts_flag is 2D_TS (two-dimensional transform skip mode) (for example, 1 (true)) or the transform quantization bypass flag transquant_bypass_flag is 1 (true). When it is determined that the transform skip identifier ts_idx is 2D_TS or the transform quantization bypass flag is 1 (true), the inverse orthogonal transform process ends and the process returns to Fig. 17. In this case, the inverse orthogonal transform process (inverse primary transform or inverse secondary transform) is omitted, and the transform coefficient Coeff_IQ is set as the prediction residual D'.

[0252] Also, in step S231, if it is determined that the transform skip identifier ts_idx is not 2D_TS (a mode other than two-dimensional transform skip) (for example, 0 (false)) and the transform quantization bypass flag is 0 (false), the process proceeds to step S232. In this case, the inverse secondary transform process and the inverse primary transform process are performed.

[0253] In step S232, the inverse secondary transform unit 252 performs an inverse secondary transform process on the transform coefficient Coeff_IQ based on the secondary transform identifier st_idx to derive and output the transform coefficient Coeff_IS.

[0254] In step S233, the inverse primary transform unit 253 performs an inverse primary transform process on the transform coefficient Coeff_IS to derive a transform coefficient Coeff_IP (prediction residual D') after the inverse primary transform.

[0255] When the process of step S233 ends, the inverse orthogonal transform process ends.

[0256] In the inverse orthogonal transform process described above, the process to which the present technology is applied is performed as the process of step S233. Therefore, by performing this inverse orthogonal transform process, it is possible to suppress an increase in memory capacity required for the inverse primary transform process.

[0257] <2-2. Example 1-1> <Concept> Next, the derivation examples described with reference to Fig. 10 will be described in more detail. First, in the table shown in Fig. 10, excluding the row of item names in the top row, examples in the first and second rows from the top will be described.

[0258] As described above, the derivation of the example in the first row from the top focuses on the similarity between the waveform of the lowest-order row vector of the first transformation type and the waveform of the lowest-order row vector of the transformation type of the alternative transformation matrix. In this case, the derivation unit flips the first transformation matrix to derive the second transformation matrix. That is, as shown in FIG. 19, the derivation unit derives the transformation matrix of DST7 as the base transformation matrix T base Then, it is flipped in the row direction to derive the FlipDST7 transformation matrix. This FlipDST7 transformation matrix can be used to replace the (reduced) DCT8 transformation matrix, which has a similar waveform for the 0th-order row vector.

[0259] A specific example of this derivation is shown in the upper part of Fig. 20. As shown in the upper part of Fig. 20, this derivation is base It can be expressed by the matrix product of (DST7) and a flip matrix J. Here, the flip matrix J (also called a cross-identity matrix) is an N×N unit matrix I that is flipped left and right.

[0260] The derivation of the second example from the top focuses on the similarity between the waveform of the lowest-order column vector of the first transformation type and the waveform of the lowest-order row vector of the transformation type of the alternative transformation matrix. In this case, the derivation unit transposes the first transformation matrix to derive the second transformation matrix. That is, as shown in FIG. 19, the derivation unit transposes the transformation matrix of DST7 to the base transformation matrix T base Then, it is transposed to derive the transformation matrix of TrDST7. This transformation matrix of TrDST7 can be used to replace the transformation matrix of DST1, which has a similar (mountain-shaped) waveform of the 0th-order row vector.

[0261] A specific example of this derivation is shown in the lower part of Fig. 20. As shown in the lower part of Fig. 20, this derivation is performed by using the base transformation matrix T base It can be expressed as the transpose of (DST7).

[0262] In other words, in both of the above two derivation examples, the second transformation matrix can be derived with one operation (flip or transpose). In addition, the operation is also easy. In other words, the second transformation matrix can be easily derived.

[0263] In addition, by applying the above two derivation examples, it is no longer necessary to prepare DCT8 transform matrices and DST1 transform matrices as candidates for transform matrices used in orthogonal transform and inverse orthogonal transform. In other words, it is possible to reduce the number of eigentransform types.

[0264] In this case, as shown in the table of Fig. 21, the number of transformation matrices required in the technique described in Non-Patent Document 1 (see Table A in Fig. 6) can be reduced to three, so the total LUT size can be about 40 KB. In other words, the size of the LUT can be reduced (by about 53 KB (Table A in Fig. 6)) compared to the technique described in Non-Patent Document 1. In other words, an increase in the size of the LUT can be suppressed.

[0265] As described above, in this case too, by performing orthogonal transform and inverse orthogonal transform using the derived transform matrix of the second transform type (TrDST7 or FlipDST7), coding efficiency can be obtained that is similar to that when a DST1 transform matrix or a DCT8 transform matrix is ​​used for the orthogonal transform and inverse orthogonal transform.

[0266] <Primary conversion section> Next, the configuration and processing for performing such derivation will be described. Fig. 22 is a block diagram showing an example of the main configuration of the primary conversion unit 152 in this case. As shown in Fig. 22, the primary conversion unit 152 has a primary conversion selection unit 311, a primary horizontal conversion unit 312, and a primary vertical conversion unit 313.

[0267] The primary transform selection unit 311 receives as input prediction mode information PInfo, a component identifier compID, an adaptive primary transform flag apt_flag[compID], and a primary transform identifier pt_idx[compID]. The primary transform selection unit 311 derives a transform type identifier TrTypeIdxH of the primary horizontal transform and a transform type identifier TrTypeIdxV of the primary vertical transform by referring to the information. The primary transform selection unit 311 supplies the derived transform type identifier TrTypeIdxH of the primary horizontal transform to the primary horizontal transform unit 312. In addition, the primary transform selection unit 311 supplies the derived transform type identifier TrTypeIdxV of the primary vertical transform to the primary vertical transform unit 313.

[0268] The primary horizontal transform unit 312 receives as input the prediction residual D, a transform type identifier TrTypeIdxH of the primary horizontal transform, and information on the size of the transform block (not shown). The information on the size of the transform block may be a natural number N indicating the horizontal or vertical size (number of coefficients) of the transform block, or may be log2TBWSize (logarithm of the width) indicating the width of the transform block (N = 1 << log2TBWSize). The primary horizontal transform unit 312 performs a primary horizontal transform Phor determined by the transform type identifier TrTypeIdxH and the size of the transform block on the prediction residual D, and derives transform coefficients Coeff_Phor after the primary horizontal transform. The primary horizontal transform unit 312 supplies the transform coefficients Coeff_Phor after the primary horizontal transform to the primary vertical transform unit 313.

[0269] The primary vertical transform unit 313 receives as input the transform coefficient Coeff_Phor after the primary horizontal transform, the transform type identifier TrTypeIdxV of the primary vertical transform, and information on the size of the transform block (not shown). The information on the size of the transform block may be a natural number N indicating the horizontal or vertical size (number of coefficients) of the transform block, or may be log2TBHSize (logarithm of the vertical width) indicating the vertical width of the transform block (N=1<<log2TBHSize). The primary vertical transform unit 313 performs a primary vertical transform Pver determined by the transform type identifier TrTypeIdxV and the size of the transform block on the transform coefficient Coeff_Phor after the primary horizontal transform, and derives the transform coefficient Coeff_Pver after the primary vertical transform. The primary vertical transform unit 313 outputs the transform coefficient Coeff_Pver after the primary vertical transform to the outside of the primary transform unit 152 as the transform coefficient Coeff_P after the primary transform (supplies it to the secondary transform unit 153).

[0270] In the primary transformation unit 152 configured as above, the primary horizontal transformation unit 312 and the primary vertical transformation unit 313 respectively function as a derivation unit and an orthogonal transformation unit and perform the above-described processing to which the present technology is applied.

[0271] That is, the primary horizontal transform unit 312, as a derivation unit, derives a second transform matrix for one-dimensional orthogonal transform in the horizontal direction, and further, as an orthogonal transform unit, performs one-dimensional orthogonal transform in the horizontal direction using the second transform matrix for one-dimensional orthogonal transform in the horizontal direction derived by the derivation unit. Therefore, the primary horizontal transform unit 312 can suppress an increase in memory capacity required for one-dimensional orthogonal transform in the horizontal direction.

[0272] Moreover, the primary vertical transform unit 313, as a derivation unit, derives a second transform matrix for one-dimensional orthogonal transform in the vertical direction, and further, as an orthogonal transform unit, performs one-dimensional orthogonal transform in the vertical direction using the second transform matrix for one-dimensional orthogonal transform in the vertical direction derived by the derivation unit. Therefore, the primary vertical transform unit 313 can suppress an increase in memory capacity required for one-dimensional orthogonal transform in the vertical direction.

[0273] <Primary horizontal conversion section> Fig. 23 is a block diagram showing an example of the main configuration of primary horizontal transformation unit 312 in Fig. 22. As shown in Fig. 23, primary horizontal transformation unit 312 has a transformation matrix derivation unit 321, a matrix operation unit 322, a scaling unit 323, and a clip unit 324.

[0274] The transformation matrix derivation unit 321 receives as input the transformation type identifier TrTypeIdxH of the primary horizontal transform and information regarding the size of the transform block, and derives a transformation matrix T H (Transformation matrix T for one-dimensional orthogonal transformation in the horizontal direction H The transformation matrix derivation unit 321 derives the transformation matrix T H is supplied to the matrix calculation unit 322.

[0275] The matrix calculation unit 322 uses the transformation matrix T H Using the input data X in (i.e., the transform block of the prediction residual D) is subjected to a horizontal one-dimensional orthogonal transform to obtain intermediate data Y1. This calculation can be expressed as a determinant as in the following equation (9).

[0276]

number

[0277] The matrix calculation unit 322 supplies the intermediate data Y 1 to the scaling unit 323 .

[0278] The scaling unit 323 shifts the coefficient Y1[i,j] of each i-th row and j-th column component of the intermediate data Y1 by a predetermined shift amount S H to obtain intermediate data Y2. This scaling can be expressed as in the following equation (10). Hereinafter, the i-th row and j-th column component ((i,j) component) of a two-dimensional matrix (two-dimensional array) X is represented as X[i,j].

[0279]

number

[0280] The scaling unit 323 supplies the intermediate data Y2 to the clipping unit 324.

[0281] The clipping unit 324 clips the value of the coefficient Y2[i,j] of each i-th row and j-th column component of the intermediate data Y2, and outputs the output data X out (That is, the transform coefficient Coeff_Phor after the primary horizontal transform is derived.) This process can be expressed as in the following equation (11).

[0282]

number

[0283] The clip unit 324 clips the output data X out (The transform coefficient after primary horizontal transformation Coeff_Phor) is output to the outside of the primary horizontal transform unit 312 (supplied to the primary vertical transform unit 313).

[0284] In the primary horizontal transform unit 312 configured as above, the transform matrix derivation unit 321 performs the above-mentioned processing to which the present technology is applied as a derivation unit. Also, the matrix operation unit 322 performs the above-mentioned processing to which the present technology is applied as an orthogonal transform unit. Therefore, the primary horizontal transform unit 312 can suppress an increase in memory capacity required for one-dimensional orthogonal transform in the horizontal direction.

[0285] <Transformation matrix derivation part> Fig. 24 is a block diagram showing an example of a main configuration of the transformation matrix derivation unit 321 in Fig. 23. As shown in Fig. 24, the transformation matrix derivation unit 321 has a transformation matrix LUT 331, a flip unit 332, and a transposition unit 333. Note that although arrows indicating data exchange are omitted in Fig. 24, the transformation matrix derivation unit 321 can exchange any data between any processing units (processing blocks).

[0286] The transformation matrix LUT 331 is a lookup table for holding (storing) a transformation matrix corresponding to the transformation type identifier TrTypeIdxH of the primary horizontal transform and the size N of the transformation block. When the transformation type identifier TrTypeIdxH of the primary horizontal transform and the size N of the transformation block are specified, the transformation matrix LUT 331 selects and outputs a transformation matrix corresponding to them. In the case of this derivation example, the transformation matrix LUT 331 converts the transformation matrix into a base transformation matrix T base to the flip section 332 or the transposition section 333, or to both.

[0287] The flip unit 332 flips the input transformation matrix T with N rows and N columns, and outputs the transformed matrix T after the flip. flip In this derivation example, the flip unit 332 outputs the base transformation matrix T baseThe input is the base transformation matrix T base Flip in the row direction (horizontal direction) to obtain the transformation matrix T flip Let us use the transformation matrix T H and outputs it to the outside of the transformation matrix derivation unit 321 (supplying it to the matrix calculation unit 322).

[0288] The transposition unit 333 transposes the input transformation matrix T with N rows and N columns to obtain the transposed transformation matrix T transpose In this derivation example, the transposition unit 333 outputs the base transformation matrix T base The input is the base transformation matrix T base Transpose the transformation matrix T transpose Let us use the transformation matrix T H and outputs it to the outside of the transformation matrix derivation unit 321 (supplying it to the matrix calculation unit 322).

[0289] As described above, the transformation matrix derivation unit 321 has the flip unit 332 and the transposition unit 333. Therefore, the transformation matrix derivation unit 321 can realize the derivation example of the example in the first row from the top of the table shown in Fig. 10, for example, by using the flip unit 332. Also, the transformation matrix derivation unit 321 can realize the derivation example of the example in the second row from the top of the table shown in Fig. 10, for example, by using the transposition unit 333.

[0290] <Primary vertical conversion section> Fig. 25 is a block diagram showing an example of the main configuration of the primary vertical transform unit 313 in Fig. 22. As shown in Fig. 25, the primary vertical transform unit 313 has a transform matrix derivation unit 351, a matrix operation unit 352, a scaling unit 353, and a clip unit 354.

[0291] The transformation matrix derivation unit 351 receives the transformation type identifier TrTypeIdxV of the primary vertical transformation and information about the size of the transformation block, and derives a transformation matrix T V (Transformation matrix T for one-dimensional orthogonal transformation in the vertical direction VThe transformation matrix derivation unit 351 derives the transformation matrix T V is supplied to the matrix calculation unit 352.

[0292] The matrix calculation unit 352 uses the transformation matrix T V Using the input data X in (i.e., the transform block of the transform coefficients Coeff_Phor after the primary horizontal transform) is subjected to a one-dimensional orthogonal transform in the vertical direction to obtain intermediate data Y1. This calculation can be expressed as a determinant as in the following equation (12).

[0293]

number

[0294] The matrix calculation unit 352 supplies the intermediate data Y 1 to the scaling unit 353 .

[0295] The scaling unit 353 shifts the coefficient Y1[i,j] of each i-th row and j-th column component of the intermediate data Y1 by a predetermined shift amount S V This scaling can be expressed by the following equation (13):

[0296]

number

[0297] The scaling unit 353 supplies the intermediate data Y2 to the clipping unit 354.

[0298] The clipping unit 354 clips the value of the coefficient Y2[i,j] of each i-th row and j-th column component of the intermediate data Y2, and outputs the output data X out (That is, the transform coefficient Coeff_Pver after the primary vertical transform is derived.) This process can be expressed as in the following equation (14).

[0299]

number

[0300] The clip unit 324 clips the output data X out (The transform coefficient after primary vertical transform Coeff_Pver) is output to the outside of the primary vertical transform unit 313 as the transform coefficient after primary transform Coeff_P (supplied to the secondary transform unit 153).

[0301] In the primary vertical transform unit 313 configured as above, the transform matrix derivation unit 351 performs the process to which the present technology is applied as a derivation unit. Also, the matrix operation unit 352 performs the process to which the present technology is applied as an orthogonal transform unit. Therefore, the primary vertical transform unit 313 can suppress an increase in memory capacity required for one-dimensional orthogonal transform in the vertical direction.

[0302] <Transformation matrix derivation part> Fig. 26 is a block diagram showing an example of a main configuration of the transformation matrix derivation unit 351 in Fig. 25. As shown in Fig. 26, the transformation matrix derivation unit 351 has a transformation matrix LUT 361, a flip unit 362, and a transposition unit 363. Note that although arrows indicating data exchange are omitted in Fig. 26, the transformation matrix derivation unit 351 can exchange any data between any processing units (processing blocks).

[0303] The transformation matrix LUT 361 is a lookup table for holding (storing) a transformation matrix corresponding to the transformation type identifier TrTypeIdxV of the primary vertical transformation and the size N of the transformation block. When the transformation type identifier TrTypeIdxV of the primary vertical transformation and the size N of the transformation block are specified, the transformation matrix LUT 361 selects and outputs a transformation matrix corresponding to them. In this derivation example, the transformation matrix LUT 361 converts the transformation matrix into a base transformation matrix T base to the flip section 362 or the transposition section 363, or both.

[0304] The flip unit 362 flips the input transformation matrix T with N rows and N columns, and outputs the transformed matrix T after the flip. flipIn this derivation example, the flip unit 362 outputs the base transformation matrix T base The input is the base transformation matrix T base Flip in the row direction (horizontal direction) to obtain the transformation matrix T flip Let us use the transformation matrix T V and outputs it to the outside of the transformation matrix derivation unit 351 (supplying it to the matrix calculation unit 352).

[0305] The transposition unit 363 transposes the input transformation matrix T with N rows and N columns, and obtains the transposed transformation matrix T transpose In this derivation example, the transposition unit 363 outputs the base transformation matrix T base The input is the base transformation matrix T base Transpose the transformation matrix T transpose Let us use the transformation matrix T V and outputs it to the outside of the transformation matrix derivation unit 351 (supplying it to the matrix calculation unit 352).

[0306] As described above, the transformation matrix derivation unit 351 has the flip unit 362 and the transposition unit 363. Therefore, the transformation matrix derivation unit 351 can realize the derivation example of the example in the first row from the top of the table shown in Fig. 10, for example, by using the flip unit 362. Also, the transformation matrix derivation unit 351 can realize the derivation example of the example in the second row from the top of the table shown in Fig. 10, for example, by using the transposition unit 363.

[0307] <Primary conversion process flow> Next, an example of the flow of the process performed by the above-mentioned configuration will be described. In this case, an example of the flow of the primary conversion process executed in step S132 in FIG. 14 will be described with reference to the flowchart in FIG.

[0308] When the primary transformation process is started, in step S301, the primary transformation selection unit 311 (Figure 22) of the primary transformation unit 152 selects the transformation type identifier TrTypeIdxH (and the transformation type TrTypeH specified by the identifier) ​​of the primary horizontal transformation and the transformation type identifier TrTypeIdxV (and the transformation type TrTypeV specified by the identifier) ​​of the primary vertical transformation, as described above.

[0309] In step S302, the primary horizontal transform unit 312 performs primary horizontal transform processing corresponding to the transform type identifier TrTypeIdxH of the primary horizontal transform obtained in step S301, on the prediction residuals D, and derives transform coefficients Coeff_Phor after the primary horizontal transform.

[0310] In step S303, the primary vertical transformation unit 313 performs a primary vertical transformation process corresponding to the transformation type identifier TrTypeIdxV of the primary vertical transformation obtained in step S301 on the primary horizontal transformation result (the transformation coefficient Coeff_Phor after the primary horizontal transformation), and derives the transformation coefficient Coeff_Pver after the primary vertical transformation (the transformation coefficient Coeff_P after the primary transformation).

[0311] When the process of step S303 ends, the primary conversion process ends, and the process returns to FIG.

[0312] In the primary conversion process described above, the process to which the present technology is applied is performed as the process in step S302 or step S303. Therefore, by performing this primary conversion process, it is possible to suppress an increase in memory capacity required for the primary horizontal conversion process or the primary vertical conversion process.

[0313] <Primary horizontal conversion process flow> The flow of the primary horizontal transformation process executed in step S302 in FIG. 27 will be described with reference to the flowchart in FIG.

[0314] When the primary horizontal transform process is started, in step S321, the transform matrix derivation unit 321 (FIG. 23) of the primary horizontal transform unit 312 calculates a transform matrix T H Derive.

[0315] In step S322, the matrix calculation unit 322 calculates the derived transformation matrix T H Using the input data X in A horizontal one-dimensional orthogonal transform is performed on (the prediction residual D) to obtain intermediate data Y1. This process can be expressed as a determinant, as shown in the above formula (9). Furthermore, this process can be expressed as an element-by-element calculation, as shown in the following formula (15).

[0316]

number

[0317] That is, as shown in FIG. 29, the coefficient Y1[i,j] of the i-th row and j-th column component of the intermediate data Y1 is in The row vector X in the i-th row of in [i,:] and the transformation matrix T H The jth row vector T H Transpose matrix T of [j,:] H T Set the dot product with [:,j] (j=0,···,M-1, i=0,···,N-1), where M is the input data X in is the size in the x direction of the input data in is the size in the y direction of the pixel. M and N can be expressed as in the following equation (16).

[0318]

number

[0319] Returning to FIG. 28, in step S323, the scaling unit 323 scales the coefficient Y1[i,j] of each i-th row and j-th column component of the intermediate data Y1 derived by the process of step S322 by a shift amount S H The intermediate data Y2 is derived by scaling using the above formula (10).

[0320] In step S324, the clipping unit 324 clips the value of the coefficient Y2[i,j] of each i-th row and j-th column component of the intermediate data Y2 derived by the process in step S323, and obtains the output data X out (That is, the transform coefficient Coeff_Phor after the primary horizontal transform is obtained.) This process can be expressed as in the above-mentioned equation (11).

[0321] When the process of step S324 ends, the primary horizontal transformation process ends, and the process returns to FIG.

[0322] In the primary horizontal transform process as described above, the process to which the present technology is applied is performed as the process of step S321 and step S322. Therefore, by performing this primary horizontal transform process, it is possible to suppress an increase in memory capacity required for one-dimensional orthogonal transform in the horizontal direction.

[0323] <Flow of transformation matrix derivation process> Next, an example of the flow of the transformation matrix derivation process executed in step S321 of FIG. 28 will be described with reference to the flowchart of FIG.

[0324] When the transformation matrix derivation process is started, in step S341, the transformation matrix derivation unit 321 determines the base transformation type BaseTrType corresponding to the transformation type identifier TrTypeIdxH by referring to a correspondence table such as that shown in FIG. 31. This process can be expressed mathematically, for example, as shown in equation (17). Furthermore, the transformation matrix of N rows and N columns of the determined base transformation type is read from the transformation matrix LUT, and is converted into the base transformation matrix T base Set to.

[0325]

number

[0326] Moreover, the transformation matrix derivation unit 321 sets a value corresponding to the transformation type identifier TrTypeIdxH to ​​the flip flag FlipFlag as shown in the following formula (19). Moreover, the transformation matrix derivation unit 321 sets a value corresponding to the transformation type identifier TrTypeIdxH to ​​the transpose flag TransposeFlag as shown in the following formula (20).

[0327]

number

[0328] In step S342, the transformation matrix derivation unit 321 determines whether or not the flip flag FlipFlag and the transpose flag TransposeFlag satisfy a condition (ConditionA1) expressed by the following equation (21).

[0329]

number

[0330] If it is determined that the above condition (ConditionA1) is satisfied (if both the flip flag FlipFlag and the transpose flag TransposeFlag are false (0)), the process proceeds to step S343.

[0331] In step S343, the transformation matrix derivation unit 321 calculates the base transformation matrix T base Transform the matrix T H Set it to.

[0332]

number

[0333] When the process of step S343 ends, the transformation matrix derivation process ends, and the process returns to Fig. 28. Also, if it is determined in step S342 that the above-mentioned condition (ConditionA1) is not satisfied (if the flip flag FlipFlag or the transpose flag TransposeFlag is true (1)), the process proceeds to step S344.

[0334] In step S344, the transformation matrix derivation unit 321 determines whether or not the flip flag FlipFlag and the transpose flag TransposeFlag satisfy a condition (ConditionA2) expressed by the following equation (23).

[0335]

number

[0336] If it is determined that the above condition (ConditionA2) is satisfied (if the flip flag FlipFlag is false (0) and the transpose flag TransposeFlag is true (1)), the process proceeds to step S345.

[0337] In step S345, the transformation matrix derivation unit 321 derives the base transformation matrix T base Transpose the transformation matrix T H This process can be expressed as a determinant as shown in the following equation (24).

[0338]

number

[0339] In addition, when this process is expressed as an element-by-element calculation, the transformation matrix derivation unit 321 calculates the base transformation matrix T base The i-th row and j-th column components ((i,j) components) of the transformation matrix T H Set it to the (j,i) component of

[0340]

number

[0341] Here, the transformation matrix T H The i-th row and j-th column component ((i,j) component) of T H The notation is [i,j]. Also, the second line, "for i,j=0,...,N-1", indicates that i and j have values ​​from 0 to N-1. H [j,i] is the N-row, N-column transformation matrix T H means to show all elements of

[0342] In this way, by expressing the process of step S345 as an element-by-element operation, the transposition operation can be realized by simply accessing a two-dimensional array. When the process of step S345 ends, the transformation matrix derivation process ends, and the process returns to FIG.

[0343] Also, if it is determined in step S344 that the above-mentioned condition (ConditionA2) is not satisfied (if the flip flag FlipFlag is true (1) or the transpose flag TransposeFlag is false (0)), the process proceeds to step S346.

[0344] In step S346, the transformation matrix derivation unit 321 receives the base transformation matrix T base Flip the transformation matrix T H This process can be expressed as a determinant as shown in the following equation (26).

[0345]

number

[0346] Here, × is an operator representing matrix multiplication. Also, the flip matrix J (Cross-Identity Matrix) is the left-right flip of the N-row, N-column unit matrix I.

[0347] When this process is expressed as an element-by-element calculation, the transformation matrix derivation unit 321 calculates the transformation matrix T H The (i, N-1-j) component of the base transformation matrix Tbase is set to the i-th row and j-th column component ((i, j) component) of

[0348]

number

[0349] Here, the transformation matrix T H The i-th row and j-th column component ((i,j) component) of T H The notation is [i,j]. Also, the second line "for i,j=0,...,N-1" indicates that i and j each have a value from 0 to N-1. That is, T H [i,j] is the N-row, N-column transformation matrix T H means to show all elements of

[0350] In this way, by expressing the process of step S346 as an element-by-element operation, the base transformation matrix T base The transposition operation can be realized by simply accessing a two-dimensional array, without performing a matrix operation between and the flip matrix J. Also, the flip matrix J becomes unnecessary. When the process of step S346 ends, the transformation matrix derivation process ends, and the process returns to FIG. 28.

[0351] In the above transformation matrix derivation process, the above-mentioned processing of the present technology is performed as the processing of step S345 and step S346. Therefore, by performing this transformation matrix derivation process, it is possible to realize the derivation examples in the first and second rows from the top of the table shown in Fig. 10 in the horizontal one-dimensional orthogonal transformation. Therefore, it is possible to suppress an increase in the required memory capacity.

[0352] Note that a branch as described below may be inserted between the process of step S344 and the process of step S346. That is, in that step, the transformation matrix derivation unit 321 determines whether or not the flip flag FlipFlag and the transpose flag TransposeFlag satisfy a condition (ConditionA3) expressed by the following formula (28).

[0353]

number

[0354] If the transformation matrix derivation unit 321 determines that the above condition (ConditionA3) is satisfied (if the flip flag FlipFlag is true (1) and the transpose flag TransposeFlag is false (0)), the process proceeds to step S346.

[0355] Also, if it is determined that the above-mentioned condition (Condition A3) is not satisfied (if the flip flag FlipFlag is false (0) or the transpose flag TransposeFlag is true (1)), the transformation matrix derivation process ends and the process returns to Figure 28.

[0356] <Primary vertical conversion process flow> Next, the flow of the primary vertical transformation process executed in step S303 in FIG. 27 will be described with reference to the flowchart in FIG.

[0357] When the primary vertical transformation process is started, in step S361, the transformation matrix derivation unit 351 (FIG. 25) of the primary vertical transformation unit 313 executes the transformation matrix derivation process to derive a transformation matrix T V Derive.

[0358] The flow of this transformation matrix derivation process is the same as that of the primary horizontal transformation described with reference to the flowchart in Fig. 30, and therefore the description thereof will be omitted. For example, the transformation type identifier TrTypeIdxH of the primary horizontal transformation may be replaced with the transformation type identifier TrTypeIdxV of the primary vertical transformation, or the transformation matrix T H Let us use the transformation matrix T for the primary vertical transformation. V 30. For example, the horizontal direction in the description given with reference to FIG. 30 may be replaced with the vertical direction.

[0359] In step S362, the matrix calculation unit 352 calculates the derived transformation matrix T V Using the input data X in A one-dimensional orthogonal transform is performed in the vertical direction on the transform coefficient Coeff_Phor after the primary horizontal transform to obtain intermediate data Y1. This process can be expressed as a determinant, as shown in the above formula (12). Furthermore, this process can be expressed as an element-by-element calculation, as shown in the following formula (29).

[0360]

number

[0361] That is, in this case, as shown in FIG. 33, the coefficient Y1[i,j] of the i-th row and j-th column component of the intermediate data Y1 is represented by the transformation matrix T V The row vector T of the i-th row V [i,:] and input data X in The j-th column vector X in Set the dot product with [:,j], where j=0,···,M-1, i=0,···,N-1.

[0362] In step S363, the scaling unit 353 scales the coefficient Y1[i,j] of each i-th row and j-th column component of the intermediate data Y1 derived in the process of step S322 by a shift amount S V The intermediate data Y2 is derived by scaling using the above equation (13).

[0363] In step S364, the clipping unit 354 clips the value of the coefficient Y2[i,j] of each i-th row and j-th column component of the intermediate data Y2 derived in the process of step S363, and obtains the output data X out (That is, the transform coefficient Coeff_Pver after the primary vertical transform is obtained.) This process can be expressed as in the above-mentioned equation (14).

[0364] When the process of step S364 ends, the primary horizontal transform process ends, and the process returns to FIG.

[0365] In the primary vertical transform process as described above, the process to which the present technology is applied is performed as the process of step S361 or step S362. Therefore, by performing this primary vertical transform process, it is possible to suppress an increase in memory capacity required for one-dimensional orthogonal transform in the vertical direction.

[0366] <Inverse primary conversion section> Next, a configuration of the image decoding device 200 in this embodiment will be described. Fig. 34 is a block diagram showing a main configuration example of the inverse primary transform unit 253 (Fig. 16) in this case. As shown in Fig. 34, the inverse primary transform unit 253 has an inverse primary transform selection unit 411, an inverse primary vertical transform unit 412, and an inverse primary horizontal transform unit 413.

[0367] The inverse primary transform selection unit 411 receives prediction mode information PInfo, a component identifier compID, an adaptive primary transform flag apt_flag[compID], and a primary transform identifier pt_idx[compID] as input. The inverse primary transform selection unit 411 derives a transform type identifier TrTypeIdxV of the inverse primary vertical transform and a transform type identifier TrTypeIdxH of the inverse primary vertical transform by referring to the information. The inverse primary transform selection unit 411 supplies the derived transform type identifier TrTypeIdxV of the inverse primary vertical transform to the inverse primary vertical transform unit 412. In addition, the inverse primary transform selection unit 411 supplies the derived transform type identifier TrTypeIdxH of the inverse primary horizontal transform to the inverse primary horizontal transform unit 413.

[0368] The inverse primary vertical transform unit 412 receives as input the transform coefficients Coeff_IS after the inverse secondary transform, the transform type identifier TrTypeIdxV of the inverse primary vertical transform, and information on the size of the transform block. The information on the size of the transform block may be a natural number N indicating the horizontal or vertical size (number of coefficients) of the transform block, or may be log2TBHSize (logarithm of the vertical width) indicating the vertical width of the transform block (N = 1 << log2TBHSize). The inverse primary vertical transform unit 412 executes an inverse primary vertical transform IPver determined by the transform type identifier TrTypeIdxV and the size of the transform block on the transform coefficients Coeff_IS after the inverse secondary transform, and derives the transform coefficients Coeff_IPver after the inverse primary vertical transform. The inverse primary vertical transform unit 412 supplies the transform coefficients Coeff_IPver after the inverse primary vertical transform to the inverse primary horizontal transform unit 413.

[0369] The inverse primary horizontal transform unit 413 receives as input the transform coefficients Coeff_IPver after the inverse primary vertical transform, the transform type identifier TrTypeIdxH of the inverse primary horizontal transform, and information on the size of the transform block. The information on the size of the transform block may be a natural number N indicating the horizontal or vertical size (number of coefficients) of the transform block, or may be log2TBWSize (logarithm of the width) indicating the width of the transform block (N = 1 << log2TBWSize). The inverse primary horizontal transform unit 413 executes an inverse primary horizontal transform IPhor determined by the transform type identifier TrTypeIdxH and the size of the transform block on the transform coefficients Coeff_IPver after the inverse primary vertical transform supplied from the inverse primary vertical transform unit 412, and derives the transform coefficients Coeff_IPhor after the inverse primary horizontal transform (i.e., the transform coefficients Coeff_IP after the inverse primary transform). The inverse primary horizontal transform unit 413 outputs the transform coefficients Coeff_IPhor after the inverse primary horizontal transform as prediction residuals D' to the outside of the inverse primary transform unit 253 (to the calculation unit 215).

[0370] In the inverse primary transform unit 253 configured as above, the inverse primary vertical transform unit 412 and the inverse primary horizontal transform unit 413 respectively function as a derivation unit and an orthogonal transform unit and perform the above-described processing to which the present technology is applied.

[0371] That is, the inverse primary vertical transform unit 412, as a derivation unit, derives a second transform matrix for inverse one-dimensional orthogonal transform in the vertical direction, and further, as an inverse orthogonal transform unit, performs inverse one-dimensional orthogonal transform in the vertical direction using the second transform matrix for inverse one-dimensional orthogonal transform in the vertical direction derived by the derivation unit. Therefore, the inverse primary vertical transform unit 412 can suppress an increase in memory capacity required for inverse one-dimensional orthogonal transform in the vertical direction.

[0372] Moreover, the inverse primary horizontal transform unit 413, as a derivation unit, derives a second transform matrix for horizontal inverse one-dimensional orthogonal transform, and further, as an orthogonal transform unit, performs horizontal inverse one-dimensional orthogonal transform using the second transform matrix for horizontal inverse one-dimensional orthogonal transform derived by the derivation unit. Therefore, the inverse primary horizontal transform unit 413 can suppress an increase in memory capacity required for horizontal inverse one-dimensional orthogonal transform.

[0373] <Inverse primary vertical conversion section> Fig. 35 is a block diagram showing an example of the main configuration of the inverse primary vertical transform unit 412 in Fig. 34. As shown in Fig. 35, the inverse primary vertical transform unit 412 has a transformation matrix derivation unit 421, a matrix operation unit 422, a scaling unit 423, and a clip unit 424.

[0374] The transformation matrix derivation unit 421 receives the transformation type identifier TrTypeIdxV of the inverse primary vertical transform and information about the size of the transform block, and derives a transformation matrix T V (Transformation matrix T for inverse 1D orthogonal transformation in the vertical direction V The transformation matrix derivation unit 421 derives the transformation matrix T V is supplied to the matrix calculation unit 422.

[0375] The matrix calculation unit 422 uses the transformation matrix T V Using the input data X in (i.e., the transform block of the transform coefficients Coeff_IS after the inverse secondary transform) is subjected to a vertical inverse one-dimensional orthogonal transform to obtain intermediate data Y1. This calculation can be expressed as a determinant as in the following equation (30).

[0376]

number

[0377] The matrix calculation unit 422 supplies the intermediate data Y 1 to the scaling unit 423 .

[0378] The scaling unit 423 shifts the coefficient Y1[i,j] of each i-th row and j-th column component of the intermediate data Y1 by a predetermined shift amount S IV This scaling can be expressed by the following equation (31):

[0379]

number

[0380] The scaling unit 423 supplies the intermediate data Y2 to the clip unit 424.

[0381] The clipping unit 424 clips the value of the coefficient Y2[i,j] of each i-th row and j-th column component of the intermediate data Y2, and outputs the output data X out (That is, the transform coefficient Coeff_IPver after the inverse primary vertical transform is derived.) This process can be expressed as in the above-mentioned equation (11).

[0382] The clip unit 424 clips the output data X out (The transform coefficient Coeff_IPver after the inverse primary vertical transform) is output to the outside of the inverse primary vertical transform unit 412 (supplied to the inverse primary horizontal transform unit 413).

[0383] In the inverse primary vertical transform unit 412 configured as above, the transform matrix derivation unit 421 performs the process to which the present technology is applied as a derivation unit. Also, the matrix operation unit 422 performs the process to which the present technology is applied as an orthogonal transform unit. Therefore, the inverse primary vertical transform unit 412 can suppress an increase in memory capacity required for the inverse one-dimensional orthogonal transform in the vertical direction.

[0384] <Transformation matrix derivation part> Fig. 36 is a block diagram showing an example of a main configuration of the transformation matrix derivation unit 421 in Fig. 35. As shown in Fig. 36, the transformation matrix derivation unit 421 has a transformation matrix LUT 431, a flip unit 432, and a transposition unit 433. Note that although arrows indicating data exchange are omitted in Fig. 36, the transformation matrix derivation unit 421 can exchange any data between any processing units (processing blocks).

[0385] The transformation matrix LUT 431 is a lookup table for holding (storing) a transformation matrix corresponding to the transformation type identifier TrTypeIdxV of the inverse primary vertical transformation and the size N of the transformation block. When the transformation type identifier TrTypeIdxV of the inverse primary vertical transformation and the size N of the transformation block are specified, the transformation matrix LUT 431 selects and outputs a transformation matrix corresponding to them. In the case of this derivation example, the transformation matrix LUT 431 converts the transformation matrix into a base transformation matrix T base to the flip section 432 or the transposition section 433, or both.

[0386] The flip unit 432 flips the input transformation matrix T with N rows and N columns, and outputs the transformed matrix T after the flip. flip In this derivation example, the flip unit 432 outputs the base transformation matrix T base The input is the base transformation matrix T base Flip in the row direction (horizontal direction) to obtain the transformation matrix T flip Let us use the transformation matrix T V and outputs it to the outside of the transformation matrix derivation unit 321 (supplying it to the matrix calculation unit 422).

[0387] The transposition unit 433 transposes the input transformation matrix T with N rows and N columns to obtain the transposed transformation matrix T transpose In this derivation example, the transposition unit 433 outputs the base transformation matrix T base The input is the base transformation matrix T base Transpose the transformation matrix T transpose Let us use the transformation matrix T Vand outputs it to the outside of the transformation matrix derivation unit 421 (supplying it to the matrix calculation unit 422).

[0388] As described above, the transformation matrix derivation unit 421 has the flip unit 432 and the transposition unit 433. Therefore, the transformation matrix derivation unit 421 can realize the derivation example of the example in the first row from the top of the table shown in Fig. 10, for example, by using the flip unit 432. Also, the transformation matrix derivation unit 421 can realize the derivation example of the example in the second row from the top of the table shown in Fig. 10, for example, by using the transposition unit 433.

[0389] <Inverse primary horizontal conversion section> Fig. 37 is a block diagram showing an example of the main configuration of the inverse primary horizontal transform unit 413 in Fig. 34. As shown in Fig. 37, the inverse primary horizontal transform unit 413 has a transformation matrix derivation unit 451, a matrix operation unit 452, a scaling unit 453, and a clip unit 454.

[0390] The transformation matrix derivation unit 451 receives the transformation type identifier TrTypeIdxH of the inverse primary horizontal transform and information about the size of the transform block as input, and derives a transformation matrix T H (Transformation matrix T for inverse 1D orthogonal transformation in the horizontal direction H The transformation matrix derivation unit 451 derives the transformation matrix T H is supplied to the matrix calculation unit 452.

[0391] The matrix calculation unit 452 uses the transformation matrix T H Using the input data X in (i.e., the transform block of the transform coefficients Coeff_IPver after the inverse primary vertical transform) is subjected to a horizontal inverse one-dimensional orthogonal transform to obtain intermediate data Y1. This calculation can be expressed as a determinant as in the following equation (32).

[0392]

number

[0393] The matrix calculation unit 452 supplies the intermediate data Y 1 to the scaling unit 453 .

[0394] The scaling unit 453 shifts the coefficient Y1[i,j] of each i-th row and j-th column component of the intermediate data Y1 by a predetermined shift amount S IH This scaling can be expressed by the following equation (33):

[0395]

number

[0396] The scaling unit 453 supplies the intermediate data Y2 to the clip unit 454.

[0397] The clipping unit 454 clips the value of the coefficient Y2[i,j] of each i-th row and j-th column component of the intermediate data Y2, and outputs the output data X out (That is, the transform coefficient Coeff_IPhor after the inverse primary horizontal transform is derived.) This process can be expressed as in the above-mentioned equation (11).

[0398] The clip unit 454 clips the output data X out (The transform coefficient Coeff_IPhor after the inverse primary horizontal transform (the transform coefficient Coeff_IP after the inverse primary transform)) is output to the outside of the inverse primary horizontal transform unit 413 as a prediction residual D' (supplied to the calculation unit 215).

[0399] In the inverse primary horizontal transform unit 413 configured as above, the transform matrix derivation unit 451 performs the process to which the present technology is applied as a derivation unit as described above. Also, the matrix operation unit 452 performs the process to which the present technology is applied as an inverse orthogonal transform unit as described above. Therefore, the inverse primary horizontal transform unit 413 can suppress an increase in memory capacity required for the inverse one-dimensional orthogonal transform in the horizontal direction.

[0400] <Transformation matrix derivation part> Fig. 38 is a block diagram showing an example of the main configuration of the transformation matrix derivation unit 451 in Fig. 37. As shown in Fig. 38, the transformation matrix derivation unit 451 has a transformation matrix LUT 461, a flip unit 462, and a transposition unit 463. Note that although arrows indicating data exchange are omitted in Fig. 38, the transformation matrix derivation unit 451 can exchange any data between any processing units (processing blocks).

[0401] The transformation matrix LUT 461 is a lookup table for holding (storing) a transformation matrix corresponding to the transformation type identifier TrTypeIdxH of the inverse primary horizontal transform and the size N of the transformation block. When the transformation type identifier TrTypeIdxH of the inverse primary horizontal transform and the size N of the transformation block are specified, the transformation matrix LUT 461 selects and outputs a transformation matrix corresponding to them. In the case of this derivation example, the transformation matrix LUT 461 converts the transformation matrix into a base transformation matrix T base to the flip section 462 or the transposition section 463, or both.

[0402] The flip unit 462 flips the input transformation matrix T with N rows and N columns, and outputs the transformed matrix T after the flip. flip In this derivation example, the flip unit 462 outputs the base transformation matrix T base The input is the base transformation matrix T base Flip in the row direction (horizontal direction) to obtain the transformation matrix T flip Let us use the transformation matrix T H and outputs it to the outside of the transformation matrix derivation unit 451 (supplying it to the matrix calculation unit 452).

[0403] The transposition unit 463 transposes the input transformation matrix T with N rows and N columns, and obtains the transposed transformation matrix T transpose In this derivation example, the transposition unit 463 outputs the base transformation matrix T base The input is the base transformation matrix T base Transpose the transformation matrix T transpose Let us use the transformation matrix TH and outputs it to the outside of the transformation matrix derivation unit 451 (supplying it to the matrix calculation unit 452).

[0404] As described above, the transformation matrix derivation unit 451 has the flip unit 462 and the transposition unit 463. Therefore, the transformation matrix derivation unit 451 can realize the derivation example of the example in the first row from the top of the table shown in Fig. 10, for example, by using the flip unit 462. Also, the transformation matrix derivation unit 451 can realize the derivation example of the example in the second row from the top of the table shown in Fig. 10, for example, by using the transposition unit 463.

[0405] <Flow of reverse primary conversion process> Next, a description will be given of an example of the flow of processing performed by the above-mentioned configuration of the image decoding device 200. In this case, a description will be given of an example of the flow of the inverse primary transform processing performed in step S233 in Fig. 18 with reference to the flowchart in Fig. 39.

[0406] When the inverse primary transform process is started, in step S401, the inverse primary transform selection unit 411 (Figure 34) of the inverse primary transform unit 253 performs an inverse primary transform selection process and selects the transform type identifier TrTypeIdxV (or the transform type TrTypeV) of the inverse primary vertical transform and the transform type identifier TrTypeIdxH (or the transform type TrTypeH) of the inverse primary horizontal transform.

[0407] In step S402, the inverse primary vertical transform unit 412 performs an inverse primary vertical transform process corresponding to the transform type identifier TrTypeIdxV of the inverse primary vertical transform obtained in step S401 on the transform coefficients Coeff_IS after the inverse secondary transform, to derive transform coefficients Coeff_IPver after the inverse primary vertical transform.

[0408] In step S403, the inverse primary horizontal transform unit 413 performs an inverse primary horizontal transform process corresponding to the transform type identifier TrTypeIdxH of the inverse primary horizontal transform obtained in step S401 on the transform coefficient Coeff_IPver after the inverse primary vertical transform, and derives the transform coefficient Coeff_IPhor after the inverse primary horizontal transform (i.e., the transform coefficient Coeff_IP(prediction residual D') after the inverse primary transform).

[0409] When the process of step S403 ends, the inverse primary transformation process ends, and the process returns to FIG.

[0410] In the inverse primary transform process described above, the process to which the present technology is applied is performed as the process of step S402 or step S403. Therefore, by performing this inverse primary transform process, it is possible to suppress an increase in the memory capacity required for the inverse primary vertical transform process or the inverse primary horizontal transform process.

[0411] <Flow of inverse primary conversion selection process> Next, an example of the flow of the inverse primary transformation selection process executed in step S401 in FIG. 39 will be described with reference to the flowchart in FIG.

[0412] When the inverse primary transform selection process is started, in step S421, the inverse primary transform selection unit 411 determines whether the adaptive primary transform flag apt_flag[compID] of the component identifier compID is true (1). If it is determined that the adaptive primary transform flag apt_flag[compID] of the component identifier compID is true (1), the process proceeds to step S422.

[0413] In step S422, the inverse primary transform selection unit 411 selects transform sets TrSetH and TrSetV of each direction from the transform set group based on the prediction mode information Pinfo.

[0414] In step S423, the inverse primary transform selection unit 411 derives a transform type identifier TrTypeIdxH of the inverse primary horizontal transform based on the transform set TrSetH and the primary transform identifier pt_idx[compID]. This process can be expressed, for example, as in the following equation (34).

[0415]

number

[0416] In step S424, the inverse primary transform selection unit 411 derives a transform type identifier TrTypeIdxV of the inverse primary vertical transform based on the transform set TrSetV and the inverse primary transform identifier pt_idx[compID]. This process can be expressed, for example, as in the following equation (35).

[0417]

number

[0418] When the process of step S424 ends, the inverse primary transformation selection process ends, and the process returns to FIG.

[0419] Also, if it is determined in step S421 that the adaptive primary transformation flag apt_flag[compID] of the component identifier compID is false (0), the process proceeds to step S425.

[0420] In step S425, the inverse primary transform selection unit 411 sets (selects) a predetermined orthogonal transform (for example, DCT2) in the transform type identifier TrTypeIdxH of the inverse primary horizontal transform. This process can be expressed, for example, as in the following equation (36).

[0421]

number

[0422] In step S426, the inverse primary transform selection unit 411 sets (selects) a predetermined orthogonal transform (for example, DCT2) in the transform type identifier TrTypeIdxV of the inverse primary vertical transform. This process can be expressed, for example, as in the following equation (37).

[0423]

number

[0424] When the process of step S426 ends, the inverse primary transformation selection process ends, and the process returns to FIG.

[0425] <Flow of inverse primary vertical transformation process> Next, the flow of the inverse primary vertical transformation process executed in step S402 in FIG. 39 will be described with reference to the flowchart in FIG.

[0426] When the inverse primary vertical transform process is started, in step S441, the transform matrix derivation unit 421 (FIG. 35) of the inverse primary vertical transform unit 412 executes the transform matrix derivation process to derive the transform matrix T V Derive.

[0427] The transformation matrix derivation process in this case is performed in the same manner as in the case of the primary horizontal transform described with reference to the flowchart in Fig. 30. Therefore, the description thereof will be omitted. For example, the transformation type identifier TrTypeIdxH of the primary horizontal transform may be replaced with the transformation type identifier TrTypeIdxV of the inverse primary vertical transform, or the derived transformation matrix T H Let us use the transformation matrix T for the inverse primary vertical transformation. V 30 can be applied as an explanation of the transformation matrix derivation process in this case.

[0428] In step S442, the matrix calculation unit 422 calculates the derived transformation matrix TV Using the input data X in (That is, the transform coefficient Coeff_IS after the inverse secondary transform is subjected to a vertical inverse one-dimensional orthogonal transform to obtain intermediate data Y1. If this process is expressed as a determinant, it can be expressed as the above-mentioned equation (30).

[0429] In step S443, the scaling unit 423 scales the coefficient Y1[i,j] of each i-th row and j-th column component of the intermediate data Y1 derived in the process of step S442 by a shift amount S IV The intermediate data Y2 is derived by scaling using the above equation (31).

[0430] In step S444, the clipping unit 424 clips the value of the coefficient Y2[i,j] of each i-th row and j-th column component of the intermediate data Y2 derived by the process in step S443, and obtains the output data X out (That is, the transform coefficient Coeff_IPver after the inverse primary vertical transform is obtained.) This process can be expressed as in the above-mentioned formula (11).

[0431] When the process of step S444 ends, the inverse primary vertical transform process ends, and the process returns to FIG.

[0432] In the inverse primary vertical transform process described above, the process to which the present technology is applied is performed as the process of step S441 or step S442. Therefore, by performing this inverse primary vertical transform process, it is possible to suppress an increase in memory capacity required for the vertical inverse one-dimensional orthogonal transform.

[0433] <Flow of inverse primary horizontal transform processing> Next, the flow of the inverse primary horizontal transform process executed in step S403 in FIG. 39 will be described with reference to the flowchart in FIG.

[0434] When the inverse primary horizontal transform process is started, in step S461, the transform matrix derivation unit 451 (FIG. 37) of the inverse primary horizontal transform unit 413 executes the transform matrix derivation process to obtain the transform matrix T H Derive.

[0435] The transformation matrix derivation process in this case is performed in the same manner as in the case of the primary horizontal transform described with reference to the flowchart of Fig. 30. Therefore, the description thereof will be omitted. For example, if the primary horizontal transform is replaced with an inverse primary horizontal transform, the description made with reference to Fig. 30 can be applied as the description of the transformation matrix derivation process in this case.

[0436] In step S462, the matrix calculation unit 452 calculates the derived transformation matrix T H Using the input data X in (That is, the transform coefficients Coeff_IPver after the inverse primary vertical transform are subjected to a horizontal inverse one-dimensional orthogonal transform to obtain intermediate data Y1. If this process is expressed as a determinant, it can be expressed as in the above-mentioned equation (32).

[0437] In step S463, the scaling unit 453 scales the coefficient Y1[i,j] of each i-th row and j-th column component of the intermediate data Y1 derived in the process of step S462 by a shift amount S IH The intermediate data Y2 is derived by scaling using the above equation (33).

[0438] In step S464, the clipping unit 454 clips the value of the coefficient Y2[i,j] of each i-th row and j-th column component of the intermediate data Y2 derived in the process of step S463, and obtains the output data X out (That is, the transform coefficient Coeff_IPhor after the inverse primary horizontal transform is obtained.) This process can be expressed as in the above-mentioned equation (14).

[0439] When the process of step S464 ends, the inverse primary horizontal transform process ends, and the process returns to FIG.

[0440] In the inverse primary horizontal transform process described above, the process to which the present technology is applied is performed as the process of step S461 and step S462. Therefore, by performing this inverse primary horizontal transform process, it is possible to suppress an increase in memory capacity required for the horizontal inverse one-dimensional orthogonal transform.

[0441] <2-3. Example 1-2> <Concept> Next, an example of the third row from the top in the table shown in FIG. 10, excluding the top row of item names, will be described.

[0442] As described above, the derivation of the example in the third row from the top focuses on the characteristics between paired DCT / DSTs. More specifically, between paired DCT / DSTs (for example, DST7 and DCT8), attention is paid to the fact that even-numbered row vectors are axially symmetric and odd-numbered row vectors are point symmetric. In this case, the derivation unit flips the first transformation matrix, and inverts the signs of odd-numbered row vectors in the first transformation matrix after the flip to derive the second transformation matrix. That is, as shown in FIG. 43, the derivation unit flips the transformation matrix of DST7 in the row direction, and further inverts the signs of odd-order row vectors to losslessly derive the transformation matrix of DCT8. Therefore, the derived DCT8 can naturally replace the transformation matrix of (reduced type) DCT8, which has the same waveform of the 0th order row vector.

[0443] A specific example of this derivation is shown in Fig. 44. As shown in Fig. 44, this derivation is performed by dividing a unit matrix D whose odd-numbered row elements are negative and a base transformation matrix T base It can be expressed as a matrix product of (DST7) and the flip matrix J.

[0444] In other words, in this derivation example, the second transformation matrix can be derived with two operations (flip and sign inversion). In addition, each operation is easy. In other words, the second transformation matrix can be easily derived.

[0445] Moreover, by applying this derivation example, it becomes unnecessary to prepare a DCT8 transformation matrix as a candidate for a transformation matrix used in an orthogonal transformation and an inverse orthogonal transformation, which means that the number of eigentransformation types can be reduced.

[0446] In this case, as shown in the table of Fig. 45, the number of transformation matrices required, which was five in the case of the technique described in Non-Patent Document 1 (see Table A in Fig. 6), can be reduced to four, so that the total LUT size can be about 47 KB. In other words, the size of the LUT can be reduced (by about 53 KB (Table A in Fig. 6)) compared to the case of the technique described in Non-Patent Document 1. In other words, an increase in the size of the LUT can be suppressed.

[0447] As described above, in this case, a transform matrix of the transform type DCT8 can be obtained as the second transform type, and by performing orthogonal transform and inverse orthogonal transform using the transform matrix of the second transform type, it is possible to obtain the same coding efficiency as when the transform matrix of DCT8 is used for the orthogonal transform and inverse orthogonal transform.

[0448] <Transformation matrix derivation part> Next, the configuration and processing for performing such derivation will be described. First, the technical configuration of the image encoding device 100 according to this embodiment will be described. In this case, the configurations of the primary transform unit 152, the primary horizontal transform unit 312, the primary vertical transform unit 313, and the like are the same as those in <2-2. Example 1-1> described with reference to Fig. 22, and therefore the description thereof will be omitted.

[0449] Fig. 46 is a block diagram showing an example of the main configuration of transformation matrix derivation unit 321 (transformation matrix derivation unit 321 in primary horizontal transformation unit 312 (Fig. 23)) in this case. As shown in Fig. 46, transformation matrix derivation unit 321 in this case has transformation matrix LUT 331, flip unit 332, and sign inversion unit 501. Note that although arrows indicating data exchange are omitted in Fig. 46, transformation matrix derivation unit 321 can exchange any data between any processing units (processing blocks).

[0450] The transformation matrix LUT 331 and the flip section 332 are the same as those in FIG.

[0451] The sign inversion unit 501 receives a transformation matrix T with N rows and N columns, inverts the signs of predetermined portions of the transformation matrix T, and outputs the transformation matrix T after the sign inversion. InvSign In this derivation example (Example 1-2), the transformation matrix derivation unit 321 outputs the base transformation matrix T base is flipped horizontally via the flip unit 332, and the signs of the odd-order row vectors are further inverted via the sign inversion unit 501 to obtain the transformation matrix T InvSign Transform the matrix T H and outputs it to the outside of the transformation matrix derivation unit 321 (supplying it to the matrix calculation unit 322).

[0452] As described above, the transformation matrix derivation unit 321 can realize the derivation example shown in the third row from the top of the table in FIG.

[0453] <Transformation matrix derivation part> Fig. 47 is a block diagram showing an example of the main configuration of the transformation matrix derivation unit 351 in the primary vertical transformation unit 313 in this case. As shown in Fig. 47, the transformation matrix derivation unit 351 in this case has a transformation matrix LUT 361, a flip unit 362, and a sign inversion unit 502, similar to the transformation matrix derivation unit 321. Note that although arrows indicating data exchange are omitted in Fig. 47, the transformation matrix derivation unit 351 can exchange any data between any processing units (processing blocks).

[0454] The transformation matrix LUT 361 and the flip section 362 are the same as those in FIG.

[0455] The sign inversion unit 502, like the sign inversion unit 501, receives the transformation matrix T with N rows and N columns, inverts the signs of predetermined positions of the transformation matrix T, and generates the transformation matrix T after the sign inversion. InvSign In this derivation example, the transformation matrix derivation unit 351 outputs the base transformation matrix T base is flipped horizontally via the flip unit 362, and the signs of the odd-order row vectors are further inverted via the sign inversion unit 502 to obtain the transformation matrix T InvSign Transform the matrix T V and outputs it to the outside of the transformation matrix derivation unit 351 (supplying it to the matrix calculation unit 352).

[0456] As described above, the transformation matrix derivation unit 351 can realize the derivation example shown in the third row from the top of the table in FIG.

[0457] <Flow of transformation matrix derivation process> In this case, the primary transformation process, primary horizontal transformation process, and primary vertical transformation process are performed in the same manner as in <2-2. Example 1-1>, so a description of these processes will be omitted.

[0458] Next, an example of the flow of the transformation matrix derivation process in this case, which is executed by transformation matrix derivation section 321 in primary horizontal transformation section 312 in step S321 in FIG. 28, will be described with reference to the flowchart in FIG.

[0459] When the transformation matrix derivation process is started, in step S501, the transformation matrix derivation unit 321 determines the base transformation type BaseTrType corresponding to the transformation type identifier TrTypeIdxH by referring to a correspondence table such as that shown in FIG. 49. This process can be expressed mathematically, for example, as in the above-mentioned equation (17). Furthermore, the transformation matrix derivation unit 321 reads out the transformation matrix of N rows and N columns of the determined base transformation type from the transformation matrix LUT, and converts it into the base transformation matrix T base Set to.

[0460] Moreover, the transformation matrix derivation unit 321 sets a value corresponding to the transformation type identifier TrTypeIdxH to ​​the flip flag FlipFlag as shown in the above-mentioned formula (19). Moreover, the transformation matrix derivation unit 321 sets a value corresponding to the transformation type identifier TrTypeIdxH to ​​the sign inversion flag InvSignFlag as shown in the following formula (38).

[0461]

number

[0462] In step S502, the transformation matrix derivation unit 321 determines whether or not the flip flag FlipFlag and the transpose flag TransposeFlag satisfy a condition (ConditionB1) expressed by the following equation (39).

[0463]

number

[0464] If it is determined that the condition shown in the above equation (39) is not satisfied (i.e., if it is determined that the flip flag FlipFlag is false (0) or the sign inversion flag InvSignFlag is false (0)), the process proceeds to step S503.

[0465] In step S503, the transformation matrix derivation unit 321 calculates the base transformation matrix Tbase Transform the matrix T H When the process of step S503 ends, the transformation matrix derivation process ends, and the process returns to FIG.

[0466] Also, in step S502, if it is determined that the condition shown in the above equation (39) is satisfied (i.e., if it is determined that the flip flag FlipFlag is true (1) and the sign inversion flag InvSignFlag is true (1)), the processing proceeds to step S504.

[0467] In step S504, the transformation matrix derivation unit 321 uses the flip unit 332 to derive the base transformation matrix T base Flip and convert it to a transformation matrix T H That is, the transformation matrix derivation unit 321 sets Flip(T base ) transformation matrix T H Derive.

[0468] In step S505, the transformation matrix derivation unit 321 uses the sign inversion unit 501 to derive the transformation matrix T H The sign of the odd-numbered row vector of H This process can be expressed as a determinant as shown in the following equation (40).

[0469]

number

[0470] Here, × is the operator representing matrix multiplication, and the sign-negation matrix D is Diag(1,-1,…,(-1) N-1 ) is a diagonal matrix consisting of

[0471] This process can also be expressed as an element-by-element calculation, as in the following equation (41). In this way, it can be realized without having a sign-negation matrix D. In this case, the transformation matrix derivation unit 321 calculates the transformation matrix T HInvert the sign of the odd-numbered row (i%2==1) components of the i-th row and j-th column ((i,j) components) of

[0472]

number

[0473] Here, the transformation matrix T H The i-th row and j-th column component ((i,j) component) of T H It is written as [i,j]. Also, "for i,j=0,...,N-1 and i%2 == 1" in the second line indicates that j has a value from 0 to N-1, and i has an odd value in the range from 0 to N-1. That is, T H [i,j] is the N-row, N-column transformation matrix T H This means that it indicates the odd row vector (each element of the odd row vector).

[0474] When the process of step S505 ends, the transformation matrix derivation process ends, and the process returns to FIG.

[0475] <Flow of transformation matrix derivation process> Note that the flow of the transformation matrix derivation process performed in step S361 (Figure 32) of the primary vertical transformation process is similar to the transformation matrix derivation process performed in the primary horizontal transformation described with reference to the flowchart of Figure 48, so its description is omitted.

[0476] <Transformation matrix derivation part> Next, a description will be given of the configuration of the image decoding device 200 in this case. In this case as well, the configurations of the inverse primary transform unit 253, the inverse primary vertical transform unit 412, the inverse primary horizontal transform unit 413, etc., of the image decoding device 200 are similar to those in <2-2. Example 1-1>, and therefore the description thereof will be omitted.

[0477] Fig. 50 is a block diagram showing an example of the main configuration of the transformation matrix derivation unit 421 (the transformation matrix derivation unit 421 included in the inverse primary vertical transformation unit 412) in this case. As shown in Fig. 50, the transformation matrix derivation unit 421 in this case has a transformation matrix LUT 431, a flip unit 432, and a sign inversion unit 511, similar to the transformation matrix derivation unit 321. Note that although arrows indicating data exchange are omitted in Fig. 50, the transformation matrix derivation unit 421 can exchange any data between any processing units (processing blocks).

[0478] The transformation matrix LUT 431 and the flip unit 432 are the same as those in Fig. 36. Similar to the sign inversion unit 501, the sign inversion unit 511 receives the transformation matrix T with N rows and N columns as input, inverts the signs of predetermined portions of the transformation matrix T, and outputs the transformation matrix T after the sign inversion. InvSign In this derivation example, the transformation matrix derivation unit 421 outputs the base transformation matrix T base is flipped horizontally via the flip unit 432, and the signs of the odd-order row vectors are further inverted via the sign inversion unit 511 to obtain the transformation matrix T InvSign Transform the matrix T V and outputs it to the outside of the transformation matrix derivation unit 421 (supplying it to the matrix calculation unit 422).

[0479] As described above, the transformation matrix derivation unit 421 can use the flip unit 432 and the sign inversion unit 511 to realize the derivation example shown in the third row from the top of the table in FIG.

[0480] <Transformation matrix derivation part> Fig. 51 is a block diagram showing an example of the main configuration of the transformation matrix derivation unit 451 in the inverse primary horizontal transformation unit 413 in this case. As shown in Fig. 51, the transformation matrix derivation unit 451 in this case has a transformation matrix LUT 461, a flip unit 462, and a sign inversion unit 512, similar to the transformation matrix derivation unit 421. Note that although arrows indicating data exchange are omitted in Fig. 51, the transformation matrix derivation unit 451 can exchange any data between any processing units (processing blocks).

[0481] The transformation matrix LUT 461 and the flip unit 462 are the same as those in FIG. 50. The sign inversion unit 512, like the sign inversion unit 501, receives the transformation matrix T with N rows and N columns, inverts the signs of predetermined portions of the transformation matrix T, and outputs the transformation matrix T after the sign inversion. InvSign In this derivation example, the transformation matrix derivation unit 451 outputs the base transformation matrix T base is flipped horizontally via the flip unit 462, and the signs of the odd-order row vectors are further inverted via the sign inversion unit 512 to obtain the transformation matrix T InvSign Transform the matrix T V and outputs it to the outside of the transformation matrix derivation unit 451 (supplying it to the matrix calculation unit 452).

[0482] As described above, the transformation matrix derivation unit 451 can realize the derivation example shown in the third row from the top of the table in FIG.

[0483] <Flow of transformation matrix derivation process> Note that the transformation matrix derivation units 421 and 451 perform the transformation matrix derivation process in the same manner as described with reference to the flowchart in FIG. 48, and therefore a description thereof will be omitted.

[0484] <2-4. Example 1-3> <Concept> Next, in the table shown in FIG. 10, excluding the top row of item names, examples of the fourth and fifth rows from the top will be described.

[0485] As described above, the derivation of the fourth example from the top, like the first example from the top, focuses on the similarity between the waveform of the lowest row vector of the first transformation type and the waveform of the lowest row vector of the transformation type of the alternative transformation matrix. In this case, the derivation unit flips the first transformation matrix to derive the second transformation matrix. That is, as shown in FIG. 52, the derivation unit converts the transformation matrix of DCT8 into a base transformation matrix T baseThen, it is flipped in the row direction to derive the transformation matrix of FlipDCT8. This transformation matrix of FlipDCT8 can replace the transformation matrix of DST7 (increasing type) whose waveform of the 0th order row vector is similar.

[0486] Moreover, the derivation of the example in the fifth row from the top focuses on the similarity between the waveform of the highest-order column vector of the first transformation type and the waveform of the lowest-order row vector of the transformation type of the substituting transformation matrix. In this case, the derivation unit flips the first transformation matrix, transposes the first transformation matrix after the flip, and derives the second transformation matrix. That is, as shown in FIG. 52, the derivation unit converts the transformation matrix of DCT8 into a base transformation matrix T base Then, it is flipped in the row direction and further transposed to derive the transformation matrix of TrFlipDCT8. This transformation matrix of TrFlipDCT8 can replace the transformation matrix of DST1, which has a similar (mountain-shaped) waveform of the 0th-order row vector.

[0487] In other words, in the above two derivation examples, the second transformation matrix can be derived with one or two operations (flip or flip + transpose). In addition, the operation is also easy. In other words, the second transformation matrix can be easily derived.

[0488] In addition, by applying the above two derivation examples, it is no longer necessary to prepare the DST7 transformation matrix and the DST1 transformation matrix as candidates for the transformation matrix used in the orthogonal transformation and the inverse orthogonal transformation. In other words, it is possible to reduce the number of eigentransformation types.

[0489] In this case, as shown in the table of Fig. 53, the number of transformation matrices required in the technique described in Non-Patent Document 1 (see Table A in Fig. 6) can be reduced to three, so the total LUT size can be about 40 KB. In other words, the size of the LUT can be reduced (by about 53 KB (Table A in Fig. 6)) compared to the technique described in Non-Patent Document 1. In other words, an increase in the size of the LUT can be suppressed.

[0490] As described above, in this case too, by performing orthogonal transform and inverse orthogonal transform using the derived transform matrix of the second transform type (FlipDCT8 or TrFlipDCT8), coding efficiency can be obtained that is similar to that when the transform matrix of DST7 or the transform matrix of DST1 is used for the orthogonal transform and inverse orthogonal transform.

[0491] <Configuration> In this case, the configurations of the image encoding device 100 and the image decoding device 200 are similar to those explained in <2-2. Example 1-1>, and therefore explanations thereof will be omitted.

[0492] <Flow of transformation matrix derivation process> Next, the flow of the process will be described. The processes other than the transformation matrix derivation process are performed in the same flow as the example described above in <2-2. Example 1-1>, etc., so the description thereof will be omitted.

[0493] An example of the flow of the transformation matrix derivation process executed by transformation matrix derivation section 321 in primary horizontal transformation section 312 in step S321 in FIG. 28 will be described with reference to the flowchart in FIG.

[0494] When the transformation matrix derivation process is started, in step S521, the transformation matrix derivation unit 321 determines the base transformation type BaseTrType corresponding to the transformation type identifier TrTypeIdxH by referring to a correspondence table such as that shown in FIG.

[0495] Also, the transformation matrix derivation unit 321 sets the value of the flip flag FlipFlag corresponding to the transformation type identifier TrTypeIdxH and the value of the transpose flag TransposeFlag corresponding to the transformation type identifier TrTypeIdxH, similarly to step S341 in the flowchart of FIG.

[0496] In step S522, the transformation matrix derivation unit 321 determines whether or not the flip flag FlipFlag satisfies a condition (Condition C1) expressed by the following equation (42).

[0497]

number

[0498] If it is determined that the above condition (Condition C1) is not satisfied (if the flip flag FlipFlag is false (0)), the process proceeds to step S523.

[0499] In step S523, the transformation matrix derivation unit 321 calculates the base transformation matrix T base Transform the matrix T H When the process of step S523 ends, the transformation matrix derivation process ends, and the process returns to FIG.

[0500] Also, in step S522, if it is determined that the above-mentioned condition (Condition C1) is satisfied (if the flip flag FlipFlag is true (1)), the process proceeds to step S524.

[0501] In step S524, the transformation matrix derivation unit 321 calculates the base transformation matrix T base Flip horizontally and use the flipped transformation matrix Flip(T base ) into the transformation matrix T tmp This process can be expressed as in the following equation (43).

[0502]

number

[0503] In step S525, the transformation matrix derivation unit 321 determines whether or not the transposition flag TransposeFlag satisfies a condition (ConditionC2) expressed by the following equation (44).

[0504]

number

[0505] If it is determined that the above condition (ConditionC2) is not satisfied (if the transpose flag TransposeFlag is false (0)), the process proceeds to step S526.

[0506] In step S526, the transformation matrix derivation unit 321 calculates the transformation matrix T tmp Transform the matrix T H This process can be expressed as in the following equation (45).

[0507]

number

[0508] When the process of step S526 ends, the transformation matrix derivation process ends, and the process returns to Fig. 28. Also, if it is determined in step S525 that the above-mentioned condition (Condition C2) is satisfied (if the transpose flag TransposeFlag is true (1)), the process proceeds to step S527.

[0509] In step S527, the transformation matrix derivation unit 321 calculates the transformation matrix T tmp Transposed Tr(T tmp ) into the transformation matrix T H This process can be expressed, for example, as in the following equation (46).

[0510]

number

[0511] When the process of step S527 ends, the transformation matrix derivation process ends, and the process returns to FIG.

[0512] By executing the transformation matrix derivation process as described above, it is possible to realize the derivation examples in the fourth and fifth rows from the top of the table shown in Fig. 10 in the horizontal one-dimensional orthogonal transformation. Therefore, it is possible to suppress an increase in the required memory capacity.

[0513] In the above, the flip and transpose are described in order, but the flip and transpose may be performed together (in one calculation). For example, if the determinations in steps S522 and S525 in Fig. 54 are performed together and both are true (1), the flip and transpose may be performed together.

[0514] For example, the flip and transpose operations can be expressed as element-wise operations for the transformation matrix T H The jth row and ith column components ((j,i) components) of the base transformation matrix T base In other words, it can be expressed as the following equation (47).

[0515]

number

[0516] Here, the transformation matrix T H The i-th row and j-th column component ((i,j) component) of T H The notation is [i,j]. Also, the second line "for i,j=0,...,N-1" indicates that i and j each have a value from 0 to N-1. That is, T H [i,j] is the N-row, N-column transformation matrix T H means to show all elements of

[0517] By doing this, flip and transpose can be realized with a single operation, moreover, by accessing a simple two-dimensional array. Also, since the flip matrix J is not required, the increase in memory capacity can be suppressed.

[0518] In this embodiment, in step S361 in FIG. 32, the transformation matrix derivation process executed by the transformation matrix derivation unit 351 included in the primary vertical transformation unit 313 is also performed in the same manner as in the flowchart in FIG.

[0519] In this embodiment, in step S441 in FIG. 41, the transformation matrix derivation process executed by the transformation matrix derivation unit 421 in the inverse primary vertical transformation unit 412 is also performed in the same manner as in the flowchart in FIG.

[0520] Furthermore, in this embodiment, in step S461 in FIG. 42, the transformation matrix derivation process executed by the transformation matrix derivation unit 451 included in the inverse primary horizontal transformation unit 413 is also performed in the same manner as in the flowchart in FIG.

[0521] Therefore, a description of these will be omitted.

[0522] <2-5. Example 1-4> <Concept> Next, an example of the sixth row from the top in the table shown in FIG. 10, excluding the top row of item names, will be described.

[0523] As described above, the derivation of the example in the sixth row from the top focuses on the characteristics between paired DCT / DSTs, similar to the case in the third row from the top (Example 1-2). More specifically, it focuses on the fact that between paired DCT / DSTs (for example, DCT8 and DST7), even-numbered row vectors are axially symmetric and odd-numbered row vectors are point symmetric. In this case, the derivation unit flips the first transformation matrix, and inverts the signs of the odd-numbered row vectors of the first transformation matrix after the flip to derive the second transformation matrix. That is, as shown in FIG. 56, the derivation unit converts the transformation matrix of DCT8 into a base transformation matrix T base Then, it is flipped in the row direction, and the signs of the odd-order row vectors are inverted to derive a DST7 transformation matrix losslessly. Naturally, the derived DST7 transformation matrix can be used to replace a (increasing) DST7 transformation matrix with the same waveform for the 0th-order row vector.

[0524] That is, in this derivation example, the second transformation matrix can be derived with two operations (flip + sign inversion). In addition, the operation is also easy. That is, the second transformation matrix can be easily derived.

[0525] In addition, by applying this derivation example, it becomes unnecessary to prepare a DST7 transformation matrix as a candidate for a transformation matrix used in an orthogonal transformation and an inverse orthogonal transformation, which means that the number of eigentransformation types can be reduced.

[0526] In this case, as shown in the table of Fig. 57, the number of transformation matrices required, which was five in the case of the technique described in Non-Patent Document 1 (see Table A in Fig. 6), can be reduced to four, so the total LUT size can be about 47 KB. In other words, the size of the LUT can be reduced (by about 53 KB (Table A in Fig. 6)) compared to the case of the technique described in Non-Patent Document 1. In other words, an increase in the size of the LUT can be suppressed.

[0527] Furthermore, in the case of this derivation example, since a DST7 transform matrix can be derived as the second transform matrix, by performing an orthogonal transform and an inverse orthogonal transform using this second transform matrix, it is natural that the same coding efficiency can be obtained as when a DST7 transform matrix is ​​used for the orthogonal transform and inverse orthogonal transform.

[0528] <Configuration and processing flow> In this case, the configurations of the image encoding device 100 and the image decoding device 200 are the same as those described in <2-3. Example 1-2>, so the description thereof will be omitted. Also, the processes performed by the image encoding device 100 and the image decoding device 200 are the same as those described in <2-3. Example 1-2>, so the description thereof will be omitted.

[0529] However, in the case of this embodiment, when determining the base transformation type corresponding to the transformation type identifier TrTypeIdxH in the transformation matrix derivation process, the determination is made by referring to a correspondence table such as that shown in Fig. 58. Other than that, the same flow as that described with reference to the flowchart in Fig. 48 may be followed.

[0530] <2-6. Example 1-5> <Concept> Next, in the table shown in FIG. 10, excluding the top row of item names, examples of the seventh and eighth rows from the top will be described.

[0531] As described above, the derivation of the example in the seventh row from the top, like the first row from the top, focuses on the similarity between the waveform of the lowest row vector of the first transformation type and the waveform of the lowest row vector of the transformation type of the alternative transformation matrix. However, in this case, the derivation unit transposes the first transformation matrix to derive the second transformation matrix. In other words, as shown in FIG. 59, the derivation unit derives the transformation matrix of DCT2 as a base transformation matrix T base Then, it is transposed to derive the DCT3 transformation matrix. This DCT3 transformation matrix can be used to replace the (reduced) DCT8 transformation matrix, which has a similar waveform for the 0th-order row vector.

[0532] Also, the derivation of the example in the eighth row from the top, like the fifth row from the top, focuses on the similarity between the waveform of the highest-order column vector of the first transformation type and the waveform of the lowest-order row vector of the transformation type of the substituting transformation matrix. In this case, the derivation unit transposes the first transformation matrix, flips the first transformation matrix after transposition, and derives the second transformation matrix. That is, as shown in FIG. 59, the derivation unit converts the transformation matrix of DCT2 into a base transformation matrix T base Then, it is transposed and further flipped in the row direction to derive the transformation matrix of FlipDCT3. This transformation matrix of FlipDCT3 can replace the transformation matrix of DST7 (increasing type) whose waveform of the 0th order row vector is similar.

[0533] In other words, in the above two derivation examples, the second transformation matrix can be derived with one or two operations (transpose or transpose + flip). In addition, the operation is also easy. In other words, the second transformation matrix can be easily derived.

[0534] In addition, by applying the above two derivation examples, it is no longer necessary to prepare DCT8 transform matrices and DST7 transform matrices as candidates for transform matrices used in orthogonal transform and inverse orthogonal transform. In other words, it is possible to reduce the number of eigentransform types.

[0535] In this case, as shown in the table of Fig. 60, the number of transformation matrices required, which was five in the case of the technique described in Non-Patent Document 1 (see Table A in Fig. 6), can be reduced to three, so the total LUT size can be about 40 KB. In other words, the size of the LUT can be reduced (by about 53 KB (Table A in Fig. 6)) compared to the case of the technique described in Non-Patent Document 1. In other words, an increase in the size of the LUT can be suppressed.

[0536] As described above, in this case too, by performing orthogonal transform and inverse orthogonal transform using the derived transform matrix of the second transform type (DCT3 or FlipDCT3), coding efficiency can be obtained that is similar to that when a DCT8 transform matrix or a DST7 transform matrix is ​​used for the orthogonal transform and inverse orthogonal transform.

[0537] <Configuration> In this case, the configurations of the image encoding device 100 and the image decoding device 200 are similar to those explained in <2-2. Example 1-1>, and therefore explanations thereof will be omitted.

[0538] <Flow of transformation matrix derivation process> Next, the flow of the process will be described. The processes other than the transformation matrix derivation process are performed in the same flow as the example described above in <2-2. Example 1-1>, etc., so the description thereof will be omitted.

[0539] An example of the flow of the transformation matrix derivation process executed by transformation matrix derivation section 321 in primary horizontal transformation section 312 in step S321 in FIG. 28 will be described with reference to the flowchart in FIG.

[0540] When the transformation matrix derivation process is started, in step S541, the transformation matrix derivation unit 321 determines the base transformation type BaseTrType corresponding to the transformation type identifier TrTypeIdxH by referring to a correspondence table such as that shown in FIG. 62, for example.

[0541] Also, the transformation matrix derivation unit 321 sets the value of the flip flag FlipFlag corresponding to the transformation type identifier TrTypeIdxH and the value of the transpose flag TransposeFlag corresponding to the transformation type identifier TrTypeIdxH, similarly to step S341 in the flowchart of FIG.

[0542] In step S542, the transformation matrix derivation unit 321 determines whether or not the transpose flag TransposeFlag satisfies a condition (ConditionD1) expressed by the following equation (48).

[0543]

number

[0544] If it is determined that the above condition (ConditionD1) is not satisfied (if the transpose flag TransposeFlag is false (0)), the process proceeds to step S543.

[0545] In step S543, the transformation matrix derivation unit 321 calculates the base transformation matrix T base Transform the matrix T H When the process of step S543 ends, the transformation matrix derivation process ends, and the process returns to FIG.

[0546] Also, in step S542, if it is determined that the above-mentioned condition (ConditionD1) is satisfied (if the transpose flag TransposeFlag is true (1)), the process proceeds to step S544.

[0547] In step S544, the transformation matrix derivation unit 321 calculates the base transformation matrix T base Transpose horizontally and use the resulting transposed transformation matrix Tr(T base ) into the transformation matrix T tmp This process can be expressed as in the following equation (49).

[0548]

number

[0549] In step S545, the transformation matrix derivation unit 321 determines whether or not the flip flag FlipFlag satisfies a condition (ConditionD2) expressed by the following equation (50).

[0550]

number

[0551] If it is determined that the above-mentioned condition (Condition D2) is not satisfied (if the flip flag FlipFlag is false (0)), the process proceeds to step S546.

[0552] In step S546, the transformation matrix derivation unit 321 calculates the transformation matrix T tmp Transform the matrix T H Set it to.

[0553] When the process of step S546 ends, the transformation matrix derivation process ends, and the process returns to Fig. 28. Also, if it is determined in step S545 that the above-mentioned condition (Condition D2) is satisfied (if the flip flag FlipFlag is true (1)), the process proceeds to step S547.

[0554] In step S547, the transformation matrix derivation unit 321 calculates the transformation matrix T tmp Flip(T tmp ) into the transformation matrix T H This process can be expressed, for example, as in the following equation (51).

[0555]

number

[0556] When the process of step S547 ends, the transformation matrix derivation process ends, and the process returns to FIG.

[0557] By executing the transformation matrix derivation process as described above, it is possible to realize the derivation examples in the seventh and eighth rows from the top of the table shown in Fig. 10 in the horizontal one-dimensional orthogonal transformation. Therefore, it is possible to suppress an increase in the required memory capacity.

[0558] In the above, the transposition and flipping are performed in sequence, but the transposition and flipping may be performed together (in one calculation). For example, if the determinations in steps S542 and S545 in Fig. 61 are performed together and both are true (1), the transposition and flipping may be performed together.

[0559] For example, the transpose and flip operations can be expressed as element-wise operations for the transformation matrix T H The base transformation matrix T base In other words, it can be expressed as the following equation (52).

[0560]

number

[0561] Here, the transformation matrix T H The i-th row and j-th column component ((i,j) component) of T H The notation is [i,j]. Also, the second line "for i,j=0,...,N-1" indicates that i and j each have a value from 0 to N-1. That is, T H [i,j] is the N-row, N-column transformation matrix T H means to show all elements of

[0562] In this way, transposition and flipping can be performed in a single operation, by simply accessing a two-dimensional array. Also, since the flip matrix J is not required, the increase in memory capacity can be suppressed.

[0563] In this embodiment, in step S361 in Fig. 32, the transformation matrix derivation process executed by the transformation matrix derivation unit 351 of the primary vertical transformation unit 313 is also performed in the same manner as in the flowchart in Fig. 61. For example, in the flowchart in Fig. 61, the transformation type identifier TrTypeIdxH is changed to TrTypeIdxV, and the transformation matrix T H Transform the matrix T V This can be interpreted by replacing it with.

[0564] In this embodiment, in step S441 in Fig. 41, the transformation matrix derivation process executed by the transformation matrix derivation unit 421 of the inverse primary vertical transformation unit 412 is also performed in the same manner as in the flowchart in Fig. 61. For example, in the flowchart in Fig. 61, the transformation type identifier TrTypeIdxH is changed to TrTypeIdxV, and the transformation matrix T H Transform the matrix T V This can be interpreted by replacing it with.

[0565] Furthermore, in this embodiment, in step S461 in FIG. 42, the transformation matrix derivation process executed by the transformation matrix derivation unit 451 in the inverse primary horizontal transformation unit 413 is also performed in the same manner as in the flowchart in FIG.

[0566] Therefore, a description of these will be omitted.

[0567] <3. Second embodiment> <3-1. Common Concept> <Replacement of prediction residuals> In the first embodiment, a transform matrix is ​​manipulated to generate another transform matrix, but not only the transform matrix but also the prediction residual may be manipulated. That is, the prediction residual of an image may be permuted, and the permuted prediction residual may be orthogonally transformed using a base transform matrix, and the coefficient data obtained by orthogonally transforming the prediction residual may be coded to generate a bit stream.

[0568] For example, an image processing device may include an operation unit that performs a substitution operation on a prediction residual of an image, an orthogonal transformation unit that performs an orthogonal transform on the prediction residual that has been replaced by the operation unit using a base transformation matrix, and an encoding unit that encodes coefficient data obtained by orthogonally transforming the prediction residual by the orthogonal transformation unit to generate a bit stream.

[0569] Even in this way, it is possible to realize a two-dimensional orthogonal transformation equivalent to that in the first embodiment. That is, an orthogonal transformation using a certain transformation matrix can be realized by substantially realizing an orthogonal transformation using another transformation matrix. Therefore, as in the first embodiment, it is possible to suppress an increase in the number of transformation matrices prepared for the orthogonal transformation, and to suppress an increase in the memory capacity required for the orthogonal transformation. In addition, since the number of operations can be reduced compared to the first embodiment, an increase in the amount of processing for the orthogonal transformation can be suppressed.

[0570] Alternatively, the bit stream may be decoded to obtain coefficient data obtained by orthogonally transforming the prediction residual of the image, the obtained coefficient data may be inversely orthogonally transformed, and the result of the inverse orthogonal transform of the obtained coefficient data may be replaced.

[0571] For example, an image processing device may include a decoding unit that decodes a bit stream to obtain coefficient data obtained by orthogonal transforming the prediction residual of an image, an inverse orthogonal transform unit that performs an inverse orthogonal transform on the coefficient data obtained by the decoding unit, and an operation unit that performs a substitution operation on the inverse orthogonal transform result of the coefficient data obtained by the inverse orthogonal transform unit.

[0572] Even in this way, it is possible to realize an inverse two-dimensional orthogonal transform equivalent to that in the first embodiment. That is, by using an inverse orthogonal transform using a certain transform matrix, it is possible to substantially realize an inverse orthogonal transform using another transform matrix. Therefore, as in the first embodiment, it is possible to suppress an increase in the number of transform matrices prepared for the inverse orthogonal transform, and it is possible to suppress an increase in the memory capacity required for the inverse orthogonal transform. In addition, since the number of operations can be reduced compared to the first embodiment, it is possible to suppress an increase in the processing amount of the inverse orthogonal transform.

[0573] To explain more specifically, when focusing on the spatial symmetry of two different two-dimensional orthogonal transforms (including prediction residuals), there may be cases where there is axial symmetry between one-dimensional orthogonal transforms in a certain direction (horizontal or vertical direction), such as between the transform matrix of DST7 and the transform matrix of FlipDST7 obtained by flipping the transform matrix of DST7. In addition, there may be cases where there is axial symmetry between prediction residuals in a certain direction (horizontal or vertical direction), such as between a prediction residual (X) and a flipped version of that prediction residual (X·J).

[0574] In such a case, for example, as shown in Figure 63, the original prediction residual can be flipped and then orthogonally transformed using a first transformation matrix, thereby replacing the orthogonal transformation of the original prediction residual using a second transformation matrix obtained by flipping the first transformation matrix.

[0575] In the example of Figure 63, on the left side of the figure, a one-dimensional orthogonal transform in the vertical direction using a first transformation matrix and a one-dimensional orthogonal transform in the horizontal direction using the flipped first transformation matrix are performed on the original prediction residual, as shown in the following equation (53).

[0576]

number

[0577] In contrast, the right side of the figure shows the flip of the prediction residual (equation (54) below) and the vertical and horizontal one-dimensional orthogonal transform (equation (55) below) of the flipped prediction residual using a first transform matrix.

[0578]

number

[0579] Two such two-dimensional orthogonal transformations have spatial symmetry, so one can be substituted for the other.

[0580] In other words, when spatial symmetry exists between a base transformation matrix (transformation type) and a transformation matrix derived using that transformation matrix and used for orthogonal transformation of the prediction residual, as described above, the prediction residual can be flipped in the direction of the spatial symmetry, and the flipped prediction residual can be orthogonally transformed using the base transformation matrix, thereby replacing the orthogonal transformation using another transformation matrix (the orthogonal transformation of the prediction residual before flipping, using the transformation matrix derived using the base transformation matrix).

[0581] That is, the operation unit may flip the prediction residual in the direction of spatial symmetry between one-dimensional orthogonal transformation, and the orthogonal transformation unit may orthogonally transform the prediction residual flipped by the operation unit using a base transformation matrix. In this way, it is possible to perform an orthogonal transformation equivalent to the orthogonal transformation using another transformation matrix. In other words, as described above, it is possible to practically realize an orthogonal transformation using another transformation matrix. Therefore, it is possible to suppress an increase in the number of transformation matrices prepared for orthogonal transformation, and to suppress an increase in memory capacity required for orthogonal transformation.

[0582] This is also true for inverse orthogonal transform. That is, the coefficient data obtained by decoding the bit stream may be inversely orthogonally transformed using a base transform matrix, and the operation unit may flip the inverse orthogonal transform result in the direction of spatial symmetry between one-dimensional orthogonal transforms. In this way, since it is possible to practically realize inverse orthogonal transform using another transform matrix, it is possible to suppress the increase in the number of transform matrices prepared for inverse orthogonal transform, and it is possible to suppress the increase in memory capacity required for inverse orthogonal transform.

[0583] For example, the left side of FIG. 64 shows a two-dimensional orthogonal transform in which a one-dimensional orthogonal transform in the horizontal direction is performed on a prediction residual X using FlipDST7 obtained by flipping the transform matrix of DST7, and a one-dimensional orthogonal transform in the vertical direction is performed using the transform matrix of DST7. Also, the right side of FIG. 64 shows a two-dimensional orthogonal transform in which a one-dimensional orthogonal transform in the horizontal and vertical directions is performed on a prediction residual X' (= X·J) obtained by flipping the prediction residual X in the horizontal direction using the transform matrix of DST7. These are symmetrical with each other in the horizontal direction and are equivalent. Therefore, for example, the latter two-dimensional orthogonal transform can replace the former two-dimensional orthogonal transform. The former two-dimensional orthogonal transform is expressed as shown in the following formula (56), and the latter two-dimensional orthogonal transform is expressed as shown in the following formula (57).

[0584]

number

[0585] That is, for example, when there is horizontal symmetry between a base transformation matrix (transformation type) and a transformation matrix derived using the base transformation matrix and used for orthogonal transformation of a prediction residual, the operation unit may flip the prediction residual in the horizontal direction, and the orthogonal transformation unit may orthogonally transform the prediction residual flipped in the horizontal direction by the operation unit using the base transformation matrix. In this way, it is possible to practically realize an orthogonal transformation using another transformation matrix that has horizontal symmetry with the orthogonal transformation using the base transformation matrix.

[0586] The same applies to the inverse orthogonal transform. In other words, if there is horizontal symmetry between a base transform matrix (transform type) and a transform matrix derived using the base transform matrix and used for inverse orthogonal transform of coefficient data, the inverse orthogonal transform unit may perform inverse orthogonal transform on the coefficient data obtained by decoding the bit stream using the base transform matrix, and the operation unit may flip the inverse orthogonal transform result in the horizontal direction. In this way, it is possible to practically realize inverse orthogonal transform using another transform matrix that has horizontal symmetry with the inverse orthogonal transform using the base transform matrix.

[0587] Also, for example, the left side of FIG. 65 shows a two-dimensional orthogonal transform in which a one-dimensional orthogonal transform in the horizontal direction using the transform matrix of DST7 is performed on the prediction residual X, and a one-dimensional orthogonal transform in the vertical direction using FlipDST7 obtained by flipping the transform matrix of DST7 is performed on the prediction residual X' (= J·X) obtained by flipping the prediction residual X in the vertical direction. These are symmetrical to each other in the vertical direction and are equivalent. Therefore, for example, the latter two-dimensional orthogonal transform can replace the former two-dimensional orthogonal transform. The former two-dimensional orthogonal transform is expressed as in the following formula (58), and the latter two-dimensional orthogonal transform is expressed as in the following formula (59).

[0588]

number

[0589] That is, for example, when there is vertical symmetry between a base transformation matrix (transformation type) and a transformation matrix derived using the base transformation matrix and used for orthogonal transformation of a prediction residual, the operation unit may flip the prediction residual in the vertical direction, and the orthogonal transformation unit may orthogonally transform the prediction residual flipped in the vertical direction by the operation unit using the base transformation matrix. In this way, it is possible to practically realize an orthogonal transformation using another transformation matrix that has vertical symmetry with the orthogonal transformation using the base transformation matrix.

[0590] The same applies to the inverse orthogonal transform. In other words, if there is vertical symmetry between a base transform matrix (transform type) and a transform matrix derived using the base transform matrix and used for inverse orthogonal transform of coefficient data, the inverse orthogonal transform unit may perform inverse orthogonal transform on the coefficient data obtained by decoding the bit stream using the base transform matrix, and the operation unit may flip the inverse orthogonal transform result in the vertical direction. In this way, it is possible to practically realize inverse orthogonal transform using another transform matrix that has vertical symmetry with the inverse orthogonal transform using the base transform matrix.

[0591] Further, for example, the left side of FIG. 66 shows a two-dimensional orthogonal transform in which a one-dimensional orthogonal transform is performed in the horizontal and vertical directions on a prediction residual X using FlipDST7 obtained by flipping the transform matrix of DST7. Also, the right side of FIG. 66 shows a two-dimensional orthogonal transform in which a one-dimensional orthogonal transform is performed in the horizontal and vertical directions on a prediction residual X' (= J·X·J) obtained by flipping the prediction residual X in the horizontal and vertical directions using the transform matrix of DST7. These are symmetrical with each other in both the horizontal and vertical directions and are equivalent. Therefore, for example, the latter two-dimensional orthogonal transform can replace the former two-dimensional orthogonal transform. The former two-dimensional orthogonal transform is expressed as in the following formula (60), and the latter two-dimensional orthogonal transform is expressed as in the following formula (61).

[0592]

number

[0593] That is, for example, when there is horizontal and vertical symmetry between a base transformation matrix (transformation type) and a transformation matrix derived using the transformation matrix and used for orthogonal transformation of a prediction residual, the operation unit may flip the prediction residual in the horizontal and vertical directions, and the orthogonal transformation unit may orthogonally transform the prediction residual flipped in the horizontal and vertical directions by the operation unit using the base transformation matrix. In this way, it is possible to practically realize orthogonal transformation using another transformation matrix that has horizontal and vertical symmetry with the orthogonal transformation using the base transformation matrix.

[0594] The same applies to the inverse orthogonal transform. In other words, when there is horizontal and vertical symmetry between a base transform matrix (transform type) and a transform matrix derived using the transform matrix and used for inverse orthogonal transform of coefficient data, the inverse orthogonal transform unit may perform inverse orthogonal transform on the coefficient data obtained by decoding the bit stream using the base transform matrix, and the operation unit may flip the inverse orthogonal transform result in the horizontal and vertical directions. In this way, it is possible to practically realize inverse orthogonal transform using another transform matrix that has horizontal and vertical symmetry with the inverse orthogonal transform using the base transform matrix.

[0595] In the case of the above formula (60), since the transformation matrices T1 and T2 are flipped, two operations are required. In contrast, in the case of the above formula (61), the flip of the prediction residual (J·X·J) can be replaced with the matrix product of the permutation matrix P and the prediction residual X. That is, it can be performed in one operation. Therefore, compared with the case of the above formula (60) (when the transformation matrix is ​​flipped), it is possible to reduce the number of operations and suppress an increase in the amount of processing of the orthogonal transform and the inverse orthogonal transform (the orthogonal transform and the inverse orthogonal transform can be performed more easily). That is, it is possible to suppress an increase in the processing load of the orthogonal transform and the inverse orthogonal transform. In addition, the orthogonal transform and the inverse orthogonal transform can be performed faster.

[0596] <Sign inversion of transformation matrix> In addition to the above-mentioned replacement operation of the prediction residual, the operation of the transformation matrix may also be applied. For example, the derivation unit may derive the second transformation matrix using the first transformation matrix, the operation unit may flip the prediction residual in the direction of spatial symmetry between the one-dimensional orthogonal transformation, and the orthogonal transformation unit may orthogonally transform the prediction residual flipped by the operation unit using the second transformation matrix derived by the derivation unit. In this way, it is possible to practically realize the orthogonal transformation by another transformation matrix. Therefore, it is possible to suppress the increase in the number of transformation matrices prepared for the orthogonal transformation, and to suppress the increase in the memory capacity required for the orthogonal transformation.

[0597] The same applies to the inverse orthogonal transform. For example, the derivation unit may derive a second transform matrix using the first transform matrix, the inverse orthogonal transform unit may perform inverse orthogonal transform on the coefficient data obtained by the decoding unit using the second transform matrix derived by the derivation unit, and the operation unit may perform a substitution operation on the inverse orthogonal transform result of the coefficient data obtained by the inverse orthogonal transform unit. In this way, it is possible to practically realize an inverse orthogonal transform using another transform matrix. Therefore, it is possible to suppress an increase in the number of transform matrices prepared for the inverse orthogonal transform, and to suppress an increase in the memory capacity required for the inverse orthogonal transform.

[0598] In addition, when deriving the second transformation matrix in the orthogonal transformation or inverse orthogonal transformation, the derivation unit may derive a second transformation matrix having the same number of rows and columns as the first transformation matrix. When the number of rows and columns is changed, the type of waveform may change unintentionally. Therefore, by making the number of rows and columns the same as the first transformation matrix, the possibility of an unintentional change in the type of waveform can be suppressed, and the derivation of the second transformation matrix can be more easily performed.

[0599] In addition, when deriving the second transformation matrix, the derivation unit may invert the sign of the odd-numbered row vector of the first transformation matrix to derive the second transformation matrix. By using the second transformation matrix derived in this way to perform an orthogonal transformation with a permutation operation on the prediction residual as described above, it is possible to realize (substitute) an orthogonal transformation of the prediction residual using another transformation matrix in which the even-numbered row vectors are axially symmetric and the odd-numbered row vectors are point symmetric with respect to the first transformation matrix. In addition, by performing an inverse orthogonal transformation using the second transformation matrix derived in this way and performing a permutation operation on the inverse orthogonal transformation result as described above, it is possible to realize (substitute) an inverse orthogonal transformation using another transformation matrix in which the even-numbered row vectors are axially symmetric and the odd-numbered row vectors are point symmetric with respect to the first transformation matrix.

[0600] <Example of operation of prediction residual> FIG. 67 shows a list of examples of operations on prediction residuals to replace the orthogonal transform and inverse orthogonal transform using other transform matrices as described above.

[0601] In the table shown in Figure 67, excluding the row with the topmost item name, the operation of the example in the first row from the top focuses on the fact that in two different two-dimensional orthogonal transforms, there is axial symmetry between the one-dimensional orthogonal transforms in a certain direction (the horizontal direction, the vertical direction, or both).

[0602] In this case, the operation unit flips the prediction residual in the direction of spatial symmetry between one-dimensional orthogonal transforms, and the orthogonal transform unit orthogonally transforms the prediction residual flipped by the operation unit using a base transformation matrix. More specifically, the operation unit flips the prediction residual, and the orthogonal transform unit transforms the transformation matrix of DST7 into a base transformation matrix T base Then, the flipped prediction residual is orthogonally transformed using the transformation matrix of the DST7. In this way, it is possible to substitute the orthogonal transformation of the non-flipped prediction residual using the transformation matrix of FlipDST7. Note that the direction of flip of the prediction residual corresponds to the direction of spatial symmetry (horizontal or vertical, or both).

[0603] The same is true for the inverse orthogonal transform. The inverse orthogonal transform unit performs inverse orthogonal transform on the coefficient data obtained by decoding the bit stream using a base transform matrix, and the operation unit flips the inverse orthogonal transform result in the direction of spatial symmetry between one-dimensional orthogonal transforms. More specifically, the inverse orthogonal transform unit performs inverse orthogonal transform on the coefficient data using the transform matrix of DST7, and the operation unit flips the inverse orthogonal transform result. In this way, it is possible to substitute the inverse orthogonal transform of the coefficient data using the transform matrix of FlipDST7. The flip direction of the inverse orthogonal transform result corresponds to the direction of spatial symmetry (horizontal or vertical, or both).

[0604] By applying such operations to the prediction residuals (including the inverse orthogonal transform results), it becomes unnecessary to prepare the transform matrix of FlipDST7 as a candidate for the transform matrix used in the orthogonal transform and inverse orthogonal transform. That is, as in the case of the first embodiment, it is possible to reduce the eigentransform type. That is, it is possible to suppress an increase in the LUT size. Furthermore, by performing the orthogonal transform and inverse orthogonal transform involving operations on such prediction residuals (including the inverse orthogonal transform results), it is possible to obtain coding efficiency equivalent to that of the orthogonal transform and inverse orthogonal transform using the transform matrix of FlipDST7.

[0605] Furthermore, compared to the case of orthogonal transform and inverse orthogonal transform using the transform matrix of FlipDST7, two-dimensional flip operations on prediction residuals (including inverse orthogonal transform results) can be performed in a single operation, thereby reducing the number of flip operations. Therefore, it is possible to suppress an increase in the amount of processing for orthogonal transform and inverse orthogonal transform (orthogonal transform and inverse orthogonal transform can be performed more easily). In other words, it is possible to suppress an increase in the processing load for orthogonal transform and inverse orthogonal transform. Also, it is possible to perform orthogonal transform and inverse orthogonal transform at higher speeds.

[0606] The operation in the example one level below (second from the top) focuses on the similarity between one-dimensional orthogonal transformations in a certain direction (horizontal or vertical direction, or both) in two different two-dimensional orthogonal transformations. More specifically, it focuses on the fact that even-numbered row vectors have axial symmetry and odd-numbered row vectors have point symmetry.

[0607] In this case, the derivation unit derives the second transformation matrix using the first transformation matrix, the operation unit performs a substitution operation on the prediction residual of the image, and the orthogonal transformation unit performs an orthogonal transformation on the prediction residual substituted by the operation unit using the second transformation matrix derived by the derivation unit. More specifically, the derivation unit inverts the sign of the odd-numbered row vector of the transformation matrix of DST7, the operation unit flips the prediction residual, and the orthogonal transformation unit orthogonally transforms the flipped prediction residual using the transformation matrix derived by the derivation unit and obtained by inverting the sign of the odd-numbered row vector of the transformation matrix of DST7. In this way, the orthogonal transformation of the prediction residual that is not flipped can be substituted using the transformation matrix of DCT8.

[0608] The direction of flip of the prediction residual corresponds to the direction of spatial symmetry (horizontal or vertical, or both). In addition, since the operation on the transformation matrix is ​​only sign inversion for odd-numbered row vectors, the number of rows and columns of the transformation matrix derived by the derivation unit is the same as that of the DST7 transformation matrix.

[0609] The same is true for the inverse orthogonal transform. The derivation unit derives a second transform matrix using a first transform matrix, the inverse orthogonal transform unit performs inverse orthogonal transform on the coefficient data obtained by decoding the bit stream using the second transform matrix, and the operation unit flips the inverse orthogonal transform result in the direction of spatial symmetry between one-dimensional orthogonal transforms. More specifically, the derivation unit inverts the sign of odd-numbered row vectors of the transform matrix of DST7, the inverse orthogonal transform unit inverts the coefficient data using the transform matrix of DST7 with the odd-numbered row vectors inverted, and the operation unit flips the inverse orthogonal transform result. In this way, it is possible to substitute the inverse orthogonal transform of the coefficient data using the transform matrix of DCT8.

[0610] The flip direction of the inverse orthogonal transform result corresponds to the direction of spatial symmetry (horizontal or vertical, or both). In addition, since the only operation on the transformation matrix is ​​sign inversion for odd-numbered row vectors, the number of rows and columns of the transformation matrix derived by the derivation unit is the same as that of DST7.

[0611] By applying such operations to the prediction residuals (including the inverse orthogonal transform results), it becomes unnecessary to prepare a DCT8 transform matrix as a candidate for the transform matrix used in the orthogonal transform and inverse orthogonal transform. That is, as in the case of the first embodiment, it is possible to reduce the eigentransform type. That is, it is possible to suppress an increase in the LUT size. Furthermore, by performing the orthogonal transform and inverse orthogonal transform involving operations on such prediction residuals (including the inverse orthogonal transform results), it is possible to obtain the same coding efficiency as in the case of the orthogonal transform and inverse orthogonal transform using the DCT transform matrix.

[0612] Furthermore, the operation in the example one level below (the third level from the top) focuses on the fact that in two different two-dimensional orthogonal transforms, there is axial symmetry between the one-dimensional orthogonal transforms in a certain direction (the horizontal or vertical direction, or both), and is the same operation as the first level from the top.

[0613] However, in this case, the DCT8 transformation matrix is ​​the base transformation matrix T base Therefore, by doing this, it is possible to substitute the orthogonal transform of the prediction residual that is not flipped using the transform matrix of FlipDCT8. The same is true for the inverse orthogonal transform. That is, by doing this, it is possible to substitute the inverse orthogonal transform of the coefficient data using the transform matrix of FlipDCT8.

[0614] In addition, the operation in the example one row below (fourth row from the top) focuses on the similarity between one-dimensional orthogonal transforms in a certain direction (horizontal or vertical, or both) in two different two-dimensional orthogonal transforms (even-numbered row vectors have axial symmetry, and odd-numbered row vectors have point symmetry), and is the same as the operation in the second row from the top.

[0615] However, in this case, the DCT8 transformation matrix is ​​the base transformation matrix T base Therefore, by doing this, it is possible to substitute the orthogonal transform of the non-flipped prediction residual using the transform matrix of DST7. The same is true for the inverse orthogonal transform. In other words, by doing this, it is possible to substitute the inverse orthogonal transform of the coefficient data using the transform matrix of DST7.

[0616] Each of the above-mentioned derivation examples may be implemented independently, or a plurality of derivation examples may be implemented in combination. In the above-mentioned examples, DST7 or DCT8 is used as an example of the base transform type (first transform type), but it can be realized by replacing it with a transform type having the same waveform shape as shown in FIG. 8. For example, DST7 may be replaced with other orthogonal transforms having the same waveform shape, such as DST4, DST8, or DST3. Similarly, DCT8 may be replaced with other orthogonal transforms having the same waveform shape, such as DCT3, DCT7, or DCT4.

[0617] <Configuration and processing flow> The configuration of the image coding device 100 that performs such a replacement operation of the prediction residual is the same as that of the first embodiment. In this image coding device 100, the orthogonal transform unit 113 performs the process to which the present technology is applied as an operation unit and an orthogonal transform unit. Furthermore, the encoding unit 115 performs the process to which the present technology is applied as an encoding unit. Furthermore, the inverse orthogonal transform unit 118 performs the process to which the present technology is applied as an inverse orthogonal transform unit and an operation unit. Therefore, the image coding device 100 can suppress an increase in memory capacity required for the orthogonal transform and the inverse orthogonal transform.

[0618] The configuration of the orthogonal transform unit 113 is the same as that of the first embodiment. In the orthogonal transform unit 113, the primary transform unit 152 performs the above-mentioned processing to which the present technology is applied as an operation unit and an orthogonal transform unit. That is, the operation unit performs a substitution operation on the prediction residual of the image, and the orthogonal transform unit performs a primary transformation on the prediction residual that has been subjected to the substitution operation by the operation unit, using a base transformation matrix. Therefore, it is possible to suppress an increase in memory capacity required for the primary transformation.

[0619] As described above, the primary transform unit 152 performs primary horizontal transform and primary vertical transform as primary transform. That is, the operation unit performs a substitution operation on the prediction residual of the image, and the orthogonal transform unit performs a one-dimensional orthogonal transform in the horizontal direction on the prediction residual subjected to the substitution operation as primary transform using a second transform matrix for one-dimensional orthogonal transform in the horizontal direction as a base, and further performs a one-dimensional orthogonal transform in the vertical direction using a second transform matrix for one-dimensional orthogonal transform in the vertical direction as a base. Therefore, it is possible to suppress an increase in memory capacity required for such primary transform in which one-dimensional orthogonal transform in the horizontal direction and one-dimensional orthogonal transform in the vertical direction are performed.

[0620] The flow of the image encoding process executed by the image encoding device 100 is the same as that of the first embodiment. That is, in the image encoding process having the above flow, the process to which the present technology is applied is performed as the process of step S106. Furthermore, the process to which the present technology is applied is performed as the process of step S109. Furthermore, the process to which the present technology is applied is performed as the process of step S113. Therefore, by performing this image encoding process, it is possible to suppress an increase in memory capacity required for orthogonal transform and inverse orthogonal transform.

[0621] In addition, the flow of the orthogonal transform process executed in step S106 of the image encoding process is the same as that of the first embodiment. That is, in the orthogonal transform process of the above flow, the process to which the present technology is applied is executed as the process of step S132. Therefore, by executing this orthogonal transform process, it is possible to suppress an increase in memory capacity required for primary transform.

[0622] The same is true for the decoding side. That is, the configuration of the image decoding device 200 that performs such a replacement operation of the inverse orthogonal transform result is the same as that of the first embodiment. In this image decoding device 200, the inverse orthogonal transform unit 214 performs the above-mentioned processing to which the present technology is applied as an inverse orthogonal transform unit and an operation unit. Also, the decoding unit 212 performs the above-mentioned processing to which the present technology is applied as a decoding unit. Therefore, the image decoding device 200 can suppress an increase in memory capacity required for the inverse orthogonal transform.

[0623] The configuration of the inverse orthogonal transform unit 214 is the same as that of the first embodiment. In the inverse orthogonal transform unit 214, the inverse primary transform unit 253 performs the above-mentioned processing to which the present technology is applied as an inverse orthogonal transform unit and an operation unit. That is, the inverse orthogonal transform unit performs an inverse primary transform on the inverse secondary transform result using a base transform matrix, and the operation unit performs a substitution operation on the obtained inverse primary transform result. Therefore, it is possible to suppress an increase in the memory capacity required for the inverse primary transform.

[0624] As described above, the inverse primary transform unit 253 performs an inverse primary vertical transform and an inverse primary horizontal transform as the inverse primary transform. That is, the inverse orthogonal transform unit performs an inverse one-dimensional orthogonal transform in the vertical direction as the inverse primary transform on the inverse secondary transform result using a second transform matrix for the inverse one-dimensional orthogonal transform in the vertical direction as the base, and further performs an inverse one-dimensional orthogonal transform in the horizontal direction using a second transform matrix for the inverse one-dimensional orthogonal transform in the horizontal direction as the base. Therefore, it is possible to suppress an increase in memory capacity required for such primary transform in which an inverse one-dimensional orthogonal transform in the vertical direction and an inverse one-dimensional orthogonal transform in the horizontal direction are performed.

[0625] The flow of the image decoding process executed by the image decoding device 200 is the same as that of the first embodiment. That is, in the image decoding process having the above flow, the process to which the present technology is applied is performed as the process of step S202. Also, the process to which the present technology is applied is performed as the process of step S204. Therefore, by performing this image decoding process, it is possible to suppress an increase in memory capacity required for inverse orthogonal transform.

[0626] In addition, the flow of the inverse orthogonal transform process executed in step S204 of the image decoding process is the same as that of the first embodiment. That is, in the inverse orthogonal transform process of the above flow, the process to which the present technology is applied is executed as the process of step S233. Therefore, by executing this inverse orthogonal transform process, it is possible to suppress the increase in memory capacity required for the inverse primary transform.

[0627] <3-2. Example 2-1> <Concept> Next, each of the derivation examples described with reference to Fig. 67 will be described in more detail. First, in the table shown in Fig. 67, except for the row of the top item name, the operation of the example in the first row from the top focuses on the fact that there is axial symmetry between one-dimensional orthogonal transforms in a certain direction (horizontal direction or vertical direction, or both) in two different two-dimensional orthogonal transforms.

[0628] In this case, the operation unit flips the prediction residual. The orthogonal transformation unit orthogonally transforms the flipped prediction residual using a base transformation matrix (the transformation matrix of DST7). In this way, it is possible to substitute the orthogonal transformation of the prediction residual using the transformation matrix of FlipDST7 (see <2-2. Example 1-1>) obtained by flipping the transformation matrix of DST7.

[0629] The same applies to the inverse orthogonal transform. In this case, the inverse orthogonal transform unit performs inverse orthogonal transform on the coefficient data using a base transform matrix (the transform matrix of DST7), and the operation unit flips the inverse orthogonal transform result. In this way, it is possible to substitute the inverse orthogonal transform of the coefficient data (the transform coefficient Coeff_IQ) using the transform matrix of FlipDST7, which is obtained by flipping the transform matrix of DST7.

[0630] By applying such an operation to the prediction residual (including the inverse orthogonal transform result), it becomes unnecessary to prepare a transform matrix of FlipDST7 as a candidate for a transform matrix used in the orthogonal transform and the inverse orthogonal transform. In other words, as in the first embodiment, it is possible to reduce the number of eigentransform types.

[0631] In this case, as shown in the table of Fig. 68, the number of transformation matrices required, which was five in the case of the technique described in Non-Patent Document 1 (see Table A in Fig. 6), can be reduced to four, so the total LUT size can be about 47 KB. In other words, the size of the LUT can be reduced (by about 53 KB (Table A in Fig. 6)) compared to the case of the technique described in Non-Patent Document 1. In other words, an increase in the size of the LUT can be suppressed.

[0632] In addition, compared to the case of orthogonal transform and inverse orthogonal transform using the transform matrix of FlipDST7, two-dimensional flip operations on the prediction residuals (including the inverse orthogonal transform results) can be performed in a lump, so the number of flip operations can be reduced. Therefore, it is possible to suppress an increase in the amount of processing for the orthogonal transform and inverse orthogonal transform (orthogonal transform and inverse orthogonal transform can be performed more easily).

[0633] In addition, by performing orthogonal transform and inverse orthogonal transform involving manipulation of such prediction residuals (including the inverse orthogonal transform results), coding efficiency can be achieved that is equivalent to that of orthogonal transform and inverse orthogonal transform using the FlipDST7 transform matrix.

[0634] <Primary conversion section> Next, a configuration and processing for performing such processing will be described. Fig. 69 is a block diagram showing a main configuration example of the primary conversion unit 152 in this case. As shown in Fig. 69, the primary conversion unit 152 in this case has a prediction residual replacement operation unit 551 in addition to the configuration shown in Fig. 22.

[0635] The primary transform selection unit 311 also supplies the derived transform type identifier TrTypeIdxH of the primary horizontal transform and the transform type identifier TrTypeIdxV of the primary vertical transform to the prediction residual replacement operation unit 551 .

[0636] The prediction residual replacement operation unit 551 receives the prediction residual D supplied from the switch 151, as well as the transformation type identifier TrTypeIdxH of the primary horizontal transform and the transformation type identifier TrTypeIdxV of the primary vertical transform. The prediction residual replacement operation unit 551 flips the prediction residual D in accordance with the transformation type identifier TrTypeIdxH and the transformation type identifier TrTypeIdxV. The prediction residual replacement operation unit 551 flips the prediction residual D after the flip. flip is supplied to the primary horizontal converter 312.

[0637] In the primary transform unit 152 configured as above, the prediction residual replacement operation unit 551 performs the above-mentioned processing to which the present technology is applied as an operation unit. The primary horizontal transform unit 312 and the primary vertical transform unit 313 each perform the above-mentioned processing to which the present technology is applied as an orthogonal transform unit.

[0638] That is, the primary horizontal transform unit 312, as an orthogonal transform unit, uses a transformation matrix for one-dimensional orthogonal transform in the horizontal direction as a base to transform the post-flip prediction residual D flipTherefore, primary horizontal transform unit 312 can suppress an increase in memory capacity required for one-dimensional orthogonal transform in the horizontal direction.

[0639] Furthermore, the primary vertical transform unit 313, as an orthogonal transform unit, performs a vertical one-dimensional orthogonal transform on the transform coefficients Coeff_Phor after the primary horizontal transform, using a transform matrix for a vertical one-dimensional orthogonal transform as a base. Therefore, the primary vertical transform unit 313 can suppress an increase in memory capacity required for the vertical one-dimensional orthogonal transform.

[0640] <Primary conversion process flow> Next, an example of the flow of the process performed by the above-mentioned configuration will be described. In this case, an example of the flow of the primary conversion process executed in step S132 in FIG. 14 will be described with reference to the flowchart in FIG.

[0641] When the primary transform process starts, in step S561, the primary transform selection unit 311 of the primary transform unit 152 selects the transform type identifier TrTypeIdxH (or the transform type TrTypeH) of the primary horizontal transform and the transform type identifier TrTypeIdxV (or the transform type TrTypeV) of the primary vertical transform.

[0642] In step S562, the prediction residual replacement operation unit 551 executes a prediction residual replacement operation process, flips the prediction residual D according to the transformation type identifier TrTypeIdxH and the transformation type identifier TrTypeIdxV obtained in step S561, and obtains the prediction residual D after the flip. flip Derive.

[0643] In step S563, the primary horizontal transform unit 312 performs a primary horizontal transform process, and converts the post-flip prediction residual D obtained in step S562 using a transform matrix corresponding to the transform type identifier TrTypeIdxH of the primary horizontal transform obtained in step S561. flipPerform the primary horizontal transform on .

[0644] In step S564, the primary vertical transformation unit 313 performs a primary vertical transformation process, and uses a transformation matrix corresponding to the transformation type identifier TrTypeIdxV of the primary vertical transformation obtained in step S561 to perform a primary vertical transformation on the transformation coefficient Coeff_Phor after the primary horizontal transformation obtained by the processing of step S563.

[0645] When the process of step S564 ends, the primary conversion process ends, and the process returns to FIG.

[0646] In the primary conversion process as described above, the process to which the present technology is applied is performed as each process from step S562 to step S564. Therefore, by performing this primary conversion process, it is possible to suppress an increase in memory capacity required for the primary horizontal conversion process and the primary vertical conversion process.

[0647] <Prediction residual replacement process flow> The flow of the prediction residual replacing operation process executed in step S562 in FIG. 70 will be described with reference to the flowchart in FIG.

[0648] When the prediction residual replacement operation process is started, the prediction residual replacement operation unit 551 derives, in step S581, a flip flag FlipFlagH corresponding to the transformation type identifier TrTypeIdxH (or the transformation type TrTypeH) of the primary horizontal transform and a flip flag FlipFlagV corresponding to the transformation type identifier TrTypeIdxV (or the transformation type TrTypeV) of the primary vertical transform. At this time, the prediction residual replacement operation unit 551 obtains the flip flag FlipFlagH and the flip flag FlipFlagV by referring to a correspondence table (LUT_TrTypeIdxToFlipFlag) as shown in Fig. 72, for example. The derivation of these flip flags can be expressed, for example, as the following formulas (62) and (63).

[0649]

number

[0650] In step S582, the prediction residual replacement operation unit 551 determines whether or not the flip flag FlipFlagH and the flip flag FlipFlagV derived in step S581 satisfy the condition represented by the following equation (64).

[0651]

number

[0652] If it is determined that this condition is met, that is, if it is determined that FlipFlagH is true (1) and FlipFlagV is false (0), the process proceeds to step S583.

[0653] In step S583, the prediction residual replacement operation unit 551 flips the prediction residual D in the horizontal direction to obtain the prediction residual D after the flip flip This operation can be expressed as a matrix as shown in the following equation (65).

[0654]

number

[0655] Here, × is an operator representing matrix multiplication, FlipH(X) is an operator representing a horizontal flip operation of matrix X, and the flip matrix J corresponding to FlipV(·) is a left-right flip of the N-row, N-column unit matrix I. When this operation is expressed as an element-by-element calculation, the prediction residual replacement operation unit 551 calculates the prediction residual D after flip as shown in the following equation (66): flip The (i,N-1-j) component of the prediction residual D is set to the i-th row and j-th column component ((i,j) component) of

[0656]

number

[0657] Here, the size of the prediction residual D is a width of N and a height of M (M rows and N columns), where N and M satisfy the following equations (67) and (68).

[0658]

number

[0659] When the process of step S583 ends, the process proceeds to step S588. If it is determined in step S582 that the condition is not satisfied, that is, if it is determined that FlipFlagH is false (0) or FlipFlagV is true (1), the process proceeds to step S584.

[0660] In step S584, the prediction residual replacement operation unit 551 determines whether or not the flip flag FlipFlagH and the flip flag FlipFlagV derived in step S581 satisfy the condition represented by the following equation (69).

[0661]

number

[0662] If it is determined that this condition is met, that is, if it is determined that FlipFlagH is false (0) and FlipFlagV is true (1), the process proceeds to step S585.

[0663] In step S585, the prediction residual replacement operation unit 551 flips the prediction residual D in the vertical direction to obtain the prediction residual D after the flip flip This operation can be expressed as a matrix as shown in the following equation (70).

[0664]

number

[0665] Here, × is an operator representing matrix multiplication, FlipV(X) is an operator representing a flip operation of matrix X in the vertical direction, and the flip matrix J corresponding to FlipV(·) is a left-right flip of the M-row, M-column unit matrix I. When this operation is expressed as an element-by-element calculation, the prediction residual replacement operation unit 551 calculates the prediction residual D after flipping as shown in the following equation (71). flip The (M-1-i,j) component of the prediction residual D is set to the i row and j column component ((i,j) component) of the prediction residual D. Here, the size of the prediction residual D is N in width and M in height (M rows and N columns), where N and M satisfy the above formulas (67) and (68).

[0666]

number

[0667] When the process of step S585 ends, the process proceeds to step S588. If it is determined in step S584 that the condition is not satisfied, that is, if it is determined that FlipFlagH is true (1) or FlipFlagV is false (0), the process proceeds to step S586.

[0668] In step S586, the prediction residual replacement operation unit 551 determines whether or not the flip flag FlipFlagH and the flip flag FlipFlagV derived in step S581 satisfy the condition represented by the following equation (72).

[0669]

number

[0670] If it is determined that this condition is met, that is, if it is determined that FlipFlagH is true (1) and FlipFlagV is true (1), the process proceeds to step S587.

[0671] In step S587, the prediction residual replacement operation unit 551 flips the prediction residual D in the horizontal and vertical directions to obtain the prediction residual D after the flip flip This operation can be expressed as a matrix as shown in the following equation (73).

[0672]

number

[0673] Here, × is an operator representing matrix multiplication, and flip matrix J1 is a left-right inversion of unit matrix I with N rows and N columns, and flip matrix J2 is a left-right inversion of unit matrix I with M rows and M columns. Note that flip matrix J is equivalent to flip matrix J even when transposed (J T When this operation is expressed as an element-by-element calculation, the prediction residual replacement operation unit 551 replaces the prediction residual D after the flip as shown in the following equation (74): flip The (Mi-1, Nj-1) component of the prediction residual D is set to the i-th row and j-th column component ((i, j) component) of the prediction residual D. Here, the size of the prediction residual D is N in width and M in height (M rows and N columns), where N and M satisfy the above formulas (67) and (68).

[0674]

number

[0675] When the process of step S587 ends, the process proceeds to step S588.

[0676] In step S588, the prediction residual replacement operation unit 551 calculates the prediction residual D after the flip as shown in the following equation (75): flip Let D be the prediction residual. In other words, this prediction residual D (prediction residual after flip D flip ) is subjected to primary horizontal transformation processing (step S563 in FIG. 70).

[0677]

number

[0678] When the process of step S588 ends, the prediction residual replacement operation process ends, and the process returns to Fig. 70. Also, when it is determined in step S586 that the condition is not satisfied, that is, when it is determined that FlipFlagH is false (0) or FlipFlagV is false (0), the prediction residual replacement operation process ends, and the process returns to Fig. 70.

[0679] By performing the prediction residual replacement operation process as described above, the prediction residual replacement operation unit 551 can flip the prediction residual in the direction of spatial symmetry that the two-dimensional orthogonal transform (including the prediction residual) has (i.e., in the direction corresponding to the transform type identifier TrTypeIdxH (or transform type TrTypeH) of the primary horizontal transform and the transform type identifier TrTypeIdxV (or transform type TrTypeV) of the primary vertical transform).

[0680] <Inverse primary conversion section> Next, the configuration of the image decoding device 200 in this embodiment will be described. Fig. 73 is a block diagram showing a main configuration example of the inverse primary transform unit 253 (Fig. 16) in this case. As shown in Fig. 73, the inverse primary transform unit 253 in this case has a prediction residual replacement operation unit 552 in addition to the configuration shown in Fig. 34.

[0681] The inverse primary transform selection unit 411 also supplies the derived transform type identifier TrTypeIdxV of the inverse primary vertical transform and the derived transform type identifier TrTypeIdxH of the inverse primary vertical transform to the prediction residual replacement manipulation unit 551. In addition, the inverse primary horizontal transform unit 413 supplies the derived transform coefficient Coeff_IPhor after the inverse primary horizontal transform to the prediction residual replacement manipulation unit 552.

[0682] The prediction residual replacement operation unit 552 receives as input the transform coefficient Coeff_IPhor after the inverse primary horizontal transform supplied from the inverse primary horizontal transform unit 413, as well as the transform type identifier TrTypeIdxH (or transform type TrTypeH) of the primary horizontal transform and the transform type identifier TrTypeIdxV (or transform type TrTypeV) of the primary vertical transform. The prediction residual replacement operation unit 552 flips the transform coefficient Coeff_IPhor after the inverse primary horizontal transform according to the transform type identifier TrTypeIdxH (or transform type TrTypeH) and the transform type identifier TrTypeIdxV (or transform type TrTypeV). The prediction residual replacement operation unit 552 outputs the flipped transform coefficient Coeff_IPhor (the transform coefficient Coeff_IP after the inverse primary transform) as a prediction residual D' to the outside of the inverse primary transform unit 253 (to the calculation unit 215).

[0683] In the inverse primary transform unit 253 configured as above, the prediction residual replacement operation unit 552 performs the above-mentioned processing to which the present technology is applied as an operation unit. The inverse primary vertical transform unit 412 and the inverse primary horizontal transform unit 413 each perform the above-mentioned processing to which the present technology is applied as an inverse orthogonal transform unit.

[0684] That is, the inverse primary vertical transform unit 412, as an inverse orthogonal transform unit, performs a vertical inverse one-dimensional orthogonal transform on the transform coefficients Coeff_IS after the inverse secondary transform, using a transform matrix for the inverse one-dimensional orthogonal transform in the vertical direction as a base. This transform coefficient is then flipped by the prediction residual replacement operation unit 552 as described above, so that it can replace the vertical inverse one-dimensional orthogonal transform using another transform matrix. Therefore, the inverse primary vertical transform unit 412 can suppress an increase in memory capacity required for the vertical inverse one-dimensional orthogonal transform.

[0685] Moreover, the inverse primary horizontal transform unit 413, as an inverse orthogonal transform unit, performs a horizontal inverse one-dimensional orthogonal transform on the transform coefficients Coeff_IPver after the inverse primary vertical transform, using a transform matrix for the horizontal inverse one-dimensional orthogonal transform that serves as a base. This transform coefficient is then flipped by the prediction residual replacement operation unit 552 as described above, thereby replacing the horizontal inverse one-dimensional orthogonal transform using another transform matrix. Therefore, the inverse primary horizontal transform unit 413 can suppress an increase in memory capacity required for the horizontal inverse one-dimensional orthogonal transform.

[0686] <Flow of reverse primary conversion process> Next, an example of the flow of the process performed by the above-mentioned configuration will be described. In this case, an example of the flow of the inverse primary conversion process executed in step S233 in FIG. 18 will be described with reference to the flowchart in FIG.

[0687] When the inverse primary transform process is started, in step S601, the inverse primary transform selection unit 411 of the inverse primary transform unit 253 performs an inverse primary transform selection process and selects the transform type identifier TrTypeIdxV (or the transform type TrTypeV) of the inverse primary vertical transform and the transform type identifier TrTypeIdxH (or the transform type TrTypeH) of the inverse primary horizontal transform.

[0688] In step S602, the inverse primary vertical transform unit 412 performs an inverse primary vertical transform process, and performs a vertical inverse one-dimensional orthogonal transform on the transform coefficient Coeff_IS after the inverse secondary transform, using a transform matrix corresponding to the transform type identifier TrTypeIdxV of the inverse primary vertical transform obtained in step S601.

[0689] In step S603, the inverse primary horizontal transform unit 413 performs an inverse primary horizontal transform process, and performs a horizontal inverse one-dimensional orthogonal transform on the transform coefficients Coeff_IPver after the inverse primary vertical transform derived in step S602, using a transform matrix corresponding to the transform type identifier TrTypeIdxH of the inverse primary horizontal transform obtained in step S601.

[0690] In step S604, the prediction residual replacement operation unit 552 performs a prediction residual replacement operation process, and flips the transformation coefficient Coeff_IPhor after the inverse primary horizontal transformation derived in step S603 using the transformation type identifier TrTypeIdxV of the inverse primary vertical transformation and the transformation type identifier TrTypeIdxH of the inverse primary horizontal transformation obtained in step S601, to derive the transformation coefficient Coeff_IP(prediction residual D') after the inverse primary transformation.

[0691] This prediction residual replacement operation process is performed in the same manner as the example described with reference to the flowchart of Fig. 71. Therefore, the description of Fig. 71 can be applied to the process of this step S604 by replacing the prediction residual D with the transform coefficient Coeff_IPhor as appropriate, and therefore the description thereof will be omitted.

[0692] When the process of step S604 ends, the inverse primary transformation process ends, and the process returns to FIG.

[0693] In the inverse primary transform process described above, the process to which the present technology is applied is performed as each process from step S602 to step S604. Therefore, by performing this inverse primary transform process, it is possible to suppress an increase in memory capacity required for the inverse primary vertical transform process and the inverse primary horizontal transform process.

[0694] <3-3. Example 2-2> <Concept> Next, in the table shown in Fig. 67, excluding the row with the topmost item name, an example in the second row from the top will be described. This operation focuses on the similarity between one-dimensional orthogonal transforms in a certain direction (horizontal or vertical direction, or both) in two different two-dimensional orthogonal transforms. More specifically, it focuses on the fact that even-numbered row vectors have axial symmetry and odd-numbered row vectors have point symmetry.

[0695] In this case, the derivation unit derives the second transformation matrix using the first transformation matrix, the operation unit performs a permutation operation on the prediction residual, and the orthogonal transformation unit performs an orthogonal transformation on the permutation-operated prediction residual using the second transformation matrix. For example, the transformation matrix of DST7 is used as a base transformation matrix, the derivation unit inverts the sign of the odd-numbered row vector of the transformation matrix of DST7, the operation unit flips the prediction residual, and the orthogonal transformation unit performs an orthogonal transformation on the flipped prediction residual using the transformation matrix of DST7 with the sign of the odd-numbered row vector inverted. In this way, the orthogonal transformation of the prediction residual using the transformation matrix of DST7 flipped and the transformation matrix with the sign of the odd-numbered row vector inverted (i.e., DCT8 (see <2-3. Example 1-2>)) can be substituted.

[0696] The same applies to the inverse orthogonal transform. In this case, the derivation unit derives a second transform matrix using a first transform matrix, the inverse orthogonal transform unit performs inverse orthogonal transform on the coefficient data using the second transform matrix, and the operation unit performs a replacement operation on the inverse orthogonal transform result. For example, the transform matrix of DST7 is used as a base transform matrix, the derivation unit inverts the signs of odd-numbered row vectors of the transform matrix of DST7, the inverse orthogonal transform unit performs inverse orthogonal transform on the coefficient data using the transform matrix of DST7 with the signs of the odd-numbered row vectors inverted, and the operation unit flips the inverse orthogonal transform result. In this way, the transform matrix of DST7 is flipped, and the inverse orthogonal transform of the coefficient data (transform coefficient Coeff_IQ) using the transform matrix with the signs of the odd-numbered row vectors inverted (i.e., DCT8 (see <2-3. Example 1-2>)) can be substituted.

[0697] By applying such an operation to the prediction residual (including the inverse orthogonal transform result), it becomes unnecessary to prepare a DCT8 transform matrix as a candidate for the transform matrix used in the orthogonal t...

Claims

1. a derivation unit that derives a second transformation matrix using the first transformation matrix; an orthogonal transformation unit that performs an orthogonal transformation on a prediction residual of an image by using the second transformation matrix derived by the derivation unit; a coding unit that codes coefficient data obtained by orthogonally transforming the prediction residuals by the orthogonal transform unit, and generates a bit stream; An image processing device comprising:

2. The derivation unit uses the first transformation matrix to derive the second transformation matrix having a transformation type different from the first transformation matrix. The image processing device according to claim 1 .

3. The derivation unit uses the first transformation matrix to derive the second transformation matrix having the same number of rows and columns as the first transformation matrix. The image processing device according to claim 1 .

4. The derivation unit derives the second transformation matrix by operating on elements of the first transformation matrix. The image processing device according to claim 1 .

5. The derivation unit derives the second transformation matrix by performing the operation a plurality of times. The image processing device according to claim 4.

6. The derivation unit derives the second transformation matrix by using the first transformation matrix stored in a lookup table. The image processing device according to claim 1 .

7. The orthogonal transform unit performs a primary transform on the prediction residual using the second transform matrix derived by the derivation unit, and further performs a secondary transform on a result of the primary transform. The image processing device according to claim 1 .

8. The deriving unit uses the first transformation matrix to derive the second transformation matrix, in which a lowest-order row vector has a desired type of waveform. The image processing device according to claim 1 .

9. The orthogonal transform unit performs an orthogonal transform of the prediction residual, with a TU (Transform Unit) of a Quad-Tree Block Structure or a QTBT (Quad Tree Plus Binary Tree) Block Structure as a processing unit. The image processing device according to claim 1 .

10. The encoding unit encodes the coefficient data using a coding unit (CU) of a quad-tree block structure or a quad tree plus binary tree (QTBT) block structure as a processing unit. The image processing device according to claim 1 .

11. deriving a second transformation matrix using the first transformation matrix; orthogonally transforming the prediction residual of the image using the derived second transformation matrix; The prediction residual is orthogonally transformed to obtain coefficient data, which is then encoded to generate a bitstream. Image processing methods.

12. a decoding unit that decodes the bit stream and obtains coefficient data obtained by orthogonally transforming the prediction residual of the image; a derivation unit that derives a second transformation matrix using the first transformation matrix; an inverse orthogonal transform unit that performs an inverse orthogonal transform on the coefficient data obtained by the decoding unit, using the second transform matrix derived by the derivation unit; An image processing device comprising:

13. The inverse orthogonal transform unit performs an inverse secondary transform on the coefficient data, and further performs an inverse primary transform on a result of the inverse secondary transform using the second transform matrix derived by the derivation unit. The image processing device according to claim 12.

14. The decoding unit decodes the bit stream using a coding unit (CU) of a quad-tree block structure or a quad tree plus binary tree (QTBT) block structure as a processing unit. The image processing device according to claim 12.

15. The inverse orthogonal transform unit performs an inverse orthogonal transform on the coefficient data, with a TU (Transform Unit) of a Quad-Tree Block Structure or a QTBT (Quad Tree Plus Binary Tree) Block Structure as a processing unit. The image processing device according to claim 12.

16. The bit stream is decoded to obtain coefficient data obtained by orthogonally transforming the prediction residual of the image, deriving a second transformation matrix using the first transformation matrix; The derived second transformation matrix is ​​used to perform an inverse orthogonal transformation on the obtained coefficient data. Image processing methods.

17. an operation unit for performing a replacement operation on a prediction residual of an image; an orthogonal transformation unit that performs an orthogonal transformation on the prediction residual that has been replaced by the operation unit, using a base transformation matrix; a coding unit that codes coefficient data obtained by orthogonally transforming the prediction residuals by the orthogonal transform unit, and generates a bit stream; An image processing device comprising:

18. The manipulation unit flips the prediction residual in a direction of spatial symmetry during one-dimensional orthogonal transformation; The orthogonal transformation unit performs an orthogonal transformation on the prediction residual flipped by the operation unit using the transformation matrix. The image processing device according to claim 17.

19. A derivation unit that derives a second transformation matrix using the first transformation matrix, The orthogonal transformation unit is configured to perform an orthogonal transformation on the prediction residual flipped by the operation unit, using the second transformation matrix derived by the derivation unit. The image processing device according to claim 17.

20. The derivation unit inverts signs of odd-numbered row vectors in the first transformation matrix to derive the second transformation matrix. The image processing device according to claim 19.

21. The orthogonal transform unit performs an orthogonal transform of the prediction residual, with a TU (Transform Unit) of a Quad-Tree Block Structure or a QTBT (Quad Tree Plus Binary Tree) Block Structure as a processing unit. The image processing device according to claim 17.

22. The encoding unit encodes the coefficient data using a coding unit (CU) of a quad-tree block structure or a quad tree plus binary tree (QTBT) block structure as a processing unit. The image processing device according to claim 17.

23. A permutation operation is performed on the prediction residual of the image, orthogonally transforming the prediction residual that has been subjected to the permutation operation using a base transformation matrix; The prediction residual is orthogonally transformed to obtain coefficient data, which is then encoded to generate a bitstream. Image processing methods.

24. a decoding unit that decodes the bit stream and obtains coefficient data obtained by orthogonally transforming the prediction residual of the image; an inverse orthogonal transform unit that performs inverse orthogonal transform on the coefficient data obtained by the decoding unit; an operation unit that performs a permutation operation on the inverse orthogonal transform result of the coefficient data obtained by the inverse orthogonal transform unit; An image processing device comprising:

25. A derivation unit that derives a second transformation matrix using the first transformation matrix, the inverse orthogonal transform unit is configured to perform an inverse orthogonal transform on the coefficient data obtained by the decoding unit, using the second transform matrix derived by the derivation unit; The operation unit is configured to perform a permutation operation on the inverse orthogonal transform result of the coefficient data obtained by the inverse orthogonal transform unit. The image processing device according to claim 24.

26. The decoding unit decodes the bitstream using a CU (Coding Unit) of a Quad-Tree Block Structure or a QTBT (Quad Tree Plus Binary Tree) Block Structure as a processing unit. The image processing device according to claim 24.

27. The inverse orthogonal transform unit performs an inverse orthogonal transform on the coefficient data, with a TU (Transform Unit) of a Quad-Tree Block Structure or a QTBT (Quad Tree Plus Binary Tree) Block Structure as a processing unit. The image processing device according to claim 24.

28. The bit stream is decoded to obtain coefficient data obtained by orthogonally transforming the prediction residual of the image, The obtained coefficient data is subjected to an inverse orthogonal transform; The inverse orthogonal transform result of the obtained coefficient data is replaced. Image processing methods.

Images