Image processing apparatus and method

Abstract

This will help to suppress the increase in memory capacity required for orthogonal and inverse orthogonal transformations. [Solution] A second transformation matrix is ​​derived using a first transformation matrix, the predicted residuals of the image are orthogonally transformed using the derived second transformation matrix, the coefficient data obtained by orthogonally transforming the predicted residuals is encoded, and a bitstream is generated. This disclosure can be applied, for example, to an image processing device, an image encoding device, or an image decoding device.

Description

[Technical Field]

[0001] This disclosure relates to an image processing apparatus and method, and more particularly to an image processing apparatus and method that can suppress the increase in memory capacity required for orthogonal and inverse orthogonal transforms. [Background technology]

[0002] Conventionally, for luminance, Adaptive Multiple Core Transforms (AMTs) have been disclosed, which adaptively select a primary transform from multiple different orthogonal transforms for each Transform Unit (TU) unit, specifically for the horizontal primary transform PThor (also called the primary horizontal transform) and the vertical primary transform PTver (also called the primary vertical transform) (see, for example, Non-Patent Document 1).

[0003] Non-Patent Document 1 lists five one-dimensional orthogonal transformations as candidates for the primary transformation: DCT-II, DST-VII, DCT-VIII, DST-I, and DCT-VI. Furthermore, it proposes adding two more one-dimensional orthogonal transformations, DST-IV and IDT (Identity Transform: one-dimensional transformation skip), bringing the total number of candidate one-dimensional orthogonal transformations to seven (see, for example, Non-Patent Document 2). [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Jianle Chen, Elena Alshina, Gary J. Sullivan, Jens-Rainer, Jill Boyce, "Algorithm Description of Joint Exploration Test Model 4", JVET-G1001_v1, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 7th Meeting: Torino, IT, 13-21 July 2017 [Non-Patent Document 2] V. Lorcy, P. Philippe, "Proposed improvements to the Adaptive multiple Core transform", JVET-C0022, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 3rd Meeting: Geneva, CH, 26 May - 1 June 2016 [Overview of the project] [Problems that the invention aims to solve]

[0005] However, these methods risked increasing the size of the LUT (Look Up Table) required to store the entire transformation matrix of the primary transformation. In other words, when considering the hardware implementation of this primary transformation, there was a risk that the memory size required to store the coefficients of the transformation matrix would increase.

[0006] This disclosure is made in view of these circumstances and aims to suppress the increase in memory capacity required for orthogonal and inverse orthogonal transforms. [Means for solving the problem]

[0007] One aspect of this technology is an image processing device that includes an operation unit for replacing the predicted residual using horizontal identification data, which is set according to the transformation type of the orthogonal transformation applied to the predicted residual of an image, to identify whether to flip the predicted residual horizontally, and vertical identification data, which is set according to the transformation type of the orthogonal transformation applied to the predicted residual of an image, to identify whether to flip the predicted residual vertically.

[0008] One aspect of this technology is an image processing method that includes replacing the predicted residual using horizontal identification data, which is set according to the transformation type of the orthogonal transformation applied to the predicted residual of an image, to identify whether to flip the predicted residual horizontally, and vertical identification data, which is set according to the transformation type of the orthogonal transformation applied to the predicted residual of an image, to identify whether to flip the predicted residual vertically.

[0009] Another aspect of this technology is an image processing device that includes an operation unit that performs an operation to replace the result of an inverse orthogonal transformation of coefficient data obtained by replacing the predicted residuals of an image with an orthogonal transformation, using horizontal identification data that identifies whether to flip the predicted residuals horizontally, which is set according to the transformation type of the orthogonal transformation of the predicted residuals of an image, and vertical identification data that identifies whether to flip the predicted residuals vertically, which is set according to the transformation type of the orthogonal transformation of the predicted residuals of an image.

[0010] Another aspect of this technology is an image processing method that includes replacing the result of an inverse orthogonal transformation of coefficient data obtained by replacing the predicted residuals of an image with horizontal identification data that identifies whether to flip the predicted residuals horizontally, which is set according to the transformation type of the orthogonal transformation of the predicted residuals of an image, and vertical identification data that identifies whether to flip the predicted residuals vertically, which is set according to the transformation type of the orthogonal transformation of the predicted residuals of an image.

[0011] In one aspect of this technology, an image processing apparatus and method is used to perform a replacement operation on the predicted residuals of an image. This operation utilizes horizontal identification data, which is set according to the transformation type of the orthogonal transformation applied to the predicted residuals of the image, to identify whether the predicted residuals should be flipped horizontally, and vertical identification data, which is set according to the transformation type of the orthogonal transformation applied to the predicted residuals of the image, to identify whether the predicted residuals should be flipped vertically.

[0012] In other aspects of the present technology, the image processing apparatus and method utilize horizontal identification data, which is set according to the transformation type of the orthogonal transformation applied to the predicted residuals of an image, to identify whether the predicted residuals are flipped horizontally, and vertical identification data, which is set according to the transformation type of the orthogonal transformation applied to the predicted residuals of an image, to identify whether the predicted residuals are flipped vertically. The result of the inverse orthogonal transformation of the coefficient data obtained by orthogonally transforming the predicted residuals of the image is then replaced. [Effects of the Invention]

[0013] According to this disclosure, images can be processed. In particular, the increase in memory capacity required for orthogonal and inverse orthogonal transformations can be suppressed. [Brief explanation of the drawing]

[0014] [Figure 1] This figure shows the correspondence between the transformation set and the selected orthogonal transformation. [Figure 2] This diagram shows the correspondence between the type of orthogonal transformation and the function used. [Figure 3] This figure shows the correspondence between the transformation set and the prediction mode. [Figure 4] This figure shows examples of the types of orthogonal transformations stored in the LUT. [Figure 5] This figure shows an example of the LUT size required to preserve the transformation matrix in HEVC. [Figure 6] This figure shows an example of the LUT size required to store the transformation matrix. [Figure 7] This diagram illustrates an example of similarity between transformation matrices. [Figure 8] This diagram illustrates examples of transformation types that can be replaced by flipping. [Figure 9] This diagram illustrates examples of transformation types that can be replaced by transposition. [Figure 10] This figure shows a list of the main examples of alternatives to transformation matrices. [Figure 11] This is a block diagram showing a typical configuration of an image coding device. [Figure 12] This block diagram shows the main configuration examples of the orthogonal transformation unit. [Figure 13] This is a flowchart illustrating an example of the image encoding process. [Figure 14] This is a flowchart illustrating an example of the orthogonal transformation process. [Figure 15] This block diagram shows a typical configuration of an image decoding device. [Figure 16] This block diagram shows the main configuration examples of the inverse orthogonal transform section. [Figure 17] This is a flowchart illustrating an example of the image decoding process. [Figure 18] This is a flowchart illustrating an example of the inverse orthogonal transform process. [Figure 19] This figure shows an example of a conversion type derivation. [Figure 20] This figure shows a concrete example of deriving a conversion type. [Figure 21] This figure shows an example of the LUT size required to store the transformation matrix. [Figure 22] This block diagram shows the main configuration examples of the primary conversion unit. [Figure 23] This block diagram shows an example of the main configuration of the primary horizontal conversion unit. [Figure 24] This is a block diagram showing the main configuration examples of the transformation matrix derivation section. [Figure 25] This block diagram shows an example of the main configuration of the primary vertical transformer. [Figure 26] This is a block diagram showing the main configuration examples of the transformation matrix derivation section. [Figure 27]This is a flowchart illustrating an example of the primary conversion process flow. [Figure 28] This flowchart illustrates an example of the primary horizontal transformation process flow. [Figure 29] This figure shows an example of an element-wise operation representation. [Figure 30] This flowchart illustrates an example of the process for deriving transformation matrices. [Figure 31] This figure shows an example of assigning a conversion type to a conversion type identifier. [Figure 32] This flowchart illustrates an example of the flow of a primary vertical transformation process. [Figure 33] This figure shows an example of an element-wise operation representation. [Figure 34] This block diagram shows the main configuration examples of the inverse primary transform section. [Figure 35] This block diagram shows a typical configuration of the inverse primary vertical transformer. [Figure 36] This is a block diagram showing the main configuration examples of the transformation matrix derivation section. [Figure 37] This block diagram shows the main configuration examples of the inverse primary horizontal conversion unit. [Figure 38] This is a block diagram showing the main configuration examples of the transformation matrix derivation section. [Figure 39] This is a flowchart illustrating an example of the inverse primary transformation process. [Figure 40] This flowchart illustrates an example of the inverse primary transformation selection process. [Figure 41] This flowchart illustrates an example of the inverse primary vertical transformation process. [Figure 42] This flowchart illustrates an example of the inverse primary horizontal transformation process. [Figure 43] This figure shows an example of a conversion type derivation. [Figure 44] This figure shows a concrete example of deriving a conversion type. [Figure 45] This figure shows an example of the LUT size required to store the transformation matrix. [Figure 46] This is a block diagram showing the main configuration examples of the transformation matrix derivation section. [Figure 47] This is a block diagram showing the main configuration examples of the transformation matrix derivation section. [Figure 48] This is a block diagram showing the main configuration examples of the transformation matrix derivation section. [Figure 49] This figure shows an example of assigning a conversion type to a conversion type identifier. [Figure 50] This is a block diagram showing the main configuration examples of the transformation matrix derivation section. [Figure 51] This is a block diagram showing the main configuration examples of the transformation matrix derivation section. [Figure 52] This figure shows an example of a conversion type derivation. [Figure 53] This figure shows an example of the LUT size required to store the transformation matrix. [Figure 54] This flowchart illustrates an example of the process for deriving transformation matrices. [Figure 55] This figure shows an example of assigning a conversion type to a conversion type identifier. [Figure 56] This figure shows an example of a conversion type derivation. [Figure 57] This figure shows an example of the LUT size required to store the transformation matrix. [Figure 58] This figure shows an example of assigning a conversion type to a conversion type identifier. [Figure 59] This figure shows an example of a conversion type derivation. [Figure 60] This figure shows an example of the LUT size required to store the transformation matrix. [Figure 61] This flowchart illustrates an example of the process for deriving transformation matrices. [Figure 62] This figure shows an example of assigning a conversion type to a conversion type identifier. [Figure 63] This diagram illustrates the spatial symmetry of a two-dimensional orthogonal transformation. [Figure 64] This diagram illustrates horizontal symmetry. [Figure 65]This is a diagram illustrating vertical symmetry. [Figure 66] This diagram illustrates horizontal and vertical symmetry. [Figure 67] This figure shows a list of the main examples of alternative transformation matrices that involve transforming the predicted residuals. [Figure 68] This figure shows an example of the LUT size required to store the transformation matrix. [Figure 69] This block diagram shows the main configuration examples of the primary conversion unit. [Figure 70] This is a flowchart illustrating an example of the primary conversion process flow. [Figure 71] This flowchart illustrates an example of the flow of the predicted residual replacement operation. [Figure 72] This figure shows an example of assigning a conversion type to a conversion type identifier. [Figure 73] This block diagram shows the main configuration examples of the inverse primary transform section. [Figure 74] This is a flowchart illustrating an example of the inverse primary transformation process. [Figure 75] This figure shows an example of the LUT size required to store the transformation matrix. [Figure 76] This is a block diagram showing the main configuration examples of the transformation matrix derivation section. [Figure 77] This is a block diagram showing the main configuration examples of the transformation matrix derivation section. [Figure 78] This flowchart illustrates an example of the process for deriving transformation matrices. [Figure 79] This figure shows an example of assigning a conversion type to a conversion type identifier. [Figure 80] This is a block diagram showing the main configuration examples of the transformation matrix derivation section. [Figure 81] This is a block diagram showing the main configuration examples of the transformation matrix derivation section. [Figure 82] This figure shows an example of assigning a conversion type to a conversion type identifier. [Figure 83]This figure shows an example of assigning a conversion type to a conversion type identifier. [Figure 84] This figure illustrates the derivation of a transformation matrix using submatrices. [Figure 85] This figure shows a list of the main examples of transformation matrices derived from submatrices. [Figure 86] This figure shows an example of how the transformation matrix is ​​derived. [Figure 87] This figure shows an example of how the transformation matrix is ​​derived. [Figure 88] This figure shows an example of how the transformation matrix is ​​derived. [Figure 89] This figure shows an example of how the transformation matrix is ​​derived. [Figure 90] This figure shows an example of how the transformation matrix is ​​derived. [Figure 91] This figure shows an example of how the transformation matrix is ​​derived. [Figure 92] This is a block diagram showing the main configuration examples of the transformation matrix derivation section. [Figure 93] This is a block diagram showing the main configuration examples of the transformation matrix derivation section. [Figure 94] This flowchart illustrates an example of the process for deriving transformation matrices. [Figure 95] This figure shows an example of assigning a conversion type to a conversion type identifier. [Figure 96] This is a block diagram showing the main configuration examples of the transformation matrix derivation section. [Figure 97] This is a block diagram showing the main configuration examples of the transformation matrix derivation section. [Figure 98] This figure shows an example of the LUT size required to store the transformation matrix. [Figure 99] This is a block diagram showing the main configuration examples of the transformation matrix derivation section. [Figure 100] This flowchart illustrates an example of the process for deriving transformation matrices. [Figure 101] This figure shows an example of assigning a conversion type to a conversion type identifier. [Figure 102] This is a block diagram showing common computer configurations. [Modes for carrying out the invention]

[0015] The following describes the forms for implementing this disclosure (hereinafter referred to as embodiments). The explanation will be given in the following order. 1. Adaptive primary conversion

[0022] 2. First Embodiment (Deriving a Transformation Matrix from a Transformation Matrix)

[0047] 2-1. Common Concept 2-2. Example 1-1 2-3. Example 1-2 2-4. Example 1-3 2-5. Example 1-4 2-6. Examples 1-5 3. Second Embodiment (Predicted Residual Transformation)

[0567] 3-1. Common Concept 3-2. Example 2-1 3-3. Example 2-2 3-4. Example 2-3 3-5. Example 2-4 4. Third Embodiment (Deriving a Transformation Matrix from a Submatrix)

[0740] 4-1. Common Concept 5. Fourth Embodiment (Combination of Embodiments)

[0846] 5-1. Common Concept 6. Addendum

[0016] <1. Adaptive Primary Conversion> <Supporting literature and other resources for technical content and terminology> The scope disclosed in this technology includes not only the contents described in the examples, but also the contents described in the following non-patent documents that were publicly known at the time of filing.

[0017] Non-patent document 1: (mentioned above) Non-patent document 3: TELECOMMUNICATION STANDARDIZATION SECTOR OF ITU (International Telecommunication Union), "Advanced video coding for generic audiovisual services", H.264, 04 / 2017 Non-patent document 4: TELECOMMUNICATION STANDARDIZATION SECTOR OF ITU (International Telecommunication Union), "High efficiency video coding", H.265, 12 / 2016

[0018] In other words, the content described in the aforementioned non-patent documents also serves as a basis for determining the support requirement. For example, even if the Quad-Tree Block Structure described in Non-Patent Document 4 and the QTBT (Quad Tree Plus Binary Tree) Block Structure described in Non-Patent Document 1 are not directly described in the examples, they are within the scope of the disclosure of this technology and satisfy the support requirement of the claims. Similarly, even if technical terms such as Parsing, Syntax, and Semantics are not directly described in the examples, they are within the scope of the disclosure of this technology and satisfy the support requirement of the claims.

[0019] Furthermore, in this specification, unless otherwise specified, the term "block" (not a block indicating a processing unit) used in the description as a sub-region or processing unit of an image (picture) refers to any sub-region within a picture, and its size, shape, and characteristics are not limited. For example, "block" includes any sub-region (processing unit) such as TB (Transform Block), TU (Transform Unit), PB (Prediction Block), PU (Prediction Unit), SCU (Smallest Coding Unit), CU (Coding Unit), LCU (Largest Coding Unit), CTB (Coding Tree Block), CTU (Coding Tree Unit), transformation block, sub-block, macroblock, tile, or slice, as described in Non-Patent Documents 1, 3, and 4 above.

[0020] Furthermore, when specifying the size of such blocks, it is possible to specify the block size not only directly but also indirectly. For example, the block size may be specified using identification information that identifies the size. Alternatively, the block size may be specified by a ratio or difference with the size of a reference block (e.g., LCU or SCU). For example, when transmitting information that specifies the block size as a syntax element, the information that specifies the size indirectly as described above may be used. By doing so, the amount of information can be reduced, and encoding efficiency may be improved. In addition, this specification of block size also includes specifying a range of block sizes (e.g., specifying a range of acceptable block sizes).

[0021] <Adaptive Primary Conversion> In the test model described in Non-Patent Literature 1 (JEM4 (Joint Exploration Test Model 4)), for the luminance transformation block, an adaptive primary transformation (AMT (Adaptive Multiple Core Transforms)) is disclosed, which adaptively selects the primary transformation from multiple different one-dimensional orthogonal transformations for each horizontal primary transformation PThor (also referred to as primary horizontal transformation) and vertical primary transformation PTver (also referred to as primary vertical transformation).

[0022] Specifically, for the luminance conversion block, if the adaptive primary conversion flag apt_flag, which indicates whether or not to perform an adaptive primary conversion, is 0 (false), then, as shown in the table in Figure 1 (LUT_TrSetToTrTypIdx), for example, DCT (Discrete Cosine Transform)-II or DST (Discrete Sine Transform)-VII is uniquely determined by the mode information (TrSetIdx = 4).

[0023] When the adaptive primary transformation flag apt_flag is 1 (true) and the current CU (Coding Unit) containing the luminance transformation block to be processed is an intraCU, a transformation set TrSet containing orthogonal transformations that are candidates for the primary transformation in the horizontal (x-direction) and vertical (y-direction) directions is selected from the three transformation sets TrSet (TrSetIdx = 0,1,2) shown in Figure 1, as shown in the table. Note that DST-VII, DCT-VIII, etc. shown in Figure 1 indicate the type of orthogonal transformation, and the functions shown in the table in Figure 2 are used for each.

[0024] This conversion set TrSet is uniquely determined based on the correspondence table (intra-predicted mode information) between mode information and conversion sets shown in Figure 3. For example, for each conversion set TrSetH, TrSetV, a conversion set identifier TrSetIdx is set to specify the corresponding conversion set TrSet, as shown in equations (1) and (2) below.

[0025]

number

[0026] Here, TrSetH represents the transformation set of the primary horizontal transformation PThor, TrSetV represents the transformation set of the primary vertical transformation PTver, and the lookup table LUT_IntraModeToTrSet is the correspondence table in Figure 3. The first array of the lookup table LUT_IntraModeToTrSet[][] takes the intra prediction mode IntraMode as an argument, and the second array takes {H=0, V=1} as an argument.

[0027] For example, in the case of intra-prediction mode number 19 (IntraMode == 19), the transformation set TrSetH (also referred to as the primary horizontal transformation set) for the primary horizontal transformation PThor is selected, and the transformation set TrSetV (also referred to as the primary vertical transformation set) for the primary vertical transformation PTver is selected, and the transformation set TrSetIdx=2 (also referred to as the primary vertical transformation set) is selected, as shown in the table in Figure 1.

[0028] Furthermore, if the adaptive primary conversion flag apt_flag is 1 (true) and the current CU containing the luminance conversion block to be processed is an interCU, the primary horizontal conversion set TrSetH and the primary vertical conversion set TrSetV are assigned the interCU-specific conversion set InterTrSet(TrSetIdx = 3).

[0029] Next, for both the horizontal and vertical directions, the system selects which orthogonal transformation to apply from the selected transformation set TrSet, based on the corresponding primary horizontal transformation flag pt_hor_flag and primary vertical transformation flag pt_ver_flag.

[0030] For example, as shown in equations (3) and (4) below, the primary {horizontal, vertical} transformation set TrSet{H,V} and the primary {horizontal, vertical} transformation specification flag pt_{hor,ver}_flag are used as arguments to derive the transformation set definition table (LUT_TrSetToTrTypeIdx) shown in Figure 1.

[0031]

number

[0032] For example, if the intra-prediction mode number is 34 (IntraMode == 34) (i.e., the primary horizontal transformation set TraSetH is 0) and the primary horizontal transformation designation flag pt_hor_flag is 0, then from the transformation set definition table (LUT_TrSetToTrTypeIdx) in Figure 1, the value of the transformation type identifier TrTypeIdxH in equation (3) is 4, and the transformation type TrTypeH corresponding to the value of the transformation type identifier TrTypeIdxH is DST-VII, as shown in Figure 2. In other words, DST-VII is selected as the transformation type of the primary horizontal transformation PThor for the transformation set where the transformation set identifier TrSetIdx is 0. Also, if the primary horizontal transformation designation flag pt_hor_flag is 1, DCT-VIII is selected as the transformation type. Note that selecting a transformation type TrType includes selecting the transformation type specified by the transformation type identifier TrTypeIdx via the transformation type identifier TrTypeIdx.

[0033] Furthermore, the primary transformation identifier pt_idx is derived from the primary horizontal transformation specification flag pt_hor_flag and the primary vertical transformation specification flag pt_ver_flag based on the following equation (5). That is, the upper bit of the primary transformation identifier pt_idx corresponds to the value of the primary vertical transformation specification flag, and the lower bit corresponds to the value of the primary horizontal transformation specification flag.

[0034]

number

[0035] Encoding is performed by applying arithmetic coding to the bin sequence of the derived primary transformation identifier pt_idx to generate a bit sequence. The adaptive primary transformation flag apt_flag and the primary transformation identifier pt_idx are signaled in the luminance transformation block.

[0036] As described above, Non-Patent Document 1 proposes five one-dimensional orthogonal transformations as candidates for the primary transformation: DCT-II (DCT2), DST-VII (DST7), DCT-VIII (DCT8), DST-I (DST1), and DCT-V (DCT5). Furthermore, Non-Patent Document 2 adds two more one-dimensional orthogonal transformations, DST-IV (DST4) and IDT (Identity Transform: one-dimensional transformation skip), proposing a total of seven one-dimensional orthogonal transformations as candidates for the primary transformation.

[0037] In other words, in the case of Non-Patent Document 1, as shown in Figure 4, a one-dimensional orthogonal transform is stored in the LUT as a candidate for the primary transform. In the case of Non-Patent Document 2, in addition to those, DST-IV (DST4) and IDT are also stored in the LUT (see Figure 4).

[0038] In the case of HEVC (High Efficiency Video Coding), the size of the LUT (Look Up Table) required to hold the transformation matrix is ​​as shown in the table in Figure 5. In other words, the total size of the LUT is approximately 1.3KB. In contrast, in the case of the method described in Non-Patent Literature 1, for example, DCT2 requires that transformation matrices of sizes 2 / 4 / 8 / 16 / 32 / 64 / 128 points be held on the LUT. In addition, other one-dimensional transformations (DST7 / DST1 / DCT8) require that transformation matrices of sizes 4 / 8 / 16 / 32 / 64 points be held on the LUT. In this case, assuming that the bit precision of each coefficient of the transformation matrix is ​​10 bits, the size of the LUT required to hold the entire transformation matrix of the primary transformation is as shown in Figure 6A. In other words, the total size of the LUT in this case is approximately 53KB. That is, the size of the LUT in this case is increased by approximately 50 times compared to the case of HEVC.

[0039] Similarly, in the case of the method described in Non-Patent Document 2, the size of the LUT required to store the entire transformation matrix of the primary transformation is as shown in the table in Figure 6B. In other words, the total size of the LUT in this case is approximately 67KB. That is, the size of the LUT in this case is increased by about 60 times compared to the HEVC case.

[0040] When considering the hardware implementation of the primary transformation, the size of the LUT is reflected in the storage capacity (memory capacity). In other words, in the case of the methods described in Non-Patent Document 1 and Non-Patent Document 2, the circuit size (memory capacity required to hold the coefficients of the transformation matrix) could increase by about 50 to 60 times compared to the case of HEVC.

[0041] <2. First Embodiment> <2-1. Common Concepts> <Derivation of the transformation matrix> Therefore, the first transformation matrix is ​​used to derive the second transformation matrix, the derived second transformation matrix is ​​used to orthogonally transform the predicted residuals of the image, the coefficient data obtained from the orthogonal transformation of the predicted residuals is encoded, and a bitstream is generated.

[0042] For example, an image processing device may include a derivation unit that derives a second transformation matrix using a first transformation matrix, an orthogonal transformation unit that orthogonally transforms the predicted residuals of an image using the second transformation matrix derived by the derivation unit, and an encoding unit that encodes the coefficient data obtained by orthogonally transforming the predicted residuals by the orthogonal transformation unit and generates a bitstream.

[0043] By doing so, transformation matrices can be derived from other transformation matrices, which helps to suppress the increase in the number of transformation matrices required for orthogonal transformations, and thus suppresses the increase in the memory capacity required for orthogonal transformations.

[0044] Furthermore, the bitstream is decoded to obtain coefficient data obtained by orthogonally transforming the predicted residuals of the image. A second transformation matrix is ​​derived using the first transformation matrix, and the obtained coefficient data is then inversely orthogonally transformed using the derived second transformation matrix.

[0045] For example, an image processing device may include a decoding unit that decodes a bitstream and obtains coefficient data obtained by orthogonally transforming the predicted residuals of an image; a derivation unit that derives a second transformation matrix using a first transformation matrix; and an inverse orthogonal transformation unit that performs an inverse orthogonal transformation on the coefficient data obtained by the decoding unit using the second transformation matrix derived by the derivation unit.

[0046] By doing so, transformation matrices can be derived from other transformation matrices, which helps to suppress the increase in the number of transformation matrices required for inverse orthogonal transformations, and thus suppresses the increase in the memory capacity required for inverse orthogonal transformations.

[0047] <Characteristics of transformation matrices> One of the main roles of a transformation matrix is ​​to bias the signal towards the DC component, particularly the low-order (especially 0th-order) frequency components, and the way these frequency components are collected is a crucial characteristic. How the frequency components are biased depends on the waveform components of the low-order (especially 0th-order) basis vectors (row vectors). In other words, transformation matrices with similar waveform components of these basis vectors can be expected to have similar performance for orthogonal and inverse orthogonal transformations (the way the frequency components are biased is similar).

[0048] Therefore, we focus on the waveforms of the low-order (especially 0th order) basis vectors (row vectors) of the transformation matrix. For example, in the transformation matrix 30 in Figure 7, let's assume that the waveforms (trends of the values ​​of each element) of the low-order (especially 0th order) row vectors within frame 31 are shown as in graph 32.

[0049] Graph 32 shows the trend of lower frequency components on the left side of the graph and the trend of higher frequency components on the right side. Furthermore, the upper part of Graph 32 indicates a larger value, and the lower part indicates a smaller value. The vertical center of Graph 32 represents 0, values ​​above the center represent positive values, and values ​​below the center represent negative values.

[0050] Waveform 32A in Graph 32 shows the waveform of the zero-order row vector of the transformation matrix 30. As shown in waveform 32A, in this case, the zero-order row vector of the transformation matrix 30 tends to increase in value from low-frequency components to high-frequency components.

[0051] Furthermore, for example, in the transformation matrix 30 of Figure 7, suppose the waveform (trend of the values ​​of each element) of the low-order (especially 0th-order) column vectors within frame 33 is shown as in graph 34. In graph 34, the upper part of the graph shows the values ​​(trends) of elements with lower frequency components, and the lower part of the graph shows the values ​​(trends) of elements with higher frequency components. Also, in graph 34, the left side of the graph shows a larger value, and the right side shows a smaller value. Note that the center of graph 34 in the horizontal direction represents 0, the left side of the center represents a positive value, and the right side of the center represents a negative value.

[0052] Waveform 34A in Graph 34 shows the waveform of the zero-order column vector of the transformation matrix 30. As shown in waveform 34A, in this case, the zero-order column vector of the transformation matrix 30 has a peak in the intermediate frequency components (i.e., the value tends to decrease towards lower frequency components on the low-frequency side and towards higher frequency components on the high-frequency side).

[0053] In this specification, the waveform of the zero-order column vector of the transformation matrix 30 may also be represented in a transposed state as shown in Graph 35. The structure of Graph 35 is the same as that of Graph 32. Waveform 35A is equivalent to waveform 34A.

[0054] As mentioned above, transformation matrices with similar waveforms for low-order (especially zero-order) basis vectors (row vectors) tend to have similar performance. In other words, a transformation matrix can be replaced by another transformation matrix with similar waveforms for low-order (especially zero-order) basis vectors (row vectors). Therefore, by utilizing this, the increase in the number of transformation matrices stored in the LUT can be suppressed.

[0055] Focusing on the transformation types described in Non-Patent Documents 1 and 2, the waveforms of the zero-order row vectors and zero-order column vectors of the transformation matrices of these transformation types can be classified into four types. An example is shown in Figure 8.

[0056] The first type is the flat type. This flat type is a waveform type in which the value is approximately uniform across all frequency components. The second type is the increasing type. This increasing type is a waveform type in which the value tends to increase from low-frequency components to high-frequency components. The third type is the decreasing type. This decreasing type is a waveform type in which the value tends to decrease from low-frequency components to high-frequency components. The fourth type is the bell type. This bell type is a waveform type in which there tends to be a peak (maximum value) in the middle. In other words, in the case of this bell type, the waveform tends to decrease in value towards lower frequencies on the low-frequency side, and also decreases towards higher frequencies on the high-frequency side.

[0057] Note that these types only indicate the approximate shape of the waveform and do not need to be exactly the same. For example, in the case of an increasing waveform, the waveform as a whole only needs to tend to increase in value from the low-frequency side to the high-frequency side, and does not need to be strictly monotonically increasing from the low-frequency side to the high-frequency side.

[0058] Similarly, in the case of a decreasing waveform, the waveform as a whole only needs to tend to decrease in value from the low-frequency side to the high-frequency side, and does not need to be strictly monotonically decreasing from the low-frequency side to the high-frequency side.

[0059] Similarly, in the case of a bell-shaped waveform, the waveform as a whole should have a peak (maximum value) near the center, and the values ​​should tend to decrease away from the peak on both sides; it is not necessary for the values ​​to decrease monotonically away from the peak on both sides. Furthermore, the peak does not have to be formed by a single component; for example, the approximate position and value of the peak can be determined from multiple components. Also, the position of the peak does not have to be exactly in the center.

[0060] Similarly, in the case of a flat waveform, the waveform only needs to have approximately uniform values ​​overall; it does not need to be strictly flat. In other words, some variation in values ​​is acceptable. To put it another way, any waveform that cannot be classified into the other three types can be considered a flat waveform.

[0061] The above waveform classification is merely an example and is not limited to the examples given. In other words, waveforms may be classified into types other than those described above, and the number of classification types is arbitrary and not limited to the four types mentioned above. Note that this classification is performed for the convenience of explaining this technology and is not performed as an actual processing step.

[0062] According to this classification, as shown in Figure 8, the waveform of the zero-order row vector of the DCT2 transformation matrix is ​​classified as flat, and the waveform of the zero-order column vector is classified as decreasing. Similarly, the waveform of the zero-order row vector of the DST7 transformation matrix is ​​classified as increasing, and the waveform of the zero-order column vector is classified as bell-shaped. Furthermore, the waveform of the zero-order row vector of the DCT8 transformation matrix is ​​classified as decreasing, and the waveform of the zero-order column vector is also decreasing. Additionally, the waveform of the zero-order row vector of the DCT5 transformation matrix is ​​classified as flat, and the waveform of the zero-order column vector is also flat. Finally, the waveform of the zero-order row vector of the DST4 transformation matrix is ​​classified as increasing, and the waveform of the zero-order column vector is also increasing.

[0063] As described above, a transformation matrix can be substituted by another transformation matrix whose waveform of a zero-order row vector is similar. In other words, transformation types whose waveforms of zero-order row vectors are isomorphic can substitute for each other's transformation matrices.

[0064] In other words, when the derivation unit derives the second transformation matrix using the first transformation matrix described above, it may derive a second transformation matrix in which the lowest-order row vector has a waveform of the desired type. By doing so, it is possible to suppress the increase in the number of transformation matrices prepared for orthogonal and inverse orthogonal transformations, and to suppress the increase in the memory capacity required for orthogonal and inverse orthogonal transformations.

[0065] For example, the derivation unit may use the first transformation matrix to derive a second transformation matrix in which the lowest-order row vector has a flat waveform. In this way, the derived second transformation matrix can substitute for the transformation matrix in which the lowest-order row vector has a flat waveform. Alternatively, for example, the derivation unit may use the first transformation matrix to derive a second transformation matrix in which the lowest-order row vector has an increasing waveform. In this way, the derived second transformation matrix can substitute for the transformation matrix in which the lowest-order row vector has an increasing waveform.

[0066] Furthermore, for example, the derivation unit may use the first transformation matrix to derive a second transformation matrix in which the lowest-order row vector has a decreasing waveform. In this way, the derived second transformation matrix can substitute for the transformation matrix in which the lowest-order row vector has a decreasing waveform. Furthermore, for example, the derivation unit may use the first transformation matrix to derive a second transformation matrix in which the lowest-order row vector has a bell-shaped waveform. In this way, the derived second transformation matrix can substitute for the transformation matrix in which the lowest-order row vector has a bell-shaped waveform.

[0067] For example, in Figure 8, DST7, DST4, DST8, and DST3 all have increasing waveforms for their zero-order row vectors, so they can substitute for each other's transformation matrices. In other words, even transformation matrices of different transformation types can be substituted for each other.

[0068] In other words, when the derivation unit derives the second transformation matrix using the first transformation matrix described above, it may derive a second transformation matrix of a different transformation type than the first transformation matrix. By doing so, it is possible to suppress the increase in the number of transformation types prepared for orthogonal and inverse orthogonal transformations, and thus suppress the increase in the memory capacity required for orthogonal and inverse orthogonal transformations.

[0069] Furthermore, during the derivation process, the derivation unit may derive a second transformation matrix that has the same number of rows and columns as the first transformation matrix. When the number of rows and columns is changed, the waveform type may change unintentionally. Therefore, by making the number of rows and columns the same as the first transformation matrix, the possibility of unintended changes in the waveform type can be suppressed, and the derivation of the second transformation matrix can be made easier.

[0070] Furthermore, matrices allow for easy manipulation of their elements. These operations include, for example, rearranging elements. More specifically, a matrix can have its element order flipped (reversed) in a predetermined direction, or its elements transposed to swap rows and columns. Note that transposition is equivalent to flipping (reversing) the matrix around the diagonal connecting the top-left and bottom-right ends. In other words, transposition can be considered a part of flipping. It is also easy to reverse the sign of each element (from positive to negative, or negative to positive).

[0071] By utilizing such operations, the waveform (type) of a zero-order row vector can be intentionally changed. For example, if a matrix with an increasing waveform for its zero-order row vectors is flipped row by row, the waveform of that zero-order row vector will change to a decreasing waveform. Conversely, if a matrix with a decreasing waveform for its zero-order row vectors is flipped row by row, the waveform of that zero-order row vector will change to an increasing waveform.

[0072] In other words, as shown in Figure 8, by flipping the transformation matrix of an increasing transformation type (e.g., DST7, DST4, DST8, DST3) in the row direction, a transformation matrix that can substitute for a decreasing transformation type (e.g., DCT7, DCT4, DCT8, DCT3) of a decreasing transformation type of a zero-order row vector can be obtained.

[0073] Furthermore, for example, transposing a matrix swaps the waveform types of the zeroth row vectors and the zeroth column vectors. In other words, transposition makes the waveform of the zeroth row vectors of the matrix the same type as the waveform of the zeroth column vectors of the matrix before transposition.

[0074] For example, as shown in Figure 9, by transposing the transformation matrix of DCT2 (DCT6), where the waveform of the zero-order column vector is decreasing, a transformation matrix that can substitute for the transformation matrix of transformation types (DCT3, DCT7, DCT4, DCT8) where the waveform of the zero-order row vector is decreasing can be obtained. Also, for example, as shown in Figure 9, by transposing the transformation matrix of DST7 (DST3), where the waveform of the zero-order column vector is bell-shaped, a transformation matrix that can substitute for the transformation matrix of transformation types (DST2, DST6, DST1, DST5) where the waveform of the zero-order row vector is bell-shaped can be obtained.

[0075] In other words, the derivation unit may derive a second transformation matrix by performing operations on the elements of such a first transformation matrix. Furthermore, these operations on the elements may include rearranging (changing the order of) the elements, as described above. By doing so, the waveform type can be intentionally changed, and a wider variety of second transformation matrices can be derived from the first transformation matrix. Therefore, it is possible to suppress an increase in the number of transformation types prepared for orthogonal and inverse orthogonal transformations, and to suppress an increase in the memory capacity required for orthogonal and inverse orthogonal transformations.

[0076] Of course, the derivation unit may perform such operations multiple times to derive the second transformation matrix. For example, operations such as flipping and transposing can be combined arbitrarily. Also, the same operation may be repeated multiple times. In this way, a wider variety of second transformation matrices can be derived from the first transformation matrix.

[0077] As mentioned above, the transformation matrices used for orthogonal and inverse orthogonal transformations are stored in the Lookup Table (LUT). Therefore, the derivation unit may derive the second transformation matrix using the first transformation matrix stored in this Lookup Table (LUT). By doing so, the increase in the size of the LUT can be suppressed. Consequently, the increase in the memory capacity required for orthogonal and inverse orthogonal transformations can be suppressed.

[0078] <Derivation Example> Figure 10 shows a list of examples of derivations of transformation matrices involving the operations described above. Note that the transformation matrix used in the derivation (the first transformation matrix) is the base transformation matrix T. base It is also called the first transformation type. The transformation type of the base transformation matrix is ​​also called the base transformation type or the first transformation type. Furthermore, the transformation type of the derived transformation matrix (the second transformation matrix) is also called the second transformation type.

[0079] In the table shown in Figure 10, the derivation of the first example from the top, excluding the top row of item names, focuses on the similarity between the waveform of the lowest-order row vector of the first transformation type and the waveform of the lowest-order row vector of the transformation type of the alternative transformation matrix.

[0080] In this case, the derivation unit flips the first transformation matrix to derive the second transformation matrix. More specifically, the derivation unit uses the transformation matrix of DST7 as the base transformation matrix T base Then, we flip it in the row direction to derive the transformation matrix FlipDST7. Since the waveform of the 0th row vector of the DST7 transformation matrix is ​​increasing, the waveform of the 0th row vector of the derived FlipDST7 transformation matrix is ​​decreasing. Therefore, this FlipDST7 transformation matrix can be used to substitute for the DCT8 transformation matrix, in which the waveform of the 0th row vector is decreasing.

[0081] By applying this derivation, it becomes unnecessary to prepare DCT8 transformation matrices as candidates for transformation matrices used in orthogonal and inverse orthogonal transformations. In other words, the number of intrinsic transformation types can be reduced, which means that the increase in LUT size can be suppressed. Furthermore, by performing orthogonal and inverse orthogonal transformations using the derived second transformation type (FlipDST7) transformation matrix, coding efficiency comparable to that obtained when using the DCT8 transformation matrix for orthogonal and inverse orthogonal transformations can be achieved. In this case, the second transformation matrix (a decreasing alternative transformation matrix) can be derived in a single operation.

[0082] Furthermore, the derivation of the example one row below (the second row from the top) focuses on the similarity between the waveform of the lowest-order column vector of the first transformation type and the waveform of the lowest-order row vector of the transformation type of the alternative transformation matrix.

[0083] In this case, the derivation unit transposes the first transformation matrix to derive the second transformation matrix. More specifically, the derivation unit uses the transformation matrix of DST7 as the base transformation matrix T base We then transpose it to derive the transformation matrix TrDST7. Since the waveform of the 0th order column vector of the DST7 transformation matrix is ​​bell-shaped, the waveform of the 0th order row vector of the derived TrDST7 transformation matrix is ​​also bell-shaped. Therefore, this TrDST7 transformation matrix can be used to substitute for the DST1 transformation matrix, whose 0th order row vector waveform is bell-shaped.

[0084] By applying this derivation, it becomes unnecessary to prepare the DST1 transformation matrix as a candidate for the transformation matrix used in orthogonal and inverse orthogonal transformations. In other words, the number of intrinsic transformation types can be reduced, which means that the increase in LUT size can be suppressed. Furthermore, by performing orthogonal and inverse orthogonal transformations using the derived second transformation type (TrDST7) transformation matrix, coding efficiency comparable to that obtained when using the DST1 transformation matrix for orthogonal and inverse orthogonal transformations can be achieved. In this case, the second transformation matrix (the bell-shaped alternative transformation matrix) can be derived in a single operation.

[0085] Furthermore, the derivation of the example one level below (the third level from the top) focuses on the characteristics between pairs of DCT / DST. More specifically, it focuses on the fact that between pairs of DCT / DST (for example, DST7 and DCT8), even-numbered row vectors are axisymmetric, and odd-numbered row vectors are point-symmetric.

[0086] In this case, the derivation unit flips the first transformation matrix and reverses the sign of the odd-numbered row vectors in the flipped first transformation matrix to derive the second transformation matrix. More specifically, the derivation unit uses the transformation matrix of DST7 as the base transformation matrix T baseThen, we flip it in the row direction and further invert the sign of the odd-order row vectors to derive the DCT8 transformation matrix. Note that sign inversion of row vectors can be done by transforming the most significant bit of each element of the row vector. Naturally, the derived DCT8 transformation matrix can be used to replace the DST8 transformation matrix for the waveform of the zero-order row vector, which is a decreasing type.

[0087] By applying this derivation, it becomes unnecessary to prepare DCT8 transformation matrices as candidates for transformation matrices used in orthogonal and inverse orthogonal transformations. In other words, the number of intrinsic transformation types can be reduced, which means that the increase in LUT size can be suppressed. Furthermore, by performing orthogonal and inverse orthogonal transformations using the derived second transformation type (DCT8) transformation matrix, the same coding efficiency as when using the DCT8 transformation matrix for orthogonal and inverse orthogonal transformations can be obtained. Also, in this case, the second transformation matrix (the paired alternative transformation matrix) can be derived in just two operations.

[0088] Furthermore, the derivation of the example one row below (the fourth row from the top) focuses on the similarity between the waveform of the lowest-order row vector of the first transformation type and the waveform of the lowest-order row vector of the transformation type of the substitute transformation matrix, similar to the case of the first row from the top.

[0089] In this case, the derivation unit flips the first transformation matrix to derive the second transformation matrix. More specifically, the derivation unit uses the transformation matrix of DCT8 as the base transformation matrix T base Then, by flipping it in the row direction, we derive the transformation matrix of FlipDCT8. Since the waveform of the 0th row vector of the DCT8 transformation matrix is ​​decreasing, the waveform of the 0th row vector of the derived FlipDCT8 transformation matrix is ​​increasing. Therefore, this FlipDCT8 transformation matrix can be used to replace the DST7 transformation matrix in which the waveform of the 0th row vector is increasing.

[0090] By applying this derivation, it becomes unnecessary to prepare the DST7 transformation matrix as a candidate for the transformation matrix used for orthogonal and inverse orthogonal transformations. In other words, the number of intrinsic transformation types can be reduced, which means that the increase in LUT size can be suppressed. Furthermore, by performing orthogonal and inverse orthogonal transformations using the derived second transformation type (FlipDCT8) transformation matrix, coding efficiency comparable to that obtained when using the DST7 transformation matrix for orthogonal and inverse orthogonal transformations can be achieved. In this case, the second transformation matrix (a decreasing alternative transformation matrix) can be derived in a single operation.

[0091] Furthermore, the derivation of the example one row below (the fifth row from the top) focuses on the similarity between the waveform of the highest-order column vector of the first transformation type and the waveform of the lowest-order row vector of the transformation type of the substitute transformation matrix.

[0092] In this case, the derivation unit flips the first transformation matrix, transposes the first transformation matrix after the flip, and derives the second transformation matrix. More specifically, the derivation unit uses the transformation matrix of DCT8 as the base transformation matrix T base Then, we flip it in the row direction and transpose it to derive the transformation matrix TrFlipDCT8. Since the waveform of the highest-order column vector of the DCT8 transformation matrix is ​​bell-shaped, the waveform of the 0th-order row vector of the derived TrFlipDCT8 transformation matrix is ​​also bell-shaped. Therefore, this TrFlipDCT8 transformation matrix can be used to replace the DST1 transformation matrix in which the waveform of the 0th-order row vector is bell-shaped.

[0093] By applying this derivation, it becomes unnecessary to prepare the DST1 transformation matrix as a candidate for the transformation matrix used in orthogonal and inverse orthogonal transformations. In other words, the number of intrinsic transformation types can be reduced, which means that the increase in LUT size can be suppressed. Furthermore, by performing orthogonal and inverse orthogonal transformations using the derived second transformation type (TrFlipDCT8) transformation matrix, coding efficiency comparable to that obtained when using the DST1 transformation matrix for orthogonal and inverse orthogonal transformations can be achieved. In this case, the second transformation matrix (a bell-shaped alternative transformation matrix) can be derived in just two operations.

[0094] Furthermore, the derivation of the example one level below (the sixth level from the top) focuses on the characteristics between pairs of DCT / DST, similar to the case of the third level from the top. More specifically, it focuses on the fact that between pairs of DCT / DST (for example, DCT8 and DST7), even-numbered row vectors are axisymmetric, and odd-numbered row vectors are point-symmetric.

[0095] In this case, the derivation unit flips the first transformation matrix and reverses the sign of the odd-numbered row vectors in the flipped first transformation matrix to derive the second transformation matrix. More specifically, the derivation unit uses the transformation matrix of DCT8 as the base transformation matrix T base Then, by flipping it in the row direction and further inverting the sign of the odd-order row vectors, we derive the DST7 transformation matrix. Naturally, the derived DST7 transformation matrix can substitute for the DST7 transformation matrix where the waveform of the zero-order row vector is increasing.

[0096] By applying this derivation, it becomes unnecessary to prepare a DST7 transformation matrix as a candidate for the transformation matrix used in orthogonal and inverse orthogonal transformations. In other words, the number of intrinsic transformation types can be reduced, which means that the increase in LUT size can be suppressed. Furthermore, by performing orthogonal and inverse orthogonal transformations using the derived second transformation type (DST7) transformation matrix, the same coding efficiency as when using the DST7 transformation matrix for orthogonal and inverse orthogonal transformations can be obtained. Also, in this case, the second transformation matrix (the paired alternative transformation matrix) can be derived in just two operations.

[0097] Furthermore, the derivation of the example one row below (the seventh row from the top) focuses on the similarity between the waveform of the lowest-order column vector of the first transformation type and the waveform of the lowest-order row vector of the transformation type of the alternative transformation matrix.

[0098] In this case, the derivation unit transposes the first transformation matrix to derive the second transformation matrix. More specifically, the derivation unit uses the transformation matrix of DCT2 as the base transformation matrix T baseand transpose it to derive the transformation matrix of DCT3. Since the waveform of the 0th column vector of the transformation matrix of DCT2 is of the decreasing type, the waveform of the 0th row vector of the derived transformation matrix of DCT3 is of the decreasing type. Therefore, with this transformation matrix of DCT3, the transformation matrix of DCT8 with the waveform of the 0th row vector being of the decreasing type can be substituted.

[0099] By applying such a derivation, there is no need to prepare the transformation matrix of DCT8 as a candidate for the transformation matrix used in orthogonal transformation and inverse orthogonal transformation. That is, the number of specific transformation types can be reduced. Namely, an increase in the LUT size can be suppressed. Also, by performing orthogonal transformation and inverse orthogonal transformation using the derived transformation matrix of the second transformation type (DCT3), the same coding efficiency as when using the transformation matrix of DCT8 in orthogonal transformation and inverse orthogonal transformation can be obtained. Moreover, in this case, the second transformation matrix (the alternative transformation matrix of the decreasing type) can be derived with one operation.

[0100] Also, the derivation of the example one level below (the 8th level from the top) focuses on the similarity between the waveform of the highest-order column vector of the first transformation type and the waveform of the lowest-order row vector of the transformation type of the alternative transformation matrix.

[0101] In this case, the derivation unit transposes the first transformation matrix, flips the transposed first transformation matrix, and derives the second transformation matrix. More specifically, the derivation unit uses the transformation matrix of DCT2 as the base transformation matrix T base and transposes it, and further flips it in the row direction to derive the transformation matrix of FlipDCT3. Since the waveform of the highest-order column vector of the transformation matrix of DCT2 is of the increasing type, the waveform of the 0th row vector of the derived transformation matrix of FlipDCT3 is of the increasing type. Therefore, with this transformation matrix of FlipDCT3, the transformation matrix of DST7 with the waveform of the 0th row vector being of the increasing type can be substituted.

[0102] By applying this derivation, it becomes unnecessary to prepare the DST7 transformation matrix as a candidate for the transformation matrix used for orthogonal and inverse orthogonal transformations. In other words, the number of intrinsic transformation types can be reduced, which means that the increase in LUT size can be suppressed. Furthermore, by performing orthogonal and inverse orthogonal transformations using the derived second transformation type (FlipDCT3) transformation matrix, coding efficiency comparable to that obtained when using the DST7 transformation matrix for orthogonal and inverse orthogonal transformations can be achieved. In this case, the second transformation matrix (an augmented alternative transformation matrix) can be derived in just two operations.

[0103] Each of the above derivation examples may be implemented individually, or multiple derivation examples may be combined.

[0104] <Image encoding device> Next, we will describe the configuration for deriving the transformation matrix as described above. Figure 11 is a block diagram showing an example of the configuration of an image encoding device, which is one embodiment of an image processing device to which this technology is applied. The image encoding device 100 shown in Figure 11 is a device for encoding image data of moving images. For example, the image encoding device 100 implements the technology described in Non-Patent Document 1, Non-Patent Document 3, or Non-Patent Document 4, and encodes image data of moving images in a manner compliant with the standards described in any of those documents.

[0105] Note that Figure 11 shows the main components such as the processing unit and data flow, and does not necessarily represent everything. In other words, there may be processing units in the image encoding device 100 that are not shown as blocks in Figure 11, and there may be processes and data flows that are not shown as arrows or other symbols in Figure 11. The same applies to other figures that explain the processing units within the image encoding device 100.

[0106] As shown in FIG. 11, the image encoding apparatus 100 includes a control unit 101, a rearrangement buffer 111, an arithmetic unit 112, an orthogonal transformation unit 113, a quantization unit 114, an encoding unit 115, an accumulation buffer 116, an inverse quantization unit 117, an inverse orthogonal transformation unit 118, an arithmetic unit 119, an in-loop filter unit 120, a frame memory 121, a prediction unit 122, and a rate control unit 123.

[0107] <Control Unit> Based on the external or the block size of a preset processing unit, the control unit 101 divides the moving image data held by the rearrangement buffer 111 into blocks (such as CU, PU, transformation blocks, etc.) of the processing unit. Also, the control unit 101 determines encoding parameters (header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, filter information Finfo, etc.) to be supplied to each block, for example, based on RDO (Rate-Distortion Optimization).

[0108] Details of these encoding parameters will be described later. When the control unit 101 determines the encoding parameters as described above, it supplies them to each block. Specifically, it is as follows.

[0109] The header information Hinfo is supplied to each block. The prediction mode information Pinfo is supplied to the encoding unit 115 and the prediction unit 122. The transformation information Tinfo is supplied to the encoding unit 115, the orthogonal transformation unit 113, the quantization unit 114, the inverse quantization unit 117, and the inverse orthogonal transformation unit 118. The filter information Finfo is supplied to the in-loop filter unit 120.

[0110] <Rearrangement Buffer> In the image encoding device 100, each field (input image) of moving image data is input in its playback order (display order). The rearrangement buffer 111 acquires and holds (stores) each input image in its playback order (display order). Based on the control of the control unit 101, the rearrangement buffer 111 rearranges the input images in the encoding order (decoding order) or divides them into blocks of a processing unit. The rearrangement buffer 111 supplies each processed input image to the arithmetic unit 112. Also, the rearrangement buffer 111 supplies each input image (original image) to the prediction unit 122 and the in-loop filter unit 120 as well.

[0111] <Arithmetic unit> The arithmetic unit 112 takes as inputs the image I corresponding to a block of the processing unit and the prediction image P supplied from the prediction unit 122, subtracts the prediction image P from the image I as shown in the following formula (6) to derive a prediction residue D, and supplies it to the orthogonal transformation unit 113.

[0112]

Equation

[0113] <Orthogonal transformation unit> The orthogonal transformation unit 113 takes as inputs the prediction residue D supplied from the arithmetic unit 112 and the transformation information Tinfo supplied from the control unit 101, performs an orthogonal transformation on the prediction residue D based on the transformation information Tinfo, and derives transformation coefficients Coeff. The orthogonal transformation unit 113 supplies the obtained transformation coefficients Coeff to the quantization unit 114.

[0114] <Quantization unit> The quantization unit 114 takes as inputs the transformation coefficients Coeff supplied from the orthogonal transformation unit 113 and the transformation information Tinfo supplied from the control unit 101, and scales (quantizes) the transformation coefficients Coeff based on the transformation information Tinfo. Note that the rate of this quantization is controlled by the rate control unit 123. The quantization unit 114 supplies the quantized transformation coefficients, that is, the quantization transformation coefficient level level, obtained by such quantization to the encoding unit 115 and the inverse quantization unit 117.

[0115] <Encoding part> The encoding unit 115 receives the quantization conversion coefficient level level supplied from the quantization unit 114, various encoding parameters supplied from the control unit 101 (header information Hinfo, prediction mode information Pinfo, conversion information Tinfo, filter information Finfo, etc.), filter information such as filter coefficients supplied from the in-loop filter unit 120, and information on the optimal prediction mode supplied from the prediction unit 122 as input. The encoding unit 115 performs variable-length encoding (for example, arithmetic encoding) on ​​the quantization conversion coefficient level level to generate a bit sequence (encoded data).

[0116] Furthermore, the encoding unit 115 derives residual information Rinfo from the quantization conversion coefficient level level, encodes the residual information Rinfo, and generates a bit sequence.

[0117] Furthermore, the encoding unit 115 includes information about the filter supplied from the in-loop filter unit 120 in the filter information Finfo, and information about the optimal prediction mode supplied from the prediction unit 122 in the prediction mode information Pinfo. Then, the encoding unit 115 encodes the various encoding parameters mentioned above (header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, filter information Finfo, etc.) and generates a bit sequence.

[0118] Furthermore, the encoding unit 115 multiplexes the bit sequences of the various information generated as described above to generate encoded data. The encoding unit 115 supplies the encoded data to the storage buffer 116.

[0119] <Storage buffer> The storage buffer 116 temporarily holds the encoded data obtained by the encoding unit 115. At a predetermined timing, the storage buffer 116 outputs the stored encoded data to the outside of the image encoding device 100, for example, as a bitstream. For example, this encoded data is transmitted to the decoding side via any recording medium, any transmission medium, any information processing device, etc. In other words, the storage buffer 116 also functions as a transmission unit that transmits encoded data (bitstream).

[0120] <Inverse quantization part> The inverse quantization unit 117 performs processing related to inverse quantization. For example, the inverse quantization unit 117 takes the quantization conversion coefficient level level supplied from the quantization unit 114 and the conversion information Tinfo supplied from the control unit 101 as input, and scales (inverse quantizes) the value of the quantization conversion coefficient level level based on the conversion information Tinfo. This inverse quantization is the reverse process of the quantization performed in the quantization unit 114. The inverse quantization unit 117 supplies the conversion coefficient Coeff_IQ obtained by such inverse quantization to the inverse orthogonal transformation unit 118.

[0121] <Inverse orthogonal transformation section> The inverse orthogonal transform unit 118 performs processing related to the inverse orthogonal transform. For example, the inverse orthogonal transform unit 118 takes the transformation coefficient Coeff_IQ supplied from the inverse quantization unit 117 and the transformation information Tinfo supplied from the control unit 101 as input, and performs an inverse orthogonal transform on the transformation coefficient Coeff_IQ based on the transformation information Tinfo to derive the predicted residual D'. This inverse orthogonal transform is the inverse process of the orthogonal transform performed in the orthogonal transform unit 113. The inverse orthogonal transform unit 118 supplies the predicted residual D' obtained by such an inverse orthogonal transform to the calculation unit 119. Since the inverse orthogonal transform unit 118 is the same as the inverse orthogonal transform unit on the decoding side (described later), the explanation given for the decoding side (described later) can be applied to the inverse orthogonal transform unit 118.

[0122] <Arithmetic section> The calculation unit 119 receives the predicted residual D' supplied from the inverse orthogonal transform unit 118 and the predicted image P supplied from the prediction unit 122 as input. The calculation unit 119 adds the predicted residual D' and the predicted image P corresponding to the predicted residual D' to obtain the local decoded image R local The calculation unit 119 derives the derived local decoded image R local This is supplied to the in-loop filter unit 120 and the frame memory 121.

[0123] <In-loop filter section> The in-loop filter unit 120 performs processing related to in-loop filtering. For example, the in-loop filter unit 120 processes the locally decoded image R supplied from the calculation unit 119. local The in-loop filter unit 120 receives filter information Finfo supplied from the control unit 101 and the input image (original image) supplied from the sorting buffer 111 as input. The information input to the in-loop filter unit 120 is arbitrary, and information other than this may be input. For example, if necessary, information such as prediction mode, motion information, code amount target value, quantization parameter QP, picture type, and block (CU, CTU, etc.) may be input to the in-loop filter unit 120.

[0124] The in-loop filter unit 120 processes the local decoded image R based on its filter information Finfo. local Appropriate filtering is performed on the input image. The in-loop filter unit 120 also uses the input image (original image) and other input information in its filtering process as needed.

[0125] For example, the in-loop filter section 120 applies four in-loop filters in this order, as described in Non-Patent Document 1: a bilateral filter, a deblocking filter (DBF), an adaptive offset filter (SAO), and an adaptive loop filter (ALF). Note that the order in which these filters are applied is arbitrary and can be selected as appropriate.

[0126] Of course, the filtering process performed by the in-loop filter unit 120 is arbitrary and is not limited to the above example. For example, the in-loop filter unit 120 may apply a Wiener filter or the like.

[0127] The in-loop filter unit 120 supplies the filtered local decoded image R local to the frame memory 121. When transmitting information related to the filter, such as filter coefficients, to the decoding side, the in-loop filter unit 120 supplies the information related to the filter to the encoding unit 115.

[0128] <Frame Memory> The frame memory 121 performs processing related to storing data related to the image. For example, the frame memory 121 receives the local decoded image R local supplied from the arithmetic unit 119 and the filtered local decoded image R local supplied from the in-loop filter unit 120, and holds (stores) it. Further, the frame memory 121 reconstructs and holds the decoded image R for each picture using the local decoded image R local (stores it in a buffer in the frame memory 121). The frame memory 121 supplies the decoded image R (or a part thereof) to the prediction unit 122 in response to a request from the prediction unit 122.

[0129] <Prediction Unit> The prediction unit 122 performs processing related to the generation of a predicted image. For example, the prediction unit 122 takes prediction mode information Pinfo supplied from the control unit 101, an input image (original image) supplied from the sorting buffer 111, and a decoded image R (or a part thereof) read from the frame memory 121 as input. The prediction unit 122 uses the prediction mode information Pinfo and the input image (original image) to perform prediction processing such as interpretation or intraprediction, makes predictions by referencing the decoded image R as a reference image, performs motion compensation processing based on the prediction results, and generates a predicted image P. The prediction unit 122 supplies the generated predicted image P to the calculation unit 112 and the calculation unit 119. The prediction unit 122 also supplies information about the prediction mode selected by the above processing, i.e., the optimal prediction mode, to the encoding unit 115 as needed.

[0130] <Rate Control Section> The rate control unit 123 performs processing related to rate control. For example, the rate control unit 123 controls the rate of the quantization operation of the quantization unit 114 based on the amount of encoded data stored in the storage buffer 116, so as to prevent overflow or underflow.

[0131] In the image encoding device 100 configured as described above, the orthogonal transformation unit 113 performs processing that applies the aforementioned technology as both a derivation unit and an orthogonal transformation unit. The encoding unit 115 also performs processing that applies the aforementioned technology as an encoding unit. Furthermore, the inverse orthogonal transformation unit 118 performs processing that applies the aforementioned technology as both an inverse orthogonal transformation unit and a derivation unit. Therefore, the image encoding device 100 can suppress the increase in memory capacity required for orthogonal and inverse orthogonal transformations.

[0132] <Details of the orthogonal transformation unit> Figure 12 is a block diagram showing an example of the main configuration of the orthogonal conversion unit 113 in Figure 11. As shown in Figure 12, the orthogonal conversion unit 113 includes a switch 151, a primary conversion unit 152, and a secondary conversion unit 153.

[0133] Switch 151 takes the predicted residual D and the conversion skip flag ts_flag[compID] corresponding to the component identifier compID as input. If the value of the conversion skip flag ts_flag[compID] is NO_TS (=0) (meaning no conversion skip is applied), it supplies the predicted residual D to the primary conversion unit 152. If the value of the conversion skip flag ts_flag[compID] is 2D_TS (=1) (indicating the application of a two-dimensional conversion skip), it skips the primary conversion unit 152 and the secondary conversion unit 153, and outputs the predicted residual D as a conversion coefficient Coeff to the outside of the orthogonal conversion unit 113 (supplied to the quantization unit 114).

[0134] The primary transformation unit 152 performs processing related to primary transformations, such as predetermined transformation processes like orthogonal transformations. For example, the primary transformation unit 152 takes the component identifier compID, the adaptive primary transformation flag apt_flag[compID] of the component identifier compID, the primary transformation identifier pt_idx[compID] of the component identifier compID, the prediction mode information PInfo, the size of the transformation block (logarithm of the width log2TBWSize, logarithm of the height log2TBHSize), and the prediction residual D as input. The width of the transformation block TBWSize is also referred to as TBWidth, and its logarithm is also referred to as log2TBWidth. Similarly, the height of the transformation block TBHSize is also referred to as TBHeight, and its logarithm is also referred to as log2TBHeight.

[0135] The primary transformation unit 152 refers to its prediction mode information PInfo, component identifier compID, adaptive primary transformation flag apt_flag[compID] of component identifier compID, and primary transformation identifier pt_idx[compID] of component identifier compID, and selects the transformation type TrTypeH (and primary horizontal transformation type identifier TrTypeIdxH) indicating the transformation type and the transformation type TrTypeV (and primary vertical transformation type identifier TrTypeIdxV) indicating the transformation type for the primary horizontal transformation that correspond to the component identifier compID.

[0136] Furthermore, the primary transformation unit 152 performs a primary horizontal transformation on the predicted residual D, which is determined by the primary horizontal transformation type identifier TrTypeIdxH (or primary horizontal transformation type TrTypeH) and the width of the transformation block log2TBWSize, and a primary vertical transformation, which is determined by the primary vertical transformation type identifier TrTypeIdxV (or primary vertical transformation type TrTypeV) and the height of the transformation block log2TBHSize, and derives the transformation coefficient Coeff_P after the primary transformation. The primary horizontal transformation is a one-dimensional orthogonal transformation in the horizontal direction, and the primary vertical transformation is a one-dimensional orthogonal transformation in the vertical direction.

[0137] The primary conversion unit 152 supplies the derived conversion coefficient Coeff_P to the secondary conversion unit 153.

[0138] The secondary transformation unit 153 performs processing related to secondary transformation, which is a predetermined transformation process such as orthogonal transformation. For example, the secondary transformation unit 153 takes a secondary transformation identifier st_idx, a scan identifier scanIdx indicating the method of scanning the transformation coefficients, and a transformation coefficient Coeff_P as input. Based on the secondary transformation identifier st_idx and the scan identifier scanIdx, the secondary transformation unit 153 performs a secondary transformation on the transformation coefficient Coeff_P and derives the transformation coefficient Coeff_S after secondary transformation.

[0139] More specifically, if the secondary transformation identifier st_idx indicates that a secondary transformation should be applied (st_idx > 0), the secondary transformation unit 153 performs the secondary transformation process corresponding to the secondary transformation identifier st_idx on the transformation coefficient Coeff_P and derives the transformation coefficient Coeff_S after the secondary transformation.

[0140] The secondary transformation unit 153 outputs its secondary transformation coefficient Coeff_S as the transformation coefficient Coeff to the outside of the orthogonal transformation unit 113 (supplied to the quantization unit 114).

[0141] Furthermore, if the secondary transformation identifier st_idx indicates that the secondary transformation should not be applied (st_idx==0), the secondary transformation unit 153 skips the secondary transformation and outputs the transformation coefficient Coeff_P after the primary transformation as the transformation coefficient Coeff (the transformation coefficient Coeff_S after the secondary transformation) to the outside of the orthogonal transformation unit 113 (supplied to the quantization unit 114).

[0142] In the orthogonal transformation unit 113 configured as described above, the primary transformation unit 152 performs processing that applies the aforementioned technology as both a derivation unit and an orthogonal transformation unit. Specifically, the derivation unit derives a second transformation matrix using the first transformation matrix, and the orthogonal transformation unit performs a primary transformation on the predicted residuals using the second transformation matrix derived by the derivation unit. Therefore, the increase in memory capacity required for the primary transformation can be suppressed.

[0143] As described above, the primary transformation unit 152 performs a primary horizontal transformation and a primary vertical transformation as primary transformations. Specifically, the derivation unit derives a second transformation matrix for a horizontal one-dimensional orthogonal transformation and a second transformation matrix for a vertical one-dimensional orthogonal transformation. The orthogonal transformation unit performs a horizontal one-dimensional orthogonal transformation using the second transformation matrix for the horizontal one-dimensional orthogonal transformation derived by the derivation unit as a primary transformation, and then performs a vertical one-dimensional orthogonal transformation using the second transformation matrix for the vertical one-dimensional orthogonal transformation derived by the derivation unit. Therefore, it is possible to suppress the increase in memory capacity required for primary transformations that perform such horizontal one-dimensional orthogonal transformations and vertical one-dimensional orthogonal transformations.

[0144] <Image encoding process flow> Next, we will explain the flow of each process performed by the image encoding device 100 as described above. First, we will explain an example of the image encoding process flow with reference to the flowchart in Figure 13.

[0145] When the image encoding process is started, in step S101, the sorting buffer 111 is controlled by the control unit 101 to sort the order of the frames of the input video data from display order to encoding order.

[0146] In step S102, the control unit 101 sets processing units (performs block division) for the input image held by the sorting buffer 111.

[0147] In step S103, the control unit 101 determines (sets) the encoding parameters for the input image held by the sorting buffer 111.

[0148] In step S104, the prediction unit 122 performs prediction processing and generates predicted images, etc., for the optimal prediction mode. For example, in this prediction processing, the prediction unit 122 performs intra-prediction to generate predicted images, etc., for the optimal intra-prediction mode, performs inter-prediction to generate predicted images, etc., for the optimal inter-prediction mode, and selects the optimal prediction mode from among them based on cost function values, etc.

[0149] In step S105, the calculation unit 112 calculates the difference between the input image and the predicted image of the optimal mode selected by the prediction process in step S104. In other words, the calculation unit 112 generates a predicted residual D between the input image and the predicted image. The predicted residual D obtained in this way has a reduced data size compared to the original image data. Therefore, the amount of data can be compressed compared to encoding the image as is.

[0150] In step S106, the orthogonal transformation unit 113 performs an orthogonal transformation on the predicted residual D generated by the processing in step S105, and derives the transformation coefficient Coeff.

[0151] In step S107, the quantization unit 114 quantizes the conversion coefficient Coeff obtained in the process of step S106 by using the quantization parameters calculated by the control unit 101, and derives the quantization conversion coefficient level level.

[0152] In step S108, the inverse quantization unit 117 inversely quantizes the quantization conversion coefficient level generated by the processing in step S107 using characteristics corresponding to the quantization characteristics of step S107, and derives the conversion coefficient Coeff_IQ.

[0153] In step S109, the inverse orthogonal transform unit 118 performs an inverse orthogonal transform on the transformation coefficient Coeff_IQ obtained in step S108 in a manner corresponding to the orthogonal transform process in step S106, and derives the predicted residual D'. Since this inverse orthogonal transform process is the same as the inverse orthogonal transform process performed on the decoding side (described later), the explanation given for the decoding side (described later) can be applied to the inverse orthogonal transform process in step S109.

[0154] In step S110, the calculation unit 119 generates a locally decoded decoded image by adding the predicted image obtained by the prediction process in step S104 to the predicted residual D' derived by the processing in step S109.

[0155] In step S111, the in-loop filter unit 120 performs in-loop filtering on the locally decoded decoded image derived from the processing in step S110.

[0156] In step S112, the frame memory 121 stores the locally decoded decoded images derived from the processing in step S110, and the locally decoded decoded images that have been filtered in step S112.

[0157] In step S113, the encoding unit 115 encodes the quantization conversion coefficient level obtained in step S107. For example, the encoding unit 115 encodes the quantization conversion coefficient level, which is information about the image, using arithmetic encoding or the like to generate encoded data. At this time, the encoding unit 115 also encodes various encoding parameters (header information Hinfo, prediction mode information Pinfo, and conversion information Tinfo). Furthermore, the encoding unit 115 derives residual information RInfo from the quantization conversion coefficient level and encodes the residual information RInfo.

[0158] In step S114, the storage buffer 116 stores the encoded data obtained in this way and outputs it to the outside of the image encoding device 100 as a bitstream, for example. This bitstream is transmitted to the decoding side via a transmission line or recording medium, for example. The rate control unit 123 also performs rate control as needed.

[0159] Once step S114 is completed, the image encoding process is finished.

[0160] In the image encoding process described above, the process applying the aforementioned technology is performed as step S106. Furthermore, the process applying the aforementioned technology is performed as step S109. In addition, the process applying the aforementioned technology is performed as step S113. Therefore, by executing this image encoding process, it is possible to suppress the increase in memory capacity required for orthogonal and inverse orthogonal transformations.

[0161] <Flow of orthogonal transformation processing> Next, an example of the flow of the orthogonal transformation process performed in step S106 of Figure 13 will be explained with reference to the flowchart in Figure 14.

[0162] When the orthogonal transformation process begins, switch 151 determines in step S131 whether the transformation skip flag ts_flag is 2D_TS (indicating a 2D transformation skip) (e.g., 1 (true)) or whether the transformation quantization bypass flag transquant_bypass_flag is 1 (true). If it is determined that the transformation skip flag ts_flag is 2D_TS (e.g., 1 (true)) or the transformation quantization bypass flag is 1 (true), the orthogonal transformation process ends and the process returns to Figure 13. In this case, the orthogonal transformation process (primary and secondary transformations) is omitted, and the input predicted residual D is used as the transformation coefficient Coeff.

[0163] Furthermore, in step S131 of Figure 14, if it is determined that the transformation skip flag ts_flag is not 2D_TS (not a 2D transformation skip) (for example, 0 (false)) and the transformation quantization bypass flag transquant_bypass_flag is 0 (false), the process proceeds to step S132. In this case, the primary transformation process and the secondary transformation process are performed.

[0164] In step S132, the primary transformation unit 152 performs primary transformation processing on the input predicted residual D based on the adaptive primary transformation information specified by the component identifier compID, and derives the transformation coefficient Coeff_P after primary transformation.

[0165] In step S133, the secondary conversion unit 153 performs a secondary conversion process on the conversion coefficient Coeff_P and derives the conversion coefficient Coeff_S (conversion coefficient Coeff) after secondary conversion.

[0166] Once the processing in step S133 is complete, the orthogonal transformation process is finished.

[0167] In the orthogonal transformation process described above, the process applying the aforementioned technology is performed as step S132. Therefore, by executing this orthogonal transformation process, the increase in memory capacity required for the primary transformation can be suppressed.

[0168] <Image Decoding Device> Figure 15 is a block diagram showing an example of the configuration of an image decoding device, which is one embodiment of an image processing device to which this technology is applied. The image decoding device 200 shown in Figure 15 is a device that decodes encoded data in which the predicted residual between an image and its predicted image is encoded, such as AVC or HEVC. For example, the image decoding device 200 implements the technology described in Non-Patent Document 1, Non-Patent Document 3, or Non-Patent Document 4, and decodes encoded data in which the image data of a moving image is encoded in a manner compliant with the standard described in any of those documents. For example, the image decoding device 200 decodes the encoded data (bitstream) generated by the image encoding device 100 described above.

[0169] Note that Figure 15 shows the main components such as the processing unit and data flow, and does not necessarily represent everything. In other words, there may be processing units in the image decoding device 200 that are not shown as blocks in Figure 15, and there may be processes and data flows that are not shown as arrows or other symbols in Figure 15. The same applies to other figures that explain the processing units within the image decoding device 200.

[0170] In Figure 15, the image decoding device 200 comprises a storage buffer 211, a decoding unit 212, an inverse quantization unit 213, an inverse orthogonal transform unit 214, an arithmetic unit 215, an in-loop filter unit 216, a sorting buffer 217, a frame memory 218, and a prediction unit 219. The prediction unit 219 includes an intra-prediction unit and an inter-prediction unit (not shown). The image decoding device 200 is a device for generating moving image data by decoding encoded data (bitstream).

[0171] <Storage buffer> The storage buffer 211 acquires and stores (stores) the bitstream input to the image decoding device 200. The storage buffer 211 supplies the stored bitstream to the decoding unit 212 at a predetermined timing or when predetermined conditions are met.

[0172] <Decryption section> The decoding unit 212 performs processing related to image decoding. For example, the decoding unit 212 takes the bitstream supplied from the storage buffer 211 as input, and decodes the syntax value of each syntax element from the bit sequence in accordance with the definition of the syntax table, thereby deriving the parameters.

[0173] The parameters derived from the syntax elements and their syntax values ​​include, for example, header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, residual information Rinfo, and filter information Finfo. In other words, the decoding unit 212 parses (analyzes and obtains) this information from the bitstream. This information is described below.

[0174] <Header Information Hinfo> The header information Hinfo includes, for example, header information such as VPS (Video Parameter Set), SPS (Sequence Parameter Set), PPS (Picture Parameter Set), and SH (Slice Header). The header information Hinfo includes information that defines, for example, the image size (width PicWidth, height PicHeight), bit depth (luminance bitDepthY, chrominance bitDepthC), chrominance array type ChromaArrayType, maximum CU size MaxCUSize / minimum CUSize, maximum depth MaxQTDepth / minimum depth MinQTDepth for quad-tree partitioning, maximum depth MaxBTDepth / minimum depth MinBTDepth for binary-tree partitioning, maximum value MaxTSSize (also called maximum conversion skip block size), and on / off flags (also called enable flags) for each encoding tool.

[0175] For example, the on / off flags for encoding tools included in the header information Hinfo include the on / off flags related to the following transformation and quantization processes. Note that the on / off flag for an encoding tool can also be interpreted as a flag indicating whether or not the syntax related to that encoding tool exists in the encoded data. Furthermore, a value of 1 (true) for the on / off flag indicates that the encoding tool is usable, and a value of 0 (false) indicates that the encoding tool is unavailable. Note that the interpretation of the flag values ​​may be reversed.

[0176] Cross-Component Prediction (CCP_enabled_flag): This flag indicates whether cross-component prediction (CCP, also known as CC prediction) is enabled or disabled. For example, a flag value of "1" (true) indicates that it is enabled, while a flag value of "0" (false) indicates that it is disabled.

[0177] This CCP is also known as inter-component linear prediction (CCLM or CCLMP).

[0178] <Predictive Mode Information Pinfo> The prediction mode information Pinfo includes, for example, information such as the size information of the PB (prediction block) to be processed (PBSize, prediction block size), intra prediction mode information IPinfo, and motion prediction information MVinfo.

[0179] The intra-prediction mode information IPinfo includes, for example, prev_intra_luma_pred_flag, mpm_idx, rem_intra_pred_mode in the JCTVC-W1005, 7.3.8.5 Coding Unit syntax, and the luminance intra-prediction mode IntraPredModeY derived from that syntax.

[0180] Furthermore, the intra-prediction mode information IPinfo includes, for example, the inter-component prediction flag (ccp_flag(cclmp_flag)), the multi-class linear prediction mode flag (mclm_flag), the color difference sample location type identifier (chroma_sample_loc_type_idx), the color difference MPM identifier (chroma_mpm_idx), and the luminance intra-prediction mode (IntraPredModeC) derived from these syntaxes.

[0181] The inter-component prediction flag (ccp_flag(cclmp_flag)) is a flag indicating whether or not to apply inter-component linear prediction. For example, ccp_flag==1 indicates that inter-component prediction will be applied, and ccp_flag==0 indicates that inter-component prediction will not be applied.

[0182] The multi-class linear prediction mode flag (mclm_flag) is information about the linear prediction mode (linear prediction mode information). More specifically, the multi-class linear prediction mode flag (mclm_flag) is flag information that indicates whether or not to use multi-class linear prediction mode. For example, if it is "0", it indicates that it is a one-class mode (single-class mode) (e.g., CCLMP), and if it is "1", it indicates that it is a two-class mode (multi-class mode) (e.g., MCLMP).

[0183] The chroma_sample_loc_type_idx is an identifier that identifies the type of pixel position of a chroma difference component (also called the chroma_sample_loc_type). For example, if the ChromaArrayType, which is information about the color format, indicates a 420 format, the chroma_sample_loc_type identifier will be assigned as shown in equation (7) below.

[0184]

number

[0185] This color difference sample location type identifier (chroma_sample_loc_type_idx) is transmitted as information about the pixel location of the color difference component (chroma_sample_loc_info()), which is stored within it.

[0186] The chroma_mpm_idx identifier is an identifier that indicates which prediction mode candidate from the intraPredModeCandListC chroma_mpm_idx is designated as the intraPredMode prediction mode.

[0187] The motion prediction information MVinfo includes information such as merge_idx, merge_flag, inter_pred_idc, ref_idx_LX, mvp_lX_flag, X={0,1}, and mvd (see, for example, JCTVC-W1005, 7.3.8.6 Prediction Unit Syntax).

[0188] Of course, the information included in the predictive mode information Pinfo is arbitrary, and other information may be included as well.

[0189] <Conversion Information Tinfo> The conversion information Tinfo may include, for example, the following information. Of course, the information included in the conversion information Tinfo is arbitrary, and it may include information other than these.

[0190] The width (TBWSize) and height (TBHSize) of the conversion block to be processed (or the base-2 logarithms of each TBWSize and TBHSize, log2TBWSize and log2TBHSize). Conversion skip flag (ts_flag): This flag indicates whether or not to skip the (inverse) primary and (inverse) secondary conversions. Scan identifier (scanIdx) Quantization parameter (qp) Quantization matrix (scaling_matrix (e.g., JCTVC-W1005, 7.3.4 Scaling list data syntax))

[0191] <Residual Information Rinfo> The residual information Rinfo (see, for example, 7.3.8.11 Residual Coding syntax in JCTVC-W1005) includes syntax such as the following:

[0192] cbf(coded_block_flag): residual data presence / absence flag last_sig_coeff_x_pos: Last non-zero coefficient X coordinate last_sig_coeff_y_pos: Last non-zero coefficient Y coordinate coded_sub_block_flag: Subblock non-zero coefficient flag sig_coeff_flag: Flag indicating presence or absence of non-zero coefficients gr1_flag: A flag indicating whether the level of non-zero coefficients is greater than 1 (also called the GR1 flag). gr2_flag: A flag indicating whether the level of non-zero coefficients is greater than 2 (also called the GR2 flag). sign_flag: The sign indicating the positive or negative sign of non-zero coefficients (also called the sine sign). coeff_abs_level_remaining: Residual level of non-zero coefficients (also called non-zero coefficient residual level) etc.

[0193] Of course, the information included in the residual information Rinfo is arbitrary, and it is also possible to include information other than this.

[0194] <Filter Information Finfo> The filter information Finfo includes, for example, control information related to each of the following filter processes:

[0195] Control information regarding deblocking filters (DBFs) Control information regarding Pixel Adaptive Offset (SAO) Control information for Adaptive Loop Filters (ALF) Other control information related to linear and nonlinear filters

[0196] More specifically, this includes information such as the picture to which each filter applies, the area within the picture, filter On / Off control information for each CU, and filter On / Off control information related to slice and tile boundaries. Of course, the information included in the filter information Finfo is arbitrary, and other information may also be included.

[0197] Returning to the description of the decoding unit 212, the decoding unit 212 refers to the residual information Rinfo and derives the quantization transformation coefficient level level for each coefficient position within each transformation block. The decoding unit 212 supplies this quantization transformation coefficient level level to the inverse quantization unit 213.

[0198] Furthermore, the decoding unit 212 supplies parsed header information Hinfo, prediction mode information Pinfo, quantization conversion coefficient level level, conversion information Tinfo, and filter information Finfo to each block. Specifically, this is done as follows:

[0199] The header information Hinfo is supplied to the inverse quantization unit 213, the inverse orthogonal transform unit 214, the prediction unit 219, and the in-loop filter unit 216. The prediction mode information Pinfo is supplied to the inverse quantization unit 213 and the prediction unit 219. The conversion information Tinfo is supplied to the inverse quantization unit 213 and the inverse orthogonal transformation unit 214. The filter information Finfo is supplied to the in-loop filter unit 216.

[0200] Of course, the above example is just one example and is not limited to this example. For example, each encoding parameter may be supplied to any processing unit. Also, other information may be supplied to any processing unit.

[0201] <Inverse quantization part> The inverse quantization unit 213 performs processing related to inverse quantization. For example, the inverse quantization unit 213 takes the conversion information Tinfo and the quantization conversion coefficient level level supplied from the decoding unit 212 as input, scales (inverse quantizes) the value of the quantization conversion coefficient level level based on the conversion information Tinfo, and derives the conversion coefficient Coeff_IQ after inverse quantization.

[0202] This inverse quantization is performed as the reverse process of quantization by the quantization unit 114. Furthermore, this inverse quantization is the same process as the inverse quantization performed by the inverse quantization unit 117. In other words, the inverse quantization unit 117 performs the same process (inverse quantization) as the inverse quantization unit 213.

[0203] The inverse quantization unit 213 supplies the derived conversion coefficient Coeff_IQ to the inverse orthogonal transformation unit 214.

[0204] <Inverse orthogonal transformation section> The inverse orthogonal transform unit 214 performs processing related to the inverse orthogonal transform. For example, the inverse orthogonal transform unit 214 takes the transformation coefficient Coeff_IQ supplied from the inverse quantization unit 213 and the transformation information Tinfo supplied from the decoding unit 212 as input, and performs an inverse orthogonal transform on the transformation coefficient Coeff_IQ based on the transformation information Tinfo to derive the predicted residual D'.

[0205] This inverse orthogonal transformation is performed as the reverse of the orthogonal transformation performed by the orthogonal transformation unit 113. Furthermore, this inverse orthogonal transformation is the same process as the inverse orthogonal transformation performed by the inverse orthogonal transformation unit 118. In other words, the inverse orthogonal transformation unit 118 performs the same process (inverse orthogonal transformation) as the inverse orthogonal transformation unit 214.

[0206] The inverse orthogonal transform unit 214 supplies the derived predicted residual D' to the calculation unit 215.

[0207] <Arithmetic section> The calculation unit 215 performs processing related to the addition of information about the image. For example, the calculation unit 215 takes the predicted residual D' supplied from the inverse orthogonal transform unit 214 and the predicted image P supplied from the prediction unit 219 as input. The calculation unit 215 adds the predicted residual D' and the predicted image P (predicted signal) corresponding to the predicted residual D' as shown in equation (8) below, and obtains the local decoded image R local Derive the following.

[0208]

number

[0209] The calculation unit 215 calculates the derived local decoded image R local This is supplied to the in-loop filter unit 216 and the frame memory 218.

[0210] <In-loop filter section> The in-loop filter unit 216 performs processing related to in-loop filtering. For example, the in-loop filter unit 216 processes the local decoded image R supplied from the calculation unit 215. local The inputs are the filter information Finfo supplied from the decoding unit 212. The information input to the in-loop filter unit 216 is arbitrary, and information other than this may be input.

[0211] The in-loop filter unit 216 performs appropriate filtering on the locally decoded image Rlocal based on its filter information Finfo.

[0212] For example, the in-loop filter unit 216 applies four in-loop filters in this order, as described in Non-Patent Literature 1: a bilateral filter, a deblocking filter (DBF), an adaptive offset filter (SAO), and an adaptive loop filter (ALF). Note that the order in which these filters are applied is arbitrary and can be selected as appropriate.

[0213] The in-loop filter unit 216 performs filtering corresponding to the filtering performed by the encoding side (for example, the in-loop filter unit 120 of the image encoding device 100). Of course, the filtering performed by the in-loop filter unit 216 is arbitrary and not limited to the example described above. For example, the in-loop filter unit 216 may apply a Wiener filter or the like.

[0214] The in-loop filter unit 216 filters the local decoded image R local The data is then sorted and supplied to the buffer 217 and the frame memory 218.

[0215] <sort buffer> The sorting buffer 217 receives the local decoded image R supplied from the in-loop filter unit 216. local It takes the input and stores (stores) it. The sorting buffer 217 stores its local decoded image R local The decoded image R for each picture unit is reconstructed and stored (stored in a buffer) using the sorting buffer 217. The sorting buffer 217 sorts the obtained decoded images R from the decoding order to the playback order. The sorting buffer 217 outputs the sorted group of decoded images R as moving image data to the outside of the image decoding device 200.

[0216] <Frame Memory> The frame memory 218 performs processing related to the storage of image data. For example, the frame memory 218 stores the locally decoded image R supplied by the arithmetic unit 215. local The input is used to reconstruct the decoded image R for each picture unit and store it in a buffer in the frame memory 218.

[0217] Furthermore, the frame memory 218 receives the locally decoded image R that has been filtered in the in-loop filter from the in-loop filter unit 216. localThe input is used to reconstruct the decoded image R for each picture unit and store it in a buffer in the frame memory 218. The frame memory 218 then supplies the stored decoded image R (or a part thereof) to the prediction unit 219 as a reference image as appropriate.

[0218] The frame memory 218 may also store header information Hinfo, prediction mode information Pinfo, conversion information Tinfo, filter information Finfo, etc., related to the generation of the decoded image.

[0219] <Prediction Section> The prediction unit 219 performs processing related to the generation of a predicted image. For example, the prediction unit 219 takes prediction mode information Pinfo supplied from the decoding unit 212 as input, performs prediction according to the prediction method specified by the prediction mode information Pinfo, and derives a predicted image P. In this deriving process, the prediction unit 219 uses the pre-filtered or filtered decoded image R (or a part thereof) stored in the frame memory 218, as specified by the prediction mode information Pinfo, as a reference image. The prediction unit 219 supplies the derived predicted image P to the calculation unit 215.

[0220] In the image decoding device 200 configured as described above, the inverse orthogonal transform unit 214 performs processing that applies the aforementioned technology as both a derivation unit and an inverse orthogonal transform unit. The decoding unit 212 also performs processing that applies the aforementioned technology as a decoding unit. Therefore, the image decoding device 200 can suppress the increase in memory capacity required for inverse orthogonal transformation.

[0221] <Details of the inverse orthogonal transformer> Figure 16 is a block diagram showing an example of the main configuration of the inverse orthogonal transformer 214 shown in Figure 15. As shown in Figure 16, the inverse orthogonal transformer 214 includes a switch 251, an inverse secondary transformer 252, and an inverse primary transformer 253.

[0222] Switch 251 receives the conversion coefficient Coeff_IQ and the conversion skip flag ts_flag[compID] as inputs. If the value of the conversion skip flag ts_flag[compID] is NO_TS (=0), i.e., if conversion skipping is not applied, switch 251 supplies the conversion coefficient Coeff_IQ to the inverse secondary conversion unit 252. Also, if the value of the conversion skip flag ts_flag[compID] is 2D_TS (=1), i.e., if 2D conversion skipping is applied, switch 251 skips the inverse secondary conversion unit 252 and the inverse primary conversion unit 253, and outputs the conversion coefficient Coeff_IQ as the predicted residual D' to the outside of the inverse orthogonal conversion unit 214 (supplied to the calculation unit 215).

[0223] The inverse secondary transform unit 252 performs processing related to the inverse secondary transform, which is the inverse process of the secondary transform performed on the encoding side (for example, the secondary transform unit 153 of the image encoding device 100). For example, the inverse secondary transform unit 252 takes the secondary transform identifier st_idx, the scan identifier scanIdx indicating the method of scanning the transform coefficients, and the transform coefficient Coeff_IQ supplied from the switch 251 as inputs.

[0224] The inverse secondary transformation unit 252 performs an inverse secondary transformation on the transformation coefficient Coeff_IQ based on the secondary transformation identifier st_idx and the scan identifier scanIdx, and derives the transformation coefficient Coeff_IS after the inverse secondary transformation.

[0225] More specifically, if the secondary transformation identifier st_idx indicates that an inverse secondary transformation should be applied (st_idx > 0), the inverse secondary transformation unit 252 performs the inverse secondary transformation process corresponding to the secondary transformation identifier st_idx on the transformation coefficient Coeff_IQ and derives the transformation coefficient Coeff_IS after the inverse secondary transformation. The inverse secondary transformation unit 252 then supplies the transformation coefficient Coeff_IS after the inverse secondary transformation to the inverse primary transformation unit 253.

[0226] Furthermore, if the secondary transformation identifier st_idx indicates that the inverse secondary transformation should not be applied (st_idx==0), the inverse secondary transformation unit 252 skips the inverse secondary transformation and supplies the transformation coefficient Coeff_IQ to the inverse primary transformation unit 253 as the transformation coefficient Coeff_IS after the inverse secondary transformation.

[0227] The inverse primary transform unit 253 performs processing related to the inverse primary transform, which is the inverse process of the primary transform performed on the encoding side (for example, the primary transform unit 152 of the image encoding device 100). For example, the inverse primary transform unit 253 takes the component identifier compID, the adaptive primary transform flag apt_flag[compID] of the component identifier compID, the primary transform identifier pt_idx[compID] of the component identifier compID, the prediction mode information PInfo, the size of the transform block (logarithm of the width log2TBWSize, logarithm of the height log2TBHSize), and the transform coefficient Coeff_IS after the inverse secondary transform as input.

[0228] The inverse primary transform unit 253 refers to its prediction mode information PInfo, component identifier compID, adaptive primary transform flag apt_flag[compID] of component identifier compID, and primary transform identifier pt_idx[compID] of component identifier compID, and selects the transform type TrTypeH (and the inverse primary horizontal transform type identifier TrTypeIdxH) indicating the transform type, and the transform type TrTypeV (and the inverse primary vertical transform type identifier TrTypeIdxV) indicating the transform type, for the inverse primary horizontal transform that corresponds to the component identifier compID.

[0229] Furthermore, the inverse primary transformation unit 253 performs an inverse primary vertical transformation on the transformation coefficient Coeff_IS after the inverse secondary transformation, which is determined by the inverse primary vertical transformation type identifier TrTypeIdxV (or inverse primary vertical transformation type TrTypeV) and the vertical width log2TBHSize of the transformation block, and an inverse primary horizontal transformation, which is determined by the inverse primary horizontal transformation type identifier TrTypeIdxH (or inverse primary horizontal transformation type TrTypeH) and the horizontal width log2TBWSize of the transformation block, to derive the transformation coefficient Coeff_IP after the inverse primary transformation. The inverse primary vertical transformation is an inverse one-dimensional orthogonal transformation in the vertical direction, and the inverse primary horizontal transformation is an inverse one-dimensional orthogonal transformation in the horizontal direction.

[0230] The inverse primary transform unit 253 outputs the transformation coefficient Coeff_IP after the inverse primary transform as the predicted residual D' to the outside of the inverse orthogonal transform unit 214 (supplied to the calculation unit 215).

[0231] In the inverse orthogonal transform unit 214 configured as described above, the inverse primary transform unit 253 performs processing that applies the aforementioned technology as a derivation unit and an inverse orthogonal transform unit. That is, the derivation unit derives a second transformation matrix using the first transformation matrix, and the inverse orthogonal transform unit performs an inverse primary transform on the inverse secondary transform result using the second transformation matrix derived by the derivation unit. Therefore, it is possible to suppress the increase in memory capacity required for such an inverse primary transform.

[0232] As described above, the inverse primary transformation unit 253 performs an inverse primary vertical transformation and an inverse primary horizontal transformation as inverse primary transformations. Specifically, the derivation unit derives a second transformation matrix for the inverse one-dimensional orthogonal transformation in the vertical direction and a second transformation matrix for the inverse one-dimensional orthogonal transformation in the horizontal direction. The inverse orthogonal transformation unit performs an inverse one-dimensional orthogonal transformation in the vertical direction using the second transformation matrix for the inverse one-dimensional orthogonal transformation in the vertical direction derived by the derivation unit, and then performs an inverse one-dimensional orthogonal transformation in the horizontal direction using the second transformation matrix for the inverse one-dimensional orthogonal transformation in the horizontal direction derived by the derivation unit. Therefore, it is possible to suppress the increase in memory capacity required for primary transformations in which such inverse one-dimensional orthogonal transformations in the vertical direction and inverse one-dimensional orthogonal transformations in the horizontal direction are performed.

[0233] <Image Decoding Process Flow> Next, we will explain the flow of each process performed by the image decoding device 200 as described above. First, we will explain an example of the image decoding process flow with reference to the flowchart in Figure 17.

[0234] When the image decoding process starts, the storage buffer 211 acquires and holds (stores) the encoded data (bitstream) supplied from outside the image decoding device 200 in step S201.

[0235] In step S202, the decoding unit 212 decodes the encoded data (bitstream) and obtains the quantization conversion coefficient level. The decoding unit 212 also parses (analyzes and obtains) various encoding parameters from the encoded data (bitstream) through this decoding.

[0236] In step S203, the inverse quantization unit 213 performs inverse quantization, which is the reverse process of quantization performed on the encoding side, on the quantization conversion coefficient level level obtained in the processing of step S202, to obtain the conversion coefficient Coeff_IQ.

[0237] In step S204, the inverse orthogonal transform unit 214 performs an inverse orthogonal transform on the transformation coefficient Coeff_IQ obtained in step S203, which is the inverse of the orthogonal transform process performed on the encoding side, to obtain the predicted residual D'.

[0238] In step S205, the prediction unit 219 performs prediction processing based on the information parsed in step S202 using a prediction method specified by the encoding side, and generates a predicted image P by referring to a reference image stored in the frame memory 218, etc.

[0239] In step S206, the calculation unit 215 adds the predicted residual D' obtained in step S204 and the predicted image P obtained in step S205 to obtain the local decoded image R local Derive the following.

[0240] In step S207, the in-loop filter unit 216 processes the local decoded image R obtained by the processing in step S206. local In-loop filtering is performed on this.

[0241] In step S208, the sorting buffer 217 is used to sort the filtered local decoded image R obtained by the processing in step S207. local The decoded image R is derived using this method, and the order of the decoded image R group is rearranged from the decoding order to the playback order. The decoded image R group rearranged in playback order is output to the outside of the image decoding device 200 as a moving image.

[0242] Furthermore, in step S209, the frame memory 218 stores the local decoded image R obtained by the processing in step S206. local , and the filtered local decoded image R obtained by the processing in step S207 local Remember at least one of them.

[0243] Once the process in step S209 is completed, the image decoding process is finished.

[0244] In the image decoding process described above, the process applying the aforementioned technology is performed as step S202. Furthermore, the process applying the aforementioned technology is performed as step S204. Therefore, by executing this image decoding process, the increase in memory capacity required for the inverse orthogonal transform can be suppressed.

[0245] <Processing flow of inverse orthogonal transform> Next, an example of the flow of the inverse orthogonal transform process performed in step S204 of Figure 17 will be explained with reference to the flowchart in Figure 18. When the inverse orthogonal transform process is started, switch 251 determines in step S231 whether the transform skip flag ts_flag is 2D_TS (mode of 2D transform skip) (e.g., 1 (true)) or whether the transform quantization bypass flag transquant_bypass_flag is 1 (true). If it is determined that the transform skip identifier ts_idx is 2D_TS or the transform quantization bypass flag is 1 (true), the inverse orthogonal transform process is terminated and the process returns to Figure 17. In this case, the inverse orthogonal transform process (inverse primary transform and inverse secondary transform) is omitted, and the transform coefficient Coeff_IQ is set to the predicted residual D'.

[0246] Furthermore, if it is determined in step S231 that the transformation skip identifier ts_idx is not 2D_TS (a mode other than 2D transformation skip) (for example, 0 (false)) and the transformation quantization bypass flag is 0 (false), the process proceeds to step S232. In this case, the inverse secondary transformation process and the inverse primary transformation process are performed.

[0247] In step S232, the inverse secondary transformation unit 252 performs an inverse secondary transformation on the transformation coefficient Coeff_IQ based on the secondary transformation identifier st_idx, derives the transformation coefficient Coeff_IS, and outputs it.

[0248] In step S233, the inverse primary transformation unit 253 performs an inverse primary transformation on the transformation coefficient Coeff_IS and derives the transformation coefficient Coeff_IP (predicted residual D') after the inverse primary transformation.

[0249] Once step S233 is completed, the inverse orthogonal transform process is finished.

[0250] In the inverse orthogonal transform process described above, the process using the aforementioned technology is performed as step S233. Therefore, by executing this inverse orthogonal transform process, the increase in memory capacity required for the inverse primary transform process can be suppressed.

[0251] <2-2. Example 1-1> <Concept> Next, we will explain each derivation example described in Figure 10 in more detail. First, we will explain the first and second row examples from the top, excluding the row of item names at the top of the table shown in Figure 10.

[0252] As described above, the derivation of the first example from the top focuses on the similarity between the waveform of the lowest-order row vector of the first transformation type and the waveform of the lowest-order row vector of the transformation type of the alternative transformation matrix. In this case, the derivation unit flips the first transformation matrix to derive the second transformation matrix. That is, as shown in Figure 19, the derivation unit uses the transformation matrix of DST7 as the base transformation matrix T base Then, we flip it in the row direction to derive the transformation matrix of FlipDST7. This transformation matrix of FlipDST7 can be used to substitute for the transformation matrix of DCT8, which has a similar waveform for the 0th row vector (a decreasing type).

[0253] A concrete example of this derivation is shown in the upper part of Figure 20. As shown in the upper part of Figure 20, this derivation is based on the base transformation matrix T base It can be expressed by the matrix product of (DST7) and the flip matrix J. Here, the flip matrix J (also called the Cross-Identity Matrix) is the horizontally flipped version of the N×N identity matrix I.

[0254] Furthermore, the derivation of the second example from the top focuses on the similarity between the waveform of the lowest-order column vector of the first transformation type and the waveform of the lowest-order row vector of the transformation type of the substitute transformation matrix. In this case, the derivation unit transposes the first transformation matrix to derive the second transformation matrix. That is, as shown in Figure 19, the derivation unit uses the transformation matrix of DST7 as the base transformation matrix T base Then, we transpose it to derive the transformation matrix for TrDST7. This transformation matrix for TrDST7 can be used to substitute for the transformation matrix for DST1, which has a similar (mountain-shaped) waveform for the 0th row vector.

[0255] A concrete example of this derivation is shown in the lower part of Figure 20. As shown in the lower part of Figure 20, this derivation is based on the base transformation matrix T base It can be expressed by transposing (DST7).

[0256] In other words, in both of the above derivation examples, the second transformation matrix can be derived in a single operation (flipping or transposing). Furthermore, this operation is easy. That is, the second transformation matrix can be easily derived.

[0257] Furthermore, by applying the two derivation examples above, it becomes unnecessary to prepare DCT8 and DST1 transformation matrices as candidates for transformation matrices used in orthogonal and inverse orthogonal transformations. In other words, the number of intrinsic transformation types can be reduced.

[0258] In this case, as shown in the table in Figure 21, the number of transformation matrices required, which was five in the case of the technology described in Non-Patent Literature 1 (see table in Figure 6A), can be reduced to three, so the total LUT size can be reduced to approximately 40KB. In other words, the LUT size can be reduced by approximately 53KB (table in Figure 6A) compared to the case of the technology described in Non-Patent Literature 1. That is, the increase in LUT size can be suppressed.

[0259] As mentioned above, in this case as well, by performing orthogonal and inverse orthogonal transformations using the transformation matrices of the derived second transformation types (TrDST7 and FlipDST7), coding efficiency comparable to that obtained when using the DST1 or DCT8 transformation matrices for orthogonal and inverse orthogonal transformations can be obtained.

[0260] <Primary Conversion Unit> Next, the configuration and processing for performing such derivation will be explained. Figure 22 is a block diagram showing an example of the main configuration of the primary conversion unit 152 in this case. As shown in Figure 22, the primary conversion unit 152 has a primary conversion selection unit 311, a primary horizontal conversion unit 312, and a primary vertical conversion unit 313.

[0261] The primary transformation selection unit 311 receives the prediction mode information PInfo, the component identifier compID, the adaptive primary transformation flag apt_flag[compID], and the primary transformation identifier pt_idx[compID] as inputs. The primary transformation selection unit 311 refers to this information and derives the transformation type identifier TrTypeIdxH for the primary horizontal transformation and the transformation type identifier TrTypeIdxV for the primary vertical transformation. The primary transformation selection unit 311 supplies the derived primary horizontal transformation type identifier TrTypeIdxH to ​​the primary horizontal transformation unit 312. The primary transformation selection unit 311 also supplies the derived primary vertical transformation type identifier TrTypeIdxV to the primary vertical transformation unit 313.

[0262] The primary horizontal transformation unit 312 takes the predicted residual D, the transformation type identifier TrTypeIdxH for the primary horizontal transformation, and information regarding the size of the transformation block (not shown) as input. This information regarding the size of the transformation block may be a natural number N indicating the horizontal or vertical size (number of coefficients) of the transformation block, or it may be log2TBWSize (logarithm of the width) indicating the width of the transformation block (N = 1 << log2TBWSize). The primary horizontal transformation unit 312 performs a primary horizontal transformation Phor, determined by the transformation type identifier TrTypeIdxH and the size of the transformation block, on the predicted residual D, and derives the transformation coefficients Coeff_Phor after the primary horizontal transformation. The primary horizontal transformation unit 312 supplies these transformation coefficients Coeff_Phor to the primary vertical transformation unit 313.

[0263] The primary vertical transformation unit 313 takes as input the transformation coefficient Coeff_Phor after primary horizontal transformation, the transformation type identifier TrTypeIdxV for primary vertical transformation, and information regarding the size of the transformation block (not shown). This information regarding the size of the transformation block may be a natural number N indicating the horizontal or vertical size (number of coefficients) of the transformation block, or it may be log2TBHSize (logarithm of the vertical width) indicating the vertical width of the transformation block (N = 1 << log2TBHSize). The primary vertical transformation unit 313 performs a primary vertical transformation Pver, determined by the transformation type identifier TrTypeIdxV and the size of the transformation block, on the transformation coefficient Coeff_Phor after primary horizontal transformation, and derives the transformation coefficient Coeff_Pver after primary vertical transformation. The primary vertical transformation unit 313 outputs the transformation coefficient Coeff_Pver after primary vertical transformation as the transformation coefficient Coeff_P after primary transformation to the outside of the primary transformation unit 152 (supplied to the secondary transformation unit 153).

[0264] In the primary conversion unit 152 configured as described above, the primary horizontal conversion unit 312 and the primary vertical conversion unit 313 perform processing to which the above-described technology is applied, as the derivation unit and the orthogonal conversion unit, respectively.

[0265] In other words, the primary horizontal transformation unit 312 derives a second transformation matrix for horizontal one-dimensional orthogonal transformation as a derivation unit, and then performs horizontal one-dimensional orthogonal transformation using the second transformation matrix for horizontal one-dimensional orthogonal transformation derived by the derivation unit as an orthogonal transformation unit. Therefore, the primary horizontal transformation unit 312 can suppress the increase in memory capacity required for horizontal one-dimensional orthogonal transformation.

[0266] Furthermore, the primary vertical transformation unit 313 derives a second transformation matrix for vertical one-dimensional orthogonal transformation as a derivation unit, and then performs vertical one-dimensional orthogonal transformation using the second transformation matrix for vertical one-dimensional orthogonal transformation derived by the derivation unit as an orthogonal transformation unit. Therefore, the primary vertical transformation unit 313 can suppress the increase in memory capacity required for vertical one-dimensional orthogonal transformation.

[0267] <Primary Horizontal Conversion Unit> Figure 23 is a block diagram showing an example of the main configuration of the primary horizontal transformation unit 312 in Figure 22. As shown in Figure 23, the primary horizontal transformation unit 312 includes a transformation matrix derivation unit 321, a matrix operation unit 322, a scaling unit 323, and a clipping unit 324.

[0268] The transformation matrix derivation unit 321 takes the transformation type identifier TrTypeIdxH of the primary horizontal transformation and information about the size of the transformation block as input, and generates a transformation matrix T for the primary horizontal transformation that corresponds to the transformation type identifier TrTypeIdxH of the primary horizontal transformation and is the same size as the transformation block. H (Transformation matrix T for horizontal one-dimensional orthogonal transformation) H The transformation matrix derivation unit 321 derives the transformation matrix T. H This is supplied to the matrix operation unit 322.

[0269] The matrix operation unit 322 receives the transformation matrix T from the transformation matrix derivation unit 321. H Using input data X in A horizontal one-dimensional orthogonal transformation is performed on (i.e., the transformation block of the predicted residual D) to obtain the intermediate data Y1. This operation can be expressed as a matrix as shown in equation (9) below.

[0270]

number

[0271] The matrix operation unit 322 supplies the intermediate data Y1 to the scaling unit 323.

[0272] The scaling unit 323 shifts the coefficients Y1[i,j] of each i row and j column component of the intermediate data Y1 by a predetermined shift amount S H We scale the matrix to obtain intermediate data Y2. This scaling can be expressed as shown in equation (10) below. Hereafter, the i-th row and j-th column component ((i,j) component) of a two-dimensional matrix (two-dimensional array) X will be denoted as X[i,j].

[0273]

number

[0274] The scaling unit 323 supplies the intermediate data Y2 to the clipping unit 324.

[0275] The clipping unit 324 clips the coefficient Y2[i,j] of each i row and j column component of the intermediate data Y2, and outputs the X out (That is, the transformation coefficient Coeff_Phor after the primary horizontal transformation) is derived. This process can be expressed as shown in equation (11) below.

[0276]

number

[0277] The clip unit 324 outputs data X out The conversion coefficient Coeff_Phor (after primary horizontal conversion) is output to the outside of the primary horizontal conversion unit 312 (supplied to the primary vertical conversion unit 313).

[0278] In the primary horizontal transformation unit 312 configured as described above, the transformation matrix derivation unit 321 performs processing that applies the aforementioned technology as a derivation unit. The matrix operation unit 322 also performs processing that applies the aforementioned technology as an orthogonal transformation unit. Therefore, the primary horizontal transformation unit 312 can suppress the increase in memory capacity required for one-dimensional orthogonal transformation in the horizontal direction.

[0279] <Transformation Matrix Derivation Section> Figure 24 is a block diagram showing an example of the main configuration of the transformation matrix derivation unit 321 in Figure 23. As shown in Figure 24, the transformation matrix derivation unit 321 includes a transformation matrix LUT 331, a flip unit 332, and a transpose unit 333. Although the arrows indicating data exchange are omitted in Figure 24, the transformation matrix derivation unit 321 can exchange any data between any processing units (processing blocks).

[0280] The transformation matrix LUT331 is a lookup table for holding (storing) transformation matrices corresponding to the transformation type identifier TrTypeIdxH and the size N of the transformation block of the primary horizontal transformation. When the transformation type identifier TrTypeIdxH and the size N of the transformation block of the primary horizontal transformation are specified, the transformation matrix LUT331 selects and outputs the corresponding transformation matrices. In this derivation example, the transformation matrix LUT331 uses its transformation matrix as the base transformation matrix T base It is supplied to the flip section 332, the transfer section 333, or both.

[0281] The flip unit 332 flips the input N x N transformation matrix T, and the resulting transformation matrix T flip The output is as follows: In this derivation example, the flip unit 332 outputs the N x N base transformation matrix T supplied from the transformation matrix LUT331. baseThe input is the base transformation matrix T. base Flip it in the row direction (horizontally), and the resulting transformation matrix T flip The transformation matrix T H The result is output to the outside of the transformation matrix derivation unit 321 (supplied to the matrix operation unit 322).

[0282] The transpose unit 333 transposes the input N x N transformation matrix T, and the transposed transformation matrix T transpose The output is... In this derivation example, the transpose unit 333 outputs the N x N base transformation matrix T supplied from the transformation matrix LUT331. base The input is the base transformation matrix T. base Transpose the result, and the transformation matrix T after transposition transpose The transformation matrix T H The result is output to the outside of the transformation matrix derivation unit 321 (supplied to the matrix operation unit 322).

[0283] As described above, the transformation matrix derivation unit 321 includes a flip unit 332 and a transpose unit 333. Therefore, the transformation matrix derivation unit 321 can, for example, use the flip unit 332 to realize the derivation example of the first row from the top of the table shown in Figure 10. Also, the transformation matrix derivation unit 321 can, for example, use the transpose unit 333 to realize the derivation example of the second row from the top of the table shown in Figure 10.

[0284] <Primary Vertical Transformer> Figure 25 is a block diagram showing an example of the main configuration of the primary vertical transformation unit 313 in Figure 22. As shown in Figure 25, the primary vertical transformation unit 313 includes a transformation matrix derivation unit 351, a matrix operation unit 352, a scaling unit 353, and a clipping unit 354.

[0285] The transformation matrix derivation unit 351 takes the transformation type identifier TrTypeIdxV of the primary vertical transformation and information regarding the size of the transformation block as input, and generates a transformation matrix T for the primary vertical transformation that corresponds to the transformation type identifier TrTypeIdxV of the primary vertical transformation and is the same size as the transformation block. V (Transformation matrix T for vertical one-dimensional orthogonal transformation) VThe transformation matrix derivation unit 351 derives the transformation matrix T. V This is supplied to the matrix operation unit 352.

[0286] The matrix operation unit 352 uses the transformation matrix T supplied from the transformation matrix derivation unit 351. V Using input data X in A vertical one-dimensional orthogonal transformation is performed on the transformation block of the transformation coefficient Coeff_Phor after the primary horizontal transformation (i.e., the transformation block of the transformation coefficient Coeff_Phor after the primary horizontal transformation) to obtain the intermediate data Y1. This operation can be expressed as a matrix as shown in equation (12) below.

[0287]

number

[0288] The matrix operation unit 352 supplies the intermediate data Y1 to the scaling unit 353.

[0289] The scaling unit 353 shifts the coefficients Y1[i,j] of each i-row, j-column component of the intermediate data Y1 by a predetermined shift amount S V We scale the data to obtain intermediate data Y2. This scaling can be expressed as shown in equation (13) below.

[0290]

number

[0291] The scaling unit 353 supplies the intermediate data Y2 to the clipping unit 354.

[0292] The clipping unit 354 clips the coefficient Y2[i,j] of each i row and j column component of the intermediate data Y2, and outputs the X out (That is, we derive the transformation coefficient Coeff_Pver after the primary vertical transformation.) This process can be expressed as shown in equation (14) below.

[0293]

number

[0294] The clip unit 324 outputs data X out The conversion coefficient Coeff_Pver (after primary vertical conversion) is output to the outside of the primary vertical conversion unit 313 as the conversion coefficient Coeff_P (supplied to the secondary conversion unit 153).

[0295] In the primary vertical transformation unit 313 configured as described above, the transformation matrix derivation unit 351 performs processing that applies the aforementioned technology as a derivation unit. Furthermore, the matrix operation unit 352 performs processing that applies the aforementioned technology as an orthogonal transformation unit. Therefore, the primary vertical transformation unit 313 can suppress the increase in memory capacity required for one-dimensional orthogonal transformations in the vertical direction.

[0296] <Transformation Matrix Derivation Section> Figure 26 is a block diagram showing an example of the main configuration of the transformation matrix derivation unit 351 in Figure 25. As shown in Figure 26, the transformation matrix derivation unit 351 includes a transformation matrix LUT 361, a flip unit 362, and a transpose unit 363. Although the arrows indicating data exchange are omitted in Figure 26, the transformation matrix derivation unit 351 can exchange any data between any processing units (processing blocks).

[0297] The transformation matrix LUT361 is a lookup table for holding (storing) transformation matrices corresponding to the transformation type identifier TrTypeIdxV and the size N of the transformation block of the primary vertical transformation. When the transformation type identifier TrTypeIdxV and the size N of the transformation block of the primary vertical transformation are specified, the transformation matrix LUT361 selects and outputs the corresponding transformation matrices. In this derivation example, the transformation matrix LUT361 uses its transformation matrix as the base transformation matrix T base It is supplied to the flip section 362, the transfer section 363, or both.

[0298] The flip unit 362 flips the input N x N transformation matrix T, and the resulting transformation matrix T flipThe output is... In this derivation example, the flip unit 362 outputs the N x N base transformation matrix T supplied from the transformation matrix LUT361. base The input is the base transformation matrix T. base Flip it in the row direction (horizontally), and the resulting transformation matrix T flip The transformation matrix T V The result is output to the outside of the transformation matrix derivation unit 351 (supplied to the matrix operation unit 352).

[0299] The transpose unit 363 transposes the input N x N transformation matrix T, and the transposed transformation matrix T transpose The output is... In this derivation example, the transpose unit 363 outputs the N x N base transformation matrix T supplied from the transformation matrix LUT361. base The input is the base transformation matrix T. base Transpose the result, and the transformation matrix T after transposition transpose The transformation matrix T V The result is output to the outside of the transformation matrix derivation unit 351 (supplied to the matrix operation unit 352).

[0300] As described above, the transformation matrix derivation unit 351 includes a flip unit 362 and a transpose unit 363. Therefore, the transformation matrix derivation unit 351 can, for example, use the flip unit 362 to realize the derivation example of the first row from the top of the table shown in Figure 10. Also, the transformation matrix derivation unit 351 can, for example, use the transpose unit 363 to realize the derivation example of the second row from the top of the table shown in Figure 10.

[0301] <Primary conversion process flow> Next, we will explain an example of the processing flow performed by the above-described configuration. In this case, an example of the primary conversion processing flow performed in step S132 of Figure 14 will be explained with reference to the flowchart in Figure 27.

[0302] When the primary conversion process is started, the primary conversion selection unit 311 (Figure 22) of the primary conversion unit 152 selects the conversion type identifier TrTypeIdxH (and the conversion type TrTypeH) for the primary horizontal conversion and the conversion type identifier TrTypeIdxV (and the conversion type TrTypeV) for the primary vertical conversion, respectively, in step S301, as described above.

[0303] In step S302, the primary horizontal transformation unit 312 performs a primary horizontal transformation process on the predicted residual D corresponding to the transformation type identifier TrTypeIdxH of the primary horizontal transformation obtained in step S301, and derives the transformation coefficient Coeff_Phor after the primary horizontal transformation.

[0304] In step S303, the primary vertical transformation unit 313 performs primary vertical transformation processing on the primary horizontal transformation result (transformation coefficient Coeff_Phor after primary horizontal transformation) corresponding to the primary vertical transformation type identifier TrTypeIdxV obtained in step S301, and derives the transformation coefficient Coeff_Pver (transformation coefficient Coeff_P after primary transformation) after primary vertical transformation.

[0305] Once the processing in step S303 is complete, the primary conversion process ends, and the process returns to Figure 14.

[0306] In the primary conversion process described above, the process applying the aforementioned technology is performed as steps S302 and S303. Therefore, by executing this primary conversion process, it is possible to suppress the increase in memory capacity required for the primary horizontal conversion process and the primary vertical conversion process.

[0307] <Primary Horizontal Transformation Processing Flow> The flow of the primary horizontal transformation process performed in step S302 of Figure 27 will be explained with reference to the flowchart in Figure 28.

[0308] When the primary horizontal transformation process is started, the transformation matrix derivation unit 321 (Figure 23) of the primary horizontal transformation unit 312 calculates the transformation matrix T corresponding to the transformation type identifier TrTypeIdxH (or transformation type TrTypeH) of the primary horizontal transformation in step S321. H Derive the following.

[0309] In step S322, the matrix operation unit 322 performs the derived transformation matrix T H Using input data X in A horizontal one-dimensional orthogonal transformation is performed on the (predicted residual D) to obtain intermediate data Y1. This process can be expressed as a matrix, as shown in equation (9) above. Alternatively, this process can be expressed as an element-wise operation, as shown in equation (15) below.

[0310]

number

[0311] In other words, as shown in Figure 29, the coefficient Y1[i,j] of the i-th row and j-th column component of the intermediate data Y1 is the input data X in Row vector X of the i-th row in [i,:] and the transformation matrix T H Row vector T of the jth row H The transpose matrix T of [j,:] H T Set the dot product with [:,j] (j=0,···,M-1, i=0,···,N-1). Here, M is the input data X in The size in the x-direction is N, where N is the input data X in This is the size in the y direction. M and N can be expressed as shown in equation (16) below.

[0312]

number

[0313] Returning to Figure 28, in step S323, the scaling unit 323 shifts the coefficients Y1[i,j] of each i-row, j-column component of the intermediate data Y1 derived by the processing in step S322 by a shift amount S H We scale the data and derive the intermediate data Y2. This scaling can be expressed as shown in equation (10) above.

[0314] In step S324, the clipping unit 324 clips the coefficient Y2[i,j] of each i row and j column component of the intermediate data Y2 derived by the processing in step S323, and outputs the X out (That is, we obtain the transformation coefficient Coeff_Phor after the primary horizontal transformation.) This process can be expressed as shown in equation (11) above.

[0315] Once the processing in step S324 is complete, the primary horizontal transformation process ends, and the process returns to Figure 27.

[0316] In the primary horizontal transformation process described above, the process applying the aforementioned technology is performed as steps S321 and S322. Therefore, by executing this primary horizontal transformation process, the increase in memory capacity required for the one-dimensional orthogonal transformation in the horizontal direction can be suppressed.

[0317] <Flow of transformation matrix derivation process> Next, an example of the flow of the transformation matrix derivation process performed in step S321 of Figure 28 will be explained with reference to the flowchart in Figure 30.

[0318] When the transformation matrix derivation process is started, the transformation matrix derivation unit 321, in step S341, obtains the base transformation type BaseTrType corresponding to the transformation type identifier TrTypeIdxH by referring to a correspondence table such as the one shown in Figure 31. This process can be expressed mathematically as, for example, equation (17). Furthermore, the N x N transformation matrix of the obtained base transformation type is read from the transformation matrix LUT and converted into the base transformation matrix T as shown in equation (18) below. base Set to this.

[0319]

number

[0320] Furthermore, the transformation matrix derivation unit 321 sets the flip flag FlipFlag to a value corresponding to the transformation type identifier TrTypeIdxH, as shown in equation (19) below. Also, the transformation matrix derivation unit 321 sets the transpose flag TransposeFlag to a value corresponding to the transformation type identifier TrTypeIdxH, as shown in equation (20) below.

[0321]

number

[0322] In step S342, the transformation matrix derivation unit 321 determines whether the flip flag and the transpose flag satisfy the condition (Condition A1) expressed by the following equation (21).

[0323]

number

[0324] If it is determined that the above condition (ConditionA1) is met (i.e., both the flip flag and the transpose flag are false (0)), the process proceeds to step S343.

[0325] In step S343, the transformation matrix derivation unit 321 calculates the base transformation matrix T as shown in equation (22) below. base Transform matrix T H Set it to [this].

[0326]

number

[0327] When the process of step S343 ends, the conversion matrix derivation process ends, and the process returns to FIG. 28. Also, in step S342, when it is determined that the above-mentioned condition (ConditionA1) is not satisfied (when the flip flag FlipFlag or the transpose flag TransposeFlag is true (1)), the process proceeds to step S344.

[0328] In step S344, the conversion matrix derivation unit 321 determines whether the flip flag FlipFlag and the transpose flag TransposeFlag satisfy the condition (ConditionA2) represented by the following equation (23).

[0329]

Equation

[0330] When it is determined that the above-mentioned condition (ConditionA2) is satisfied (when the flip flag FlipFlag is false (0) and the transpose flag TransposeFlag is true (1)), the process proceeds to step S345.

[0331] In step S345, the conversion matrix derivation unit 321 transposes the base conversion matrix T base through the transpose unit 333 to obtain the conversion matrix T H This process can be expressed as a determinant as shown in the following equation (24).

[0332]

Equation

[0333] Also, when this process is expressed as an element-by-element operation, the conversion matrix derivation unit 321 sets the i-th row and j-th column component ((i,j) component) of the base conversion matrix T base to the (j,i) component of the conversion matrix T H as shown in the following equation (25).

[0334]

Number

[0335] Here, the transformation matrix T of N rows and N columns H The i-th row and j-th column component ((i, j) component) of is denoted as T H [i, j]. Also, the "for i, j = 0,..., N - 1" in the second line indicates that i and j have values from 0 to N - 1. That is, T H [j, i] means indicating all the elements of the transformation matrix T of N rows and N columns H .

[0336] In this way, by expressing the processing of step S345 as an operation for each element, the transposition operation can be realized by accessing a simple two-dimensional array. When the processing of step S345 ends, the transformation matrix derivation processing ends, and the processing returns to FIG. 28.

[0337] Also, in step S344, when it is determined that the above-mentioned condition (ConditionA2) is not satisfied (when the flip flag FlipFlag is true (1), or when the transpose flag TransposeFlag is false (0)), the processing proceeds to step S346.

[0338] In step S346, the transformation matrix derivation unit 321 flips the base transformation matrix T base through the flip unit 332 to obtain the transformation matrix T H . This processing can be expressed as a determinant as shown in the following formula (26).

[0339]

Number

[0340] Here, × is an operator representing matrix multiplication. Also, the flip matrix J (Cross-Identity Matrix) is obtained by reversing the N - row and N - column identity matrix I left and right.

[0341] Furthermore, when this process is expressed as an element-wise operation, the transformation matrix derivation unit 321 calculates the transformation matrix T as shown in equation (27) below. H The i-th row and j-th column element ((i,j) element) of the base transformation matrix Tbase is set to the (i, N-1-j) element.

[0342]

number

[0343] Here, we have a transformation matrix T of N rows and N columns. H The i-th row and j-th column component ((i,j) component) of T H It is denoted as [i,j]. Also, the second line "for i,j=0,...,N-1" indicates that i and j each have values ​​from 0 to N-1. That is, T H [i,j] is a transformation matrix T with N rows and N columns. H This means showing all elements of [the object].

[0344] In this way, by representing the process in step S346 as an element-wise operation, the base transformation matrix T base The transpose operation can be achieved by accessing a simple two-dimensional array without performing matrix operations with the flip matrix J. Furthermore, the flip matrix J becomes unnecessary. When the processing in step S346 is completed, the transformation matrix derivation process is finished, and the process returns to Figure 28.

[0345] In the transformation matrix derivation process described above, the process applying the aforementioned technology is performed as steps S345 and S346. Therefore, by executing this transformation matrix derivation process, it is possible to realize the derivation examples of the first and second rows from the top of the table shown in Figure 10 for horizontal one-dimensional orthogonal transformations. Consequently, the increase in required memory capacity can be suppressed.

[0346] Furthermore, a branch as described below may be inserted between the processing in step S344 and the processing in step S346. That is, in that step, the transformation matrix derivation unit 321 determines whether the flip flag and the transpose flag satisfy the condition (Condition A3) expressed by the following equation (28).

[0347]

number

[0348] If the transformation matrix derivation unit 321 determines that the above-mentioned condition (Condition A3) is met (i.e., the flip flag FlipFlag is true (1) and the transpose flag TransposeFlag is false (0)), the process proceeds to step S346.

[0349] Furthermore, if it is determined that the above condition (Condition A3) is not met (i.e., the flip flag is false (0) or the transpose flag is true (1)), the transformation matrix derivation process ends and the process returns to Figure 28.

[0350] <Primary Vertical Transformation Process Flow> Next, the flow of the primary vertical transformation process performed in step S303 of Figure 27 will be explained with reference to the flowchart in Figure 32.

[0351] When the primary vertical transformation process is started, the transformation matrix derivation unit 351 (Figure 25) of the primary vertical transformation unit 313 performs the transformation matrix derivation process in step S361, and generates the transformation matrix T corresponding to the transformation type identifier TrTypeIdxV (or transformation type TrTypeV) of the primary vertical transformation. V Derive the following.

[0352] The process for deriving this transformation matrix is ​​the same as that for the primary horizontal transformation, as explained with reference to the flowchart in Figure 30, so its explanation will be omitted. For example, the transformation type identifier TrTypeIdxH for the primary horizontal transformation is replaced with the transformation type identifier TrTypeIdxV for the primary vertical transformation, or the transformation matrix T for the derived primary horizontal transformation is obtained. H The transformation matrix T for the primary vertical transformation is used. V You can simply replace the explanation regarding the horizontal direction in the explanation given with reference to Figure 30, for example, by replacing it with the explanation regarding the vertical direction.

[0353] In step S362, the matrix operation unit 352 performs the derived transformation matrix T V Using input data X in A one-dimensional orthogonal transformation is performed vertically on (the transformation coefficient Coeff_Phor after the primary horizontal transformation) to obtain the intermediate data Y1. This process can be expressed as a matrix, as shown in equation (12) above. Furthermore, this process can be expressed as an element-wise operation, as shown in equation (29) below.

[0354]

number

[0355] In other words, in this case, as shown in Figure 33, the coefficient Y1[i,j] of the i-th row and j-th column component of the intermediate data Y1 is the transformation matrix T V Row vector T of the i-th row V [i,:] and input data X in The column vector X of the jth column. in Set the inner product with [:,j] (j=0,···,M-1, i=0,···,N-1).

[0356] In step S363, the scaling unit 353 shifts the coefficients Y1[i,j] of each i-row, j-column component of the intermediate data Y1 derived by the processing in step S322 by a shift amount S V We scale the data and derive the intermediate data Y2. This scaling can be expressed as shown in equation (13) above.

[0357] In step S364, the clipping unit 354 clips the coefficient Y2[i,j] values ​​of each i row and j column component of the intermediate data Y2 derived by the processing in step S363, and outputs the X out (That is, we obtain the conversion coefficient Coeff_Pver after the primary vertical transformation.) This process can be expressed as shown in equation (14) above.

[0358] Once step S364 is completed, the primary horizontal transformation process finishes, and the process returns to Figure 27.

[0359] In the primary vertical transformation process described above, the process applying the aforementioned technology is performed as steps S361 and S362. Therefore, by executing this primary vertical transformation process, the increase in memory capacity required for the one-dimensional orthogonal transformation in the vertical direction can be suppressed.

[0360] <Inverse Primary Transformer> Next, the configuration of the image decoding device 200 in this embodiment will be described. Figure 34 is a block diagram showing an example of the main configuration of the inverse primary conversion unit 253 (Figure 16) in this case. As shown in Figure 34, the inverse primary conversion unit 253 has an inverse primary conversion selection unit 411, an inverse primary vertical conversion unit 412, and an inverse primary horizontal conversion unit 413.

[0361] The inverse primary transformation selection unit 411 takes the prediction mode information PInfo, component identifier compID, adaptive primary transformation flag apt_flag[compID], and primary transformation identifier pt_idx[compID] as inputs. The inverse primary transformation selection unit 411 refers to this information and derives the transformation type identifier TrTypeIdxV and the transformation type identifier TrTypeIdxH for the inverse primary vertical transformation. The inverse primary transformation selection unit 411 supplies the derived transformation type identifier TrTypeIdxV for the inverse primary vertical transformation to the inverse primary vertical transformation unit 412. The inverse primary transformation selection unit 411 also supplies the derived transformation type identifier TrTypeIdxH for the inverse primary horizontal transformation to the inverse primary horizontal transformation unit 413.

[0362] The inverse primary vertical transformation unit 412 takes as input the transformation coefficient Coeff_IS after the inverse secondary transformation, the transformation type identifier TrTypeIdxV for the inverse primary vertical transformation, and information regarding the size of the transformation block. This information regarding the size of the transformation block may be a natural number N indicating the horizontal or vertical size (number of coefficients) of the transformation block, or it may be log2TBHSize (logarithm of the vertical width) indicating the vertical width of the transformation block (N = 1 << log2TBHSize). The inverse primary vertical transformation unit 412 performs an inverse primary vertical transformation IPver on the transformation coefficient Coeff_IS after the inverse secondary transformation, which is determined by the transformation type identifier TrTypeIdxV and the size of the transformation block, and derives the transformation coefficient Coeff_IPver after the inverse primary vertical transformation. The inverse primary vertical transformation unit 412 supplies the transformation coefficient Coeff_IPver after the inverse primary vertical transformation to the inverse primary horizontal transformation unit 413.

[0363] The inverse primary horizontal transformation unit 413 takes as input the transformation coefficient Coeff_IPver after inverse primary vertical transformation, the transformation type identifier TrTypeIdxH for inverse primary horizontal transformation, and information regarding the size of the transformation block. This information regarding the size of the transformation block may be a natural number N indicating the horizontal or vertical size (number of coefficients) of the transformation block, or it may be log2TBWSize (logarithm of the width) indicating the width of the transformation block (N = 1 << log2TBWSize). The inverse primary horizontal transformation unit 413 performs the inverse primary horizontal transformation IPhor, which is determined by the transformation type identifier TrTypeIdxH and the size of the transformation block, on the transformation coefficient Coeff_IPver after inverse primary vertical transformation supplied from the inverse primary vertical transformation unit 412, and derives the transformation coefficient Coeff_IPhor (i.e., the transformation coefficient Coeff_IP after inverse primary transformation). The inverse primary horizontal transformation unit 413 outputs the transformation coefficient Coeff_IPhor after the inverse primary horizontal transformation as the predicted residual D' to the outside of the inverse primary transformation unit 253 (supplied to the calculation unit 215).

[0364] In the inverse primary transformer 253 configured as described above, the inverse primary vertical transformer 412 and the inverse primary horizontal transformer 413 perform processing to which the above-described technology is applied, as the derivation unit and the orthogonal transformer unit, respectively.

[0365] In other words, the inverse primary vertical transformation unit 412 derives a second transformation matrix for the inverse one-dimensional orthogonal transformation in the vertical direction as a derivation unit, and then performs the inverse one-dimensional orthogonal transformation in the vertical direction using the second transformation matrix for the inverse one-dimensional orthogonal transformation in the vertical direction derived by the derivation unit. Therefore, the inverse primary vertical transformation unit 412 can suppress the increase in memory capacity required for the inverse one-dimensional orthogonal transformation in the vertical direction.

[0366] Furthermore, the inverse primary horizontal transformation unit 413 derives a second transformation matrix for the horizontal inverse one-dimensional orthogonal transformation as a derivation unit, and then performs the horizontal inverse one-dimensional orthogonal transformation using the second transformation matrix for the horizontal inverse one-dimensional orthogonal transformation derived by the derivation unit as an orthogonal transformation unit. Therefore, the inverse primary horizontal transformation unit 413 can suppress the increase in memory capacity required for the horizontal inverse one-dimensional orthogonal transformation.

[0367] <Inverse Primary Vertical Transformer> Figure 35 is a block diagram showing an example of the main configuration of the inverse primary vertical transform unit 412 of Figure 34. As shown in Figure 35, the inverse primary vertical transform unit 412 includes a transform matrix derivation unit 421, a matrix operation unit 422, a scaling unit 423, and a clipping unit 424.

[0368] The transformation matrix derivation unit 421 takes the transformation type identifier TrTypeIdxV of the inverse primary vertical transformation and information about the size of the transformation block as input, and generates a transformation matrix T for the inverse primary vertical transformation that corresponds to the transformation type identifier TrTypeIdxV of the inverse primary vertical transformation and is the same size as the transformation block. V (Transformation matrix T for inverse 1D orthogonal transformation in the vertical direction) V The transformation matrix derivation unit 421 derives the transformation matrix T. V This is supplied to the matrix operation unit 422.

[0369] The matrix operation unit 422 receives the transformation matrix T from its transformation matrix derivation unit 421. V Using input data X in A vertical inverse one-dimensional orthogonal transformation is performed on the transformation block of the transformation coefficients Coeff_IS after the inverse secondary transformation (i.e., the transformation block of the transformation coefficients Coeff_IS after the inverse secondary transformation) to obtain the intermediate data Y1. This operation can be expressed as a matrix as shown in equation (30) below.

[0370]

number

[0371] The matrix operation unit 422 supplies the intermediate data Y1 to the scaling unit 423.

[0372] The scaling unit 423 shifts the coefficients Y1[i,j] of each i-row, j-column component of the intermediate data Y1 by a predetermined shift amount S IV The data is scaled to obtain intermediate data Y2. This scaling can be expressed as shown in equation (31) below.

[0373]

number

[0374] The scaling unit 423 supplies the intermediate data Y2 to the clipping unit 424.

[0375] The clipping unit 424 clips the coefficient Y2[i,j] of each i row and j column component of the intermediate data Y2, and outputs the X out (That is, the conversion coefficient Coeff_IPver after the inverse primary vertical transformation) is derived. This process can be expressed as shown in equation (11) above.

[0376] The clip unit 424 outputs data X out The conversion coefficient Coeff_IPver (after inverse primary vertical conversion) is output to the outside of the inverse primary vertical conversion unit 412 (supplied to the inverse primary horizontal conversion unit 413).

[0377] In the inverse primary vertical transformation unit 412 configured as described above, the transformation matrix derivation unit 421 performs processing that applies the aforementioned technology as a derivation unit. The matrix operation unit 422 also performs processing that applies the aforementioned technology as an orthogonal transformation unit. Therefore, the inverse primary vertical transformation unit 412 can suppress the increase in memory capacity required for inverse one-dimensional orthogonal transformation in the vertical direction.

[0378] <Transformation Matrix Derivation Section> Figure 36 is a block diagram showing an example of the main configuration of the transformation matrix derivation unit 421 in Figure 35. As shown in Figure 36, the transformation matrix derivation unit 421 includes a transformation matrix LUT 431, a flip unit 432, and a transpose unit 433. Although the arrows indicating data exchange are omitted in Figure 36, the transformation matrix derivation unit 421 can exchange any data between any processing units (processing blocks).

[0379] The transformation matrix LUT431 is a lookup table for holding (storing) transformation matrices corresponding to the transformation type identifier TrTypeIdxV and the transformation block size N of the inverse primary vertical transformation. When the transformation type identifier TrTypeIdxV and the transformation block size N of the inverse primary vertical transformation are specified, the transformation matrix LUT431 selects and outputs the corresponding transformation matrices. In this derivation example, the transformation matrix LUT431 uses its transformation matrix as the base transformation matrix T base It is supplied to the flip section 432, the transfer section 433, or both.

[0380] The flip unit 432 flips the input N x N transformation matrix T, and the resulting transformation matrix T flip The output is... In this derivation example, the flip unit 432 outputs the N x N base transformation matrix T supplied from the transformation matrix LUT 431. base The input is the base transformation matrix T. base Flip it in the row direction (horizontally), and the resulting transformation matrix T flip The transformation matrix T V The result is output to the outside of the transformation matrix derivation unit 321 (supplied to the matrix calculation unit 422).

[0381] The transpose unit 433 transposes the input N x N transformation matrix T, and the transposed transformation matrix T transpose The output is... In this derivation example, the transpose unit 433 outputs the N x N base transformation matrix T supplied from the transformation matrix LUT431. base The input is the base transformation matrix T. base Transpose the result, and the transformation matrix T after transposition transpose The transformation matrix T VThe result is output to the outside of the transformation matrix derivation unit 421 (supplied to the matrix operation unit 422).

[0382] As described above, the transformation matrix derivation unit 421 has a flip unit 432 and a transpose unit 433. Therefore, the transformation matrix derivation unit 421 can, for example, use the flip unit 432 to realize the derivation example of the first row from the top of the table shown in Figure 10. Also, the transformation matrix derivation unit 421 can, for example, use the transpose unit 433 to realize the derivation example of the second row from the top of the table shown in Figure 10.

[0383] <Inverse Primary Horizontal Conversion Unit> Figure 37 is a block diagram showing an example of the main configuration of the inverse primary horizontal transformation unit 413 in Figure 34. As shown in Figure 37, the inverse primary horizontal transformation unit 413 includes a transformation matrix derivation unit 451, a matrix operation unit 452, a scaling unit 453, and a clipping unit 454.

[0384] The transformation matrix derivation unit 451 takes the transformation type identifier TrTypeIdxH of the inverse primary horizontal transformation and information about the size of the transformation block as input, and generates a transformation matrix T for the inverse primary horizontal transformation that corresponds to the transformation type identifier TrTypeIdxH of the inverse primary horizontal transformation and is the same size as the transformation block. H (Transformation matrix T for inverse 1D orthogonal transformation in the horizontal direction) H The transformation matrix derivation unit 451 derives the transformation matrix T. H This is supplied to the matrix operation unit 452.

[0385] The matrix operation unit 452 uses the transformation matrix T supplied from the transformation matrix derivation unit 451. H Using input data X in A horizontal inverse 1D orthogonal transformation is performed on the transformation block of the transformation coefficient Coeff_IPver after the inverse primary vertical transformation (i.e., the transformation block), and intermediate data Y1 is obtained. This operation can be expressed as a matrix as shown in equation (32) below.

[0386]

number

[0387] The matrix operation unit 452 supplies the intermediate data Y1 to the scaling unit 453.

[0388] The scaling unit 453 shifts the coefficients Y1[i,j] of each i-row, j-column component of the intermediate data Y1 by a predetermined shift amount S IH We scale the data to obtain intermediate data Y2. This scaling can be expressed as shown in equation (33) below.

[0389]

number

[0390] The scaling unit 453 supplies the intermediate data Y2 to the clipping unit 454.

[0391] The clipping unit 454 clips the coefficient Y2[i,j] of each i row and j column component of the intermediate data Y2, and outputs the data X out (That is, the transformation coefficient Coeff_IPhor after the inverse primary horizontal transformation) is derived. This process can be expressed as shown in equation (11) above.

[0392] The clip unit 454 outputs data X out The conversion coefficient Coeff_IPhor (conversion coefficient Coeff_IP after inverse primary conversion) is output to the outside of the inverse primary horizontal conversion unit 413 as the predicted residual D' (supplied to the calculation unit 215).

[0393] In the inverse primary horizontal transformation unit 413 configured as described above, the transformation matrix derivation unit 451 performs processing that applies the aforementioned technology as a derivation unit. The matrix operation unit 452 also performs processing that applies the aforementioned technology as an inverse orthogonal transformation unit. Therefore, the inverse primary horizontal transformation unit 413 can suppress the increase in memory capacity required for inverse one-dimensional orthogonal transformation in the horizontal direction.

[0394] <Transformation Matrix Derivation Section> Figure 38 is a block diagram showing an example of the main configuration of the transformation matrix derivation unit 451 in Figure 37. As shown in Figure 38, the transformation matrix derivation unit 451 has a transformation matrix LUT 461, a flip unit 462, and a transpose unit 463. Although the arrows indicating data exchange are omitted in Figure 38, the transformation matrix derivation unit 451 can exchange any data between any processing units (processing blocks).

[0395] The transformation matrix LUT461 is a lookup table for holding (storing) transformation matrices corresponding to the transformation type identifier TrTypeIdxH and the transformation block size N of the inverse primary horizontal transformation. When the transformation type identifier TrTypeIdxH and the transformation block size N of the inverse primary horizontal transformation are specified, the transformation matrix LUT461 selects and outputs the corresponding transformation matrices. In this derivation example, the transformation matrix LUT461 uses its transformation matrix as the base transformation matrix T base It is supplied to the flip section 462, the transfer section 463, or both.

[0396] The flip unit 462 flips the input N x N transformation matrix T, and the resulting transformation matrix T flip The output is... In this derivation example, the flip unit 462 outputs the N x N base transformation matrix T supplied from the transformation matrix LUT 461. base The input is the base transformation matrix T. base Flip it in the row direction (horizontally), and the resulting transformation matrix T flip The transformation matrix T H The result is output to the outside of the transformation matrix derivation unit 451 (supplied to the matrix operation unit 452).

[0397] The transpose unit 463 transposes the input N x N transformation matrix T, and the transposed transformation matrix T transpose The output is... In this derivation example, the transpose unit 463 outputs the N x N base transformation matrix T supplied from the transformation matrix LUT461. base The input is the base transformation matrix T. base Transpose the result, and the transformation matrix T after transposition transpose The transformation matrix TH The result is output to the outside of the transformation matrix derivation unit 451 (supplied to the matrix operation unit 452).

[0398] As described above, the transformation matrix derivation unit 451 has a flip unit 462 and a transpose unit 463. Therefore, the transformation matrix derivation unit 451 can, for example, use the flip unit 462 to realize the derivation example of the first row from the top of the table shown in Figure 10. Also, the transformation matrix derivation unit 451 can, for example, use the transpose unit 463 to realize the derivation example of the second row from the top of the table shown in Figure 10.

[0399] <Flow of inverse primary transformation process> Next, we will describe an example of the processing flow performed by the image decoding device 200 with the above-described configuration. In this case, an example of the inverse primary transformation process performed in step S233 of Figure 18 will be explained with reference to the flowchart in Figure 39.

[0400] When the inverse primary conversion process is started, the inverse primary conversion selection unit 411 (Figure 34) of the inverse primary conversion unit 253 performs the inverse primary conversion selection process in step S401 and selects the conversion type identifier TrTypeIdxV (or conversion type TrTypeV) for the inverse primary vertical conversion and the conversion type identifier TrTypeIdxH (or conversion type TrTypeH) for the inverse primary horizontal conversion, respectively.

[0401] In step S402, the inverse primary vertical transformation unit 412 performs an inverse primary vertical transformation process on the transformation coefficient Coeff_IS after the inverse secondary transformation, corresponding to the transformation type identifier TrTypeIdxV of the inverse primary vertical transformation obtained in step S401, and derives the transformation coefficient Coeff_IPver after the inverse primary vertical transformation.

[0402] In step S403, the inverse primary horizontal transformation unit 413 performs an inverse primary horizontal transformation process corresponding to the transformation type identifier TrTypeIdxH of the inverse primary horizontal transformation obtained in step S401 on the transformation coefficient Coeff_IPver after the inverse primary vertical transformation, and derives the transformation coefficient Coeff_IPhor after the inverse primary horizontal transformation (i.e., the transformation coefficient Coeff_IP (predicted residual D') after the inverse primary transformation).

[0403] Once the processing in step S403 is complete, the inverse primary transformation process is finished, and the process returns to Figure 18.

[0404] In the inverse primary conversion process described above, the process applying the aforementioned technology is performed as steps S402 and S403. Therefore, by executing this inverse primary conversion process, it is possible to suppress the increase in memory capacity required for the inverse primary vertical conversion process and the inverse primary horizontal conversion process.

[0405] <Flowchart of the inverse primary transformation selection process> Next, an example of the flow of the inverse primary transformation selection process performed in step S401 of Figure 39 will be explained with reference to the flowchart in Figure 40.

[0406] When the reverse primary transformation selection process is started, the reverse primary transformation selection unit 411 determines in step S421 whether the adaptive primary transformation flag apt_flag[compID] of the component identifier compID is true (1). If it is determined that the adaptive primary transformation flag apt_flag[compID] of the component identifier compID is true (1), the process proceeds to step S422.

[0407] In step S422, the inverse primary conversion selection unit 411 selects conversion sets TrSetH and TrSetV for each direction from the conversion set group based on the prediction mode information Pinfo.

[0408] In step S423, the inverse primary transformation selection unit 411 derives the transformation type identifier TrTypeIdxH for the inverse primary horizontal transformation based on the transformation set TrSetH and the primary transformation identifier pt_idx[compID]. This process can be expressed, for example, as shown in equation (34) below.

[0409]

number

[0410] In step S424, the inverse primary transformation selection unit 411 derives the transformation type identifier TrTypeIdxV for the inverse primary vertical transformation based on the transformation set TrSetV and the inverse primary transformation identifier pt_idx[compID]. This process can be expressed, for example, as shown in equation (35) below.

[0411]

number

[0412] Once step S424 is completed, the inverse primary transformation selection process ends, and the process returns to Figure 39.

[0413] Furthermore, if in step S421 the adaptive primary conversion flag apt_flag[compID] of the component identifier compID is determined to be false (0), the process proceeds to step S425.

[0414] In step S425, the inverse primary transformation selection unit 411 sets (selects) a predetermined orthogonal transformation (e.g., DCT2) for the transformation type identifier TrTypeIdxH of the inverse primary horizontal transformation. This process can be expressed, for example, as shown in equation (36) below.

[0415]

number

[0416] In step S426, the inverse primary transformation selection unit 411 sets (selects) a predetermined orthogonal transformation (e.g., DCT2) to the transformation type identifier TrTypeIdxV of the inverse primary vertical transformation. This process can be expressed, for example, as shown in equation (37) below.

[0417]

number

[0418] Once the processing in step S426 is complete, the inverse primary transformation selection process ends, and the process returns to Figure 39.

[0419] <Flowchart of the inverse primary vertical transform process> Next, the flow of the inverse primary vertical transformation process performed in step S402 of Figure 39 will be explained with reference to the flowchart in Figure 41.

[0420] When the inverse primary vertical transformation process is started, the transformation matrix derivation unit 421 (Figure 35) of the inverse primary vertical transformation unit 412 performs the transformation matrix derivation process in step S441, and generates the transformation matrix T corresponding to the transformation type identifier TrTypeIdxV of the inverse primary vertical transformation. V Derive the following.

[0421] The transformation matrix derivation process in this case follows the same procedure as for the primary horizontal transformation, as explained with reference to the flowchart in Figure 30. Therefore, the explanation is omitted. For example, the transformation type identifier TrTypeIdxH of the primary horizontal transformation is replaced with the transformation type identifier TrTypeIdxV of the inverse primary vertical transformation, and the resulting transformation matrix T for the primary horizontal transformation is derived. H The transformation matrix T for the inverse primary vertical transformation is used. V By making substitutions, the explanation given with reference to Figure 30 can be applied as an explanation of the transformation matrix derivation process in this case.

[0422] In step S442, the matrix operation unit 422 performs the derived transformation matrix TV Using input data X in (That is, an inverse one-dimensional orthogonal transformation is performed perpendicular to the transformation coefficients Coeff_IS after the inverse secondary transformation) to obtain the intermediate data Y1. When this process is expressed as a matrix, it can be represented as shown in equation (30) above.

[0423] In step S443, the scaling unit 423 shifts the coefficients Y1[i,j] of each i-row, j-column component of the intermediate data Y1 derived by the processing in step S442 by a shift amount S IV We scale the data and derive the intermediate data Y2. This scaling can be expressed as shown in equation (31) above.

[0424] In step S444, the clipping unit 424 clips the coefficient Y2[i,j] of each i row and j column component of the intermediate data Y2 derived by the processing in step S443, and outputs the X out (That is, we obtain the conversion coefficient Coeff_IPver after the inverse primary vertical transformation.) This process can be expressed as shown in equation (11) above.

[0425] Once step S444 is complete, the inverse primary vertical transformation process is finished, and the process returns to Figure 39.

[0426] In the inverse primary vertical transformation process described above, the process applying the aforementioned technology is performed as steps S441 and S442. Therefore, by executing this inverse primary vertical transformation process, the increase in memory capacity required for the inverse one-dimensional orthogonal transformation in the vertical direction can be suppressed.

[0427] <Flowchart of Inverse Primary Horizontal Transformation Process> Next, the flow of the inverse primary horizontal transformation process performed in step S403 of Figure 39 will be explained with reference to the flowchart in Figure 42.

[0428] When the inverse primary horizontal transformation process is started, the transformation matrix derivation unit 451 (Figure 37) of the inverse primary horizontal transformation unit 413 performs the transformation matrix derivation process in step S461, and generates the transformation matrix T corresponding to the transformation type identifier TrTypeIdxH of the inverse primary horizontal transformation. H Derive the following.

[0429] The transformation matrix derivation process in this case follows the same procedure as for the primary horizontal transformation, as explained with reference to the flowchart in Figure 30. Therefore, the explanation is omitted. For example, by replacing the primary horizontal transformation with the inverse primary horizontal transformation, the explanation given with reference to Figure 30 can be applied as an explanation of the transformation matrix derivation process in this case.

[0430] In step S462, the matrix operation unit 452 performs the derived transformation matrix T H Using input data X in (That is, the transformation coefficient Coeff_IPver after the inverse primary vertical transformation) is subjected to an inverse one-dimensional orthogonal transformation in the horizontal direction to obtain the intermediate data Y1. When this process is expressed as a matrix, it can be represented as shown in equation (32) above.

[0431] In step S463, the scaling unit 453 shifts the coefficients Y1[i,j] of each i-row, j-column component of the intermediate data Y1 derived by the processing in step S462 by a shift amount S IH We scale the data and derive the intermediate data Y2. This scaling can be expressed as shown in equation (33) above.

[0432] In step S464, the clipping unit 454 clips the coefficient Y2[i,j] of each i row and j column component of the intermediate data Y2 derived by the processing in step S463, and outputs the X out (That is, the transformation coefficient Coeff_IPhor after the inverse primary horizontal transformation is obtained.) This process can be expressed as shown in equation (14) above.

[0433] Once step S464 is completed, the inverse primary horizontal transformation process is finished, and the process returns to Figure 39.

[0434] In the inverse primary horizontal transformation process described above, the process applying the aforementioned technology is performed as steps S461 and S462. Therefore, by executing this inverse primary horizontal transformation process, the increase in memory capacity required for the inverse one-dimensional orthogonal transformation in the horizontal direction can be suppressed.

[0435] <2-3. Example 1-2> <Concept> Next, we will explain the example in the third row from the top, excluding the top row of item names, in the table shown in Figure 10.

[0436] As described above, the derivation of the third example from the top focuses on the characteristics between pairs of DCT / DST. More specifically, it focuses on the fact that between pairs of DCT / DST (for example, DST7 and DCT8), even-numbered row vectors are axisymmetric, and odd-numbered row vectors are point-symmetric. In this case, the derivation unit flips the first transformation matrix, reverses the sign of the odd-numbered row vectors in the flipped first transformation matrix, and derives the second transformation matrix. That is, as shown in Figure 43, the derivation unit flips the transformation matrix of DST7 in the row direction, and further reverses the sign of the odd-numbered row vectors to derive the transformation matrix of DCT8 losslessly. Therefore, naturally, the derived DCT8 can be used to substitute the transformation matrix of DCT8 with the same (decreasing) waveform of the 0th row vector.

[0437] A concrete example of this derivation is shown in Figure 44. As shown in Figure 44, this derivation is based on an identity matrix D with negative elements in odd-numbered rows and a base transformation matrix T base It can be expressed by the matrix product of (DST7) and the flip matrix J.

[0438] In other words, this derivation example allows us to derive the second transformation matrix in just two operations (flipping and sign reversal). Furthermore, each operation is straightforward. Therefore, the second transformation matrix can be easily derived.

[0439] Furthermore, by applying this derivation example, it becomes unnecessary to prepare DCT8 transformation matrices as candidates for transformation matrices used in orthogonal and inverse orthogonal transformations. In other words, the number of intrinsic transformation types can be reduced.

[0440] In this case, as shown in the table in Figure 45, the number of transformation matrices required, which was five in the case of the technology described in Non-Patent Literature 1 (see table in Figure 6A), can be reduced to four, so the total LUT size can be reduced to approximately 47KB. In other words, the LUT size can be reduced by approximately 53KB (table in Figure 6A) compared to the case of the technology described in Non-Patent Literature 1. That is, the increase in LUT size can be suppressed.

[0441] As mentioned above, in this case, a transformation matrix of transformation type DCT8 can be obtained as a second transformation type. By performing orthogonal and inverse orthogonal transformations using this second transformation matrix, the same coding efficiency as when using the DCT8 transformation matrix for orthogonal and inverse orthogonal transformations can be obtained.

[0442] <Transformation Matrix Derivation Section> Next, we will describe the configuration and processing for performing such derivations. First, we will describe the technical configuration of this embodiment of the image encoding device 100. In this case, the configuration of the primary conversion unit 152, primary horizontal conversion unit 312, and primary vertical conversion unit 313, etc., is the same as in the case of <2-2. Example 1-1> which was explained with reference to Figure 22, so we will omit the explanation.

[0443] Figure 46 is a block diagram showing an example of the main configuration of the transformation matrix derivation unit 321 (transformation matrix derivation unit 321 within the primary horizontal transformation unit 312 (Figure 23)) in this case. As shown in Figure 46, the transformation matrix derivation unit 321 in this case has a transformation matrix LUT 331, a flip unit 332, and a sign inversion unit 501. Although the arrows indicating data exchange are omitted in Figure 46, the transformation matrix derivation unit 321 can exchange any data between any processing units (processing blocks).

[0444] The transformation matrix LUT331 and the flip section 332 are the same as in the case of Figure 24.

[0445] The sign inversion unit 501 takes an N x N transformation matrix T as input, inverts the sign of a predetermined location in the transformation matrix T, and returns the sign-inverted transformation matrix T InvSign The output is as follows. In this derivation example (Example 1-2), the transformation matrix derivation unit 321 outputs the N x N base transformation matrix T selected in the transformation matrix LUT331. base The vectors are flipped horizontally via the flip unit 332, and the signs of the odd-order row vectors are reversed via the sign reversal unit 501, resulting in the transformation matrix T InvSign Transform matrix T H The result is output to the outside of the transformation matrix derivation unit 321 (supplied to the matrix operation unit 322).

[0446] As described above, the transformation matrix derivation unit 321 can use the flip unit 332 and the sign inversion unit 501 to realize the derivation example of the third row from the top of the table shown in Figure 10.

[0447] <Transformation Matrix Derivation Section> Figure 47 is a block diagram showing an example of the main configuration of the transformation matrix derivation unit 351 in the primary vertical transformation unit 313 in this case. As shown in Figure 47, the transformation matrix derivation unit 351 in this case has a transformation matrix LUT 361, a flip unit 362, and a sign inversion unit 502, similar to the transformation matrix derivation unit 321. Although the arrows indicating data exchange are omitted in Figure 47, the transformation matrix derivation unit 351 can exchange any data between any processing units (processing blocks).

[0448] The transformation matrix LUT361 and the flip section 362 are the same as in the case of Figure 26.

[0449] The sign inversion unit 502, similar to the sign inversion unit 501, takes an N x N transformation matrix T as input, inverts the sign of a predetermined location in the transformation matrix T, and returns the sign-inverted transformation matrix T InvSign The output is as follows: In this derivation example, the transformation matrix derivation unit 351 outputs the N x N base transformation matrix T selected in the transformation matrix LUT361. base The vectors are flipped horizontally via the flip unit 362, and the signs of the odd-order row vectors are reversed via the sign reversal unit 502, resulting in the transformation matrix T InvSign Transform matrix T V The result is output to the outside of the transformation matrix derivation unit 351 (supplied to the matrix operation unit 352).

[0450] As described above, the transformation matrix derivation unit 351 can use the flip unit 362 and the sign inversion unit 502 to realize the derivation example of the third row from the top of the table shown in Figure 10.

[0451] <Flow of transformation matrix derivation process> In this case, the primary transformation process, primary horizontal transformation process, and primary vertical transformation process are carried out in the same manner as in <2-2. Example 1-1>, so a description of these processes will be omitted.

[0452] Next, an example of the flow of the transformation matrix derivation process in this case, which is performed by the transformation matrix derivation unit 321 in the primary horizontal transformation unit 312 in step S321 of Figure 28, will be explained with reference to the flowchart in Figure 48.

[0453] When the transformation matrix derivation process is started, the transformation matrix derivation unit 321, in step S501, obtains the base transformation type BaseTrType corresponding to the transformation type identifier TrTypeIdxH by referring to a correspondence table such as the one shown in Figure 49. This process can be expressed mathematically as, for example, equation (17) above. Furthermore, the transformation matrix derivation unit 321 reads the N x N transformation matrix of the obtained base transformation type from the transformation matrix LUT and converts it into the base transformation matrix T as shown in equation (18) above. base Set to this.

[0454] Furthermore, the transformation matrix derivation unit 321 sets the flip flag FlipFlag to a value corresponding to the transformation type identifier TrTypeIdxH, as shown in equation (19) above. Also, the transformation matrix derivation unit 321 sets the sign inversion flag InvSignFlag to a value corresponding to the transformation type identifier TrTypeIdxH, as shown in equation (38) below.

[0455]

number

[0456] In step S502, the transformation matrix derivation unit 321 determines whether the flip flag and the transpose flag satisfy the condition (Condition B1) expressed by the following equation (39).

[0457]

number

[0458] If it is determined that the conditions shown in equation (39) above are not met (i.e., if it is determined that the flip flag FlipFlag is false (0) or the sign inversion flag InvSignFlag is false (0)), the process proceeds to step S503.

[0459] In step S503, the transformation matrix derivation unit 321 calculates the base transformation matrix Tbase Transform matrix T H Set to this value. Once the process in step S503 is complete, the transformation matrix derivation process is finished, and the process returns to Figure 28.

[0460] Furthermore, if it is determined in step S502 that the conditions shown in equation (39) above are met (i.e., if the flip flag FlipFlag is true (1) and the sign inversion flag InvSignFlag is true (1)), the process proceeds to step S504.

[0461] In step S504, the transformation matrix derivation unit 321 uses the flip unit 332 to determine the base transformation matrix T base Flip it and transform it into the transformation matrix T H Set to this. In other words, the transformation matrix derivation unit 321 is Flip(T base ) Transformation matrix T H Derive the following.

[0462] In step S505, the transformation matrix derivation unit 321 uses the sign inversion unit 501 to obtain the transformation matrix T H Reverse the sign of the odd-numbered row vectors, and transform matrix T H Reset it. This process can be expressed as a determinant, as shown in equation (40) below.

[0463]

number

[0464] Here, × is the operator representing matrix multiplication, and the sign-inverted matrix D is Diag(1,-1,…,(-1) N-1 It is a diagonal matrix consisting of ).

[0465] Furthermore, this process can also be expressed as an element-wise operation, as shown in equation (41) below. In this way, it can be implemented without having a sign-inverting matrix D. In that case, the transformation matrix derivation unit 321 is the transformation matrix T HIn the i-th row and j-th column component ((i,j) component), reverse the sign of the component in the odd-numbered row (i%2==1).

[0466]

number

[0467] Here, we have a transformation matrix T of N rows and N columns. H The i-th row and j-th column component ((i,j) component) of T H It is denoted as [i,j]. Also, the second line, "for i,j=0,...,N-1 and i%2 == 1", indicates that j has values ​​from 0 to N-1 and i has odd values ​​in the range of 0 to N-1. That is, T H [i,j] is a transformation matrix T with N rows and N columns. H This means showing each element of the odd-numbered row vector.

[0468] Once step S505 is completed, the transformation matrix derivation process finishes, and the process returns to Figure 28.

[0469] <Flow of transformation matrix derivation process> The flow of the transformation matrix derivation process performed in step S361 (Figure 32) of the primary vertical transformation process is the same as that of the transformation matrix derivation process performed in the primary horizontal transformation, as explained with reference to the flowchart in Figure 48, so its explanation will be omitted.

[0470] <Transformation Matrix Derivation Section> Next, the configuration of the image decoding device 200 in this case will be described. In this case as well, the configuration of the inverse primary conversion unit 253, inverse primary vertical conversion unit 412, inverse primary horizontal conversion unit 413, etc., of the image decoding device 200 is the same as in the case of <2-2. Example 1-1>, so its explanation will be omitted.

[0471] Figure 50 is a block diagram showing an example of the main configuration of the transformation matrix derivation unit 421 (the transformation matrix derivation unit 421 of the inverse primary vertical transformation unit 412) in this case. As shown in Figure 50, the transformation matrix derivation unit 421 in this case has a transformation matrix LUT 431, a flip unit 432, and a sign inversion unit 511, similar to the transformation matrix derivation unit 321. Although the arrows indicating data exchange are omitted in Figure 50, the transformation matrix derivation unit 421 can exchange any data between any processing units (processing blocks).

[0472] The transformation matrix LUT431 and the flip unit 432 are the same as in Figure 36. The sign inversion unit 511, like the sign inversion unit 501, takes an N x N transformation matrix T as input, inverts the sign of a predetermined location in the transformation matrix T, and converts the sign of the transformed matrix T InvSign The output is as follows: In this derivation example, the transformation matrix derivation unit 421 outputs the N x N base transformation matrix T selected in the transformation matrix LUT431. base The vectors are flipped horizontally via the flip unit 432, and the signs of the odd-order row vectors are reversed via the sign reversal unit 511, resulting in the transformation matrix T InvSign Transform matrix T V The result is output to the outside of the transformation matrix derivation unit 421 (supplied to the matrix operation unit 422).

[0473] As described above, the transformation matrix derivation unit 421 can use the flip unit 432 and the sign inversion unit 511 to realize the derivation example of the third row from the top of the table shown in Figure 10.

[0474] <Transformation Matrix Derivation Section> Figure 51 is a block diagram showing an example of the main configuration of the transformation matrix derivation unit 451 of the inverse primary horizontal transformation unit 413 in this case. As shown in Figure 51, the transformation matrix derivation unit 451 in this case has a transformation matrix LUT 461, a flip unit 462, and a sign inversion unit 512, similar to the transformation matrix derivation unit 421. Although the arrows indicating data exchange are omitted in Figure 51, the transformation matrix derivation unit 451 can exchange any data between any processing units (processing blocks).

[0475] The transformation matrix LUT461 and the flip unit 462 are the same as in Figure 50. The sign inversion unit 512, like the sign inversion unit 501, takes an N x N transformation matrix T as input, inverts the sign of a predetermined location in the transformation matrix T, and converts the sign of the transformed matrix T InvSign The output is as follows: In this derivation example, the transformation matrix derivation unit 451 outputs the N x N base transformation matrix T selected in the transformation matrix LUT461. base The vectors are flipped horizontally via the flip unit 462, and the signs of the odd-order row vectors are reversed via the sign reversal unit 512, resulting in the transformation matrix T InvSign Transform matrix T V The result is output to the outside of the transformation matrix derivation unit 451 (supplied to the matrix operation unit 452).

[0476] As described above, the transformation matrix derivation unit 451 can use the flip unit 462 and the sign inversion unit 512 to realize the derivation example of the third row from the top of the table shown in Figure 10.

[0477] <Flow of transformation matrix derivation process> Note that the transformation matrix derivation units 421 and 451 perform the transformation matrix derivation process in the same manner as described with reference to the flowchart in Figure 48, so their explanation will be omitted.

[0478] <2-4. Examples 1-3> <Concept> Next, we will explain the examples of the fourth and fifth rows from the top, excluding the top row of item names in the table shown in Figure 10.

[0479] As described above, the derivation of the fourth example from the top, like the first example from the top, focuses on the similarity between the waveform of the lowest-order row vector of the first transformation type and the waveform of the lowest-order row vector of the transformation type of the alternative transformation matrix. In this case, the derivation unit flips the first transformation matrix to derive the second transformation matrix. That is, as shown in Figure 52, the derivation unit uses the transformation matrix of DCT8 as the base transformation matrix T baseThen, by flipping it in the row direction, we derive the transformation matrix of FlipDCT8. This transformation matrix of FlipDCT8 can be used to substitute for the transformation matrix of DST7 (an augmentative type) whose waveform of the 0th row vector is similar.

[0480] Furthermore, the derivation of the fifth example from the top focuses on the similarity between the waveform of the highest-order column vector of the first transformation type and the waveform of the lowest-order row vector of the transformation type of the substitute transformation matrix. In this case, the derivation unit flips the first transformation matrix, transposes the flipped first transformation matrix, and derives the second transformation matrix. In other words, as shown in Figure 52, the derivation unit uses the transformation matrix of DCT8 as the base transformation matrix T base Then, by flipping it in the row direction and transposing it, we derive the transformation matrix TrFlipDCT8. This TrFlipDCT8 transformation matrix can be used to substitute for the transformation matrix of DST1, which has a similar (mountain-shaped) waveform for the 0th row vector.

[0481] In other words, the two derivation examples above allow the second transformation matrix to be derived in one or two operations (flip or flip + transpose). Furthermore, these operations are straightforward. That is, the second transformation matrix can be easily derived.

[0482] Furthermore, by applying the two derivation examples above, it becomes unnecessary to prepare the DST7 and DST1 transformation matrices as candidates for the transformation matrices used in orthogonal and inverse orthogonal transformations. In other words, the number of intrinsic transformation types can be reduced.

[0483] In this case, as shown in the table in Figure 53, the number of transformation matrices required, which was five in the case of the technology described in Non-Patent Literature 1 (see table in Figure 6A), can be reduced to three, so the total LUT size can be reduced to approximately 40KB. In other words, the LUT size can be reduced by approximately 53KB (table in Figure 6A) compared to the case of the technology described in Non-Patent Literature 1. That is, the increase in LUT size can be suppressed.

[0484] As mentioned above, in this case as well, by performing orthogonal and inverse orthogonal transformations using the transformation matrices of the derived second transformation type (FlipDCT8 or TrFlipDCT8), coding efficiency comparable to that obtained when using the DST7 or DST1 transformation matrices for orthogonal and inverse orthogonal transformations can be obtained.

[0485] <Structure> The configurations of the image encoding device 100 and the image decoding device 200 in this case are the same as those described in <2-2. Example 1-1>, so their description will be omitted.

[0486] <Flow of transformation matrix derivation process> Next, we will explain the processing flow. Processing other than the transformation matrix derivation process is carried out in the same manner as in the example described above in <2-2. Example 1-1>, etc., so we will omit their explanation.

[0487] In step S321 of Figure 28, an example of the flow of the transformation matrix derivation process performed by the transformation matrix derivation unit 321 of the primary horizontal transformation unit 312 will be explained with reference to the flowchart in Figure 54.

[0488] When the transformation matrix derivation process is started, the transformation matrix derivation unit 321, in step S521, determines the base transformation type BaseTrType corresponding to the transformation type identifier TrTypeIdxH by referring to a correspondence table such as the one shown in Figure 55.

[0489] Furthermore, the transformation matrix derivation unit 321 sets the value of the flip flag FlipFlag corresponding to the transformation type identifier TrTypeIdxH and the value of the transpose flag TransposeFlag corresponding to the transformation type identifier TrTypeIdxH, similar to the case of step S341 in the flowchart of Figure 30.

[0490] In step S522, the transformation matrix derivation unit 321 determines whether the flip flag satisfies the condition (Condition C1) expressed by the following equation (42).

[0491]

number

[0492] If it is determined that the above condition (ConditionC1) is not met (i.e., the flip flag FlipFlag is false (0)), the process proceeds to step S523.

[0493] In step S523, the transformation matrix derivation unit 321 performs the base transformation matrix T, similar to the case of step S343 in Figure 30. base Transform matrix T H Set to this value. Once the process in step S523 is complete, the transformation matrix derivation process is finished, and the process returns to Figure 28.

[0494] Furthermore, if it is determined in step S522 that the above-mentioned condition (ConditionC1) is met (i.e., the flip flag FlipFlag is true (1)), the process proceeds to step S524.

[0495] In step S524, the transformation matrix derivation unit 321 calculates the base transformation matrix T base Flip it horizontally, and the flipped transformation matrix Flip(T base ) transform matrix T tmp Set to this. This process can be expressed as shown in equation (43) below.

[0496]

number

[0497] In step S525, the transformation matrix derivation unit 321 determines whether the transpose flag satisfies the condition (Condition C2) expressed by the following equation (44).

[0498]

number

[0499] If it is determined that the above condition (ConditionC2) is not met (i.e., the transpose flag TransposeFlag is false (0)), the process proceeds to step S526.

[0500] In step S526, the transformation matrix derivation unit 321 calculates the transformation matrix T tmp Transform matrix T H Set to this. This process can be expressed as shown in equation (45) below.

[0501]

number

[0502] Once the processing in step S526 is complete, the transformation matrix derivation process is finished, and the process returns to Figure 28. Also, if it is determined in step S525 that the above condition (Condition C2) is met (i.e., the transpose flag TransposeFlag is true (1)), the process proceeds to step S527.

[0503] In step S527, the transformation matrix derivation unit 321 calculates the transformation matrix T tmp Transpose Tr(T tmp ) transform matrix T H Set to this. This process can be expressed, for example, as shown in equation (46) below.

[0504]

number

[0505] Once step S527 is completed, the transformation matrix derivation process is finished, and the process returns to Figure 28.

[0506] By performing the transformation matrix derivation process as described above, it is possible to realize the derivation examples shown in the fourth and fifth rows from the top of the table in Figure 10 for horizontal one-dimensional orthogonal transformations. Therefore, the increase in required memory capacity can be suppressed.

[0507] In the above explanation, the flip and transpose operations were described as being performed sequentially, but the flip and transpose operations may also be performed together (in a single operation). For example, the determinations in steps S522 and S525 in Figure 54 may be performed together, and if both are true (1), the flip and transpose operations may be performed together.

[0508] For example, if these flip and transpose operations are expressed as element-wise operations, the transformation matrix T H The base transformation matrix T is used for the j-th row and i-th column component ((j,i) component). base This involves setting the (i, N-1-j) component. In other words, it can be expressed as shown in equation (47) below.

[0509]

number

[0510] Here, we have a transformation matrix T of N rows and N columns. H The i-th row and j-th column component ((i,j) component) of T H We denote it as [i,j]. Also, the second line, "for i,j=0,...,N-1", indicates that i and j each have values ​​from 0 to N-1. That is, T H [i,j] is a transformation matrix T with N rows and N columns. H This means showing all elements of [the object].

[0511] This approach allows for both flipping and transposing to be performed in a single operation, and with simple access to a two-dimensional array. Furthermore, since the flip matrix J is unnecessary, the increase in memory capacity can be suppressed accordingly.

[0512] In this embodiment, the transformation matrix derivation process performed by the transformation matrix derivation unit 351 of the primary vertical transformation unit 313 in step S361 of Figure 32 is carried out in the same manner as the flowchart in Figure 54.

[0513] Furthermore, in this embodiment, the transformation matrix derivation process performed by the transformation matrix derivation unit 421 of the inverse primary vertical transformation unit 412 in step S441 of Figure 41 is also carried out in the same manner as the flowchart in Figure 54.

[0514] Furthermore, in this embodiment, the transformation matrix derivation process performed by the transformation matrix derivation unit 451 of the inverse primary horizontal transformation unit 413 in step S461 of Figure 42 is also carried out in the same manner as the flowchart in Figure 54.

[0515] Therefore, I will omit further explanation regarding these matters.

[0516] <2-5. Example 1-4> <Concept> Next, we will explain the example in the sixth row from the top, excluding the top row of item names, in the table shown in Figure 10.

[0517] As described above, the derivation of the sixth example from the top focuses on the characteristics between pairs of DCTs / DSTs, similar to the third example from the top (Example 1-2). More specifically, it focuses on the fact that between pairs of DCTs / DSTs (for example, DCT8 and DST7), the even-numbered row vectors are axisymmetric, and the odd-numbered row vectors are point-symmetric. In this case, the derivation unit flips the first transformation matrix, reverses the sign of the odd-numbered row vectors in the first transformation matrix after the flip, and derives the second transformation matrix. That is, as shown in Figure 56, the derivation unit uses the transformation matrix of DCT8 as the base transformation matrix T base Then, by flipping it in the row direction and further inverting the sign of the odd-order row vectors, the DST7 transformation matrix is ​​derived losslessly. Therefore, naturally, the derived DST7 transformation matrix can be used to substitute for the DST7 transformation matrix of the same (increasing) type, where the waveform of the zero-order row vector is identical.

[0518] In other words, this derivation example shows that the second transformation matrix can be derived in just two operations (flip + sign reversal). Furthermore, these operations are simple. That is, the second transformation matrix can be easily derived.

[0519] Furthermore, by applying this derivation example, it becomes unnecessary to prepare DST7 transformation matrices as candidates for transformation matrices used in orthogonal and inverse orthogonal transformations. In other words, the number of eigentransformation types can be reduced.

[0520] In this case, as shown in the table in Figure 57, the number of transformation matrices required, which was five in the case of the technology described in Non-Patent Literature 1 (see table in Figure 6A), can be reduced to four, so the total LUT size can be reduced to approximately 47KB. In other words, the LUT size can be reduced by approximately 53KB (table in Figure 6A) compared to the case of the technology described in Non-Patent Literature 1. That is, the increase in LUT size can be suppressed.

[0521] Furthermore, in this derivation example, since the DST7 transformation matrix can be derived as the second transformation matrix, by performing orthogonal and inverse orthogonal transformations using this second transformation matrix, the same coding efficiency as when using the DST7 transformation matrix for orthogonal and inverse orthogonal transformations can naturally be obtained.

[0522] <Configuration and Processing Flow> The configurations of the image encoding device 100 and the image decoding device 200 in this case are the same as those described in <2-3. Example 1-2>, so their description is omitted. Furthermore, the processing performed by the image encoding device 100 and the image decoding device 200 is also the same as that described in <2-3. Example 1-2>, so their description is omitted.

[0523] However, in this embodiment, when determining the base transformation type corresponding to the transformation type identifier TrTypeIdxH in the transformation matrix derivation process, it is done by referring to a correspondence table such as the one shown in Figure 58. Otherwise, the process can be carried out in the same manner as described with reference to the flowchart in Figure 48.

[0524] <2-6. Examples 1-5> <Concept> Next, we will explain the examples of the 7th and 8th rows from the top, excluding the top row of item names, in the table shown in Figure 10.

[0525] As described above, the derivation of the seventh example from the top, like the first example from the top, focuses on the similarity between the waveform of the lowest-order row vector of the first transformation type and the waveform of the lowest-order row vector of the transformation type of the alternative transformation matrix. However, in this case, the derivation unit derives the second transformation matrix by transposing the first transformation matrix. That is, as shown in Figure 59, the derivation unit uses the transformation matrix of DCT2 as the base transformation matrix T base Then, we transpose it to derive the transformation matrix for DCT3. This DCT3 transformation matrix can be used to substitute for the transformation matrix of DCT8, which has a similar (decreasing) waveform for the 0th row vector.

[0526] Furthermore, the derivation of the eighth example from the top, similar to the fifth example from the top, focuses on the similarity between the waveform of the highest-order column vector of the first transformation type and the waveform of the lowest-order row vector of the transformation type of the substitute transformation matrix. In this case, the derivation unit transposes the first transformation matrix, flips the transposed first transformation matrix, and derives the second transformation matrix. That is, as shown in Figure 59, the derivation unit uses the transformation matrix of DCT2 as the base transformation matrix T base Then, we transpose it and flip it in the row direction to derive the transformation matrix of FlipDCT3. This transformation matrix of FlipDCT3 can be used to substitute for the transformation matrix of DST7 (an augmented type) whose waveform of the 0th row vector is similar.

[0527] In other words, the two derivation examples above allow the second transformation matrix to be derived in one or two operations (transpose or transpose + flip). Furthermore, these operations are easy. That is, the second transformation matrix can be easily derived.

[0528] Furthermore, by applying the two derivation examples above, it becomes unnecessary to prepare DCT8 and DST7 transformation matrices as candidates for transformation matrices used in orthogonal and inverse orthogonal transformations. In other words, the number of intrinsic transformation types can be reduced.

[0529] In this case, as shown in the table in Figure 60, the number of transformation matrices required, which was five in the case of the technology described in Non-Patent Literature 1 (see table in Figure 6A), can be reduced to three, so the total LUT size can be approximately 40KB. In other words, the LUT size can be reduced by approximately 53KB (table in Figure 6A) compared to the case of the technology described in Non-Patent Literature 1. That is, the increase in LUT size can be suppressed.

[0530] As mentioned above, in this case as well, by performing orthogonal and inverse orthogonal transformations using the transformation matrices of the derived second transformation type (DCT3 or FlipDCT3), coding efficiency comparable to that obtained when using the DCT8 or DST7 transformation matrices for orthogonal and inverse orthogonal transformations can be obtained.

[0531] <Structure> The configurations of the image encoding device 100 and the image decoding device 200 in this case are the same as those described in <2-2. Example 1-1>, so their description will be omitted.

[0532] <Flow of transformation matrix derivation process> Next, we will explain the processing flow. Processing other than the transformation matrix derivation process is carried out in the same manner as in the example described above in <2-2. Example 1-1>, etc., so we will omit their explanation.

[0533] In step S321 of Figure 28, an example of the flow of the transformation matrix derivation process performed by the transformation matrix derivation unit 321 of the primary horizontal transformation unit 312 will be explained with reference to the flowchart in Figure 61.

[0534] When the transformation matrix derivation process is started, the transformation matrix derivation unit 321, in step S541, determines the base transformation type BaseTrType corresponding to the transformation type identifier TrTypeIdxH by referring to a correspondence table such as the one shown in Figure 62.

[0535] Furthermore, the transformation matrix derivation unit 321 sets the value of the flip flag FlipFlag corresponding to the transformation type identifier TrTypeIdxH and the value of the transpose flag TransposeFlag corresponding to the transformation type identifier TrTypeIdxH, similar to the case of step S341 in the flowchart of Figure 30.

[0536] In step S542, the transformation matrix derivation unit 321 determines whether the transpose flag satisfies the condition (Condition D1) expressed by the following equation (48).

[0537]

number

[0538] If it is determined that the above condition (ConditionD1) is not met (i.e., the transpose flag TransposeFlag is false (0)), the process proceeds to step S543.

[0539] In step S543, the transformation matrix derivation unit 321 performs the base transformation matrix T, similar to the case of step S343 in Figure 30. base Transform matrix T H Set to this value. Once the process in step S543 is complete, the transformation matrix derivation process is finished, and the process returns to Figure 28.

[0540] Furthermore, if it is determined in step S542 that the above-mentioned condition (ConditionD1) is met (i.e., the transpose flag TransposeFlag is true (1)), the process proceeds to step S544.

[0541] In step S544, the transformation matrix derivation unit 321 calculates the base transformation matrix T base Transpose it horizontally, and the resulting transposed transformation matrix Tr(T base ) transform matrix T tmp Set to this. This process can be expressed as shown in equation (49) below.

[0542]

number

[0543] In step S545, the transformation matrix derivation unit 321 determines whether the flip flag satisfies the condition (Condition D2) expressed by the following equation (50).

[0544]

number

[0545] If it is determined that the above condition (ConditionD2) is not met (i.e., the flip flag FlipFlag is false (0)), the process proceeds to step S546.

[0546] In step S546, the transformation matrix derivation unit 321 calculates the transformation matrix T as shown in equation (45) above. tmp Transform matrix T H Set it to [this].

[0547] Once the processing in step S546 is complete, the transformation matrix derivation process is finished, and the process returns to Figure 28. Also, if it is determined in step S545 that the above condition (Condition D2) is met (if the flip flag FlipFlag is true (1)), the process proceeds to step S547.

[0548] In step S547, the transformation matrix derivation unit 321 calculates the transformation matrix T tmp Flip(T tmp ) transform matrix T H Set to this. This process can be expressed, for example, as shown in equation (51) below.

[0549]

number

[0550] Once step S547 is completed, the transformation matrix derivation process is finished, and the process returns to Figure 28.

[0551] By performing the transformation matrix derivation process as described above, it is possible to realize the derivation examples shown in the 7th and 8th rows from the top of the table in Figure 10 for horizontal one-dimensional orthogonal transformations. Therefore, the increase in required memory capacity can be suppressed.

[0552] In the above explanation, transposition and flipping are described as being performed sequentially, but transposition and flipping may also be performed together (in a single operation). For example, the judgments in steps S542 and S545 in Figure 61 may be performed together, and if both are true (1), transposition and flipping may be performed together.

[0553] For example, if we express these transpose and flip operations as element-wise operations, the transformation matrix T H The i-th row (N-1-j) column component ((i,(N-1-j)) component) of the base transformation matrix T base This involves setting the (j,i) component. In other words, it can be expressed as shown in equation (52) below.

[0554]

number

[0555] Here, we have a transformation matrix T of N rows and N columns. H The i-th row and j-th column component ((i,j) component) of T H We denote it as [i,j]. Also, the second line, "for i,j=0,...,N-1", indicates that i and j each have values ​​from 0 to N-1. That is, T H [i,j] is a transformation matrix T with N rows and N columns. H This means showing all elements of [the object].

[0556] This approach allows transposition and flipping to be performed in a single operation, and with simple access to a two-dimensional array. Furthermore, since the flip matrix J is unnecessary, the increase in memory capacity can be suppressed accordingly.

[0557] In this embodiment, the transformation matrix derivation process performed by the transformation matrix derivation unit 351 of the primary vertical transformation unit 313 in step S361 of Figure 32 is also carried out in the same manner as the flowchart in Figure 61. For example, in the flowchart of Figure 61, the transformation type identifier TrTypeIdxH is converted to TrTypeIdxV, and the transformation matrix T H Transform matrix T V You can interpret it by substituting it with [this].

[0558] Furthermore, in this embodiment, the transformation matrix derivation process performed by the transformation matrix derivation unit 421 of the inverse primary vertical transformation unit 412 in step S441 of Figure 41 is also carried out in the same manner as the flowchart in Figure 61. For example, in the flowchart of Figure 61, the transformation type identifier TrTypeIdxH is converted to TrTypeIdxV, and the transformation matrix T H Transform matrix T V You can interpret it by substituting it with [this].

[0559] Furthermore, in this embodiment, the transformation matrix derivation process performed by the transformation matrix derivation unit 451 of the inverse primary horizontal transformation unit 413 in step S461 of Figure 42 is carried out in the same manner as the flowchart in Figure 61.

[0560] Therefore, I will omit further explanation regarding these matters.

[0561] <3. Second Embodiment> <3-1. Common Concepts> <Permutation operation of predicted residuals> In the first embodiment, we described how to generate other transformation matrices by manipulating a transformation matrix, but it is also possible to manipulate not only the transformation matrix but also the predicted residuals. That is, the predicted residuals of the image may be permuted, the permuted predicted residuals may be orthogonally transformed using the base transformation matrix, the coefficient data obtained from the orthogonal transformation of the predicted residuals may be encoded, and a bitstream may be generated.

[0562] For example, an image processing device may include an operation unit that performs a substitution operation on the predicted residuals of an image, an orthogonal transformation unit that orthogonally transforms the predicted residuals that have been substituted by the operation unit using a base transformation matrix, and an encoding unit that encodes the coefficient data obtained by orthogonally transforming the predicted residuals by the orthogonal transformation unit and generates a bitstream.

[0563] Even in this way, a two-dimensional orthogonal transformation equivalent to that in the first embodiment can be achieved. In other words, an orthogonal transformation using one transformation matrix can substantially achieve an orthogonal transformation using another transformation matrix. Therefore, as in the first embodiment, the increase in the number of transformation matrices prepared for the orthogonal transformation can be suppressed, and the increase in the memory capacity required for the orthogonal transformation can be suppressed. In addition, since the number of operations can be reduced compared to the first embodiment, the increase in the processing load of the orthogonal transformation can be suppressed.

[0564] Alternatively, the bitstream may be decoded to obtain coefficient data obtained by orthogonally transforming the predicted residuals of the image, the obtained coefficient data may be inversely orthogonal transformed, and the result of the inverse orthogonal transformation of the obtained coefficient data may be replaced.

[0565] For example, an image processing device may include a decoding unit that decodes a bitstream and obtains coefficient data obtained by orthogonally transforming the predicted residuals of an image; an inverse orthogonal transformation unit that performs an inverse orthogonal transformation on the coefficient data obtained by the decoding unit; and an operation unit that performs a substitution operation on the inverse orthogonal transformation result of the coefficient data obtained by the inverse orthogonal transformation unit.

[0566] Even in this way, an inverse two-dimensional orthogonal transformation equivalent to that in the first embodiment can be realized. In other words, an inverse orthogonal transformation using one transformation matrix can substantially realize an inverse orthogonal transformation using another transformation matrix. Therefore, as in the first embodiment, the increase in the number of transformation matrices prepared for the inverse orthogonal transformation can be suppressed, and the increase in the memory capacity required for the inverse orthogonal transformation can be suppressed. In addition, since the number of operations can be reduced compared to the first embodiment, the increase in the processing load of the inverse orthogonal transformation can be suppressed.

[0567] Let's explain this in more detail. Focusing on the spatial symmetry of two different 2D orthogonal transformations (including prediction residuals), there may be axial symmetry between 1D orthogonal transformations in a certain direction (horizontal or vertical), such as between the DST7 transformation matrix and the FlipDST7 transformation matrix obtained by flipping the DST7 transformation matrix. Similarly, with respect to prediction residuals, for example, there may be axial symmetry between the prediction residual (X) and its flipped version (X·J) in a certain direction (horizontal or vertical).

[0568] In such cases, as shown in Figure 63, for example, the original predicted residual can be flipped and then orthogonally transformed using the first transformation matrix. This can be used as a substitute for the orthogonal transformation of the original predicted residual using the second transformation matrix, which is obtained by flipping the first transformation matrix.

[0569] In the example shown in Figure 63, on the left side of the figure, a vertical one-dimensional orthogonal transformation using the first transformation matrix and a horizontal one-dimensional orthogonal transformation using the flipped first transformation matrix are performed on the original predicted residual, as shown in equation (53) below.

[0570]

number

[0571] In contrast, the right side of the figure shows the flip of the predicted residual (equation (54) below) and the vertical and horizontal one-dimensional orthogonal transformations (equation (55) below) using the first transformation matrix for the flipped predicted residual.

[0572]

number

[0573] Such two two-dimensional orthogonal transformations result in spatial symmetry; therefore, one can be substituted for the other.

[0574] In other words, if there is spatial symmetry between the base transformation matrix (transformation type) and the transformation matrix used for the orthogonal transformation of the predicted residuals, then, as described above, the predicted residuals can be flipped in the direction of that spatial symmetry, and the flipped predicted residuals can be orthogonally transformed by the base transformation matrix, thereby substituting for orthogonal transformations using other transformation matrices (orthogonal transformations of the predicted residuals before flipping, using the transformation matrix derived using the base transformation matrix).

[0575] In other words, the operation unit may flip the predicted residual in the direction of spatial symmetry between one-dimensional orthogonal transformations, and the orthogonal transformation unit may orthogonally transform the predicted residual flipped by the operation unit using the base transformation matrix. By doing so, an orthogonal transformation equivalent to an orthogonal transformation using other transformation matrices can be performed. That is, as described above, an orthogonal transformation using other transformation matrices can be substantially realized. Therefore, the increase in the number of transformation matrices prepared for orthogonal transformations can be suppressed, and the increase in the memory capacity required for orthogonal transformations can be suppressed.

[0576] The same applies to inverse orthogonal transformations. That is, the coefficient data obtained by decoding the bitstream may be subjected to an inverse orthogonal transformation using the base transformation matrix, and the operation unit may flip the result of the inverse orthogonal transformation in the direction of the spatial symmetry between the one-dimensional orthogonal transformations. By doing so, an inverse orthogonal transformation can be realized using other transformation matrices in effect, thereby suppressing the increase in the number of transformation matrices prepared for the inverse orthogonal transformation and thus suppressing the increase in the memory capacity required for the inverse orthogonal transformation.

[0577] For example, the left side of Figure 64 shows a two-dimensional orthogonal transformation that performs a horizontal one-dimensional orthogonal transformation using FlipDST7 (a transformation matrix of DST7 flipped) and a vertical one-dimensional orthogonal transformation using the DST7 transformation matrix on the predicted residual X. The right side of Figure 64 shows a two-dimensional orthogonal transformation that performs horizontal and vertical one-dimensional orthogonal transformations using the DST7 transformation matrix on the predicted residual X' (= X·J), which is obtained by horizontally flipping the predicted residual X. These two transformations are symmetrical in the horizontal direction and are equivalent. Therefore, for example, the latter two-dimensional orthogonal transformation can substitute for the former two-dimensional orthogonal transformation. The former two-dimensional orthogonal transformation is expressed as equation (56) below, and the latter two-dimensional orthogonal transformation is expressed as equation (57) below.

[0578]

number

[0579] In other words, for example, if there is horizontal symmetry between the base transformation matrix (transformation type) and the transformation matrix used for the orthogonal transformation of the predicted residuals derived using that transformation matrix, the operation unit may flip the predicted residuals horizontally, and the orthogonal transformation unit may orthogonally transform the predicted residuals flipped horizontally by the operation unit using the base transformation matrix. By doing so, it is possible to realize an orthogonal transformation using another transformation matrix that has horizontal symmetry with the orthogonal transformation using the base transformation matrix in effect.

[0580] The same applies to inverse orthogonal transformations. That is, if there is horizontal symmetry between the base transformation matrix (transformation type) and the transformation matrix used for the inverse orthogonal transformation of the coefficient data derived using that transformation matrix, the inverse orthogonal transformation unit may perform an inverse orthogonal transformation on the coefficient data obtained by decoding the bitstream using the base transformation matrix, and the operation unit may flip the result of the inverse orthogonal transformation horizontally. In this way, it is possible to realize an inverse orthogonal transformation using another transformation matrix that has horizontal symmetry with the inverse orthogonal transformation using the base transformation matrix in effect.

[0581] For example, the left side of Figure 65 shows a two-dimensional orthogonal transformation performed on the predicted residual X using a horizontal one-dimensional orthogonal transformation using the DST7 transformation matrix, and a vertical one-dimensional orthogonal transformation using FlipDST7, which is the DST7 transformation matrix flipped. The right side of Figure 65 shows a two-dimensional orthogonal transformation performed on the predicted residual X' (= J·X), which is obtained by flipping the predicted residual X vertically, using the DST7 transformation matrix for horizontal and vertical one-dimensional orthogonal transformations. These two transformations are symmetrical in the vertical direction and are equivalent. Therefore, for example, the latter two-dimensional orthogonal transformation can substitute for the former two-dimensional orthogonal transformation. The former two-dimensional orthogonal transformation is expressed as equation (58) below, and the latter two-dimensional orthogonal transformation is expressed as equation (59) below.

[0582]

number

[0583] In other words, for example, if there is a vertical symmetry between the base transformation matrix (transformation type) and the transformation matrix used for the orthogonal transformation of the predicted residuals, derived using that transformation matrix, the operation unit may flip the predicted residuals vertically, and the orthogonal transformation unit may orthogonally transform the predicted residuals flipped vertically by the operation unit using the base transformation matrix. By doing so, it is possible to realize an orthogonal transformation using another transformation matrix that has a vertical symmetry with the orthogonal transformation using the base transformation matrix in effect.

[0584] The same applies to inverse orthogonal transformations. That is, if there is a vertical symmetry between the base transformation matrix (transformation type) and the transformation matrix used for the inverse orthogonal transformation of the coefficient data derived using that transformation matrix, the inverse orthogonal transformation unit may perform an inverse orthogonal transformation on the coefficient data obtained by decoding the bitstream using the base transformation matrix, and the operation unit may flip the result of the inverse orthogonal transformation vertically. In this way, it is possible to realize an inverse orthogonal transformation using another transformation matrix that has a vertical symmetry with the inverse orthogonal transformation using the base transformation matrix in effect.

[0585] Furthermore, for example, the left side of Figure 66 shows a two-dimensional orthogonal transformation performed on the predicted residual X using FlipDST7, which is obtained by flipping the transformation matrix of DST7, in both horizontal and vertical directions. The right side of Figure 66 shows a two-dimensional orthogonal transformation performed on the predicted residual X' (= J·X·J), which is obtained by flipping the predicted residual X in both horizontal and vertical directions, using the transformation matrix of DST7, in both horizontal and vertical directions. These two transformations are equivalent and have symmetry in both the horizontal and vertical directions. Therefore, for example, the latter two-dimensional orthogonal transformation can substitute for the former two-dimensional orthogonal transformation. The former two-dimensional orthogonal transformation is expressed as shown in equation (60) below, and the latter two-dimensional orthogonal transformation is expressed as shown in equation (61) below.

[0586]

number

[0587] In other words, for example, if there is horizontal and vertical symmetry between the base transformation matrix (transformation type) and the transformation matrix used for the orthogonal transformation of the predicted residuals, derived using that transformation matrix, the operation unit may flip the predicted residuals horizontally and vertically, and the orthogonal transformation unit may orthogonally transform the predicted residuals flipped horizontally and vertically by the operation unit using the base transformation matrix. By doing so, it is possible to realize an orthogonal transformation using other transformation matrices that have horizontal and vertical symmetry with the orthogonal transformation using the base transformation matrix in effect.

[0588] The same applies to inverse orthogonal transformations. That is, if there is horizontal and vertical symmetry between the base transformation matrix (transformation type) and the transformation matrix used for the inverse orthogonal transformation of the coefficient data derived using that transformation matrix, the inverse orthogonal transformation unit may perform an inverse orthogonal transformation on the coefficient data obtained by decoding the bitstream using the base transformation matrix, and the operation unit may flip the inverse orthogonal transformation result horizontally and vertically. In this way, it is possible to realize an inverse orthogonal transformation using other transformation matrices that have horizontal and vertical symmetry with the inverse orthogonal transformation using the base transformation matrix in effect.

[0589] In the case of equation (60) above, the transformation matrices T1 and T2 are flipped, requiring two operations. In contrast, in the case of equation (61) above, the flip of the predicted residual (J·X·J) can be replaced by the matrix product of the permutation matrix P and the predicted residual X. That is, it can be done in one operation. Therefore, compared to the case of equation (60) above (when the transformation matrix is ​​flipped), the number of operations can be reduced, and the increase in processing load for orthogonal and inverse orthogonal transformations can be suppressed (orthogonal and inverse orthogonal transformations can be performed more easily). In other words, the increase in processing load for orthogonal and inverse orthogonal transformations can be suppressed. Furthermore, orthogonal and inverse orthogonal transformations can be performed at a faster speed.

[0590] <Sign inversion of the transformation matrix> In addition to the permutation operation of the predicted residuals described above, operations on the transformation matrix may also be applied. For example, the derivation unit may derive a second transformation matrix using the first transformation matrix, the operation unit may flip the predicted residuals in the direction of spatial symmetry between one-dimensional orthogonal transformations, and the orthogonal transformation unit may orthogonally transform the predicted residuals flipped by the operation unit using the second transformation matrix derived by the derivation unit. In this way, an orthogonal transformation using other transformation matrices can be substantially realized. Therefore, the increase in the number of transformation matrices prepared for orthogonal transformations can be suppressed, and the increase in the memory capacity required for orthogonal transformations can be suppressed.

[0591] The same applies to inverse orthogonal transformations. For example, the derivation unit may derive a second transformation matrix using the first transformation matrix, the inverse orthogonal transformation unit may perform an inverse orthogonal transformation on the coefficient data obtained by the decoding unit using the second transformation matrix derived by the derivation unit, and the operation unit may perform a substitution operation on the result of the inverse orthogonal transformation of the coefficient data obtained by the inverse orthogonal transformation unit. In this way, an inverse orthogonal transformation can be effectively realized using other transformation matrices. Therefore, it is possible to suppress the increase in the number of transformation matrices prepared for inverse orthogonal transformations and to suppress the increase in the memory capacity required for inverse orthogonal transformations.

[0592] Furthermore, when deriving the second transformation matrix in this orthogonal or inverse orthogonal transformation, the derivation unit may derive a second transformation matrix with the same number of rows and columns as the first transformation matrix. If the number of rows and columns is changed, the waveform type may change unintentionally. Therefore, by making the number of rows and columns the same as the first transformation matrix, the possibility of unintended changes in the waveform type can be suppressed, and the derivation of the second transformation matrix can be made easier.

[0593] Furthermore, when deriving the second transformation matrix, the derivation unit may derive the second transformation matrix by inverting the sign of the odd-numbered row vectors in the first transformation matrix. By using the second transformation matrix derived in this way and performing an orthogonal transformation on the predicted residuals as described above, it is possible to realize (substitute) an orthogonal transformation of the predicted residuals using other transformation matrices in which the even-numbered row vectors are axisymmetric with respect to the first transformation matrix and the odd-numbered row vectors are point-symmetric. Also, by performing an inverse orthogonal transformation using the second transformation matrix derived in this way and performing an inverse orthogonal transformation on the result of the inverse orthogonal transformation as described above, it is possible to realize (substitute) an inverse orthogonal transformation using other transformation matrices in which the even-numbered row vectors are axisymmetric with respect to the first transformation matrix and the odd-numbered row vectors are point-symmetric.

[0594] <Example of manipulating predicted residuals> Figure 67 shows a list of examples of operations on predictive residuals to replace orthogonal and inverse orthogonal transformations using other transformation matrices as described above.

[0595] In the table shown in Figure 67, excluding the top row of item names, the operation in the first example from the top focuses on the fact that there is axis symmetry between two different 2D orthogonal transformations in a certain direction (horizontal, vertical, or both) compared to a 1D orthogonal transformation.

[0596] In this case, the operation unit flips the predicted residual in the direction of spatial symmetry between the one-dimensional orthogonal transformations, and the orthogonal transformation unit orthogonally transforms the predicted residual flipped by the operation unit using the base transformation matrix. More specifically, the operation unit flips the predicted residual, and the orthogonal transformation unit transforms the transformation matrix of DST7 into the base transformation matrix T base The flipped predicted residual is then orthogonally transformed using its DST7 transformation matrix. This replaces the orthogonal transformation of the unflipped predicted residual using the FlipDST7 transformation matrix. The direction of the flip of the predicted residual corresponds to the direction of spatial symmetry (horizontal, vertical, or both).

[0597] The same applies to inverse orthogonal transformations. The inverse orthogonal transformation unit decodes the bitstream and performs an inverse orthogonal transformation on the coefficient data obtained using the base transformation matrix, and the operation unit flips the result of this inverse orthogonal transformation in the direction of spatial symmetry between the one-dimensional orthogonal transformations. More specifically, the inverse orthogonal transformation unit performs an inverse orthogonal transformation on the coefficient data using the DST7 transformation matrix, and the operation unit flips the result of this inverse orthogonal transformation. In this way, the inverse orthogonal transformation of the coefficient data using the FlipDST7 transformation matrix can be replaced. Note that the direction of the flip of the inverse orthogonal transformation result corresponds to the direction of spatial symmetry (horizontal direction, vertical direction, or both).

[0598] By applying such operations to the predicted residuals (including the inverse orthogonal transform results), it becomes unnecessary to prepare the FlipDST7 transformation matrix as a candidate for the transformation matrix used in orthogonal and inverse orthogonal transformations. In other words, as in the first embodiment, the number of intrinsic transformation types can be reduced. That is, the increase in LUT size can be suppressed. Furthermore, by performing orthogonal and inverse orthogonal transformations with such operations on the predicted residuals (including the inverse orthogonal transform results), coding efficiency comparable to that of orthogonal and inverse orthogonal transformations using the FlipDST7 transformation matrix can be obtained.

[0599] Furthermore, compared to the case of orthogonal and inverse orthogonal transformations using the transformation matrix of FlipDST7, the 2D flip operation on the predicted residual (including the inverse orthogonal transformation result) can be performed in a single operation, thus reducing the number of flip operations. Therefore, the increase in processing load for orthogonal and inverse orthogonal transformations can be suppressed (orthogonal and inverse orthogonal transformations can be performed more easily). In other words, the increase in processing load for orthogonal and inverse orthogonal transformations can be suppressed. In addition, orthogonal and inverse orthogonal transformations can be performed at a faster speed.

[0600] Furthermore, the operation in the example one row below (the second row from the top) focuses on the similarity between two different two-dimensional orthogonal transformations and one-dimensional orthogonal transformations in a certain direction (horizontal, vertical, or both). More specifically, it focuses on the fact that even-numbered row vectors have axisymmetricity, and odd-numbered row vectors have point symmetry.

[0601] In this case, the derivation unit derives a second transformation matrix using the first transformation matrix, the operation unit performs a substitution operation on the predicted residuals of the image, and the orthogonal transformation unit orthogonally transforms the predicted residuals substituted by the operation unit using the second transformation matrix derived by the derivation unit. More specifically, the derivation unit reverses the sign of the odd-numbered row vectors of the DST7 transformation matrix, the operation unit flips the predicted residuals, and the orthogonal transformation unit orthogonally transforms the flipped predicted residuals using the transformation matrix derived by the derivation unit, which is obtained by reversing the sign of the odd-numbered row vectors of the DST7 transformation matrix. In this way, the orthogonal transformation of the unflipped predicted residuals using the DCT8 transformation matrix can be replaced.

[0602] Note that the direction of the flip of the predicted residual corresponds to the direction of spatial symmetry (horizontal, vertical, or both). Also, since the operation on the transformation matrix is ​​only sign inversion for odd-numbered row vectors, the number of rows and columns of the transformation matrix derived by the derivation section is the same as that of the DST7 transformation matrix.

[0603] The same applies to inverse orthogonal transformations. The derivation unit derives a second transformation matrix using the first transformation matrix, the inverse orthogonal transformation unit performs an inverse orthogonal transformation on the coefficient data obtained by decoding the bitstream using the second transformation matrix, and the operation unit flips the result of the inverse orthogonal transformation in the direction of spatial symmetry between the one-dimensional orthogonal transformations. More specifically, the derivation unit reverses the sign of the odd-numbered row vectors of the DST7 transformation matrix, the inverse orthogonal transformation unit performs an inverse orthogonal transformation on the coefficient data using the DST7 transformation matrix with the sign reversed odd-numbered row vectors, and the operation unit flips the result of the inverse orthogonal transformation. In this way, the inverse orthogonal transformation of coefficient data using the DCT8 transformation matrix can be replaced.

[0604] Note that the direction of the flip in the inverse orthogonal transformation result corresponds to the direction of spatial symmetry (horizontal, vertical, or both). Also, since the operation on the transformation matrix is ​​only sign inversion for odd-numbered row vectors, the number of rows and columns of the transformation matrix derived by the derivation section is the same as that of the DST7 transformation matrix.

[0605] By applying such operations to the prediction residuals (including the inverse orthogonal transform results), it becomes unnecessary to prepare DCT8 transformation matrices as candidates for the transformation matrices used in orthogonal and inverse orthogonal transformations. In other words, as in the first embodiment, the number of intrinsic transformation types can be reduced. That is, the increase in LUT size can be suppressed. Furthermore, by performing orthogonal and inverse orthogonal transformations with such operations on the prediction residuals (including the inverse orthogonal transform results), the same coding efficiency as in the case of orthogonal and inverse orthogonal transformations using DCT transformation matrices can be obtained.

[0606] Furthermore, the operation in the example one row below (the third row from the top) focuses on the fact that there is axisymmetric between one-dimensional orthogonal transformations in a certain direction (horizontal, vertical, or both) in two different two-dimensional orthogonal transformations, and is the same operation as the first row from the top.

[0607] However, in this case, the transformation matrix of DCT8 is the base transformation matrix T base Therefore, by doing so, it is possible to substitute the orthogonal transformation of the unflipped predicted residuals using the FlipDCT8 transformation matrix. The same applies to the inverse orthogonal transformation. In other words, by doing so, it is possible to substitute the inverse orthogonal transformation of the coefficient data using the FlipDCT8 transformation matrix.

[0608] Furthermore, the operation in the example one row below (the fourth row from the top) focuses on the similarity between two different two-dimensional orthogonal transformations and one-dimensional orthogonal transformations in a certain direction (horizontal, vertical, or both) (even-numbered row vectors have axisymmetricity, and odd-numbered row vectors have point symmetry), and is the same operation as the second row from the top.

[0609] However, in this case, the transformation matrix of DCT8 is the base transformation matrix T base Therefore, by doing so, it is possible to substitute the orthogonal transformation of the unflipped predicted residuals using the DST7 transformation matrix. The same applies to the inverse orthogonal transformation. In other words, by doing so, it is possible to substitute the inverse orthogonal transformation of the coefficient data using the DST7 transformation matrix.

[0610] Each of the derivation examples described above may be implemented individually, or multiple derivation examples may be combined. Furthermore, although the above examples used DST7 or DCT8 as examples of base conversion types (first conversion types), they can also be implemented by replacing them with conversion types having the same waveform shape, as shown in Figure 8. For example, DST7 may be replaced with other orthogonal conversions having the same waveform shape, such as DST4, DST8, or DST3. Similarly, DCT8 may be replaced with other orthogonal conversions having the same waveform shape, such as DCT3, DCT7, or DCT4.

[0611] <Configuration and Processing Flow> The configuration of the image coding device 100 that performs such a prediction residual replacement operation is the same as in the first embodiment. In this image coding device 100, the orthogonal transformation unit 113 performs processing to which the above-described technology is applied as both an operation unit and an orthogonal transformation unit. The coding unit 115 also performs processing to which the above-described technology is applied as an coding unit. Furthermore, the inverse orthogonal transformation unit 118 performs processing to which the above-described technology is applied as both an inverse orthogonal transformation unit and an operation unit. Therefore, the image coding device 100 can suppress the increase in memory capacity required for orthogonal and inverse orthogonal transformations.

[0612] Furthermore, the configuration of the orthogonal transformation unit 113 is the same as in the first embodiment. In this orthogonal transformation unit 113, the primary transformation unit 152 performs processing to which the above-described technology is applied, acting as both an operation unit and an orthogonal transformation unit. That is, the operation unit performs a substitution operation on the predicted residuals of the image, and the orthogonal transformation unit uses the base transformation matrix to perform a primary transformation on the predicted residuals substituted by the operation unit. Therefore, it is possible to suppress the increase in memory capacity required for the primary transformation.

[0613] As described above, the primary transformation unit 152 performs a primary horizontal transformation and a primary vertical transformation as primary transformations. In other words, the operation unit performs a substitution operation on the predicted residuals of the image, and the orthogonal transformation unit performs a horizontal one-dimensional orthogonal transformation on the substituted predicted residuals as a primary transformation, using a second transformation matrix for the base horizontal one-dimensional orthogonal transformation, and further performs a vertical one-dimensional orthogonal transformation using a second transformation matrix for the base vertical one-dimensional orthogonal transformation. Therefore, it is possible to suppress the increase in memory capacity required for primary transformations in which such horizontal one-dimensional orthogonal transformations and vertical one-dimensional orthogonal transformations are performed.

[0614] The image encoding process performed by the image encoding device 100 is the same as in the first embodiment. That is, in the image encoding process described above, the process applying the present technology is performed as step S106. Furthermore, the process applying the present technology is performed as step S109. In addition, the process applying the present technology is performed as step S113. Therefore, by performing this image encoding process, it is possible to suppress the increase in memory capacity required for orthogonal and inverse orthogonal transformations.

[0615] Furthermore, the flow of the orthogonal transformation process performed in step S106 of the image encoding process is the same as in the first embodiment. In other words, in the orthogonal transformation process described above, the process to which the present technology is applied is performed as the process in step S132. Therefore, by performing this orthogonal transformation process, it is possible to suppress the increase in the memory capacity required for primary transformation.

[0616] The same applies to the decoding side. In other words, the configuration of the image decoding device 200 that performs the substitution operation of such inverse orthogonal transformation results is the same as in the first embodiment. In this image decoding device 200, the inverse orthogonal transformation unit 214 performs processing to which the above-described technology is applied as both an inverse orthogonal transformation unit and an operation unit. The decoding unit 212 also performs processing to which the above-described technology is applied as a decoding unit. Therefore, the image decoding device 200 can suppress the increase in memory capacity required for inverse orthogonal transformation.

[0617] Furthermore, the configuration of the inverse orthogonal transformation unit 214 is the same as in the first embodiment. In this inverse orthogonal transformation unit 214, the inverse primary transformation unit 253 performs processing to which the above-described technology is applied, acting as both an inverse orthogonal transformation unit and an operation unit. That is, the inverse orthogonal transformation unit performs an inverse primary transformation on the inverse secondary transformation result using the base transformation matrix, and the operation unit performs a substitution operation on the obtained inverse primary transformation result. Therefore, it is possible to suppress the increase in memory capacity required for the inverse primary transformation.

[0618] As described above, the inverse primary transformation unit 253 performs an inverse primary vertical transformation and an inverse primary horizontal transformation as inverse primary transformations. In other words, the inverse orthogonal transformation unit performs an inverse vertical inverse one-dimensional orthogonal transformation on the inverse secondary transformation result, using a second transformation matrix for the base vertical inverse one-dimensional orthogonal transformation, and then performs an inverse horizontal inverse one-dimensional orthogonal transformation using a second transformation matrix for the base horizontal inverse one-dimensional orthogonal transformation. Therefore, it is possible to suppress the increase in memory capacity required for primary transformations in which such vertical inverse one-dimensional orthogonal transformations and horizontal inverse one-dimensional orthogonal transformations are performed.

[0619] The image decoding process performed by the image decoding device 200 is the same as in the first embodiment. That is, in the image decoding process described above, the process applying the present technology described above is performed as step S202. Also, the process applying the present technology described above is performed as step S204. Therefore, by performing this image decoding process, it is possible to suppress the increase in memory capacity required for the inverse orthogonal transform.

[0620] Furthermore, the flow of the inverse orthogonal transform process performed in step S204 of the image decoding process is the same as in the first embodiment. That is, in the inverse orthogonal transform process as described above, the process to which the present technology is applied is performed as the process in step S233. Therefore, by performing this inverse orthogonal transform process, it is possible to suppress the increase in memory capacity required for the inverse primary transform.

[0621] <3-2. Example 2-1> <Concept> Next, we will explain each derivation example described with reference to Figure 67 in more detail. First, in the table shown in Figure 67, excluding the top row of item names, the operation in the first example from the top focuses on the fact that there is axis symmetry between one-dimensional orthogonal transformations in a certain direction (horizontal, vertical, or both) between two different two-dimensional orthogonal transformations.

[0622] In this case, the operation unit flips the predicted residual. The orthogonal transformation unit orthogonally transforms the flipped predicted residual using the base transformation matrix (the transformation matrix of DST7). In this way, it is possible to substitute the orthogonal transformation of the predicted residual using the transformation matrix of FlipDST7 (see <2-2. Example 1-1>), which is obtained by flipping the transformation matrix of DST7.

[0623] The same applies to inverse orthogonal transformations. In this case, the inverse orthogonal transformation unit performs an inverse orthogonal transformation on the coefficient data using the base transformation matrix (the transformation matrix of DST7), and the operation unit flips the inverse orthogonal transformation result. By doing so, it is possible to substitute the inverse orthogonal transformation of the coefficient data (transformation coefficient Coeff_IQ) using the FlipDST7 transformation matrix, which is the transformation matrix of DST7 flipped.

[0624] By applying such operations to the predicted residuals (including the inverse orthogonal transformation results), it becomes unnecessary to prepare the transformation matrix of FlipDST7 as a candidate for the transformation matrix used in the orthogonal and inverse orthogonal transformations. In other words, as in the first embodiment, the number of intrinsic transformation types can be reduced.

[0625] In this case, as shown in the table in Figure 68, the number of transformation matrices required, which was five in the case of the technology described in Non-Patent Literature 1 (see table in Figure 6A), can be reduced to four, so the total LUT size can be approximately 47KB. In other words, the LUT size can be reduced by approximately 53KB (table in Figure 6A) compared to the case of the technology described in Non-Patent Literature 1. That is, the increase in LUT size can be suppressed.

[0626] Furthermore, compared to the case of orthogonal and inverse orthogonal transformations using the transformation matrix of FlipDST7, the 2D flip operation on the predicted residual (including the inverse orthogonal transformation result) can be performed in a single operation, thus reducing the number of flip operations. Therefore, the increase in processing load for orthogonal and inverse orthogonal transformations can be suppressed (orthogonal and inverse orthogonal transformations can be performed more easily).

[0627] Furthermore, by performing orthogonal and inverse orthogonal transformations that involve manipulating such prediction residuals (including the inverse orthogonal transformation results), coding efficiency comparable to that of orthogonal and inverse orthogonal transformations using the FlipDST7 transformation matrix can be obtained.

[0628] <Primary Conversion Unit> Next, the configuration and processing for performing such processing will be described. Figure 69 is a block diagram showing an example of the main configuration of the primary transformation unit 152 in this case. As shown in Figure 69, the primary transformation unit 152 in this case has a predicted residual replacement operation unit 551 in addition to the configuration shown in Figure 22.

[0629] The primary transformation selection unit 311 also supplies the derived primary horizontal transformation type identifier TrTypeIdxH and primary vertical transformation type identifier TrTypeIdxV to the predicted residual replacement operation unit 551.

[0630] The predicted residual replacement operation unit 551 receives the predicted residual D supplied from the switch 151, as well as the transformation type identifier TrTypeIdxH for the primary horizontal transformation and the transformation type identifier TrTypeIdxV for the primary vertical transformation as input. The predicted residual replacement operation unit 551 flips the predicted residual D according to the transformation type identifiers TrTypeIdxH and TrTypeIdxV. The predicted residual replacement operation unit 551 then flips the predicted residual D flip This is supplied to the primary horizontal conversion unit 312.

[0631] In the primary transformation unit 152 configured as described above, the predicted residual replacement operation unit 551 performs processing that applies the aforementioned technology as an operation unit. The primary horizontal transformation unit 312 and the primary vertical transformation unit 313 each perform processing that applies the aforementioned technology as orthogonal transformation units.

[0632] In other words, the primary horizontal transformation unit 312, as an orthogonal transformation unit, uses the transformation matrix for the base horizontal one-dimensional orthogonal transformation to predict the residual D after the flip. flipA one-dimensional orthogonal transformation is performed in the horizontal direction. Therefore, the primary horizontal transformation unit 312 can suppress the increase in memory capacity required for the one-dimensional orthogonal transformation in the horizontal direction.

[0633] Furthermore, the primary vertical transformation unit 313, acting as an orthogonal transformation unit, uses the transformation matrix for the base vertical one-dimensional orthogonal transformation to perform a vertical one-dimensional orthogonal transformation on the transformation coefficients Coeff_Phor after the primary horizontal transformation. Therefore, the primary vertical transformation unit 313 can suppress the increase in memory capacity required for the vertical one-dimensional orthogonal transformation.

[0634] <Primary conversion process flow> Next, we will explain an example of the processing flow performed by the above-described configuration. In this case, an example of the primary conversion processing flow performed in step S132 of Figure 14 will be explained with reference to the flowchart in Figure 70.

[0635] When the primary conversion process is started, the primary conversion selection unit 311 of the primary conversion unit 152 selects the conversion type identifier TrTypeIdxH (or conversion type TrTypeH) for the primary horizontal conversion and the conversion type identifier TrTypeIdxV (or conversion type TrTypeV) for the primary vertical conversion in step S561.

[0636] In step S562, the prediction residual replacement operation unit 551 executes the prediction residual replacement operation process and flips the prediction residual D according to the conversion type identifier TrTypeIdxH and conversion type identifier TrTypeIdxV obtained in step S561, and the predicted residual D after flipping flip Derive the following.

[0637] In step S563, the primary horizontal transformation unit 312 performs primary horizontal transformation processing and uses the transformation matrix corresponding to the transformation type identifier TrTypeIdxH of the primary horizontal transformation obtained in step S561 to determine the predicted residual D after flipping obtained in step S562. flipPerform a primary horizontal transformation on it.

[0638] In step S564, the primary vertical transformation unit 313 performs primary vertical transformation processing and uses the transformation matrix corresponding to the primary vertical transformation type identifier TrTypeIdxV obtained in step S561 to perform primary vertical transformation on the transformation coefficient Coeff_Phor obtained in step S563 after primary horizontal transformation.

[0639] Once step S564 is completed, the primary conversion process ends, and the process returns to Figure 14.

[0640] In the primary conversion process described above, the processes to which the present technology is applied are performed as steps S562 to S564. Therefore, by executing this primary conversion process, it is possible to suppress the increase in memory capacity required for the primary horizontal conversion process and the primary vertical conversion process.

[0641] <Flowchart of Predicted Residual Replacement Operation> The flow of the predicted residual replacement operation process performed in step S562 of Figure 70 will be explained with reference to the flowchart in Figure 71.

[0642] When the predicted residual replacement operation process is started, the predicted residual replacement operation unit 551 derives a flip flag FlipFlagH corresponding to the primary horizontal transformation type identifier TrTypeIdxH (or transformation type TrTypeH) and a flip flag FlipFlagV corresponding to the primary vertical transformation type identifier TrTypeIdxV (or transformation type TrTypeV) in step S581. At that time, the predicted residual replacement operation unit 551 obtains the flip flags FlipFlagH and FlipFlagV by referring to a correspondence table (LUT_TrTypeIdxToFlipFlag) such as the one shown in Figure 72. The derivation of these flip flags can be expressed, for example, as shown in equations (62) and (63) below.

[0643]

number

[0644] In step S582, the prediction residual replacement operation unit 551 determines whether the flip flags FlipFlagH and FlipFlagV derived in step S581 satisfy the conditions shown in the following equation (64).

[0645]

number

[0646] If it is determined that this condition is met, that is, if FlipFlagH is true (1) and FlipFlagV is false (0), the process proceeds to step S583.

[0647] In step S583, the predicted residual replacement operation unit 551 flips the predicted residual D horizontally to obtain the predicted residual D after flipping. flip We obtain this. This operation can be expressed in matrix form as shown in equation (65) below.

[0648]

number

[0649] Here, × is the operator representing matrix multiplication, FlipH(X) is the operator representing the horizontal flip operation of matrix X, and the flip matrix J corresponding to FlipV(·) is the horizontally flipped identity matrix I of N rows and N columns. Furthermore, when this operation is expressed as an element-wise operation, the prediction residual replacement operation unit 551 calculates the predicted residual D after the flip as shown in equation (66) below. flip The i-th row, j-th column component ((i,j) component) of the predicted residual D is set to the (i,N-1-j) component.

[0650]

number

[0651] However, the size of the predicted residual D is N in width and M in height (M rows and N columns). N and M satisfy the following equations (67) and (68).

[0652]

number

[0653] When the processing in step S583 is completed, the process proceeds to step S588. Also, if it is determined in step S582 that the condition is not met, that is, if FlipFlagH is determined to be false (0) or FlipFlagV is determined to be true (1), the process proceeds to step S584.

[0654] In step S584, the prediction residual replacement operation unit 551 determines whether the flip flags FlipFlagH and FlipFlagV derived in step S581 satisfy the conditions shown in the following equation (69).

[0655]

number

[0656] If it is determined that this condition is met, that is, if FlipFlagH is false (0) and FlipFlagV is true (1), the process proceeds to step S585.

[0657] In step S585, the predicted residual replacement operation unit 551 flips the predicted residual D vertically to obtain the predicted residual D after flipping. flip We obtain this. This operation can be expressed in matrix form as shown in equation (70) below.

[0658]

number

[0659] Here, × is the operator representing matrix multiplication, FlipV(X) is the operator representing the vertical flip operation of matrix X, and the flip matrix J corresponding to FlipV(·) is the horizontally flipped identity matrix I of M rows and M columns. Furthermore, when this operation is expressed as an element-wise operation, the prediction residual replacement operation unit 551 calculates the predicted residual D after the flip as shown in equation (71) below. flip The i-th row, j-th column component ((i,j) component) of the predicted residual D is set to the (M-1-i,j) component. Here, the size of the predicted residual D is width N and height M (M rows, N columns), where N and M satisfy equations (67) and (68) above.

[0660]

number

[0661] When the processing in step S585 is completed, the process proceeds to step S588. Also, if it is determined in step S584 that the condition is not met, that is, if FlipFlagH is determined to be true (1) or FlipFlagV is determined to be false (0), the process proceeds to step S586.

[0662] In step S586, the prediction residual replacement operation unit 551 determines whether the flip flags FlipFlagH and FlipFlagV derived in step S581 satisfy the conditions shown in the following equation (72).

[0663]

number

[0664] If it is determined that this condition is met, that is, if it is determined that FlipFlagH is true (1) and FlipFlagV is true (1), the process proceeds to step S587.

[0665] In step S587, the predicted residual replacement operation unit 551 flips the predicted residual D horizontally and vertically to obtain the predicted residual D after flipping. flip We obtain this. This operation can be expressed in matrix form as shown in equation (73) below.

[0666]

number

[0667] Here, × is the operator representing matrix multiplication, and the flip matrix J1 is the horizontally flipped version of the N x N identity matrix I, and the flip matrix J2 is the horizontally flipped version of the M x M identity matrix I. Note that the flip matrix J is equivalent to the flip matrix J when transposed (J T = J). Also, when this operation is expressed as an element-wise calculation, the predicted residual replacement operation unit 551 calculates the predicted residual D after the flip as shown in equation (74) below. flip The i-th row, j-th column component ((i,j) component) of the predicted residual D is set to the (Mi-1,Nj-1) component. Here, the size of the predicted residual D is width N and height M (M rows, N columns), where N and M satisfy equations (67) and (68) above.

[0668]

number

[0669] Once the processing in step S587 is completed, the process proceeds to step S588.

[0670] In step S588, the predicted residual replacement operation unit 551 replaces the predicted residual D after the flip as shown in equation (75) below. flip Let D be the predicted residual. In other words, this predicted residual D (predicted residual D after flip) flip A primary horizontal transformation process (step S563 in Figure 70) is performed on the result.

[0671]

number

[0672] When the process in step S588 is completed, the prediction residual replacement operation process is finished, and the process returns to Figure 70. Also, in step S586, if it is determined that the conditions are not met, that is, if FlipFlagH is determined to be false (0) or FlipFlagV is determined to be false (0), the prediction residual replacement operation process is finished, and the process returns to Figure 70.

[0673] By performing the predicted residual replacement operation as described above, the predicted residual replacement operation unit 551 can flip the predicted residual in the direction of the spatial symmetry of the two-dimensional orthogonal transformation (including the predicted residual) (i.e., in the direction corresponding to the transformation type identifier TrTypeIdxH (or transformation type TrTypeH) of the primary horizontal transformation and the transformation type identifier TrTypeIdxV (or transformation type TrTypeV) of the primary vertical transformation).

[0674] <Inverse Primary Transformer> Next, the configuration of the image decoding device 200 in this embodiment will be described. Figure 73 is a block diagram showing an example of the main configuration of the inverse primary transform unit 253 (Figure 16) in this case. As shown in Figure 73, the inverse primary transform unit 253 in this case has a predictive residual replacement operation unit 552 in addition to the configuration shown in Figure 34.

[0675] The inverse primary transformation selection unit 411 also supplies the derived inverse primary vertical transformation type identifier TrTypeIdxV and the derived inverse primary vertical transformation type identifier TrTypeIdxH to ​​the prediction residual replacement operation unit 551. In addition, the inverse primary horizontal transformation unit 413 supplies the derived inverse primary horizontal transformation coefficient Coeff_IPhor to the prediction residual replacement operation unit 552.

[0676] The prediction residual replacement operation unit 552 receives the conversion coefficient Coeff_IPhor after the inverse primary horizontal transformation supplied from the inverse primary horizontal transformation unit 413, as well as the transformation type identifier TrTypeIdxH (or transformation type TrTypeH) for the primary horizontal transformation and the transformation type identifier TrTypeIdxV (or transformation type TrTypeV) for the primary vertical transformation as input. The prediction residual replacement operation unit 552 flips the conversion coefficient Coeff_IPhor after the inverse primary horizontal transformation according to the transformation type identifier TrTypeIdxH (or transformation type TrTypeH) and the transformation type identifier TrTypeIdxV (or transformation type TrTypeV). The prediction residual replacement operation unit 552 outputs the flipped conversion coefficient Coeff_IPhor (conversion coefficient Coeff_IP after the inverse primary transformation) as the prediction residual D' to the outside of the inverse primary transformation unit 253 (supplied to the calculation unit 215).

[0677] In the inverse primary transform unit 253 configured as described above, the predicted residual replacement operation unit 552 performs processing that applies the aforementioned technology as an operation unit. The inverse primary vertical transform unit 412 and the inverse primary horizontal transform unit 413 each perform processing that applies the aforementioned technology as inverse orthogonal transform units.

[0678] In other words, the inverse primary vertical transformation unit 412, acting as an inverse orthogonal transformation unit, uses the transformation matrix for the base vertical inverse one-dimensional orthogonal transformation to perform a vertical inverse one-dimensional orthogonal transformation on the transformation coefficients Coeff_IS after the inverse secondary transformation. These transformation coefficients are then flipped by the prediction residual substitution operation unit 552 as described above, thereby substituting for a vertical inverse one-dimensional orthogonal transformation using other transformation matrices. Therefore, the inverse primary vertical transformation unit 412 can suppress the increase in memory capacity required for the vertical inverse one-dimensional orthogonal transformation.

[0679] Furthermore, the inverse primary horizontal transformation unit 413, acting as an inverse orthogonal transformation unit, uses the transformation matrix for the base horizontal inverse one-dimensional orthogonal transformation to perform a horizontal inverse one-dimensional orthogonal transformation on the transformation coefficients Coeff_IPver after the inverse primary vertical transformation. These transformation coefficients are then flipped by the prediction residual substitution operation unit 552 as described above, thereby substituting for horizontal inverse one-dimensional orthogonal transformations using other transformation matrices. Therefore, the inverse primary horizontal transformation unit 413 can suppress the increase in memory capacity required for horizontal inverse one-dimensional orthogonal transformations.

[0680] <Flow of inverse primary transformation process> Next, we will explain an example of the processing flow performed by the above-described configuration. In this case, an example of the inverse primary transformation process performed in step S233 of Figure 18 will be explained with reference to the flowchart in Figure 74.

[0681] When the inverse primary conversion process is started, the inverse primary conversion selection unit 411 of the inverse primary conversion unit 253 performs an inverse primary conversion selection process in step S601 and selects the conversion type identifier TrTypeIdxV (or conversion type TrTypeV) for the inverse primary vertical conversion and the conversion type identifier TrTypeIdxH (or conversion type TrTypeH) for the inverse primary horizontal conversion, respectively.

[0682] In step S602, the inverse primary vertical transformation unit 412 performs an inverse primary vertical transformation process and uses the transformation matrix corresponding to the transformation type identifier TrTypeIdxV of the inverse primary vertical transformation obtained in step S601 to perform a vertical inverse one-dimensional orthogonal transformation on the transformation coefficients Coeff_IS after the inverse secondary transformation.

[0683] In step S603, the inverse primary horizontal transformation unit 413 performs an inverse primary horizontal transformation process and uses the transformation matrix corresponding to the transformation type identifier TrTypeIdxH of the inverse primary horizontal transformation obtained in step S601 to perform an inverse one-dimensional orthogonal transformation in the horizontal direction on the transformation coefficient Coeff_IPver after the inverse primary vertical transformation derived in step S602.

[0684] In step S604, the prediction residual replacement operation unit 552 performs a prediction residual replacement operation, and using the transformation type identifier TrTypeIdxV of the inverse primary vertical transformation and the transformation type identifier TrTypeIdxH of the inverse primary horizontal transformation obtained in step S601, it flips the transformation coefficient Coeff_IPhor after the inverse primary horizontal transformation derived in step S603, and derives the transformation coefficient Coeff_IP (predicted residual D') after the inverse primary transformation.

[0685] This prediction residual replacement operation is performed in the same manner as the example described with reference to the flowchart in Figure 71. Therefore, the explanation regarding Figure 71 can be applied to the process in step S604 by appropriately replacing the prediction residual D with the conversion coefficient Coeff_IPhor, etc., so the explanation is omitted.

[0686] Once step S604 is completed, the inverse primary transformation process ends, and the process returns to Figure 18.

[0687] In the inverse primary conversion process described above, the processes to which the aforementioned technology is applied are performed as steps S602 to S604. Therefore, by executing this inverse primary conversion process, it is possible to suppress the increase in memory capacity required for the inverse primary vertical conversion process and the inverse primary horizontal conversion process.

[0688] <3-3. Example 2-2> <Concept> Next, we will explain the example in the second row from the top, excluding the top row of item names, in the table shown in Figure 67. This operation focuses on the similarity between two different two-dimensional orthogonal transformations and one-dimensional orthogonal transformations in a certain direction (horizontal, vertical, or both). More specifically, it focuses on the fact that even-numbered row vectors have axisymmetricity and odd-numbered row vectors have point symmetry.

[0689] In this case, the derivation unit derives a second transformation matrix using the first transformation matrix, the operation unit performs a substitution operation on the predicted residuals, and the orthogonal transformation unit orthogonally transforms the substituted predicted residuals using the second transformation matrix. For example, the DST7 transformation matrix is ​​used as the base transformation matrix, the derivation unit reverses the sign of the odd-numbered row vectors of the DST7 transformation matrix, the operation unit flips the predicted residuals, and the orthogonal transformation unit orthogonally transforms the flipped predicted residuals using the DST7 transformation matrix with the odd-numbered row vectors having their signs reversed. In this way, it is possible to substitute the orthogonal transformation of the predicted residuals using a transformation matrix o...

Claims

1. An operation unit that replaces the predicted residuals using horizontal identification data that identifies whether to flip the predicted residuals horizontally, set according to the transformation type of the orthogonal transformation to the predicted residuals of the image, and vertical identification data that identifies whether to flip the predicted residuals vertically, set according to the transformation type of the orthogonal transformation to the predicted residuals of the image. An image processing device equipped with the following features.

2. The horizontal identification data is set according to the horizontal transformation type of the orthogonal transformation. The vertical identification data is set according to the vertical transformation type of the orthogonal transformation. Image processing apparatus according to claim 1

3. The operation unit performs a horizontal replacement operation on the predicted residual using the horizontal identification data, and a vertical replacement operation on the predicted residual using the vertical identification data. The image processing apparatus according to claim 2.

4. The predicted residuals are replaced using horizontal identification data, which is set according to the transformation type of the orthogonal transformation applied to the predicted residuals of an image, to identify whether to flip the predicted residuals horizontally, and vertical identification data, which is set according to the transformation type of the orthogonal transformation applied to the predicted residuals of an image, to identify whether to flip the predicted residuals vertically. Image processing methods including [specific details omitted].

5. An operation unit that replaces the result of an inverse orthogonal transformation of coefficient data obtained by replacing the predicted residuals of an image with an orthogonal transformation, using horizontal identification data that identifies whether to flip the predicted residuals horizontally, set according to the transformation type of the orthogonal transformation of the predicted residuals of an image, and vertical identification data that identifies whether to flip the predicted residuals vertically, set according to the transformation type of the orthogonal transformation of the predicted residuals of an image. An image processing device equipped with the following features.

6. The operation unit uses the horizontal identification data set according to the horizontal transformation type of the orthogonal transformation and the vertical identification data set according to the vertical transformation type of the orthogonal transformation to perform a substitution operation on the result of the inverse orthogonal transformation of the coefficient data. The image processing apparatus according to claim 5.

7. The aforementioned operating unit is Using the aforementioned horizontal identification data, the result of the inverse orthogonal transformation of the coefficient data is replaced horizontally. Using the aforementioned vertical identification data, the result of the inverse orthogonal transformation of the coefficient data is replaced vertically. The image processing apparatus according to claim 6.

8. Using horizontal identification data that identifies whether to flip the predicted residuals horizontally, set according to the transformation type of the orthogonal transformation on the predicted residuals of the image, and vertical identification data that identifies whether to flip the predicted residuals vertically, set according to the transformation type of the orthogonal transformation on the predicted residuals of the image, the result of the inverse orthogonal transformation of the coefficient data obtained by replacing the predicted residuals of the image with an orthogonal transformation. Image processing methods including [specific details omitted].

Images