Image encoding device, image decoding device, and program

Abstract

To improve coding efficiency.SOLUTION: An image encoding device (1) encodes a target image in block units obtained by dividing a current image in frame units. The image encoding device (1) includes: a prediction unit (172) for predicting the target image using a plurality of reference images to generate a predicted image; an evaluation unit (180) for generating map information indicating a distribution of an error in the predicted image by evaluating similarity between the plurality of reference images in pixel units; a subtraction unit (110) for calculating a prediction residual indicating a difference between the target block and the prediction image in pixel units; a determination unit (190) for determining an orthogonal transformation to be applied to the prediction residual on the basis of the map information; and a transformation unit (121) for performing an orthogonal transformation process on the prediction residual by the determined orthogonal transformation.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image encoding device, an image decoding device, and a program. [Background technology]

[0002] Conventionally, in an image coding device that codes a target image in units of blocks obtained by dividing a current image in units of frames, a method is known in which the target image is predicted using a plurality of reference images to generate a predicted image, a prediction residual indicating the difference between the target image and the predicted image is subjected to an orthogonal transform process to calculate transform coefficients, and the transform coefficients are quantized and entropy coded to output coded data.

[0003] Similarly to the image coding device, the image decoding device generates a predicted image by predicting a target image using multiple reference images. The image decoding device decodes coded data to obtain transform coefficients and inversely quantizes them, performs an inverse orthogonal transform process on the inversely quantized transform coefficients to calculate a prediction residual, and decodes the target image by combining the predicted image and the prediction residual.

[0004] In HEVC, two types of orthogonal transform, DCT-2 and DST-7, are defined as applicable to transform processes (orthogonal transform process and inverse orthogonal transform process) (see Non-Patent Document 1). Specifically, in HEVC, which of the two types of orthogonal transform to apply is determined based on the block size of a target image and the intra prediction mode applied to the target image. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] Recommendation ITU-T H.265,(12 / 2016), “High efficiency video coding”, International Telecommunication Union Summary of the Invention [Problem to be solved by the invention]

[0006] However, simply determining the type of orthogonal transform based on the block size of the target image and the mode of intra prediction applied to the target image does not allow for applying the optimal type of orthogonal transform according to the energy distribution of the prediction residual. For example, even if DCT-2 would actually be more efficient in concentrating the energy of the prediction residual, DST-7 may be selected as the orthogonal transform to be applied, resulting in a problem of reduced coding efficiency.

[0007] Furthermore, KLT is an orthogonal transform method that can more efficiently concentrate the energy of prediction residuals. However, since the image decoding device requires information for the inverse processing of the KLT performed in the image encoding device, the amount of information to be transmitted increases, resulting in a problem of reduced encoding efficiency.

[0008] Therefore, an object of the present invention is to provide an image encoding device, an image decoding device, and a program that can improve encoding efficiency. [Means for solving the problem]

[0009]

[0013] A first aspect of the image coding apparatus is an image coding apparatus that codes a target image in units of blocks obtained by dividing a current image in units of frames, the image coding apparatus comprising: a prediction unit that predicts the target image using a plurality of reference images to generate a predicted image; an evaluation unit that generates map information indicating an error distribution in the predicted image by evaluating similarities between the plurality of reference images on a pixel-by-pixel basis; a subtraction unit that calculates, on a pixel-by-pixel basis, a prediction residual indicating a difference between the target block and the predicted image; a determination unit that determines an orthogonal transform to apply to the prediction residual based on the map information; and a transformation unit that performs orthogonal transform processing on the prediction residual using the determined orthogonal transform.

[0014] Another aspect of the image coding apparatus is an image coding apparatus that codes a target image in units of blocks obtained by dividing a current image in units of frames, the image coding apparatus comprising: a prediction unit that predicts the target image using a plurality of reference images to generate a predicted image; and an evaluation unit that calculates, for each region smaller than the block and consisting of a plurality of pixels, a sum of absolute differences indicating the similarity between the plurality of reference images; and controls the coding process based on the sum of absolute differences calculated by the evaluation unit for each region.

[0010]

[0013] A second aspect of the image decoding device is an image decoding device that decodes a target image in units of blocks obtained by dividing a current image in units of frames, the image decoding device comprising: a decoding unit that obtains transform coefficients by decoding encoded data; a prediction unit that predicts the target image using multiple reference images to generate a predicted image; an evaluation unit that generates map information indicating an error distribution in the predicted image by evaluating similarities between the multiple reference images on a pixel-by-pixel basis; a determination unit that determines an inverse orthogonal transform to apply to the transform coefficients based on the map information; and an inverse transform unit that performs inverse orthogonal transform processing on the transform coefficients using the determined inverse orthogonal transform.

[0014] Another aspect of the image decoding device is an image decoding device that decodes a target image in units of blocks obtained by dividing a current image in units of frames, the image decoding device comprising: a prediction unit that predicts the target image using multiple reference images to generate a predicted image; and an evaluation unit that calculates a sum of absolute differences indicating the similarity between the multiple reference images for each region smaller than the block, the region consisting of multiple pixels, and controls decoding based on the sum of absolute differences calculated by the evaluation unit for each region.

[0011] A third aspect of the present invention relates to a program for causing a computer to function as the image encoding device according to the first aspect.

[0012] A fourth aspect of the present invention relates to a program for causing a computer to function as the image decoding device according to the second aspect. [Effects of the Invention]

[0013] According to the present invention, it is possible to provide an image encoding device, an image decoding device, and a program that can improve encoding efficiency. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 1 is a diagram showing the configuration of an image encoding device according to a first embodiment. [Figure 2] FIG. 2 is a diagram showing an example of inter prediction according to the first to third embodiments. [Figure 3] FIG. 2 is a diagram illustrating an example of the configuration of an evaluation unit according to the first to third embodiments. [Figure 4] FIG. 4 is a diagram illustrating the operation of an adaptive transformation generation unit according to the first embodiment. [Figure 5] FIG. 1 is a diagram showing the configuration of an image decoding device according to a first embodiment. [Figure 6] FIG. 10 is a diagram showing the configuration of an image encoding device according to a second embodiment. [Figure 7] FIG. 10 is a diagram showing the configuration of an image decoding device according to a second embodiment. [Figure 8] FIG. 10 is a diagram showing the configuration of an image encoding device according to a third embodiment. [Figure 9] FIG. 11 is a diagram illustrating the operation of a feature amount evaluation unit according to the third embodiment. [Figure 10] FIG. 11 is a diagram showing the configuration of an image decoding device according to a third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0015] An image encoding device and an image decoding device according to an embodiment will be described with reference to the drawings. In the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals.

[0016] First Embodiment A description will be given of an image encoding device and an image decoding device according to the first embodiment. The image encoding device and the image decoding device according to the first embodiment encode and decode moving images, as typified by MPEG.

[0017] (Image encoding device) Fig. 1 is a diagram showing the configuration of an image coding device 1 according to the first embodiment. As shown in Fig. 1, the image coding device 1 includes a block division unit 100, a subtraction unit 110, a transform / quantization unit 120, an entropy coding unit 130, an inverse quantization / inverse transform unit 140, a synthesis unit 150, a memory 160, a prediction unit 170, an evaluation unit 180, and a determination unit 190.

[0018] The block division unit 100 divides an input image in units of frames (or pictures) constituting a moving image into small block-shaped regions, and outputs the blocks obtained by division to the subtraction unit 110. The size of a block is, for example, 32×32 pixels, 16×16 pixels, 8×8 pixels, or 4×4 pixels. The shape of a block is not limited to a square, but may also be a rectangle. A block is a unit for encoding by the image encoding device 1 and a unit for decoding by the image decoding device 2.

[0019] The subtraction unit 110 calculates a prediction residual indicating a difference in pixel units between the block input from the block division unit 100 and a prediction image (prediction block) obtained by predicting the block by the prediction unit 170. Specifically, the subtraction unit 110 calculates the prediction residual by subtracting each pixel value of the prediction image from each pixel value of the block, and outputs the calculated prediction residual to the transformation and quantization unit 120.

[0020] The transform / quantization unit 120 performs orthogonal transform processing and quantization processing on a block-by-block basis. The transform / quantization unit 120 includes a transform unit 121 and a quantization unit 122.

[0021] The transform unit 121 performs orthogonal transform processing on the prediction residuals input from the subtraction unit 110 to calculate transform coefficients, and outputs the calculated transform coefficients to the quantization unit 122. Examples of orthogonal transform include a discrete cosine transform (DCT), a discrete sine transform (DST), and a Karhunen Loeve transform (KLT). In the first embodiment, the transform unit 121 performs orthogonal transform processing using the KLT.

[0022] The quantization unit 122 quantizes the transform coefficients input from the transform unit 121 using a quantization parameter (Qp) and a quantization matrix, and outputs the quantized transform coefficients to the entropy coding unit 130 and the inverse quantization and inverse transform unit 140. The quantization parameter (Qp) is a parameter commonly applied to each transform coefficient in a block, and determines the coarseness of quantization. The quantization matrix is ​​a matrix having, as elements, quantization values ​​used when quantizing each transform coefficient.

[0023] The entropy coding unit 130 performs entropy coding on the transform coefficients input from the quantization unit 122, compresses the data, generates coded data (bit stream), and outputs the coded data to the outside of the image coding device 1. For entropy coding, Huffman coding, CABAC (Context-based Adaptive Binary Arithmetic Coding), etc. may be used. Note that the entropy coding unit 130 receives control information related to prediction from the prediction unit 170 and also performs entropy coding on the input control information.

[0024] The inverse quantization and inverse transform unit 140 performs inverse quantization processing and inverse orthogonal transform processing on a block-by-block basis. The inverse quantization and inverse transform unit 140 includes an inverse quantization unit 141 and an inverse transform unit 142.

[0025] The inverse quantization unit 141 performs inverse quantization processing corresponding to the quantization processing performed by the quantization unit 122. Specifically, the inverse quantization unit 141 restores the transform coefficients by inverse quantizing the transform coefficients input from the quantization unit 122 using a quantization parameter (Qp) and a quantization matrix, and outputs the restored transform coefficients to the inverse transform unit 142.

[0026] The inverse transform unit 142 performs inverse orthogonal transform processing corresponding to the orthogonal transform processing performed by the transform unit 121. For example, if the transform unit 121 performs discrete cosine transform, the inverse transform unit 142 performs inverse discrete cosine transform. The inverse transform unit 142 performs inverse orthogonal transform processing on the transform coefficients input from the inverse quantization unit 141 to restore prediction residuals, and outputs restored prediction residuals, which are the restored prediction residuals, to the synthesis unit 150.

[0027] The synthesis unit 150 synthesizes, on a pixel-by-pixel basis, the reconstructed prediction residual input from the inverse transform unit 142 with the predicted image input from the prediction unit 170. The synthesis unit 150 adds each pixel value of the reconstructed prediction residual to each pixel value of the predicted image to reconstruct (decode) a block, and outputs the decoded image in block units to the memory 160. Such a decoded image may be referred to as a reconstructed image.

[0028] The memory 160 stores the decoded image input from the synthesis unit 150. The memory 160 stores the decoded image in units of frames. The memory 160 outputs the stored decoded image to the prediction unit 170. Note that a loop filter may be provided between the synthesis unit 150 and the memory 160.

[0029] The prediction unit 170 performs prediction on a block-by-block basis and includes an intra prediction unit 171, an inter prediction unit 172, and a switching unit 173.

[0030] The intra prediction unit 171 generates an intra prediction image by referring to decoded pixel values ​​in the vicinity of the block to be predicted among the decoded images stored in the memory 160, and outputs the generated intra prediction image to the switching unit 173. The intra prediction unit 171 also selects an optimal intra prediction mode to be applied to the current block from among multiple intra prediction modes, and performs intra prediction using the selected intra prediction mode. The intra prediction unit 171 outputs control information related to the selected intra prediction mode to the entropy coding unit 130. Note that the intra prediction modes include planar prediction, DC prediction, and directional prediction.

[0031] The inter prediction unit 172 uses the decoded image stored in the memory 160 as a reference image to calculate a motion vector by a technique such as block matching, predicts the block to be predicted, generates an inter prediction image, and outputs the generated inter prediction image to the switching unit 173. The inter prediction unit 172 selects an optimal inter prediction method from inter prediction using multiple reference images (typically, bi-prediction) and inter prediction using one reference image (unidirectional prediction), and performs inter prediction using the selected inter prediction method. The inter prediction unit 172 outputs control information related to the inter prediction (information on the inter prediction method, motion vector, etc.) to the entropy coding unit 130. When multiple reference images are used to generate the inter prediction image, the inter prediction unit 172 outputs the multiple reference images to the evaluation unit 180.

[0032] Note that prediction using multiple reference images is typically performed by bi-prediction in inter prediction, but is not limited to this. The prediction unit 170 may perform prediction by intra block copying using multiple reference images. In intra block copying, a reference image in the same frame as the current frame is used to predict a block in the current frame. When performing prediction by intra block copying using multiple reference images, the prediction unit 170 outputs the multiple reference images to the evaluation unit 180.

[0033] The switching unit 173 switches between the intra-predicted image input from the intra-prediction unit 171 and the inter-predicted image input from the inter-prediction unit 172, and outputs either of the predicted images to the subtraction unit 110 and the synthesis unit 150.

[0034] The evaluation unit 180 evaluates the similarity between the plurality of reference images input from the prediction unit 170 on a pixel-by-pixel basis, thereby generating map information indicating the distribution of errors in the predicted image generated using the plurality of reference images, and outputs the generated map information to the determination unit 190. Details of the evaluation unit 180 will be described later.

[0035] The determination unit 190 determines an orthogonal transform to be applied to a prediction residual corresponding to a predicted image whose prediction accuracy has been evaluated by the evaluation unit 180, based on the map information input from the evaluation unit 180, and outputs the determined orthogonal transform to the transformation unit 121 and the inverse transformation unit 142. In the first embodiment, the determination unit 190 includes an adaptive transformation generation unit 191 that generates an orthogonal transformation in the vertical direction and an orthogonal transformation in the horizontal direction, based on the map information. The transformation unit 121 performs orthogonal transformation processing in accordance with the orthogonal transformation input from the determination unit 190. The inverse transformation unit 142 performs orthogonal transformation processing in accordance with the orthogonal transformation input from the determination unit 190. Details of the adaptive transformation generation unit 191 will be described later.

[0036] (An example of inter prediction) Fig. 2 is a diagram showing an example of inter prediction, where Fig. 2(a) shows bi-prediction as an example of inter prediction, and Fig. 2(b) shows an example of a predicted image generated by bi-prediction.

[0037] As shown in Figure 2(a), bi-prediction refers to frames temporally before and after a target frame (current frame). In the example of Figure 2(a), prediction of a block in an image of the tth frame is performed by referring to the t-1th frame and the t+1th frame. In motion estimation, a portion (block) similar to the target image block is detected within a search range set by the system from the reference frames of the t-1st and t+1st frames.

[0038] The detected location is the reference image. Information indicating the relative position of the reference image with respect to the target image block is indicated by the arrow in the figure and is called a motion vector. The motion vector information is coded by entropy coding together with frame information of the reference image in the image coding device 1. Meanwhile, the image decoding device detects the reference image based on the motion vector information generated by the image coding device 1.

[0039] As shown in Figures 2(a) and 2(b), reference images 1 and 2 detected by motion detection are similar partial images aligned with the target image block within the reference frame, and are therefore similar to the target image block (image to be encoded). In the example of Figure 2(b), the target image block includes a star pattern and a partial circle pattern. Reference image 1 includes a star pattern and a full circle pattern. Reference image 2 includes a star pattern but does not include a circle pattern.

[0040] A predicted image is generated from these reference images 1 and 2. Note that prediction processing generally generates a predicted image with the characteristics of each reference image by averaging reference images 1 and 2, which have different characteristics but are partially similar. However, a predicted image may also be generated by using more advanced processing, for example, signal enhancement processing using a low-pass filter or a high-pass filter. Here, since reference image 1 contains a circular pattern and reference image 2 does not contain a circular pattern, when reference images 1 and 2 are averaged to generate a predicted image, the signal of the circular pattern in the predicted image is reduced by half compared to reference image 1.

[0041] The difference between the predicted image obtained from reference images 1 and 2 and the target image block (image to be encoded) is the prediction residual. In the prediction residual shown in Figure 2(b), a large difference occurs only in the misaligned parts of the edges of the star pattern and the misaligned parts of the circle pattern (shaded areas), but for other parts, predictions are made with high accuracy and the difference is small (no difference occurs in the example of Figure 2(b)).

[0042] The areas where no difference occurs (non-edge areas of the star pattern and background areas) are areas where the similarity between reference image 1 and reference image 2 is high and where highly accurate predictions have been made. On the other hand, the areas where a large difference occurs are areas specific to each reference image, that is, areas where the similarity between reference image 1 and reference image 2 is significantly low. Therefore, it can be seen that areas where the similarity between reference image 1 and reference image 2 is significantly low have low prediction accuracy and cause large differences (residuals).

[0043] When a prediction residual containing a mixture of large and small differences is orthogonally transformed and quantization causes degradation of the transform coefficients, the degradation propagates throughout the image (block) through inverse quantization and inverse orthogonal transform. When the prediction residual (restored prediction residual) restored by inverse quantization and inverse orthogonal transform is synthesized with a predicted image to reconstruct the target image block, the degradation of image quality propagates to areas where high-precision predictions were made, such as the non-edge and background areas of the star pattern shown in Figure 2(b).

[0044] (Evaluation Department) Fig. 3 is a diagram showing an example of the configuration of the evaluation section 180. As shown in Fig. 3, the evaluation section 180 includes a difference calculation section (subtraction section) 180a, a normalization section 180b, and an adjustment section 180c.

[0045] The difference calculation unit 180a calculates the difference value (absolute value of the difference) between reference image 1 and reference image 2 on a pixel-by-pixel basis, and outputs the calculated difference value to the normalization unit 180b. Such difference value is an example of a value indicating the similarity. It can be said that the smaller the difference value, the higher the similarity, and the larger the difference value, the lower the similarity. The difference calculation unit 180a may calculate the difference value after performing a filter process on each reference image. The difference calculation unit 180a may calculate a statistic such as a squared error, and use the statistic as the similarity.

[0046] The normalization unit 180b normalizes the difference value input from the difference calculation unit 180a by the largest difference value in the block (i.e., the maximum value of the difference values ​​in the block) and outputs the normalized value. The smaller the difference value, the higher the similarity and the higher the prediction accuracy. On the other hand, the larger the difference value, the lower the similarity and the lower the prediction accuracy (the larger the prediction error).

[0047] The normalization unit 180b normalizes the difference value of each pixel input from the difference calculation unit 180a by the difference value of the pixel with the largest difference value in the block (i.e., the maximum value of the difference values ​​in the block), and outputs the normalized difference value. Such a normalized difference value can be used as an estimate representing the magnitude of the prediction error.

[0048] The adjustment unit 180c adjusts the normalized difference value input from the normalization unit 180b based on a quantization parameter (Qp) that determines the coarseness of quantization, and outputs the adjusted normalized difference value. Since the degree of degradation of the restored prediction residual increases as the coarseness of quantization increases, the adjustment unit 180c adjusts the normalized difference value (weight) based on the quantization parameter (Qp).

[0049] The estimated value Rij of the prediction error at each pixel position (ij) output by the evaluation unit 180 can be expressed, for example, as in the following equation (1).

[0050] Rij = (abs(Xij-Yij) / maxD × Scale(Qp)) ···(1)

[0051] In equation (1), Xij is the pixel value of pixel ij in reference image 1, Yij is the pixel value of pixel ij in reference image 2, and abs is a function for obtaining the absolute value. The difference calculation unit 180a outputs abs(Xij-Yij).

[0052] In addition, in equation (1), maxD is the maximum value of the difference values ​​abs(Xij-Yij) within the block. To calculate maxD, it is necessary to calculate the difference values ​​for all pixels within the block. However, to omit this process, the maximum value of an adjacent block that has already been coded may be used instead. Alternatively, maxD may be calculated from the quantization parameter (Qp) using a table that defines the correspondence between the quantization parameter (Qp) and maxD. Alternatively, a fixed value defined in advance in the specifications may be used as maxD. The normalization unit 180b outputs abs(Xij-Yij) / maxD.

[0053] In addition, in equation (1), Scale(Qp) is a coefficient multiplied according to the quantization parameter (Qp). Scale(Qp) is designed to approach 1.0 when Qp is large and to approach 0 when Qp is small, and the degree of this is adjusted depending on the system. Alternatively, a fixed value defined in advance in the specifications may be used as Scale(Qp). Furthermore, to simplify processing, Scale(Qp) may be a fixed value, such as 1.0, designed according to the system.

[0054] The adjustment unit 180c outputs abs(Xij-Yij) / maxD×Scale(Qp) as the error estimate Rij. This Rij may be weighted and adjusted using a sensitivity function designed for the system. For example, the sensitivity may be adjusted by setting abs(Xij-Yij) / maxD×Scale(Qp)=Rij and setting Rij=Clip(Rij,1.0,0.0), or by adding an offset such as Rij=Clip(Rij+offset,1.0,0.0). Note that Clip(x,max,min) indicates a process of clipping at max if x exceeds max, or at min if x is below min.

[0055] The error estimate Rij for each pixel position calculated in this way is a value within the range from 0 to 1.0. Basically, the error estimate Rij approaches 1.0 when the difference value of pixel position ij between the reference images is large (i.e., the prediction accuracy is low), and approaches 0 when the difference value of pixel position ij between the reference images is small (i.e., the prediction accuracy is high). The evaluation unit 180 outputs two-dimensional map information (hereinafter referred to as "error map") consisting of the error estimate Rij for each pixel position ij in the block.

[0056] (Adaptive transformation generation unit) 4 is a diagram showing the operation of the adaptive transform generation unit 191 according to the first embodiment. The adaptive transform generation unit 191 uses the error map input from the evaluation unit 180 to generate a vertical adaptive orthogonal transform to be applied to the prediction residual in the vertical direction and a horizontal adaptive orthogonal transform to be applied to the prediction residual in the horizontal direction by principal component analysis.

[0057] Specifically, the adaptive transformation generation unit 191 regards the error map as a set of column vectors to generate a covariance matrix, and calculates the eigenvectors of the generated covariance matrix. The adaptive transformation generation unit 191 outputs the obtained eigenvectors as a vertical adaptive orthogonal transform. The adaptive transformation generation unit 191 also regards the matrix obtained by applying the generated vertical adaptive orthogonal transform in the vertical direction as a set of row vectors to generate a covariance matrix, and calculates its eigenvectors. The adaptive transformation generation unit 191 outputs the obtained eigenvectors as a horizontal adaptive orthogonal transform.

[0058] As shown in FIG. 4(a), when the error map has width w and height h, the adaptive transformation generation unit 191 regards the error map as w column vectors of h×1, and calculates a covariance matrix Λ h The adaptive transformation generation unit 191 calculates the eigenvectors by diagonalizing the obtained covariance matrix. Here, the covariance matrix is ​​diagonalized by iterative calculation using, for example, the Jacobi method. The adaptive transformation generation unit 191 calculates the eigenvectors e0 to e h The transformation combination outputs an hxh matrix as the vertical adaptive orthogonal transformation.

[0059] Furthermore, as shown in FIG. 4(b), the adaptive transformation generator 191 regards the error map as h 1×w row vectors and calculates a covariance matrix Λ h The adaptive transformation generation unit 191 calculates eigenvectors by diagonalizing the obtained covariance matrix. The adaptive transformation generation unit 191 outputs a w × w matrix as a horizontal adaptive orthogonal transformation by combining the obtained eigenvectors.

[0060] In video coding methods such as HEVC (see Non-Patent Document 1) and the latest video coding technology (JEM) currently being studied by an international standardization organization, transform coefficients with integer precision and bit shifting are used to achieve high-speed and lightweight transform processing. The adaptive transform generation unit 191 according to this embodiment may also approximate the obtained eigenvectors to integer coefficients, and the amount of bit shifting that does not expand the dynamic range of the transform coefficients may be specified in advance in the image coding device and the image decoding device.

[0061] As shown in Figure 1, the transform unit 121 calculates transform coefficients by performing orthogonal transform processing in the vertical and horizontal directions on the prediction residuals generated by the subtraction unit 110 using the vertical adaptive orthogonal transform and horizontal adaptive orthogonal transform input from the adaptive transform generation unit 191, and outputs the calculated transform coefficients to the quantization unit 122.

[0062] In addition, the inverse transform unit 142 performs an inverse orthogonal transform process corresponding to the orthogonal transform process performed by the transform unit 121 on the transform coefficients input from the inverse quantization unit 141, using the vertical adaptive orthogonal transform and horizontal adaptive orthogonal transform input from the adaptive transform generation unit 191.

[0063] (Image decoding device) Fig. 5 is a diagram showing the configuration of an image decoding device 2 according to the first embodiment. As shown in Fig. 5, the image decoding device 2 includes an entropy code decoding unit 200, an inverse quantization and inverse transform unit 210, a synthesis unit 220, a memory 230, a prediction unit 240, an evaluation unit 250, and a determination unit 260.

[0064] The entropy code decoding unit 200 decodes the coded data generated by the image coding device 1, and outputs the quantized transform coefficients to the inverse quantization and inverse transform unit 210. The entropy code decoding unit 200 also obtains control information related to prediction (intra prediction and inter prediction), and outputs the obtained control information to the prediction unit 240.

[0065] The inverse quantization and inverse transform unit 210 performs inverse quantization processing and inverse orthogonal transform processing on a block-by-block basis. The inverse quantization and inverse transform unit 210 includes an inverse quantization unit 211 and an inverse transform unit 212.

[0066] The inverse quantization unit 211 performs inverse quantization processing corresponding to the quantization processing performed by the quantization unit 122 of the image encoding device 1. The inverse quantization unit 211 restores the transform coefficients by inverse quantizing the quantized transform coefficients input from the entropy encoding / decoding unit 200 using a quantization parameter (Qp) and a quantization matrix, and outputs the restored transform coefficients to the inverse transform unit 212.

[0067] The inverse transform unit 212 performs inverse orthogonal transform processing corresponding to the orthogonal transform processing performed by the transform unit 121 of the image encoding device 1. The inverse transform unit 212 performs inverse orthogonal transform processing on the transform coefficients input from the inverse quantization unit 211 to restore prediction residuals, and outputs the restored prediction residuals (restored prediction residuals) to the synthesis unit 220.

[0068] The synthesis unit 220 reconstructs (decodes) the original block by synthesizing the prediction residual input from the inverse transform unit 212 and the prediction image input from the prediction unit 240 on a pixel-by-pixel basis, and outputs the decoded image on a block-by-block basis to the memory 230.

[0069] The memory 230 stores the decoded image input from the synthesis unit 220. The memory 230 stores the decoded image in units of frames. The memory 230 outputs the decoded image in units of frames to the outside of the image decoding device 2. Note that a loop filter may be provided between the synthesis unit 220 and the memory 230.

[0070] The prediction unit 240 performs prediction on a block-by-block basis and includes an intra prediction unit 241, an inter prediction unit 242, and a switching unit 243.

[0071] The intra prediction unit 241 references the decoded image stored in the memory 230 and performs intra prediction according to the control information input from the entropy coding decoding unit 200 to generate an intra prediction image, and outputs the generated intra prediction image to the switching unit 243.

[0072] The inter prediction unit 242 performs inter prediction to predict a block to be predicted using a decoded image stored in the memory 230 as a reference image. The inter prediction unit 242 generates an inter predicted image by performing inter prediction in accordance with control information (inter prediction method, motion vector information, etc.) input from the entropy code decoding unit 200, and outputs the generated inter predicted image to the switching unit 243. When multiple reference images are used to generate the inter predicted image, the inter prediction unit 242 outputs the multiple reference images to the evaluation unit 250.

[0073] Note that prediction using multiple reference images is typically performed by bi-prediction in inter prediction, but is not limited to this. The prediction unit 240 may perform prediction by intra block copying using multiple reference images. When performing prediction by intra block copying using multiple reference images, the prediction unit 240 outputs the multiple reference images to the evaluation unit 250.

[0074] The switching unit 243 switches between the intra-predicted image input from the intra-prediction unit 241 and the inter-predicted image input from the inter-prediction unit 242 , and outputs either of the predicted images to the synthesis unit 220 .

[0075] The evaluation unit 250 performs the same operation as the evaluation unit 180 (see FIG. 3 ) of the image encoding device 1. The evaluation unit 250 evaluates the similarity between the multiple reference images input from the prediction unit 240 on a pixel-by-pixel basis to generate an error map indicating the distribution of errors in the predicted image generated using the multiple reference images, and outputs the generated error map to the determination unit 260.

[0076] The determination unit 260 determines, based on the error map input from the evaluation unit 250, an inverse orthogonal transform to be applied to a prediction residual corresponding to a predicted image whose prediction accuracy has been evaluated by the evaluation unit 250, and outputs the determined inverse orthogonal transform to the inverse transform unit 212. The determination unit 260 includes an adaptive transform generation unit 261 that generates an orthogonal transform in the vertical direction and an orthogonal transform in the horizontal direction based on the error map. The adaptive transform generation unit 261 performs the same operation as the adaptive transform generation unit 191 of the image encoding device 1 (see FIG. 4 ). The inverse transform unit 212 performs inverse orthogonal transform processing in accordance with the inverse orthogonal transform input from the determination unit 260.

[0077] (Summary of the first embodiment) An image encoding device 1 according to the first embodiment encodes a target image in units of blocks obtained by dividing a current image in units of frames. The image encoding device 1 includes a prediction unit 170 that predicts the target image using multiple reference images to generate a predicted image, an evaluation unit 180 that generates an error map indicating the distribution of errors in the predicted image by evaluating the similarity between the multiple reference images on a pixel-by-pixel basis, a subtraction unit 110 that calculates a prediction residual indicating the difference between the target block and the predicted image on a pixel-by-pixel basis, a determination unit 190 that determines an orthogonal transform to apply to the prediction residual based on the error map, and a transformation unit 121 that performs orthogonal transform processing on the prediction residual using the determined orthogonal transform. The determination unit 190 includes an adaptive transform generation unit 191 that generates an orthogonal transform based on map information.

[0078] The image decoding device 2 according to the first embodiment decodes target images in blocks obtained by dividing a current image in frames. The image decoding device 2 includes an entropy code decoding unit 200 that obtains transform coefficients by decoding encoded data, a prediction unit 240 that predicts a target image using multiple reference images to generate a predicted image, an evaluation unit 250 that generates an error map indicating the distribution of errors in the predicted image by evaluating the similarity between the multiple reference images on a pixel-by-pixel basis, a determination unit 260 that determines an inverse orthogonal transform to be applied to the transform coefficients based on the error map, and an inverse transform unit 212 that performs inverse orthogonal transform processing on the transform coefficients using the determined inverse orthogonal transform. The determination unit 260 includes an adaptive transform generation unit 261 that generates an inverse orthogonal transform based on the map information.

[0079] As described above, according to the first embodiment, by generating an orthogonal transform based on an error map indicating the distribution of errors in a predicted image, it is possible to apply an optimal orthogonal transform according to the energy distribution in the prediction residuals.

[0080] Furthermore, since the image encoding device 1 and the image decoding device 2 can each generate an orthogonal transform based on an error map, the image decoding device 2 does not need information for the inverse process of the orthogonal transform (KLT) performed by the image encoding device 1. This makes it possible to suppress an increase in the amount of information to be transmitted.

[0081] Therefore, the image encoding device 1 and the image decoding device 2 according to the first embodiment make it possible to apply adaptive orthogonal transform that efficiently concentrates the energy of prediction residuals, thereby improving encoding efficiency.

[0082] Second Embodiment The image encoding device 1 and image decoding device 2 according to the second embodiment will be described, focusing on the differences from the first embodiment.

[0083] (Image encoding device) Fig. 6 is a diagram showing the configuration of an image encoding device 1 according to the second embodiment. As shown in Fig. 6, the image encoding device 1 according to the second embodiment differs from that according to the first embodiment in the configuration of a decision unit 190. As in the first embodiment, the decision unit 190 includes an adaptive transform generation unit 191 that generates an orthogonal transform by principal component analysis of an error map (see Fig. 4). In the second embodiment, the decision unit 190 further includes a candidate selection unit (first selection unit) 192 and an orthogonal transform selection unit (second selection unit) 193.

[0084] The adaptive transform generating unit 191 outputs the adaptive orthogonal transforms (vertical adaptive orthogonal transform and horizontal adaptive orthogonal transform) generated based on the error map to the candidate selecting unit 192.

[0085] The candidate selection unit 192 selects one or more orthogonal transform candidates from among a plurality of predefined types of orthogonal transforms in descending order of correlation with the adaptive orthogonal transform input from the adaptive transform generation unit 191, and outputs the one or more selected orthogonal transform candidates to the orthogonal transform selection unit 263. The plurality of predefined types of orthogonal transforms are shared by the image encoding device 1 and the image decoding device 2. In the second embodiment, it is assumed that DCT-2, DST-7, DCT-8, DST-1, and DCT-5 are predefined as the plurality of types of orthogonal transforms.

[0086] Specifically, the candidate selection unit 192 evaluates the correlation between the horizontal adaptive orthogonal transform input from the adaptive transform generation unit 191 and each orthogonal transform (DCT-2, DST-7, DCT-8, DST-1, DCT-5). Then, based on the correlation evaluation result, the candidate selection unit 192 outputs one or more orthogonal transforms from among the multiple types of orthogonal transforms in descending order of correlation as adaptive orthogonal transform candidates to the orthogonal transform selection unit 193.

[0087] The number of adaptive orthogonal transform candidates output by the candidate selection unit 192 is specified in advance to be the same for the image encoding device 1 and the image decoding device 2. The number of adaptive orthogonal transform candidates output by the candidate selection unit 192 may be variable depending on the block size and color components (luminance component, chrominance component) of the image block to be encoded.

[0088] Furthermore, the candidate selection unit 192 may select adaptive orthogonal transform candidates separately for the horizontal and vertical directions. In this case, the candidate selection unit 192 further evaluates the correlation between the vertical adaptive orthogonal transform input from the adaptive transform generation unit 191 and each orthogonal transform (DCT-2, DST-7, DCT-8, DST-1, DCT-5), and outputs adaptive vertical orthogonal transform candidates in addition to the adaptive horizontal orthogonal transform candidates to the orthogonal transform selection unit 193.

[0089] The orthogonal transform selection unit 193 selects an orthogonal transform to be applied to the prediction residual from one or more orthogonal transform candidates input from the candidate selection unit 192, and outputs the selected orthogonal transform to the transform unit 121 and the inverse transform unit 142. The orthogonal transform selection unit 193 also outputs an index indicating the type of the selected orthogonal transform to the entropy coding unit 130.

[0090] For example, the orthogonal transform selection unit 193 calculates the coding efficiency when each orthogonal transform candidate is applied by simulation, and selects the optimal orthogonal transform according to the results of the simulation. Note that the orthogonal transform selection unit 193 may be configured to select the same type of orthogonal transform in the horizontal and vertical directions, or may be configured to select different types of orthogonal transform in the horizontal and vertical directions.

[0091] The entropy coding unit 130 entropy codes the adaptive orthogonal transform index input from the orthogonal transform selection unit 193. When selecting different types of orthogonal transform in the horizontal and vertical directions, the entropy coding unit 130 entropy codes different adaptive orthogonal transform indices for the horizontal and vertical directions. However, when the adaptive orthogonal transform candidates are configured by one type of orthogonal transform, the orthogonal transform can be uniquely identified in the image decoding device 2, and therefore such indices do not need to be entropy coded.

[0092] (Image decoding device) Fig. 7 is a diagram showing the configuration of an image decoding device 2 according to the second embodiment. As shown in Fig. 7, the image decoding device 2 according to the second embodiment differs from the first embodiment in the configuration of the decision unit 260. As in the first embodiment, the decision unit 260 includes an adaptive transform generation unit 261 that generates an orthogonal transform by principal component analysis of an error map (see Fig. 4). In the second embodiment, the decision unit 260 further includes a candidate selection unit (first selection unit) 262 and an orthogonal transform selection unit (second selection unit) 263.

[0093] The adaptive transform generation unit 261 outputs the adaptive orthogonal transforms (vertical adaptive orthogonal transform and horizontal adaptive orthogonal transform) generated based on the error map to the candidate selection unit 262.

[0094] The candidate selection unit 262 selects one or more orthogonal transform candidates from among a plurality of predefined types of orthogonal transforms in descending order of correlation with the adaptive orthogonal transform input from the adaptive transform generation unit 261, and outputs the one or more selected orthogonal transform candidates to the orthogonal transform selection unit 263. As described above, it is assumed that DCT-2, DST-7, DCT-8, DST-1, and DCT-5 are predefined as a plurality of types of orthogonal transforms.

[0095] Specifically, the candidate selection unit 262 evaluates the correlation between the horizontal adaptive orthogonal transform input from the adaptive transform generation unit 261 and each orthogonal transform (DCT-2, DST-7, DCT-8, DST-1, DCT-5). Then, based on the result of the correlation evaluation, the candidate selection unit 262 outputs one or more orthogonal transforms from among the multiple types of orthogonal transforms in descending order of correlation as adaptive orthogonal transform candidates to the orthogonal transform selection unit 263.

[0096] As described above, the number of adaptive orthogonal transform candidates output by the candidate selection unit 262 is specified in advance to be the same for the image encoding device 1 and the image decoding device 2. Furthermore, the number of adaptive orthogonal transform candidates output by the candidate selection unit 262 may be variable depending on the block size and color components (luminance component, chrominance component) of the image block to be decoded, etc.

[0097] Furthermore, the candidate selection unit 262 may select adaptive orthogonal transform candidates separately for the horizontal and vertical directions. In this case, the candidate selection unit 262 further evaluates the correlation between the vertical adaptive orthogonal transform input from the adaptive transform generation unit 261 and each orthogonal transform (DCT-2, DST-7, DCT-8, DST-1, DCT-5), and outputs adaptive vertical orthogonal transform candidates in addition to the adaptive horizontal orthogonal transform candidates to the orthogonal transform selection unit 263.

[0098] On the other hand, the entropy code decoding unit 200 decodes an index indicating the orthogonal transform selected by the image encoding device 1 from one or more orthogonal transform candidates, and outputs the index to the orthogonal transform selection unit 263. When different types of orthogonal transform are selected in the horizontal and vertical directions, the entropy code decoding unit 200 decodes different adaptive orthogonal transform indices for the horizontal and vertical directions.

[0099] The orthogonal transform selection unit 263 selects an inverse orthogonal transform to be applied to the transform coefficients from one or more orthogonal transform candidates input from the candidate selection unit 262, based on the index input from the entropy coding / decoding unit 200, and outputs the selected inverse orthogonal transform to the inverse transform unit 212. Note that the orthogonal transform selection unit 263 may be configured to select the same type of orthogonal transform in the horizontal and vertical directions, or may be configured to select different types of orthogonal transform in the horizontal and vertical directions.

[0100] (Summary of the second embodiment) In the image coding device 1 according to the second embodiment, the determination unit 190 includes an adaptive transform generation unit 191 that generates an orthogonal transform by principal component analysis of an error map, a candidate selection unit 192 that selects one or more orthogonal transform candidates from among multiple predefined types of orthogonal transforms in descending order of correlation with the generated orthogonal transform, and an orthogonal transform selection unit 193 that selects an orthogonal transform to apply to a prediction residual from among the one or more orthogonal transform candidates. The entropy coding unit 130 codes an index indicating the orthogonal transform selected by the orthogonal transform selection unit 193 from among the one or more orthogonal transform candidates.

[0101] Furthermore, in the image decoding device 2 according to the second embodiment, the determination unit 260 includes an adaptive transform generation unit 261 that generates an orthogonal transform by principal component analysis of an error map, a candidate selection unit 262 that selects one or more orthogonal transform candidates from among multiple predefined types of orthogonal transforms in descending order of correlation with the generated orthogonal transform, and an orthogonal transform selection unit 263 that selects an inverse orthogonal transform to apply to the transform coefficients from among the one or more orthogonal transform candidates. The entropy code decoding unit 200 decodes an index indicating the orthogonal transform selected by the image encoding device 1 from among the one or more orthogonal transform candidates. The orthogonal transform selection unit 263 selects an inverse orthogonal transform to apply to the transform coefficients from among the one or more orthogonal transform candidates based on the index.

[0102] As described above, according to the second embodiment, from among a plurality of predefined orthogonal transforms, orthogonal transform candidates highly correlated with the orthogonal transform (KLT) generated by principal component analysis of the error map are selected, and an orthogonal transform to be applied to the transform processing is selected from among the orthogonal transform candidates. The predefined orthogonal transforms (DCT-2, DST-7, DCT-8, DST-1, DCT-5) require less computational processing than the KLT.

[0103] Therefore, according to the second embodiment, it is possible to apply an orthogonal transform that efficiently concentrates the energy of prediction residuals, thereby improving coding efficiency, while reducing the amount of calculation required for the transform process compared to the first embodiment.

[0104] <Third embodiment> The image encoding device 1 and image decoding device 2 according to the third embodiment will be described, focusing on the differences from the first and second embodiments.

[0105] In the second embodiment, an orthogonal transform candidate highly correlated with an orthogonal transform (KLT) generated by principal component analysis of an error map is selected from among a plurality of predefined types of orthogonal transforms. In contrast, in the third embodiment, an orthogonal transform candidate is selected from among a plurality of predefined types of orthogonal transforms by evaluating the feature quantities of an error map.

[0106] (Image encoding device) 8 is a diagram showing the configuration of an image encoding device 1 according to the third embodiment. As shown in Fig. 8, the image encoding device 1 according to the third embodiment differs from the second embodiment in that the determining unit 190 includes a feature amount evaluating unit 191a. The feature amount evaluating unit 191a evaluates the feature amounts of the error map input from the evaluating unit 180 and outputs the evaluation result to the candidate selecting unit 192.

[0107] 9 is a diagram showing the operation of the feature amount evaluation unit 191a. As shown in FIG. 9, in order to evaluate the energy distribution of the error map, the feature amount evaluation unit 191a divides the error map into four parts horizontally and four parts vertically, and calculates the sum of the error estimate values ​​Exy(E 00~E 33 The feature amount evaluation unit 191a calculates the evaluated energy distribution E 00 ~E 33 to the candidate selection unit 192.

[0108] The candidate selection unit 192 selects an adaptive orthogonal transform candidate from among multiple predefined types of orthogonal transform (DCT-2, DST-7, DCT-8, DST-1, DCT-5) based on the following conditions, and outputs the selected adaptive orthogonal transform candidate to the orthogonal transform selection unit 193.

[0109] (a) The candidate selection unit 192 selects the candidate in the horizontal direction as follows:

[0110]

number

[0111] select DCT-2 and DST-1 as horizontal adaptive orthogonal transform candidates when

[0112]

number

[0113] When , DCT-2 and DST-7 are selected as horizontal adaptive orthogonal transform candidates;

[0114]

number

[0115] select DCT-2 and DCT-5 as horizontal adaptive orthogonal transform candidates when If neither of these applies, DCT-8 and DST-7 are selected as horizontally adaptive orthogonal transform candidates.

[0116] (b) The candidate selection unit 192 selects the candidate in the vertical direction as follows:

[0117]

number

[0118] select DCT-2 and DST-1 as vertical adaptive orthogonal transform candidates when

[0119]

number

[0120] select DCT-2 and DST-7 as vertical adaptive orthogonal transform candidates when

[0121]

number

[0122] select DCT-2 and DCT-5 as vertical adaptive orthogonal transform candidates when If neither of these applies, DCT-8 and DST-7 are selected as candidates for vertical adaptive orthogonal transform.

[0123] Here, the number of adaptive orthogonal transform candidates output by the candidate selection unit 192 is two in each of the horizontal and vertical directions, but the number of adaptive orthogonal transform candidates output by the candidate selection unit 192 may be variable depending on the block size and color components (luminance component, chrominance component) of the image block to be coded, etc.

[0124] As in the second embodiment, the orthogonal transform selection unit 193 selects an orthogonal transform to be applied to the prediction residual from the orthogonal transform candidates input from the candidate selection unit 192, and outputs the selected orthogonal transform to the transform unit 121 and the inverse transform unit 142. The orthogonal transform selection unit 193 also outputs an index indicating the type of the selected orthogonal transform to the entropy coding unit 130. The entropy coding unit 130 entropy codes the adaptive orthogonal transform index input from the orthogonal transform selection unit 193.

[0125] (Image decoding device) Fig. 10 is a diagram showing the configuration of an image decoding device 2 according to the third embodiment. As shown in Fig. 10, the image decoding device 2 according to the third embodiment differs from the second embodiment in that the determination unit 260 includes a feature amount evaluation unit 261a. The feature amount evaluation unit 261a performs the same operation as the feature amount evaluation unit 191a of the image encoding device 1 (see Fig. 9).

[0126] In order to evaluate the energy distribution of the error map, the feature amount evaluation unit 261a divides the error map into four parts horizontally and vertically, and calculates the sum of the error estimates Exy(E 00 ~E 33 The feature evaluation unit 261a calculates the evaluated energy distribution E 00 ~E 33 to the candidate selection unit 262.

[0127] The candidate selection unit 262 selects an adaptive orthogonal transform candidate from among multiple predefined types of orthogonal transform (DCT-2, DST-7, DCT-8, DST-1, DCT-5) based on conditions similar to those of the candidate selection unit 192 of the image encoding device 1, and outputs the selected adaptive orthogonal transform candidate to the orthogonal transform selection unit 263.

[0128] Meanwhile, the entropy code decoding unit 200 decodes an index indicating the orthogonal transform selected by the image encoding device 1 from among the orthogonal transform candidates, and outputs the index to the orthogonal transform selection unit 263. The orthogonal transform selection unit 263 selects an inverse orthogonal transform to apply to the transform coefficients from among the orthogonal transform candidates input from the candidate selection unit 262, based on the index input from the entropy code decoding unit 200, and outputs the selected inverse orthogonal transform to the inverse transform unit 212.

[0129] (Summary of the third embodiment) In the image coding device 1 according to the third embodiment, the determination unit 190 includes a feature evaluation unit 191a that evaluates feature amounts of the error map, a candidate selection unit 192 that selects one or more orthogonal transform candidates from multiple predefined types of orthogonal transform based on the evaluated feature amounts, and an orthogonal transform selection unit 193 that selects an orthogonal transform to be applied to the prediction residual from the one or more orthogonal transform candidates. The entropy coding unit 130 codes an index indicating the orthogonal transform selected by the orthogonal transform selection unit 193 from the one or more orthogonal transform candidates.

[0130] Furthermore, in the image decoding device 2 according to the third embodiment, the determination unit 260 includes a feature evaluation unit 261a that evaluates features of the error map, a candidate selection unit 262 that selects one or more orthogonal transform candidates from multiple predefined types of orthogonal transform based on the evaluated features, and an orthogonal transform selection unit 263 that selects an inverse orthogonal transform to be applied to the transform coefficients from the one or more orthogonal transform candidates. The entropy code decoding unit 200 decodes an index indicating the orthogonal transform selected by the image encoding device 1 from the one or more orthogonal transform candidates. The orthogonal transform selection unit 263 selects an inverse orthogonal transform to be applied to the transform coefficients from the one or more orthogonal transform candidates based on the index.

[0131] As described above, according to the third embodiment, the feature amount of the error map can be evaluated by simple calculation processing, and therefore the amount of calculation processing can be reduced compared to the second embodiment in which an orthogonal transform (KLT) is generated by principal component analysis of the error map.

[0132] Therefore, according to the third embodiment, it is possible to apply an orthogonal transform that efficiently concentrates the energy of the prediction residual, thereby improving coding efficiency, while reducing the amount of computation required to analyze the error map compared to the second embodiment.

[0133] <Other embodiments> In the first to third embodiments described above, an example has been described in which one-dimensional orthogonal transform is used to perform transform processing separately in the vertical direction and the vertical direction. However, instead of one-dimensional orthogonal transform, two-dimensional orthogonal transform may be used to perform transform processing in the vertical direction and the vertical direction together.

[0134] Alternatively, the image encoding device 1 may be provided by a program causing a computer to execute each process performed by the image encoding device 1, and a program causing a computer to execute each process performed by the image decoding device 2. The program may be recorded on a computer-readable medium. Using the computer-readable medium makes it possible to install the program on a computer. Here, the computer-readable medium on which the program is recorded may be a non-transitory recording medium. The non-transitory recording medium is not particularly limited, and may be, for example, a recording medium such as a CD-ROM or a DVD-ROM.

[0135] Furthermore, circuits that execute the processes performed by the image encoding device 1 may be integrated, and the image encoding device 1 may be configured as a semiconductor integrated circuit (chip set, SoC). Similarly, circuits that execute the processes performed by the image decoding device 2 may be integrated, and the image decoding device 2 may be configured as a semiconductor integrated circuit (chip set, SoC).

[0136] The above describes the embodiments in detail with reference to the drawings, but the specific configuration is not limited to that described above, and various design changes can be made within the scope that does not deviate from the gist of the invention. [Explanation of symbols]

[0137] 1: Image encoding device 2: Image decoding device 100: Block division section 110: Subtraction section 120: Transformation and quantization unit 121: Conversion section 122: Quantization section 130: Entropy coding unit 140: Inverse quantization and inverse transformation unit 141: Inverse quantization section 142: Inverse conversion unit 150: Synthesis section 160: Memory 170: Prediction Department 171: Intra prediction unit 172: Inter prediction unit 173: Switching section 180: Evaluation section 180a: Difference calculation section 180b: Normalization section 180c: Adjustment section 190: Decision section 191: Adaptive transformation generation unit 191a: Feature evaluation unit 192: Candidate selection section 193: Orthogonal transform selection unit 200: Entropy coding and decoding unit 210: Inverse quantization and inverse transformation unit 211: Inverse quantization section 212: Inverse conversion unit 220: Synthesis section 230: Memory 240: Prediction section 241: Intra prediction unit 242: Inter prediction unit 243: Switching section 250: Evaluation section 260: Decision section 261: Adaptive transformation generation unit 261a: Feature evaluation unit 262: Candidate selection section 263: Orthogonal transform selection unit

Claims

1. An image encoding device that encodes a target image in units of blocks obtained by dividing a current image in units of frames, comprising: a block dividing unit that divides the current image into blocks; a prediction unit that predicts the target image by inter prediction including bi-prediction using a plurality of reference images to generate a predicted image; an evaluation unit that calculates a value indicating a similarity between the plurality of reference images in units of regions smaller than the block and consisting of a plurality of pixels only when the prediction unit performs the bi-prediction using a reference image that is temporally earlier and a reference image that is temporally later than the target image, an image encoding device, characterized in that the evaluation unit controls encoding processing based on the value calculated for each region.

2. An image decoding device that decodes a target image in units of blocks obtained by dividing a current image in units of frames, comprising: a prediction unit that predicts the target image by inter prediction including bi-prediction using a plurality of reference images to generate a predicted image; an evaluation unit that calculates a value indicating a similarity between the plurality of reference images in units of regions smaller than the block and consisting of a plurality of pixels only when the prediction unit performs the bi-prediction using a reference image that is temporally earlier and a reference image that is temporally later than the target image, The image decoding device controls a decoding process based on the value calculated by the evaluation unit for each region.

3. A program that causes a computer to function as the image encoding device according to claim 1.

4. A program that causes a computer to function as the image decoding device according to claim 2.

Images