Image encoding device, image encoding method, image decoding device, image decoding method, and computer program

Abstract

To provide a technique for improving compression efficiency, which enables control such that an appropriate correction value can be added even to a reproduced orthogonal transform coefficient generated by an inverse quantization process using a quantization matrix.SOLUTION: For a block of interest of a given size in an image to be coded, a predicted image is generated from coded pixels, a prediction error is derived which is the difference between the block of interest and the predicted image, the prediction error is frequency transformed, and the orthogonal transform coefficients obtained by the frequency transformation are quantized. The quantized orthogonal transform coefficients are then entropy coded, and the quantized orthogonal transform coefficients are inverse quantized to generate reconstructed orthogonal transform coefficients. The reconstructed orthogonal transform coefficients are corrected.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an image encoding device, an image encoding method, an image decoding device, an image decoding method, and a computer program. [Background technology]

[0002] The Versatile Video Coding (VVC) coding method (hereafter referred to as VVC) is known as a coding method for compressing and recording moving images. In VVC, basic blocks with a maximum size of 128 pixels x 128 pixels are divided into rectangular sub-blocks in addition to the conventional square sub-blocks to improve coding efficiency.

[0003] In addition, VVC uses a process in which coefficients after orthogonal transformation (hereinafter referred to as orthogonal transformation coefficients) are weighted according to frequency components using a quantization matrix. By further reducing data of high frequency components, degradation of which is less noticeable to human vision, it is possible to improve compression efficiency while maintaining image quality. Patent Document 1 discloses a technology for encoding such a quantization matrix.

[0004] In recent years, the Joint Video Experts Team (JVET), which standardized VVC, has been studying technologies to achieve compression efficiency that exceeds that of VVC. To improve coding efficiency, a new inverse quantization method (hereinafter referred to as inverse quantization correction) that adds correction values ​​to the orthogonal transform coefficients generated by inverse quantization processing (hereinafter referred to as reconstructed orthogonal transform coefficients) is being studied. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2013-38758 A Summary of the Invention [Problem to be solved by the invention]

[0006] This inverse quantization correction is a process of adding a correction value to the reproduced orthogonal transform coefficient, but a predetermined correction value is added regardless of whether a quantization matrix is ​​used. Therefore, it is not possible to perform an appropriate correction value addition process on the reproduced orthogonal transform coefficient generated by the inverse quantization process using the quantization matrix, which causes a problem that the compression efficiency cannot be improved.

[0007] The present invention provides a technique for improving compression efficiency by enabling control so that an appropriate correction value can be added even to reproduced orthogonal transform coefficients generated by inverse quantization processing using a quantization matrix. [Means for solving the problem]

[0008] One aspect of the present invention is an image coding device that codes an image on a block-by-block basis, comprising: a prediction means for generating a predicted image from already-coded pixels for a target block of a predetermined size in the image to be coded, and deriving a prediction error which is the difference between the target block and the predicted image; a transformation means for frequency transforming the prediction error derived by the prediction means; a quantization means for quantizing an orthogonal transform coefficient obtained by frequency transform by the transformation means; an encoding means for entropy encoding the orthogonal transform coefficient quantized by the quantization means; and an inverse quantization means for inverse quantizing the orthogonal transform coefficient quantized by the quantization means to generate reproduced orthogonal transform coefficients, wherein the inverse quantization means corrects the reproduced orthogonal transform coefficients. Effect of the Invention

[0009] According to the present invention, it is possible to provide a technique for improving compression efficiency by enabling control so that an appropriate correction value can be added to the reproduced orthogonal transform coefficients generated by inverse quantization processing using a quantization matrix. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram showing an example of the functional configuration of an image encoding device. [Diagram 2] FIG. 2 is a block diagram showing an example of the functional configuration of an image decoding device. [Diagram 3] 4 is a flowchart of a process performed by the image encoding device to encode one frame of an input image. [Figure 4] 11 is a flowchart of a process performed by an image decoding device to decode a bitstream for one frame. [Diagram 5] FIG. 1 is a block diagram showing an example of the hardware configuration of a computer device applicable to an image encoding device or an image decoding device. [Figure 6] FIG. 2 is a diagram showing an example of a data structure of a bit stream. [Figure 7] FIG. 13 is a diagram showing an example of a division pattern of sub-blocks. [Figure 8] FIG. 8 shows an example of the configuration of a quantization matrix 800. [Figure 9] FIG. [Figure 10] FIG. 2 is a diagram showing an example of the configuration of a one-dimensional difference matrix. [Figure 11] FIG. 4 is a diagram showing an example of the configuration of an encoding table. [Figure 12] FIG. 1 is a diagram showing an example of region division used in mixed intra / inter prediction. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0012] [First embodiment] First, an example of the functional configuration of an image encoding device according to this embodiment will be described with reference to the block diagram of Fig. 1. The image encoding device can be applied to a PC (personal computer), a smartphone, a tablet terminal device, an imaging device, a dedicated image processing circuit, and the like.

[0013] The image coding device acquires an image to be coded as an input image via the input unit 101. The method for the image coding device to acquire the input image is not limited to a specific method. For example, the image coding device may acquire an image output from an imaging device (an image of each frame in a moving image, a still image captured periodically or irregularly, etc.) as an input image. Note that, when the image coding device and the imaging device are integrated, the image coding device acquires an image captured by the imaging device owned by the image coding device as an input image. For example, the image coding device may acquire an image held in an external device such as a server device as an input image via a network such as a LAN or the Internet. For example, the image coding device may acquire an image held in a storage device owned by the image coding device as an input image.

[0014] The block division unit 102 divides an input image into a plurality of basic blocks (hereinafter, simply referred to as blocks, as appropriate). The quantization matrix holding unit 103 acquires and holds a plurality of quantization matrices used in the quantization process. The method by which the quantization matrix holding unit 103 acquires the quantization matrix is ​​not limited to a specific method. For example, the quantization matrix holding unit 103 may acquire a quantization matrix input by a user operating an operation unit (not shown), may calculate a quantization matrix from characteristics of the input image, or may acquire a quantization matrix designated in advance as an initial value. In this embodiment, the quantization matrix holding unit 103 acquires and holds three types of "two-dimensional quantization matrices 800 corresponding to orthogonal transformation (frequency transformation) of 8 pixels x 8 pixels" as exemplified in Figs. 8(a) to (c).

[0015] The inverse quantization correction control unit 114 acquires inverse quantization correction control information, which is information for controlling the inverse quantization correction process executed in the inverse quantization and inverse transform unit 106 at the subsequent stage. The method for the inverse quantization correction control unit 114 to acquire the inverse quantization correction control information is not limited to a specific method. For example, the inverse quantization correction control unit 114 may acquire the inverse quantization correction control information input by the user by operating an operation unit (not shown), or may calculate the inverse quantization correction control information based on the characteristics of the input image. Also, for example, the inverse quantization correction control unit 114 may acquire the inverse quantization correction control information set in advance as an initial value from a memory in the image encoding device or a memory outside the image encoding device.

[0016] The prediction unit 104 divides each basic block divided by the block division unit 102 into one or more sub-blocks, performs prediction processing such as intra-prediction, which is intra-frame prediction, or inter-prediction, which is inter-frame prediction, on the sub-block to generate a predicted image corresponding to the sub-block, and derives the difference (error) between the sub-block and the predicted image as a prediction error. The prediction unit 104 also outputs information required for prediction (such as information on sub-block division, prediction mode, motion vector, etc.) as prediction information.

[0017] The transform / quantization unit 105 generates orthogonal transform coefficients corresponding to the sub-block by performing orthogonal transform (frequency transform) on the prediction error corresponding to the sub-block, and generates quantization coefficients by quantizing the orthogonal transform coefficients using the quantization matrix stored in the quantization matrix storage unit 103. Note that, as an example, a configuration for performing orthogonal transform and a configuration for performing quantization are represented by one block. However, the configuration for performing orthogonal transform and the configuration for performing quantization may be separate.

[0018] The inverse quantization and inverse transform unit 106 generates reproduced orthogonal transform coefficients by inverse quantizing the quantization coefficients generated by the transform and quantization unit 105 using the quantization matrix stored in the quantization matrix storage unit 103. The inverse quantization and inverse transform unit 106 then corrects the generated reproduced orthogonal transform coefficients based on the inverse quantization correction control information acquired by the inverse quantization correction control unit 114, and generates (reproduces) prediction errors by inverse orthogonal transforming the corrected reproduced orthogonal transform coefficients. Note that, as an example, a configuration for performing inverse quantization and a configuration for performing inverse orthogonal transform are represented by one block. However, the configuration for performing inverse quantization and the configuration for performing inverse transform may be separate.

[0019] The image reproduction unit 107 generates a predicted image by appropriately referring to the frame memory 108 based on the prediction information output from the prediction unit 104, generates a reproduced image from the predicted image and the prediction error generated (reproduced) by the inverse quantization / inverse transform unit 106, and stores the reproduced image in the frame memory 108.

[0020] The in-loop filter unit 109 performs in-loop filtering such as deblocking filtering and sample adaptive offset on the reconstructed image stored in the frame memory 108 .

[0021] The encoding unit 110 generates coded data by encoding the quantization coefficients generated by the transform / quantization unit 105 and the prediction information output from the prediction unit 104. The quantization matrix encoding unit 113 generates coded data by encoding the quantization matrix held in the quantization matrix holding unit 103.

[0022] The integrated coding unit 111 generates header code data using header information required for coding image data, such as the inverse quantization correction control information acquired by the inverse quantization correction control unit 114, and the code data generated by the quantization matrix coding unit 113. Furthermore, the integrated coding unit 111 generates a bit stream by combining the generated header code data with the code data generated by the coding unit 110, and outputs the generated bit stream to the outside via the output unit 112. Note that the output destination of the bit stream is not limited to a specific output destination. For example, the integrated coding unit 111 may output (transmit) the bit stream to an external device (for example, a memory device or a server device) via the output unit 112, or may store the bit stream in a memory in the image coding device. The control unit 150 controls the operation of the entire image coding device, including the above-mentioned functional units.

[0023] Next, a description will be given of the operation of the image encoding device in the functional configuration shown in Fig. 1. The inverse quantization correction control unit 114 acquires inverse quantization correction control information. The relationship between the value of the inverse quantization correction control information and the inverse quantization correction process will be described later.

[0024] The quantization matrix storage unit 103 acquires and stores a plurality of quantization matrices, which are generated according to the size of the sub-block and the type of prediction method. In this embodiment, the quantization matrix storage unit 103 generates a quantization matrix 800 of 8 pixels x 8 pixels, which corresponds to the sub-block of 8 pixels x 8 pixels shown in Figures 8(a) to 8(c) as described above.

[0025] A quantization matrix 800 in FIG. 8(a) shows an example of a quantization matrix corresponding to intra prediction. A quantization matrix 800 in FIG. 8(b) shows an example of a quantization matrix corresponding to inter prediction. A quantization matrix 800 in FIG. 8(c) shows an example of a quantization matrix corresponding to mixed intra-inter prediction. As shown in FIG. 8, the quantization matrix is ​​composed of 8×8 elements (quantization step values). In this embodiment, a case will be described in which the three types of quantization matrices shown in FIG. 8(a) to (c) are stored in the quantization matrix storage unit 103 as a two-dimensional array, but the elements in the quantization matrix are not limited to this. In addition, it is also possible to store multiple quantization matrices for the same prediction method depending on the size of the subblock or whether the encoding target is a luminance block or a chrominance block. In general, a quantization matrix realizes a quantization process according to human visual characteristics, so that, as shown in FIG. 8(a) to (c), the elements for DC components corresponding to the upper left corner of the quantization matrix are small, and the elements for AC components corresponding to the lower right corner are large.

[0026] However, the generated quantization matrix is ​​not limited to this, and a quantization matrix corresponding to the shape of a subblock, such as 4 pixels x 8 pixels, 8 pixels x 4 pixels, or 4 pixels x 4 pixels, may be generated. The method of determining each element in the quantization matrix is ​​not particularly limited. For example, a predetermined initial value may be used for each element in the quantization matrix, each element in the quantization matrix may be set individually, or the quantization matrix may be generated according to the characteristics of the image.

[0027] The quantization matrix encoding unit 113 sequentially reads out the quantization matrices stored in the quantization matrix storage unit 103 as a two-dimensional array, scans each element in the quantization matrix, calculates the difference, and arranges the result in a one-dimensional matrix (difference matrix). In this embodiment, a scanning method is used in which elements in each quantization matrix 800 shown in Figs. 8(a) to (c) are scanned in the order indicated by the arrows in Fig. 9, and the difference between each element and the previous element is calculated in the scanning order. For example, the quantization matrix 800 of 8 pixels x 8 pixels shown in Fig. 8(c) is scanned by the scanning method shown in Fig. 9, and after the first element "8" located in the upper left corner, the element "11" located immediately below it is scanned, and the difference "+3" is calculated. In addition, the first element of the quantization matrix (in the example of Fig. 8(c), "8" is encoded by calculating the difference between the first element and a predetermined initial value (for example, "8"), but this is of course not limited to this, and the difference between any value or the value of the first element itself may be used.

[0028] In this manner, in this embodiment, the quantization matrices 800 in each of Figures 8(a) to 8(c) are scanned using the scanning method in Figure 9, and one-dimensional difference matrices 1000 shown in Figures 10(a) to 10(c) are generated. The quantization matrix encoding unit 113 further encodes the difference matrix to generate code data of the quantization matrix. In this embodiment, the encoding is performed using the encoding table shown in Figure 11(a), but the encoding table is not limited to this, and for example, the encoding table shown in Figure 11(b) may be used.

[0029] 1, the integrated coding unit 111 generates header code data by integrating the coded data of the quantization matrix with header information, including the inverse quantization correction control information, necessary for coding the image. Next, the coding of the image will be described.

[0030] The block division unit 102 divides an input image input via the input unit 101 into a plurality of basic blocks. In this embodiment, the size of a basic block is 8 pixels x 8 pixels. The prediction unit 104 performs a prediction process for each basic block. More specifically, the prediction unit 104 first determines a sub-block division method, which is a method for dividing a basic block into smaller sub-blocks, and then determines a prediction mode, such as intra prediction, inter prediction, or mixed intra-inter prediction, for each sub-block.

[0031] 7(a) to (f) show examples of subblock division patterns. In FIG. 7, the outer thick frame 700 represents a basic block, which has a size of 8 pixels x 8 pixels in this embodiment. The rectangles within the thick frame 700 represent subblocks. FIG. 7(a) shows an example of a basic block = subblock. FIG. 7(b) shows an example of a conventional square subblock division, in which a basic block of 8 pixels x 8 pixels is divided into four subblocks of 4 pixels x 4 pixels. FIG. 7(c) to (f) show an example of a rectangular subblock division. In FIG. 7(c), the basic block is divided into two vertically long subblocks of 4 pixels x 8 pixels, and in FIG. 7(d), the basic block is divided into two horizontally long rectangular subblocks of 8 pixels x 4 pixels. In addition, in FIG. 7(e) and (f), the basic block is divided into rectangular subblocks at a ratio of 1:2:1. In this way, not only square but also rectangular subblocks are used for encoding processing.

[0032] For ease of explanation, this embodiment employs a subblock division method in which a basic block of 8 pixels x 8 pixels is not divided into subblocks (FIG. 7(a)). However, quadtree division as shown in FIG. 7(b), ternary tree division as shown in FIG. 7(e) and (f), or binary tree division as shown in FIG. 7(c) and FIG. 7(d) may also be used. When a subblock division other than that shown in FIG. 7(a) is used, the quantization matrix storage unit 103 generates a quantization matrix corresponding to the subblock to be used. The generated quantization matrix is ​​then coded by the quantization matrix coding unit 113.

[0033] In addition, the prediction mode (prediction method) used in this embodiment will be explained again. In this embodiment, three types of prediction methods are used: intra prediction, inter prediction, and mixed intra-inter prediction. In intra prediction, a predicted pixel of a block to be coded is generated using coded pixels located spatially around the block to be coded, and an intra prediction mode indicating an intra prediction method such as horizontal prediction, vertical prediction, and DC prediction is also generated. In inter prediction, a predicted pixel of a block to be coded is generated using coded pixels of a frame that is temporally different from the block to be coded, and motion information indicating a reference frame, a motion vector, and the like is also generated.

[0034] In intra-inter mixed prediction, first, the block to be coded is divided by a diagonal line segment to generate two regions. Then, pixel values ​​generated by the above-mentioned intra prediction are used in one region, and pixel values ​​generated by the above-mentioned inter prediction are used in the other region to generate predicted pixels of the block to be coded. Figures 12(a) and 12(b) show an example of region division used in this intra-inter mixed prediction. Figure 12(a) shows an example of a case where two regions are generated in the block to be coded 1200 by a diagonal line from the upper left vertex to the lower right vertex. For example, pixel values ​​generated by intra prediction can be used in the upper right region, and pixel values ​​generated by inter prediction can be used in the lower left region. Also, Figure 12(b) shows an example of a case where two regions are generated in the block to be coded 1200 by a diagonal line segment from the upper right vertex to the midpoint between the upper left vertex and the lower left vertex. For example, pixel values ​​generated by intra prediction can be placed in the upper left region, and pixel values ​​generated by inter prediction can be placed in the lower right region. In this way, in mixed intra / inter prediction, predicted pixels for the block to be coded are generated, and the intra prediction mode used to generate the predicted pixels, motion information, and information regarding region division are also generated.

[0035] The prediction unit 104 generates a prediction image of the sub-block to be coded from the determined prediction mode and coded pixels. The prediction unit 104 then calculates the difference (error) between the sub-block to be coded and the predicted image of the sub-block to generate a prediction error. The prediction unit 104 also outputs prediction information such as a sub-block division method, a prediction mode (information indicating whether the prediction mode is intra prediction, inter prediction, or mixed intra-inter prediction), and vector data.

[0036] The transform / quantization unit 105 generates quantization coefficients by performing orthogonal transform and quantization on the prediction error. Specifically, the transform / quantization unit 105 performs orthogonal transform processing corresponding to the size of the prediction error to generate orthogonal transform coefficients. Next, the transform / quantization unit 105 selects a quantization matrix corresponding to the prediction mode from among the quantization matrices held by the quantization matrix holding unit 103, and generates quantization coefficients by quantizing the orthogonal transform coefficients using the selected quantization matrix. In this embodiment, the quantization matrix of FIG. 8(a) is selected for quantization of the orthogonal transform coefficients of the subblocks on which prediction processing has been performed by intra prediction, and the quantization matrix of FIG. 8(b) is selected for quantization of the orthogonal transform coefficients of the subblocks on which inter prediction has been performed. Also, in this embodiment, the quantization matrix of FIG. 8(c) is selected for quantization of the orthogonal transform coefficients of the subblocks on which intra / inter mixed prediction has been performed. However, the quantization matrix used is not limited to this.

[0037] The inverse quantization and inverse transform unit 106 generates reproduced orthogonal transform coefficients by inverse quantizing the quantization coefficients of the sub-block using the quantization matrix used for quantizing the orthogonal transform coefficients of the sub-block, among the quantization matrices stored in the quantization matrix storage unit 103. Then, the inverse quantization and inverse transform unit 106 performs inverse quantization correction processing on the reproduced orthogonal transform coefficients based on the inverse quantization correction control information.

[0038] Here, the inverse quantization correction process in this embodiment will be described. The inverse quantization process and the inverse quantization correction process in this embodiment are performed using, for example, the following equation (1).

[0039] dz[x][y]=L[x][y]×Q[x][y]+Shift … (1) In formula (1), dz[x][y] is a corrected reproduction orthogonal transform coefficient corresponding to the position (x, y), and L[x][y] is a quantization coefficient corresponding to the position (x, y). Also, Q[x][y] is a "quantization scale calculated by taking into account the element of the quantization matrix" corresponding to the position (x, y). Also, Shift is a correction value used in the inverse quantization correction process of this embodiment, and is determined based on the quantization coefficient L and the inverse quantization correction control information. Specifically, when the value of the inverse quantization correction control information is 0, the inverse quantization and inverse transform unit 106 sets the value of Shift in formula (1) to 0. In this case, in the inverse quantization correction process, the correction of the reproduction orthogonal transform coefficient is not substantially performed. On the other hand, when the value of the inverse quantization correction control information is 1, the inverse quantization and inverse transform unit 106 derives the value of Shift in formula (1) using the following formula (2).

[0040] Shift = T × L [x] [y] … (2) In formula (2), T is a real number that takes a value greater than or equal to 0 and less than 1. In this embodiment, T is a fixed value, but is not limited to this and may take a variable value depending on the position (x, y), or T itself may be calculated using the quantization coefficient L[x][y]. For example, the value of T may be calculated according to the absolute value (|L[x][y]|) of the quantization coefficient L[x][y] as shown in the table below.

[0041] [Table 1]

[0042] In this case, when the quantization coefficient L[x][y] is 0, the value of T is also 0, and essentially no correction is performed. Also, as the absolute value of the non-zero quantization coefficient L[x][y] (|L[x][y]|) increases, the value of T decreases, and when |L[x][y]| is greater than a predetermined value, the value of T also becomes 0, and essentially no correction is performed in this case either.

[0043] Then, the inverse quantization and inverse transform unit 106 generates (reproduces) prediction errors by performing inverse orthogonal transform on the reproduced orthogonal transform coefficients generated using the above equation (1). For the inverse quantization process, a quantization matrix corresponding to the prediction mode of the block to be coded is used, as in the transform and quantization unit 105. Specifically, the same quantization matrix as that used by the transform and quantization unit 105 is used.

[0044] The image reproduction unit 107 generates (reproduces) a predicted image by appropriately referring to the frame memory 108 based on the prediction information input from the prediction unit 104. Then, the image reproduction unit 107 generates (reproduces) a reconstructed image of the corresponding sub-block by adding the reconstructed predicted image and the prediction error generated (reconstructed) by the inverse quantization and inverse transform unit 106, and stores the generated reconstructed image in the frame memory 108.

[0045] The in-loop filter unit 109 reads the reconstructed image from the frame memory 108, and performs in-loop filtering on the read reconstructed image using a filter such as a deblocking filter. Then, the in-loop filter unit 109 stores the reconstructed image to which the in-loop filtering has been applied in the frame memory 108 again.

[0046] The encoding unit 110 generates coded data for each sub-block by entropy coding the quantization coefficient of the sub-block generated by the transform / quantization unit 105 and the prediction information of the sub-block input from the prediction unit 104. The entropy coding method is not limited to a specific method, and Golomb coding, arithmetic coding, Huffman coding, etc. can be used.

[0047] The integrated encoding unit 111 multiplexes each piece of encoded data to generate a bit stream, and outputs the generated bit stream. FIG. 6(a) shows an example of the data structure of a bit stream output in this embodiment. The sequence header includes encoded data of the inverse quantization correction control information and the quantization matrix, and is composed of encoded data of each element. However, the position at which the information is encoded is not limited to this, and the information may be encoded in a picture header or other headers. In addition, when changing the inverse quantization correction control information or the quantization matrix within one sequence, it is also possible to update the information by newly encoding the inverse quantization correction control information or the quantization matrix.

[0048] Next, the process performed by the image coding device to code one frame of an input image will be described with reference to the flowchart in Fig. 3. When the image coding device codes input images of multiple frames, the image coding device performs the processes of steps S305 to S312 on the input image of each frame.

[0049] First, prior to encoding of an image, in step S301, the inverse quantization correction control unit 114 acquires inverse quantization correction control information. In step S302, the quantization matrix holding unit 103 acquires and holds a plurality of quantization matrices used in the quantization process.

[0050] In step S303, the quantization matrix encoding unit 113 scans the quantization matrix generated in step S302, calculates the difference between each element, and generates a one-dimensional difference matrix. In step S304, the integrated encoding unit 111 generates header code data using header information required for encoding image data, such as the inverse quantization correction control information acquired in step S301, and the code data of the quantization matrix generated in step S303.

[0051] In step S305, the block division unit 102 divides the input image input via the input unit 101 into a plurality of basic blocks. In step S306, the prediction unit 104 selects one of the basic blocks divided in step S305 that has not been selected as a selected basic block. The prediction unit 104 then divides the selected basic block into sub-blocks (including the case where the selected basic block is the sub-block), derives prediction errors for the sub-blocks, and outputs prediction information.

[0052] A specific example of processing by the prediction unit 104 is as follows. The prediction unit 104 performs intra prediction processing on a subblock of interest (block of interest) to be coded by referring to a coded area in the same input image as the input image to which the subblock of interest belongs, and generates an intra predicted image. The prediction unit 104 also performs inter prediction processing by referring to a coded input image (for example, an input image of the immediately preceding frame) different from the input image to which the subblock of interest to be coded belongs, and generates an inter predicted image. The prediction unit 104 then divides the subblock of interest into two regions as in the example of FIG. 12 shown above, and places an intra predicted image in one region and an inter predicted image in the other region to generate an intra / inter mixed predicted image. The prediction unit 104 then derives the squared sum (or the absolute sum) of the difference between pixel values ​​of pixels that correspond to each other in position between each of these three predicted images and the subblock of interest, and determines the prediction mode of the predicted image with the smallest squared sum as the prediction mode of the subblock of interest. The prediction unit 104 then performs a prediction process on the sub-block according to the prediction mode to generate a predicted image, and derives the difference between the sub-block and the predicted image as a prediction error.

[0053] In step S307, the transform / quantization unit 105 performs orthogonal transform on the prediction error derived in step S306 for each subblock to generate orthogonal transform coefficients. Then, for each subblock, the transform / quantization unit 105 selects one of the quantization matrices stored in the quantization matrix storage unit 103 based on the prediction information, and quantizes the orthogonal transform coefficients using the selected quantization matrix to generate quantization coefficients.

[0054] In step S308, the inverse quantization and inverse transform unit 106 performs inverse quantization on the quantization coefficients generated in step S307 for each subblock using the quantization matrix selected in step S307 to generate reproduced orthogonal transform coefficients, performs inverse quantization correction processing on the reproduced orthogonal transform coefficients based on the inverse quantization correction control information, and then performs inverse orthogonal transform to generate (reproduce) prediction errors. That is, the inverse quantization and inverse transform unit 106 performs inverse quantization processing and inverse quantization correction processing on the reproduced orthogonal transform coefficients in accordance with the above equation (1).

[0055] In step S309, the image reconstructor 107 generates a predicted image for each sub-block based on the prediction information output in step S306 by referring to the frame memory 108, and generates a reconstructed image using the predicted image and the prediction error generated in step S308.

[0056] In step S310, the encoding unit 110 generates coded data for each subblock of the basic block by encoding the prediction information output in step S306 and the quantization coefficients generated in step S307 for each subblock. Then, the integrated encoding unit 111 multiplexes the header coded data generated in step S304, the coded data generated by the encoding unit 110, and the like to generate a bitstream.

[0057] In step S311, the control unit 150 determines whether or not all basic blocks in the input image have been selected as selected basic blocks (that is, whether or not the encoding of all basic blocks (the processes of steps S306 to S310) has been completed).

[0058] If the result of this determination is that all basic blocks in the input image have been selected as selected basic blocks, the process proceeds to step S312; if there are any basic blocks remaining in the input image that have not yet been selected as selected basic blocks, the process proceeds to step S306.

[0059] In step S312, the in-loop filter unit 109 reads the reconstructed image from the frame memory 108, performs in-loop filter processing on the reconstructed image, and stores the reconstructed image having been subjected to the in-loop filter processing back in the frame memory 108.

[0060] With the above configuration and operation, particularly in step S308, by correcting the reproduced orthogonal transform coefficients based on the inverse quantization correction control information, appropriate correction can be performed on the reproduced orthogonal transform coefficients generated by the inverse quantization process using a quantization matrix, thereby improving compression efficiency.

[0061] In this embodiment, the quantization matrix is ​​used for the inverse quantization process of all subblocks in a frame to generate reproduced orthogonal transform coefficients, and correction is performed based on the inverse quantization correction control information, but the present invention is not limited to this. For example, some subblocks in a frame (e.g., subblocks with a size of 8 pixels x 8 pixels) may be inverse quantized using a quantization matrix, and other subblocks (e.g., subblocks with a size of 4 pixels x 4 pixels) may be inverse quantized without using a quantization matrix (i.e., the same quantization scale is used for all frequency components). In this case, the subblocks inverse quantized using the quantization matrix may be corrected according to formula (1) according to the inverse quantization correction control information, and the subblocks inverse quantized without using the quantization matrix may be always corrected according to formula (1) regardless of the inverse quantization correction control information. As a result, even when subblocks inverse quantized using a quantization matrix and subblocks inverse quantized without using a quantization matrix are mixed, appropriate correction can be performed on each subblock, improving compression efficiency.

[0062] In addition, in this embodiment, the quantization correction control information is coded and included in the bit stream, but this is not limiting. For example, by always setting the quantization correction control information to 0, the code of the quantization correction control information to be included in the bit stream can be omitted as shown in FIG. 6(b). In this case, the sub-blocks that perform inverse quantization without using a quantization matrix are always subjected to inverse quantization correction processing, and the sub-blocks that perform inverse quantization using a quantization matrix are never subjected to inverse quantization correction processing. This simplifies the relationship between whether or not the quantization matrix is ​​applied and whether or not the inverse quantization correction processing is applied, making the control easier, and furthermore, the amount of code for the code of the inverse quantization correction control information can be reduced.

[0063] In addition, in this embodiment, the inverse quantization correction control information is configured to indicate only whether or not the inverse quantization correction process is applied, but the value of the inverse quantization correction control information can also be set to the value of a parameter used in the inverse quantization correction process. For example, the value of the inverse quantization correction control information can also be set to the value of Shift in the above-mentioned formula (1) or the value of T in formula (2). This makes it possible to control the strength of the inverse quantization correction according to the characteristics of the image, and as a result, the compression efficiency can be improved.

[0064] In addition, in this embodiment, three types of prediction methods, intra prediction, inter prediction, and intra / inter mixed prediction, are used, but since the characteristics of the prediction and the error thereof are different, it is also possible to configure different inverse quantization correction processing according to the prediction method. For example, it is also possible to apply inverse quantization correction control suitable for each prediction method by individually setting inverse quantization correction control information corresponding to a subblock using intra prediction, inverse quantization correction control information corresponding to a subblock using inter prediction, and inverse quantization correction control information corresponding to a subblock using intra / inter mixed prediction. In this case, it is possible to configure each inverse quantization correction control information to be coded and included in the bit stream, or it is possible to configure each inverse quantization correction control information to be set to a fixed value and omit coding each.

[0065] In this embodiment, the encoding process is performed on an image per frame to generate and output a bit stream, but the target of the encoding process is not limited to an image. For example, a feature amount used in machine learning such as object recognition may be a two-dimensional array of data, and the data may be encoded. This makes it possible to efficiently encode the feature amount used in machine learning.

[0066] [Second embodiment] The image decoding device according to this embodiment decodes a bit stream for each frame generated by the image encoding device according to the first embodiment. An example of the functional configuration of the image decoding device according to this embodiment will be described with reference to the block diagram of FIG.

[0067] The separation decoding unit 202 acquires a bit stream via the input unit 201. The method for the separation decoding unit 202 to acquire the bit stream is not limited to a specific method. For example, the separation decoding unit 202 may acquire a bit stream stored in an external device such as a server device via a network, or may acquire a bit stream generated by an imaging device from the imaging device. The separation decoding unit 202 then separates header code data and code data in sub-block units of a basic block from the bit stream. In short, the separation decoding unit 202 performs the reverse operation of the integration encoding unit 111 in FIG. 1. The separation decoding unit 202 also extracts inverse quantization correction control information from the header code data.

[0068] The quantization matrix decoding unit 209 reproduces the quantization matrix by decoding the header encoded data separated by the separate decoding unit 202. The decoding unit 203 decodes the encoded data of each sub-block of the basic block separated by the separate decoding unit 202, and reproduces the quantization coefficients and prediction information.

[0069] The inverse quantization and inverse transform unit 204 generates reproduced orthogonal transform coefficients by inverse quantizing the quantization coefficients reproduced by the decoding unit 203 using the quantization matrix reproduced by the quantization matrix decoding unit 209 in the same manner as the inverse quantization and inverse transform unit 106. Then, like the inverse quantization and inverse transform unit 106, the inverse quantization and inverse transform unit 204 performs inverse quantization correction processing on the reproduced orthogonal transform coefficients based on the inverse quantization correction control information extracted by the separate decoding unit 202, and then performs inverse orthogonal transform to generate (reproduce, derive) prediction errors.

[0070] Similar to the image reproduction unit 107, the image reproduction unit 205 generates a predicted image by appropriately referring to the frame memory 206 based on the prediction information reproduced by the decoding unit 203. Then, similar to the image reproduction unit 107, the image reproduction unit 205 generates a reproduced image by adding the prediction error reproduced by the inverse quantization and inverse transform unit 204 to the predicted image, and stores the reproduced image in the frame memory 206.

[0071] Similar to the in-loop filter unit 109, the in-loop filter unit 207 reads out a reconstructed image from the frame memory 206, performs in-loop filter processing on the reconstructed image, and stores the reconstructed image to which the in-loop filter processing has been applied back in the frame memory 206. The reconstructed image to which the in-loop filter processing has been applied by the in-loop filter unit 207 is output to an external device via the output unit 208 under the control of the control unit 250.

[0072] The output destination of the reproduced image is not limited to a specific output destination. For example, the control unit 250 may transmit the reproduced image to an external device via a network, or may output the reproduced image to a display device connected to the image decoding device and display the reproduced image on the display device. The control unit 250 controls the operation of the entire image decoding device, including the above-mentioned functional units.

[0073] Next, the operation of the image decoding device in the functional configuration shown in Fig. 2 will be described. A bit stream for one frame input via an input unit 201 is input to a separate decoding unit 202. The separate decoding unit 202 according to this embodiment extracts inverse quantization correction control information from the sequence header of the bit stream shown in Fig. 6(a), and also extracts the code data of the quantization matrices of Fig. 8(a) to (c) from the sequence header. The separate decoding unit 202 also reproduces the code data of each sub-block of a basic block of picture data.

[0074] The quantization matrix decoding unit 209 decodes the coded data of the quantization matrix to reproduce the one-dimensional difference matrix shown in Fig. 10(a)-(c). In this embodiment, as in the first embodiment, the decoding is performed using the coding table shown in Fig. 11(a) (or Fig. 11(b)). However, the coding table is not limited to this, and other coding tables may be used as long as they are the same as those in the first embodiment. Then, the quantization matrix decoding unit 209 reverse-scans the reproduced one-dimensional difference matrix to reproduce the quantization matrix as a two-dimensional array. That is, the quantization matrix decoding unit 209 performs an operation opposite to that of the quantization matrix encoding unit 113. That is, the quantization matrix decoding unit 209 reproduces the three types of quantization matrices shown in Fig. 8(a)-(c) from the difference matrix shown in Fig. 10(a)-(c) using the scanning method shown in Fig. 9.

[0075] The decoding unit 203 decodes the coded data of each sub-block of the basic block to reproduce the quantization coefficients and prediction information. The inverse quantization and inverse transform unit 204 selects one of the quantization matrices reproduced by the quantization matrix decoding unit 209, and generates reproduced orthogonal transform coefficients by inverse quantizing the quantization coefficients reproduced by the decoding unit 203 using the selected quantization matrix. Then, like the inverse quantization and inverse transform unit 106, the inverse quantization and inverse transform unit 204 performs inverse quantization correction processing on the generated reproduced orthogonal transform coefficients based on the inverse quantization correction control information extracted by the separation decoding unit 202, and then performs inverse orthogonal transform to generate (reproduce) prediction errors.

[0076] The inverse quantization and inverse transform unit 204 of this embodiment determines a quantization matrix to be used in the inverse quantization process according to the prediction mode of the sub-block to be decoded, which is determined according to the prediction information reproduced by the decoding unit 203. That is, the quantization matrix of FIG. 8(a) is selected for the sub-block using intra prediction, the quantization matrix of FIG. 8(b) is selected for the sub-block using inter prediction, and the quantization matrix of FIG. 8(c) is selected for the sub-block using mixed intra / inter prediction. However, the quantization matrix to be used is not limited to this, and may be the same as the quantization matrix used in the transform and quantization unit 105 and the inverse quantization and inverse transform unit 106 of the first embodiment.

[0077] The image reproduction unit 205, like the image reproduction unit 107, generates a predicted image by appropriately referring to the frame memory 206 based on the prediction information reproduced by the decoding unit 203. In this embodiment, like the prediction unit 104, three types of prediction methods are used: intra prediction, inter prediction, and mixed intra / inter prediction. Then, like the image reproduction unit 107, the image reproduction unit 205 generates a reproduced image by adding the prediction error reproduced by the inverse quantization / inverse transform unit 204 to the predicted image, and stores the reproduced image in the frame memory 206. The stored reproduced image becomes a prediction reference candidate when decoding other sub-blocks.

[0078] Similar to the in-loop filter unit 109, the in-loop filter unit 207 performs in-loop filter processing on the reconstructed image stored in the frame memory 206. As described above, the reconstructed image to which the in-loop filter processing has been applied by the in-loop filter unit 207 is output to an external device via the output unit 208.

[0079] Next, the process performed by the image decoding device to decode a bitstream for one frame will be described with reference to the flowchart in Fig. 4. When the image decoding device decodes bitstreams of multiple frames, the image decoding device performs the process according to the flowchart in Fig. 4 on the bitstreams of each frame.

[0080] In step S401, the separate decoding unit 202 extracts (decodes) the inverse quantization correction control information from the bit stream, and reproduces (separates) from the bit stream the coded data of the quantization matrix and the coded data of each sub-block of the basic block.

[0081] In step S402, the quantization matrix decoding unit 209 reconstructs a one-dimensional difference matrix by decoding the encoded data of the quantization matrix reconstructed in step S401, and inversely scans the reconstructed one-dimensional difference matrix to reconstruct the quantization matrix as a two-dimensional array.

[0082] In step S403, the decoding unit 203 decodes the coded data of the basic block reconstructed in step S401 on a sub-block basis to reconstruct the quantization coefficients and prediction information. In step S404, the inverse quantization and inverse transform unit 204 selects one of the quantization matrices reconstructed in step S402, and generates reconstructed orthogonal transform coefficients by inverse quantizing the quantization coefficients reconstructed in step S403 using the selected quantization matrix. The inverse quantization and inverse transform unit 204 then performs inverse quantization correction processing on the reconstructed orthogonal transform coefficients based on the inverse quantization correction control information extracted in step S401, and then performs inverse orthogonal transform to generate (reconstruct) prediction errors.

[0083] In step S405, the image reproducing unit 205 generates a predicted image based on the prediction information reproduced in step S403 by appropriately referring to the frame memory 206. Then, the image reproducing unit 205 generates a reproduced image by adding the prediction error reproduced in step S404 to the predicted image, and stores the reproduced image in the frame memory 206.

[0084] In step S406, the control unit 250 determines whether or not the processes of steps S403 to S405 have been performed for all basic blocks. If the result of this determination is that the processes of steps S403 to S405 have been performed for all basic blocks, the process proceeds to step S407. On the other hand, if there are any basic blocks remaining for which the processes of steps S403 to S405 have not been performed, the process proceeds to step S403 to perform the processes of steps S403 to S405 for those basic blocks. In step S407, the in-loop filter unit 207 performs in-loop filter processing on the reconstructed image stored in the frame memory 206.

[0085] With the above configuration and operation, by correcting the reproduced orthogonal transform coefficients based on the inverse quantization correction control information, appropriate correction can be performed on the reproduced orthogonal transform coefficients generated by the inverse quantization process using a quantization matrix, and a bitstream with improved compression efficiency can be decoded.

[0086] In this embodiment, as in the first embodiment, the reconstructed orthogonal transform coefficients are generated using a quantization matrix in the inverse quantization process of all subblocks in a frame, and correction is performed based on the inverse quantization correction control information, but this is not limited to the above. For example, as in the first embodiment, the present embodiment can be configured to perform correction according to formula (1) according to the inverse quantization correction control information for subblocks inverse quantized using a quantization matrix, and to always perform correction according to formula (1) regardless of the inverse quantization correction control information for subblocks inverse quantized without using a quantization matrix. As a result, even when subblocks inverse quantized using a quantization matrix and subblocks inverse quantized without using a quantization matrix are mixed, appropriate correction can be performed on each subblock, and a bitstream with improved compression efficiency can be decoded.

[0087] In addition, in this embodiment, the bit stream in which the quantization correction control information is coded is decoded, but the present invention is not limited to this. For example, by always setting the value of the quantization correction control information to 0, a bit stream in which the code of the quantization correction control information is omitted as shown in FIG. 6(b) can be decoded. In this case, as in the first embodiment, the sub-block in which the dequantization is performed without using the quantization matrix is ​​always subjected to the dequantization correction process, and the sub-block in which the dequantization is performed using the quantization matrix is ​​never subjected to the dequantization correction process. This simplifies the relationship between the application of the quantization matrix and the application of the dequantization correction process, making the control easier, and furthermore, it is possible to decode a bit stream in which the code amount of the code of the dequantization correction control information is reduced.

[0088] In this embodiment, as in the first embodiment, the parameters used in the inverse quantization correction process can be the inverse quantization correction control information. This makes it possible to control the strength of the inverse quantization correction according to the characteristics of the image, and as a result, it is possible to decode a bitstream with improved compression efficiency.

[0089] Also in this embodiment, similarly to the first embodiment, it is possible to configure the configuration to perform different inverse quantization correction processing depending on the prediction method. In this case, similarly to the first embodiment, it is possible to configure the configuration to decode a bit stream in which each inverse quantization correction control information is coded, or to set each inverse quantization correction control information to a fixed value and omit each decoding.

[0090] In this embodiment, a bit stream is decoded in units of frames, but the target of the decoding process is not limited to a bit stream obtained by encoding an image. For example, a feature amount used in machine learning such as object recognition may be a two-dimensional array of data, and the data may be encoded to generate a bit stream, which may be decoded. This makes it possible to efficiently decode a bit stream obtained by encoding the feature amount used in machine learning.

[0091] [Third embodiment] In the first embodiment, each of the functional units shown in Fig. 1 has been described as being implemented in hardware. However, each of the functional units except for the frame memory 108 may be implemented in software (computer program). In this case, a computer device capable of executing this software is applicable to an image encoding device.

[0092] In the second embodiment, each of the functional units shown in Fig. 2 has been described as being implemented in hardware. However, each of the functional units except for the frame memory 206 may be implemented in software (computer program). In this case, a computer device capable of executing this software is applicable to an image decoding device.

[0093] An example of the hardware configuration of a computer device applicable to an image coding device and an image decoding device will be described with reference to the block diagram of Fig. 5. Note that the configuration shown in Fig. 5 is an example of the hardware configuration of a computer device applicable to an image coding device and an image decoding device, and can be changed / modified as appropriate. Also, computer devices with different configurations may be applied to the image coding device and the image decoding device. Also, the image coding device and the image decoding device may be the same device.

[0094] The CPU 501 executes various processes using computer programs and data stored in the RAM 502 and the ROM 503. As a result, the CPU 501 controls the operation of the entire computer device, and executes or controls the various processes described as the processes performed by the image encoding device and the image decoding device.

[0095] The RAM 502 has an area for storing computer programs and data loaded from the ROM 503 or storage device 506, and an area for storing computer programs and data received from the outside via the I / F 507. The RAM 502 further has a work area used when the CPU 501 executes various processes. In this way, the RAM 502 can provide various areas as needed.

[0096] The ROM 503 stores setting data for the computer device, computer programs and data relating to the startup of the computer device, computer programs and data relating to the basic operation of the computer device, and the like.

[0097] An operation unit 504 is a user interface such as a keyboard, a mouse, a touch panel screen, etc., and the user can operate it to input various instructions and information to the computer device.

[0098] Display unit 505 has a liquid crystal screen or a touch panel screen, and can display the results of processing by CPU 501 as images, characters, etc. Display unit 505 may also be a projection device such as a projector that projects images and characters.

[0099] The storage device 506 is a non-volatile memory device such as a hard disk drive, etc. The storage device 506 stores an OS (operating system), computer programs and data for causing the CPU 501 to execute or control the various processes described as the processes performed by the image encoding device and the image decoding device, and the like.

[0100] The computer programs stored in the storage device 506 include computer programs for causing the CPU 501 to execute or control the various processes described as the processes performed by each functional unit (excluding frame memory 108) shown in Fig. 1. The computer programs stored in the storage device 506 also include computer programs for causing the CPU 501 to execute or control the various processes described as the processes performed by each functional unit (excluding frame memory 206) shown in Fig. 2. The frame memory 108 and the frame memory 206 can be implemented using the RAM 502 and the storage device 506.

[0101] The I / F 507 is a communication interface for performing data communication with an external device. For example, the computer device can obtain an input image or a bit stream from an image capture device or a server device, or transmit a bit stream to the server device by performing data communication with the image capture device or a server device via the I / F 507.

[0102] The CPU 501, the RAM 502, the ROM 503, the operation unit 504, the display unit 505, the storage device 506, and the I / F 507 are all connected to a system bus 508. In this configuration, when the computer device is powered on, the CPU 501 executes a boot program stored in the ROM 503, loads the OS stored in the storage device 506 into the RAM 502, and starts the OS. As a result, the computer device becomes capable of communication via the I / F 507. Then, under the control of the OS, the CPU 501 loads an application (corresponding to FIG. 3) relating to image encoding from the storage device 506 into the RAM 502 and executes it, so that the computer device functions as an image encoding device. On the other hand, when the CPU 501 loads an application (corresponding to FIG. 4) relating to image decoding from the storage device 506 into the RAM 502 and executes it, the computer device functions as an image decoding device.

[0103] The numerical values, processing timing, processing order, processing subject, data (information) structure / acquisition method / send destination / send source / storage location, etc. used in each of the above embodiments are given as examples to provide a concrete explanation, and are not intended to be limited to these examples.

[0104] In addition, a part or all of the embodiments described above may be used in appropriate combination. In addition, a part or all of the embodiments described above may be used selectively.

[0105] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0106] The invention of this specification includes the following image encoding device, image encoding method, image decoding device, image decoding method, and computer program. (Item 1) An image encoding device that encodes an image on a block-by-block basis, comprising: a prediction means for generating a predicted image from encoded pixels for a block of interest having a predetermined size to be encoded in an image, and deriving a prediction error which is a difference between the block of interest and the predicted image; a transform means for frequency transforming the prediction error derived by the prediction means; a quantization means for quantizing an orthogonal transform coefficient obtained by the frequency transform by the transform means; a coding means for entropy coding the orthogonal transform coefficients quantized by the quantization means; an inverse quantization means for inverse quantizing the orthogonal transform coefficients quantized by the quantization means to generate reproduced orthogonal transform coefficients; having The image coding apparatus according to claim 1, wherein the inverse quantization means corrects the reconstructed orthogonal transform coefficients. (Item 2) The inverse quantization means performs control so as to correct the reproduced orthogonal transform coefficients when the inverse quantization is performed without using a quantization matrix, and so as not to correct the reproduced orthogonal transform coefficients when the inverse quantization is performed using a quantization matrix. 2. The image encoding device according to item 1, (Item 3) 3. The image encoding device according to item 1 or 2, wherein the encoding means further encodes inverse quantization correction control information for controlling the correction process in the inverse quantization means. (Item 4) The image coding device according to any one of items 1 to 3, characterized in that the inverse quantization means performs different corrections on the reconstructed orthogonal transform coefficients depending on a method for generating a predicted image in the prediction means. (Item 5) An image decoding device that decodes an image on a block-by-block basis, comprising: A decoding means for decoding the quantized orthogonal transform coefficients; an inverse quantization means for inverse quantizing the quantized orthogonal transform coefficients to generate reproduced orthogonal transform coefficients; an inverse transform means for inversely transforming the reconstructed orthogonal transform coefficients to derive a prediction error; A prediction means for generating a predicted image and decoding a target block using the predicted image and the prediction error, The inverse quantization means corrects the reproduced orthogonal transform coefficients. 2. An image decoding device comprising: (Item 6) The inverse quantization means performs control so as to correct the reproduced orthogonal transform coefficients when the inverse quantization is performed without using a quantization matrix, and so as not to correct the reproduced orthogonal transform coefficients when the inverse quantization is performed using a quantization matrix. 6. The image decoding device according to item 5, (Item 7) 7. The image decoding device according to item 5 or 6, wherein the decoding means further decodes inverse quantization correction control information for controlling the correction process in the inverse quantization means. (Item 8) The image decoding device according to any one of items 5 to 7, characterized in that the inverse quantization means performs different corrections on the reconstructed orthogonal transform coefficients depending on a method for generating a predicted image in the prediction means. (Item 9) An image coding method performed by an image coding device that codes an image on a block-by-block basis, comprising: a prediction step in which a prediction means of the image encoding device generates a predicted image from encoded pixels for a target block of a predetermined size to be encoded in an image, and derives a prediction error which is a difference between the target block and the predicted image; a transform step in which a transform means of the image encoding device frequency-transforms the prediction error derived in the prediction step; a quantization step in which a quantization means of the image coding device quantizes an orthogonal transform coefficient obtained by the frequency transform in the transform step; an encoding step in which an encoding means of the image encoding device entropy-encodes the orthogonal transform coefficients quantized by the quantization step; an inverse quantization step in which an inverse quantization means of the image encoding device inversely quantizes the orthogonal transform coefficients quantized by the quantization step to generate reproduced orthogonal transform coefficients; having 4. An image coding method comprising the steps of: correcting the reconstructed orthogonal transform coefficients in the inverse quantization step. (Item 10) An image decoding method performed by an image decoding device that decodes an image on a block-by-block basis, comprising: a decoding step in which a decoding means of the image decoding device decodes the quantized orthogonal transform coefficients; an inverse quantization step in which an inverse quantization means of the image decoding device inverse quantizes the quantized orthogonal transform coefficients to generate reproduced orthogonal transform coefficients; an inverse transform step in which an inverse transform means of the image decoding device inversely transforms the reconstructed orthogonal transform coefficients to derive a prediction error; a prediction step in which a prediction means of the image decoding device generates a predicted image and decodes a target block using the predicted image and the prediction error; In the inverse quantization step, the reconstructed orthogonal transform coefficients are corrected. 2. An image decoding method comprising: (Item 11) A computer program for causing a computer of an image encoding device that encodes an image on a block-by-block basis to function as each of the means of the image encoding device according to any one of items 1 to 4. (Item 12) A computer program for causing a computer of an image decoding device that decodes an image on a block-by-block basis to function as each of the means of the image decoding device according to any one of items 5 to 8.

[0107] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0108] 101...input unit, 102...block division unit, 103...quantization matrix storage unit, 104...prediction unit, 105...transformation and quantization unit, 106...inverse quantization and inverse transformation unit, 107...image reproduction unit, 108...frame memory, 109...in-loop filter unit, 110...encoding unit, 111...integrated encoding unit, 112...output unit, 113...quantization matrix encoding unit, 114...inverse quantization correction control unit, 150...control unit

Claims

1. An image encoding device that encodes an image on a block-by-block basis, comprising: a prediction means for generating a predicted image from encoded pixels for a block of interest having a predetermined size to be encoded in an image, and deriving a prediction error which is a difference between the block of interest and the predicted image; a transform means for frequency transforming the prediction error derived by the prediction means; a quantization means for quantizing an orthogonal transform coefficient obtained by the frequency transform by the transform means; a coding means for entropy coding the orthogonal transform coefficients quantized by the quantization means; an inverse quantization means for inverse quantizing the orthogonal transform coefficients quantized by the quantization means to generate reproduced orthogonal transform coefficients; having The image coding apparatus according to claim 1, wherein the inverse quantization means corrects the reconstructed orthogonal transform coefficients.

2. The inverse quantization means performs control so as to correct the reproduced orthogonal transform coefficients when the inverse quantization is performed without using a quantization matrix, and so as not to correct the reproduced orthogonal transform coefficients when the inverse quantization is performed using a quantization matrix.

2. The image encoding device according to claim 1 .

3. 2. The image encoding device according to claim 1, wherein said encoding means further encodes inverse quantization correction control information for controlling the correction process in said inverse quantization means.

4. 2. The image coding apparatus according to claim 1, wherein the inverse quantization means performs different corrections on the reconstructed orthogonal transform coefficients depending on a method for generating a predicted image in the prediction means.

5. An image decoding device that decodes an image on a block-by-block basis, comprising: A decoding means for decoding the quantized orthogonal transform coefficients; an inverse quantization means for inverse quantizing the quantized orthogonal transform coefficients to generate reproduced orthogonal transform coefficients; an inverse transform means for inversely transforming the reconstructed orthogonal transform coefficients to derive a prediction error; A prediction means for generating a predicted image and decoding a target block using the predicted image and the prediction error, The inverse quantization means corrects the reproduced orthogonal transform coefficients.

2. An image decoding device comprising:

6. The inverse quantization means performs control so as to correct the reproduced orthogonal transform coefficients when the inverse quantization is performed without using a quantization matrix, and so as not to correct the reproduced orthogonal transform coefficients when the inverse quantization is performed using a quantization matrix.

6. The image decoding device according to claim 5.

7. 6. The image decoding device according to claim 5, wherein said decoding means further decodes inverse quantization correction control information for controlling the correction process in said inverse quantization means.

8. 6. The image decoding apparatus according to claim 5, wherein the inverse quantization means performs different corrections on the reconstructed orthogonal transform coefficients depending on a method for generating a predicted image in the prediction means.

9. An image coding method performed by an image coding device that codes an image on a block-by-block basis, comprising: a prediction step in which a prediction means of the image encoding device generates a predicted image from encoded pixels for a target block of a predetermined size to be encoded in an image, and derives a prediction error which is a difference between the target block and the predicted image; a transform step in which a transform means of the image encoding device frequency-transforms the prediction error derived in the prediction step; a quantization step in which a quantization means of the image coding device quantizes an orthogonal transform coefficient obtained by the frequency transform in the transform step; an encoding step in which an encoding means of the image encoding device entropy-encodes the orthogonal transform coefficients quantized by the quantization step; an inverse quantization step in which an inverse quantization means of the image encoding device inversely quantizes the orthogonal transform coefficients quantized by the quantization step to generate reproduced orthogonal transform coefficients; having 4. An image coding method comprising the steps of: correcting the reconstructed orthogonal transform coefficients in the inverse quantization step.

10. An image decoding method performed by an image decoding device that decodes an image on a block-by-block basis, comprising: a decoding step in which a decoding means of the image decoding device decodes the quantized orthogonal transform coefficients; an inverse quantization step in which an inverse quantization means of the image decoding device inverse quantizes the quantized orthogonal transform coefficients to generate reproduced orthogonal transform coefficients; an inverse transform step in which an inverse transform means of the image decoding device inversely transforms the reconstructed orthogonal transform coefficients to derive a prediction error; a prediction step in which a prediction means of the image decoding device generates a predicted image and decodes a target block using the predicted image and the prediction error; In the inverse quantization step, the reconstructed orthogonal transform coefficients are corrected.

2. An image decoding method comprising:

11. 5. A computer program for causing a computer of an image coding apparatus that codes an image on a block-by-block basis to function as each of the means of the image coding apparatus according to claim 1.

12. 9. A computer program for causing a computer of an image decoding device that decodes an image in units of blocks to function as each of the means of the image decoding device according to claim 5.

Images