Image encoding device, image encoding method, image decoding device, image decoding method, and computer program

Abstract

According to the present invention, a prediction image is generated from an encoded pixel for a target block of a predetermined size to be encoded in an image, a prediction error that is the difference between the target block and the prediction image is derived, the prediction error is frequency-converted, and an orthogonal transform coefficient obtained by the frequency conversion is quantized. Moreover, the quantized orthogonal transform coefficient is entropy-encoded, and the quantized orthogonal transform coefficient is dequantized to generate a reproduction orthogonal transform coefficient. The reproduction orthogonal transform coefficient is corrected.

Description

Image encoding device, image encoding method, image decoding device, image decoding method, and computer program

[0001] The present invention relates to an image encoding device, an image encoding method, an image decoding device, an image decoding method, and a computer program.

[0002] Versatile Video Coding (VVC) is a well-known coding method for compressing and recording moving images. In VVC, basic blocks with a maximum size of 128 pixels x 128 pixels are divided into rectangular sub-blocks in addition to the conventional square sub-blocks to improve coding efficiency.

[0003] Furthermore, VVC uses a process in which coefficients after orthogonal transformation (hereinafter referred to as orthogonal transform coefficients) are weighted according to frequency components using a quantization matrix. By further reducing data of high-frequency components, degradation of which is less noticeable to the human eye, it is possible to improve compression efficiency while maintaining image quality. Patent Document 1 discloses a technique for encoding such a quantization matrix.

[0004] In recent years, the Joint Video Experts Team (JVET), which standardized VVC, has been conducting technical studies to achieve compression efficiency exceeding that of VVC. To improve coding efficiency, a new inverse quantization method (hereinafter referred to as inverse quantization correction) is being studied, in which correction values ​​are added to orthogonal transform coefficients generated by inverse quantization processing (hereinafter referred to as reproduced orthogonal transform coefficients).

[0005] JP 2013-38758 A

[0006] This inverse quantization correction is a process of adding a correction value to the reproduced orthogonal transform coefficients, but it is a process of adding a predetermined correction value regardless of whether a quantization matrix is ​​used. Therefore, it is not possible to add an appropriate correction value to the reproduced orthogonal transform coefficients generated by inverse quantization processing using a quantization matrix, which poses a problem in that it is not possible to improve compression efficiency.

[0007] The present invention provides a technique for improving compression efficiency by enabling control so that appropriate correction values ​​can be added to reproduced orthogonal transform coefficients generated by inverse quantization processing using a quantization matrix.

[0008] One aspect of the present invention is an image coding device that codes an image on a block-by-block basis, comprising: prediction means that generates a predicted image from coded pixels for a target block of a predetermined size to be coded in the image, and derives a prediction error, which is the difference between the target block and the predicted image; transformation means that frequency transforms the prediction error derived by the prediction means; quantization means that quantizes orthogonal transform coefficients obtained by the frequency transform by the transformation means; coding means that entropy encodes the orthogonal transform coefficients quantized by the quantization means; and inverse quantization means that inverse quantizes the orthogonal transform coefficients quantized by the quantization means to generate reproduced orthogonal transform coefficients, wherein the inverse quantization means corrects the reproduced orthogonal transform coefficients.

[0009] According to the present invention, it is possible to provide a technology for improving compression efficiency by enabling control so that appropriate correction values ​​can be added to reproduced orthogonal transform coefficients generated by inverse quantization processing using a quantization matrix.

[0010] Other features and advantages of the present invention will become apparent from the following description taken in conjunction with the accompanying drawings, in which the same or similar elements are designated by the same reference numerals.

[0011] The accompanying drawings are incorporated into and constitute a part of the specification, illustrate embodiments of the present invention, and are used, together with the description, to explain the principles of the present invention. A block diagram showing an example of the functional configuration of an image encoding device. A block diagram showing an example of the functional configuration of an image decoding device. A flowchart of processing performed by an image encoding device to encode one frame of an input image. A flowchart of processing performed by an image decoding device to decode one frame of a bitstream. A block diagram showing an example of the hardware configuration of a computer device applicable to an image encoding device or an image decoding device. A diagram showing an example of the data structure of a bitstream. A diagram showing an example of an example of a data structure of a bitstream. A diagram showing an example of a sub-block division pattern. A diagram showing an example of an sub-block division pattern. A diagram showing an example of an sub-block division pattern. A diagram showing an example of a sub-block division pattern. A diagram showing an example of a quantization matrix 800. A diagram showing an example of a quantization matrix 800. A diagram showing an example of a quantization matrix 800. A diagram showing a scanning method. A diagram showing an example of a one-dimensional difference matrix. A diagram showing an example of a one-dimensional difference matrix. A diagram showing an example of the configuration of a one-dimensional difference matrix. A diagram showing an example of the configuration of a coding table. A diagram showing an example of the configuration of a coding table. A diagram showing an example of region division used in intra-inter mixed prediction. A diagram showing an example of region division used in intra-inter mixed prediction.

[0012] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention claimed. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0013] First Embodiment First, an example of the functional configuration of an image encoding device according to this embodiment will be described using the block diagram in Fig. 1. The image encoding device can be applied to a PC (personal computer), a smartphone, a tablet terminal device, an imaging device, a dedicated image processing circuit, and the like.

[0014] The image coding device acquires an image to be coded as an input image via the input unit 101. The method by which the image coding device acquires the input image is not limited to a specific method. For example, the image coding device may acquire an image output from an imaging device (such as an image of each frame in a moving image or a still image captured periodically or irregularly) as the input image. Note that, when the image coding device and the imaging device are integrated, the image coding device acquires an image captured by its own imaging device as the input image. For another example, the image coding device may acquire an image stored in an external device such as a server device via a network such as a LAN or the Internet as the input image. For another example, the image coding device may acquire an image stored in its own storage device as the input image.

[0015] The block division unit 102 divides the input image into a plurality of basic blocks (hereinafter, simply referred to as blocks, as appropriate). The quantization matrix storage unit 103 acquires and stores a plurality of quantization matrices used in the quantization process. The method by which the quantization matrix storage unit 103 acquires the quantization matrices is not limited to a specific method. For example, the quantization matrix storage unit 103 may acquire a quantization matrix input by a user operating an operation unit (not shown), may calculate a quantization matrix from the characteristics of the input image, or may acquire a quantization matrix specified in advance as an initial value. In this embodiment, the quantization matrix storage unit 103 acquires and stores three types of "two-dimensional quantization matrices 800 corresponding to 8 pixel x 8 pixel orthogonal transforms (frequency transforms)" as exemplified in FIGS. 8A to 8C.

[0016] The inverse quantization correction control unit 114 acquires inverse quantization correction control information, which is information for controlling the inverse quantization correction process executed in the downstream inverse quantization / inverse transform unit 106. The method by which the inverse quantization correction control unit 114 acquires the inverse quantization correction control information is not limited to a specific method. For example, the inverse quantization correction control unit 114 may acquire inverse quantization correction control information input by a user operating an operation unit (not shown), or may calculate the inverse quantization correction control information based on the characteristics of the input image. Furthermore, for example, the inverse quantization correction control unit 114 may acquire inverse quantization correction control information set in advance as an initial value from a memory within the image encoding device or a memory outside the image encoding device.

[0017] The prediction unit 104 divides each basic block divided by the block division unit 102 into one or more sub-blocks, performs prediction processing on the sub-blocks such as intra-frame prediction or inter-frame prediction to generate a predicted image corresponding to the sub-block, and derives the difference (error) between the sub-block and the predicted image as a prediction error. The prediction unit 104 also outputs information necessary for prediction (e.g., information on sub-block division, prediction mode, motion vector, etc.) as prediction information.

[0018] The transform / quantization unit 105 generates orthogonal transform coefficients corresponding to the sub-block by orthogonal transforming (frequency transforming) the prediction errors corresponding to the sub-block, and generates quantized coefficients by quantizing the orthogonal transform coefficients using the quantization matrix stored in the quantization matrix storage unit 103. Note that, as an example, a configuration for performing orthogonal transform and a configuration for performing quantization are represented by a single block. However, the configuration for performing orthogonal transform and the configuration for quantization may be separate.

[0019] The inverse quantization / inverse transform unit 106 generates reproduced orthogonal transform coefficients by inverse quantizing the quantization coefficients generated by the transform / quantization unit 105 using the quantization matrix stored in the quantization matrix storage unit 103. The inverse quantization / inverse transform unit 106 then corrects the generated reproduced orthogonal transform coefficients based on the inverse quantization correction control information acquired by the inverse quantization correction control unit 114, and generates (reproduces) prediction errors by performing inverse orthogonal transform on the corrected reproduced orthogonal transform coefficients. Note that, as an example, a configuration for performing inverse quantization and a configuration for performing inverse orthogonal transform are represented by a single block. However, the configuration for performing inverse quantization and the configuration for performing inverse transform may be separate.

[0020] The image reproduction unit 107 generates a predicted image by appropriately referring to the frame memory 108 based on the prediction information output from the prediction unit 104, generates a reproduced image from the predicted image and the prediction error generated (reproduced) by the inverse quantization and inverse transform unit 106, and stores the reproduced image in the frame memory 108.

[0021] The in-loop filter unit 109 performs in-loop filtering such as deblocking filtering and sample adaptive offset on the reconstructed image stored in the frame memory 108 .

[0022] The encoding unit 110 generates coded data by encoding the quantization coefficients generated by the transform / quantization unit 105 and the prediction information output from the prediction unit 104. The quantization matrix encoding unit 113 generates coded data by encoding the quantization matrix stored in the quantization matrix storage unit 103.

[0023] The integrative coding unit 111 generates header coded data using header information required for coding image data, such as inverse quantization correction control information acquired by the inverse quantization correction control unit 114, and coded data generated by the quantization matrix coding unit 113. Furthermore, the integrative coding unit 111 generates a bitstream by combining the generated header coded data with the coded data generated by the coding unit 110, and outputs the generated bitstream to an external device via the output unit 112. Note that the output destination of the bitstream is not limited to a specific output destination. For example, the integrative coding unit 111 may output (transmit) the bitstream to an external device (e.g., a memory device or a server device) via the output unit 112, or may store the bitstream in a memory within the image coding device. The control unit 150 controls the operation of the entire image coding device, including the above-mentioned functional units.

[0024] Next, a description will be given of the operation of the image encoding device in the functional configuration shown in Fig. 1. The inverse quantization correction control unit 114 acquires inverse quantization correction control information. The relationship between the value of the inverse quantization correction control information and the inverse quantization correction process will be described later.

[0025] The quantization matrix storage unit 103 acquires and stores a plurality of quantization matrices, which are generated according to the size of the sub-block and the type of prediction method. In this embodiment, the quantization matrix storage unit 103 generates a quantization matrix 800 of 8 pixels x 8 pixels, which corresponds to the sub-block of 8 pixels x 8 pixels shown in Figures 8A to 8C, as described above.

[0026] The quantization matrix 800 in FIG. 8A shows an example of a quantization matrix corresponding to intra prediction. The quantization matrix 800 in FIG. 8B shows an example of a quantization matrix corresponding to inter prediction. The quantization matrix 800 in FIG. 8C shows an example of a quantization matrix corresponding to mixed intra-inter prediction. As shown in FIG. 8, the quantization matrix is ​​composed of 8 x 8 elements (quantization step values). In this embodiment, a case will be described in which the three types of quantization matrices shown in FIGS. 8A to 8C are stored in the quantization matrix storage unit 103 as a two-dimensional array, but the elements in the quantization matrix are not limited to this. It is also possible to store multiple quantization matrices for the same prediction method depending on the size of the sub-block or whether the encoding target is a luminance block or a chrominance block. Generally, quantization matrices realize quantization processing that is in accordance with human visual characteristics, so that, as shown in FIGS. 8A to 8C, the elements for DC components corresponding to the upper left corner of the quantization matrix are small, and the elements for AC components corresponding to the lower right corner are large.

[0027] However, the generated quantization matrix is ​​not limited to this, and a quantization matrix corresponding to the shape of a sub-block, such as 4 pixels x 8 pixels, 8 pixels x 4 pixels, or 4 pixels x 4 pixels, may be generated. The method of determining each element in the quantization matrix is ​​not particularly limited. For example, a predetermined initial value may be used for each element in the quantization matrix, or each element in the quantization matrix may be set individually, or the quantization matrix may be generated according to the characteristics of the image.

[0028] The quantization matrix encoding unit 113 sequentially reads out the quantization matrices stored in the quantization matrix storage unit 103 as a two-dimensional array, scans each element in the quantization matrix, calculates the difference, and arranges the result in a one-dimensional matrix (difference matrix). In this embodiment, a scanning method is used in which elements in each quantization matrix 800 shown in FIGS. 8A to 8C are scanned in the order indicated by the arrows in FIG. 9, and the difference between each element and the previous element in the scanning order is calculated. For example, the 8-pixel by 8-pixel quantization matrix 800 shown in FIG. 8C is scanned using the scanning method shown in FIG. 9. After the first element "8" located in the upper left corner, the element "11" located immediately below it is scanned, and the difference "+3" is calculated. Furthermore, to encode the first element of the quantization matrix ("8" in the example of FIG. 8C), the difference between the first element and a predetermined initial value (e.g., "8") is calculated; however, this is not limiting, and the difference between any value or the value of the first element itself may also be used.

[0029] In this manner, in this embodiment, the quantization matrices 800 in each of Figures 8A to 8C are scanned using the scanning method of Figure 9, and the one-dimensional difference matrices 1000 shown in Figures 10A to 10C are generated. The quantization matrix encoding unit 113 further encodes the difference matrices to generate coded data of the quantization matrices. In this embodiment, the coding table shown in Figure 11A is used for coding, but the coding table is not limited to this, and for example, the coding table shown in Figure 11B may also be used.

[0030] 1, the integrated encoding unit 111 generates header encoded data by integrating encoded data of the quantization matrix with header information necessary for encoding an image, including inverse quantization correction control information. Next, image encoding will be described.

[0031] The block division unit 102 divides the input image input via the input unit 101 into a plurality of basic blocks. In this embodiment, the size of each basic block is 8 pixels x 8 pixels. The prediction unit 104 performs prediction processing on each basic block. More specifically, the prediction unit 104 first determines a sub-block division method, which is a method for dividing the basic block into smaller sub-blocks, and then determines a prediction mode, such as intra prediction, inter prediction, or mixed intra-inter prediction, for each sub-block.

[0032] Figures 7A to 7F show examples of sub-block division patterns. In Figure 7, the outer bold frame 700 represents a basic block, which in this embodiment has a size of 8 pixels x 8 pixels. The rectangles within the bold frame 700 represent sub-blocks. Figure 7A shows an example of a basic block = sub-block. Figure 7B shows an example of conventional square sub-block division, in which an 8 pixel x 8 pixel basic block is divided into four 4 pixel x 4 pixel sub-blocks. Figures 7C to 7F show an example of rectangular sub-block division. In Figure 7C, the basic block is divided into two 4 pixel x 8 pixel vertical sub-blocks, and in Figure 7D, the basic block is divided into two 8 pixel x 4 pixel horizontal rectangular sub-blocks. Furthermore, in Figures 7E and 7F, the basic block is divided into rectangular sub-blocks at a ratio of 1:2:1. In this way, encoding is performed using not only square but also rectangular sub-blocks.

[0033] For simplicity of explanation, this embodiment employs a subblock partitioning method in which an 8 pixel x 8 pixel basic block is not divided into subblocks ( FIG. 7A ). However, quadtree partitioning as shown in FIG. 7B , ternary tree partitioning as shown in FIGS. 7E and 7F , or binary tree partitioning as shown in FIGS. 7C and 7D may also be used. When a subblock partitioning method other than that shown in FIG. 7A is used, the quantization matrix storage unit 103 generates a quantization matrix corresponding to the subblock to be used. The generated quantization matrix is ​​then coded by the quantization matrix coding unit 113.

[0034] Next, the prediction modes (prediction methods) used in this embodiment will be explained again. In this embodiment, three types of prediction methods are used: intra prediction, inter prediction, and mixed intra-inter prediction. In intra prediction, predicted pixels for the block to be coded are generated using coded pixels located spatially around the block to be coded, and an intra prediction mode indicating an intra prediction method such as horizontal prediction, vertical prediction, or DC prediction is also generated. In inter prediction, predicted pixels for the block to be coded are generated using coded pixels of a frame temporally different from the block to be coded, and motion information indicating a reference frame, a motion vector, etc. is also generated.

[0035] In intra-inter mixed prediction, the current block to be coded is first divided by a diagonal line segment to generate two regions. Then, pixel values ​​generated by the above-mentioned intra prediction are used in one region, and pixel values ​​generated by the above-mentioned inter prediction are used in the other region to generate predicted pixels for the current block to be coded. FIGS. 12A and 12B show an example of region division used in this intra-inter mixed prediction. FIG. 12A shows an example in which two regions are generated from the current block to be coded 1200 along a diagonal line from the upper left vertex to the lower right vertex. For example, pixel values ​​generated by intra prediction can be used in the upper right region, and pixel values ​​generated by inter prediction can be used in the lower left region. FIG. 12B also shows an example in which two regions are generated from the current block to be coded 1200 along a diagonal line from the upper right vertex to the midpoint between the upper left vertex and the lower left vertex. For example, pixel values ​​generated by intra prediction can be placed in the upper left region, and pixel values ​​generated by inter prediction can be placed in the lower right region. In this way, in mixed intra / inter prediction, predicted pixels for the block to be coded are generated, and information on the intra prediction mode used to generate the predicted pixels, motion information, and region division is also generated.

[0036] The prediction unit 104 generates a predicted image of the sub-block to be coded based on the determined prediction mode and coded pixels. The prediction unit 104 then calculates the difference (error) between the sub-block to be coded and the predicted image of the sub-block to generate a prediction error. The prediction unit 104 also outputs prediction information such as a sub-block division method, a prediction mode (information indicating whether the prediction mode is intra prediction, inter prediction, or mixed intra-inter prediction), and vector data.

[0037] The transform / quantization unit 105 generates quantized coefficients by performing orthogonal transform and quantization on the prediction error. Specifically, the transform / quantization unit 105 generates orthogonal transform coefficients by performing orthogonal transform processing corresponding to the size of the prediction error. Next, the transform / quantization unit 105 selects a quantization matrix corresponding to the prediction mode from the quantization matrices stored in the quantization matrix storage unit 103, and quantizes the orthogonal transform coefficients using the selected quantization matrix to generate quantized coefficients. In this embodiment, the quantization matrix of FIG. 8A is selected for quantizing the orthogonal transform coefficients of sub-blocks predicted using intra prediction, and the quantization matrix of FIG. 8B is selected for quantizing the orthogonal transform coefficients of sub-blocks predicted using inter prediction. Furthermore, in this embodiment, the quantization matrix of FIG. 8C is selected for quantizing the orthogonal transform coefficients of sub-blocks predicted using mixed intra / inter prediction. However, the quantization matrices used are not limited to these.

[0038] The inverse quantization and inverse transform unit 106 generates reproduced orthogonal transform coefficients by inverse quantizing the quantization coefficients of the sub-block using the quantization matrix used for quantizing the orthogonal transform coefficients of the sub-block, among the quantization matrices stored in the quantization matrix storage unit 103. Then, the inverse quantization and inverse transform unit 106 performs inverse quantization correction processing on the reproduced orthogonal transform coefficients based on the inverse quantization correction control information.

[0039] Here, the inverse quantization correction process in this embodiment will be described. The inverse quantization process and the inverse quantization correction process in this embodiment are performed using, for example, the following equation (1).

[0040] dz[x][y]=L[x][y]×Q[x][y]+Shift (1) In equation (1), dz[x][y] is a corrected reproduction orthogonal transform coefficient corresponding to position (x, y), and L[x][y] is a quantization coefficient corresponding to position (x, y). Furthermore, Q[x][y] is a "quantization scale calculated taking into account the elements of the quantization matrix" corresponding to position (x, y). Furthermore, Shift is a correction value used in the inverse quantization correction process of this embodiment, and is determined based on the quantization coefficient L and the inverse quantization correction control information. Specifically, when the value of the inverse quantization correction control information is 0, the inverse quantization / inverse transform unit 106 sets the value of Shift in equation (1) to 0. In this case, the inverse quantization correction process does not actually correct the reproduction orthogonal transform coefficient. On the other hand, when the value of the inverse quantization correction control information is 1, the inverse quantization and inverse transform unit 106 derives the value of Shift in equation (1) using the following equation (2).

[0041] Shift = T × L[x][y] (2) In equation (2), T is a real number that takes a value greater than or equal to 0 and less than 1. In this embodiment, T is a fixed value, but is not limited to this and may take a variable value depending on the position (x, y), or T itself may be calculated using the quantization coefficient L[x][y]. For example, the value of T may be calculated according to the absolute value (|L[x][y]|) of the quantization coefficient L[x][y], as shown in the table below.

[0042]

[0043] In this case, when the quantization coefficient L[x][y] is 0, the value of T is also 0, and essentially no correction is performed. Furthermore, as the absolute value of the non-zero quantization coefficient L[x][y] (|L[x][y]|) increases, the value of T decreases, and when |L[x][y]| becomes greater than a predetermined value, the value of T also becomes 0, and essentially no correction is performed in this case either.

[0044] The inverse quantization and inverse transform unit 106 then generates (reproduces) prediction errors by inverse orthogonally transforming the reproduced orthogonal transform coefficients generated using the above equation (1). For the inverse quantization process, a quantization matrix corresponding to the prediction mode of the block to be coded is used, as in the transform and quantization unit 105. Specifically, the same quantization matrix as that used by the transform and quantization unit 105 is used.

[0045] The image reproduction unit 107 generates (reproduces) a predicted image by appropriately referring to the frame memory 108 based on the prediction information input from the prediction unit 104. The image reproduction unit 107 then adds the reproduced predicted image and the prediction error generated (reproduced) by the inverse quantization and inverse transform unit 106 to generate (reproduce) a reproduced image of the corresponding sub-block, and stores the generated reproduced image in the frame memory 108.

[0046] The in-loop filter unit 109 reads the reconstructed image from the frame memory 108, and performs in-loop filtering on the read reconstructed image using a filter such as a deblocking filter. Then, the in-loop filter unit 109 stores the reconstructed image to which the in-loop filtering has been applied back into the frame memory 108.

[0047] The encoding unit 110 generates coded data for each sub-block by entropy encoding the quantization coefficients of the sub-block generated by the transform / quantization unit 105 and the prediction information of the sub-block input from the prediction unit 104. The entropy encoding method is not limited to a specific method, and Golomb coding, arithmetic coding, Huffman coding, etc. can be used.

[0048] The integrated encoding unit 111 multiplexes each piece of encoded data to generate a bitstream and outputs the generated bitstream. Fig. 6A shows an example of the data structure of the bitstream output in this embodiment. The sequence header includes encoded data of the inverse quantization correction control information and the quantization matrix, and is composed of encoded data of each element. However, the encoding position is not limited to this, and the information may be encoded in a picture header or other header. Furthermore, when the inverse quantization correction control information or the quantization matrix is ​​changed within a single sequence, it is also possible to update the information by newly encoding the inverse quantization correction control information or the quantization matrix.

[0049] Next, the process performed by the image coding device to code one frame of input image will be described with reference to the flowchart in Fig. 3. When the image coding device codes input images of multiple frames, the image coding device performs the processes of steps S305 to S312 on the input images of each frame.

[0050] First, prior to encoding an image, in step S301, the inverse quantization correction control unit 114 acquires inverse quantization correction control information. In step S302, the quantization matrix storage unit 103 acquires and stores a plurality of quantization matrices to be used in the quantization process.

[0051] In step S303, the quantization matrix encoding unit 113 scans the quantization matrix generated in step S302, calculates the difference between each element, and generates a one-dimensional difference matrix. In step S304, the integrated encoding unit 111 generates header code data using header information necessary for encoding image data, such as the inverse quantization correction control information acquired in step S301, and the code data of the quantization matrix generated in step S303.

[0052] In step S305, the block division unit 102 divides the input image input via the input unit 101 into a plurality of basic blocks. In step S306, the prediction unit 104 selects one of the basic blocks divided in step S305 that has not yet been selected as a selected basic block. The prediction unit 104 then divides the selected basic block into sub-blocks (including the case where the selected basic block is the same as the sub-block), derives prediction errors for the sub-blocks, and outputs prediction information.

[0053] A specific example of processing by the prediction unit 104 is as follows. The prediction unit 104 performs intra prediction processing on a target sub-block (target block) to be coded by referencing an already-coded region in the same input image as the input image to which the target sub-block belongs, thereby generating an intra-predicted image. The prediction unit 104 also performs inter prediction processing by referencing an already-coded input image (e.g., the input image of the immediately preceding frame) different from the input image to which the target sub-block to be coded belongs, thereby generating an inter-predicted image. The prediction unit 104 then divides the target sub-block into two regions, as in the example of FIG. 12 shown above, and places the intra-predicted image in one region and the inter-predicted image in the other region, thereby generating an intra-inter mixed predicted image. The prediction unit 104 then derives the sum of squares (or the sum of absolute values) of the differences in pixel values ​​between positionally corresponding pixels in each of these three predicted images and the target sub-block, and determines the prediction mode of the predicted image with the smallest sum of squares as the prediction mode for the target sub-block. The prediction unit 104 then performs a prediction process on the sub-block in accordance with the prediction mode to generate a predicted image, and derives the difference between the sub-block and the predicted image as a prediction error.

[0054] In step S307, the transform / quantization unit 105 performs orthogonal transform on the prediction errors derived in step S306 for each subblock to generate orthogonal transform coefficients. Then, for each subblock, the transform / quantization unit 105 selects one of the quantization matrices stored in the quantization matrix storage unit 103 based on the prediction information, and quantizes the orthogonal transform coefficients using the selected quantization matrix to generate quantized coefficients.

[0055] In step S308, the inverse quantization / inverse transform unit 106 performs inverse quantization on the quantization coefficients generated in step S307 for each sub-block using the quantization matrix selected in step S307 to generate reproduced orthogonal transform coefficients, performs inverse quantization correction processing on the reproduced orthogonal transform coefficients based on the inverse quantization correction control information, and then performs inverse orthogonal transform to generate (reproduce) prediction errors. That is, the inverse quantization / inverse transform unit 106 performs inverse quantization processing and inverse quantization correction processing on the reproduced orthogonal transform coefficients in accordance with the above equation (1).

[0056] In step S309, the image reproduction unit 107 generates a predicted image for each sub-block by referring to the frame memory 108 based on the prediction information output in step S306, and generates a reproduced image using the predicted image and the prediction error generated in step S308.

[0057] In step S310, the encoding unit 110 generates coded data for each subblock of the basic block by encoding the prediction information output in step S306 and the quantization coefficients generated in step S307. The integrated encoding unit 111 then multiplexes the header coded data generated in step S304, the coded data generated by the encoding unit 110, and the like to generate a bitstream.

[0058] In step S311, the control unit 150 determines whether or not all basic blocks in the input image have been selected as selected basic blocks (that is, whether or not the encoding of all basic blocks (the processing of steps S306 to S310) has been completed).

[0059] If, as a result of this determination, all basic blocks in the input image have been selected as selected basic blocks, the process proceeds to step S312; if there are any basic blocks remaining in the input image that have not yet been selected as selected basic blocks, the process proceeds to step S306.

[0060] In step S312, the in-loop filter unit 109 reads the reconstructed image from the frame memory 108, performs in-loop filtering on the reconstructed image, and stores the reconstructed image to which the in-loop filtering has been applied back into the frame memory 108.

[0061] With the above configuration and operation, particularly in step S308, by correcting the reproduced orthogonal transform coefficients based on the inverse quantization correction control information, appropriate correction can be made to the reproduced orthogonal transform coefficients generated by the inverse quantization process using a quantization matrix, thereby improving compression efficiency.

[0062] In the present embodiment, a quantization matrix is ​​used in the inverse quantization process for all sub-blocks in a frame to generate reproduced orthogonal transform coefficients, and correction is performed based on the inverse quantization correction control information. However, this is not limiting. For example, some sub-blocks in a frame (e.g., sub-blocks with a size of 8 pixels x 8 pixels) may be inverse quantized using a quantization matrix, while other sub-blocks (e.g., sub-blocks with a size of 4 pixels x 4 pixels) may be inverse quantized without using a quantization matrix (i.e., using the same quantization scale for all frequency components). In this case, correction according to Equation (1) may be performed for sub-blocks inverse quantized using a quantization matrix in accordance with the inverse quantization correction control information, while correction according to Equation (1) may always be performed for sub-blocks inverse quantized without using a quantization matrix, regardless of the inverse quantization correction control information. This allows appropriate correction to be performed on each sub-block, improving compression efficiency, even when sub-blocks inverse quantized using a quantization matrix and sub-blocks inverse quantized without using a quantization matrix are mixed.

[0063] Furthermore, in this embodiment, the quantization correction control information is coded and included in the bitstream, but this is not limiting. For example, by always setting the quantization correction control information to 0, it is possible to omit the code of the quantization correction control information to be included in the bitstream, as shown in FIG. 6B . In this case, the inverse quantization correction process is always performed on sub-blocks that are inverse quantized without using a quantization matrix, and the inverse quantization correction process is never performed on sub-blocks that are inverse quantized using a quantization matrix. This simplifies the relationship between whether or not a quantization matrix is ​​applied and whether or not the inverse quantization correction process is applied, making control easier, and furthermore, it is possible to reduce the amount of code corresponding to the code of the inverse quantization correction control information.

[0064] Furthermore, in this embodiment, the inverse quantization correction control information is configured to indicate only whether or not the inverse quantization correction process is applied, but the value of the inverse quantization correction control information can also be set as the value of a parameter used in the inverse quantization correction process. For example, the value of the inverse quantization correction control information can also be set as the value of Shift in the above-mentioned equation (1) or the value of T in equation (2). This makes it possible to control the strength of the inverse quantization correction according to the characteristics of the image, and as a result, the compression efficiency can be improved.

[0065] Furthermore, in this embodiment, three types of prediction methods are used: intra prediction, inter prediction, and intra-inter mixed prediction. However, since the characteristics of each prediction and its error are different, a different inverse quantization correction process can be performed depending on the prediction method. For example, by individually setting inverse quantization correction control information corresponding to a sub-block using intra prediction, inverse quantization correction control information corresponding to a sub-block using inter prediction, and inverse quantization correction control information corresponding to a sub-block using intra-inter mixed prediction, it is possible to apply inverse quantization correction control suitable for each prediction method. In this case, each inverse quantization correction control information may be coded and included in the bitstream, or each inverse quantization correction control information may be set to a fixed value and coding of each may be omitted.

[0066] In this embodiment, an image is encoded frame by frame to generate and output a bitstream, but the encoding process is not limited to images. For example, features used in machine learning such as object recognition may be encoded as two-dimensional array data. This allows the features used in machine learning to be efficiently encoded.

[0067] [Second Embodiment] An image decoding device according to this embodiment decodes a frame-by-frame bitstream generated by the image encoding device according to the first embodiment. An example of the functional configuration of the image decoding device according to this embodiment will be described with reference to the block diagram in Fig. 2 .

[0068] The separate decoding unit 202 acquires a bitstream via the input unit 201. The method by which the separate decoding unit 202 acquires the bitstream is not limited to a specific method. For example, the separate decoding unit 202 may acquire a bitstream stored in an external device such as a server device via a network, or may acquire a bitstream generated by an imaging device from the imaging device. The separate decoding unit 202 then separates header coded data and coded data in sub-block units of a basic block from the bitstream. In short, the separate decoding unit 202 performs the reverse operation of the integrated coding unit 111 in FIG. 1 . The separate decoding unit 202 also extracts inverse quantization correction control information from the header coded data.

[0069] The quantization matrix decoding unit 209 reproduces the quantization matrix by decoding the header coded data separated by the separation decoding unit 202. The decoding unit 203 decodes the coded data of each sub-block of the basic block separated by the separation decoding unit 202, and reproduces the quantization coefficients and prediction information.

[0070] The inverse quantization and inverse transform unit 204 generates reproduced orthogonal transform coefficients by inverse quantizing the quantization coefficients reproduced by the decoding unit 203 using the quantization matrix reproduced by the quantization matrix decoding unit 209, in the same manner as the inverse quantization and inverse transform unit 106. Then, like the inverse quantization and inverse transform unit 106, the inverse quantization and inverse transform unit 204 performs inverse quantization correction processing on the reproduced orthogonal transform coefficients based on the inverse quantization correction control information extracted by the separate decoding unit 202, and then performs inverse orthogonal transform to generate (reproduce, derive) prediction errors.

[0071] Similar to the image reproduction unit 107, the image reproduction unit 205 generates a predicted image by appropriately referring to the frame memory 206 based on the prediction information reproduced by the decoding unit 203. Similar to the image reproduction unit 107, the image reproduction unit 205 then generates a reproduced image by adding the prediction error reproduced by the inverse quantization and inverse transform unit 204 to the predicted image, and stores the reproduced image in the frame memory 206.

[0072] Similar to the in-loop filter unit 109, the in-loop filter unit 207 reads a reproduced image from the frame memory 206, performs in-loop filtering on the reproduced image, and stores the reproduced image after the in-loop filtering again in the frame memory 206. The reproduced image after the in-loop filtering by the in-loop filter unit 207 is output to an external device via the output unit 208 under the control of the control unit 250.

[0073] The output destination of the reproduced image is not limited to a specific output destination. For example, the control unit 250 may transmit the reproduced image to an external device via a network, or may output the reproduced image to a display device connected to the image decoding device and display the reproduced image on the display device. The control unit 250 controls the operation of the entire image decoding device, including the above-mentioned functional units.

[0074] Next, the operation of the image decoding device in the functional configuration shown in Fig. 2 will be described. A bitstream for one frame input via an input unit 201 is input to a separate decoding unit 202. The separate decoding unit 202 according to this embodiment extracts inverse quantization correction control information from the sequence header of the bitstream shown in Fig. 6A, and also extracts coded data of the quantization matrices shown in Figs. 8A to 8C from the sequence header. The separate decoding unit 202 also reproduces coded data for each sub-block of the basic block of picture data.

[0075] The quantization matrix decoding unit 209 decodes the coded data of the quantization matrix and reproduces the one-dimensional difference matrix shown in FIGS. 10A to 10C. In this embodiment, as in the first embodiment, decoding is performed using the coding table shown in FIG. 11A (or FIG. 11B). However, the coding table is not limited to this, and other coding tables may be used as long as they are the same as those in the first embodiment. The quantization matrix decoding unit 209 then inversely scans the reproduced one-dimensional difference matrix to reproduce the quantization matrix as a two-dimensional array. In other words, the quantization matrix decoding unit 209 performs the reverse operation of the quantization matrix encoding unit 113. In other words, the quantization matrix decoding unit 209 reproduces the three types of quantization matrices shown in FIGS. 8A to 8C from the difference matrix shown in FIGS. 10A to 10C using the scanning method shown in FIG. 9.

[0076] The decoding unit 203 decodes the coded data of each sub-block of the basic block to reproduce the quantization coefficients and prediction information. The inverse quantization and inverse transform unit 204 selects one of the quantization matrices reproduced by the quantization matrix decoding unit 209 and generates reproduced orthogonal transform coefficients by inverse quantizing the quantization coefficients reproduced by the decoding unit 203 using the selected quantization matrix. Then, similar to the inverse quantization and inverse transform unit 106, the inverse quantization and inverse transform unit 204 performs inverse quantization correction processing on the generated reproduced orthogonal transform coefficients based on the inverse quantization correction control information extracted by the separate decoding unit 202, and then performs inverse orthogonal transform to generate (reproduce) prediction errors.

[0077] The inverse quantization and inverse transform unit 204 of this embodiment determines the quantization matrix to be used in the inverse quantization process according to the prediction mode of the sub-block to be decoded, which is determined in accordance with the prediction information reproduced by the decoding unit 203. That is, the quantization matrix of Fig. 8A is selected for the sub-block using intra prediction, the quantization matrix of Fig. 8B for the sub-block using inter prediction, and the quantization matrix of Fig. 8C for the sub-block using mixed intra / inter prediction. However, the quantization matrix to be used is not limited to this, and may be the same as the quantization matrix used in the transform and quantization unit 105 and the inverse quantization and inverse transform unit 106 of the first embodiment.

[0078] Similar to the image reproduction unit 107, the image reproduction unit 205 generates a predicted image by appropriately referring to the frame memory 206 based on the prediction information reproduced by the decoding unit 203. In this embodiment, similar to the prediction unit 104, three types of prediction methods are used: intra prediction, inter prediction, and mixed intra-inter prediction. Similar to the image reproduction unit 107, the image reproduction unit 205 generates a reproduced image by adding the prediction error reproduced by the inverse quantization and inverse transform unit 204 to the predicted image, and stores the reproduced image in the frame memory 206. The stored reproduced image becomes a prediction reference candidate when decoding other sub-blocks.

[0079] Similar to the in-loop filter unit 109, the in-loop filter unit 207 performs in-loop filtering on the reproduced image stored in the frame memory 206. As described above, the reproduced image to which the in-loop filtering has been applied by the in-loop filter unit 207 is output to an external device via the output unit 208.

[0080] Next, the process performed by the image decoding device to decode a bitstream for one frame will be described with reference to the flowchart in Fig. 4. When the image decoding device decodes bitstreams for multiple frames, the image decoding device performs the process according to the flowchart in Fig. 4 on the bitstream for each frame.

[0081] In step S401, the separate decoding unit 202 extracts (decodes) the inverse quantization correction control information from the bit stream, and reproduces (separates) from the bit stream the coded data of the quantization matrix and the coded data of each sub-block of the basic block.

[0082] In step S402, the quantization matrix decoding unit 209 reproduces a one-dimensional difference matrix by decoding the coded data of the quantization matrix reproduced in step S401, and then inversely scans the reproduced one-dimensional difference matrix to reproduce the quantization matrix as a two-dimensional array.

[0083] In step S403, the decoding unit 203 decodes the coded data of each sub-block of the basic block reconstructed in step S401 to reconstruct quantization coefficients and prediction information. In step S404, the inverse quantization and inverse transform unit 204 selects one of the quantization matrices reconstructed in step S402 and inverse quantizes the quantization coefficients reconstructed in step S403 using the selected quantization matrix to generate reconstructed orthogonal transform coefficients. The inverse quantization and inverse transform unit 204 then performs inverse quantization correction processing on the reconstructed orthogonal transform coefficients based on the inverse quantization correction control information extracted in step S401, and then performs inverse orthogonal transform to generate (reconstruct) prediction errors.

[0084] In step S405, the image reproduction unit 205 generates a predicted image based on the prediction information reproduced in step S403 by appropriately referring to the frame memory 206. Then, the image reproduction unit 205 generates a reproduced image by adding the prediction error reproduced in step S404 to the predicted image, and stores the reproduced image in the frame memory 206.

[0085] In step S406, the control unit 250 determines whether the processing of steps S403 to S405 has been performed for all basic blocks. If the result of this determination is that the processing of steps S403 to S405 has been performed for all basic blocks, the process proceeds to step S407. On the other hand, if there are any basic blocks that have not been subjected to the processing of steps S403 to S405, the process proceeds to step S403 to perform the processing of steps S403 to S405 for those basic blocks. In step S407, the in-loop filter unit 207 performs in-loop filtering on the reconstructed image stored in the frame memory 206.

[0086] With the above configuration and operation, by correcting the reproduced orthogonal transform coefficients based on the inverse quantization correction control information, appropriate correction can be made to the reproduced orthogonal transform coefficients generated by the inverse quantization process using a quantization matrix, and a bitstream with improved compression efficiency can be decoded.

[0087] Note that, like the first embodiment, this embodiment is also configured to generate reproduced orthogonal transform coefficients using a quantization matrix in the inverse quantization process for all sub-blocks in a frame and perform correction based on the inverse quantization correction control information, but is not limited to this. For example, like the first embodiment, this embodiment can also be configured to perform correction using equation (1) in accordance with the inverse quantization correction control information for sub-blocks that have been inverse quantized using a quantization matrix, and to always perform correction using equation (1) regardless of the inverse quantization correction control information for sub-blocks that have been inverse quantized without using a quantization matrix. As a result, even when sub-blocks that have been inverse quantized using a quantization matrix and sub-blocks that have been inverse quantized without using a quantization matrix are mixed, it is possible to perform appropriate correction on each sub-block and decode a bitstream with improved compression efficiency.

[0088] Furthermore, in this embodiment, a bitstream in which quantization correction control information is coded is decoded, but this is not limiting. For example, by always setting the value of the quantization correction control information to 0, a bitstream in which the code of the quantization correction control information is omitted can be decoded as shown in FIG. 6B . In this case, as in the first embodiment, the inverse quantization correction process is always applied to subblocks that are inverse quantized without using a quantization matrix, and the inverse quantization correction process is never applied to subblocks that are inverse quantized using a quantization matrix. This simplifies the relationship between whether or not a quantization matrix is ​​applied and whether or not the inverse quantization correction process is applied, making control easier, and furthermore, it is possible to decode a bitstream in which the code amount corresponding to the code of the inverse quantization correction control information is reduced.

[0089] In this embodiment, as in the first embodiment, the parameters used in the inverse quantization correction process can also be used as inverse quantization correction control information. This makes it possible to control the strength of the inverse quantization correction according to the characteristics of the image, and as a result, it is possible to decode a bitstream with improved compression efficiency.

[0090] Also in this embodiment, similarly to the first embodiment, it is possible to configure the present invention to perform different inverse quantization correction processing depending on the prediction method. In this case, similarly to the first embodiment, it is possible to decode a bit stream in which each piece of inverse quantization correction control information is coded, or it is possible to set each piece of inverse quantization correction control information to a fixed value and omit decoding each piece of information.

[0091] In this embodiment, a bitstream is decoded frame by frame, but the target of the decoding process is not limited to a bitstream obtained by encoding an image. For example, features used in machine learning such as object recognition may be treated as two-dimensional array data, and the bitstream generated by encoding the data may be decoded. This makes it possible to efficiently decode a bitstream obtained by encoding features used in machine learning.

[0092] [Third Embodiment] In the first embodiment, the functional units shown in Fig. 1 are all described as being implemented in hardware. However, the functional units except for the frame memory 108 may be implemented in software (computer program). In this case, a computer device capable of executing this software is applicable to an image encoding device.

[0093] In the second embodiment, the functional units shown in Fig. 2 are described as being implemented in hardware. However, the functional units except for the frame memory 206 may be implemented in software (computer program). In this case, a computer device capable of executing this software is applicable to the image decoding device.

[0094] An example of the hardware configuration of a computer device applicable to an image encoding device and an image decoding device will be described using the block diagram of Fig. 5. Note that the configuration shown in Fig. 5 is an example of the hardware configuration of a computer device applicable to an image encoding device and an image decoding device, and can be changed / modified as appropriate. Furthermore, computer devices with different configurations may be applied to the image encoding device and the image decoding device. Furthermore, the image encoding device and the image decoding device may be the same device.

[0095] The CPU 501 executes various processes using computer programs and data stored in the RAM 502 and the ROM 503. As a result, the CPU 501 controls the operation of the entire computer device, and also executes or controls the various processes described as processes performed by the image encoding device and the image decoding device.

[0096] The RAM 502 has an area for storing computer programs and data loaded from the ROM 503 or storage device 506, and an area for storing computer programs and data received from the outside via the I / F 507. The RAM 502 also has a work area used by the CPU 501 when executing various processes. In this way, the RAM 502 can provide various areas as needed.

[0097] The ROM 503 stores setting data for the computer device, computer programs and data related to the startup of the computer device, computer programs and data related to the basic operation of the computer device, and the like.

[0098] The operation unit 504 is a user interface such as a keyboard, a mouse, and a touch panel screen, and allows the user to input various instructions and information to the computer device by operating it.

[0099] The display unit 505 has a liquid crystal screen or a touch panel screen, and can display the processing results by the CPU 501 as images, text, etc. The display unit 505 may also be a projection device such as a projector that projects images and text.

[0100] The storage device 506 is a non-volatile memory device such as a hard disk drive, etc. The storage device 506 stores an operating system (OS), computer programs and data for causing the CPU 501 to execute or control the various processes described as processes performed by the image encoding device and the image decoding device, and the like.

[0101] The computer programs stored in the storage device 506 include computer programs for causing the CPU 501 to execute or control the various processes described as processes performed by the functional units (excluding the frame memory 108) shown in Fig. 1. The computer programs stored in the storage device 506 also include computer programs for causing the CPU 501 to execute or control the various processes described as processes performed by the functional units (excluding the frame memory 206) shown in Fig. 2. The frame memory 108 and the frame memory 206 can be implemented using the RAM 502 and the storage device 506.

[0102] The I / F 507 is a communication interface for performing data communication with an external device. For example, by performing data communication with an image capture device or a server device via the I / F 507, the computer device can acquire input images or bitstreams from the image capture device or the server device, or transmit bitstreams to the server device.

[0103] The CPU 501, RAM 502, ROM 503, operation unit 504, display unit 505, storage device 506, and I / F 507 are all connected to a system bus 508. In this configuration, when the computer is powered on, the CPU 501 executes a boot program stored in ROM 503, loads an OS stored in storage device 506 into RAM 502, and starts the OS. As a result, the computer becomes capable of communication via the I / F 507. Then, under control of the OS, the CPU 501 loads an application related to image encoding (corresponding to FIG. 3 ) from the storage device 506 into RAM 502 and executes it, thereby causing the computer to function as an image encoding device. On the other hand, when the CPU 501 loads an application related to image decoding (corresponding to FIG. 4 ) from the storage device 506 into RAM 502 and executes it, the computer functions as an image decoding device.

[0104] The numerical values, processing timing, processing order, processing subject, data (information) configuration / acquisition method / sending destination / sending source / storage location, etc. used in each of the above embodiments are given as examples to provide a concrete explanation, and are not intended to be limited to these examples.

[0105] Furthermore, some or all of the above-described embodiments may be used in appropriate combination. Furthermore, some or all of the above-described embodiments may be used selectively. (Other Examples) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., an ASIC) that realizes one or more functions.

[0106] The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention.

[0107] This application claims priority based on Japanese Patent Application No. 2023-168873, filed September 28, 2023, the entire contents of which are incorporated herein by reference.

Claims

1. An image coding device which codes an image on a block-by-block basis, comprising: a prediction means which generates a predicted image from already-coded pixels for a target block of a predetermined size in the image to be coded, and derives a prediction error which is the difference between the target block and the predicted image; a transformation means which frequency-transforms the prediction error derived by the prediction means; a quantization means which quantizes an orthogonal transform coefficient obtained by frequency transformation by the transformation means; an encoding means which entropy-codes the orthogonal transform coefficient quantized by the quantization means; and an inverse quantization means which inverse quantizes the orthogonal transform coefficient quantized by the quantization means to generate reproduced orthogonal transform coefficients, wherein the inverse quantization means corrects the reproduced orthogonal transform coefficients.

2. The image coding device according to claim 1, characterized in that the inverse quantization means corrects the reproduced orthogonal transform coefficients when inverse quantization is performed without using a quantization matrix, and controls so as not to correct the reproduced orthogonal transform coefficients when inverse quantization is performed using a quantization matrix.

3. The image encoding device according to claim 1 or 2, wherein said encoding means further encodes inverse quantization correction control information for controlling the correction process in said inverse quantization means.

4. An image coding apparatus according to any one of claims 1 to 3, characterized in that the inverse quantization means performs different corrections on the reconstructed orthogonal transform coefficients depending on a method for generating a predicted image in the prediction means.

5. An image decoding device that decodes an image on a block-by-block basis, comprising: a decoding means for decoding quantized orthogonal transform coefficients; an inverse quantization means for inverse quantizing the quantized orthogonal transform coefficients to generate reconstructed orthogonal transform coefficients; an inverse transform means for inverse transforming the reconstructed orthogonal transform coefficients to derive prediction errors; and a prediction means for generating a predicted image and decoding a block of interest using the predicted image and the prediction error, wherein the inverse quantization means corrects the reconstructed orthogonal transform coefficients.

6. The image decoding device according to claim 5, characterized in that the inverse quantization means corrects the reconstructed orthogonal transform coefficients when inverse quantization is performed without using a quantization matrix, and controls so as not to correct the reconstructed orthogonal transform coefficients when inverse quantization is performed using a quantization matrix.

7. The image decoding device according to claim 5 or 6, wherein said decoding means further decodes inverse quantization correction control information for controlling the correction process in said inverse quantization means.

8. An image decoding apparatus according to any one of claims 5 to 7, characterized in that the inverse quantization means performs different corrections on the reconstructed orthogonal transform coefficients depending on a method for generating a predicted image in the prediction means.

9. An image coding method performed by an image coding device that codes an image on a block-by-block basis, comprising: a prediction step in which a prediction means of the image coding device generates a predicted image from already-coded pixels for a target block of a predetermined size in the image to be coded, and derives a prediction error which is the difference between the target block and the predicted image; a transformation step in which a transformation means of the image coding device frequency-transforms the prediction error derived in the prediction step; a quantization step in which a quantization means of the image coding device quantizes an orthogonal transform coefficient obtained by frequency transformation in the transformation step; an encoding step in which an encoding means of the image coding device entropy-codes the orthogonal transform coefficient quantized in the quantization step; and an inverse quantization step in which a dequantization means of the image coding device inversely quantizes the orthogonal transform coefficient quantized in the quantization step to generate reproduced orthogonal transform coefficients, wherein the reproduced orthogonal transform coefficients are corrected in the inverse quantization step.

10. An image decoding method performed by an image decoding device that decodes an image on a block-by-block basis, comprising: a decoding step in which a decoding means of the image decoding device decodes quantized orthogonal transform coefficients; an inverse quantization step in which an inverse quantization means of the image decoding device inverse quantizes the quantized orthogonal transform coefficients to generate reconstructed orthogonal transform coefficients; an inverse transform step in which an inverse transform means of the image decoding device inversely transforms the reconstructed orthogonal transform coefficients to derive prediction errors; and a prediction step in which a prediction means of the image decoding device generates a predicted image and decodes a block of interest using the predicted image and the prediction error, wherein the reconstructed orthogonal transform coefficients are corrected in the inverse quantization step.

11. A computer program for causing a computer of an image coding device that codes an image on a block-by-block basis to function as each of the means of the image coding device according to any one of claims 1 to 4.

12. A computer program for causing a computer of an image decoding device that decodes an image in units of blocks to function as each of the means of the image decoding device according to any one of claims 5 to 8.

Images