Image encoding device, image encoding method, image decoding device, image decoding method, and computer program

Abstract

According to the present invention, a prediction image is generated from encoded pixels for a block of interest having a predetermined size as an encoding object in an image, a prediction error that is a difference between the block of interest and the prediction image is derived, the prediction error is frequency-converted, and an orthogonal conversion coefficient obtained by the frequency conversion is quantized. Then, entropy encoding is performed on the quantized orthogonal transformation coefficients, and reproduced orthogonal transformation coefficients are generated by performing inverse quantization on the quantized orthogonal transformation coefficients. And correcting the reproduced orthogonal transformation coefficient.

Description

Technical Field

[0001] This invention relates to an image encoding device, an image encoding method, an image decoding device, an image decoding method, and a computer program. Background Technology

[0002] The VVC (Video Coding Universal) coding method (hereinafter referred to as VVC) is known as an encoding method for compressed recording of moving images. In VVC, in order to improve coding efficiency, basic blocks with a maximum size of 128 pixels × 128 pixels are divided into sub-blocks that have not only a regular square shape but also a rectangular shape.

[0003] Furthermore, in VVC, a process is used to weight the coefficients after orthogonal transformation (hereinafter referred to as orthogonal transformation coefficients) according to the frequency components using a quantization matrix. This reduces the amount of data degraded for high-frequency components that are not visually apparent to the human eye, thereby improving compression efficiency while maintaining image quality. Patent Document 1 discloses a technique for encoding such a quantization matrix.

[0004] In recent years, the JVET (Joint Video Experts Group), which standardized VVC, has explored techniques to achieve better compression efficiency than VVC. To improve coding efficiency, a novel inverse quantization method (hereinafter referred to as inverse quantization correction) has been investigated, which adds a correction value to the orthogonal transform coefficients (hereinafter referred to as reconstructed orthogonal transform coefficients) generated through inverse quantization processing. Existing technical documents Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2013-38758 Summary of the Invention The problem the invention aims to solve

[0006] This inverse quantization correction is a process used to add correction values ​​to the reconstructed orthogonal transform coefficients, and it adds predetermined correction values ​​regardless of whether a quantization matrix is ​​used. Therefore, it is not possible to perform a process to add appropriate correction values ​​to the reconstructed orthogonal transform coefficients generated by inverse quantization using a quantization matrix, and it does not improve compression efficiency.

[0007] The present invention provides a technique for improving compression efficiency by enabling control that allows appropriate correction values ​​to be added even to the reproducible orthogonal transform coefficients generated by inverse quantization of the quantization matrix. Solution for solving the problem

[0008] According to one aspect of the present invention, an image encoding apparatus is provided for encoding an image in blocks, characterized in that the image encoding apparatus comprises: a prediction unit for generating a prediction image from encoded pixels for a block of interest of a predetermined size in the image as the object of encoding, and deriving a prediction error as the difference between the block of interest and the prediction image; a transform unit for performing a frequency transform on the prediction error derived by the prediction unit; a quantization unit for quantizing orthogonal transform coefficients obtained by the frequency transform using the transform unit; an encoding unit for entropy encoding the orthogonal transform coefficients quantized by the quantization unit; and an inverse quantization unit for generating reproducible orthogonal transform coefficients by inverse quantizing the orthogonal transform coefficients quantized by the quantization unit, wherein the inverse quantization unit corrects the reproducible orthogonal transform coefficients. Advantages of the invention

[0009] According to the present invention, a technique can be provided for improving compression efficiency by enabling control that allows appropriate correction values ​​to be added even to the reproducible orthogonal transform coefficients generated by inverse quantization of the quantization matrix.

[0010] Other features and advantages of the invention will become apparent from the following description taken in conjunction with the accompanying drawings. Note that throughout the drawings, the same reference numerals denote the same or similar components. Attached Figure Description

[0011] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and, together with the specification, serve to explain the principles of the invention. Figure 1 This is a block diagram illustrating an example of the functional configuration of an image encoding device; Figure 2 This is a block diagram illustrating an example of the functional configuration of an image decoding device; Figure 3 This is a flowchart of the process by which an image encoding device encodes an input image frame. Figure 4 This is a flowchart of the process performed by an image decoding device to decode the bitstream of a frame; Figure 5 This is a block diagram illustrating an example of the hardware configuration of a computer device suitable for an image encoding device or an image decoding device; Figure 6A This is a diagram illustrating an example of a bitstream data structure; Figure 6B This is a diagram illustrating an example of a bitstream data structure; Figure 7A This is a diagram illustrating an example of a sub-block segmentation pattern; Figure 7B This is a diagram illustrating an example of a sub-block segmentation pattern; Figure 7C This is a diagram illustrating an example of a sub-block segmentation pattern; Figure 7D This is a diagram illustrating an example of a sub-block segmentation pattern; Figure 7E This is a diagram illustrating an example of a sub-block segmentation pattern; Figure 7F This is a diagram illustrating an example of a sub-block segmentation pattern; Figure 8A This is a diagram illustrating an example of the configuration of the quantization matrix 800; Figure 8B This is a diagram illustrating an example of the configuration of the quantization matrix 800; Figure 8C This is a diagram illustrating an example of the configuration of the quantization matrix 800; Figure 9 This is a diagram illustrating the scanning method; Figure 10A This is a diagram illustrating an example of the configuration of a one-dimensional difference matrix; Figure 10B This is a diagram illustrating an example of the configuration of a one-dimensional difference matrix; Figure 10C This is a diagram illustrating an example of the configuration of a one-dimensional difference matrix; Figure 11A This is a diagram illustrating an example of the structure of the encoding table; Figure 11B This is a diagram illustrating an example of the structure of the encoding table; Figure 12A This is a diagram illustrating an example of region segmentation used in hybrid intra-frame / inter-frame prediction; and Figure 12B This is a diagram illustrating an example of region segmentation used in hybrid intra-frame-inter-frame prediction. Detailed Implementation

[0012] In the following, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments are not intended to limit the claimed invention. Multiple features are described in the embodiments, but this does not limit the invention to requiring all of these features, and multiple such features can be appropriately combined. Furthermore, in the drawings, the same reference numerals are given the same or similar configuration, and redundant descriptions are omitted.

[0013] [First Embodiment] First, refer to Figure 1The block diagram illustrates an example of the functional configuration of an image encoding device according to this embodiment. PCs (personal computers), smartphones, tablet terminals, camera devices, or dedicated image processing circuits can be used with the image encoding device.

[0014] The image encoding device acquires an image of the object to be encoded as an input image via input unit 101. The method of acquiring the input image using the image encoding device is not limited to a specific method. For example, the image encoding device can acquire images output from a camera device (images of individual frames in a moving image, or still images captured periodically or irregularly, etc.) as input images. Note that if the image encoding device and the camera device are integrated, the image encoding device acquires images captured by its own camera device as input images. Alternatively, for example, the image encoding device can acquire images stored in an external device such as a server device via a network such as a LAN or the Internet as input images. Alternatively, for example, the image encoding device can acquire images stored in its own storage device as input images.

[0015] The block segmentation unit 102 segments the input image into multiple basic blocks (hereinafter also appropriately referred to as blocks). The quantization matrix holding unit 103 acquires and holds multiple quantization matrices to be used for quantization processing. The method of acquiring the quantization matrices using the quantization matrix holding unit 103 is not limited to a specific method. For example, the quantization matrix holding unit 103 may acquire the quantization matrix input by the user through operations on the operation unit (not shown), calculate the quantization matrix from the characteristics of the input image, or acquire the quantization matrix that has been pre-specified with initial values. In this embodiment, the quantization matrix holding unit 103 acquires... Figures 8A to 8C The three types of "two-dimensional quantization matrices 800 corresponding to an 8-pixel × 8-pixel orthogonal transformation (frequency transformation)" are illustrated and preserved.

[0016] The inverse quantization correction control unit 114 acquires inverse quantization correction control information, which is information used to control the inverse quantization correction processing performed by the subsequent inverse quantization / inverse transform unit 106. The method by which the inverse quantization correction control unit 114 acquires the inverse quantization correction control information is not limited to a specific method. For example, the inverse quantization correction control unit 114 may acquire inverse quantization correction control information input by a user operating an operation unit (not shown), or it may calculate the inverse quantization correction control information based on the characteristics of the input image. Alternatively, for example, the inverse quantization correction control unit 114 may acquire inverse quantization correction control information preset to initial values ​​from memory inside or outside the image encoding device.

[0017] For each basic block segmented by the block segmentation unit 102, the prediction unit 104 divides the basic block into one or more sub-blocks, generates a prediction image corresponding to the sub-block by performing prediction processing on the sub-block (such as intra-prediction as intra-prediction or inter-prediction as inter-prediction), and derives the difference (error) between the sub-block and the prediction image as the prediction error. In addition, the prediction unit 104 outputs the information required for prediction (e.g., information representing sub-block segmentation, prediction mode, and motion vectors) as prediction information.

[0018] The transform / quantization unit 105 generates orthogonal transform coefficients corresponding to the sub-block by performing an orthogonal transform (frequency transform) on the prediction error corresponding to the sub-block, and generates quantized coefficients by quantizing the orthogonal transform coefficients using a quantization matrix held by the quantization matrix holding unit 103. Note that, as an example, the component for performing the orthogonal transform and the component for performing the quantization are represented by a single block. However, the component for performing the orthogonal transform and the component for performing the quantization can be separate.

[0019] The inverse quantization / inverse transform unit 106 generates reproducible quadrature transform coefficients by inverse quantization of the quantization coefficients generated by the transform / quantization unit 105 using the quantization matrix held by the quantization matrix holding unit 103. Then, the inverse quantization / inverse transform unit 106 corrects the generated reproducible quadrature transform coefficients based on inverse quantization correction control information obtained by the inverse quantization correction control unit 114, and performs an inverse quadrature transform on the corrected reproducible quadrature transform coefficients, thereby generating (reproducible) prediction error. Note that, as an example, the components for inverse quantization and the components for inverse quadrature transform are represented by a single block. However, the components for inverse quantization and the components for inverse transform can be separate.

[0020] The image reconstruction unit 107 generates a prediction image based on the prediction information output from the prediction unit 104 by appropriately referencing the frame memory 108, generates a reconstruction image from the prediction image and the prediction error generated (reconstructed) by the inverse quantization / inverse transform unit 106, and stores the reconstruction image in the frame memory 108.

[0021] The in-loop filtering unit 109 performs in-loop filtering on the reconstructed image stored in the frame memory 108, such as deblocking filtering or sample adaptive shifting.

[0022] Encoding unit 110 generates encoded data by encoding the quantization coefficients generated by transform / quantization unit 105 and the prediction information output from prediction unit 104. Quantization matrix encoding unit 113 generates encoded data by encoding the quantization matrix held by quantization matrix holding unit 103.

[0023] The integrated encoding unit 111 generates header encoded data using header information required for encoding the image data (such as inverse quantization correction control information acquired by the inverse quantization correction control unit 114) and encoded data generated by the quantization matrix encoding unit 113. Furthermore, the integrated encoding unit 111 generates a bitstream by combining the encoded data generated by the encoding unit 110 with the generated header encoded data, and outputs the generated bitstream to an external device via the output unit 112. Note that the output destination of the bitstream is not limited to a specific destination. For example, the integrated encoding unit 111 may output (send) the bitstream to an external device (e.g., a memory device or server device) via the output unit 112, or store the bitstream in the memory of the image encoding device. The control unit 150 controls the operation of the entire image encoding device including the aforementioned functional units.

[0024] Next, we will describe Figure 1 The operation of the image encoding device in the functional configuration shown. The inverse quantization correction control unit 114 acquires inverse quantization correction control information. The relationship between the value of the inverse quantization correction control information and the inverse quantization correction process will be described later.

[0025] The quantization matrix holding unit 103 acquires and holds multiple quantization matrices, and generates quantization matrices based on the size of the sub-blocks or the type of prediction method. This embodiment assumes that, as described above, the quantization matrix holding unit 103 generates quantization matrices that are consistent with the size of the sub-blocks or the type of prediction method. Figures 8A to 8C Each of the 8-pixel × 8-pixel sub-blocks shown in the diagram corresponds to a quantization matrix 800 with a size of 8 pixels × 8 pixels.

[0026] Figure 8A The quantization matrix 800 shown is an example of a quantization matrix corresponding to intra-frame prediction. Figure 8B The quantization matrix 800 shown is an example of a quantization matrix corresponding to inter-frame prediction. Figure 8C The quantization matrix 800 shown is an example of a quantization matrix corresponding to hybrid intra-frame / inter-frame prediction. For example... Figures 8A to 8C As shown, the quantization matrix consists of 8×8 elements (quantization step size values). This embodiment will describe the following situation: Figures 8A to 8CThe three types of quantization matrices shown are held as a two-dimensional array in quantization matrix holding unit 103, but the elements in the quantization matrix are not limited to these. Furthermore, depending on the size of the sub-block or whether the encoded object is a luma block or a chromaticity block, multiple quantization matrices can be held corresponding to the same prediction method. Generally, since the quantization matrix performs quantization processing corresponding to human visual characteristics, therefore... Figures 8A to 8C As shown, the DC component, which corresponds to the upper left part of the quantization matrix, uses small elements, while the AC component, which corresponds to the lower right part, uses large elements.

[0027] The generated quantization matrix is ​​not limited to this. For example, a quantization matrix corresponding to the shape of the sub-block can be generated, such as 4 pixels × 8 pixels, 8 pixels × 4 pixels, or 4 pixels × 4 pixels. There are no particular restrictions on the method for determining the elements in the quantization matrix. For example, predetermined initial values ​​can be used for the elements in the quantization matrix, or the elements in the quantization matrix can be set individually or generated according to the characteristics of the image.

[0028] The quantization matrix encoding unit 113 sequentially reads the quantization matrix, which is held as a two-dimensional array by the quantization matrix holding unit 103, calculates the differences by scanning each element in the quantization matrix, and arranges these differences in a one-dimensional matrix (difference matrix). In this embodiment, a method for... Figure 9 The sequence of scans indicated by the arrows in the image. Figures 8A to 8C The scanning method for each element in the quantization matrix 800 shown in the diagram calculates the difference between each element and the previous element in the scanning order. For example, by... Figure 9 The scanning method shown is used for scanning. Figure 8C The diagram shows a quantization matrix 800 with a size of 8 pixels × 8 pixels. After the first element "8" in the top left corner, the element "11" located directly below the first element is scanned, and the difference "+3" is calculated. Here, it is assumed that the calculation is performed with respect to the first element in the quantization matrix (in...). Figure 8C In the example shown, a predetermined initial value (e.g., "8") is used for the encoding of "8". However, the invention is by no means limited to this, and the difference with any value or the value of the first element can be used.

[0029] As described above, in this embodiment, regarding Figures 8A to 8C The quantization matrix 800 shown uses Figure 9 The scanning methods shown generate respectively Figures 10A to 10C The one-dimensional difference matrix 1000 is shown. The quantization matrix encoding unit 113 further encodes the difference matrix, thereby generating quantization matrix encoded data. In this embodiment, using... Figure 11A The encoding table shown is used for encoding. However, the encoding table is not limited to this, and for example, it can be used... Figure 11BThe encoding table shown.

[0030] Return to reference Figure 1 The integration coding unit 111 integrates the quantization matrix coding data with the header information (including inverse quantization correction control information) required for image coding, thereby generating header coding data. Subsequently, the image coding will be described.

[0031] The block segmentation unit 102 segments the input image received via the input unit 101 into multiple basic blocks. In this embodiment, the size of the basic block is 8 pixels × 8 pixels. The prediction unit 104 performs prediction processing on each basic block. More specifically, first, the prediction unit 104 determines a sub-block segmentation method as a way to segment the basic blocks into finer sub-blocks, and further determines a prediction mode such as intra-frame prediction, inter-frame prediction, or hybrid intra-inter-frame prediction on a sub-block basis.

[0032] Figures 7A to 7F An example of a sub-block splitting pattern is shown. Figures 7A to 7F The thick outer frame 700 represents a basic block, and in this embodiment, it has a size of 8 pixels × 8 pixels. The rectangles within the thick frame 700 represent sub-blocks. Figure 7A An example of a base block = sub-block is shown. Figure 7B This example shows a regular square sub-block division, where a basic block of 8 pixels × 8 pixels is divided into four sub-blocks, each of which is 4 pixels × 4 pixels. Figures 7C to 7F An example of rectangular sub-block segmentation is shown. Figure 7C In the diagram, the basic block is divided into two vertically elongated sub-blocks, each measuring 4 pixels × 8 pixels. Figure 7D In the image, the basic block is divided into two horizontally elongated rectangular sub-blocks, each measuring 8 pixels x 4 pixels. Figure 7E and Figure 7F In this process, the basic blocks are divided into rectangular sub-blocks in a 1:2:1 ratio. As mentioned above, both square and rectangular sub-blocks are used for encoding.

[0033] In this embodiment, for the sake of simplicity, a sub-block segmentation method is used where the 8-pixel × 8-pixel basic block is not divided into sub-blocks. Figure 7A However, it is possible to use, such as Figure 7B The quadtree partitioning shown is as follows: Figure 7E or Figure 7F The ternary tree partition shown, or as... Figure 7C or Figure 7D The binary tree partition shown. If using division... Figure 7A For sub-block partitions other than those shown, the quantization matrix holding unit 103 generates quantization matrices corresponding to each sub-block to be used. The generated quantization matrices are encoded by the quantization matrix encoding unit 113.

[0034] The prediction mode (prediction method) used in this embodiment will be described again. In this embodiment, three types of prediction methods are used (that is, intra-frame prediction, inter-frame prediction, and hybrid intra-frame-inter-frame prediction). In intra-frame prediction, the predicted pixels of the coded object block are generated using the coded pixels spatially located around the coded object block, and an intra-frame prediction mode representing an intra-frame prediction method such as horizontal prediction, vertical prediction, or DC prediction is also generated. In inter-frame prediction, the predicted pixels of the coded object block are generated using the coded pixels of frames that are temporally different from the coded object block, and motion information representing the frame to be referenced and motion vectors, etc., is also generated.

[0035] In hybrid intra-frame and inter-frame prediction, the coded object block is first segmented using diagonal line segments, thereby generating two regions. The pixel values ​​generated by the intra-frame prediction are used in one of the two regions, and the pixel values ​​generated by the inter-frame prediction are used in the other region, thereby generating the predicted pixels for the coded object block. Figure 12A and Figure 12B An example of region segmentation used in hybrid intra-frame-inter-frame prediction is shown. Figure 12A An example is shown where, from coded object block 1200, two regions are generated using a diagonal line from the top-left vertex to the bottom-right vertex. For example, pixel values ​​generated by intra-frame prediction can be used for the top-right region, and pixel values ​​generated by inter-frame prediction can be used for the bottom-left region. Figure 12B An example is shown where, from coded object block 1200, two regions are generated using line segments running diagonally from the top-right vertex to the midpoint between the top-left and bottom-left vertices. For example, pixel values ​​generated by intra-frame prediction can be placed in the top-left region, and pixel values ​​generated by inter-frame prediction can be placed in the bottom-right region. As described above, in hybrid intra-frame-inter-frame prediction, predicted pixels for the coded object block are generated, and intra-frame prediction modes, motion information, and region segmentation-related information used to generate the predicted pixels are also generated.

[0036] The prediction unit 104 generates a predicted image of the coded target sub-block based on the determined prediction mode and the encoded pixels. Then, the prediction unit 104 calculates the difference (error) between the coded target sub-block and its predicted image, thereby generating a prediction error. The prediction unit 104 also outputs prediction information, such as the sub-block segmentation method, prediction mode (indicating which of the following is used: intra-frame prediction, inter-frame prediction, or a hybrid intra-inter-frame prediction), and vector data.

[0037] The transform / quantization unit 105 generates quantization coefficients by performing an orthogonal transform and quantization on the prediction error. More specifically, the transform / quantization unit 105 generates orthogonal transform coefficients by performing an orthogonal transform process corresponding to the magnitude of the prediction error. Next, the transform / quantization unit 105 selects a quantization matrix corresponding to the prediction mode from the quantization matrix held by the quantization matrix holding unit 103, and uses the selected quantization matrix to quantize the orthogonal transform coefficients, thereby generating quantization coefficients. In this embodiment, the selected... Figure 8A The quantization matrix shown is used to quantize the orthogonal transform coefficients of the sub-blocks that have already undergone prediction processing using intra-frame prediction, and the quantization matrix is ​​selected. Figure 8B The quantization matrix shown is used to quantize the orthogonal transform coefficients of the sub-blocks that have already undergone inter-frame prediction. Furthermore, in this embodiment, the quantization matrix is ​​selected... Figure 8C The quantization matrices shown are used to quantize the orthogonal transform coefficients of sub-blocks that have undergone mixed intra- and inter-frame prediction. However, the quantization matrices that can be used are not limited to these.

[0038] The inverse quantization / inverse transform unit 106 generates reproducible orthogonal transform coefficients by inverse quantizing the quantization coefficients of the sub-block using the quantization matrix for quantizing the orthogonal transform coefficients of the sub-block stored in the quantization matrix holding unit 103. Then, the inverse quantization / inverse transform unit 106 performs inverse quantization correction processing on the reproducible orthogonal transform coefficients based on inverse quantization correction control information.

[0039] The inverse quantization correction process according to this embodiment will now be described. For example, the inverse quantization process and inverse quantization correction process according to this embodiment are performed using the following equation (1).

[0040] dz[x][y] = L[x][y] × Q[x][y] + Shift...(1) In equation (1), dz[x][y] represents the corrected reconstructed orthogonal transform coefficients corresponding to position (x, y), and L[x][y] represents the quantization coefficients corresponding to position (x, y). Additionally, Q[x][y] represents the quantization scale calculated considering the elements of the quantization matrix corresponding to position (x, y). Shift represents the correction value used in the inverse quantization correction process of this embodiment, and is determined based on the quantization coefficients L and the inverse quantization correction control information. More specifically, if the value of the inverse quantization correction control information is 0, the inverse quantization / inverse transform unit 106 sets the value of Shift in equation (1) to 0. In this case, the reconstructed orthogonal transform coefficients are essentially not corrected in the inverse quantization correction process. On the other hand, if the value of the inverse quantization correction control information is 1, the inverse quantization / inverse transform unit 106 derives the value of Shift in equation (1) using the following equation (2).

[0041] Shift = T × L[x][y]...(2) In equation (2), T represents a real number ranging from 0 (including the endpoints) to 1 (excluding the endpoints). This embodiment assumes that T is a fixed value, but is not limited to this, and can take a variable value depending on the position (x, y), or can be calculated using the quantization coefficient L[x][y]. For example, as indicated in the table below, the value of T can be calculated based on the absolute value of the quantization coefficient L[x][y] (|L[x][y]|).

[0042] [Table 1]

[0043]

[0044] In this case, if the quantization coefficient L[x][y] is 0, the value of T is also 0, and essentially no correction is performed. Furthermore, as the absolute value of the non-zero quantization coefficient L[x][y] (|L[x][y]|) increases, the value of T decreases. When |L[x][y]| becomes greater than a predetermined value, the value of T becomes 0, and essentially no correction is performed.

[0045] Then, the inverse quantization / inverse transform unit 106 generates (reproduces) the prediction error by performing an inverse orthogonal transform on the reproducible orthogonal transform coefficients generated using equation (1) above. In the inverse quantization process, as with transform / quantization unit 105, a quantization matrix corresponding to the prediction mode of the encoded object block is used. More specifically, the same quantization matrix as used by transform / quantization unit 105 is used.

[0046] The image reconstruction unit 107 generates (reconstructs) a predicted image based on the prediction information input from the prediction unit 104 by appropriately referencing the frame memory 108. Then, the image reconstruction unit 107 generates (reconstructs) a reconstructed image of the corresponding sub-block by adding the reconstructed predicted image to the prediction error generated (reconstructed) by the inverse quantization / inverse transform unit 106, and stores the generated reconstructed image in the frame memory 108.

[0047] The in-loop filtering unit 109 reads the reproduced image from the frame memory 108 and performs in-loop filtering on the read reproduced image using a filter such as a deblocking filter. Then, the in-loop filtering unit 109 stores the reproduced image with the in-loop filtering applied back into the frame memory 108.

[0048] The encoding unit 110 generates encoded data for each sub-block by entropy encoding the quantization coefficients of the sub-blocks generated by the transform / quantization unit 105 and the prediction information of the sub-blocks input from the prediction unit 104. The entropy encoding method is not limited to a specific method and can use Golomb coding, arithmetic coding, or Huffman coding, etc.

[0049] The integrated coding unit 111 generates a bit stream by multiplexing coded data and outputs the generated bit stream. Figure 6A An example of the data structure of the output bitstream according to this embodiment is shown. The sequence header includes inverse quantization correction control information and quantization matrix encoded data, and is formed by the encoded data of each element. However, the location to be encoded is not limited to this, and the data can be encoded in the image header or other headers. If the inverse quantization correction control information or quantization matrix is ​​to be changed in a sequence, it can be updated by re-encoding the inverse quantization correction control information or quantization matrix.

[0050] Next, we will refer to Figure 3 The flowchart describes the process performed by the image encoding device to encode an input image frame. If the image encoding device encodes multiple input images, the image encoding device performs the processing steps S305 to S312 on each input image frame.

[0051] First, before image encoding, in step S301, the inverse quantization correction control unit 114 acquires inverse quantization correction control information. In step S302, the quantization matrix holding unit 103 acquires and holds multiple quantization matrices to be used for quantization processing.

[0052] In step S303, the quantization matrix encoding unit 113 scans the quantization matrix generated in step S302 to calculate the differences between elements and generates a one-dimensional difference matrix. In step S304, the integration encoding unit 111 uses the header information required for encoding the image data (such as the inverse quantization correction control information obtained in step S301) and the quantization matrix encoding data generated in step S303 to generate header encoding data.

[0053] In step S305, the block segmentation unit 102 segments the input image input via the input unit 101 into multiple basic blocks. In step S306, the prediction unit 104 selects the unselected basic blocks from the basic blocks segmented in step S305 as selected basic blocks. Then, the prediction unit 104 segments the selected basic block into sub-blocks (including the case where the selected basic block = a sub-block), derives the prediction error of each sub-block, and outputs prediction information.

[0054] Note that a detailed example of the processing using prediction unit 104 is as follows. Prediction unit 104 performs intra-frame prediction processing on the sub-block of interest (block) to be encoded by referring to the encoded region of the same input image to which the sub-block of interest belongs, thereby generating an intra-frame prediction image. In addition, prediction unit 104 performs inter-frame prediction processing by referring to an encoded input image different from the input image to which the sub-block of interest belongs (e.g., the input image of the immediate preceding frame), thereby generating an inter-frame prediction image. Then, as shown in the example above in FIG12, prediction unit 104 divides the sub-block of interest into two regions, and arranges the intra-frame prediction image in one region and the inter-frame prediction image in the other region, thereby generating a hybrid intra-frame-inter-frame prediction image. Prediction unit 104 derives the sum of squares (or the sum of absolute values) of the differences between each of the three prediction images and the pixel values ​​corresponding to the position of the sub-block of interest, and determines the prediction mode of the prediction image with the smallest sum of squares as the prediction mode of the sub-block of interest. The prediction unit 104 generates a prediction image by performing prediction processing on the sub-blocks according to the prediction mode, and derives the difference between the sub-blocks and the prediction image as the prediction error.

[0055] In step S307, for each sub-block, the transform / quantization unit 105 performs an orthogonal transform on the prediction error derived in step S306, thereby generating orthogonal transform coefficients. Next, for each sub-block, the transform / quantization unit 105 selects one of the quantization matrices held by the quantization matrix holding unit 103 based on the prediction information, and uses the selected quantization matrix to quantize the orthogonal transform coefficients, thereby generating quantization coefficients.

[0056] In step S308, for each sub-block, the inverse quantization / inverse transform unit 106 performs inverse quantization on the quantization coefficients generated in step S307 using the quantization matrix selected in step S307 to generate reproducible quadrature transform coefficients. After performing inverse quantization correction processing based on inverse quantization correction control information, it performs inverse quadrature transformation on the reproducible quadrature transform coefficients, thereby generating (reproducible) prediction error. That is, the inverse quantization / inverse transform unit 106 performs inverse quantization processing and inverse quantization correction processing on the reproducible quadrature transform coefficients according to the above equation (1).

[0057] In step S309, for each sub-block, the image reproduction unit 107 generates a prediction image based on the prediction information output in step S306 through the reference frame memory 108, and uses the prediction image and the prediction error generated in step S308 to generate a reproduced image.

[0058] In step S310, for each sub-block, the encoding unit 110 generates encoded data for each sub-block of the basic block by encoding the prediction information output in step S306 and the quantization coefficients generated in step S307. Then, the integration encoding unit 111 generates a bitstream by reusing the header encoded data generated in step S304 and the encoded data generated by the encoding unit 110.

[0059] In step S311, the control unit 150 determines whether all basic blocks in the input image have been selected as selected basic blocks (that is, whether the encoding of all basic blocks (processing in steps S306 to S310) has been completed).

[0060] As a result of this determination, if all basic blocks in the input image have been selected as selected basic blocks, the process proceeds to step S312. If there are remaining basic blocks in the input image that have not yet been selected as selected basic blocks, the process returns to step S306.

[0061] In step S312, the in-loop filtering unit 109 reads the reproduced image from the frame memory 108, performs in-loop filtering on the reproduced image, and stores the reproduced image with the applied in-loop filtering in the frame memory 108 again.

[0062] Using the above configuration and operation, particularly in step S308, the reconstructed quadrature transform coefficients are corrected based on the inverse quantization correction control information. This allows for proper correction of the reconstructed quadrature transform coefficients generated by inverse quantization of the quantization matrix, and improves compression efficiency.

[0063] Note that in this embodiment, a quantization matrix is ​​used to generate reproducible orthogonal transform coefficients during the inverse quantization process of all sub-blocks in the frame, and the reproducible orthogonal transform coefficients are corrected based on inverse quantization correction control information. However, the present invention is not limited to this. For example, a quantization matrix can be used to inverse quantize sub-blocks in the frame (e.g., sub-blocks of size 8 pixels × 8 pixels), and other sub-blocks (e.g., sub-blocks of size 4 pixels × 4 pixels) can be inverse quantized without using a quantization matrix (i.e., by using the same quantization scale for all frequency components). In this case, the sub-blocks that have been inverse quantized using a quantization matrix can be corrected according to Equation (1) based on the inverse quantization correction control information, and the sub-blocks that have been inverse quantized without using a quantization matrix can always be corrected according to Equation (1) regardless of the inverse quantization correction control information. Therefore, even if sub-blocks that have been inverse quantized using a quantization matrix and sub-blocks that have been inverse quantized without using a quantization matrix are mixed, each sub-block can be appropriately corrected, thereby improving compression efficiency.

[0064] In this embodiment, the quantization correction control information is encoded and included in the bit stream, but the invention is not limited thereto. For example, by always setting the quantization correction control information to 0, such as... Figure 6B As shown, the code for quantization correction control information included in the bitstream can be omitted. In this case, sub-blocks that have undergone inverse quantization without using a quantization matrix are always subjected to inverse quantization correction, while sub-blocks that have undergone inverse quantization using a quantization matrix are never subjected to inverse quantization correction. This simplifies the relationship between whether to apply a quantization matrix and whether to apply inverse quantization correction for easier control, and reduces the amount of code corresponding to the inverse quantization correction control information.

[0065] In this embodiment, the inverse quantization correction control information only indicates whether inverse quantization correction processing is applied, but the value of the inverse quantization correction control information can also be set to the value of the parameter to be used for inverse quantization correction processing. For example, the value of the inverse quantization correction control information can also be set in Shift in equation (1) above or in the value of T in equation (2). This can control the intensity of inverse quantization correction according to the characteristics of the image, thereby improving compression efficiency.

[0066] In this embodiment, three types of prediction methods are used (i.e., intra-frame prediction, inter-frame prediction, and hybrid intra-inter-frame prediction). However, due to the different characteristics of prediction and error, it can be configured to perform different inverse quantization correction processes based on the prediction method. For example, inverse quantization correction control suitable for each prediction method can be applied by separately setting inverse quantization correction control information corresponding to sub-blocks using intra-frame prediction, inverse quantization correction control information corresponding to sub-blocks using inter-frame prediction, and inverse quantization correction control information corresponding to sub-blocks using hybrid intra-inter-frame prediction. In this case, it can be configured to encode each inverse quantization correction control information and include it in the bitstream, or to set each inverse quantization correction control information to a fixed value and omit encoding.

[0067] Note that in this embodiment, image encoding is performed on each frame to generate a bitstream and then output. However, the object of encoding is not limited to images. For example, features used in machine learning, such as object recognition, can be represented as two-dimensional array data, and this data can be set as the encoding object. This allows for efficient encoding of features used in machine learning.

[0068] [Second Embodiment] The image decoding device according to this embodiment decodes the bitstream of each frame generated by the image encoding device according to the first embodiment. (Refer to...) Figure 2 The block diagram describes an example of the functional configuration of the image decoding device according to this embodiment.

[0069] The separation decoding unit 202 acquires the bit stream via the input unit 201. The method used by the separation decoding unit 202 to acquire the bit stream is not limited to a specific method. For example, the separation decoding unit 202 can acquire the bit stream held in an external device such as a server device via a network, or acquire the bit stream generated by a camera device. Then, the separation decoding unit 202 separates the header encoded data and encoded data of each sub-block of the basic block from the bit stream. In short, the separation decoding unit 202 performs... Figure 1 The operation of the integrated coding unit 111 shown is the opposite of that of the integrated coding unit 111. Furthermore, the separate decoding unit 202 extracts inverse quantization correction control information from the header encoded data.

[0070] The quantization matrix decoding unit 209 decodes the header encoded data separated by the separation decoding unit 202 and reproduces the quantization matrix. The decoding unit 203 decodes the encoded data of each sub-block of the basic block separated by the separation decoding unit 202 and reproduces the quantization coefficients and prediction information.

[0071] The inverse quantization / inverse transform unit 204 uses the quantization matrix reproduced by the quantization matrix decoding unit 209 to inverse quantize the quantization coefficients reproduced by the decoding unit 203, as in the inverse quantization / inverse transform unit 106, thereby generating reproduced quadrature transform coefficients. Then, as in the inverse quantization / inverse transform unit 106, the inverse quantization / inverse transform unit 204 performs inverse quadrature transform on the reproduced quadrature transform coefficients after performing inverse quantization correction processing based on the inverse quantization correction control information extracted by the separate decoding unit 202, thereby generating (reproducing or deriving) the prediction error.

[0072] Similar to image reconstruction unit 107, image reconstruction unit 205 generates a predicted image based on the prediction information reconstructed by decoding unit 203 by appropriately referencing frame memory 206. Then, similar to image reconstruction unit 107, image reconstruction unit 205 generates a reconstructed image by adding the prediction error reconstructed by inverse quantization / inverse transform unit 204 to the predicted image, and stores the reconstructed image in frame memory 206.

[0073] Similar to the in-loop filtering unit 109, the in-loop filtering unit 207 reads the reproduced image from the frame memory 206, performs in-loop filtering on the reproduced image, and stores the reproduced image with the applied in-loop filtering in the frame memory 206 again. The reproduced image with the applied in-loop filtering by the in-loop filtering unit 207 is output to an external device via the output unit 208 under the control of the control unit 250.

[0074] The output destination of the reproduced image is not limited to a specific destination. For example, the control unit 250 can send the reproduced image to an external device via a network, or output the reproduced image to a display device connected to the image decoding device and display the reproduced image on the display device. The control unit 250 controls the operation of the entire image decoding device, including the aforementioned functional units.

[0075] Next, we will describe Figure 2 The operation of the image decoding device in the illustrated functional configuration. A bitstream of a frame input via input unit 201 is input to separation decoding unit 202. According to this embodiment, separation decoding unit 202... Figure 6A Inverse quantization correction control information is extracted from the sequence header of the bitstream shown, and from this sequence header, inverse quantization correction control information is extracted. Figures 8A to 8C The quantization matrix shown is encoded data. Additionally, the separation decoding unit 202 reproduces the encoded data of each sub-block of the basic block of the image data.

[0076] The quantization matrix decoding unit 209 decodes the encoded data of the quantization matrix, thereby reproducing the encoded data. Figures 10A to 10C The one-dimensional difference matrix is ​​shown. Similar to the first embodiment, this embodiment assumes the use of... Figure 11A (or Figure 11B Decoding is performed using the encoding table shown in the first embodiment, but the encoding table is not limited to this one, and other encoding tables can be used, as long as they use the same content as in the first embodiment. Then, the quantization matrix decoding unit 209 inversely scans the reconstructed one-dimensional difference matrix, thereby reconstructing the quantization matrix as a two-dimensional array. That is, the quantization matrix decoding unit 209 performs the opposite operation to that of the quantization matrix encoding unit 113. In other words, the quantization matrix decoding unit 209 uses... Figure 9 The scanning method shown will thus Figures 10A to 10C The difference matrices shown are reproduced as follows: Figures 8A to 8C The three types of quantization matrices are shown.

[0077] Decoding unit 203 decodes the encoded data of each sub-block of the basic block, thereby reproducing the quantization coefficients and prediction information. Inverse quantization / inverse transform unit 204 selects one of the quantization matrices reproduced by quantization matrix decoding unit 209, and uses the selected quantization matrix to inverse quantize the quantization coefficients reproduced by decoding unit 203, thereby generating reproduced quadrature transform coefficients. Then, as with inverse quantization / inverse transform unit 106, inverse quantization / inverse transform unit 204 performs inverse quantization correction processing based on inverse quantization correction control information extracted by separate decoding unit 202, and performs inverse quadrature transform on the generated reproduced quadrature transform coefficients, thereby generating (reproduced) prediction error.

[0078] According to this embodiment, the inverse quantization / inverse transform unit 204 determines the quantization matrix to be used in the inverse quantization process based on the prediction mode of the decoded target sub-block determined by the prediction information reproduced by the decoding unit 203. That is, intra-frame prediction is used to select the quantization matrix for the sub-block. Figure 8A The quantization matrix shown uses inter-frame prediction to select sub-blocks. Figure 8B The quantization matrix is ​​shown, and hybrid intra-frame / inter-frame prediction is used to select sub-blocks. Figure 8C The quantization matrix shown is not limited to these. However, the quantization matrix to be used is not limited to these, and the same quantization matrix used by the transform / quantization unit 105 and the inverse quantization / inverse transform unit 106 in the first embodiment is used.

[0079] Similar to image reconstruction unit 107, image reconstruction unit 205 generates a predicted image based on the prediction information reconstructed by decoding unit 203 by appropriately referencing frame memory 206. In this embodiment, like prediction unit 104, three types of prediction methods are used (i.e., intra-frame prediction, inter-frame prediction, and hybrid intra-inter-frame prediction). Then, like image reconstruction unit 107, image reconstruction unit 205 generates a reconstructed image by adding the prediction error reconstructed by inverse quantization / inverse transform unit 204 to the predicted image, and stores the reconstructed image in frame memory 206. The stored reconstructed image serves as a prediction reference candidate when decoding other sub-blocks.

[0080] Similar to the in-loop filtering unit 109, the in-loop filtering unit 207 performs in-loop filtering on the reproduced image stored in the frame memory 206. As described above, the reproduced image processed by the in-loop filtering unit 207 is output to an external device via the output unit 208.

[0081] Next, we will refer to Figure 4 The flowchart describes the process performed by the image decoding device to decode the bitstream of a single frame. When decoding the bitstream of multiple frames, the image decoding device performs processing based on the bitstream of each frame. Figure 4 The processing of flowcharts.

[0082] In step S401, the separation decoding unit 202 extracts (decodes) the inverse quantization correction control information from the bit stream, and reproduces (separates) the encoded data of the quantization matrix and the encoded data of each sub-block of the basic block from the bit stream.

[0083] In step S402, the quantization matrix decoding unit 209 reproduces the one-dimensional difference matrix by decoding the encoded data of the quantization matrix reproduced in step S401, and reproduces the quantization matrix as a two-dimensional array by inverse scanning the reproduced one-dimensional difference matrix.

[0084] In step S403, decoding unit 203 decodes the encoded data of each sub-block of the basic block reproduced in step S401, thereby reproducing the quantization coefficients and prediction information. In step S404, inverse quantization / inverse transform unit 204 selects one of the quantization matrices reproduced in step S402 and uses the selected quantization matrix to inverse quantize the quantization coefficients reproduced in step S403, thereby generating reproduced orthogonal transform coefficients. Then, after performing inverse quantization correction processing based on the inverse quantization correction control information extracted in step S401, inverse quantization / inverse transform unit 204 performs inverse orthogonal transform on the reproduced orthogonal transform coefficients, thereby generating (reproduced) prediction error.

[0085] In step S405, the image reconstruction unit 205 generates a predicted image based on the predicted information reconstructed in step S403 by appropriately referencing the frame memory 206. Then, the image reconstruction unit 205 generates a reconstructed image by adding the predicted error reconstructed in step S404 to the predicted image, and stores the reconstructed image in the frame memory 206.

[0086] In step S406, the control unit 250 determines whether steps S403 to S405 have been processed for all basic blocks. If, as a result of this determination, steps S403 to S405 have been processed for all basic blocks, the process proceeds to step S407. Otherwise, if there are remaining basic blocks that have not yet undergone steps S403 to S405, the process returns to step S403 to perform steps S403 to S405 on those basic blocks. In step S407, the in-loop filtering unit 207 performs in-loop filtering processing on the reproduced image stored in the frame memory 206.

[0087] Using the above configuration and operation, by correcting the reconstructed quadrature transform coefficients based on inverse quantization correction control information, the reconstructed quadrature transform coefficients generated by inverse quantization processing of the quantization matrix can be appropriately corrected, and the bit stream can be decoded with improved compression efficiency.

[0088] Note that, similarly in this embodiment, in the inverse quantization process of all sub-blocks in the frame, reconstructed orthogonal transform coefficients are generated using a quantization matrix, and the reconstructed orthogonal transform coefficients are corrected based on inverse quantization correction control information. However, the present invention is not limited thereto. For example, similarly in this embodiment, in the first embodiment, sub-blocks that have undergone inverse quantization using a quantization matrix can be corrected according to the inverse quantization correction control information using equation (1), and sub-blocks that have undergone inverse quantization without using a quantization matrix can be corrected using equation (1) regardless of the inverse quantization correction control information. Therefore, even if sub-blocks that have undergone inverse quantization using a quantization matrix and sub-blocks that have undergone inverse quantization without using a quantization matrix are mixed, each sub-block can be appropriately corrected, thereby decoding the bitstream with improved compression efficiency.

[0089] In this embodiment, the bitstream encoded with quantization correction control information is decoded, but the invention is not limited thereto. For example, the value of the quantization correction control information can always be set to 0, such as... Figure 6B As shown, a bitstream with omitted quantization correction control information can be decoded. In this case, similar to the first embodiment, sub-blocks that have undergone inverse quantization without using a quantization matrix are always subjected to inverse quantization correction, while sub-blocks that have undergone inverse quantization using a quantization matrix are never subjected to inverse quantization correction. This simplifies the relationship between whether to apply a quantization matrix and whether to apply inverse quantization correction for easier control, and allows decoding of a bitstream in which the amount of code corresponding to the inverse quantization correction control information is reduced.

[0090] Note that, also in this embodiment, similar to the first embodiment, the parameters used in the inverse quantization correction process can also be used as inverse quantization correction control information. This allows the intensity of inverse quantization correction to be controlled according to the characteristics of the image, thereby enabling decoding of the bitstream with improved compression efficiency.

[0091] Similarly, in this embodiment, as in the first embodiment, it can also be configured to perform different inverse quantization correction processes based on the prediction method. In this case, similar to the first embodiment, it can also be configured to decode the bitstream in which each inverse quantization correction control information is encoded, or to set each inverse quantization correction control information to a fixed value and omit decoding.

[0092] Note that in this embodiment, the bitstream of each frame is decoded, but the object of the decoding process is not limited to the bitstream obtained by encoding the image. For example, features used for machine learning such as object recognition can be represented as two-dimensional array data, and the bitstream generated by encoding the data can be decoded. This allows for the decoding of bitstreams generated by efficiently encoding features used for machine learning.

[0093] [Third Embodiment] The first embodiment assumes Figure 1 The functional units shown are implemented in hardware. However, functional units other than the frame memory 108 can be implemented in software (computer program). In this case, a computer device capable of executing software can be applied to an image encoding device.

[0094] The second embodiment assumes that Figure 2 The functional units shown are implemented in hardware. However, functional units other than frame memory 206 can be implemented in software (computer program). In this case, a computer device capable of executing software can be applied to an image decoding device.

[0095] Reference Figure 5 The block diagram describes an example of the hardware configuration of a computer device suitable for image encoding and image decoding devices. Note that... Figure 5 The configurations shown are merely examples of hardware configurations applicable to computer devices used for image encoding and image decoding, and can be appropriately changed / modified. Computer devices with different configurations can be applied to image encoding and image decoding devices. Alternatively, the image encoding and image decoding devices can be implemented using the same device.

[0096] CPU 501 uses computer programs and data stored in RAM 502 or ROM 503 to perform various processes. Therefore, CPU 501 controls the operation of the entire computer device and executes or controls various processes described as being performed by image encoding devices or image decoding devices.

[0097] RAM 502 has areas configured to store computer programs and data loaded from ROM 503 or storage device 506, and areas configured to store computer programs and data received from the outside via I / F 507. RAM 502 also has working areas used by CPU 501 when performing various processes. RAM 502 can thus be appropriately provided with various areas.

[0098] ROM 503 stores computer device configuration data, computer programs and data associated with the activation of the computer device, and computer programs and data associated with the basic operation of the computer device.

[0099] The operation unit 504 is a user interface such as a keyboard, mouse or touch panel screen, and the user can input various instructions and information to the computer device by operating the operation unit 504.

[0100] The display unit 505 includes a liquid crystal display or a touch panel display, and can display the processing results of the CPU 501 as images or characters. The display unit 505 can be a projection device such as a projector for projecting images or characters.

[0101] Storage device 506 is a non-volatile memory device such as a hard disk drive. Storage device 506 stores an OS (operating system), computer programs and data for causing CPU 501 to execute or control various processes described as being executed by an image encoding device or an image decoding device.

[0102] The computer program stored in storage device 506 includes functions for causing CPU 501 to execute or control what is described as being performed by... Figure 1 The computer programs for various processes executed by the functional units shown (excluding frame memory 108) are also included. Additionally, the computer programs stored in storage device 506 include those for causing CPU 501 to execute or control processes described by [the processor / system]. Figure 2 The functional units shown (excluding frame memory 206) execute various processing computer programs. Note that frame memories 108 and 206 can be implemented using RAM 502 and storage device 506.

[0103] I / F 507 is a communication interface used for data communication with external devices. For example, by communicating with a camera device or server device via I / F 507, a computer device can acquire an input image or bitstream from the camera device or server device and send the bitstream to the server device.

[0104] CPU 501, RAM 502, ROM 503, operation unit 504, display unit 505, storage device 506, and I / F 507 are all connected to the system bus 508. In this configuration, when the computer device is powered on, CPU 501 executes the boot program stored in ROM 503, loads the OS stored in storage device 506 into RAM 502, and activates the OS. As a result, the computer device can perform communication via I / F 507. Under the control of the OS, CPU 501 will handle applications associated with image encoding (corresponding to...) Figure 3 The application is loaded from storage device 506 into RAM 502 and executed, thereby turning the computer device into an image encoding device. On the other hand, when the CPU 501 loads the application associated with image decoding (corresponding to...) Figure 4 When the application is loaded from storage device 506 into RAM 502 and executed, the computer device is used as an image decoding device.

[0105] The numerical values, processing timing, processing order, main components of processing, and data (information) structure / acquisition method / sending destination / sending source / storage location used in the above embodiments are merely examples for detailed description and are not intended to be limited to these examples.

[0106] Some or all of the above embodiments can be appropriately combined and used. Alternatively, some or all of the above embodiments can be selectively used. (Other embodiments) This invention can be implemented by supplying a program for implementing one or more functions of the above embodiments to a system or device via a network or storage medium, and causing one or more processors in the computer of the system or device to read and execute the program. This invention can also be implemented by circuitry (e.g., an ASIC) for implementing one or more functions.

[0107] This invention is not limited to the embodiments described above, and various changes and modifications can be made within the spirit and scope of this invention. Therefore, the appended claims are made to inform the public of the scope of this invention.

[0108] This application claims priority to Japanese Patent Application 2023-168873, filed on September 28, 2023, which is incorporated herein by reference.

Claims

1. An image encoding device for encoding images in blocks, characterized in that, The image encoding device includes: A prediction unit is used to generate a prediction image from encoded pixels for a block of interest of a predetermined size in an image, which is an object of encoding, and to derive a prediction error as the difference between the block of interest and the prediction image. A transformation component is used to perform frequency transformation on the prediction error derived by the prediction component; A quantization unit is used to quantize the orthogonal transform coefficients obtained by frequency transformation using the transform unit; An encoding component is used to perform entropy encoding on the orthogonal transform coefficients quantized by the quantization component; and The inverse quantization unit is used to generate reproducible orthogonal transform coefficients by inverse quantizing the orthogonal transform coefficients quantized by the quantization unit. The inverse quantization component corrects the reproducible orthogonal transform coefficients.

2. The image encoding device according to claim 1, characterized in that, The inverse quantization component is controlled to correct the reconstructed orthogonal transform coefficients without using a quantization matrix for inverse quantization, and not to correct the reconstructed orthogonal transform coefficients when using a quantization matrix for inverse quantization.

3. The image encoding device according to claim 1 or 2, characterized in that, The encoding component further encodes the inverse quantization correction control information used to control the correction processing utilizing the inverse quantization component.

4. The image encoding device according to any one of claims 1 to 3, characterized in that, The inverse quantization component performs different corrections on the reconstructed orthogonal transform coefficients according to the prediction image generation method using the prediction component.

5. An image decoding device for decoding images in blocks, characterized in that, The image decoding device includes: The decoding unit is used to decode the quantized orthogonal transform coefficients; An inverse quantization unit is used to inverse quantize the quantized orthogonal transform coefficients to generate reproducible orthogonal transform coefficients; Inverse transform unit, used to perform inverse transform on the reconstructed orthogonal transform coefficients to derive the prediction error; and A prediction component is used to generate a prediction image and to decode the block of interest using the prediction image and the prediction error. The inverse quantization component corrects the reproducible orthogonal transform coefficients.

6. The image decoding device according to claim 5, characterized in that, The inverse quantization component is controlled to correct the reconstructed orthogonal transform coefficients without using a quantization matrix for inverse quantization, and not to correct the reconstructed orthogonal transform coefficients when using a quantization matrix for inverse quantization.

7. The image decoding device according to claim 5 or 6, characterized in that, The decoding unit further decodes the inverse quantization correction control information used to control the correction processing utilizing the inverse quantization unit.

8. The image decoding device according to any one of claims 5 to 7, characterized in that, The inverse quantization component performs different corrections on the reconstructed orthogonal transform coefficients according to the prediction image generation method using the prediction component.

9. An image encoding method, performed by an image encoding device, said image encoding device being used to encode an image in blocks, characterized in that, The image encoding method includes: The prediction step is used to generate a prediction image from encoded pixels for a block of interest of a predetermined size in the image that is the object of encoding, using the prediction component of the image encoding device, and to derive a prediction error as the difference between the block of interest and the prediction image. The transformation step is used to perform frequency transformation on the prediction error derived in the prediction step using the transformation component of the image encoding device. The quantization step is used to quantize the orthogonal transform coefficients obtained by the frequency transform in the transform step using the quantization component of the image encoding device. The encoding step is used to perform entropy encoding on the orthogonal transform coefficients quantized in the quantization step using the encoding component of the image encoding device; and The inverse quantization step is used to generate reproducible orthogonal transform coefficients by inverse quantizing the orthogonal transform coefficients obtained in the quantization step using the inverse quantization component of the image encoding device. In the inverse quantization step, the reconstructed orthogonal transform coefficients are corrected.

10. An image decoding method, performed by an image decoding device, the image decoding device being used to decode an image in blocks, characterized in that, The image decoding method includes: The decoding step is used to decode the quantized orthogonal transform coefficients using the decoding component of the image decoding device; The inverse quantization step is used to inverse quantize the quantized orthogonal transform coefficients using the inverse quantization component of the image decoding device to generate reproducible orthogonal transform coefficients. The inverse transform step is used to perform an inverse transform on the reconstructed orthogonal transform coefficients using the inverse transform component of the image decoding device to derive the prediction error; and The prediction step involves generating a prediction image using the prediction component of the image decoding device, and using the prediction image and the prediction error to decode the block of interest. In the inverse quantization step, the reconstructed orthogonal transform coefficients are corrected.

11. A computer program configured to cause a computer for an image encoding device that encodes images in blocks to function as a component of the image encoding device according to any one of claims 1 to 4.

12. A computer program configured to cause a computer for an image decoding device used for decoding images in blocks to function as a component of the image decoding device according to any one of claims 5 to 8.

Images