Image encoding device and method, image decoding device and method, and storage medium

Abstract

An image encoding device includes: a prediction unit, which is configured to generate a prediction error, wherein the prediction error is a difference between a predicted image obtained by a prediction process on an input image and the input image; a first transform unit, which is configured to generate a first transform coefficient by performing an orthogonal transform on the prediction error; a second transform unit, which is configured to generate a second transform coefficient by performing an LFNST process on the first transform coefficient; a quantization unit, which is configured to generate a quantized coefficient by performing a quantization process on the second transform coefficient; and an encoding unit, which is configured to encode the quantized coefficient, wherein the encoding unit encodes information indicating whether a range of possible values ​​that the second transform coefficient can at least take is a range determined based on a bit depth or a fixed range.

Description

Technical Field

[0001] The present invention relates to image coding. Background Art

[0002] As a coding scheme for compressed recording of moving images, a high efficiency video coding (HEVC) coding scheme (hereinafter referred to as HEVC) is known. In HEVC, multiple profiles for defining restrictions on coding techniques are defined, and the 10-bit main profile (Main 10profile) supports images with bit depths from 8 bits to 10 bits. In addition, in HEVC, profiles supporting 12-bit images and 16-bit images are also defined as images that also support higher bit depths. Japanese Patent Laid-Open No. 2014-131172 discusses a coding technique that supports an image coding scheme that supports such a high bit depth.

[0003] In recent years, international standardization of a more efficient coding scheme as a successor to HEVC has begun. ISO / IEC and ITU-T have established a Joint Video Experts Team (JVET), and standardization of the Versatile Video Coding (VVC) coding scheme (hereinafter referred to as VVC) is already underway. In VVC, in order to improve coding efficiency, a new technology is discussed that performs a secondary transform (LFNST, hereinafter referred to as a low-frequency transform) on the low-frequency components of the orthogonally transformed coefficients (hereinafter referred to as orthogonal transform coefficients) in addition to the traditional orthogonal transform.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Laid-Open No. 2014-131172 Summary of the invention

[0007] In VVC, in encoding processes such as orthogonal transform and quantization, restrictions are added to the values ​​of the processing results to increase the ease of implementation. Specifically, the possible values ​​of the coefficients that have been quantized (hereinafter referred to as quantization coefficients) are limited to -32768 to 32767, making it easier to implement the coefficient encoding process and the inverse quantization process on the decoding side. The following is considered: In VVC, even in the case of encoding high-bit depth images, the hardware implementation cost will not increase significantly. On the other hand, the above-mentioned restrictions lead to a reduction in the calculation accuracy (especially for high-bit depth images), resulting in problems such as no improvement in image quality.

[0008] Solutions for solving problems

[0009] In order to adaptively determine the range of possible values ​​of coefficient values ​​in the encoding or decoding process, the image encoding device according to the present invention includes, for example, the following configuration. That is, the image encoding device includes: a prediction unit configured to generate a prediction error, the prediction error being the difference between a predicted image obtained by a prediction process for an input image and the input image; a first transform unit configured to generate a first transform coefficient by performing an orthogonal transform on the prediction error; a second transform unit configured to generate a second transform coefficient by performing an LFNST process on the first transform coefficient; a quantization unit configured to generate a quantized coefficient by performing a quantization process on the second transform coefficient; and an encoding unit configured to encode the quantized coefficient, wherein the encoding unit encodes information indicating whether the range of possible values ​​that the second transform coefficient can take at least is a range determined based on the bit depth or a fixed range.

[0010] In addition, in order to solve the above-mentioned problem, the image decoding device according to the present invention includes, for example, the following configuration. That is, the image decoding device for decoding an image from an input bitstream includes: a decoding unit configured to decode a quantization coefficient from the bitstream; an inverse quantization unit configured to derive a first transform coefficient by inverse quantization processing the quantization coefficient; a first transform unit configured to derive a second transform coefficient by inverse LFNST processing the first transform coefficient; and a second transform unit configured to derive a prediction error by inverse orthogonal transform processing the second transform coefficient, wherein the decoding unit decodes information indicating whether the range of possible values ​​that the first transform coefficient can take at least is a range determined based on the bit depth or a fixed range from the bitstream.

[0011] Furthermore, in order to solve the above-mentioned problem, the image encoding device according to the present invention includes, for example, the following configuration. That is, the image encoding device includes: a prediction unit configured to generate a prediction error, the prediction error being a difference between a predicted image obtained by a prediction process for an input image and the input image; and an encoding unit configured to encode the prediction error using at least a BDPCM process, wherein the encoding unit encodes information indicating whether the range of a value obtained by at least the BDPCM process is a range determined based on a bit depth or a fixed range.

[0012] Furthermore, in order to solve the above-mentioned problem, the image decoding device according to the present invention includes, for example, the following configuration. That is, the image decoding device for decoding an image from an input bit stream includes: a decoding unit configured to decode a quantization coefficient from the bit stream; and an inverse quantization unit configured to derive a prediction error from the quantization coefficient using at least a BDPCM process, wherein the decoding unit decodes from the bit stream information indicating whether the range of values ​​obtained by at least the BDPCM process is a range determined based on a bit depth or a fixed range. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a block diagram illustrating the configuration of an image encoding device.

[0014] Figure 2 is a block diagram illustrating the configuration of an image decoding device.

[0015] Figure 3 This is a flowchart illustrating the image encoding process in the image encoding device.

[0016] Figure 4 This is a flowchart illustrating the image decoding process in the image decoding device.

[0017] Figure 5 is a diagram illustrating a computer hardware configuration.

[0018] Fig. 6A is a diagram illustrating an example of a bit stream structure generated by an encoding device and decoded by a decoding device.

[0019] Figure 6B is a diagram illustrating an example of a bit stream structure generated by an encoding device and decoded by a decoding device.

[0020] Fig. 7A is a diagram illustrating an example of sub-block partitioning.

[0021] Figure 7B is a diagram illustrating an example of sub-block partitioning.

[0022] Figure 7C is a diagram illustrating an example of sub-block partitioning.

[0023] Fig.7D is a diagram illustrating an example of sub-block partitioning.

[0024] Fig. 7E is a diagram illustrating an example of sub-block partitioning.

[0025] Figure 7F is a diagram illustrating an example of sub-block partitioning.

[0026] Figure 8is a diagram illustrating the relationship between coefficient range information, the bit depth of an image, and the possible range of coefficients. DETAILED DESCRIPTION

[0027] The exemplary embodiments will be described in detail below with reference to the accompanying drawings. The configurations described in the following exemplary embodiments are merely examples, and the present invention is not limited to the configurations shown in the drawings.

[0028] (First Exemplary Embodiment)

[0029] In the exemplary embodiment, a plurality of features are described, but all of these plurality of features may not be essential to the present invention, and a plurality of features may be freely combined. In addition, in the accompanying drawings, the same or similar configurations are assigned the same reference numerals, and description thereof will not be repeated.

[0030] Figure 1 1 is a block diagram illustrating the configuration of the image encoding device 100 according to the present exemplary embodiment. The image encoding device 100 includes a control unit 150 for controlling the entire device. The control unit 150 includes a CPU, a ROM for storing a program to be executed by the CPU, and a RAM to be used as a work area of ​​the CPU. The image encoding device 100 also includes an input terminal 101, a block division unit 102, a coefficient range information generation unit 103, a prediction unit 104, a transform and quantization unit 105, an inverse quantization and inverse transform unit 106, an image reconstruction unit 107, a frame memory 108, an in-loop filtering unit 109, an encoding unit 110, an integrated encoding unit 111, and an output terminal 112.

[0031] The input terminal 101 inputs the image data to be encoded frame by frame. The image data is obtained from a camera that captures the image, a file server or a storage medium that stores the image data to be encoded, and such a source may be of any type. The output terminal 112 outputs the encoded data to a destination device, and the destination device may also be of any type, such as a storage medium or a file server, and the like.

[0032] The block segmentation unit 102 segments the image of the input frame (picture) into a plurality of basic blocks, and sequentially outputs one of the blocks as a basic block to the prediction unit 104 in the subsequent stage. In addition, for example, a block of 128×128 pixels, a block of 64×64 pixels, or a block of 32×32 pixels can be used as a basic block. In addition, a smaller block can be used as a basic block. In addition, the basic block is, for example, a coding tree unit or a coding unit. In the case where the unit can be further divided into smaller sub-blocks, the basic block can be any unit.

[0033] The coefficient range information generating unit 103 generates coefficient range information for indicating the range of possible values ​​of coefficient values ​​as a result of the various encoding processes to be described below, and outputs the generated information to the transform and quantization unit 105, the inverse quantization and inverse transform unit 106 and the integrated encoding unit 111 in the subsequent stages.

[0034] The prediction unit 104 determines sub-block division for the image data in units of basic blocks. At this time, the prediction unit 104 determines whether to divide the basic block into sub-blocks, and if the basic block is to be divided, determines how to divide the basic block. In the case where the basic block is not divided into sub-blocks, the size of the sub-block is the same as the size of the basic block. The sub-block may be a square, and may be a rectangle other than a square.

[0035] Subsequently, the prediction unit 104 generates predicted image data in units of sub-blocks by performing intra-prediction as intra-frame prediction or inter-prediction as inter-frame prediction, etc. in units of the divided sub-blocks. At this time, for example, the prediction unit 104 selects a prediction method to be performed on a certain sub-block from intra-frame prediction, inter-frame prediction, and a combination of intra-frame prediction and inter-frame prediction, and performs the selected prediction to generate predicted image data for the sub-block.

[0036] In addition, the prediction unit 104 calculates a prediction error in pixels in units of sub-blocks based on the input image data and the predicted image data, and outputs the calculated prediction error. For example, the prediction unit 104 calculates the difference between each pixel value of the image data of the sub-block and each pixel value of the predicted image data generated by prediction of the sub-block, and determines the calculated difference as the prediction error.

[0037] In addition, the prediction unit 104 outputs information required for prediction (e.g., information indicating sub-block partitioning (a state in which a basic block is partitioned into sub-blocks) (i.e., information indicating how the basic block is partitioned into sub-blocks)) together with the prediction error. The prediction unit 104 also outputs information such as a prediction mode and a motion vector used in the prediction of the sub-block together with the prediction error. This information required for prediction will hereinafter be referred to as prediction information.

[0038] The transform and quantization unit 105 obtains orthogonal transform coefficients representing the respective frequency components of the prediction error by performing an orthogonal transform on the prediction error input from the prediction unit 104 on a sub-block-by-subblock basis. In addition, the transform and quantization unit 105 determines whether to perform a low-frequency transform on the low-frequency portion of the orthogonal transform coefficient, and generates information related to the determination as low-frequency transform information. In other words, the low-frequency transform information is information indicating whether to perform a low-frequency transform on the low-frequency portion of the orthogonal transform coefficient. In the case where a low-frequency transform is to be performed on a sub-block, the transform and quantization unit 105 obtains a low-frequency transform coefficient by performing a low-frequency transform process on the low-frequency portion of the obtained orthogonal transform coefficient, and further obtains a residual coefficient (quantized orthogonal transform coefficient) by quantizing the low-frequency transform coefficient. On the other hand, in the case where a low-frequency transform is not performed on a sub-block, the transform and quantization unit 105 obtains a residual coefficient by quantizing the orthogonal transform coefficient. The low-frequency transform indicates a low-frequency non-separable transform (LFNST) process, and this is a process for reconstructing (deriving) a low-frequency transform coefficient by transforming the orthogonal transform coefficient.

[0039] The inverse quantization and inverse transform unit 106 receives the residual coefficient and the low-frequency transform information from the transform and quantization unit 105 as input, and reconstructs the transform coefficient by inverse quantizing the residual coefficient. At this time, the inverse quantization and inverse transform unit 106 reconstructs the transform coefficient using the quantization matrix and the quantization parameter for the residual coefficient input from the transform and quantization unit 105. The process of reconstructing (deriving) the transform coefficient from the residual coefficient using the quantization matrix and the quantization parameter in this way will be called inverse quantization. It may not be necessary to use the quantization matrix in the inverse quantization process. In particular, in the case where the quantization matrix is ​​not used in the quantization process of the transform and quantization unit 105, the quantization matrix is ​​not used in the inverse quantization process of the inverse quantization and inverse transform unit 106. In addition, it can be determined whether to apply the quantization matrix based on whether the sub-block is subjected to a low-frequency transform. For example, in the case where the sub-block is subjected to a low-frequency transform, inverse quantization is performed without using the quantization matrix, otherwise inverse quantization is performed using the quantization matrix.

[0040] In addition, the inverse quantization and inverse transform unit 106 determines whether the sub-block is subjected to low-frequency transform based on the input low-frequency transform information. In the case where the sub-block is subjected to low-frequency transform, the inverse quantization and inverse transform unit 106 reconstructs the orthogonal transform coefficient by performing inverse low-frequency transform processing on the transform coefficient (low-frequency transform coefficient) obtained by the inverse quantization residual coefficient, and further reconstructs the prediction error by performing inverse orthogonal transform on the orthogonal transform coefficient. On the other hand, in the case where the sub-block has not been subjected to low-frequency transform, the inverse quantization and inverse transform unit 106 reconstructs the prediction error by performing inverse orthogonal transform on the transform coefficient (orthogonal transform coefficient) obtained by the inverse quantization residual coefficient. The inverse low-frequency transform processing indicates an inverse LFNST processing, and is a processing for reconstructing (deriving) the orthogonal transform coefficient by transforming the low-frequency transform coefficient.

[0041] The image reconstruction unit 107 appropriately refers to the frame memory 10 and generates predicted image data based on the prediction information output from the prediction unit 104. The image reconstruction unit 107 generates reconstructed image data (reconstructed picture) by adding the prediction error input from the inverse quantization and inverse transform unit 106 to the predicted image data, and stores the generated reconstructed image data in the frame memory 108.

[0042] The in-loop filter 109 performs an in-loop filtering process such as a deblocking filter or a sample adaptive offset on the reconstructed image stored in the frame memory 108 , and stores the image data after the filtering process in the frame memory 108 again.

[0043] The encoding unit 110 generates code data by encoding the residual coefficient and low-frequency transform information output from the transform and quantization unit 105 and the prediction information output from the prediction unit 104 , and outputs the generated code data to the integrated encoding unit 111 .

[0044] The integrated coding unit 111 generates a coefficient range information code by encoding the coefficient range information from the coefficient range information generating unit 103. Subsequently, header code data including the coefficient range information code is generated. Then, the integrated coding unit 111 forms a bit stream by making the code data output from the coding unit 110 follow the header code data. Then, the integrated coding unit 111 outputs the formed bit stream via the output terminal 112.

[0045] Here, the image encoding operation in the image encoding device will be described in detail below. In the present exemplary embodiment, a configuration in which 16-bit motion image data is input frame by frame from the input terminal 101 is adopted, but a configuration in which 1 frame of still image data is input may also be adopted. In addition, in the present exemplary embodiment, for the purpose of description, the block division unit 101 will be described as dividing the image data input from the input terminal into basic blocks of 8×8 pixels. The basic block used here is, for example, a coding tree unit. Although an example in which a coding tree unit has a size of 8×8 pixels will be described, other sizes may be adopted. For example, any size from a size of 32×32 pixels to a size of 128×128 pixels may be adopted.

[0046] Prior to image encoding, the range of possible values ​​of coefficients in the encoding process according to the present exemplary embodiment is determined.

[0047] The coefficient range information generating unit 103 generates coefficient range information indicating whether the range of possible values ​​of the coefficient in the encoding process is variable according to the bit depth of the input image, or whether the range is fixed regardless of the bit depth. Hereinafter, the former variable range of possible values ​​of the coefficient in the encoding process according to the bit depth of the image will be referred to as a high-precision coefficient range, and the latter fixed range regardless of the bit depth will be referred to as a fixed coefficient range. In the present exemplary embodiment, in the case of selecting the former high-precision coefficient range, the coefficient range information is 1, and in the case of selecting the latter fixed coefficient range, the coefficient range information is 0. However, the combination of the selected coefficient range and the coefficient range information is not limited thereto. In addition, the method of determining the coefficient range information is not particularly limited, and the method of determining the coefficient range information can be determined before the encoding process under the assumption that the application will be used by the present encoding device and the corresponding decoding device, or can be selected by the user. For example, under the assumption that the image encoding device 100 according to the present exemplary embodiment uses an application that gives the highest priority to image quality and attaches importance to operation accuracy, the coefficient range information is 1, otherwise the coefficient range information is 0. The generated coefficient range information is output to the integrated encoding unit 111 , the transform and quantization unit 105 , and the inverse quantization and inverse transform unit 106 .

[0048] The integrated coding unit 111 generates a coefficient range information code by encoding the coefficient range information input from the coefficient range information generating unit 103, and integrates the generated coefficient range information code into header information required for encoding of image data.

[0049] Subsequently, encoding of the image data is performed. Image data of one frame input from the input terminal 101 is supplied to the block division unit 102 .

[0050] The block division unit 102 divides the input image data of 1 frame into a plurality of basic blocks, and outputs the image data in units of basic blocks to the prediction unit 104. In the present exemplary embodiment, the image data in units of basic blocks of 8×8 pixels is supplied to the prediction unit 104.

[0051] The prediction unit 104 performs prediction processing on the image data in units of basic blocks input from the block division unit 102. Specifically, subblock division for dividing the basic block into smaller subblocks is determined, and a prediction mode such as intra prediction or inter prediction is further determined on a subblock-by-subblock basis. Intra prediction generates predicted pixels of the encoding object block using encoded pixels spatially located near the encoding object block, and also generates an intra prediction mode indicating an intra prediction method such as horizontal prediction, vertical prediction, or DC prediction. Inter prediction generates predicted pixels of the encoding object block using encoded pixels of a frame temporally different from the encoding object block, and also generates a frame to be referenced and motion information indicating a motion vector, etc.

[0052] Will refer to Figures 7A to 7F Describe the sub-block segmentation method. 7A to 7F The thick frame of each of the blocks 700 to 705 in FIG. 7 has a size of 8×8 pixels, which is the same as the size of the basic block. Each quadrilateral within the thick frame represents a sub-block. Figure 7B An example of conventional square sub-block partitioning is illustrated, and a basic block 701 of 8×8 pixels is partitioned into four sub-blocks of 4×4 pixels. FIG. 7C to FIG. 7F Each illustrates an example of rectangular sub-block partitioning. Figure 7C A basic block 702 is illustrated which is divided into two sub-blocks (the longer side in the vertical direction) each having a size of 4×8 pixels. Fig.7D A basic block 703 is illustrated which is divided into two sub-blocks (the longer side in the horizontal direction) each having a size of 8×4 pixels. Fig. 7E and Figure 7F The basic blocks 704 and 705 are each divided into three rectangular sub-blocks at a ratio of 1:2:1, but the division method is different. In this way, the encoding process is performed using not only square sub-blocks but also rectangular sub-blocks.

[0053] In the present exemplary embodiment, only the basic block having a size of 8×8 pixels is not divided into sub-blocks. Fig. 7A , but the sub-block segmentation method is not limited to this. Figure 7B The quadtree partition shown in Fig. 7E and Figure 7F The ternary tree partition shown or Figure 7C and Fig.7D The binary tree partition shown.

[0054] The prediction unit 104 generates predicted image data according to the determined prediction mode and the encoded region stored in the frame memory 108, further calculates a prediction error in units of pixels according to the input image data and the predicted image data, and outputs the calculated error to the transform and quantization unit 105. In addition, the prediction unit 104 outputs information on sub-block division and prediction mode, etc., as prediction information to the encoding unit 110 and the image reconstruction unit 107.

[0055] The transform and quantization unit 105 first receives the coefficient range information from the coefficient range generation unit 103 as input, and determines the range of possible values ​​of the coefficients in the transform process or the quantization process. Figure 8 The table shown determines the ranges that coefficients obtained as the results of each operation such as one-dimensional orthogonal transformation in each of the horizontal and vertical directions, secondary transformation for further transforming the coefficients that have undergone the orthogonal transformation, and quantization can take. However, the combination of coefficient range information and the ranges that coefficients obtained as the results of each operation can take are not limited to this. In the present exemplary embodiment, since the bit depth of the input image is 16, a range of -32768 to 32767 or a range of -8388608 to 8388607 is used according to the coefficient range information. The processing to be performed when the results of each operation fall outside the above range is not particularly limited, but the result can be adjusted to fall within the above range by limiting processing or bit shifting processing.

[0056] Next, the transform and quantization unit 105 generates a residual coefficient by orthogonally transforming and quantizing the prediction error input from the prediction unit 104 based on the above-mentioned coefficient range. Specifically, the transform and quantization unit 105 first generates an orthogonal transform coefficient by performing an orthogonal transform process corresponding to the size of the subblock on the prediction error. Next, the transform and quantization unit 105 determines whether to perform low-frequency transform on the low-frequency part of the orthogonal transform coefficient, generates information related to the determination as low-frequency transform information, and outputs the low-frequency transform information to the inverse quantization and inverse transform unit 106 and the encoding unit 110. There is no particular limitation on the method of determining the low-frequency transform information; however, in cases such as the case where the orthogonal transform coefficients are concentrated on the low-frequency components and further improvement in compression efficiency can be expected by performing low-frequency transform, low-frequency transform can be further performed. In the case where it is determined that low-frequency transform is to be performed, the transform and quantization unit 105 generates a low-frequency transform coefficient by performing low-frequency transform on the low-frequency part of the orthogonal transform coefficient, and on the other hand, regardless of the value of the orthogonal transform coefficient, the high-frequency component that has not yet been low-frequency transformed is processed to 0. Then, if low frequency transform is performed, transform and quantization unit 105 generates residual coefficients by quantizing low frequency transform coefficients. On the other hand, if low frequency transform is not performed, transform and quantization unit 105 generates residual coefficients by quantizing orthogonal transform coefficients.

[0057] Similar to the transform and quantization unit 105, the inverse quantization and inverse transform unit 106 first receives the coefficient range information from the coefficient range generation unit 103 as input, and determines the range of possible values ​​of the coefficients in the inverse quantization process or the inverse transform process. In the present exemplary embodiment, similar to the transform and quantization unit 105, based on Figure 8 The table shown determines the range that the coefficients obtained as the respective arithmetic results of the inverse quantization process, the inverse low-frequency transform process for the low-frequency component, and the one-dimensional inverse orthogonal transform in each of the horizontal direction and the vertical direction can adopt. In the present exemplary embodiment, similarly to the transform and quantization unit 105, since the bit depth of the input image is 16, the range of -32768 to 32767 or the range of -8388608 to 8388607 is adopted according to the coefficient range information.

[0058] Next, the inverse quantization and inverse transform unit 106 reconstructs the transform coefficient by inverse quantizing the residual coefficient input from the transform and quantization unit 105 based on the above-mentioned coefficient range. In addition, the inverse quantization and inverse transform unit 106 determines whether the sub-block is subjected to low-frequency transform based on the low-frequency transform information input from the transform and quantization unit 105. In the case where the sub-block is subjected to low-frequency transform, the inverse quantization and inverse transform unit 106 reconstructs the orthogonal transform coefficient by inverse low-frequency transform processing on the transform coefficient, and further reconstructs the prediction error by inverse orthogonal transforming the orthogonal transform coefficient. On the other hand, in the case where the sub-block has not been subjected to low-frequency transform, the inverse quantization and inverse transform unit 106 reconstructs the prediction error by inverse orthogonally transforming the transform coefficient. The prediction error thus reconstructed is output to the image reconstruction unit 107.

[0059] y=Clip3(CoeffMin,CoeffMax,((ΣlowFreqTransMatrix[j]×x)+64)>>7)(1)

[0060] (Wherein, Clip3(a,b,c) indicates a process of clipping the value c using the minimum value a and the maximum value b, and ">>" indicates a bit shift to the right.)

[0061] The above expression (1) is one of the calculation expressions used in the inverse low frequency transform process in the inverse quantization and inverse transform unit 106 according to the present exemplary embodiment. Here, CoeffMin in expression (1) is equal to Figure 8 The minimum coefficient value in corresponds to , and CoeffMax in expression (1) corresponds to Figure 8 In the present exemplary embodiment, since the bit depth of the input image is 16 bits, the values ​​of CoeffMin and CoeffMax in expression (1) are determined based on the coefficient range information. When the coefficient range information is 0 (i.e., a fixed coefficient range is selected), CoeffMin is -32768, and CoeffMax is 32767, and the output value can be represented by a signed 16-bit. In this case, a 16-bit addition / multiplication command or the like can be used in the subsequent encoding process, so there is an advantage of not increasing the implementation cost. On the other hand, when the coefficient range information is 1 (i.e., a high-precision coefficient range is selected), CoeffMin is -8388608, and CoeffMax is 8388607. In this case, the output value cannot be represented by a signed 16-bit, so the implementation cost in the subsequent encoding process increases, but a high-precision encoding process suitable for a high bit depth such as 16 bits can be implemented. As a result, there are advantageous effects such as an improvement in compression efficiency and an improvement in image quality when decoding the bit stream of the encoding result.

[0062] The image reconstruction unit 107 appropriately refers to the frame memory 108 and reconstructs the predicted image based on the prediction information input from the prediction unit 104. Subsequently, the image reconstruction unit 107 generates reconstructed image data based on the reconstructed predicted image and the prediction error reconstructed by the inverse quantization and inverse transform unit 106, and stores the generated reconstructed image data in the frame memory 108.

[0063] The in-loop filtering unit 109 reads out the reconstructed image data from the frame memory 108 , and performs an in-loop filtering process such as a deblocking filter, etc. Subsequently, the in-loop filtering unit 109 stores the image data subjected to the filtering process into the frame memory 108 again.

[0064] The encoding unit 110 generates code data by entropy encoding the residual coefficient and low frequency transform information in units of subblocks generated in the transform and quantization unit 105, and the prediction information input from the prediction unit 104. The entropy encoding method is not particularly specified, and Golomb encoding, arithmetic encoding, Huffman encoding, etc. can be used. The encoding unit 110 outputs the generated code data to the integrated encoding unit 111.

[0065] The integration coding unit 111 forms a bit stream by multiplexing the code data and the like input from the coding unit 110 together with the header code data. Then, the integration coding unit 111 outputs the formed bit stream from the output terminal 112 to the outside (to a storage medium or a network, etc.).

[0066] Fig. 6A is an example of a data structure of a bit stream output in the present exemplary embodiment. The coefficient range information code is included in the sequence header. However, the encoded position is not limited thereto and may include Figure 6B The picture header part shown or the header part composed of multiple pictures.

[0067] Figure 3 : is a flowchart illustrating an encoding process performed on 1 frame by the control unit 150 in the image encoding device 100 according to an exemplary embodiment.

[0068] First, before image encoding, in step S301 , the control unit 150 controls the coefficient range information generating unit 103 to generate coefficient range information.

[0069] In step S302 , the control unit 150 controls the integrated encoding unit 111 to encode the coefficient range information generated in step S301 to generate a coefficient range information code.

[0070] In step S303, the control unit 150 controls the integrated encoding unit 111 to encode the generated coefficient range information code and header information required for encoding the image data, and outputs the result.

[0071] In step S304 , the control unit 150 controls the block division unit 102 to divide the input image in units of frames into basic blocks.

[0072] In step S305, the control unit 150 controls the prediction unit 104 to perform prediction processing on the image data in units of basic blocks generated in step S304, and generates prediction information such as sub-block partition information and prediction mode and predicted image data. In addition, the control unit 150 controls the prediction unit 104 to calculate a prediction error based on the input image data and the predicted image data.

[0073] In step S306, the control unit 150 controls the transform and quantization unit 105 to determine the coefficient range in the encoding process of this step based on the coefficient range information generated in step S301. For example, the control unit 150 determines the coefficient range in the encoding process of this step as a high-precision coefficient range when the coefficient range information is 1, and determines the coefficient range in the encoding process of this step as a fixed coefficient range when the coefficient range information is 0. Next, the control unit 150 controls the transform and quantization unit 105 to perform an orthogonal transform on the prediction error calculated in step S305 to generate an orthogonal transform coefficient. Subsequently, the control unit 150 controls the transform and quantization unit 105 to determine whether to perform a low-frequency transform on the orthogonal transform coefficient generated in the sub-block, and generates information related to the determination as low-frequency transform information. In the case where it is determined that a low-frequency transform is to be performed, the control unit 150 controls the transform and quantization unit 105 to generate a residual coefficient by performing a low-frequency transform on the low-frequency component of the orthogonal transform coefficient and then quantizing it. On the other hand, in the case where it is determined that the low-frequency transform is not to be performed, the control unit 150 controls the transform and quantization unit 105 to generate a residual coefficient by quantizing the orthogonal transform coefficient.

[0074] In step S307, the control unit 150 controls the inverse quantization and inverse transform unit 106 to determine the coefficient range in the encoding process of this step based on the coefficient range information generated in step S301. For example, the control unit 150 determines the coefficient range in the encoding process of this step as a high-precision coefficient range when the coefficient range information is 1, and determines the coefficient range in the encoding process of this step as a fixed coefficient range when the coefficient range information is 0. Next, the control unit 150 controls the inverse quantization and inverse transform unit 106 to reconstruct the transform coefficient by inverse quantizing the residual coefficient generated in step S306. Next, the control unit 150 controls the inverse quantization and inverse transform unit 106 to determine whether the sub-block is low-frequency transformed based on the low-frequency transform information generated in step S306. In the case where the sub-block is low-frequency transformed, the control unit 150 controls the inverse quantization and inverse transform unit 106 to reconstruct the orthogonal transform coefficient by performing inverse low-frequency transform processing on the transform coefficient, and further reconstructs the prediction error by performing inverse orthogonal transform on the orthogonal transform coefficient. On the other hand, in the case where the sub-block has not been low-frequency transformed, the control unit 150 controls the inverse quantization and inverse transform unit 106 to reconstruct the prediction error by inversely orthogonally transforming the transform coefficients.

[0075] In step S308 , the control unit 150 controls the image reconstruction unit 107 to reconstruct the predicted image based on the prediction information generated in step S305 , reconstruct the image data according to the reconstructed predicted image and the prediction error generated in step S307 , and stores the reconstructed image data in the frame memory 108 .

[0076] In step S309, the control unit 150 controls the encoding unit 110 to encode the prediction information generated in step S305 and the residual coefficient and low-frequency transform information generated in step S306 to generate code data. In addition, the encoding unit 110 outputs the generated code data to the integrated encoding unit 111. The integrated encoding unit 111 places the encoded data from the encoding unit 110 after the previously generated header and outputs the result.

[0077] In step S310, the control unit 150 determines whether the encoding of all basic blocks in the focus frame is completed. If the control unit 150 determines that the encoding is completed, the process proceeds to step S311, and if the control unit 150 determines that there is a basic block that has not been encoded, the process returns to step S304 to continue encoding the next basic block.

[0078] In step S311 , the control unit 150 controls the in-loop filtering unit 109 to perform in-loop filtering processing on the image data reconstructed in step S308 and generate a filtered image, and the processing ends.

[0079] The above configuration and operation (particularly the encoding of the coefficient range information in step S302) make it possible to generate a bit stream capable of switching between encoding processes of different operation precision and implementation cost according to the required specifications of the application.

[0080] In the present exemplary embodiment, the coefficient range is described as a range of possible values ​​of the coefficient as a result of a transform process or a quantization process, but can be used as a range of possible values ​​of the coefficient as a result of other encoding processes. For example, in VVC, in order to improve compression efficiency mainly in lossless coding, a technique called Block-based Delta Pulse Code Modulation (BDPCM) is adopted. BDPCM is a scheme for generating quantized coefficients by quantizing the prediction error only without transforming it (instead of encoding the residual coefficient obtained by transforming and quantizing the prediction error), and encoding the differential value between the generated quantized coefficient and each quantized coefficient adjacent to the left or upper side. Here, the coefficient range can be applied to the differential value. Therefore, whether BDPCM is applied or not, it is determined whether the range of possible values ​​of the coefficient to be encoded is a high-precision coefficient range or a fixed coefficient range, and thus the implementation cost can be reduced by implementing a coding unit that supports only one of these ranges.

[0081] In addition, in the present exemplary embodiment, for example, when the bit depth of the input image is 8 bits, the coefficient range information code can be omitted. This is because the minimum coefficient value and the maximum coefficient value are equal regardless of the coefficient range information, so that redundant codes can be reduced. The bit depth of the input image for which the encoding of the coefficient range information can be omitted is not limited to 8 bits, and in other exemplary embodiments, when the minimum coefficient value and the maximum coefficient value are equal regardless of the coefficient range information, the encoding of the coefficient range information can be omitted.

[0082] Figure 2 : is a block diagram illustrating a configuration of an image decoding device 200 that decodes encoded image data generated in the image encoding device 100 in the exemplary embodiment. The configuration and operation related to the decoding process will be described below with reference to this drawing.

[0083] The image decoding device 200 includes a control unit 250 for controlling the entire device. The control unit 250 includes a CPU, a ROM for storing a program to be executed by the CPU, and a RAM to be used as a work area of ​​the CPU. In addition, the image decoding device 200 has an input terminal 201, a separation decoding unit 202, a decoding unit 203, an inverse quantization and inverse transformation unit 204, an image reconstruction unit 205, a frame memory 206, an in-loop filtering unit 207, an output terminal 208, and a coefficient range information decoding unit 209.

[0084] The input terminal 201 inputs an encoded bit stream, and the input source is, for example, a storage medium for storing the encoded stream; however, the encoded bit stream may be input from a network, and any type of source may be used.

[0085] The separation decoding unit 202 separates information related to the decoding process and code data related to the coefficient from the bit stream, and decodes the code data included in the header portion of the bit stream. The separation decoding unit 202 according to the present exemplary embodiment separates the coefficient range information code and outputs the coefficient range information code to the coefficient range information decoding unit 209. In addition, the separation decoding unit 202 outputs the code data of the image to the decoding unit 203.

[0086] The coefficient range information decoding unit 209 reconstructs the coefficient range information by decoding the coefficient range information code supplied from the separation decoding unit 202 , and outputs the reconstructed coefficient range information to the inverse quantization and inverse transform unit 204 in the subsequent stage.

[0087] The decoding unit 203 reconstructs the residual coefficient, the low-frequency transform information, and the prediction information by decoding the code data of the image output from the separation decoding unit 202 .

[0088] and Figure 1 Similarly to the inverse quantization and inverse transform unit 106 in the inverse quantization and inverse transform unit 204, the inverse quantization and inverse transform unit 204 inverse quantizes the residual coefficients and reconstructs the transform coefficients which are the inverse quantized coefficients. In addition, the inverse quantization and inverse transform unit 204 uses the low-frequency transform information to determine whether the sub-block has been subjected to low-frequency transform. In the case where the sub-block has been subjected to low-frequency transform, the inverse quantization and inverse transform unit 204 reconstructs the orthogonal transform coefficients by performing inverse low-frequency transform processing on the low-frequency transform coefficients. The inverse quantization and inverse transform unit 204 also reconstructs the prediction error by performing inverse orthogonal transform on these orthogonal transform coefficients or transform coefficients. On the other hand, in the case where the sub-block has not been subjected to low-frequency transform, the inverse quantization and inverse transform unit 204 reconstructs the prediction error by performing inverse orthogonal transform on the orthogonal transform coefficients.

[0089] The image reconstruction unit 205 appropriately refers to the frame memory 206 based on the input prediction information and generates predicted image data. Subsequently, the image reconstruction unit 205 generates reconstructed image data based on the predicted image data and the prediction error reconstructed in the inverse quantization and inverse transformation unit 204, and stores the generated reconstructed image data in the frame memory 206.

[0090] and Figure 1 Similarly, the in-loop filtering unit 109 in the frame memory 206 performs in-loop filtering processing such as deblocking filtering on the reconstructed image data stored in the frame memory 206, and stores the processed image data in the frame memory 206 again.

[0091] The output terminal 208 sequentially outputs the frame images stored in the frame memory 206 to the outside. The output destination is usually a display device, but may be another device.

[0092] The operation related to the image decoding of the image decoding device 200 of the above-described exemplary embodiment will be described in more detail. In the present exemplary embodiment, a configuration is adopted in which an encoded bit stream is input frame by frame.

[0093] exist Figure 2 2, a bit stream of one frame input from input terminal 201 is supplied to separation decoding unit 202. Separation decoding unit 202 separates information related to decoding processing and code data related to coefficients from the bit stream, and decodes the code data included in the header portion of the bit stream. Subsequently, separation decoding unit 202 supplies the coefficient range information code included in the header portion to coefficient range information decoding unit 209, and supplies the code data of the image data to decoding unit 203. Specifically, separation decoding unit 202 first separates the code data from the bit stream. Fig. 6A The coefficient range information code is extracted from the sequence header of the bit stream shown, and the extracted coefficient range information code is output to the coefficient range information decoding unit 209. Subsequently, the code data in units of basic blocks of the picture data is extracted and output to the decoding unit 203.

[0094] The coefficient range information decoding unit 209 obtains the coefficient range information by decoding the coefficient range information code input from the separation decoding unit 202. As with the encoding side, a high-precision coefficient range is used when the coefficient range information is 1, and a fixed coefficient range is used when the coefficient range information is 0. The coefficient range information is output to the inverse quantization and inverse transform unit 204. In the present exemplary embodiment, a configuration is adopted in which a 16-bit image is input on the encoding side and a 16-bit image is output on the decoding side, so a range of -32768 to 32767 is used when the coefficient range information is 0, and a range of -8388608 to 8388607 is used when the coefficient range information is 1.

[0095] The decoding unit 203 decodes the code data supplied from the separation decoding unit 202, reconstructs the prediction information, and further reconstructs the residual coefficient and the low-frequency transform information. First, the decoding unit 203 reconstructs the prediction information and obtains the prediction mode used in the sub-block. The decoding unit 203 outputs the reconstructed residual coefficient and the low-frequency transform information to the inverse quantization and inverse transform unit 204, and outputs the reconstructed prediction information to the image reconstruction unit 205.

[0096] The inverse quantization and inverse transform unit 204 generates a transform coefficient by inversely quantizing the input residual coefficient based on the above-mentioned coefficient range. Then, the inverse quantization and inverse transform unit 204 determines whether a low-frequency transform is performed on the sub-block based on the input low-frequency transform information. In the case where a low-frequency transform is performed on the sub-block, the inverse quantization and inverse transform unit 204 reconstructs the orthogonal transform coefficient by performing a low-frequency transform process on the transform coefficient, and further reconstructs the prediction error by performing an inverse orthogonal transform on the orthogonal transform coefficient. On the other hand, in the case where a low-frequency transform has not been performed on the sub-block, the inverse quantization and inverse transform unit 204 reconstructs the prediction error by performing an inverse orthogonal transform on the transform coefficient. The prediction error thus reconstructed is output to the image reconstruction unit 205.

[0097] The image reconstruction unit 205 appropriately refers to the frame memory 206 based on the prediction information input from the decoding unit 203 and reconstructs the predicted image. Like the prediction unit 104 on the encoding side, the image reconstruction unit 205 according to the present exemplary embodiment uses intra-frame prediction and / or inter-frame prediction. The specific prediction process is the same as the prediction process of the prediction unit 104 on the encoding side, so the description thereof will be omitted. The image reconstruction unit 205 reconstructs the image data based on the predicted image and the prediction error input from the inverse quantization and inverse transform unit 204, and stores the reconstructed image data in the frame memory 206. The stored image data is used for reference at the time of prediction.

[0098] Similar to the in-loop filtering unit 109 on the encoding side, the in-loop filtering unit 207 reads the reconstructed image from the frame memory 206 and performs in-loop filtering processing such as deblocking filtering, etc. Then, the in-loop filtering unit 207 stores the filtered image in the frame memory 206 again.

[0099] The reconstructed image stored in the frame memory 206 is finally output from the output terminal 208 to the outside (usually to a display device).

[0100] Figure 4 is a flowchart illustrating a decoding process of the control unit 205 in the image decoding device 200 according to an exemplary embodiment.

[0101] First, in step S401, the control unit 250 controls the separation decoding unit 202 to separate the information related to the decoding process and the code data related to the coefficient from the bit stream, and decodes the code data in the header part. More specifically, the separation decoding unit 202 supplies the coefficient range information code to the coefficient range information decoding unit 209, and supplies the code data of the image to the decoding unit 203.

[0102] In step S402, the control unit 250 controls the coefficient range information decoding unit 209 to decode the coefficient range information code reconstructed in step S401. A description of the specific operation of the coefficient range information decoding unit 209 has been provided here, and thus will be omitted.

[0103] In step S403 , the control unit 250 controls the decoding unit 203 to decode the code data separated in step S401 , reconstruct the prediction information, and reconstruct the residual coefficient and the low-frequency transform information.

[0104] In step S404, the control unit 250 controls the inverse quantization and inverse transform unit 204 to determine the coefficient range in the decoding process of this step based on the coefficient range information decoded in step S402. For example, the control unit 150 determines the coefficient range in the decoding process of this step as a high-precision coefficient range when the coefficient range information is 1, and determines the coefficient range in the decoding process of this step as a fixed coefficient range when the coefficient range information is 0. Next, the control unit 250 controls the inverse quantization and inverse transform unit 204 to generate a transform coefficient by inverse quantizing the residual coefficient reconstructed in step S403. In addition, the control unit 250 controls the inverse quantization and inverse transform unit 204 to determine whether a low-frequency transform is performed on the sub-block based on the low-frequency transform information reconstructed in step S403. In the case where a low-frequency transform is performed on the sub-block, the control unit 250 controls the inverse quantization and inverse transform unit 204 to reconstruct the orthogonal transform coefficient by performing an inverse low-frequency transform process on the transform coefficient, and further reconstructs the prediction error by performing an inverse orthogonal transform on the orthogonal transform coefficient. On the other hand, in the case where the sub-block has not been low-frequency transformed, the control unit 250 controls the inverse quantization and inverse transform unit 204 to reconstruct the prediction error by inversely orthogonally transforming the transform coefficients.

[0105] In step S405, the control unit 250 controls the image reconstruction unit 205 to reconstruct the image based on the prediction information generated in step S403. Specifically, the image reconstruction unit 205 refers to the frame memory 206 based on the prediction information and reconstructs the predicted image. At this time, as in step S305 on the encoding side, the image reconstruction unit 205 uses intra-frame prediction and / or inter-frame prediction. Subsequently, the image reconstruction unit 205 reconstructs the image data based on the reconstructed predicted image and the prediction error generated in S404, and stores the reconstructed image data in the frame memory 206.

[0106] In step S406, the control unit 250 determines whether decoding of all basic blocks in the focus frame is completed. In the case of decoding completion, the process proceeds to step S407, and in the case of uncoded basic blocks, the process returns to step S403 for the next basic block to be decoded.

[0107] In step S407 , the control unit 250 controls the in-loop filtering unit 207 to perform in-loop filtering processing on the image data reconstructed in step S405 and generate a filtered image, and the processing ends.

[0108] The above configuration and operation make it possible to decode the encoded bit stream generated in the above image encoding device 100 (ie, a bit stream that enables switching between decoding processes that differ in operation precision and implementation cost according to the required specifications of the application).

[0109] In this exemplary embodiment, the coefficient range is described as a range of possible values ​​of the coefficients obtained as a result of inverse quantization processing or inverse transform processing, but can be used as a range of possible values ​​of the coefficients obtained as a result of other decoding processing; for example, similarly to the encoding side, the coefficient range can be applicable to BDPCM processing.

[0110] dz[x][y]=Clip3(CoeffMin,CoeffMax,dz[x-1][y]+dz[x][y]) (2)

[0111] (Among them, Clip3(a, b, c) indicates a process of clipping the value c using the minimum value a and the maximum value b.)

[0112] The above expression (2) is one of the calculation formulas used in the decoding process using BDPCM, and is an expression in which the difference value is added to the left quantization coefficient dz[x-1][y] in the reconstruction of the quantization coefficient dz[x][y], and the coefficient range is used for clipping. Similar to expression (1), CoeffMin and CoeffMax of expression (2) are the same as Figure 8The minimum coefficient value and the maximum coefficient value in correspondence. When the coefficient range information is 0 (i.e., a fixed coefficient range is used), the quantization coefficient dz[x][y] output as expression (2) takes a range of -32768 to 32767 and can be represented by a signed 16-bit. In this case, 16-bit addition / multiplication commands and the like can be used in the inverse quantization process as a subsequent decoding process, so there is an advantage of reducing the implementation cost. On the other hand, when the coefficient range information is 1 (i.e., a high-precision coefficient range is selected), the implementation cost in subsequent decoding processes such as inverse quantization processes increases, but decoding processes with high operational precision can be implemented. As a result, a bit stream generated on the encoding side with improved compression efficiency can be decoded to have higher image quality.

[0113] The respective processing units of the image encoding device 100 and the image decoding device 200 according to the exemplary embodiment are described as being configured by hardware. However, the processing performed by each of these processing units shown in the drawings may be configured by a computer program.

[0114] Figure 5 : is a block diagram illustrating an example of a computer hardware configuration applicable to the image encoding device 100 and the image decoding device 200 according to the above-described exemplary embodiments.

[0115] The CPU 501 controls the entire computer using computer programs and data stored in the RAM 502 and the ROM 503, and executes the above-described respective processes to be performed by the image encoding device 100 or the image decoding device 200 according to the above-described exemplary embodiments. In other words, the CPU 501 functions as Figure 1 and Figure 2 The individual processing units are shown.

[0116] The RAM 502 has an area for temporarily storing data or the like acquired from an external storage device 506 or from the outside via an interface (I / F) 507. In addition, the RAM 502 is also used as a work area used when the CPU 501 performs various types of processing. For example, the RAM 502 may be allocated as a frame memory, or other various areas may be appropriately provided.

[0117] ROM 503 stores setting data and boot programs of the computer, etc. Operation unit 504 is composed of a keyboard and a mouse, etc., and the user of the computer can input various instructions to CPU 501 by operating operation unit 504. Display unit 505 displays the result of processing by CPU 501. Display unit 505 is composed of, for example, a liquid crystal display.

[0118] The external storage device 506 is a large-capacity information storage device represented by a hard disk drive. The external storage device 506 stores an operating system (OS) and a program for the CPU 501 to implement the Figure 1 and Figure 2 The computer program (application program) for implementing the functions of each unit shown in the figure. In addition, the external storage device 506 can store various image data to be processed.

[0119] Under the control of the CPU 501, the computer program and data stored in the external storage device 506 are appropriately loaded into the RAM 502 to be processed by the CPU 501. A network such as a LAN and the Internet and other devices such as a projection device and a display device can be connected to the I / F 507, and the computer can acquire and send various information via the I / F 507. 508 is a bus for connecting the above-mentioned units.

[0120] When the present device is turned on in the above configuration, the CPU 501 executes the boot program stored in the ROM 503, loads the OS stored in the external storage device 506 into the RAM 502, and executes the loaded OS. Subsequently, the CPU 501 loads the application program related to encoding or decoding from the external storage device 506 into the RAM 502 under the control of the OS, and executes the loaded application program. As a result, the CPU 501 functions as Figure 1 or Figure 2 The individual processing units in the image encoding device or the image decoding device are used.

[0121] (Other Exemplary Embodiments)

[0122] The present invention can also be implemented by supplying a program for implementing one or more functions of the above exemplary embodiments to a system or device via a network or storage medium, and causing one or more processors in a computer of the system or device to read and execute the program. The present invention can also be implemented by a circuit (e.g., ASIC) for implementing one or more functions.

[0123] According to the various exemplary embodiments described above, the range of possible values ​​of coefficient values ​​in an encoding or decoding process can be adaptively determined.

[0124] The present invention is not limited to the above exemplary embodiments, and various changes and modifications can be made within the spirit and scope of the present invention. Therefore, in order to make the scope of the present invention public, the following claims are attached.

[0125] This application claims the benefit of Japanese Patent Application No. 2020-048201, filed on Mar. 18, 2020, which is hereby incorporated by reference herein in its entirety.

Claims

1. An image encoding device, comprising: a prediction unit configured to generate a prediction error, the prediction error being a difference between a predicted image obtained by a prediction process for a sub-block in an input image and the sub-block; A first transform unit configured to generate a first transform coefficient by performing an orthogonal transform on the prediction error; A second transform unit configured to generate a second transform coefficient by performing a low frequency non-separable transform process (LFNST) on the first transform coefficient; a quantization unit configured to generate a quantized coefficient by performing a quantization process on the second transform coefficient; as well as an encoding unit configured to encode the quantized coefficients, The encoding unit encodes information indicating whether the range of possible values ​​of the second transform coefficient is at least a first range determined based on the bit depth or a second range that is a fixed range and not based on the bit depth, wherein, in the case where the information indicates that the range of possible values ​​that the second transform coefficient can take is at least the first range determined based on the bit depth, the range of possible values ​​that can be taken by a block-based delta pulse code modulation process, i.e., a BDPCM process, is the first range determined based on the bit depth, and Herein, when the information indicates that the range of possible values ​​that the second transform coefficient can take is at least the second range that is the fixed range and not based on the bit depth, the range of possible values ​​that can be taken by the BDPCM processing is the second range.

2. The image encoding device according to claim 1, wherein: Determine whether to use a quantization matrix, and When the LFNST process is applied, it is determined not to use the quantization matrix.

3. The image encoding device according to claim 1, wherein: The BDPCM process is a process of deriving a difference between mutually adjacent quantization coefficients among quantization coefficients generated by quantization for a prediction error, wherein the prediction error is a difference between a predicted image obtained by a prediction process for a given subblock in an input image and the given subblock.

4. The image encoding device according to claim 1, wherein: In case the bit depth is 8, the first range and the second range are the same.

5. An image decoding device for decoding an image from a bit stream, the image decoding device comprising: a decoding unit configured to decode quantized coefficients from the bitstream; an inverse quantization unit configured to derive a first transform coefficient by performing an inverse quantization process on the quantized coefficient; A first transform unit configured to derive second transform coefficients by performing an inverse low frequency non-separable transform process, i.e., an inverse LFNST process, on the first transform coefficients; as well as a second transform unit configured to derive a prediction error by performing an inverse orthogonal transform process on the second transform coefficients, The decoding unit decodes information from the bitstream indicating whether the range of possible values ​​of the first transform coefficient is at least a first range determined based on the bit depth or a second range that is a fixed range and not based on the bit depth, wherein, in the case where the information indicates that the range of possible values ​​that the first transform coefficient can take is at least the first range determined based on the bit depth, the range of possible values ​​that can be taken by a block-based delta pulse code modulation process, i.e., a BDPCM process, is the first range determined based on the bit depth, and Herein, when the information indicates that the range of possible values ​​that the first transform coefficient can take is at least the second range that is the fixed range and not based on the bit depth, the range of possible values ​​that can be taken by the BDPCM processing is the second range.

6. The image decoding device according to claim 5, wherein: Determine whether to use a quantization matrix, and However, when the LFNST process is applied, it is determined that the quantization matrix is ​​not used.

7. The image decoding device according to claim 5, wherein: The BDPCM process is a process of adding a difference value to a quantized coefficient on the left side of a given quantized coefficient to derive the given quantized coefficient.

8. The image decoding device according to claim 5, wherein: In case the bit depth is 8, the first range and the second range are the same.

9. An image encoding method, comprising: A prediction step for generating a prediction error, wherein the prediction error is a difference between a predicted image obtained by a prediction process for a sub-block in an input image and the sub-block; A first transform step, for generating a first transform coefficient by performing an orthogonal transform on the prediction error; A second transform step, for generating second transform coefficients by performing a low frequency non-separable transform process (LFNST) on the first transform coefficients; A quantization step, for generating a quantized coefficient by quantizing the second transform coefficient; as well as an encoding step for encoding the quantized coefficients, wherein, in the encoding step, information indicating whether the range of possible values ​​that the second transform coefficient can take is at least a first range determined based on the bit depth or a second range that is a fixed range and not based on the bit depth is encoded, wherein, in the case where the information indicates that the range of possible values ​​that the second transform coefficient can take is at least the first range determined based on the bit depth, the range of possible values ​​that can be taken by a block-based delta pulse code modulation process, i.e., a BDPCM process, is the first range determined based on the bit depth, and Herein, when the information indicates that the range of possible values ​​that the second transform coefficient can take is at least the second range that is the fixed range and not based on the bit depth, the range of possible values ​​that can be taken by the BDPCM processing is the second range.

10. An image decoding method for decoding an image from a bit stream, the image decoding method comprising: A decoding step for decoding quantized coefficients from the bit stream; An inverse quantization step, for deriving a first transform coefficient by inverse quantizing the quantized coefficient; A first transform step for deriving second transform coefficients by performing an inverse low frequency non-separable transform process, i.e., an inverse LFNST process, on the first transform coefficients; as well as a second transform step for deriving a prediction error by performing an inverse orthogonal transform process on the second transform coefficients, wherein, in the decoding step, information indicating whether the range of possible values ​​of the first transform coefficient is at least a first range determined by the bit depth or a second range that is a fixed range and not based on the bit depth is decoded from the bit stream, wherein, in the case where the information indicates that the range of possible values ​​that the first transform coefficient can take is at least the first range determined based on the bit depth, the range of possible values ​​that can be taken by a block-based delta pulse code modulation process, i.e., a BDPCM process, is the first range determined based on the bit depth, and Herein, when the information indicates that the range of possible values ​​that the first transform coefficient can take is at least the second range that is the fixed range and not based on the bit depth, the range of possible values ​​that can be taken by the BDPCM processing is the second range.

11. A non-transitory computer-readable storage medium storing a program for causing a computer to execute a method, the method comprising: A prediction step for generating a prediction error, wherein the prediction error is a difference between a predicted image obtained by a prediction process for a sub-block in an input image and the sub-block; A first transform step, for generating a first transform coefficient by performing an orthogonal transform on the prediction error; A second transform step, for generating second transform coefficients by performing a low frequency non-separable transform process (LFNST) on the first transform coefficients; A quantization step, for generating a quantized coefficient by quantizing the second transform coefficient; as well as an encoding step for encoding the quantized coefficients, wherein, in the encoding step, information indicating whether the range of possible values ​​that the second transform coefficient can take is at least a first range determined based on the bit depth or a second range that is a fixed range and not based on the bit depth is encoded, wherein, in the case where the information indicates that the range of possible values ​​that the second transform coefficient can take is at least the first range determined based on the bit depth, the range of possible values ​​that can be taken by a block-based delta pulse code modulation process, i.e., a BDPCM process, is the first range determined based on the bit depth, and Herein, when the information indicates that the range of possible values ​​that the second transform coefficient can take is at least the second range that is the fixed range and not based on the bit depth, the range of possible values ​​that can be taken by the BDPCM processing is the second range.

12. A non-transitory computer-readable storage medium storing a method for causing a computer to execute a method for decoding an image from a bit stream, the method comprising: A decoding step for decoding quantized coefficients from the bit stream; An inverse quantization step, for deriving a first transform coefficient by inverse quantizing the quantized coefficient; A first transform step for deriving second transform coefficients by performing an inverse low frequency non-separable transform process, i.e., an inverse LFNST process, on the first transform coefficients; as well as a second transform step for deriving a prediction error by performing an inverse orthogonal transform process on the second transform coefficients, wherein, in the decoding step, information indicating whether the range of possible values ​​of the first transform coefficient is at least a first range determined based on the bit depth or a second range that is a fixed range and not based on the bit depth is decoded from the bit stream, wherein, in the case where the information indicates that the range of possible values ​​that the first transform coefficient can take is at least the first range determined based on the bit depth, the range of possible values ​​that can be taken by a block-based delta pulse code modulation process, i.e., a BDPCM process, is the first range determined based on the bit depth, and Herein, when the information indicates that the range of possible values ​​that the first transform coefficient can take is at least the second range that is the fixed range and not based on the bit depth, the range of possible values ​​that can be taken by the BDPCM processing is the second range.

Images