Encoding / decoding devices and transmitting devices
By employing the Multiple Transform Selection (MTS) method and the low-complexity DST7 transform designed using Discrete Fourier Transform (DFT), the problems of low coding efficiency and high complexity in efficiently processing high spatial resolution and high frame rate video content are solved, achieving more efficient video coding.
Patent Information
- Application Number
- CN202311021721.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2018-08-08
- Filing Date
- 2019-08-08
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2039-08-08
AI Technical Summary
Existing technologies suffer from low coding efficiency and high complexity when processing video content with high spatial resolution, high frame rate, and high dimensionality, especially in terms of transformation operations, where efficient processing is difficult.
The Multiple Transform Selection (MTS) method is adopted. By designing a low-complexity Discrete Sine Transform 7 (DST7), and designing its forward and inverse DST7 as Discrete Fourier Transform (DFT), the transformation of DST7 with a length of 8, 16 or 32 is implemented using only integer operations, and the transformation is performed in combination with one-dimensional or two-dimensional DFT.
It reduces memory usage and operational complexity, improves encoding efficiency, and enhances video encoding performance by reducing the complexity of DST7 through the FFT algorithm.
Smart Images

Figure CN116866577B_ABST
Abstract
Description
[0001] This application is a divisional application of the original patent application No. 201980066113.2 (International Application No.: PCT / KR2019 / 009990, Application Date: August 8, 2019, Invention Title: Image Encoding / Decoding Method and Apparatus for Image Encoding / Decoding Method). Technical Field
[0002] This disclosure relates to methods and apparatus for encoding / decoding video, and more specifically, to techniques for performing transform / inverse transform based on Multiple Transform Selection (MTS). Background Technology
[0003] Next-generation video content will feature high spatial resolution, high frame rates, and high-dimensional scene rendering. To handle this content, technologies such as memory storage, memory access speeds, and processing power will need to be significantly increased.
[0004] Therefore, new encoding tools must be designed to process next-generation video content more efficiently. Specifically, when applying transforms, it is necessary to design more efficient transforms in terms of both encoding efficiency and complexity. Summary of the Invention
[0005] [Technical Issues]
[0006] This disclosure aims to propose a low-complexity operation algorithm for transform kernels used in video compression.
[0007] This disclosure aims to propose a method for designing a Discrete Sine Transform 7 (DST7) with low complexity.
[0008] This disclosure aims to propose a method for designing forward and inverse DST7 as discrete Fourier transforms (DFTs).
[0009] This disclosure aims to propose a method for implementing DST7 using one-dimensional DFT or two-dimensional DFT.
[0010] This disclosure aims to propose a method for implementing DST7 using only integer operations by applying various scaling methods.
[0011] This disclosure aims to propose methods for implementing DST7 using DFT and methods for implementing DST7 using only integer operations to design DST7 with lengths of 8, 16, or 32.
[0012] This disclosure aims to propose an encoder / decoder architecture for incorporating new transform designs.
[0013] This disclosure aims to present the overall transformation process according to the embodiments proposed in this disclosure.
[0014] [Technical Solution]
[0015] In one aspect of this disclosure, a method for decoding a video signal based on Multiple Transform Selection (MTS) may include: parsing a first syntax element indicating whether to apply the MTS to the inverse transform of a current block, wherein the MTS indicates a transform mode using a transform type other than a default transform type predefined in the current block; deriving an array of inverse-quantized transform coefficients having the width and height of the current block by performing inverse quantization on the current block; determining a vertical transform type applied to the vertical direction of the current block and a horizontal transform type applied to the horizontal direction of the current block based on the first syntax element; and deriving a residual sample array having the width and height of the current block by performing an inverse transform on the array of inverse-quantized transform coefficients using the vertical transform type and the horizontal transform type.
[0016] Preferably, the default transform type can be configured as DCT2, and the other transform types besides the default transform type can be configured as DST7 and DCT8.
[0017] Preferably, if the first syntax element indicates that the MTS is not applied to the inverse transform of the current block, then the vertical transform type and the horizontal transform type can be determined as DCT2. If the first syntax element indicates that the MTS is applied to the inverse transform of the current block, then each of the vertical transform type and the horizontal transform type can be determined as either DST7 or DCT8.
[0018] Preferably, the method further includes parsing a second syntax element indicating whether the MTS can be used in an intra-coded block and a third syntax element indicating whether the MTS can be used in an inter-coded block. When the second syntax element is 1, the first syntax element can exist in the transform unit syntax for the intra-coded block, and when the third syntax element is 1, the first syntax element can exist in the transform unit syntax for the inter-coded block.
[0019] Preferably, the step of deriving the residual sample array may include performing a one-dimensional transformation on each column of the inverse quantization transform coefficient array in the vertical direction using a vertical transform type, and performing a one-dimensional transformation on each row of the intermediate sample array output by the one-dimensional transformation on each column using a horizontal transform type.
[0020] Preferably, the step of performing one-dimensional transformation processing in the horizontal direction may further include cropping intermediate sample values output by the one-dimensional transformation processing for each column based on the minimum and maximum values of predefined coefficients.
[0021] In another aspect of this disclosure, an apparatus for decoding a video signal based on Multiple Transform Selection (MTS) may include: a syntax element parsing unit configured to parse a first syntax element indicating whether the MTS is applied to the inverse transform of a current block, wherein the MTS indicates a transform mode using a transform type other than a default transform type predefined in the current block; an inverse quantization transform coefficient derivation unit configured to derive an array of inverse quantization transform coefficients having the width and height of the current block by performing inverse quantization on the current block; a transform type determination unit configured to determine, based on the first syntax element, a vertical transform type applied to the current block in the vertical direction and a horizontal transform type applied to the current block in the horizontal direction; and a residual sample derivation unit configured to derive an array of residual samples having the width and height of the current block by performing an inverse transform on the array of inverse quantization transform coefficients using the vertical transform type and the horizontal transform type.
[0022] Preferably, the default transform type can be configured as DCT2, and the other transform types besides the default transform type can be configured as DST7 and DCT8.
[0023] Preferably, if the first syntax element indicates that the MTS is not applied to the inverse transform of the current block, then the vertical transform type and the horizontal transform type can be determined as DCT2. If the first syntax element indicates that the MTS is applied to the inverse transform of the current block, then each of the vertical transform type and the horizontal transform type can be determined as either DST7 or DCT8.
[0024] Preferably, the syntax element parsing unit can be configured to parse a second syntax element indicating whether the MTS can be used in an intra-coded block and a third syntax element indicating whether the MTS can be used in an inter-coded block. When the second syntax element is 1, the first syntax element can exist in the transform unit syntax for the intra-coded block, and when the third syntax element is 1, the first syntax element can exist in the transform unit syntax for the inter-coded block.
[0025] Preferably, the residual sample derivation unit can be configured to perform a one-dimensional transformation on each column of the inverse quantization transform coefficient array in the vertical direction using a vertical transform type, and to perform a one-dimensional transformation on each row of the intermediate sample array output by the one-dimensional transformation on each column using a horizontal transform type.
[0026] Preferably, the residual sample derivation unit can be configured to prune intermediate sample values output by a one-dimensional transformation process for each column based on the minimum and maximum values of predefined coefficients.
[0027] [Beneficial Effects]
[0028] This disclosure can reduce memory usage and operational complexity by providing a method for designing a Discrete Sine Transform 7 (DST7) with low complexity.
[0029] Furthermore, this disclosure can reduce the complexity of DST7 by designing the forward and inverse DST7 as DFTs when encoding still or moving images using an FFT algorithm.
[0030] As mentioned above, by using new operation algorithms with low complexity, the complexity of operations can be reduced and the coding efficiency can be improved. Attached Figure Description
[0031] Figure 1 This is a block diagram illustrating the configuration of an encoder for encoding video signals according to an embodiment of the present disclosure.
[0032] Figure 2 This is a block diagram illustrating the configuration of a decoder for decoding video signals according to an embodiment of the present disclosure.
[0033] Figures 3A to 3D Examples of embodiments to which this disclosure can be applied are illustrated. Figure 3A This is a diagram used to describe a block partitioning structure based on a quadtree (hereinafter referred to as "QT"). Figure 3B This is a diagram used to describe a block partitioning structure based on a binary tree (hereinafter referred to as "BT"). Figure 3C This is a diagram used to describe a block partitioning structure based on a ternary tree (hereinafter referred to as "TT"), and Figure 3D This is a diagram used to describe a block partitioning structure based on an asymmetric tree (hereinafter referred to as "AT").
[0034] Figure 4 This is an embodiment of the present disclosure, and a schematic block diagram of the transformation and quantization unit 120 / 130 and the inverse quantization and inverse transformation unit 140 / 150 in the encoder is illustrated.
[0035] Figure 5 This is an embodiment of the present disclosure, and a schematic block diagram of the inverse quantization and inverse transform unit 220 / 230 in the decoder is illustrated.
[0036] Figure 6 This is a table showing the transformation configuration groups of the application multiple transformation selection (MTS) as an embodiment of the present disclosure.
[0037] Figure 7This is a flowchart illustrating the encoding process of performing Multiple Transform Selection (MTS) as an embodiment of the present disclosure.
[0038] Figure 8 This is a flowchart illustrating the decoding process of performing Multiple Transform Selection (MTS) as an embodiment of the present disclosure.
[0039] Figure 9 This is a flowchart describing the process of encoding MTS flags and MTS indexes as an implementation of this disclosure.
[0040] Figure 10 This is a flowchart describing a decoding process that applies horizontal or vertical transformations to rows or columns based on MTS flags and MTS indexes as an implementation of this disclosure.
[0041] Figure 11 This is a flowchart illustrating the inverse transformation based on transformation-related parameters as an embodiment of the present disclosure.
[0042] Figure 12 This is a table showing the assignment of transform sets for each intra-prediction mode in NSST as an implementation of the present disclosure.
[0043] Figure 13 This is a flowchart illustrating the calculation of Givens rotation as an embodiment of the present disclosure.
[0044] Figure 14 An example of a round configuration of a 4×4 NSST consisting of Givens rotating layers and permutations is illustrated as an embodiment of applying this disclosure.
[0045] Figure 15 This describes an implementation of the present disclosure and illustrates a flowchart in which a positive DST7 with a length of 16 is designed using the Discrete Fourier Transform (DFT).
[0046] Figure 16 This describes an implementation of the present disclosure and illustrates a flowchart in which a reverse DST7 with a length of 16 is designed using the Discrete Fourier Transform (DFT).
[0047] Figures 17 to 19 This describes the application of embodiments of this disclosure, and illustrates the application therein. Figure 15 and Figure 16 The flowchart of the xDST7_FFT_B16 function.
[0048] Figure 20 This describes an implementation of the present disclosure and illustrates a flowchart in which a forward DST7 with a length of 32 is designed using the Discrete Fourier Transform (DFT).
[0049] Figure 21 This describes an implementation of the present disclosure and illustrates a flowchart in which a forward DST7 with a length of 32 is designed using the Discrete Fourier Transform (DFT).
[0050] Figures 22 to 24 This describes the application of embodiments of this disclosure, and illustrates the application therein. Figure 15 and Figure 16 The flowchart of the xDST7_FFT_B16 function.
[0051] Figure 25 This describes an implementation of the present disclosure and illustrates a flowchart in which a positive DST7 with a length of 8 is designed using the Discrete Fourier Transform (DFT).
[0052] Figure 26 This describes an implementation of the present disclosure and illustrates a flowchart in which a reverse DST7 with a length of 8 is designed using the Discrete Fourier Transform (DFT).
[0053] Figure 27 This is an implementation of the present disclosure, and a block diagram of a 16×16DST7 with 33-point DFT applied is illustrated.
[0054] Figure 28 This is an implementation of the present disclosure, and a block diagram of a 32×32DST7 with 65-point DFT applied is illustrated.
[0055] Figure 29 This describes an implementation of the present disclosure, and illustrates an encoding flowchart in which the forward discrete sine transform 7 (DST7) and the forward discrete cosine transform 8 (DCT8) are performed as a discrete Fourier transform (DTF).
[0056] Figure 30 This describes an implementation of the present disclosure, and illustrates a decoding flowchart in which the inverse discrete sine transform 7 (DST7) and the inverse discrete cosine transform 8 (DCT8) are performed as a discrete Fourier transform (DTF).
[0057] Figure 31 This is a flowchart illustrating a method for decoding a video signal based on Multiple Transform Selection (MTS) according to an embodiment of the present disclosure.
[0058] Figure 32 This is a diagram illustrating an apparatus for decoding video signals based on Multiple Transform Selection (MTS) according to an embodiment of the present disclosure.
[0059] Figure 33 An example of a video coding system using this disclosure is provided.
[0060] Figure 34 This describes an implementation of the present disclosure and illustrates the structure of a content streaming system. Detailed Implementation
[0061] In the following, the configuration and operation of embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. The configuration and operation of the present disclosure described with reference to the accompanying drawings are described as embodiments, and the scope, core configuration and operation of the present disclosure are not limited thereto.
[0062] Furthermore, the terminology used in this disclosure is selected from currently widely used general terms, but in certain cases, terms randomly selected by the applicant are used. In such cases, since the meaning of the terms is clearly described in the detailed description of the corresponding sections, the terms should not be interpreted solely by their names as used in the description of this disclosure, and the meaning of the corresponding terms should be understood and interpreted.
[0063] Furthermore, when common terms chosen for describing this disclosure or other terms with similar meanings exist, the terms used in this disclosure may be replaced with more appropriate interpretations. For example, in each encoding process, signals, data, samples, images, frames, and blocks may be appropriately replaced and interpreted. Additionally, in each encoding process, segmentation, decomposition, separation, and partitioning may be appropriately replaced and interpreted.
[0064] In this disclosure, multiple transform selection (MTS) can refer to a method for performing a transform using at least two transform types. This can also be represented as adaptive multiple transform (AMT) or explicit multiple transform (EMT). Similarly, mts_idx can be represented as AMT_idx, EMT_idx, tu_mts_idx, AMT_TU_idx, EMT_TU_idx, transform index, transform combination index, and this disclosure is not limited to these representations.
[0065] Figure 1 This is a schematic block diagram of an encoder that performs video signal encoding as an embodiment of the present disclosure.
[0066] Reference Figure 1 The encoder 100 can be configured to include an image segmentation unit 110, a transform unit 120, a quantization unit 130, an inverse quantization unit 140, an inverse transform unit 150, a filtering unit 160, a decoded image buffer (DPB) 170, an inter-frame prediction unit 180, an intra-frame prediction unit 185, and an entropy coding unit 190.
[0067] The image segmentation unit 110 divides the input image (or picture or frame) input to the encoder 100 into one or more processing units. For example, the processing unit may be a coding tree unit (CTU), a coding unit (CU), a prediction unit (PU), or a transform unit (TU).
[0068] However, these terms are for convenience of description only, and the disclosure is not limited to the definitions of these terms. Furthermore, in this disclosure, for convenience of description, the term "encoding unit" is used as a unit used in encoding or decoding video signals, but the disclosure is not limited thereto and can be appropriately interpreted according to this disclosure.
[0069] The encoder 100 subtracts the prediction signal (or prediction block) output from the inter-frame prediction unit 180 or the intra-frame prediction unit 185 from the input image signal to generate a residual signal (or residual block), and the generated residual signal is sent to the transform unit 120.
[0070] Transform unit 120 can generate transform coefficients by applying transform techniques to the residual signal. Transform processing can be applied to quadtree-structured square blocks and blocks (squares or rectangles) divided according to binary, ternary, or asymmetric tree structures.
[0071] Transformer 120 can perform transformations based on multiple transformations (or combinations of transformations), and the transformation scheme can be referred to as Multiple Transform Selection (MTS). MTS can also be referred to as Adaptive Multiple Transform (AMT) or Enhanced Multiple Transform (EMT).
[0072] MTS (or AMT or EMT) can refer to a transformation scheme that is performed based on an adaptive selection of a transformation (or combination of transformations) from a plurality of transformations (or combinations of transformations).
[0073] The plurality of transformations (or combinations of transformations) may include those referenced in this disclosure. Figure 6 The transformation (or combination of transformations) described. In this disclosure, the transformation or transformation type may be written, for example, as DCT type 2, DCT-II, DCT2, or DCT-2.
[0074] The transformation unit 120 can perform the following implementation.
[0075] This disclosure provides a method for designing forward and inverse DST7 as discrete Fourier transforms (DFTs).
[0076] The transformation unit 120 can implement DST7 through one-dimensional DFT or two-dimensional DFT.
[0077] Furthermore, the transformation unit 120 can implement DST7 using only integer operations by applying various scaling methods.
[0078] Furthermore, the transform unit 120 can design a DST7 with a length of 8, 16, or 32 by using the DFT method to implement the DST7 and by using only integer operations to implement the DST7.
[0079] The detailed implementation thereof will be described in more detail in this disclosure.
[0080] The quantization unit 130 can quantize the transform coefficients and send the quantized transform coefficients to the entropy coding unit 190. The entropy coding unit 190 can entropy code the quantized signal and output the entropy-coded quantized signal as a bit stream.
[0081] Although the transformation unit 120 and the quantization unit 130 are described as separate functional units, this disclosure is not limited thereto, and they can be combined into a single functional unit. Similarly, the inverse quantization unit 140 and the inverse transformation unit 150 can also be combined into a single functional unit.
[0082] The quantized signal output from quantization unit 130 can be used to generate a prediction signal. For example, the residual signal is reconstructed by cyclically applying inverse quantization and inverse transform to the quantized signal through inverse quantization unit 140 and inverse transform unit 150. The reconstructed residual signal is added to the prediction signal output from inter-frame prediction unit 180 or intra-frame prediction unit 185 to generate the reconstructed signal.
[0083] Furthermore, due to quantization errors during this compression process, degradation that reveals block boundaries may occur. This phenomenon is known as blocking artifacts and is one of the key factors in evaluating image quality. To reduce this degradation, filtering can be performed. Filtering eliminates block degradation and reduces errors in the current image, thereby enhancing image quality.
[0084] The filtering unit 160 applies filtering to the reconstructed signal and outputs the applied reconstructed signal to the playback device, or sends the output reconstructed signal to the decoded image buffer 170. The inter-frame prediction unit 180 can use the filtered signal sent from the decoded image buffer 170 as a reference image. Thus, in inter-frame prediction mode, the filtered image is used as a reference image to enhance image quality and coding efficiency.
[0085] The decoded image buffer 170 can store the filtered image so that the filtered image can be used as a reference image in the inter-frame prediction unit 180.
[0086] Inter-frame prediction unit 180 performs temporal and / or spatial prediction to remove temporal and / or spatial redundancy by reconstructing a reference image. In this case, block artifacts or ringing artifacts may exist because the reference image used for prediction is a transformed signal that has been quantized and dequantized in blocks during encoding / decoding at a previous time.
[0087] Therefore, the inter-frame prediction unit 180 can interpolate the signal between pixels on a sub-pixel basis by applying a low-pass filter to address performance degradation caused by signal discontinuities or quantization. In this case, a sub-pixel refers to a virtual pixel generated by applying an interpolation filter, and an integer pixel refers to an actual pixel present in the reconstructed image. Linear interpolation, bilinear interpolation, Wiener filters, etc., can be used as interpolation methods.
[0088] Interpolation filters are applied to the reconstructed image to enhance prediction accuracy. For example, the inter-frame prediction unit 180 applies interpolation filters to integer pixels to generate interpolated pixels, and prediction can be performed by using interpolated blocks composed of interpolated pixels as prediction blocks.
[0089] Furthermore, the intra-prediction unit 185 can predict the current block by referencing samples near the block that will undergo the current encoding. The intra-prediction unit 185 can perform the following processes to perform intra-prediction: First, reference samples can be prepared, which are needed to generate the prediction signal. Additionally, the prediction signal can be generated using the prepared reference samples. Afterward, the prediction mode is encoded. In this case, reference samples can be prepared by zero-padding and / or filtering. Since the reference samples have undergone prediction and reconstruction processing, quantization errors may exist. Therefore, reference sample filtering can be performed for each prediction mode used for intra-prediction to reduce this error.
[0090] The prediction signal generated by the inter-frame prediction unit 180 or the intra-frame prediction unit 185 can be used to generate the reconstructed signal or the residual signal.
[0091] Figure 2 This is a schematic block diagram of a decoder that performs video signal decoding as an embodiment of the present disclosure.
[0092] Reference Figure 2 The decoder 200 can be configured to include a parsing unit (not shown), an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, a filtering unit 240, a decoded picture buffer (DPB) unit 250, an inter-frame prediction unit 260, and an intra-frame prediction unit 265.
[0093] In addition, the reconstructed video signal output by the decoder 200 can be reproduced by the reproduction device.
[0094] Decoder 200 can receive from Figure 1 The signal output by the encoder 100 can be entropy decoded by the entropy decoding unit 210.
[0095] The dequantization unit 220 obtains the transformation coefficients from the entropy-decoded signal by using quantization step size information.
[0096] The inverse transform unit 230 performs an inverse transform on the transform coefficients to obtain the residual signal.
[0097] In this context, the present disclosure provides a method for configuring transform combinations for each transform configuration group divided according to at least one of prediction mode, block size, or block shape, and the inverse transform unit 230 can perform an inverse transform based on the transform combinations configured by the present disclosure. Additionally, embodiments described in the present disclosure can be applied.
[0098] The inverse transformation unit 230 can perform the following implementation.
[0099] This disclosure provides a method for designing forward and inverse DST7 as discrete Fourier transforms (DFTs).
[0100] The inverse transform unit 230 can implement DST7 through one-dimensional DFT or two-dimensional DFT.
[0101] Furthermore, the inverse transformation unit 230 can implement DST7 using only integer operations by applying various scaling methods.
[0102] Furthermore, the inverse transform unit 230 can design a DST7 with a length of 8, 16, or 32 by using the DFT to implement the DST7 and by using only integer operations to implement the DST7.
[0103] In this implementation, the inverse transformation unit 230 can derive the transformation combination corresponding to the transformation index, and can perform an inverse transformation on the current block in the vertical or horizontal direction using DST7 or DCT8. In this case, the transformation combination can consist of a horizontal transformation and a vertical transformation, and the horizontal transformation and the vertical transformation can correspond to either DST7 or DCT8.
[0104] In an implementation, when applying a 33-point Discrete Fourier Transform (DFT) to a DST7, the steps may include dividing a row or column of the DST7 into two partial vector signals; and applying an 11-point DFT type 1 or an 11-point DFT type 2 to the two partial vector signals.
[0105] In an implementation, when a row or column of DST7 is represented as src[0…15], the two partial vector signals can be divided into src[0…4] and src[5…15].
[0106] In an implementation, when applying a 65-point Discrete Fourier Transform (DFT) to a DST7, the step may include dividing a row or column of the DST7 into three partial vector signals; and may include applying a 13-point DFT type 1 or a 13-point DFT type 2 to the three partial vector signals.
[0107] In the implementation, when a row or column of DST7 is represented as src[0…31], the three partial vector signals can be divided into src[0…5], src[6…18] and src[19…31].
[0108] In the implementation, among the three partial vector signals, a 13-point DFT type 1 can be applied to src[0…5], and a 13-point DFT type 2 can be applied to src[6…18] and src[19…31].
[0109] Furthermore, the inverse transform unit 230 can perform an inverse main transform on the transform coefficient block in the vertical direction using a vertical main transform, and can also perform an inverse main transform on the transform coefficient block in the horizontal direction using a horizontal main transform.
[0110] Furthermore, in this embodiment, a horizontal transformation is applied after a vertical transformation, but this disclosure is not limited to this. That is, a vertical transformation can be applied after a horizontal transformation.
[0111] In implementation, the combination of horizontal and vertical transformations can include... Figure 6 At least one of the implementation methods.
[0112] Although the inverse quantization unit 220 and the inverse transform unit 230 are described as separate functional units, this disclosure is not limited thereto, and they can be combined into a single functional unit.
[0113] The obtained residual signal is added to the prediction signal output from the inter-frame prediction unit 260 or the intra-frame prediction unit 265 to generate the reconstructed signal.
[0114] The filtering unit 240 applies filtering to the reconstructed signal and outputs the applied reconstructed signal to the generation device, or sends the output reconstructed signal to the decoding image buffer unit 250. The inter-frame prediction unit 260 can use the filtered signal sent from the decoding image buffer unit 250 as a reference image.
[0115] In this disclosure, the embodiments described in the corresponding functional units of the transformation unit 120 and the encoder 100 can be equivalently applied to the corresponding functional units of the inverse transformation unit 230 and the decoder, respectively.
[0116] Figures 3A to 3D Examples of embodiments to which this disclosure can be applied are illustrated. Figure 3A This is a diagram used to describe a block partitioning structure based on a quadtree (hereinafter referred to as "QT"). Figure 3B This is a diagram used to describe a block partitioning structure based on a binary tree (hereinafter referred to as "BT"). Figure 3C This is a diagram used to describe a block partitioning structure based on a ternary tree (hereinafter referred to as "TT"), and Figure 3D This is a diagram used to describe a block partitioning structure based on an asymmetric tree (hereinafter referred to as "AT").
[0117] During video encoding, a block can be partitioned based on a quadtree (QT). Furthermore, a sub-block partitioned by QT can be recursively further partitioned using QT. Leaf blocks that are no longer partitioned by QT can be partitioned using at least one of binary trees (BT), ternary trees (TT), or asymmetric trees (AT). BT can have two types of partitions: horizontal BT (2N×N, 2N×N) and vertical BT (N×2N, N×2N). TT can have two types of partitions: horizontal TT (2N×1 / 2N, 2N×N, 2N×1 / 2N) and vertical TT (1 / 2N×2N, N×2N, 1 / 2N×2N). AT can have four types of partitions: horizontal up AT (2N×1 / 2N, 2N×3 / 2N), horizontal down AT (2N×3 / 2N, 2N×1 / 2N), vertical left AT (1 / 2N×2N, 3 / 2N×2N), and vertical right AT (3 / 2N×2N, 1 / 2N×2N). Each of BT, TT, or AT can be further recursively partitioned using BT, TT, or AT.
[0118] Figure 3A An example of QT partitioning is shown. Block A can be partitioned into four sub-blocks A0, A1, A2, and A3 using QT. Sub-block A1 can be partitioned into four sub-blocks B0, B1, B2, and B3 using QT.
[0119] Figure 3B An example of BT partitioning is shown. Block B3, which is no longer partitioned by QT, can be partitioned into vertical BT C0 and C1 or horizontal BT D0 and D1. As in block C0, each sub-block can be further recursively partitioned in the form of horizontal BT E0 and E1 or vertical BT F0 and F1.
[0120] Figure 3CAn example of TT partitioning is shown. Block B3, which is no longer partitioned by QT, can be partitioned into vertical TT C0, C1, and C2 or horizontal TT D0, D1, and D2. As in block C1, each sub-block can be further recursively partitioned into the form of horizontal TT E0, E1, and E2 or vertical TT F0, F1, and F2.
[0121] Figure 3D An example of AT partitioning is shown. Block B3, which is no longer partitioned by QT, can be partitioned into vertical AT C0 and C1 or horizontal AT D0 and D1. As in block C1, each sub-block can be further recursively partitioned in the form of horizontal AT E0 and E1 or vertical TT F0 and F1.
[0122] Furthermore, BT, TT, and AT partitions can be performed together. For example, sub-blocks partitioned by BT can be partitioned by TT or AT. Similarly, sub-blocks partitioned by TT can be partitioned by BT or AT. Sub-blocks partitioned by AT can be partitioned by BT or TT. For example, after a horizontal BT partition, each sub-block can be partitioned into vertical BT partitions, or after a vertical BT partition, each sub-block can be partitioned into horizontal BT partitions. The partitioning order differs between the two types of partitioning methods, but the final partitioned shape is the same.
[0123] Furthermore, if a block is divided, the order of searching the blocks can be defined in various ways. Typically, searches are performed from left to right or from top to bottom. A search block can mean the order in which additional blocks are added to determine whether to split each sub-block, or it can mean the encoding order of each sub-block if the block is no longer divided, or it can mean the search order when referencing information from another neighboring block within a sub-block.
[0124] Figure 4 and Figure 5 This is an implementation method that applies the present disclosure. Figure 4 A schematic block diagram illustrating the transform and quantization units 120 / 130 and the inverse quantization and inverse transform units 140 / 150 within the encoder is shown, and Figure 5 A schematic block diagram illustrating the inverse quantization and inverse transform units 220 / 230 within the decoder is shown.
[0125] Reference Figure 4 The transformation and quantization units 120 / 130 may include a main transformation unit 121, a secondary transformation unit 122, and a quantization unit 130. The inverse quantization and inverse transformation units 140 / 150 may include an inverse quantization unit 140, an inverse secondary transformation unit 151, and an inverse main transformation unit 152.
[0126] Reference Figure 5The dequantization and inverse transformation units 220 / 230 may include a dequantization unit 220, an inverse quadratic transformation unit 231, and an inverse main transformation unit 232.
[0127] In this disclosure, when performing a transformation, the transformation can be performed through multiple steps. For example, as... Figure 4 In this process, two steps can be applied: the principal transform and the second transform, or more transform steps can be used depending on the algorithm. In this case, the principal transform can be called the core transform.
[0128] The main transform unit 121 can apply a main transform to the residual signal. In this case, the main transform can be predefined in tabular form in the encoder and / or decoder.
[0129] Discrete cosine transform type 2 (hereinafter, "DCT2") can be applied to the main transform.
[0130] Alternatively, Discrete Sine Transform Type 7 (hereinafter referred to as "DST7") can be applied to specific situations. For example, in intra-frame prediction mode, DST7 can be applied to 4×4 blocks.
[0131] Furthermore, the main transform can employ combinations of various transforms DST7, DCT8, DST1, and DCT5 from the Multiple Transform Selection (MTS) framework. For example, it can use... Figure 6 .
[0132] The secondary transformation unit 122 can apply a secondary transformation to the signal after the main transformation, and here, the secondary transformation can be predefined in the table of the encoder and / or decoder.
[0133] As an implementation method, the quadratic transform can conditionally employ an indivisible quadratic transform (hereinafter referred to as "NSST"). For example, NSST can be applied only to intra-prediction blocks and can have a transform set suitable for each prediction mode group.
[0134] In this case, prediction mode groups can be configured based on symmetry relative to the prediction direction. For example, since prediction mode 52 and prediction mode 16 are symmetric about prediction mode 34 (diagonal direction), they can be grouped together to apply the same set of transformations. In this case, when applying the transformation for prediction mode 52, since prediction mode 52 has the same set of transformations as prediction mode 16, the input data is transposed and then applied.
[0135] Furthermore, since there is no directional symmetry in the planar and DC modes, each mode has a different transformation set, and the corresponding transformation set can include two transformations. For the remaining directional modes, each transformation set can include three transformations.
[0136] As another implementation, the secondary transform can employ a combination of various transforms DST7, DCT8, DST1, and DCT5 from the Multiple Transform Selection (MTS) framework. For example, it can employ... Figure 6 .
[0137] In another implementation, the DST7 can be used as the main transform.
[0138] In another implementation, DCT8 can be used as the master transform.
[0139] As another implementation, instead of applying the NSST to the entire main transform block, it can be applied only to the top left 8×8 region. For example, when the block size is 8×8 or larger, an 8×8 NSST is applied, and when the block size is smaller than 8×8, a 4×4 NSST is applied. In this case, the block is divided into 4×4 blocks, and then a 4×4 NSST is applied to each of the divided blocks.
[0140] As another implementation, even in the case of 4×N / N×4 (N>=16), 4×4NSST can be applied.
[0141] pass Figures 12 to 15 NSST, 4×4NSST and 8×8NSST are described in more detail with other embodiments within this disclosure.
[0142] The quantization unit 130 can perform quantization on the signal after the second transformation.
[0143] The dequantization unit 140 and the inverse transformation unit 150 perform the above processing in opposite ways, and redundant descriptions of them will be omitted.
[0144] Figure 5 This is a schematic block diagram of the inverse quantization unit 220 and the inverse transform unit 230 in the decoder.
[0145] Refer to the above Figure 5 The dequantization unit 220 and the inverse transformation unit 230 may include the dequantization unit 220, the inverse quadratic transformation unit 231 and the inverse main transformation unit 232.
[0146] The dequantization unit 220 obtains the transformation coefficients from the entropy-decoded signal by using quantization step size information.
[0147] The inverse quadratic transform unit 231 performs an inverse quadratic transform on the transform coefficients. In this case, the inverse quadratic transform is represented with reference to the above. Figure 4 The inverse of the described quadratic transformation.
[0148] As another implementation, the secondary transform can employ a combination of various transforms DST7, DCT8, DST1, and DCT5 from the Multiple Transform Selection (MTS) framework. For example, it can employ... Figure 6 .
[0149] The inverse master transformation unit 232 performs an inverse master transformation on the signal (or block) after the inverse second transformation and obtains the residual signal. In this case, the inverse master transformation represents the above... Figure 4 The inverse transform of the described principal transform.
[0150] As an implementation method, the main transform can employ a combination of various transforms DST7, DCT8, DST1, and DCT5 from the Multiple Transform Selection (MTS) framework. For example, it can employ... Figure 6 .
[0151] As an implementation of this disclosure, DST7 can be applied to the main transformation.
[0152] As an embodiment of this disclosure, DCT8 can be applied to the main transformer.
[0153] This disclosure provides a method for configuring transform combinations for each transform configuration group divided by at least one of prediction mode, block size, or block shape, and the inverse master transform unit 232 can perform an inverse transform based on the transform combinations configured by this disclosure. Additionally, the embodiments described in this disclosure can be applied.
[0154] Figure 6 This is a table showing the transformation configuration groups of the application multiple transformation selection (MTS) as an embodiment of the present disclosure.
[0155] Transform configuration group applying Multiple Transform Selection (MTS)
[0156] In this disclosure, configuration group G is used for transformation. i The j-th transformation combination candidate is represented by the pair shown in Equation 1 below.
[0157] [Formula 1]
[0158] (H(G i ,j),V(G i ,j))
[0159] Among them, H(G) i ,j) indicates the level transformation of the j-th candidate, and V(G i ,j) indicates the vertical transformation of the j-th candidate. For example, in Figure 6 In this context, H(G3,2) = DST7 and V(G3,2) = DCT8 can be represented. Based on the context, the assignment to H(G3,2) is... i ,j) or V(G iThe value of ,j) can be a nominal value used to distinguish transformations as in the examples above, or it can be an index value indicating the transformation, or it can be a two-dimensional (D) matrix used for the transformation.
[0160] In addition, in this disclosure, the 2D matrix values of DCT and DST can be represented as shown in Equations 2 and 3 below.
[0161] [Equation 2]
[0162] DCT Type 2: DCT type 8:
[0163] [Formula 3]
[0164] DCT type 7: DST Type 4:
[0165] In this context, whether the transformation is DST or DCT is indicated by S or C, the type number is represented as a superscript in Roman numeral form, and the subscript N indicates that the transformation is an N×N transformation. Additionally, in cases such as... and In such a 2D matrix, it is assumed that the column vectors form a transformation basis.
[0166] Reference Figure 6 The transform configuration group can be determined based on the prediction mode, and the number of groups can be a total of six groups G0 to G5. In addition, G0 to G4 correspond to the case of applying intra-frame prediction, and G5 represents the transform combination (or transform set and transform combination set) applied to the residual block generated by inter-frame prediction.
[0167] A combination of transformations may include horizontal transformations (or row transformations) applied to the rows of the corresponding 2D block and vertical transformations (or column transformations) applied to the columns.
[0168] In this configuration, each of the transform configuration groups can have four transform combination candidates. The four transform combinations can be selected or determined by transform combination indices 0 to 3, and the transform combination indices can be encoded and sent from the encoder to the decoder.
[0169] As an implementation method, depending on the intra-frame prediction mode, the residual data (or residual signal) obtained through intra-frame prediction can have different statistical properties. Therefore, as described above... Figure 6 In addition to the general cosine transform (e.g., DCT2), a transform can be applied to each intra-prediction mode.
[0170] Refer to above Figure 6Examples are shown for using 35 intra-prediction modes and 67 intra-prediction modes. Multiple transform combinations can be applied to each transform configuration group divided within each intra-prediction mode column. For example, these multiple transform combinations can include four combinations (row-direction transform and column-direction transform). As a specific example, DST-7 and DST-5 can be applied in the row (horizontal) and column (vertical) directions of group 0, resulting in a total of four combinations available.
[0171] Since a total of four transform kernel combinations can be applied to each intra-prediction mode, a transform combination index for selecting one of the transform kernel combinations can be sent to each transform unit. In this disclosure, the transform combination index may be referred to as the MTS index and denoted as mts_idx.
[0172] In addition to the above Figure 6 Besides the transform kernel represented in the diagram, there may be cases where the characteristics of the residual signal make DCT-2 optimal for both the row and column directions. Therefore, an MTS flag is defined for each coding unit to adaptively perform the transform. In this case, when the MTS flag is 0, DCT-2 can be applied to both the row and column directions, and when the MTS flag is 1, one of four combinations can be selected or determined by the MTS index.
[0173] As an implementation method, when the MTS flag is 1, if the number of non-zero transform coefficients used in a transform unit is not greater than a threshold, then the above can be omitted. Figure 6 In the case of a transform kernel, DST-7 is applied in both the row and column directions. For example, the threshold can be configured to 2, and this threshold can be configured differently based on the block size or the size of the transform unit. This also applies to other embodiments in this disclosure.
[0174] As an implementation method, if the number of non-zero transform coefficients is not greater than a threshold by first parsing the transform coefficient values, the amount of additional information transmitted can be reduced by applying DST-7 without parsing the MTS index.
[0175] As an implementation method, when the MTS flag is 1, if the number of non-zero transform coefficients is greater than the threshold of a transform unit, the MTS index can be resolved, and the horizontal and vertical transforms can be determined based on the MTS index.
[0176] As an implementation method, MTS can only be applied when both the width and height of the transformation unit are equal to or less than 32.
[0177] As an implementation method, the above can be pre-configured through offline training. Figure 6 .
[0178] As an implementation method, the MTS index can be defined as an index that can simultaneously indicate both horizontal and vertical transformations. Alternatively, the MTS index can be defined as a horizontal transformation index and a vertical transformation index separately.
[0179] In implementations, an MTS flag or MTS index can be defined at at least one level of sequence, image, slice, block, coding unit, transform unit, or prediction unit. For example, an MTS flag or MTS index can be defined at at least one level of coding unit or transform unit.
[0180] Figure 7 This is a flowchart illustrating the encoding process of performing Multiple Transform Selection (MTS) as an embodiment of the present disclosure.
[0181] This disclosure basically describes an implementation in which transformations are applied in the horizontal and vertical directions respectively, but the combination of transformations can even be composed of indivisible transformations.
[0182] Alternatively, the transformation combination can be a mixed configuration of separable and inseparable transformations. In this case, when using inseparable transformations, row / column transformation selection or horizontal / vertical direction selection is not required, and the above can be used only when separable transformations are selected. Figure 6 Transformation and combination.
[0183] Furthermore, the scheme proposed in this disclosure can be applied regardless of whether it is a primary transformation or a secondary transformation. That is, there is no restriction that the scheme should be applied only to either the primary transformation or the secondary transformation, and the scheme can be applied to both the primary transformation and the secondary transformation. In this case, the primary transformation can refer to the transformation used to first transform the residual block, while the secondary transformation can refer to the transformation used to apply the transformation to the block generated as a result of the primary transformation.
[0184] First, the encoder can determine the transform configuration group corresponding to the current block (S710). In this case, the transform configuration group can refer to the above... Figure 6 The present disclosure is not limited to the transformation configuration group, and the transformation configuration group may include other transformation combinations.
[0185] The encoder can perform transformations on candidate transformation combinations available for the transformation configuration group (S720).
[0186] As a result of performing the transformation, encoder 100 can determine or select the transformation combination with the minimum rate distortion (RD) cost (S730).
[0187] The encoder can encode the transform combination index corresponding to the selected transform combination (S740).
[0188] Figure 8This is a flowchart illustrating the decoding process of performing Multiple Transform Selection (MTS) as an embodiment of the present disclosure.
[0189] First, the decoder can determine the transform configuration group for the current block (S810).
[0190] The decoder can parse (or obtain) the transform combination index from the video signal, and here, the transform combination index can correspond to any one of multiple transform combinations in a transform configuration group (S820). For example, the transform configuration group can include Discrete Sine Transform Type (DST) 7 and Discrete Cosine Transform Type (DST) 8. The transform combination index can be referred to as the MTS index.
[0191] As an implementation method, the transformation configuration group can be configured based on at least one of the prediction mode, block size, or block shape of the current block.
[0192] The decoder can derive the transform combination corresponding to the transform combination index (S830). In this case, the transform combination can include a horizontal transform and a vertical transform, and can include at least one of DST-7 or DCT-8.
[0193] Furthermore, the transformation combination may refer to the transformation combination described in Table 6 above, but this disclosure is not limited thereto. That is, depending on other embodiments of this disclosure, the transformation combination can be configured by other transformation combinations.
[0194] The decoder can perform an inverse transform on the current block based on the transform combination (S840). When the transform combination includes row (horizontal) transforms and column (vertical) transforms, the column (vertical) transform can be applied first, followed by the row (horizontal) transform. However, this disclosure is not limited to this, and the transform order can be reversed, or when the transform combination includes inseparable transforms, the inseparable transforms can be applied immediately.
[0195] As an implementation method, when the vertical or horizontal transformation is DST-7 or DCT-8, the inverse transformation of DST-7 or the inverse transformation of DCT-8 can be applied to each column and then to each row.
[0196] As an implementation method, different transformations can be applied to each row and / or each column for vertical or horizontal transformations.
[0197] As an implementation method, the transformation combination index can be obtained based on the MTS flag indicating whether MTS is performed. That is, when MTS is performed according to the MTS flag, the transformation combination index can be obtained.
[0198] As an implementation, the decoder can check whether the number of non-zero transform coefficients is greater than a threshold. In this case, when the number of non-zero transform coefficients is greater than the threshold, the transform combination index can be obtained.
[0199] As an implementation method, an MTS flag or MTS index can be defined at at least one level among sequence, image, slice, block, coding unit, transform unit or prediction unit.
[0200] As an implementation method, the inverse transformation can only be applied when both the width and height of the transformation unit are equal to or less than 32.
[0201] On the other hand, as another implementation, the process of determining the transform configuration group and the process of parsing the transform combination index can be performed simultaneously. Alternatively, the above step S810 can be pre-configured and omitted in the encoder and / or decoder.
[0202] Figure 9 This is a flowchart describing the process of encoding MTS flags and MTS indexes as an implementation of this disclosure.
[0203] The encoder can determine whether to apply Multiple Transform Selection (MTS) (S910) to the current block.
[0204] When applying Multiple Transform Selection (MTS), the encoder can encode the MTS flag as equal to 1 (S920).
[0205] Additionally, the encoder can determine the MTS index (S930) based on at least one of the prediction mode, horizontal transform, and vertical transform of the current block. In this case, the MTS index can refer to an index indicating any of the multiple transform combinations used for each intra-frame prediction mode, and the MTS index can be sent for each transform unit.
[0206] Once the MTS index is determined, the encoder can encode the MTS index (S940).
[0207] On the other hand, when multiple transform selection (MTS) is not applied, the encoder can encode the MTS flag as equal to 0 (S950).
[0208] Figure 10 This is a flowchart describing a decoding process that applies horizontal or vertical transformations to rows or columns based on MTS flags and MTS indexes as an implementation of this disclosure.
[0209] The decoder can parse the MTS flag (S1010) from the bitstream. In this case, the MTS flag can indicate whether Multiple Transform Selection (MTS) is applied to the current block.
[0210] The decoder can determine whether to apply Multiple Transform Selection (MTS) to the current block based on the MTS flag (S1020). For example, it can check whether the MTS flag is 1.
[0211] When the MTS flag is 1, the decoder can check whether the number of non-zero transform coefficients is greater than (or equal to or greater than) a threshold (S1030). For example, the threshold can be configured to 2, and this threshold can be configured differently based on the block size or the size of the transform unit.
[0212] When the number of non-zero transform coefficients exceeds a threshold, the decoder can resolve the MTS index (S1040). In this case, the MTS index can refer to any of the multiple transform combinations used for each intra-prediction mode or inter-prediction mode, and the MTS index can be sent for each transform unit. Alternatively, the MTS index can refer to an index indicating any transform combination defined in a pre-configured transform combination table, and here, the pre-configured transform combination table can refer to the above... Figure 6 However, this disclosure is not limited to this.
[0213] The decoder can derive or determine the horizontal and vertical transformations based on at least one of the MTS index and the prediction mode (S1050).
[0214] Alternatively, the decoder can derive the transform combinations corresponding to the MTS index. For example, the decoder can derive or determine the horizontal and vertical transforms corresponding to the MTS index.
[0215] Furthermore, when the number of non-zero transform coefficients is not greater than a threshold, the decoder can apply a pre-configured vertical inverse transform (S1060) to each column. For example, the vertical inverse transform can be the inverse transform of DST7.
[0216] Additionally, the decoder can apply a pre-configured horizontal inverse transform to each row (S1070). For example, the horizontal inverse transform can be the inverse transform of DST7. That is, when the number of non-zero transform coefficients is not greater than a threshold, a transform kernel pre-configured by the encoder or decoder can be used. For example, the transform kernel pre-configured by the encoder or decoder can be used in the above... Figure 6 The transform combination table shown in the example does not define a transform kernel (e.g., DCT-2) but is widely used.
[0217] Furthermore, when the MTS flag is 0, the decoder can apply a pre-configured vertical inverse transform (S1080) to each column. For example, the vertical inverse transform can be the inverse transform of DCT2.
[0218] Additionally, the decoder can apply a pre-configured horizontal inverse transform (S1090) to each line. For example, the horizontal inverse transform can be the inverse transform of DCT2. That is, when the MTS flag is 0, a transform kernel pre-configured in the encoder or decoder can be used. For example, the transform kernel pre-configured in the above... Figure 6 The transformation kernels shown in the example table are widely used but are not defined.
[0219] Figure 11 This is a flowchart illustrating the inverse transformation based on transformation-related parameters as an embodiment of the present disclosure.
[0220] The decoder using this disclosure can obtain either `sps_mts_intra_enabled_flag` or `sps_mts_inter_enabled_flag` (S1110). In this case, `sps_mts_intra_enabled_flag` indicates whether `tu_mts_flag` exists in the residual coding syntax of the intra-frame coding unit. For example, when `sps_mts_intra_enabled_flag` = 0, `tu_mts_flag` does not exist in the residual coding syntax of the intra-frame coding unit, and when `sps_mts_intra_enabled_flag` = 1, `tu_mts_flag` exists in the residual coding syntax of the intra-frame coding unit. Additionally, `sps_mts_inter_enabled_flag` indicates whether `tu_mts_flag` exists in the residual coding syntax of the inter-frame coding unit. For example, when sps_mts_inter_enabled_flag = 0, tu_mts_flag does not exist in the residual coding syntax of the inter-frame coding unit, and when sps_mts_inter_enabled_flag = 1, tu_mts_flag exists in the residual coding syntax of the inter-frame coding unit.
[0221] The decoder can obtain `tu_mts_flag` based on either `sps_mts_intra_enabled_flag` or `sps_mts_inter_enabled_flag` (S1120). For example, the decoder can obtain `tu_mts_flag` when `sps_mts_intra_enabled_flag` = 1 or `sps_mts_inter_enabled_flag` = 1. In this case, `tu_mts_flag` indicates whether to apply multiple transform selection (hereinafter referred to as "MTS") to the residual samples of the luma transform block. For example, when `tu_mts_flag` = 0, MTS is not applied to the residual samples of the luma transform block, and when `tu_mts_flag` = 1, MTS is applied to the residual samples of the luma transform block.
[0222] As another example, at least one of the implementations of this disclosure can be applied to tu_mts_flag.
[0223] The decoder can obtain mts_idx based on tu_mts_flag (S1130). For example, when tu_mts_flag = 1, the decoder can obtain mts_idx. In this case, mts_idx indicates which transform kernel is applied to the luminance residual sample along the horizontal and / or vertical direction of the current transform block.
[0224] For example, at least one of the embodiments of this disclosure can be applied to mts_idx. As a specific example, the above can be applied. Figure 6 At least one of the implementation methods.
[0225] The decoder can derive the transform kernel corresponding to mts_idx (S1140). For example, it can be defined by dividing the transform kernel corresponding to mts_idx into horizontal and vertical transforms.
[0226] As another example, different transform kernels can be applied to horizontal and vertical transforms. However, this disclosure is not limited to this, and the same transform kernel can be applied to both horizontal and vertical transforms.
[0227] As an implementation method, mts_idx can be defined as shown in Table 1 below.
[0228] [Table 1]
[0229] mts_idx[x0][y0] trTypeHor trTypeVer 0 0 0 1 1 1 2 2 1 3 1 2 4 2 2
[0230] In addition, the decoder can perform the inverse transform based on the transform kernel (S1150).
[0231] exist Figure 11 The main description focuses on an implementation method for determining whether to apply MTS by obtaining the transform kernel by acquiring the transform kernel tu_mts_flag and then determining the transform kernel mts_idx based on the value of the acquired tu_mts_flag. However, this disclosure is not limited to this. For example, the decoder can determine the transform kernel by directly parsing mts_idx without parsing tu_mts_flag. In this case, Table 1 can be used. That is, when the mts_idx value indicates 0, DCT-2 can be applied in the horizontal / vertical direction. When the mts_idx value indicates a non-zero value, DCT-7 and / or DCT-8 can be applied based on the mts_idx value.
[0232] As another embodiment of this disclosure, a decoding process that performs transformation processing is described.
[0233] The decoder can examine the transform size nTbS (S10). In this case, the transform size nTbS can be a variable representing the horizontal sample size of the scaled transform coefficients.
[0234] The decoder can examine the transform kernel type trType (S20). In this case, the transform kernel type trType can be a variable representing the transform kernel type, and various implementations of this disclosure can be applied. The transform kernel type trType can include a horizontal transform kernel type trTypeHor and a vertical transform kernel type trTypeVer.
[0235] Referring to Table 1 above, when the transform kernel type trType is 0, the transform kernel type can represent DCT2; when the transform kernel type trType is 1, the transform kernel type can represent DST7; and when the transform kernel type trType is 2, the transform kernel type can represent DCT8.
[0236] The decoder can perform transformation matrix multiplication based on at least one of the transformation size nTbS or the transformation kernel type (S30).
[0237] As another example, when the transformation kernel type is 1 and the transformation size is 4, a predefined transformation matrix 1 can be applied when performing transformation matrix multiplication.
[0238] As another example, when the transformation kernel type is 1 and the transformation size is 8, a predetermined transformation matrix 2 can be applied when performing transformation matrix multiplication.
[0239] As another example, when the transformation kernel type is 1 and the transformation size is 16, a predetermined transformation matrix 3 can be applied when performing transformation matrix multiplication.
[0240] As another example, when the transformation kernel type is 1 and the transformation size is 32, a predefined transformation matrix 4 can be applied when performing transformation matrix multiplication.
[0241] Similarly, when the transformation kernel type is 2 and the transformation size is 4, 8, 16 or 32, predefined transformation matrices 5, 6, 7 and 8 can be applied respectively.
[0242] In this case, each of the predefined transformation matrices 1 through 8 can correspond to any of the various types of transformation matrices. As an example, the above can be applied... Figure 6 The transformation matrix of the type shown in the example.
[0243] The decoder can derive the transformed samples based on the transformation matrix multiplication (S40).
[0244] Implementation methods may be used, but this disclosure is not limited thereto. The above-described embodiments and other embodiments of this disclosure may be combined and used.
[0245] Figure 12 This is a table showing the assignment of transform sets for each intra-prediction mode in NSST as an implementation of the present disclosure.
[0246] Indivisible quadratic transformation (NSST)
[0247] The secondary transformation unit can apply a secondary transformation to the signal after the main transformation, and here, the secondary transformation can be defined in the tables of the encoder and / or decoder.
[0248] As an implementation method, the quadratic transform can conditionally employ an indivisible quadratic transform (hereinafter referred to as "NSST"). For example, NSST can be applied only to intra-prediction blocks and can have a transform set suitable for each prediction mode group.
[0249] In this case, prediction mode groups can be configured based on symmetry relative to the prediction direction. For example, since prediction mode 52 and prediction mode 16 are symmetric about prediction mode 34 (diagonal direction), they can be grouped together to apply the same set of transformations. In this case, when applying the transformation for prediction mode 52, since prediction mode 52 has the same set of transformations as prediction mode 16, the input data is transposed and then applied.
[0250] Furthermore, since there is no directional symmetry in the planar and DC modes, each mode can have a different transformation set, and the corresponding transformation set can include two transformations. For the remaining directional modes, each transformation set can include three transformations. However, this disclosure is not limited thereto, and each transformation set can include multiple transformations.
[0251] Figure 13 This is a flowchart illustrating the calculation of Givens rotation as an embodiment of the present disclosure.
[0252] As another implementation, instead of applying the NSST to the entire main transform block, it can be applied only to the top left 8×8 region. For example, when the block size is 8×8 or larger, an 8×8 NSST is applied, and when the block size is smaller than 8×8, a 4×4 NSST is applied. In this case, the block is divided into 4×4 blocks, and then a 4×4 NSST is applied to each of the divided blocks.
[0253] As another implementation, even in the case of 4×N / N×4 (N>=16), 4×4NSST can be applied.
[0254] Since both the 8×8NSST and the 4×4NSST follow the transform combination configuration described in this disclosure and are inseparable transforms, the 8×8NSST receives 64 data points and outputs 64 data points, while the 4×4NSST has 16 inputs and 16 outputs.
[0255] Both 8×8NSST and 4×4NSST are composed of hierarchical combinations of Givens rotations. Equation 4 shows the matrix corresponding to a Givens rotation, and Equation 5 below shows the matrix product.
[0256] [Formula 4]
[0257]
[0258] [Formula 5]
[0259] t m =x m cosθ-x n sinθ
[0260] t n =x m sinθ+x n cosθ
[0261] As above Figure 13 As illustrated, since one Givens rotation rotates two data points, a total of 32 or 8 Givens rotations are needed to process 64 data points (for 8×8 NSST) or 16 data points (for 4×4 NSST).
[0262] Therefore, a batch of 32 or 8 is used to form Givens rotation layers. The output data of one Givens rotation layer is transmitted as the input data of the next Givens rotation layer through the determined permutation.
[0263] Figure 14 An example of a round configuration of a 4×4 NSST consisting of Givens rotating layers and permutations is illustrated as an embodiment of applying this disclosure.
[0264] Refer to above Figure 14 This illustrates the sequential processing of four Givens rotation layers in the case of 4×4 NSST. (See above.) Figure 14 As illustrated, the output data of one Givens rotation layer is transmitted as the input data of the next Givens rotation layer through a determined permutation (i.e., scrambling).
[0265] As above Figure 14 As illustrated, the patterns to be permuted are determined systematically, and in the case of 4×4NSST, four Givens rotation layers and their corresponding permutations are combined to form a round.
[0266] In the case of 8×8 NSST, six Givens rotation layers and their corresponding permutations form one round. 4×4 NSST goes through two rounds, and 8×8 NSST goes through four rounds. Different rounds use the same permutation pattern, but the applied Givens rotation angles are different. Therefore, it is necessary to store the angle data of all Givens rotations that constitute each transformation.
[0267] As the final step, a permutation is performed on the data output from the Givens rotation layer, and the corresponding permutation information is stored for each transformation. In the forward NSST, the corresponding permutation is performed last, while in the inverse NSST, the opposite is true; the corresponding inverse permutation is applied first.
[0268] In the case of reverse NSST, the permutations and Givens rotation layers applied to forward NSST are performed in reverse order, and even for each Givens rotation angle, a negative value is used for rotation.
[0269] Figure 15 This describes an implementation of the present disclosure and illustrates a flowchart in which a positive DST7 with a length of 16 is designed using the Discrete Fourier Transform (DFT).
[0270] Implementation Method 1: Design and Implementation of a DST7 with a Length of 16
[0271] This disclosure provides a detailed implementation of the design of DST7 using DFT. The implementation of this disclosure can also be used for DCT8 design and can also be applied to MTS configurations.
[0272] exist Figure 15 The signals (information) transmitted between blocks shown in the flowchart can be scalar values and can also have vector form. For example, a vector can be written as x[0…N-1]. This illustrates a signal (information) consisting of N elements such as x[0…N-1] = [x[0]x[1]…x[N-2]x[N-1]]. A portion of the signal in the vector x[0…N-1] can be indicated as x[i…j]. For example, a portion of the signal can be indicated as x[5…10] = [x[5]x[6]x[7]x[8]x[9]x
[10] , which is part of x[0…15].
[0273] Figure 15 The flowchart illustrates the implementation of DST7 for a row or column of length 16. In this case, DST7 with length 16 is written as DST7_B16. Forward DST7 is written as forward DST7_B16. Reverse DST7 is written as reverse DST7_B16.
[0274] Furthermore, the input data is x[0…15], and the final output data can be written as y[0…15].
[0275] When input data x[0…15] is input, the encoder performs preprocessing (S1510) on the positive DST7 with a length of 16.
[0276] The encoder can apply a DFT (S1520) to the output w[0…15] of step S1510. In this case, refer to Figures 17 to 19 The steps S1520 for applying the DFT are described in detail.
[0277] The encoder can perform post-processing on the output z[0…15] of step S1520 and can output the final output data y[0…15] (S1530).
[0278] Figure 16 This describes an implementation of the present disclosure and illustrates a flowchart in which a reverse DST7 with a length of 16 is designed using the Discrete Fourier Transform (DFT).
[0279] Figure 16 The flowchart illustrates the implementation of reverse DST7 for a row or column of length 16. In this case, the input data is x[0…15], and the final output data can be written as y[0…15].
[0280] When input data x[0…15] is input, the decoder performs preprocessing (S1610) on the inverse DST7 with a length of 16.
[0281] The decoder can apply a DFT to the output of step S1610 (S1620). In this case, refer to... Figures 17 to 19 The steps S1620 for applying the DFT are described in detail.
[0282] The decoder can perform post-processing on the output of step S1620 and can output the final output data y[0…15](S1630).
[0283] Figures 17 to 19 This describes the application of embodiments of this disclosure, and illustrates the application therein. Figure 15 and Figure 16 The flowchart of the xDST7_FFT_B16 function.
[0284] refer to Figure 17 src[0…15] is input to the xDST7_FFT3 block and outputs src_FFT11[0…15] (S1710). The output src_FFT11[0…15] can be divided into two part signals and sent.
[0285] For example, src_FFT11[0…4] can be sent to the xDST7_FFT11_type1 block. src_FFT11[5…15] can be sent to the xDST7_FFT11_type2 block.
[0286] The xDST7_FFT11_type1 block can receive src_FFT11[0…4] and output dst[0…4] (S1720).
[0287] The xDST7_FFT11_type2 block receives src_FFT11[5…15] and outputs dst[5…15] (S1730).
[0288] In this case, refer to Figure 18 Describe in detail the implementation of the xDST7_FFT11_type1 block, and refer to... Figure 19 Describe in detail the implementation of the xDST7_FFT11_type2 block.
[0289] refer to Figure 18 src[0…4] is input into the xDST7_FFT11_half1 block and outputs dst_half1[0…4](S1810).
[0290] The output dst_half1[0…4] is input into the xDST7_FFT11_type1 block and output dst[0…4](S1820).
[0291] Reference Figure 19 The src[0…10] block can be divided into two parts and sent. For example, src[0…4] can be sent to the xDST7_FFT11_half1 block. src[5…10] can be sent to the xDST7_FFT11_half2 block.
[0292] The xDST7_FFT11_half1 block receives src[0…4] and outputs dst_half1[0…4] (S1910).
[0293] The xDST7_FFT11_half2 block receives src[5…10] and outputs dst_half2[0…5] (S1920).
[0294] The encoder / decoder can perform post-processing on the output of step S1920 through the xDST7_FFT11_type2_Post_Processing block, and can output the final output data dst[0…10](S1930).
[0295] Figure 17 The src_FFT11[5…15] corresponds to Figure 19The src[0…10]. That is, the assignments are performed as src[0] = src_FFT11[5], src[1] = src_FFT11[6], ..., src
[10] = src_FFT11
[15] .
[0296] In addition, Figure 19 In the xDST7_FFT11_type2_Post_Processing block, dst_half1[0…4] and dst_half2[0…5] are input sequentially from left to right. They correspond to the input parameters src_half1[0…4] and src_half2[0…5], respectively. This will be described in detail in the table indicating the operation of each block.
[0297] As mentioned above, Figure 15 and Figure 16 The block diagram can be connected to Figures 17 to 19 The block diagram is then explained.
[0298] This can be described using Tables 2 to 10. Figures 15 to 19 Detailed operation of the function.
[0299]
Table 2
[0300]
[0301]
Table 3
[0302]
[0303] In Table 3, the value rnd_factor = 1 << (final_shift – 1) can be used. Furthermore, in... Figure 15 and Figure 16 In the context of using a function to apply DST7 to all rows or columns of a block, if a value called "shift" has already been passed as an argument, the value final_shift = shift - 1 can be used.
[0304]
Table 4
[0305]
[0306]
Table 5
[0307]
[0308] In Table 5, the value rnd_factor = 1 << (final_shift – 1) can be used. Furthermore, in... Figure 15 and Figure 16 In the context of using a function to apply DST7 to all rows or columns of a block, if a value called "shift" has already been passed as an argument, the value final_shift = shift - 1 can be used.
[0309] In Table 5, `outputMinimum` and `outputMaximum` indicate the minimum and maximum values that can be included in the output, respectively. The `Clip3` function executes `Clip3(A,B,C) = (C...)`.<A)?A:(C> B)? B: Operation on C. That is, the Clip3 function clips the value of C so that the value of C must exist within the range from A to B.
[0310]
Table 6
[0311]
[0312] In Table 6, the C3 value means The value is given, and examples are provided showing that the multiplication coefficient has been set to 2. 9 Scaling. In Table 6, since shift = 10, apply rnd_factor = 1 << (shift – 1) = 2. 9 Therefore, dst[i] and dst[5+i] can be calculated as in Equation 6.
[0313] [Formula 6]
[0314] dst[i]=(src[3*i+1]+src[3*i+2]+src[3*i+3]+1)>>1
[0315] dst[5+i]=((src[3*i+1]<<1)-src[3*i+2]-src[3*i+3]+2)>>2
[0316]
Table 7
[0317]
[0318] In Table 7, array C11R is illustrated by... The calculated value.
[0319]
Table 8
[0320]
[0321] In Table 8, array C11R is illustrated by... The calculated value.
[0322]
Table 9
[0323] name xDST7_FFT11_type1_Post_Processing enter src[0…4] Output dst[0…4] operate dst[0]=src[4]; dst[1]=-src[0]; dst[2]=src[1]; dst[3]=src[2]; dst[4]=src[3];
[0324]
Table 10
[0325]
[0326] If DST7 is applied to a 16×16 2D block in the horizontal (or vertical) direction, then Figure 15 and Figure 16 The flowchart can be used in 16 rows (or columns).
[0327] Figure 20 This describes an implementation of the present disclosure and illustrates a flowchart in which a forward DST7 with a length of 32 is designed using the Discrete Fourier Transform (DFT).
[0328] Implementation Method 2: Design and Implementation of a DST7 with a Length of 32
[0329] This disclosure provides a detailed implementation of the design of DST7 using DFT. The implementation of this disclosure can also be used for DCT8 design and can also be applied to MTS configurations.
[0330] Figure 20 The flowchart illustrates the implementation of DST7 for a row or column of length 32. In this case, DST7 with length 32 is written as DST7_B32, forward DST7 is written as forward DST7_B32, and reverse DST7 is written as reverse DST7_B32.
[0331] Furthermore, the input data is x[0…31], and the final output data can be written as y[0…31].
[0332] When input data x[0…31] is input, the encoder performs preprocessing on the positive DST7 with a length of 32 (S2010).
[0333] The encoder can apply a DFT (S2020) to the output w[0…31] of step S2010. In this case, refer to Figures 22 to 24 The steps for applying the DFT are described in detail in S2020.
[0334] The encoder can perform post-processing on the output z[0…31] of step S2020 and can output the final output data y[0…31] (S2030).
[0335] Figure 21 This describes an implementation of the present disclosure and illustrates a flowchart in which a forward DST7 with a length of 32 is designed using the Discrete Fourier Transform (DFT).
[0336] Figure 21 The flowchart illustrates the implementation of reverse DST7 for a row or column of length 32. In this case, the input data is x[0…31], and the final output data can be written as y[0…31].
[0337] When input data x[0…31] is input, the decoder performs preprocessing (S2110) on the positive DST7 with a length of 32.
[0338] The decoder can apply a DFT (S2120) to the output w[0…31] of step S2110. In this case, refer to Figures 22 to 24 The steps S2120 for applying the DFT are described in detail.
[0339] The decoder can perform post-processing on the output z[0…31] of step S2120 and can output the final output data y[0…31] (S2130).
[0340] Figures 22 to 24 This describes the application of embodiments of this disclosure, and illustrates the application therein. Figure 15 and Figure 16 The flowchart of the xDST7_FFT_B16 function.
[0341] refer to Figure 22 src[0…31] is input to the xDST7_FFT5 block and outputs src_FFT13[0…31] (S2210). The output src_FFT13[0…31] can be divided into three part signals and sent.
[0342] For example, src_FFT13[0…5] can be sent to the xDST7_FFT13_type1 block, src_FFT13[6…18] can be sent to the xDST7_FFT13_type2 block, and src_FFT13[19…31] can be sent to another xDST7_FFT13_type2 block.
[0343] The xDST7_FFT13_type1 block receives src_FFT13[0…5] and outputs dst[0…5] (S2220).
[0344] The xDST7_FFT11_type2 block receives src_FFT13[6…18] and outputs dst[6…18] (S2230).
[0345] The xDST7_FFT11_type2 block receives src_FFT13[19…31] and outputs dst[19…31] (S2240).
[0346] In this case, refer to Figure 23 Describe in detail the implementation of the xDST7_FFT13_type1 block, and refer to... Figure 24 Describe in detail the implementation of the xDST7_FFT13_type2 block.
[0347] refer to Figure 23 src[0…5] is input into the xDST7_FFT13_half1 block and outputs dst_half1[0…5] (S2310).
[0348] The output dst_half1[0…5] is input into the xDST7_FFT13_type1_Post_Processing block, and the xDST7_FFT13_type1_Post_Processing block outputs dst[0…5] (S2320).
[0349] Reference Figure 24 The src[0…12] block can be divided into two parts and sent. For example, src[0…5] can be sent to the xDST7_FFT13_half1 block, and src[6…12] can be sent to the xDST7_FFT13_half2 block.
[0350] The xDST7_FFT13_half1 block receives src[0…5] and outputs dst_half1[0…5] (S2410).
[0351] The xDST7_FFT13_half2 block receives src[6…12] and outputs dst_half2[0…6] (S2420).
[0352] The encoder / decoder can perform post-processing on the output of steps S2410 and S2420 through the xDST7_FFT13_type2_Post_Processing block, and can output the final output data dst[0…12](S1930).
[0353] Figure 22 The src_FFT13[0…5] corresponds to Figure 23 The src[0…5]. That is, the assignment is performed as src[0] = src_FFT13[0], src[1] = src_FFT13[1], ..., src[5] = src_FFT13[5].
[0354] also, Figure 22 The src_FFT13[6…18] or src_FFT13[19…31] corresponds to Figure 24 The src[0…12]. For example, perform assignments such as src[0] = src_FFT13[6], src[1] = src_FFT13[7], ..., src
[12] = src_FFT13
[18] .
[0355] In addition, Figure 24 In the xDST7_FFT13_type2_Post_Processing block, dst_half1[0…5] and dst_half2[0…6] are input sequentially from left to right. They correspond to the input parameters src_half1[0…5] and src_half2[0…6], respectively. This will be described in detail in the table indicating the operation of each block.
[0356] As mentioned above, Figure 20 and Figure 21 The block diagram can be connected to Figures 22 to 24 The diagram is provided and explained.
[0357] This can be described by the following tables 11 to 18. Figures 20 to 24 Detailed operation of the function.
[0358]
Table 11
[0359]
[0360]
Table 12
[0361]
[0362] In Table 12, the value rnd_factor = 1 << (final_shift – 1) can be used. Furthermore, in... Figure 20 and Figure 21 In the context of using a function to apply DST7 to all rows or columns of a block, if a value called "shift" has already been passed as an argument, the value final_shift = shift - 1 can be used.
[0363]
Table 13
[0364]
[0365]
[0366]
Table 14
[0367]
[0368] In Table 14, the value rnd_factor = 1 << (final_shift – 1) can be used. Furthermore, in... Figure 20 and Figure 21 In the context of using a function to apply DST7 to all rows or columns of a block, if a value called "shift" has already been passed as an argument, the value final_shift = shift - 1 can be used.
[0369] In Table 14, `outputMinimum` and `outputMaximum` indicate the minimum and maximum values that can be included in the output, respectively. The `Clip3` function executes `Clip3(A,B,C) = (C...)`.<A)?A:(C> B)? B: Operation on C. That is, the Clip3 function clips the value of C so that the value of C must exist within the range from A to B.
[0370]
Table 15
[0371]
[0372]
[0373] In Table 15, array C13R is illustrated by... The calculated value.
[0374]
Table 16
[0375]
[0376] In Table 16, array C13I illustrates the transmission through... The calculated value.
[0377]
Table 17
[0378]
[0379]
Table 18
[0380]
[0381]
[0382] If DST7 is applied to a 32×32 two-dimensional block in the horizontal (or vertical) direction, then Figure 20 and Figure 21 The flowchart can be used for 32 rows (or columns).
[0383] Figure 25 This describes an implementation of the present disclosure and illustrates a flowchart in which a positive DST7 with a length of 8 is designed using the Discrete Fourier Transform (DFT).
[0384] Implementation Method 3: Design and Implementation of a DST7 with a Length of 8
[0385] This disclosure provides a detailed implementation of the design of DST7 using DFT. The implementation of this disclosure can also be used for DCT8 design and can also be applied to MTS configurations.
[0386] Figure 25 The flowchart illustrates the implementation of DST7 for a row or column of length 8. In this case, DST7 with length 8 is written as DST7_B8, forward DST7 is written as forward DST7_B8, and reverse DST7 is written as reverse DST7_B8.
[0387] Furthermore, the input data is x[0…7], and the final output data can be written as y[0…7].
[0388] When input data x[0…7] is input, the encoder performs preprocessing (S2510) on the positive DST7 with a length of 8.
[0389] The encoder can apply a DFT (S2520) to the output w[0…7] of step S2510. In this case, refer to Figure 27 and Figure 28 The steps for applying the DFT are described in detail in S2520.
[0390] The encoder can perform post-processing on the output z[0…7] of step S2520 and can output the final output data y[0…7] (S2530).
[0391] Figure 26This describes an implementation of the present disclosure and illustrates a flowchart in which a reverse DST7 with a length of 8 is designed using the Discrete Fourier Transform (DFT).
[0392] Figure 26 The flowchart illustrates the reverse DST7 implementation for a row or column of length 8. In this case, the input data is x[0…7], and the final output data can be written as y[0…7].
[0393] When input data x[0…7] is input, the decoder performs preprocessing on the inverse DST7 with a length of 8 (S2610).
[0394] The decoder can apply a DFT (S2620) to the output w[0…7] of step S2610. In this case, refer to Figure 27 and Figure 28 The steps S2620 for applying the DFT are described in detail.
[0395] The decoder can perform post-processing on the output z[0…7] of step S2620 and can output the final output data y[0…7] (S2630).
[0396] This can be described using Tables 19 to 23 below. Figure 25 and Figure 26 Detailed operation of the function.
[0397]
Table 19
[0398]
[0399]
Table 20
[0400]
[0401]
[0402] In Table 20, the value rnd_factor = 1 << (shift – 1) can be used. In this case, the shift value is the value passed as a parameter when using a function to apply DST7 to all rows or columns of a block.
[0403]
Table 21
[0404]
[0405]
Table 22
[0406]
[0407] In Table 22, the value rnd_factor = 1 << (shift – 1) can be used. In this case, the shift value is the value passed as a parameter when using a function to apply DST7 to all rows or columns of a block.
[0408] In Table 5, `outputMinimum` and `outputMaximum` indicate the minimum and maximum values that can be included in the output, respectively. The `Clip3` function executes `Clip3(A,B,C) = (C...)`.<A)?A:(C> B)? B: Operation on C. That is, the Clip3 function clips the value of C so that the value of C must exist within the range from A to B.
[0409]
Table 23
[0410]
[0411]
[0412] In Table 23, array C8 illustrates the method through... The calculated value.
[0413] If DST7 is applied to an 8×8 2D block in the horizontal (or vertical) direction, then Figure 25 and Figure 26 The flowchart can be used in 8 rows (or columns).
[0414] Implementation Method 4: A method that partially applies the DST7 implementation proposed in Implementation Methods 1-3
[0415] The DST7 implementations proposed in Embodiments 1 and 2 can be applied to DST7s with a length of 16 and DST7s with a length of 32. The DST7 implementation proposed in Embodiment 3 can be applied to DST7s with a length of 8, but this disclosure is not limited thereto, and different implementations are possible. For example, if the DST7 implementation proposed in Embodiment 3 is not applied, a DST7 implementation with a common matrix multiplication form can be applied.
[0416] Implementation Method 5: DST7 Implementation Using Discrete Fourier Transform (DFT)
[0417] The matrix form of N×N DST7 can be represented as shown in Equation 7.
[0418] [Formula 7]
[0419]
[0420] In this case, if n is the row index from 0 to N-1 and k is the column index from 0 to N-1, then the matrix in Equation 7 matches the inverse DST7 matrix multiplied by the transformation coefficients in order to reconstruct the original input.
[0421] Therefore, the transpose of Equation 7 is a forward DST7 matrix. Furthermore, the forward DST7 matrix and the inverse DST7 matrix are orthogonal to each other, and each of their default vectors has a norm of 1.
[0422] The relationship between DST7 and DFT can be expressed based on Equation 7 as in Equation 8.
[0423] [Formula 8]
[0424]
[0425] in,
[0426] and
[0427] In Equation 8, R is an N×(2N+1) matrix (number of rows × number of columns), Q is a (2N+1)×N matrix, and P is an N×N matrix. N Indicates an N×N identity matrix, and J N instruct
[0428] In Equation 8, This means that after performing a DFT of length (2N+1), only the imaginary part of the DFT result is taken. Equation 8 holds only when N is even. More specifically, This means that when x is an N×1 vector as the input to the positive DST7, if z = QPx is calculated, the output is a (2N+1)×1 vector (z), and only the imaginary part is taken after performing a DFT with a length of 2N+1 using the vector (z) as input.
[0429] As shown in Equation 8, the N inputs of matrices P, Q and R are rearranged and their signs (+ / -) are assigned, so that the main computational part in the forward DST7 becomes a DF of length 2N+1.
[0430] This disclosure uses a DST7 with a size of 2n×2n (N=2n). Therefore, when N=4, 8, 16, 32, a 9-point DFT, a 17-point DFT, a 33-point DFT, and a 65-point DFT can be applied, respectively.
[0431] This disclosure primarily describes the cases where N = 8, 16, and 32, and provides a method for introducing the design of the corresponding DFT in the form of an equivalent multidimensional DFT and integrating the DFT to obtain a low-complexity DST7.
[0432] The inverse N×N DST7 that matches the forward DST6 can be represented as a 2N+1 length DFT as shown in Equation 9:
[0433] [Formula 9]
[0434]
[0435] in,
[0436] and
[0437] In this case, R is an N×(2N+1) matrix (number of rows × number of columns), Q is a (2N+1)×N matrix, and I N Indicates an N×N identity matrix. J N The definition is the same as that in Equation 8.
[0438] This means that when x is an N×1 vector as input to the positive DST7, if z = Qx is calculated, the output is a (2N+1)×1 vector (z), and only the imaginary part is taken after performing a DFT of length 2N+1 using vector (z) as input. That is, in Equation 9 The meaning is the same as the definition in Equation 8, except that z = QPx is calculated.
[0439] In Equation 9, N is an even number. Furthermore, the same 2N+1 length DFT can be used in the reverse DST7 as in the forward DST7.
[0440] Trigonometric transformations of even lengths can be applied to codec systems employing this disclosure. For example, for DST7s with lengths of 8, 16, 32, and 64 respectively, DFTs of lengths of 17, 33, 65, and 129 are necessary according to Equation 8. As shown in Equations 10 and 11, the 33-point and 65-point DFTs applicable to DST7s with lengths of 8 and 16 can be represented as one-dimensional DFTs. Equation 12 illustrates the DFT formula for a common length N.
[0441] [Formula 10]
[0442]
[0443] [Equation 11]
[0444]
[0445] [Equation 12]
[0446]
[0447] For the N×N DST7 implementation, the application of a DFT with a length of 2N+1 has been described. However, for ease of writing, a length of N instead of 2N+1 can be used for the content including Equations 10 and 11. Therefore, if the DFT is applied via Equations 8 and 9, a transformation is required when writing it correctly.
[0448] Furthermore, the one-dimensional 33-point DFT and the one-dimensional 65-point DFT can also be represented as equivalent two-dimensional DFTs through simple input / output data transformations, and their corresponding formulas are the same as those in Equations 13 and 14.
[0449] [Equation 13]
[0450]
[0451] [Formula 14]
[0452]
[0453] Where n indicates the index of the input data, and k indicates the index of the transform coefficient.
[0454] In the following text, the remainder of a number is written as <x> N =xmodN. In addition, four index variables n1, n2, k1 and k2 are introduced, which can indicate the relationship between the 33-point DFT and the 65-point DFT as in Equations 15 and 16.
[0455] [Formula 15]
[0456] n = <22n1 + 12n2> 33
[0457] k = <11k1 + 3k2> 33
[0458] [Formula 16]
[0459] n = <26n1 + 40n2> 65
[0460] k = <13k1 + 5k2> 65
[0461] Where n indicates the index of the input data, and k indicates the index of the transform coefficients. Equation 15 indicates the index mapped to the 33-point DFT, and Equation 16 indicates the index mapped to the 65-point DFT.
[0462] Similar to Equations 17 and 18, the input / output data mapping between the one-dimensional DFT and the two-dimensional DFT according to Equations 15 and 16 is given. In this disclosure, according to Equations 15 and 16, the new input / output variables can be defined as two indexed independent variables, as in Equations 17 and 18. and
[0463] [Equation 17]
[0464]
[0465]
[0466] [Formula 18]
[0467]
[0468]
[0469] in, <x> N =xmodN.
[0470] Implementation Method 5-1: Indexing Method for the Two-Dimensional DFT Constructing DST7
[0471] Equations 15 and 17 have enabled two-dimensional DFT, but this disclosure is not limited thereto. That is, if Equation 19 is satisfied, two-dimensional DFT such as Equations 13 and 14 can be formed.
[0472] [Formula 19]
[0473] N = N1N2
[0474] n =<K1n1+K2n2> N
[0475] k =<K3k1+K4k2> N
[0476] <k1k3> N =N2
[0477] <k2k4> N =N1
[0478] <k1k4> N = <k2k3> N =0
[0479] Where N1 and N2 indicate coprime factors. Furthermore, <x> N =x mod N.
[0480] The 33-point one-dimensional DFT corresponds to (N1, N2) = (3, 11), and the 65-point one-dimensional DFT corresponds to (N1, N2) = (5, 13). In both cases, since N1 and N2 are coprime factors, Equation 19 can be applied. If K1, K2, K3, and K4 satisfy Equation 20, then in Equation 19, the following condition is met. <k1k4> N = <k2k3> N =0 condition.
[0481] [Formula 20]
[0482] K1=αN2, K2=βN1, K3=γN2, K4=δN1
[0483] In addition, in order to satisfy the other conditions of Equation 19, the relation of Equation 21 needs to be satisfied.
[0484] [Equation 21]
[0485]
[0486] Therefore, all α, β, γ, δ satisfying Equation 21 can be derived from Equation 20 to satisfy K1, K2, K3, and K4 of Equation 19, thus enabling the construction of an equivalent two-dimensional DFT. Possible implementations of α, β, γ, δ are as follows.
[0487] 1)(α,β,γ,δ)=(2,4,1,1)
[0488] This corresponds to Equation 15, and is the case of (N1, N2) = (3, 11).
[0489] 2)(α,β,γ,δ)=(2,8,1,1)
[0490] This corresponds to Equation 16, and is the case of (N1, N2) = (5, 13).
[0491] 3)(α,β,γ,δ)=(1,1,2,4)
[0492] This is the case of (N1, N2) = (3, 11).
[0493] 4)(α,β,γ,δ)=(1,1,2,8)
[0494] This is the case of (N1, N2) = (5, 13).
[0495] If the corresponding two-dimensional DFT is configured by K1, K2, K3 and K4 derived from α, β, γ, δ satisfying Equation 21, then in the process of computing the two-dimensional DFT, symmetries between the input / output data and intermediate result values may occur, such as the symmetries in these equations.
[0496] Therefore, even in the case of a two-dimensional DFT with an index different from that of the implementation (i.e., with different α, β, γ, δ values), the complexity required to perform DST7 can be significantly reduced by applying the methods and structures proposed in the implementation.
[0497] In summary, a DFT of length N (N = N1N2, where N1 and N2 are coprime factors) can be computed as a two-dimensional DFT such as Equation 22 by index transformations satisfying Equations 19 to 21 (i.e., transformations between one-dimensional and two-dimensional indices).
[0498] [Equation 22]
[0499]
[0500] If a two-dimensional DFT form, such as Equation 22, is used, the two-dimensional DFT can be decomposed into DFTs with short lengths and manipulated. Therefore, the computational load can be significantly reduced compared to the equivalent one-dimensional DFT.
[0501] Implementation Method 5-2: Optimization for Low-Complexity DST7 Design
[0502] According to Equations 13 and 14, this disclosure performs the following on a given n2. 3-point DFT and 5-point DFT.
[0503] This disclosure is applicable to output generated after performing the inner DFT loops of Equations 13 and 14. To define the same as Equation 23 The real and imaginary parts.
[0504] [Equation 23]
[0505]
[0506] in, Indicates the real part, and Indicates the imaginary part.
[0507] Similarly, input and output It can be decomposed into real and imaginary parts.
[0508] [Equation 24]
[0509]
[0510]
[0511] Among them, input This could be pixel or residual data to which the specified transformation is expected to be applied. Therefore, it can be assumed that the actual... It has the value a0.
[0512] Under such assumptions, this disclosure can examine the first transform data output by imparting input symmetry to the first step DFT (i.e., a 3-point DFT in the case of a 33-point DFT and a 5-point DFT in the case of a 65-point DFT). The relationship between them. This symmetry is provided by the P and Q matrices of Equation 8 or 9, and is described in Equations 25 and 26.
[0513] [Equation 25]
[0514] Case 1)
[0515] x(0,n2)=0,x(2,n2)=-x(1,n2)
[0516] Case 2)
[0517] x(0, n2) = -x(0, n′2) = -x(1, n2) = -x(1, n′2): x(2, n′2) for some n′2
[0518] [Equation 26]
[0519] Case 1)
[0520] x(0,n2)=0, x(3,n2)=-x(2,n2), x(4,n2)=-x(1,n2)
[0521] Case 2)
[0522] x(0,n2)=-x(0,n′2)=-x(1,n2)=-x(1,n′2): x(2,n′2)=-x(3,n′2)
[0523] x(3, n²) = -x(2, n′²), x(4, n²) = -x(1, n′²) (for some n′²)
[0524] In addition, In the first step, the output relationship is the same as that in Equations 27 and 28.
[0525] [Equation 27]
[0526]
[0527]
[0528] [Equation 28]
[0529]
[0530]
[0531] Equations 25 and 27 indicate relations in a 3-point FFT belonging to a 33-point DFT. Equations 26 and 28 indicate relations in a 5-point FFT belonging to a 65-point DFT.
[0532] For example, in Equations 25 and 26, Case 1 occurs when n2 = 0, and Case 2 occurs when n2 = 11 - n′2, n′2 = 1, 2, ..., 10 (n2 = 13 - n′2, n′2 = 1, 2, ..., 12). For the input of Case 1, the real parts of all outputs from the 3-point FFT (5-point FFT) become 0. Since the remaining one output (two outputs) can be recovered according to Equations 27 and 28, this disclosure requires maintaining one (two) imaginary outputs.
[0533] In Equations 25 and 26, due to the input mode of Case 2, this disclosure has the following characteristics as shown in Equation 29. and The relationship between them.
[0534] [Equation 29]
[0535]
[0536]
[0537] In the case of Equation 29, the same relationship between the indices n2 = 11 - n′2, n′2 = 1, 2, ..., 10 (n2 = 13 - n′2, n′2 = 1, 2, ..., 12) of the 11-point FFT (13-point FFT) is also applied.
[0538] Therefore, due to Equation 29, this disclosure only performs the 3-point FFT (5-point FFT) when n2 is in the range [0, 5] ([0, 6]), thus reducing the associated computational load.
[0539] Furthermore, in each 3-point FFT (5-point FFT) calculation within the range [1, 5] ([1, 6]), the other parts of the output can be recovered according to Equation 21. Therefore, only certain outputs are calculated, namely, two (three) real part outputs and one (two) imaginary part output.
[0540] Because of the symmetry in the first step output (Equation 29), the outputs calculated from the outer loop (second step FFT) in Equations 13 and 14 are arranged symmetrically. This reduces the computational load. The input pattern of the outer loop (second step FFT) is the same as that of Equations 30 to 33.
[0541] [Formula 30]
[0542] 1) Real part
[0543]
[0544]
[0545] [Equation 31]
[0546] 1) Real part
[0547]
[0548]
[0549] [Equation 32]
[0550] 2) Imaginary part
[0551]
[0552]
[0553] [Equation 33]
[0554] 2) Imaginary part
[0555]
[0556]
[0557] Equations 30 and 32 indicate the input symmetry encountered in an 11-point FFT belonging to a 33-point FFT.
[0558] Equations 31 and 33 indicate the input symmetries encountered in the 13-point FFT, which is part of the 65-point FFT. Other symmetries are encountered in the input set of the 11-point FFT (13-point FFT) based on the outer loop iteration. This makes it possible to recover the output from one of the previous iterations.
[0559] In this disclosure, if The vector is represented as The input symmetry that exists in the iterative process can be expressed as shown in Equation 34:
[0560] [Formula 34]
[0561] Scenario 1:
[0562] Scenario 2:
[0563] In a two-dimensional DFT such as a 33-point FFT (65-point FFT), k1 has a range of [0, 2] ([0, 4]).
[0564] In Equation 34, Case 1 occurs only when k1 = 0. In Equation 34, Case 2 occurs only when k1 = 3 - k′1, k′1 = 1, 2 (k1 = 5 - k′1, k′1 = 1, 2, 3, 4).
[0565] Due to the symmetry in Equation 34, the output of the omitted iteration can be derived from one of its previous iterations. Therefore, the effective number of iterations for the 11-point FFT (15-point FFT) in the 33-point FFT (65-point FFT) can be reduced from 3 (5) to 2 (3).
[0566] Furthermore, according to Equations 8 and 9, this disclosure may take only the imaginary part of the output from the 33-point FFT (65-point FFT). Therefore, in Equation 34, the output mode for the corresponding case can be represented as in Equations 35 to 38.
[0567] [Formula 35]
[0568] Scenario 1:
[0569]
[0570] [Formula 36]
[0571] Scenario 1:
[0572]
[0573] [Formula 37]
[0574] Scenario 2:
[0575]
[0576] [Formula 38]
[0577] Scenario 2:
[0578]
[0579] Equations 35 and 37 indicate the output symmetry in an 11-point FFT belonging to a 33-point FFT. Equations 36 and 38 indicate the output symmetry in a 13-point FFT belonging to a 65-point FFT.
[0580] Due to symmetries such as Equations 35 to 38, iterations after the outer loop become unnecessary in the two-dimensional DFT. In Equation 8, based on the relationship between the forward DST7 and the DFT, the final output of k indices is k = 2m + 1. In this case, for a 16x16 DST7 (32×32DST7), the range of m is [0, 15] ([0, 31]).
[0581] Figure 27 and Figure 28 This refers to the implementation method of this disclosure. Figure 27 A block diagram of a 16×16DST7 with a 33-point DFT applied is shown, and Figure 28 A block diagram of a 32×32DST7 with 65-point DFT applied is shown.
[0582] Implementation Method 5-3: Replacing Wingrad with Simplified DFT Blocks ′ FFT block configuration
[0583] This implementation proposes a structure that uses a general DFT instead of the Winograd FFT.
[0584] For the 33-point DFT and 65-point DFT respectively, formulas for the general one-dimensional DFT are given as in Equations 7 and 8. Furthermore, formulas for the general two-dimensional DFT corresponding to the 33-point and 65-point one-dimensional DFTs are given as in Equations 13 and 14 respectively.
[0585] exist Figures 27 to 28 In this process, the first step of the DFT is either a 3-point DFT or a 5-point DFT. The general DFT formula used for the first step of the DFT is as follows.
[0586] [Formula 39]
[0587]
[0588]
[0589]
[0590] In Equation 39, a 3-point DFT is obtained when N1 = 3, and a 5-point DFT is obtained when N1 = 5. Based on the symmetry proposed in Equation 21, the corresponding DFT only needs to be calculated for the range in Equation 34 (i.e., n2 is 0 to (N2-1) / 2). That is, when N1 = 3, N2 = 11, and when N1 = 5, N2 = 13.
[0591] Case 1 in equations 25 and 26 corresponds to Figure 27 Simplified 3-point DFT type 1 and Figure 28 The simplified 5-point DFT type 1. This corresponds to the case where n² = 0.
[0592] As in Equation 40, a simplified 3-point DFT type 1 is given.
[0593] [Formula 40]
[0594]
[0595] In Equation 40, only one multiplication is required because the calculation is only needed for k1 = 1. The formula for the simplified 5-point DFT type 1 is calculated using the same method as in Equation 41.
[0596] [Formula 41]
[0597]
[0598] In Equation 41, only two multiplications are required because they are only needed for cases k1 = 1 and 2. Furthermore, the multiplication 2 output from Equations 40 and 41 cannot be considered a multiplication because it can be handled by a left shift operation.
[0599] Case 2 in Equations 25 and 26 respectively corresponds to Figure 27 Simplified 3-point DFT type 2 and Figure 28 The simplified 5-point DFT type 2 corresponds to the cases of n2 = 1 to 5 and n2 = 1 to 6, respectively.
[0600] A simplified 3-point DFT type 2 is achieved through Equation 39. In this case, if the symmetry of Equation 27 is used, calculation is only required when k1 = 0 or 1. And calculation is only required when k1 = 1.
[0601] Similarly, a simplified 5-point DFT type 2 can be achieved using Equation 39. Likewise, if the symmetry of Equation 28 is used, calculations are only required when k1 = 0, 1, 2. And calculation is only required when k1 = 1 or 2.
[0602] exist Figure 27 and Figure 28 In this context, the second-step DFT is either an 11-point DFT or a 13-point DFT. The general DFT formula used for the second-step DFT is the same as Equation 42.
[0603] [Equation 42]
[0604]
[0605]
[0606] In Equation 42, an 11-point DFT is obtained when N2 = 11, and a 13-point DFT is obtained when N2 = 13. Due to the symmetry proposed in Equations 36 to 38, it is only necessary to consider k1 in Equation 42 as... The DFT is calculated for the range. When N2 = 11, N1 = 3, and when N2 = 13, N1 = 5.
[0607] Case 1 in Equations 34 and 35 corresponds to Figure 27 The simplified 11-point DFT type 1. Furthermore, cases 1 of equations 34 and 36 correspond to... Figure 28 The simplified 13-point DFT type 1.
[0608] If the symmetries proposed in Equations 30 to 33 are used, the simplified 11-point DFT type 1 and the simplified 13-point DFT type 1 are calculated as in Equation 43. That is, this corresponds to the case where k1 = 0.
[0609] [Formula 43]
[0610]
[0611] According to Equation 43, the simplified 11-point DFT type 1 case requires five multiplications, while the simplified 13-point DFT type 1 case requires six multiplications.
[0612] Similarly, if the symmetries proposed in Equations 30 to 33 are used, simplified 11-point DFT type 2 and simplified 13-point DFT type 2 are obtained as in Equation 44. In this case, simplified 11-point DFT type 2 is executed when k1 = 1, and simplified 13-point DFT type 2 is executed when k1 = 1 or 2.
[0613] [Formula 44]
[0614]
[0615] According to Equation 44, the simplified 11-point DFT type 2 requires ten multiplications, while the simplified 13-point DFT type 2 requires twelve multiplications.
[0616] In the multiplications appearing in Equations 40 to 44, the cosine and sine values are multiplied as DFT kernel coefficients. Since the possible values of N1 and N2 are 3, 5, 11, and 13, the coefficient values appear in the corresponding multiplications as shown in Equation 45. In this case, the case of i = 0 is excluded because the corresponding cosine or sine value has 0 or 1.
[0617] [Formula 45]
[0618]
[0619]
[0620]
[0621]
[0622] In Equations 43 and 44, since the n2 index only increases up to (N2-1) / 2, the value of i is limited to (N2-1) / 2 in the last two cases of Equation 45.
[0623] The total number of coefficients appearing in Equation 45 becomes 2 × (2 + 4 + 5 + 6) = 34, the total number of coefficients for the 33-point DFT is 2 × (2 + 5) = 14, and the total number of coefficients for the 65-point DFT is 2 × (4 + 6) = 20. Each coefficient can be approximated in integer form through scaling and rounding. The input data for DST7 is residual data in integer form, therefore all associated calculations can be performed as integer operations. Of course, since intermediate result values will also be scaled values, appropriate scaling is required in each calculation step or each output step.
[0624] Furthermore, the forms of the reference cosine and sine values are: The reference order of coefficient values can vary depending on the values of k1 and k2.
[0625] Therefore, the reference order based on n1 and n2 can be obtained in the form of a table lookup by generating a sequence list with values k1 and k2 as addresses. For example, if N2 = 11 and k2 = 3, then It can become the corresponding table entry. You can configure corresponding table entries for all possible k2 values.
[0626] exist Figure 27 and Figure 28 In this process, rectangles with elongated shapes indicated as 16 and 32 perform permutations and sign transformations on the data. This is achieved through the index transformations proposed in Equations 15 and 16, and the symmetry of the input data proposed in Equations 25 and 26. Figure 27 and Figure 28 Each of the simplified 3-point DFT type 1, simplified 3-point DFT type 2, simplified 5-point DFT type 1, and simplified 5-point DFT type 2 can receive the corresponding data. Due to the symmetry of Equations 25 and 26, the sign of a certain data is converted and input.
[0627] Implementation Method 6: Implementing DST7 through multiple scaling methods
[0628] Calculated using Equation 39 Figure 27 Simplified 3-point DFT type 2 and Figure 28 The simplified 5-point DFT type 2. More specifically, in Equation 39, the case where n² ≠ 0 occurs, and and This includes many cases where the absolute value of the change depends on the value of n1. Therefore, as in Equation 39, although the value of n1 increases from 0 to N1-1, N1 multiplications are not required. In Equation 39, when n2 ≠ 0 (i.e., Figure 27 Simplified 3-point DFT type 2 and Figure 28 The simplified 5-point DFT type 2), assuming that the A / B values are scaled as in Equation 46.
[0629] [Formula 46]
[0630]
[0631]
[0632] For example, in 46, the value or It is a floating-point number with an absolute value equal to or less than 1. Therefore, if the value A is multiplied appropriately, an integer value or a floating-point number with sufficient precision can be generated. In Equation 46, 1 / B, which is ultimately multiplied, can be calculated solely based on shift operations that depend on the value B. More details regarding this are described with reference to Implementation 7.
[0633] In Equations 40 and 41, if we multiply by A / 2B instead of A / B, we get Equations 47 and 48.
[0634] [Formula 47]
[0635]
[0636] [Formula 48]
[0637]
[0638] Even in Equations 47 and 48, it is possible to... or Multiplying by value A produces an integer or floating-point number with sufficient precision. The final multiplication of 1 / B can be calculated simply by shifting based on value B. See Implementation 7 for more details.
[0639] The simplified 11-point DFT type 1 and the simplified 13-point DFT type 1 perform the operations described in Equation 43 (corresponding to k1 = 0). Equation 49 can be obtained by multiplying by the value C / 2D, which is the scaling value.
[0640] [Formula 49]
[0641]
[0642]
[0643] As in equation 49, It can be multiplied by the value C. Integer or fixed-point arithmetic can be applied. If A / B is considered as in Equation 49 (i.e., the scaling value multiplied in Equation 46), then multiplying by... The total scaling value (i.e., one of the final result data) becomes Furthermore, the value calculated from Equation 46 It can be directly used as input, as in Equation 49.
[0644] The simplified 11-point DFT type 2 and the simplified 13-point DFT type 2 are calculated using Equation 44 (the simplified 11-point DFT type 2 is executed when k1 = 1, and the simplified 13-point DFT type 2 is executed when k1 = 1 or 2). As shown in Equation 49, if C / 2D is multiplied as a scaling value, Equation 50 is obtained.
[0645] [Formula 50]
[0646]
[0647]
[0648] in,
[0649] Even in Equation 50, as in Equation 49, it can be seen that... and The value C has already been multiplied. Therefore, the cosine value can be multiplied by the sine value using integer or floating-point arithmetic. As in Equation 49, if both the value A / B multiplied in Equation 46 and the value A / 2B multiplied in Equations 47 and 48 are considered, the result is the same as in the second formula in Equation 50. If defined as in Equation 50... The values obtained through equations 46 to 48 can then be used as input data for equation 50.
[0650] In Equation 50, the possible values of k2 range from 0 to 10 in the simplified 11-point DFT type 2 case, and from 0 to 12 in the simplified 13-point DFT type 2 case. Due to the symmetry that is essentially present in the cosine and sine values, a relation such as Equation 51 is established.
[0651] [Formula 51]
[0652]
[0653]
[0654]
[0655]
[0656] In Equation 51, the N² value of the simplified 11-point DFT type 2 is 11, and the N² value of the simplified 13-point DFT type 2 is 13. All identifiers appearing in Equation 51 are defined in the same way as those in Equation 50.
[0657] Therefore, as in Equation 51, it is only necessary to calculate for f(k1,k2). The range, and only needs to be calculated for g(k1,k2). The range. Based on the same principle, even in Equation 49, it is only necessary to calculate based on the symmetry of k2. The range.
[0658] Implementation Method 7: Implement DST7 by adjusting the scaling value based solely on integer or floating-point operations.
[0659] All scaling values appearing in Implementation 6 have an A / B form. or First, multiply by A to enable integer arithmetic, then multiply by 1 / B. Furthermore, as in Equation 45, the number of cosine and sine values appearing in all formulas is limited. Therefore, the corresponding cosine and sine values can be pre-multiplied by A and stored in an array or ROM, and can be used as a table lookup method. Equation 46 can be expressed as in Equation 52.
[0660] [Equation 52]
[0661]
[0662]
[0663] Among them, or In this context, if a sufficiently large value is multiplied as a value A and then rounded, the cosine or sine value can be modified into a scaled integer value while sufficiently preserving its precision. Typically, this can be achieved using the power of 2 form (2...). n The value of ) is taken as the value A. For example, a method such as Equation 53 can be used to approximate it. or
[0664] [Formula 53]
[0665]
[0666] In Equation 53, rounding indicates the rounding operator. Any rounding method can be used for integers, but the common rounding method based on 0.5 can be used.
[0667] In Equation 52, if B is a power of 2, then a right shift operation can be used to multiply by 1 / B (i.e., divide by B). Assume B = 2. m Then, as in Equation 54, the multiplication by 1 / B can be approximated. In this case, as in Equation 54, rounding can be considered, but this disclosure is not limited to this.
[0668] [Formula 54]
[0669]
[0670] Furthermore, the value A of the product, as shown in Equation 53, does not necessarily need to be a power of 2. In particular, if it is necessary to multiply by an additional power... If the scaling factor is in the form of A, then this needs to be incorporated into the value A.
[0671] For example, in equations 49 to 51, the values multiplied as molecules are A and C. If it is possible to multiply by one side of A or C... and Then you can multiply by α on side A and by β on side C. A is not in exponential form. For another example, you can additionally multiply by, such as... Such a value. The reason for doing this is that, in a codec system applying this disclosure, it can be additionally multiplied by... This is to maintain the range of kernel coefficient values equally for transformations of all sizes.
[0672] As a similar approach, equations 40, 41, 43 and 44 can be appropriately approximated by simply performing operations on equations 55 to 58, respectively.
[0673] [Formula 55]
[0674]
[0675] [Formula 56]
[0676]
[0677] [Formula 57]
[0678]
[0679] [Formula 58]
[0680]
[0681]
[0682] ,in,
[0683] Due to symmetry, this can be achieved only in certain ranges (specifically, and The computation of f(k1,k2) and g(k1,k2) can be performed in the ), thus significantly reducing the complexity.
[0684] Furthermore, the approximation methods for multiplying by A and for multiplying by 1 / B can also be applied to Equations 47 to 51.
[0685] In DST7 with lengths of 8, 16, or 32, the approximate implementation of scaling factor multiplication is the same as in Table 24. A, B, C, and D appearing in Table 24 are the same as those appearing in Equations 46 through 51. `shift` is a value introduced as a factor into the DST7 function and can be a value determined based on a method of performing quantization (or dequantization) after the transform (or before the inverse transform).
[0686] Table 24
[0687]
[0688] Table 25 shows examples where scaling values are applied differently from those in Table 24. Specifically, scaling values obtained by multiplying the scaling values in Table 24 by 1 / 4 are used.
[0689] Table 25
[0690]
[0691]
[0692] Figure 29 This describes an implementation of the present disclosure, and illustrates an encoding flowchart in which the forward discrete sine transform-7 (DST7) and the forward discrete cosine transform-8 (DCT8) are performed as a discrete Fourier transform (DTF).
[0693] The encoder can determine (or select) a horizontal transformation and / or a vertical transformation based on at least one of the prediction mode, block shape, and / or block size of the current block (S2910). In this case, candidates for the horizontal transformation and / or vertical transformation may include... Figure 6 At least one of the implementation methods.
[0694] The encoder can determine the optimal horizontal and / or optimal vertical transform through rate distortion (RD) optimization. The optimal horizontal and / or optimal vertical transform can correspond to one of multiple transform combinations, and these combinations can be limited by transform indices.
[0695] The encoder can signal the transformation index corresponding to the optimal horizontal transformation and / or optimal vertical transformation (S2920). In this case, other embodiments described in this disclosure can be applied to the transformation index. For example, it may include... Figure 6 At least one of the implementation methods.
[0696] In another example, the horizontal transformation index of the optimal horizontal transformation and the vertical transformation index of the optimal vertical transformation can be signaled independently.
[0697] The encoder can perform a forward transform on the current block in the horizontal direction using the optimal horizontal transform (S2930). In this case, the current block can refer to the transform block, and the optimal horizontal transform can be a forward DCT8.
[0698] Furthermore, the encoder can perform a forward transformation on the current block in the vertical direction using the optimal vertical transform (S2940). In this case, the optimal vertical transform can be a forward DST7, and the forward DST7 can be designed as a DFT.
[0699] In this embodiment, a vertical transformation is performed after a horizontal transformation, but this disclosure is not limited to this. That is, a horizontal transformation can be performed after a vertical transformation.
[0700] In implementation, the combination of horizontal and vertical transformations can include... Figure 6 At least one of the implementation methods.
[0701] In addition, the encoder can generate a transform coefficient block by performing quantization on the current block (S2950).
[0702] The encoder can generate a bitstream by performing entropy encoding on the transform coefficient block.
[0703] Figure 30 This describes an implementation of the present disclosure, and illustrates a decoding flowchart in which the inverse discrete sine transform 7 (DST7) and the inverse discrete cosine transform 8 (DCT8) are performed as a discrete Fourier transform (DTF).
[0704] The decoder can obtain the transform index from the bitstream (S3010). In this case, other embodiments described in this disclosure can be applied to the transform index. For example, it may include... Figure 6 At least one of the implementation methods.
[0705] The decoder can derive the horizontal and vertical transforms corresponding to the transform indices (S3020). In this case, candidates for the horizontal and / or vertical transforms can include... Figure 6 At least one of the implementation methods.
[0706] In this context, steps S3010 and S3020 are implementation methods, and this disclosure is not limited thereto. For example, the decoder may derive the horizontal and vertical transforms based on at least one of the prediction mode, block shape, and / or block size of the current block. In another example, the transform index may include a horizontal transform index corresponding to the horizontal transform and a vertical transform index corresponding to the vertical transform.
[0707] Furthermore, the decoder can obtain the transform coefficient block by performing entropy decoding on the bit stream, and can perform dequantization on the transform coefficient block (S3030).
[0708] The decoder can perform an inverse transform on the dequantized transform coefficient block in the vertical direction using a vertical transform (S3040). In this case, the vertical transform can correspond to DST7. That is, the decoder can apply inverse DST7 to the dequantized transform coefficient block.
[0709] This disclosure provides a method for designing forward DST7 and / or inverse DST7 as Discrete Fourier Transform (DFT).
[0710] The decoder can implement DST7 using one-dimensional DFT or two-dimensional DFT.
[0711] Furthermore, the decoder can implement DST7 using only integer operations by applying various scaling methods.
[0712] Furthermore, the decoder can be designed with DST7 lengths of 8, 16, or 32 by using DFT to implement DST7 and by using only integer operations to implement DST7.
[0713] In one implementation, the decoder can derive the transform combination corresponding to the transform index and perform an inverse transform on the current block in the vertical or horizontal direction using DST7 or DCT8. In this case, the transform combination can consist of a horizontal transform and a vertical transform. The horizontal transform and the vertical transform can correspond to either DST7 or DCT8.
[0714] In an implementation, when applying a 33-point Discrete Fourier Transform (DFT) to a DST7, the steps may include dividing a row or column of the DST7 into two partial vector signals and applying an 11-point DFT type 1 or an 11-point DFT type 2 to the two partial vector signals.
[0715] In an implementation, when a row or column of DST7 is represented as src[0…15], the two partial vector signals can be divided into src[0…4] and src[5…15].
[0716] In an implementation, when applying a 65-point Discrete Fourier Transform (DFT) to a DST7, the steps may include dividing a row or column of the DST7 into three partial vector signals and applying a 13-point DFT type 1 or a 13-point DFT type 2 to the three partial vector signals.
[0717] In the implementation, when a row or column of DST7 is represented as src[0…31], the three partial vector signals can be divided into src[0…5], src[6…18] and src[19…31].
[0718] In an implementation, a 13-point DFT type 1 can be applied to src[0…5] of the three partial vector signals, and a 13-point DFT type 2 can be applied to src[6…18] and src[19…31].
[0719] In the implementation, the one-dimensional 33-point DFT required for 16×16 DST7 and the one-dimensional 65-point DFT required for 32×32 DST7 can be decomposed into equivalent two-dimensional DFTs with shorter DFTs. As described above, redundant computations can be removed by utilizing DFTs to perform DST7, and low-complexity DST7s can be designed.
[0720] Furthermore, the decoder can perform an inverse transform in the horizontal direction using a horizontal transform (S3050). In this case, the horizontal transform can correspond to DCT8. That is, the decoder can apply inverse DCT8 to the dequantized transform coefficient block.
[0721] In this embodiment, a horizontal transformation is applied after a vertical transformation, but this disclosure is not limited to this. That is, a vertical transformation can be applied after a horizontal transformation.
[0722] In implementation, the combination of horizontal and vertical transformations can include... Figure 6 At least one of the implementation methods.
[0723] The decoder generates a residual block in step S3050 and generates a reconstructed block by adding the residual block to the prediction block.
[0724] The following describes the overall modification process according to the embodiments proposed in this disclosure. That is, referring to... Figures 1 to 30 The described implementation methods can be applied to the transformation processes described below.
[0725] In embodiments of this disclosure, the decoder can derive the residual samples (or residual sample array) of the current transform block by performing transform processing. For ease of description, the decoder has been substantially described, but this disclosure is not limited thereto. The transform processing according to embodiments of this disclosure can be applied substantially the same to the encoder.
[0726] In embodiments of this disclosure, the transformation process may receive at least one of the following variables (or tags or parameters).
[0727] -Based on the top left luminance sample of the current image, indicate the luminance position (xTbY, yTbY) of the top left luminance sample (or the position and coordinates of the top left luminance sample) of the current luminance transformation block.
[0728] In this case, xTbY indicates the horizontal coordinate of the top-left luminance sample of the current luminance transformation block, and yTbY indicates the vertical coordinate of the top-left luminance sample of the current luminance transformation block.
[0729] - A variable hTbS indicating the horizontal size (or width) of the current transform block.
[0730] - A variable vTbS indicating the vertical size (or height) of the current transform block.
[0731] - The variable cIdx indicates the color components of the current block.
[0732] - An array of (hTbS)×(vTbS) inverse quantization transform coefficients with elements d[x][y].
[0733] Furthermore, the transformation process can output a (hTbS)×(vTbS) residual sample array with elements r[x][y].
[0734] If cIdx is 0, then the minimum transform coefficient coeffMin can be set to the minimum transform coefficient CoeffMinY, and the maximum transform coefficient CoeffMax can be set to the maximum transform coefficient CoeffMaxY. Otherwise, the minimum transform coefficient can be set to the minimum transform coefficient CoeffMinC, and the maximum transform coefficient can be set to the maximum transform coefficient CoeffMaxC.
[0735] In embodiments of this disclosure, the encoder / decoder can deduce the transform type (or transform kernel) in the horizontal and / or vertical directions for the main transform (or core transform) of the current transform block based on the MTS syntax (or syntax elements) indicating whether an MTS is applied to the current transform block. For example, the deduced transform type can be derived using at least one of the prediction mode of the current transform block, the width / height of the current transform block, the MTS syntax, or cIdx.
[0736] In this disclosure, the application of MTS and the non-application of MTS are distinguished and described, but this disclosure is not limited to this description. For example, whether or not MTS is applied can have the same meaning as whether or not a transformation type other than a predefined specific transformation type (which may be represented as a basic transformation type or a default transformation type) is used for the transformation. If MTS is applied, a transformation type other than the default transformation type (e.g., any of multiple transformation types or combinations of two or more transformation types) can be used for the transformation. If MTS is not applied, the default transformation type can be used for the transformation. In an implementation, the default transformation type can be configured (or defined) as DCT2.
[0737] For example, the MTS flag syntax indicating whether MTS is applied to the current transform block and the MTS index syntax indicating the transform type applied to the current block when MTS is applied can each be sent from the encoder to the decoder. As another example, a syntax (e.g., the MTS index) including both whether MTS is applied to the current transform block and the transform type applied to the current block when MTS is applied can be sent from the encoder to the decoder. That is, in the latter embodiment, a syntax (or syntax element) indicating the transform type applied to the current transform block (or unit) within the entire transform type group (or transform type set) including the default transform type can be sent from the encoder to the decoder. Therefore, despite this statement, the syntax indicating the transform type applied to the current transform block (MTS index) can include information about whether MTS is applied. In other words, in the latter embodiment, only the MTS index can be signaled without an MTS flag. In this case, DCT2 can be interpreted as being included in the MTS. However, in the following, in this disclosure, the application of DCT2 is described as the absence of MTS. Even so, the technical scope of MTS is not limited to the corresponding definition.
[0738] Furthermore, as mentioned above, MTS can use at least two transformation types. The embodiments of this disclosure basically describe the use of a total of three transformation types: DCT2, DST7, and DCT8, but this disclosure is not limited thereto. In embodiments, to indicate the transformation type, indices 0 (DCT2), 1 (DST7), or 2 (DCT8) can be assigned.
[0739] In the implementation, the transformation type can be set to 0 when the MTS syntax value indicating whether to apply MTS to the current transform block is 0. Otherwise, the transformation type can be derived according to Table 26 below. In other words, the transformation type is set to 0 when the MTS syntax value is 0. Otherwise, the transformation type can be set to 1 or 2 for each of the horizontal / vertical directions.
[0740] Table 26
[0741]
[0742] Referring to Table 26, in this implementation, the syntax indicating whether MTS is applied (i.e., the MTS flag) can be parsed first. If MTS is not applied, the transform type of the current transform block can be determined to be 0. If MTS is applied, the syntax (or syntax element) indicating the transform type (TrType) can be parsed for the horizontal / vertical direction. If MTS is applied, the transform type applied to the horizontal / vertical direction can be determined to be 1 or 2.
[0743] In another implementation, as described above, a syntax (i.e., an MTS index) indicating the transformation type applied to the current transformation block within the entire transformation type group, including the default transformation type, can be sent. In this case, unlike in Table 26, the transformation type when MTS is applied and the transformation type when MTS is not applied can be determined based on the MTS index, as shown in Table 27, without performing MTS flag resolution.
[0744] Table 27
[0745] MTS Index 0 1 2 3 4 trTypeHor 0 1 2 1 2 trTypeVer 0 1 1 2 2
[0746] Referring to Table 27, when the MTS index is 0, MTS cannot be applied. In this case, the transformation type applied to the horizontal / vertical direction can be determined (or set) to 0. Furthermore, when the MTS index is not 0, MTS can be applied. In this case, as shown in Table 27, the transformation type applied to the horizontal / vertical direction can be determined (or set) to 1 or 2 based on the MTS index value.
[0747] When the transformation type in the horizontal / vertical direction is determined according to the method mentioned above, the (hTbS)×(vTbS) array of residual samples can be derived according to the following method.
[0748] First, by calling the one-dimensional transformation process for each column, each column (i.e., the vertical direction) of the dequantized transformation coefficients d[x][y] (where x = 0..hTbS⁻¹, y = 0..vTbS⁻¹) can be (inversely) transformed into e[x][y] (where x = 0..hTbS⁻¹, y = 0..vTbS⁻¹). e[x][y] indicates the coefficients (or list) of the inverse transformation in the vertical direction. The one-dimensional transformation process can receive the height of the current transform block, the column (or list) of the dequantized transformation coefficients d[x][y], and the vertical transformation type as input, and can output e[x][y] (y = 0..vTbS⁻¹).
[0749] Second, Equation 59 can be used to derive the intermediate sample value g[x][y] (where x = 0..hTbS-1, y = 0..vTbS-1).
[0750] [Formula 59]
[0751] g[x][y]=Clip3(coeffMin,coeffMax,(e[x][y]+64)>>7)
[0752] Referring to Equation 59, the intermediate sample values (or intermediate transformation coefficient values) after the inverse transformation has been performed in the vertical direction can be determined as the values clipped from the scaled e[x][y] values between the minimum value coeffMin and the maximum value coeffMax of the predefined coefficient.
[0753] Third, each row (i.e., the horizontal direction) of the resulting array (i.e., the intermediate samples) g[x][y] (where x = 0..hTbS⁻¹, y = 0..vTbS⁻¹) can be (inversely) transformed into r[x][y] (where x = 0..hTbS⁻¹, y = 0..vTbS⁻¹) by calling a one-dimensional transformation process on each row. r[x][y] indicates the coefficients (or list) of the inverse transformation in the horizontal direction. The one-dimensional transformation process can take the width of the current transform block, the rows (or list) g[x][y] of the intermediate sample array, and the horizontal transformation type as input, and can output r[x][y] (x = 0..hTbS⁻¹).
[0754] The following describes the one-dimensional transformation process applied in the horizontal or vertical direction.
[0755] The one-dimensional transformation processing according to the embodiments of this disclosure can be applied in the horizontal or vertical direction.
[0756] If a one-dimensional transformation is invoked for a column (i.e., the vertical direction), the one-dimensional transformation process can receive the following variables (or labels or parameters) as input.
[0757] - A variable vTbS indicating the vertical size (or height) of the scaled (or dequantized) transform coefficients (or transform blocks).
[0758] - A list of scaled transformation coefficients x with elements x[j] (j = 0..nTbs–1).
[0759] -Transformation type variable trType
[0760] Furthermore, the one-dimensional transformation processing for columns can output a list of inverse transformed samples with elements y[i] (i = 0..nTbs–1).
[0761] In contrast, when a one-dimensional process is invoked for a row (i.e., the vertical direction), the one-dimensional transformation process can receive the following variables (or labels or parameters) as input.
[0762] - A variable hTbS indicating the horizontal size (or width) of the scaled (or dequantized) transform coefficients (or transform blocks).
[0763] - A list of scaled transformation coefficients x with elements x[j] (j = 0..nTbs–1).
[0764] -Transformation type variable trType
[0765] Furthermore, the one-dimensional transformation processing for rows can output a list of inverse transformed samples with elements y[i] (i = 0..nTbs–1).
[0766] In an implementation, the transformation matrix can be applied based on the value of the transformation type as follows.
[0767] If the transformation type is 0, then Equation 60 can be applied to the scaled list of transformation coefficients.
[0768] [Formula 60]
[0769]
[0770] Referring to Equation 60, if the transformation type applied to the current transform block is 0, a predefined transformation matrix can be applied. For example, if the transformation type is 0, the transformation matrix can be defined as shown in Table 28 below.
[0771] Table 28
[0772]
[0773]
[0774]
[0775]
[0776] If the transformation type is not 0, Equation 61 can be applied to the scaled list of transformation coefficients.
[0777] [Formula 61]
[0778]
[0779] Referring to Equation 61, if the transformation type applied to the current transform block is not 0 (i.e., 1 or 2), a predefined transformation matrix can be applied.
[0780] For example, if the transformation type is 1, a 4×4 transformation matrix can be defined as shown in Table 29 below.
[0781] Table 29
[0782]
[0783] Furthermore, for example, if the transformation type is 1, an 8×8 transformation matrix can be defined as shown in Table 30 below.
[0784] Table 30
[0785]
[0786] Furthermore, for example, if the transformation type is 1, a 16×16 transformation matrix can be defined as shown in Table 31 below.
[0787] Table 31
[0788]
[0789] Furthermore, for example, if the transformation type is 2, a 32×32 transformation matrix can be defined as shown in Table 32 below.
[0790] Table 32
[0791]
[0792]
[0793] Furthermore, for example, if the transformation type is 2, a 4×4 transformation matrix can be defined as shown in Table 33 below.
[0794] Table 33
[0795]
[0796] Furthermore, for example, if the transformation type is 2, an 8×8 transformation matrix can be defined as shown in Table 34 below.
[0797] Table 34
[0798]
[0799] Furthermore, for example, if the transformation type is 2, a 16×16 transformation matrix can be defined as shown in Table 35 below.
[0800] Table 35
[0801]
[0802] Furthermore, for example, if the transformation type is 2, a 32×32 transformation matrix can be defined as shown in Table 36 below.
[0803] Table 36
[0804]
[0805]
[0806] In the following text, syntactic structures to which the methods proposed above can be applied are described as examples. In implementation, more advanced syntactic structures, such as those in Table 37 below, can be defined.
[0807] Table 37
[0808]
[0809] Table 37 illustrates the sequence parameter set syntax. The encoder can signal whether an MTS syntax element can be used via the syntax parameter set. The sequence parameter set is an example, and this disclosure is not limited thereto. Syntax elements can be signaled via video parameter sets, image parameter sets, slice headers, etc.
[0810] Specifically, when mts_intra_enabled_flag is 1, it indicates that there may be syntax elements in the residual coding syntax or transform coding syntax for intra-blocks that indicate whether MTS is applied (e.g., mts_cu_flag, mts_tu_idx, mts_cu_idx). When mts_intra_enabled_flag is 0, it indicates that there are no syntax elements in the residual coding syntax or transform coding syntax for intra-blocks that indicate whether MTS is applied.
[0811] When `mts_inter_enabled_flag` is 1, it indicates that there may be a syntax element in the residual coding syntax or transform coding syntax for inter-frame blocks that indicates whether MTS is applied. When `mts_inter_enabled_flag` is 0, it indicates that there is no syntax element in the residual coding syntax or transform coding syntax for inter-frame blocks that indicates whether MTS is applied.
[0812] In implementation, a transformation unit syntax structure such as Table 38 below can be defined.
[0813] Table 38
[0814]
[0815] Referring to Table 38, the `mts_cu_flag` syntax element is parsed if the MTS can be used for the current transform block and the width and height of the current transform block are less than or equal to the predefined maximum size. `mts_cu_flag` indicates whether the MTS is applied to the associated transform block. When `mts_cu_flag` is 1, it indicates that the MTS is applied to the residual samples of the current transform unit. When `mts_cu_flag` is 0, it indicates that the MTS is not applied to the residual samples of the current transform unit. The `maxSizeMts` variable indicates the maximum size of the transform block to which the MTS is applied.
[0816] In implementation, residual coding syntax structures such as those in Table 39 below can be defined.
[0817] Table 39
[0818]
[0819] Referring to Table 39, `mts_hor_mode` indicates the transform type (or transform kernel) applied to the residual sample in the horizontal direction of the current transform unit. When `mts_hor_mode` is 0, it indicates that a DST7 transform kernel is applied to the residual sample in the horizontal direction of the current transform unit. When `mts_hor_mode` is 1, it indicates that a DCT8 transform kernel is applied to the residual sample in the horizontal direction of the current transform unit.
[0820] `mts_ver_mode` indicates the type of transform (or transform kernel) applied to the residual sample in the vertical direction of the current transform unit. When `mts_ver_mode` is 0, it indicates that a DST7 transform kernel is applied to the residual sample in the vertical direction of the current transform unit. When `mts_ver_mode` is 1, it indicates that a DCT8 transform kernel is applied to the residual sample in the vertical direction of the current transform unit.
[0821] For x = x0..x0+cbWidth-1 and y = y0..y0+cbHeight-1, the variable MtsMode[x][y], which indicates the transformation type (or transformation kernel) for the horizontal and vertical directions, can be derived from mts_hor_mode and mts_ver_mode defined in Table 40 below.
[0822] Table 40
[0823]
[0824] For ease of description, the above-mentioned embodiments of this disclosure have been divided and described, but this disclosure is not limited thereto. That is, the above-mentioned embodiments can be performed independently, or one or more embodiments can be combined and performed.
[0825] Figure 31 This is a flowchart illustrating a method for decoding a video signal based on Multiple Transform Selection (MTS) according to an embodiment of the present disclosure.
[0826] Reference Figure 31 The decoder resolves whether to apply the first syntax element of the MTS (S3101) to the inverse transform of the current block. In this case, the MTS indicates the use of a transform mode of a different transform type than the default transform type predefined in the current block.
[0827] The decoder derives an array of dequantized transform coefficients with the width and height of the current block by performing dequantization on the current block (S3102).
[0828] The decoder determines the vertical transformation type applied to the vertical direction of the current block and the horizontal transformation type applied to the horizontal direction of the current block based on the first syntax element (S3103).
[0829] The decoder derives a residual sample array with the width and height of the current block by performing an inverse transformation on the inverse quantization transform coefficient array using vertical and horizontal transform types (S3104).
[0830] In the implementation, the default transform type can be configured as DCT2, and the other transform types besides the default transform type can be configured as DST7 and DCT8.
[0831] In the implementation, if the first syntax element indicates that the MTS is not applied to the inverse transform of the current block, then the vertical transform type and the horizontal transform type can be determined as DCT2. If the first syntax element indicates that the MTS is applied to the inverse transform of the current block, then each of the vertical transform type and the horizontal transform type can be determined as either DST7 or DCT8.
[0832] In this implementation, the method further includes parsing a second syntax element indicating whether the MTS can be used in an intra-coded block and a third syntax element indicating whether the MTS can be used in an inter-coded block. When the second syntax element is 1, the first syntax element can exist in the transform unit syntax for the intra-coded block. When the third syntax element is 1, the first syntax element can exist in the transform unit syntax for the inter-coded block.
[0833] In an implementation, the step of deriving the residual sample array may further include the following steps: performing a one-dimensional transformation on each column of the inverse quantization transform coefficient array in the vertical direction using a vertical transform type; and performing a one-dimensional transformation on each row of the intermediate sample array output by the one-dimensional transformation on each column using a horizontal transform type.
[0834] In an implementation, the step of performing one-dimensional transformation processing in the horizontal direction may further include the following steps: cropping intermediate sample values output by the one-dimensional transformation processing for each column based on the minimum and maximum values of predefined coefficients.
[0835] Figure 32 This is a diagram illustrating an apparatus for decoding video signals based on Multiple Transform Selection (MTS) according to an embodiment of the present disclosure.
[0836] Reference Figure 32 The device that decodes video signals is implemented in Figures 4 to 31 The proposed functions, processing, and / or methods. Specifically, the device can be configured to include a syntax element parsing unit 3201, an inverse quantization transform coefficient derivation unit 3202, a transform type determination unit 3203, and a residual sample derivation unit 3204.
[0837] Syntax element parsing unit 3201 parses whether to apply the MTS to the first syntax element of the inverse transform of the current block. In this case, the MTS indicates the use of a transform mode of a transform type other than the default transform type predefined in the current block.
[0838] The dequantization transform coefficient derivation unit 3202 derives an array of dequantized transform coefficients having the width and height of the current block by performing dequantization on the current block.
[0839] The transformation type determination unit 3203 determines the vertical transformation type applied to the vertical direction of the current block and the horizontal transformation type applied to the horizontal direction of the current block based on the first syntax element.
[0840] The residual sample derivation unit 3204 derives a residual sample array with the width and height of the current block by performing an inverse transformation on the inverse quantization transform coefficient array using vertical and horizontal transform types.
[0841] In the implementation, the default transform type can be configured as DCT2, and the other transform types besides the default transform type can be configured as DST7 and DCT8.
[0842] In an implementation, if the first syntax element indicates that the MTS is not applied to the inverse transform of the current block, then the vertical transform type and the horizontal transform type can be determined as DCT2. If the first syntax element indicates that the MTS is applied to the inverse transform of the current block, then each of the vertical transform type and the horizontal transform type can be determined as either DST7 or DCT8.
[0843] In this implementation, the syntax element parsing unit parses a second syntax element indicating whether the MTS can be used in an intra-coded block and a third syntax element indicating whether the MTS can be used in an inter-coded block. When the second syntax element is 1, the first syntax element can exist in the transform unit syntax for the intra-coded block. When the third syntax element is 1, the first syntax element can exist in the transform unit syntax for the inter-coded block.
[0844] In the implementation, the residual sample derivation unit 3204 can use the vertical transformation type to perform one-dimensional transformation processing on each column of the inverse quantization transformation coefficient array in the vertical direction, and can use the horizontal transformation type to perform one-dimensional transformation processing on each row of the intermediate sample array output by the one-dimensional transformation processing for each column in the horizontal direction.
[0845] In the implementation, the residual sample derivation unit 3204 can trim intermediate sample values output by one-dimensional transformation processing for each column based on the minimum and maximum values of predefined coefficients.
[0846] Figure 33 An example of a video coding system using this disclosure is provided.
[0847] A video encoding system may include a source device and a receiving device. The source device may transmit encoded video / image information or data to the receiving device in the form of a file or stream via a digital storage medium or over a network.
[0848] The source device may include a video source, an encoding device, and a transmitter. The receiving device may include a receiver, a decoding device, and a renderer. The encoding device may be referred to as a video / image encoding device. The decoding device may be referred to as a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display. The display may be configured for each device or external component.
[0849] Video sources can be obtained through video / image capture, compositing, or generation processes. Video sources may include video / image capture devices and / or video / image generation devices. Video / image capture devices may include, for example, one or more cameras, video / image archives including previously captured video / images, etc. Video / image generation devices may include, for example, computers, tablets, and smartphones, and can generate video / images (electronically). For example, virtual video / images can be generated by a computer. In this case, video / image capture processing can replace the processing that generates the relevant data.
[0850] Encoding devices can encode input video / images. They can perform a series of processes such as prediction, transformation, and quantization to optimize compression and encoding efficiency. The encoded data (encoded video / image information) can be output as a bitstream.
[0851] The transmitter can send encoded video / image information or data, output as a bitstream, to the receiver of the receiving device in the form of a file or stream, via digital storage media or over a network. Digital storage media can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. The transmitter can include elements for generating media files according to predefined file formats and may include elements for transmission over broadcast / communication networks. The receiver can extract the bitstream and send it to a decoding device.
[0852] Decoding devices can decode video / images by performing a series of processes such as inverse quantization, inverse transform, and prediction, which correspond to the operations of encoding devices.
[0853] The renderer can render decoded video / images. The rendered video / images can then be displayed on a monitor.
[0854] Figure 34 An example of a content streaming system using this disclosure is provided.
[0855] Reference Figure 34 The content streaming system using this disclosure can generally include an encoding server, a streaming server, a network server, a media storage device, a user device, and a multimedia input device.
[0856] Encoding servers are essentially used to generate bitstreams by compressing content input from multimedia input devices such as smartphones, cameras, and camcorders into digital data, and then sending the bitstreams to streaming servers. Alternatively, if multimedia input devices such as smartphones, cameras, and camcorders generate bitstreams directly, then encoding servers can be omitted.
[0857] Bitstreams can be generated by applying the encoding methods or bitstream generation methods disclosed herein. A streaming server can temporarily store bitstreams during the sending or receiving of bitstreams.
[0858] A streaming server sends multimedia data to a user's device based on a user request via a web server. The web server acts as an intermediary, informing the user what services are offered. When a user requests a desired service from the web server, the web server forwards the request to the streaming server. The streaming server then sends the multimedia data to the user. In this scenario, the content streaming system may include a separate control server. In this case, the control server is used to control the commands / responses between devices within the content streaming system.
[0859] A streaming server can receive content from media storage and / or encoding servers. For example, if content is received from an encoding server, the streaming server can receive the content in real time. In this case, to provide a smooth streaming service, the streaming server can store the bitstream for a given period of time.
[0860] Examples of user equipment may include mobile phones, smartphones, portable computers, terminals for digital broadcasting, personal digital assistants (PDAs), portable multimedia players (PMPs), navigators, touchscreen tablet PCs, tablet PCs, ultrabooks, wearable devices (e.g., watch-type terminals (smartwatches), glasses-type terminals (smart glasses), and head-mounted displays (HMDs)), digital TVs, desktop computers, and digital signage.
[0861] Servers within a content streaming system can function as distributed servers. In this case, data received from the server can be distributed and processed.
[0862] As described above, the embodiments described in this disclosure can be implemented and executed on a processor, microprocessor, controller, or chip. For example, the functional units illustrated in the figures can be implemented and executed on a computer, processor, microprocessor, controller, or chip.
[0863] Furthermore, the decoders and encoders disclosed herein can be included in multimedia broadcasting transmitting and receiving devices, mobile communication terminals, home theater video devices, digital cinema video devices, surveillance cameras, video conferencing devices, real-time communication devices such as video communication, mobile streaming devices, storage media, cameras, video-on-demand (VoD) service providers, over-the-top (OTT) video devices, internet streaming service providers, three-dimensional (3D) video devices, video telephony devices, and medical video devices, and can be used to process video signals or data signals. For example, OTT video devices can include game consoles, Blu-ray players, internet access TVs, home theater systems, smartphones, tablet PCs, and digital video recorders (DVRs).
[0864] Furthermore, the processing methods of this disclosure can be generated in the form of a program executed by a computer and can be stored in a computer-readable recording medium. Multimedia data with data structures according to this disclosure can also be stored in a computer-readable recording medium. A computer-readable recording medium includes all kinds of storage devices in which computer-readable data is stored. Computer-readable recording media can include, for example, Blu-ray discs (BD), Universal Serial Bus (USB), ROM, PROM, EPROM, EEPROM, RAM, CD-ROM, magnetic tape, floppy disks, and optical data storage devices. Furthermore, computer-readable recording media include media implemented in the form of a carrier wave (e.g., transmitted via the Internet). Additionally, bitstreams generated using encoding methods can be stored in a computer-readable recording medium or transmitted via wired or wireless communication networks.
[0865] Furthermore, the embodiments of this disclosure can be implemented as a computer program product using program code. The program code can be executed by a computer according to the embodiments of this disclosure. The program code can be stored on a computer-readable medium.
[0866] In the embodiments mentioned above, the elements and features of this disclosure have been combined in a specific manner. Each of these elements or features may be considered optional unless otherwise explicitly described. Each of these elements or features may be implemented without combination with other elements or features. Furthermore, some of these elements and / or features may be combined to form embodiments of this disclosure. The order of operations described in the embodiments of this disclosure may be changed. Some elements or features of an embodiment may be included in another embodiment, or may be replaced by corresponding elements or features of another embodiment. It is apparent that embodiments may be constructed by combining claims that are not expressly referenced in the claims or may be included as new claims by amendments after filing the application.
[0867] Embodiments according to this disclosure can be implemented using various means (e.g., hardware, firmware, software, or combinations thereof). In the case of hardware implementation, embodiments of this disclosure can be implemented using one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, etc.
[0868] When implemented in firmware or software, embodiments of this disclosure can be implemented as modules, processes, or functions that perform the functions or operations mentioned above. Software code can be stored in memory and driven by a processor. The memory can be located internally or externally to the processor and can exchange data with the processor using various known means.
[0869] It will be apparent to those skilled in the art that this disclosure may be implemented in other specific forms without departing from its essential characteristics. Therefore, the specific embodiments should not be construed as restrictive, but rather as illustrative in all respects. The scope of this disclosure should be determined through a reasonable analysis of the appended claims, and all modifications within the equivalent scope of this disclosure are included within its scope.
[0870] Industrial applicability
[0871] The preferred embodiments mentioned above have been disclosed for illustrative purposes, and those skilled in the art can improve, modify, substitute or add various other embodiments without departing from the technical spirit and scope of the present disclosure as disclosed in the appended claims. < / k1k4> < / x> < / k1k4> < / k2k4> < / k1k3> < / x> < / x>
Claims
1. A decoding device for decoding video signals based on Multiple Transform Selection (MTS), the decoding device being configured as follows: obtaining a sequence parameter set from the video signal, wherein The sequence parameter set includes an MTS intra-frame enable flag indicating whether an MTS index can exist in the syntax for intra-frame coding units and an MTS inter-frame enable flag indicating whether the MTS index can exist in the syntax for inter-frame coding units. The MTS index is obtained from the video signal based on the MTS intra-frame enable flag or the MTS inter-frame enable flag, wherein the MTS index corresponds to any one of a plurality of transform combinations within a transform configuration group, and wherein the MTS indicates a transform method based on a transform combination adaptively selected from the plurality of transform combinations. An array of inverse-quantized transform coefficients having the width and height of the current block is derived by performing inverse quantization on the current block; The vertical transformation type applied to the current block in the vertical direction and the horizontal transformation type applied to the current block in the horizontal direction are determined based on the MTS index; and Based on the vertical and horizontal transform types, an inverse transform is performed on the inverse quantization transform coefficient array to derive a residual sample array having the width and height of the current block. Specifically, based on the vertical transformation type, a one-dimensional transformation process is performed on each column of the inverse quantization transformation coefficient array in the vertical direction to output an intermediate sample array. Specifically, based on the horizontal transformation type, a one-dimensional transformation process is performed on each row of the intermediate sample array in the horizontal direction, and In the one-dimensional transformation process in the horizontal direction, the intermediate sample values of the intermediate sample array are cropped based on the minimum and maximum values of predefined coefficients.
2. The decoding device according to claim 1, in, The plurality of transformation combinations includes at least one of Discrete Cosine Transform Type 2 (DCT2), Discrete Sine Transform Type 7 (DST7), and Discrete Cosine Transform Type 8 (DCT8).
3. The decoding device according to claim 1, in, When the MTS index has a value of zero, the vertical transform type and the horizontal transform type are determined to be DCT2, and When the MTS index has a non-zero value, each of the vertical transformation type and the horizontal transformation type is determined to be either DST7 or DCT8.
4. An encoding device for encoding video signals based on Multiple Transform Selection (MTS), the encoding device being configured as follows: Generate a residual sample array; Determine the vertical transformation type applied to the current block in the vertical direction and the horizontal transformation type applied to the current block in the horizontal direction; Perform a transformation on the current block based on the vertical transformation type and the horizontal transformation type; Generate an MTS index, which corresponds to one of a plurality of transformation combinations within a transformation configuration group, wherein the one of the plurality of transformation combinations includes the vertical transformation type and the horizontal transformation type, and wherein the MTS indicates a transformation method based on a transformation combination adaptively selected from the plurality of transformation combinations; A quantized transform coefficient array having the width and height of the current block is derived by performing quantization on the current block; and Entropy encoding is performed on the current block to generate a bitstream. The sequence parameter set in the bitstream includes an MTS intra-frame enable flag indicating whether the MTS index can exist in the syntax for intra-frame coding units, and an MTS inter-frame enable flag indicating whether the MTS index can exist in the syntax for inter-frame coding units. Specifically, the MTS index is encoded into the bitstream based on the MTS intra-frame enable flag or the MTS inter-frame enable flag. Specifically, based on the horizontal transformation type, a one-dimensional transformation process is performed on each row of the residual sample array in the horizontal direction to output an intermediate sample array, and Specifically, based on the vertical transformation type, a one-dimensional transformation process is performed on each column of the intermediate sample array in the vertical direction.
5. A transmitting device for transmitting a bitstream of an image encoded based on Multiple Transform Selection (MTS), the transmitting device comprising: A processor configured to execute an encoding method to generate the bitstream, the encoding method comprising: generating a residual sample array; determining a vertical transform type applied to a current block in the vertical direction and a horizontal transform type applied to the current block in the horizontal direction; performing a transform on the current block based on the vertical transform type and the horizontal transform type; generating an MTS index corresponding to one of a plurality of transform combinations within a transform configuration group, the one of the plurality of transform combinations including the vertical transform type and the horizontal transform type, and the MTS indicating a transform method based on a transform combination adaptively selected from the plurality of transform combinations; deriving a quantized transform coefficient array having the width and height of the current block by performing quantization on the current block; and performing entropy encoding on the current block; and A transmitter configured to transmit the bitstream for the image. The sequence parameter set in the bitstream includes an MTS intra-frame enable flag indicating whether the MTS index can exist in the syntax for intra-frame coding units, and an MTS inter-frame enable flag indicating whether the MTS index can exist in the syntax for inter-frame coding units. Specifically, the MTS index is encoded into the bitstream based on the MTS intra-frame enable flag or the MTS inter-frame enable flag. Specifically, based on the horizontal transformation type, a one-dimensional transformation process is performed on each row of the residual sample array in the horizontal direction to output an intermediate sample array, and Specifically, based on the vertical transformation type, a one-dimensional transformation process is performed on each column of the intermediate sample array in the vertical direction.
