Method for encoding / decoding image signal by apparatus and bitstream transmission method

Abstract

Methods of encoding / decoding an image signal by an apparatus and a bitstream transmission method. Embodiments of the present invention provide methods and apparatuses for processing a video signal. A video signal decoding method according to an embodiment of the present invention includes determining an input length and an output length of a non-separable transform based on a height and a width of a current block, determining a non-separable transform matrix corresponding to the input length and the output length of the non-separable transform, and applying the non-separable transform matrix to a number of coefficients corresponding to the input length in the current block, wherein the height and the width of the current block are greater than or equal to 8, and in a case where the height and the width of the current block are both 8, the input length of the non-separable transform is set to 8.

Description

[0001] This application is a divisional application of the original application No. 201980014843.8 (International Application No. PCT / KR2019 / 011517, filed on September 5, 2019, entitled "Method for encoding / decoding image signal by device and bitstream transmission method"). TECHNICAL FIELD

[0002] The disclosure relates to a method and apparatus for processing a video signal, and in particular, to a method and apparatus for encoding or decoding a video signal by performing a transform. BACKGROUND

[0003] Compression encoding refers to a signal processing technique for transmitting digitized information through a communication line or storing it in an appropriate form in a storage medium. Media such as video, image, and audio can be an object of compression encoding, and in particular, a technique of performing compression encoding on an image is referred to as video image compression.

[0004] Next-generation video content will have the characteristics of high spatial resolution, high frame rate, and high-dimensional scene representation. In order to process such content, memory storage, memory access rate, and processing capacity will be significantly increased.

[0005] Accordingly, it is necessary to design an encoding tool for more efficiently processing next-generation video content. In particular, a video codec standard after the High Efficiency Video Coding (HEVC) standard requires an efficient transform technique of transforming a spatial domain video signal into a frequency domain signal together with a prediction technique with higher precision. SUMMARY

[0006] TECHNICAL PROBLEM

[0007] Embodiments of the disclosure provide an image signal processing method and apparatus that apply a transform with high encoding efficiency and low complexity.

[0008] The technical problems addressed by the disclosure are not limited to the above technical problems, and other technical problems not described herein will become apparent to those skilled in the art from the following description.

[0009] TECHNICAL SOLUTION

[0010] A method of decoding an image signal according to an embodiment of the disclosure can include the steps of determining an input length and an output length of a non-separable transform based on a height and a width of a current block, determining a non-separable transform matrix corresponding to the input length and the output length of the non-separable transform, and applying the non-separable transform matrix to coefficients according to a number of the input length in the current block, wherein the height and the width of the current block are greater than or equal to 8, and wherein the input length of the non-separable transform is determined as 8 if each of the height and the width of the current block is equal to 8.

[0011] In addition, the input length of the non-separable transform can be determined as 16 if the height and the width of the current block are not equal to 8.

[0012] In addition, the output length can be determined as 48 or 64.

[0013] In addition, applying the non-separable transform matrix to the current block can include applying the non-separable transform matrix to a top-left 4x4 region of the current block if each of the height and the width of the current block is not equal to 8 and a product of the width and the height is less than a threshold value.

[0014] In addition, determining the non-separable transform matrix can include the steps of determining a non-separable transform set index based on an intra prediction mode of the current block, determining a non-separable transform kernel corresponding to a non-separable transform index in a non-separable transform set included in the non-separable transform set index, and determining the non-separable transform matrix according to the non-separable transform based on the input length and the output length.

[0015] An apparatus for decoding an image signal according to another embodiment of the disclosure can include a memory configured to store a video signal, and a processor coupled to the memory, wherein the processor is configured to determine an input length and an output length of a non-separable transform based on a height and a width of a current block, determine a non-separable transform matrix corresponding to the input length and the output length of the non-separable transform, and apply the non-separable transform matrix to coefficients according to a number of the input length in the current block, wherein the height and the width of the current block are greater than or equal to 8, and wherein the input length of the non-separable transform is determined as 8 if each of the height and the width of the current block is equal to 8.

[0016] Advantageous Effects

[0017] According to embodiments of the disclosure, a video encoding method and apparatus with high encoding efficiency and low complexity can be provided by applying a transform based on a size of a current block.

[0018] Effects of the present disclosure are not limited to the above-mentioned effects, and other effects not described herein will become apparent to those skilled in the art from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0019] The accompanying drawings, which are included as part of the specification to help understand the present disclosure, provide embodiments of the present disclosure and describe technical features of the present disclosure by way of the following description.

[0020] Figure 1 is a block diagram schematically illustrating an encoding apparatus for encoding a video / image signal according to an embodiment of the present disclosure;

[0021] Figure 2 is a block diagram schematically illustrating a decoding apparatus for decoding an image signal according to an embodiment of the present disclosure;

[0022] Figures 3a to 3d is an example to which embodiments of the present application can be applied, Figure 3a , Figure 3b , Figure 3c and Figure 3d are views respectively illustrating block partitioning structures according to a quad tree (QT), a binary tree (BT), a ternary tree (TT), and an asymmetric tree (AT) according to an embodiment of the present disclosure;

[0023] Figure 4 is a block diagram schematically illustrating an encoding apparatus including a transform and quantization unit of Figure 1 according to an embodiment of the present disclosure, and Figure 5 is a block diagram schematically illustrating a decoding apparatus including a dequantization and inverse transform unit according to an embodiment of the present disclosure;

[0024] Figure 6 is a flowchart illustrating an example of encoding a video signal via a primary transform and a secondary transform according to an embodiment of the present disclosure;

[0025] Figure 7 is a flowchart illustrating an example of decoding a video signal via a secondary inverse transform and a primary inverse transform according to an embodiment of the present disclosure;

[0026] Figure 8 illustrates an example transform configuration group to which an adaptive multiple transform (AMT) is applied according to an embodiment of the present disclosure;

[0027] Figure 9 is a flowchart illustrating encoding to which an AMT is applied according to an embodiment of the present disclosure;

[0028] Figure 10 is a flowchart illustrating decoding to which an AMT is applied according to an embodiment of the present disclosure;

[0029] Figure 11 is a flowchart illustrating an example of encoding AMT flags and AMT indices according to embodiments of the present disclosure;

[0030] Figure 12 is a flowchart illustrating an example decoding for performing a transform based on AMT flags and AMT indices;

[0031] Figure 13 is a diagram illustrating Givens rotation according to embodiments of the present disclosure, and Figure 14 is a configuration of a round in a 4x4 NSST composed of a permutation and Givens rotation layers according to embodiments of the present disclosure;

[0032] Figure 15 illustrates an example configuration of a non-partitioned transform set for each intra prediction mode according to embodiments of the present disclosure;

[0033] Figure 16 illustrates three types of forward scanning orders of transform coefficients or transform coefficient blocks applied in the HEVC (High Efficiency Video Coding) standard, where (a) shows a diagonal scan, (b) shows a horizontal scan, and (c) shows a vertical scan.

[0034] Figure 17 illustrates positions of transform coefficients when a forward diagonal scan is applied when a 4x8 block is applied with a 4x4 RST according to embodiments of the present disclosure, and Figure 18 illustrates an example of merging the significant transform coefficients of two 4x4 blocks into a single block according to embodiments of the present disclosure;

[0035] Figure 19 illustrates an example method of configuring a hybrid NSST set for each intra prediction mode according to embodiments of the present disclosure;

[0036] Figure 20 illustrates an example method of selecting an NSST set (or kernel) considering the size of a transform block and an intra prediction mode according to embodiments of the present disclosure;

[0037] Figure 21a and Figure 21b illustrates a forward and inverse simplified transform according to embodiments of the present disclosure;

[0038] Figure 22 is a flowchart illustrating an example of decoding using a simplified transform according to embodiments of the present disclosure;

[0039] Figure 23is a flowchart illustrating an example of a decoding process applying a conditionalsimplified transform according to an embodiment of the present disclosure;

[0040] Figure 24 is a flowchart illustrating an example of a decoding process applying a conditionalsimplified transform according to an embodiment of the present disclosure;

[0041] Figure 25a 、 Figure 25b 、 Figure 26a and Figure 26b illustrates an example of a simplified transform and a simplified inverse transform according to an embodiment of the present disclosure;

[0042] Figure 27 illustrates an example region to which a simplified secondary transform is applied according to an embodiment of the present disclosure;

[0043] Figure 28 illustrates a simplified transform of each simplified factor according to an embodiment of the present disclosure;

[0044] Figure 29 illustrates an example of an encoding flowchart applying an execution transform according to an embodiment of the present disclosure.

[0045] Figure 30 illustrates an example of a decoding flowchart applying an execution transform according to an embodiment of the present disclosure.

[0046] Figure 31 illustrates an example of a detailed block diagram of a transformer 120 in an encoding apparatus 100 applying an embodiment of the present disclosure.

[0047] Figure 32 illustrates an example of a detailed block diagram of an inverse transformer 230 in a decoding apparatus applying an embodiment of the present disclosure.

[0048] Figure 33 illustrates an example of a decoding flowchart applying a transform according to an embodiment of the present disclosure.

[0049] Figure 34 illustrates an example of a block diagram of an apparatus for processing a video signal applying an embodiment of the present disclosure.

[0050] Figure 35 illustrates an example of an image encoding system applying an embodiment of the present disclosure.

[0051] Figure 36 is a structural block diagram of a content streaming system applying an embodiment of the present disclosure. DETAILED DESCRIPTION

[0052] Some embodiments of the present disclosure are described in more detail with reference to the accompanying drawings. The drawings are intended to be used in conjunction with the description in order to facilitate the understanding of the present disclosure. It is to be understood that various terms and expressions have been used throughout this specification in order to describe the various embodiments of the present disclosure. Figure 1The detailed description set forth in the disclosure is intended to describe some exemplary embodiments of the disclosure, but is not intended to describe the only embodiment of the disclosure. The detailed description includes more detail to provide a full understanding of the disclosure. However, it should be understood by those skilled in the art that the disclosure can be implemented without the more detailed description.

[0053] In some cases, in order to avoid obscuring the concept of the disclosure, known structures and devices are omitted, or can be shown in block diagram form based on the core functions of various structures and devices.

[0054] Although most of the terms used in the disclosure have been selected from general terms widely used in the art, some terms have been arbitrarily selected by the applicant, and their meanings will be explained in detail in the following description as needed. Therefore, the disclosure should be understood based on the intended meaning of the terms, not their simple names or meanings.

[0055] The specific terms used in the following description are provided to help understanding of the disclosure, and the use of the specific terms can be changed to various forms without departing from the technical spirit of the disclosure. For example, signals, data, samples, pictures, frames, blocks, etc. can be appropriately replaced and interpreted in each encoding process.

[0056] In the present specification, a "processing unit" refers to a unit in which encoding / decoding processes such as prediction, transform, and / or quantization are performed. In addition, the processing unit can be interpreted to include the meaning of a unit for a luma component and a unit for a chroma component. For example, the processing unit can correspond to a block, a coding unit (CU), a prediction unit (PU), or a transform unit (TU).

[0057] In addition, the processing unit can be interpreted as a unit for a luma component or a unit for a chroma component. For example, the processing unit can correspond to a coding tree block (CTB), a coding block (CB), a PU, or a transform block (TB) for a luma component. In addition, the processing unit can correspond to a CTB, a CB, a PU, or a TB for a chroma component. Furthermore, the processing unit is not limited thereto, and can be interpreted to include the meaning of a unit for a luma component and a unit for a chroma component.

[0058] In addition, the processing unit is not necessarily limited to a square block, and can be configured to have a polygonal shape of three or more vertices.

[0059] As used herein, "pixel" and "coefficient" (e.g., a transform coefficient or a transform coefficient that has undergone a first transform) can be collectively referred to as a sample. When a sample is used, this can mean, for example, using a pixel value or a coefficient (e.g., a transform coefficient or a transform coefficient that has undergone a first transform).

[0060] Hereinafter, a method of applying and designing a reduced-set of transforms (RST) considering computational complexity in the worst case is described for encoding / decoding of still images or videos.

[0061] Embodiments of the present disclosure provide a method and apparatus for compressing images and videos. Compressed data has the form of a bitstream, and the bitstream can be stored in various types of storage devices and can be streamed to a terminal equipped with a decoder via a network. If the terminal has a display device, the terminal can display the decoded image on the display device, or can simply store the bitstream data. The method and apparatus proposed according to the embodiments of the present disclosure are applicable to both an encoder and a decoder or both a bitstream generator and a bitstream receiver, regardless of whether the terminal outputs it through a display device or not.

[0062] An image compression apparatus mainly includes a prediction unit, a transform and quantization unit, and an entropy coding unit. Figure 1 and Figure 2 are block diagrams schematically illustrating an encoding apparatus and a decoding apparatus, respectively. Among these components, the transform and quantization unit transforms a residual signal obtained by subtracting a prediction signal from an original signal into a frequency domain signal via, for example, a discrete cosine transform (DCT)-2, and applies quantization to the frequency domain signal, thereby enabling image compression in a case where the number of non-zero signals is greatly reduced.

[0063] Figure 1 is a block diagram schematically illustrating an encoding apparatus for encoding a video / image signal according to an embodiment of the present disclosure.

[0064] The image partitioner 110 can partition an image (or a picture or a frame) input to the encoding apparatus 100 into one or more processing units. As an example, the processing units can be referred to as coding units (CUs). In this case, the coding units can be recursively partitioned from a coding tree unit (CTU) or a largest coding unit (LCU) according to a quad tree binary tree (QTBT) structure. For example, one coding unit can be partitioned into a plurality of coding units having a deeper depth based on a quad tree structure and / or a binary tree structure. In this case, for example, the quad tree structure can be first applied and then the binary tree structure can be applied. Alternatively, the binary tree structure can be first applied. An encoding process according to an embodiment of the disclosure can be performed based on the final coding units that are not further partitioned. In this case, the maximum coding unit can be immediately used as the final coding unit based on, for example, coding efficiency according to an image characteristic, or if necessary, the coding unit can be recursively partitioned into coding units having a lower depth, and the coding unit having an optimal size can be used as the final coding unit. The encoding process can include, for example, prediction, transform, or reconstruction. As an example, the processing units can further include prediction units (PUs) or transform units (TUs). In this case, the prediction units and the transform units can each be partitioned from or into the final coding units described above. The prediction unit can be a unit of sample prediction, and the transform unit can be a unit of deriving transform coefficients and / or a unit of deriving a residual signal from the transform coefficients.

[0065] In some cases, the term "unit" can be used interchangeably with "block" or "area". In general, an MxN block can denote a set of samples or transform coefficients consisting of M columns and N rows. In general, a sample can denote a pixel or a pixel value, or can denote a pixel / pixel value of only a luma component or a pixel / pixel value of only a chroma component. A sample can be used as a term corresponding to a pixel or a pel of one picture (or image).

[0066] The encoding apparatus 100 can generate a residual signal (a residual block or a residual sample array) by subtracting a prediction signal (a prediction block or a prediction sample array) output from the inter-predictor 180 or the intra-predictor 185 from an input image signal (an original block or an original sample array), and the generated residual signal is transmitted to the transformer 120. In this case, as illustrated, a unit for subtracting a prediction signal (a prediction block or a prediction sample array) from an input image signal (an original block or an original sample array) in the encoding apparatus 100 can be referred to as a subtractor 115. The predictor can perform prediction on a target block to be processed (hereinafter, a current block), and generate a prediction block including prediction samples for the current block. The predictor can determine whether to apply intra-prediction or inter-prediction in each block or CU unit. The predictor can generate various information on prediction such as prediction mode information as described below in connection with each prediction mode, and transmit the generated information to the entropy encoder 190. The prediction-related information can be encoded by the entropy encoder 190 and output in the form of a bitstream.

[0067] The intra-predictor 185 can predict the current block by referring to samples in the current picture. The referred samples can be adjacent to the current block or distant from the current block according to a prediction mode. In intra-prediction, the prediction mode can include a plurality of non-directional modes and a plurality of directional modes. The non-directional modes can include, for example, a DC mode and a planar mode. The directional modes can include, for example, 33 directional prediction modes or 65 directional prediction modes according to the degree of elaboration of the prediction direction. However, this is merely an example, and more or less directional prediction modes can be used. The intra-predictor 185 can determine a prediction mode applied to the current block using a prediction mode applied to an adjacent block.

[0068] The inter predictor 180 can derive a prediction block of a current block based on reference blocks (reference sample arrays) specified by motion vectors on reference pictures. Here, to reduce the amount of motion information transmitted in the inter prediction mode, motion information can be predicted block-by-block, sub-block-by-sub-block, or sample-by-sample based on motion information correlation between neighboring blocks and the current block. The motion information can include a motion vector and a reference picture index. The motion information can also include inter prediction direction (L0 prediction, L1 prediction, or Bi prediction) information. In the case of inter prediction, the neighboring blocks can include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. The reference picture including the reference blocks can be the same as or different from the reference picture including the temporal neighboring blocks. The temporal neighboring blocks can be referred to as, for example, collocated reference blocks or collocated CUs (colCUs), and the reference picture including the temporal neighboring blocks can be referred to as a collocated picture (colPic). For example, the inter predictor 180 can construct a motion information candidate list based on the neighboring blocks and generate information indicating which candidate is used to derive the motion vector and / or the reference picture index of the current block. Inter prediction can be performed based on various prediction modes. For example, in the skip mode and the merge mode, the inter predictor 180 can use motion information of the neighboring blocks as motion information of the current block. In the skip mode, unlike in the merge mode, a residual signal cannot be transmitted. In the motion vector prediction (MVP) mode, a motion vector of a neighboring block can be used as a motion vector predictor, and a motion vector difference can be signaled, thereby indicating a motion vector of the current block.

[0069] The prediction signal generated via the inter predictor 180 or the intra predictor 185 can be used to generate a reconstructed signal or a residual signal.

[0070] The transformer 120 can apply a transform scheme to the residual signal, generating transform coefficients. For example, the transform scheme can include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-Loève transform (KLT), a graph-based transform (GBT), or a conditional nonlinear transform (CNT). The GBT means a transform obtained from a graph representing information on a relationship between pixels. The CNT means a transform obtained based on generating a prediction signal using all previously reconstructed pixels. In addition, the transform process can be applied to square pixel blocks of the same size, or can also be applied to non-square blocks of variable sizes.

[0071] The quantizer 130 can quantize the transform coefficients and transmit the quantized transform coefficients to the entropy encoder 190, and the entropy encoder 190 can encode the quantized signal (information of the quantized transform coefficients) and output the encoded signal in a bitstream. The information of the quantized transform coefficients can be referred to as residual information. The quantizer 130 can reorder the quantized transform coefficients of a block shape in a one-dimensional vector form based on a coefficient scan order, and generate the information of the quantized transform coefficients based on the quantized transform coefficients in the one-dimensional form. The entropy encoder 190 can perform various encoding methods such as, for example, exponential Golomb, context adaptive variable length coding (CAVLC), or context adaptive binary arithmetic coding (CABAC). The entropy encoder 190 can encode values of a plurality of pieces of information (e.g., syntax elements) necessary for reconstructing a video / image together with or separately from the quantized transform coefficients. The encoded information (e.g., video / image information) can be transmitted or stored in the form of a bitstream on a basis of network abstraction layer (NAL) units. The bitstream can be transmitted via a network, or stored in a digital storage medium. The network can include, for example, a broadcasting network and / or a communication network, and the digital storage medium can include, for example, a USB, an SD, a CD, a DVD, a Blu-ray, an HDD, an SSD, or other various storage media. A transmitter (not shown) for transmitting the signal output from the entropy encoder 190 and / or a storage unit (not shown) for storing the signal can be configured as an internal / external element of the encoding apparatus 100, or the transmitter can be a component of the entropy encoder 190.

[0072] The quantized transform coefficients output from the quantizer 130 can be used to generate a prediction signal. For example, a residual signal can be reconstructed by applying dequantization and inverse transform to the quantized transform coefficients via the dequantizer 140 and the inverse transformer 150 in a loop. The adder 155 can add the reconstructed residual signal to a prediction signal output from the inter-predictor 180 or the intra-predictor 185, thereby generating a reconstructed signal (a reconstructed picture, a reconstructed block, or a reconstructed sample array). As in the case of applying a skip mode, when there is no residual for a target block to be processed, a prediction block can be used as a reconstructed block. The adder 155 can be denoted as a reconstructor or a reconstructed block generator. The generated reconstructed signal can be used for intra-prediction of a next target processing block in a current picture, and filtered as described below, and then used for inter-prediction of a next picture.

[0073] The filter 160 can enhance subjective / objective picture quality by applying filtering to the reconstructed signal. For example, the filter 160 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and transmit the modified reconstructed picture to the decoded picture buffer 170. The various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, or bilateral filter. The filter 160 can generate various information about filtering, and transfer the resulting information to the entropy encoder 190, as described below in connection with each filtering method. The filtering-related information can be encoded by the entropy encoder 190 and output in the form of a bitstream.

[0074] The modified reconstructed picture transmitted to the decoded picture buffer 170 can be used as a reference picture in the inter-predictor 180. The encoding apparatus 100 can avoid prediction mismatch between the encoding apparatus 100 and the decoding apparatus, and enhance coding efficiency when inter-prediction is thus applied.

[0075] The decoded picture buffer 170 can store the modified reconstructed picture so as to use it as a reference picture in the inter-predictor 180.

[0076] Figure 2 FIG. 1 is a block diagram schematically illustrating an encoding apparatus according to an embodiment of the disclosure.

[0077] Referring to FIG. 2, Figure 2 The decoding apparatus 200 can include an entropy decoder 210, a dequantizer 220, an inverse transformer 230, a summer 235, a filter 240, a decoded picture buffer 250, an inter-predictor 260, and an intra-predictor 265. The inter-predictor 260 and the intra-predictor 265 can be collectively referred to as a predictor. In other words, the predictor can include the inter-predictor 180 and the intra-predictor 185. The dequantizer 220 and the inverse transformer 230 can be collectively referred to as a residual processor. In other words, the residual processor can include the dequantizer 220 and the inverse transformer 230. The entropy decoder 210, the dequantizer 220, the inverse transformer 230, the summer 235, the filter 240, the inter-predictor 260, and the intra-predictor 265 can be configured in a single hardware component (e.g., a decoder or a processor) according to an embodiment. According to an embodiment, the decoded picture buffer 250 can be implemented as a single hardware component (e.g., a memory or a digital storage medium).

[0078] When a bitstream including video / image information is input, the decoding apparatus 200 can perform decoding on the bitstream, and output a decoded picture according to an embodiment. Figure 2The video / image information processing in the encoding apparatus 100 correspondingly reconstructs the image. For example, the decoding apparatus 200 can perform decoding using the processing unit applied in the encoding apparatus 100. Therefore, during decoding, the processing unit can be, for example, an encoding unit, and the encoding units can be segmented from the encoding tree units or the maximum encoding unit according to a quadtree structure and / or a binary tree structure. The reconstructed image signal decoded and output by the decoding apparatus 200 can be played via a player.

[0079] Decoding device 200 can receive data from a source in the form of a bitstream. Figure 2 The signal output by the encoding device 100 can be decoded by the entropy decoder 210. For example, the entropy decoder 210 can parse the bitstream and extract information necessary for image reconstruction (or picture reconstruction) (e.g., video / image information). For example, the entropy decoder 210 can decode the information in the bitstream based on encoding methods such as exponential Golomb coding, CAVLC, or CABAC, and can output the values ​​of the syntax elements necessary for image reconstruction and the quantized values ​​of the transform coefficients with respect to the residuals. Specifically, the CABAC entropy decoding method can receive a bin corresponding to each syntax element in the bitstream, use information about the target syntax element to be decoded, decoding information for neighboring target blocks and the target block to be decoded, or information about the symbol / bin decoded in a previous step to determine a context model, predict the probability of the bin appearing based on the determined context model, and perform arithmetic decoding of the bin. At this time, after determining the context model, the CABAC entropy decoding method can update the context model using information about the symbol / bin decoded for the context model of the next symbol / bin. Among the messages decoded by entropy decoder 210, prediction information can be provided to the predictors (e.g., inter-frame predictor 260 and intra-frame predictor 265), and the residual values ​​(i.e., quantized transform coefficients and associated processor information) entropy-decoded by entropy decoder 210 can be input to dequantizer 220. Among the messages decoded by entropy decoder 210, filtering information can be provided to filter 240. Furthermore, a receiver (not shown) for receiving the signal output from encoding device 100 can be configured as an internal / external element of decoding device 200, or the receiver can be a component of entropy decoder 210.

[0080] The dequantizer 220 can dequantize the quantized transform coefficients and output the transform coefficients. The dequantizer 220 can reorder the quantized transform coefficients into two-dimensional blocks. In this case, the reordering can be performed based on the coefficient scan order already executed by the encoding device 100. The dequantizer 220 can use quantization parameters (e.g., quantization step size information) to dequantize the quantized transform coefficients to obtain the transform coefficients.

[0081] The inverse transformer 230 obtains a residual signal (a residual block or a residual sample array) by inverse-transforming the transform coefficients.

[0082] The predictor can perform prediction on the current block and generate a prediction block including predicted samples for the current block. The predictor can determine which one of intra prediction or inter prediction is applied to the current block and determine a specific intra / inter prediction mode based on information on prediction output from the entropy decoder 210.

[0083] The intra predictor 265 can predict the current block by referring to samples in the current picture. The referred samples can be adjacent to the current block or distant from the current block according to the prediction mode. In intra prediction, the prediction mode can include various non-directional modes and various directional modes. The intra predictor 265 can determine the prediction mode applied to the current block using the prediction mode applied to the neighboring block.

[0084] The inter predictor 260 can derive a prediction block of the current block based on a reference block (a reference sample array) specified by a motion vector on a reference picture. Here, to reduce the amount of motion information transmitted in the inter prediction mode, the motion information can be predicted on a block-by-block, sub-block, or sample basis based on the motion information correlation between the neighboring block and the current block. The motion information can include a motion vector and a reference picture index. The motion information can further include inter prediction direction (L0 prediction, L1 prediction, or Bi prediction) information. In the case of inter prediction, the neighboring block can include a spatial neighboring block present in the current picture and a temporal neighboring block present in the reference picture. For example, the inter predictor 260 can construct a motion information candidate list based on information related to the prediction of the neighboring block and derive a motion vector and / or a reference picture index of the current block based on received candidate selection information. The inter prediction can be performed based on various prediction modes. The information on prediction can include information indicating the mode of inter prediction for the current block.

[0085] The adder 235 can add the obtained residual signal to a prediction signal (e.g., a prediction block or a prediction sample array) output from the inter predictor 260 or the intra predictor 265, thereby generating a reconstructed signal (a reconstructed picture, a reconstructed block, or a reconstructed sample array). As in the case where the skip mode is applied, when there is no residual of the target block to be processed, the prediction block can be used as the reconstructed block.

[0086] The adder 235 can be denoted as a reconstructor or a reconstructed block generator. The generated reconstructed signal can be used for intra prediction of the next target processing block in the current picture and filtered as described below and then used for inter prediction of the next picture.

[0087] The filter 240 can enhance subjective / objective picture quality by applying filtering to the reconstructed signal. For example, the filter 240 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and transmit the modified reconstructed picture to the decoded picture buffer 250. The various filtering methods can include, for example, deblocking filtering, sample adaptive offset (SAO), adaptive loop filter (ALF), or bilateral filter.

[0088] The modified reconstructed picture transmitted to the decoded picture buffer 250 can be used as a reference picture by the inter predictor 260.

[0089] In the disclosure, the embodiments described above in connection with the filter 160, the inter predictor 180, and the intra predictor 185 of the encoding apparatus 100 can be applied in the same manner as or corresponding to the filter 240, the inter predictor 260, and the intra predictor 265 of the decoding apparatus 200.

[0090] Figure 3a 、 Figure 3b 、 Figure 3c and Figure 3d are views respectively illustrating block partitioning structures according to a quad tree (QT), a binary tree (BT), a ternary tree (TT), and an asymmetric tree (AT) according to embodiments of the disclosure.

[0091] In video encoding, a block can be partitioned based on QT. A sub-block partitioned by QT can be recursively partitioned by QT. A leaf block which is not further partitioned by QT can be partitioned by at least one of BT, TT, or AT. BT can have two types of partitioning such as horizontal BT (2N×N, 2N×N) and vertical BT (N×2N, N×2N). TT can have two types of partitioning such as horizontal TT (2N×1 / 2N, 2N×N, 2N×1 / 2N) and vertical TT (1 / 2N×2N, N×2N, 1 / 2N×2N). AT can have four types of partitioning such as horizontal upper AT (2N×1 / 2N, 2N×3 / 2N), horizontal lower AT (2N×3 / 2N, 2N×1 / 2N), vertical left AT (1 / 2N×2N, 3 / 2N×2N), and vertical right AT (3 / 2N×2N, 1 / 2N×2N). Each of BT, TT, and AT can be further recursively partitioned using BT, TT, and AT.

[0092] Figure 3a An example of QT partitioning is shown. A block A can be partitioned by QT into four sub-blocks (A0, A1, A2, A3). The sub-block A1 can be further partitioned by QT into four sub-blocks (B0, B1, B2, B3).

[0093] Figure 3bAn example of BT partitioning is shown. Block B3, which is no longer partitioned by QT, can be partitioned into vertical BT(C0,C1) or horizontal BT(D0,D1). Like block C0, each sub-block can be further recursively partitioned, for example, in the form of horizontal BT(E0,E1) or vertical BT(F0,F1).

[0094] Figure 3c An example of TT partitioning is shown. Block B3, which is no longer partitioned by QT, can be partitioned into vertical TT(C0,C1,C2) or horizontal TT(D0,D1,D2). Like block C1, each sub-block can be further recursively partitioned, for example, into horizontal TT(E0,E1,E2) or vertical TT(F0,F1,F2).

[0095] Figure 3d An example of AT partitioning is shown. Block B3, which is no longer partitioned by QT, can be partitioned into vertical AT(C0,C1) or horizontal AT(D0,D1). Like block C1, each sub-block can be further recursively partitioned, for example, in the form of horizontal AT(E0,E1) or vertical TT(F0,F1).

[0096] Furthermore, BT, TT, and AT can be used together. For example, sub-blocks divided by BT can be divided by TT or AT. Similarly, sub-blocks divided by TT can be divided by BT or AT. Sub-blocks divided by AT can be divided by BT or TT. For example, after horizontal BT division, each sub-block can be divided by vertical BT, or after vertical BT division, each sub-block can be divided by horizontal BT. In this case, despite the different division orders applied, the final shape after division may be the same.

[0097] When a block is divided, various orders can be defined for searching that block. Typically, searches are performed from left to right or from top to bottom. The order in which blocks are searched can mean the order in which each sub-block is further divided, the order in which each sub-block is encoded if the block is no longer divided, or the search order when a sub-block references other neighboring blocks.

[0098] You can follow as follows Figures 3a to 3d The segmented processing units (or transform blocks) shown in the diagram perform transformations. Specifically, it can be segmented along both row and column directions, and transformation matrices can be applied. According to embodiments of this disclosure, other types of transformations can be applied along the row or column directions of the processing units (or transform blocks).

[0099] Figure 4 and Figure 5 This is an implementation method that applies the present disclosure. Figure 4 This is an illustrative example of an embodiment of the present disclosure including transformation and quantization units 120 / 130.Figure 1 a block diagram of an encoding apparatus 100, and Figure 5 is a block diagram schematically illustrating a decoding apparatus 200 including dequantization and inverse transform units 220 / 230 according to an embodiment of the disclosure.

[0100] Referring to Figure 4 , the transform and quantization units 120 / 130 can include a primary transform unit 121, a secondary transform unit 122, and a quantizer 130. The dequantization and inverse transform units 140 / 150 can include a dequantizer 140, an inverse secondary transform unit 151, and an inverse primary transform unit 152.

[0101] Referring to Figure 5 , the dequantization and inverse transform units 220 / 230 can include a dequantizer 220, an inverse secondary transform unit 231, and an inverse primary transform unit 232.

[0102] In the disclosure, a transform can be performed through a plurality of steps. For example, as shown in Figure 4 , two steps of a primary transform and a secondary transform can be applied, or more transform steps can be applied according to an algorithm. Here, the primary transform can be referred to as a core transform.

[0103] The primary transform unit 121 can apply a primary transform to a residual signal. Here, the primary transform can be defined in advance as a table in the encoder and / or the decoder.

[0104] The secondary transform unit 122 can apply a secondary transform to a signal after the primary transform. Here, the secondary transform can be defined in advance as a table in the encoder and / or the decoder.

[0105] According to an embodiment, a non-separable secondary transform (NSST) can be conditionally applied as the secondary transform. For example, the NSST can be applied only to intra-predicted blocks, and can have a transform set suitable for each prediction mode group.

[0106] Here, the prediction mode groups can be set based on the symmetry of the prediction directions. For example, since the prediction mode 52 and the prediction mode 16 are symmetric with respect to the prediction mode 34 (diagonal direction), they can form a group, and the same transform set can be applied thereto. When a transform is applied to the prediction mode 52, the transform is applied to transposed input data after transposing the input data, because the transform set of the prediction mode 52 is the same as that of the prediction mode 16.

[0107] Further, since the planar mode and the DC mode lack directional symmetry, they have their own transform sets, and each transform set can consist of two transforms. For other directional modes, each transform set can consist of three transforms.

[0108] The quantizer 130 can perform quantization on the secondary-transformed signal.

[0109] The inverse quantization and inverse transform units 140 / 150 can perform inverse processing of the above-described processing, and no repeated description is provided.

[0110] Figure 5 is a block diagram illustrating an inverse quantization and inverse transform unit 220 / 230 in the decoding apparatus 200.

[0111] Referring to Figure 5 , the inverse quantization and inverse transform unit 220 / 230 can include an inverse quantizer 220, an inverse secondary transform unit 231, and an inverse primary transform unit 232.

[0112] The inverse quantizer 220 obtains transform coefficients from the entropy-decoded signal using quantization step information.

[0113] The inverse secondary transform unit 231 performs inverse secondary transform on the transform coefficients. Here, the inverse secondary transform denotes inverse transform of the secondary transform described above in connection with Figure 4 .

[0114] The inverse primary transform unit 232 performs inverse primary transform on the inverse secondary-transformed signal (or block), and obtains a residual signal. Here, the inverse primary transform denotes inverse transform of the primary transform described above in connection with Figure 4 .

[0115] Figure 6 is a flowchart illustrating an example of encoding a video signal via primary transform and secondary transform according to an embodiment of the disclosure. Figure 6 The operations of can be performed by the transformer 120 of the encoding apparatus 100.

[0116] The encoding apparatus 100 can determine (or select) a forward secondary transform based on at least one of a prediction mode, a block shape, and / or a block size of a current block (S610).

[0117] The encoding apparatus 100 can determine an optimal forward secondary transform via rate-distortion (RD) optimization. The optimal forward secondary transform can correspond to one of a plurality of transform combinations, and the plurality of transform combinations can be defined by a transform index. For example, for RD optimization, the encoding apparatus 100 can compare all results of performing forward secondary transform, quantization, and residual encoding on each candidate.

[0118] The encoding apparatus 100 can signal a secondary transform index corresponding to the optimal forward secondary transform (S620). Here, other embodiments described in the disclosure can be applied to the secondary transform index.

[0119] Further, the encoding apparatus 100 can perform a forward primary scan on the current block (residual block) (S630).

[0120] The encoding apparatus 100 can perform a forward secondary transform on the current block using an optimal forward secondary transform (S640). Further, the forward secondary transform can be an RST described below. The RST means a transform in which N residual data (N x 1 residual vector) is input and R (R < N) transform coefficient data (R x 1 transform coefficient vector) is output.

[0121] According to an embodiment, the RST can be applied to a specific region of the current block. For example, when the current block is N x N, the specific region can mean a left upper N / 2 x N / 2 region. However, the disclosure is not limited thereto, and the specific region can be set differently according to at least one of a prediction mode, a block shape, or a block size. For example, when the current block is N x N, the specific region can mean a left upper M x M region (M ≤ N).

[0122] Further, the encoding apparatus 100 can perform quantization on the current block, thereby generating a transform coefficient block (S650).

[0123] The encoding apparatus 100 can perform entropy encoding on the transform coefficient block, thereby generating a bitstream.

[0124] Figure 7 is a flowchart illustrating an example of decoding a video signal via a secondary inverse transform and a primary inverse transform according to an embodiment of the disclosure. Figure 7 The operations of can be performed by the inverse transformer 230 of the decoding apparatus 200.

[0125] The decoding apparatus 200 can obtain a secondary transform index from the bitstream.

[0126] The decoding apparatus 200 can cause a secondary transform corresponding to the secondary transform index.

[0127] However, steps S710 and S720 merely correspond to one embodiment, and the disclosure is not limited thereto. For example, the decoding apparatus 200 can cause a secondary transform based on at least one of a prediction mode, a block shape, and / or a block size of the current block without obtaining the secondary transform index.

[0128] Further, the decoding apparatus 200 can obtain a transform coefficient block by entropy-decoding the bitstream, and can perform inverse quantization on the transform coefficient block (S730).

[0129] The decoding apparatus 200 can perform an inverse secondary transform on the inverse-quantized transform coefficient block (S740). For example, the inverse secondary transform can be an inverse RST. The inverse RST is the inverse of the RST described above in connection with Figure 6The transpose of the RST is described, and means a transform in which R transform coefficient data (R x 1 transform coefficient vector) is input and N residual data (N x 1 residual vector) is output.

[0130] According to embodiments, a simplified secondary transform can be applied to a specific region of the current block. For example, when the current block is N x N, the specific region can mean a left-top N / 2 x N / 2 region. However, the disclosure is not limited thereto, and the specific region can be set differently according to at least one of a prediction mode, a block shape, or a block size. For example, when the current block is N x N, the specific region can mean a left-top M x M (M ≤ N) or M x L (M ≤ N, L ≤ N) region.

[0131] The decoding apparatus 200 can perform inverse primary transform on the result of the inverse secondary transform (S750).

[0132] The decoding apparatus 200 generates a residual block via step S750, and generates a reconstructed block by adding the residual block to a prediction block.

[0133] Figure 8 Example transform configuration groups to which adaptive multiple transform (AMT) according to embodiments of the disclosure is applied are exemplified.

[0134] Referring to Figure 8 The transform configuration groups can be determined based on a prediction mode, and there can be six (G0 to G5) groups in total. G0 to G4 correspond to cases in which intra prediction is applied, and G5 represents a transform combination (or transform set or transform combination set) applied to a residual block generated through inter prediction.

[0135] One transform combination can consist of a horizontal transform (or row transform) applied to rows of a two-dimensional block and a vertical transform (or column transform) applied to columns of the two-dimensional block.

[0136] Here, each transform configuration group can include four transform combination candidates. The four transform combination candidates can be selected or determined via transform combination indexes 0 to 3, and the transform combination indexes can be transmitted from the encoding apparatus 100 to the decoding apparatus 200 via an encoding process.

[0137] According to embodiments, residual data (or residual signal) obtained via intra prediction can have different statistical characteristics according to an intra prediction mode. Accordingly, a transform other than a regular cosine transform can be applied per prediction mode, as shown in Figure 8 in DCT type 2, DCT-II, or DCT-2.

[0138] Figure 8A corresponding transform set configuration is exemplified when 35 intra prediction modes are used and when 67 intra prediction modes are used. A plurality of transform combinations can be applied according to transform configuration groups distinguished in the intra prediction mode column. For example, a plurality of transform combinations (transform along the row direction, transform along the column direction) can consist of four combinations. More specifically, four combinations are possible since DST-7 and DCT-5 in group 0 can be applied to both the row (horizontal) direction and the column (vertical) direction.

[0139] Since a total of four transform kernel combinations can be applied to each intra prediction mode, a transform combination index for selecting one of them can be transmitted on a transform unit basis. In the present disclosure, the transform combination index can be denoted as an AMT index and can be denoted as amt_idx.

[0140] In a kernel other than the kernel proposed in Figure 8 , DCT-2 is sometimes best for both the row direction and the column direction due to the nature of the residual signal. Accordingly, the transform can be adaptively performed by defining an AMT flag for each coding unit. Here, if the AMT flag is 0, DCT-2 can be applied to both the row direction and the column direction, and if the AMT flag is 1, one of the four combinations can be selected or determined via an AMT index.

[0141] According to an embodiment, in the case where the AMT flag is 0, if the number of transform coefficients is 3 or less for one transform unit, the transform kernel of Figure 8 is not applied, and DST-7 can be applied to both the row direction and the column direction.

[0142] According to an embodiment, transform coefficient values are first parsed, and if the number of transform coefficients is 3 or less, the AMT index is not parsed, and DST-7 can be applied, thereby reducing the transmission of additional information.

[0143] According to an embodiment, the AMT can be applied only when both the width and the height of the transform unit are 32 or less.

[0144] According to an embodiment, the Figure 8 transform kernels can be set in advance via offline training.

[0145] According to an embodiment, the AMT index can be defined with one index that can indicate a combination of a horizontal transform and a vertical transform at the same time. Alternatively, the AMT index can be defined with a horizontal transform index and a vertical transform index, respectively.

[0146] As with the above-described AMT, a scheme of applying a transform selected from among a plurality of kernels (e.g., DCT-2, DST-7, and DCT-8) can be expressed as multiple transform selection (MTS) or enhanced multiple transform (EMT), and an AMT index can be expressed as an MTS index.

[0147] Figure 9 is a flowchart illustrating encoding to which AMT is applied according to an embodiment of the present disclosure. Figure 9 The operation of can be performed by the transformer 120 of the encoding apparatus 100.

[0148] Although the present disclosure basically describes application of a transform with respect to a horizontal direction and a vertical direction, respectively, a transform combination can consist of a non-separable transform.

[0149] Alternatively, a separable transform and a non-separable transform can be mixed. In this case, if a non-separable transform is used, transform selection in a row / column direction or selection in a horizontal / vertical direction is not necessary, and only when a separable transform is selected, a transform combination of Figure 8 can be used.

[0150] In addition, the scheme proposed in the present disclosure can be applied regardless of whether it is a primary transform or a secondary transform. In other words, there is no restriction that either one should be applied but both can be applied. Here, the primary transform can mean a transform for first transforming a residual block, and the secondary transform can mean a transform applied to a block resulting from the primary transform.

[0151] First, the encoding apparatus 100 can determine a transform configuration group corresponding to a current block (S910). Here, the transform configuration group can consist of a combination as shown in Figure 8 .

[0152] The encoding apparatus 100 can perform a transform on a candidate transform combination available in the transform configuration group (S920).

[0153] As a result of performing the transform, the encoding apparatus 100 can determine or select a transform combination having a minimum rate-distortion (RD) cost (S930).

[0154] The encoding apparatus 100 can encode a transform combination index corresponding to the selected transform combination (S940).

[0155] Figure 10 is a flowchart illustrating decoding to which AMT is applied according to an embodiment of the present disclosure. Figure 10 The operation of can be performed by the inverse transformer 230 of the decoding apparatus 200.

[0156] First, the decoding apparatus 200 can determine a transform configuration group for a current block (S1010). The decoding apparatus 200 can parse (or obtain) a transform combination index from a video signal, where the transform combination index can correspond to any one of a plurality of transform combinations in the transform configuration group (S1020). For example, the transform configuration group can include DCT-2, DST-7, or DCT-8.

[0157] The decoding apparatus 200 can invoke a transform combination corresponding to the transform combination index (S1030). Here, the transform combination can consist of a horizontal transform and a vertical transform, and can include at least one of DCT-2, DST-7, or DCT-8. In addition, as the transform combination, the above-described transform combinations can be used. Figure 8

[0158] The decoding apparatus 200 can perform an inverse transform on the current block based on the invoked transform combination (S1040). In the case where the transform combination consists of a row (horizontal) transform and a column (vertical) transform, the row (horizontal) transform can be applied first, and then the column (vertical) transform can be applied. However, the disclosure is not limited thereto, and the reverse manner thereof can be applied, or if only an inseparable transform consists, the inseparable transform can be applied immediately.

[0159] According to an embodiment, if the vertical transform or the horizontal transform is DST-7 or DCT-8, the inverse transform of DST-7 or the inverse transform of DCT-8 can be applied by column and then by row. In addition, in the vertical transform or the horizontal transform, different transforms can be applied by column and / or by row.

[0160] According to an embodiment, the transform combination index can be obtained based on an AMT flag indicating whether to perform an AMT. In other words, the transform combination index can be obtained only when the AMT is performed according to the AMT flag. In addition, the decoding apparatus 200 can identify whether the number of non-zero transform coefficients is greater than a threshold value. At this time, the transform combination index can be parsed only when the number of non-zero transform coefficients is greater than the threshold value.

[0161] According to an embodiment, the AMT flag or the AMT index can be defined at the level of at least one of a sequence, a picture, a slice, a block, a coding unit, a transform unit, or a prediction unit.

[0162] Further, according to another embodiment, the steps of determining the transform configuration group and parsing the transform combination index can be performed simultaneously. Alternatively, the step S1010 can be pre-set in the encoding apparatus 100 and / or the decoding apparatus 200 and omitted.

[0163] Figure 11 is a flowchart illustrating an example of encoding an AMT flag and an AMT index according to an embodiment of the disclosure.​Figure 11 The operation of the encoding device 100 can be performed by the transformer 120.

[0164] The encoding device 100 can determine whether to apply the AMT to the current block (S1110).

[0165] If the AMT is applied, the encoding device 100 can perform encoding in the case where the AMT flag = 1 (S1120).

[0166] The encoding device 100 can determine the AMT index based on at least one of the prediction mode, the horizontal transform, or the vertical transform of the current block (S1130). Here, the AMT index means an index indicating any one of a plurality of transform combinations for each intra prediction mode, and the AMT index can be transmitted per transform unit.

[0167] When the AMT index is determined, the encoding device 100 can encode the AMT index (S1140).

[0168] On the other hand, unless the AMT is applied, the encoding device 100 can perform encoding in the case where the AMT flag = 0 (S1150).

[0169] Figure 12 is a flowchart illustrating decoding for performing a transform based on an AMT flag and an AMT index.

[0170] The decoding device 200 can parse the AMT flag from the bitstream (S1210). Here, the AMT flag can indicate whether the AMT is applied to the current block.

[0171] The decoding device 200 can identify whether the AMT is applied to the current block based on the AMT flag (S1220). For example, the decoding device 200 can identify whether the AMT flag is 1.

[0172] If the AMT flag is 1, the decoding device 200 can parse the AMT index (S1230). Here, the AMT index means an index indicating any one of a plurality of transform combinations for each intra prediction mode, and the AMT index can be transmitted per transform unit. Alternatively, the AMT index can mean an index indicating any one of transform combinations defined in a preset transform combination table. The preset transform combination table can mean Figure 8 but the present disclosure is not limited thereto.

[0173] The decoding device 200 can cause or determine the horizontal transform and the vertical transform based on at least one of the AMT index or the prediction mode (S1240).

[0174] Alternatively, the decoding apparatus 200 can cause a transform combination corresponding to the AMT index. For example, the decoding apparatus 200 can cause or determine a horizontal transform and a vertical transform corresponding to the AMT index.

[0175] Further, if the AMT flag is 0, the decoding apparatus 200 can apply a preset vertical inverse transform by column (S1250). For example, the vertical inverse transform can be an inverse transform of DCT-2.

[0176] The decoding apparatus 200 can apply a preset horizontal inverse transform by row (S1260). For example, the horizontal inverse transform can be an inverse transform of DCT-2. That is, when the AMT flag is 0, a preset transform kernel can be used in the encoding apparatus 100 or the decoding apparatus 200. For example, a widely used transform kernel can be used, rather than a transform kernel defined in a transform combination table as shown in FIG. 6. Figure 8

[0177] NSST (Non-Separable Secondary Transform)

[0178] The secondary transform indicates that a transform kernel is applied again using an application result of a primary transform as an input. The primary transform can include DCT-2 or DST-7 in the HEVC or the AMT described above. The non-separable transform indicates that an N2×N2 transform kernel is applied only once to an N2×1 vector after an NxN two-dimensional residual block is viewed as the N2×1 vector, rather than sequentially applying an N×N transform kernel to a row direction and a column direction.

[0179] That is, the NSST can indicate a non-separable square matrix applied to a vector consisting of a coefficient array of a transform block. In addition, although the description of the embodiments of the disclosure focuses on the NSST as an example of a non-separable transform applied to an upper left region (low frequency region) determined according to a block size, the embodiments of the disclosure are not limited to the term "NSST", but any type of non-separable transform can be applied to the embodiments of the disclosure. For example, a non-separable transform applied to an upper left region (low frequency region) determined according to a block size can be expressed as a low frequency non-separable transform (LFNST). In the disclosure, an M×N transform (or transform matrix) means a matrix consisting of M rows and N columns.

[0180] ​In the NSST, the two-dimensional block data obtained by applying the primary transform is split into MxM blocks, and then an M2xM2 non-separable transform is applied to each MxM block. M can be, for example, 4 or 8. The NSST can be applied only to certain regions, not to all regions in the two-dimensional block obtained by the primary transform. For example, the NSST can be applied only to the top-left 8x8 block. In addition, the 64x64 non-separable transform can be applied to the top-left 8x8 region only when both the width and the height of the two-dimensional block obtained by the primary transform are 8 or more, and the remaining regions can be split into 4x4 blocks and the 16x16 non-separable transform can be applied to each of the 4x4 blocks.

[0181] The M2xM2 non-separable transform can be applied in the form of a matrix product, but in order to reduce the computational load and memory requirement, it can be approximated by a combination of Givens rotation layers and permutation layers. Figure 13 One Givens rotation is exemplified. As shown in Figure 13 This is described with one angle of a Givens rotation.

[0182] Figure 13 is a diagram exemplifying a Givens rotation according to an embodiment of the present disclosure, and Figure 14 is a diagram exemplifying a configuration of one round in a 4x4 NSST composed of permutation and Givens rotation layers according to an embodiment of the present disclosure.

[0183] Both the 8x8 NSST and the 4x4 NSST can be composed of a hierarchical combination of Givens rotation layers. The matrix corresponding to one Givens rotation is shown in Equation 1, and the matrix product can be represented in a diagram as shown in Figure 13

[0184] [Equation 1]

[0185]

[0186] In Figure 13 , t m and t n output by a Givens rotation can be calculated as in Equation 2. m m n m m n

[0187] [Equation 2]

[0188] m m n m m n

[0189] m m n

[0190] Since as shown in Figure 13One Givens rotation rotates two data, and thus 32 or 8 Givens rotations are required to process 64 data (in the case of 8x8 NSST) or 16 data (in the case of 4x4 NSST), respectively. Thus, a bundle of 32 or 8 Givens rotations can form a Givens rotation layer. As Figure 14 As shown in FIG. 2, the output data of one Givens rotation layer is transferred as the input data of the next Givens rotation layer through permutation (or reshuffling). As Figure 14 As shown in FIG. 2, the permutation pattern is regularly defined, and in the case of 4x4 NSST, four Givens rotation layers and their corresponding permutations form a round. 4x4 NSST is performed for two rounds, and 8x8 NSST is performed for four rounds. Although different rounds use the same permutation pattern, different Givens rotation angles are applied. Thus, angle data of all Givens rotations constituting each transform need to be stored.

[0191] In the last step, the data output via the Givens rotation layer is re-permuted, and information about the permutation is stored per transform, respectively. The permutation is performed at the end of the forward NSST, and inverse permutation is applied first to the inverse NST.

[0192] The inverse NSST performs the Givens rotation layers and permutations applied to the forward NSST in reverse order, and takes a negative (-) value for the angle of each Givens rotation to perform rotation.

[0193] Figure 15 An example configuration of a non-separable transform set for each intra prediction mode according to an embodiment of the disclosure is illustrated.

[0194] Intra prediction modes to which the same NSST or NSST set is applied can form a group. In Figure 15 In FIG. 2, 67 intra prediction modes are classified into 35 groups. For example, the mode numbered 20 and the mode numbered 48 both belong to the group numbered 20 (hereinafter, mode group).

[0195] For each mode group, multiple NSSTs rather than one NSST can be configured as a set. Each set can include a case where no NSST is applied. For example, in the case where three different NSSTs can be applied to one mode group, one of four cases including a case where no NSST is applied can be selected. At this time, an index for distinguishing one of the four cases can be transmitted in each TU. The number of NSSTs can be configured to be different for each mode group. For example, the mode group numbered 0 and the mode group numbered 1 can be signaled to select one of three cases including a case where no NSST is applied, respectively.

[0196] Embodiment 1 : RST for 4x4 block

[0197] The non-separable transform applicable to a 4x4 block is a 16x16 transform. That is, if data elements constituting a 4x4 block are arranged in a row in a row-major or column-major order, it becomes a 16x1 vector, and the non-separable transform can be applied to the 16x1 vector. The forward 16x16 transform is composed of 16 row direction transform basis vectors, and the inner product of the 16x1 vector and each transform basis vector results in a transform coefficient of the transform basis vector. The process of obtaining transform coefficients of all 16 transform basis vectors is to multiply the 16x16 non-separable transform matrix by the input 16x1 vector. The transform coefficients obtained by the matrix multiplication have the form of a 16x1 vector, and the statistical properties of each transform coefficient can be different. For example, if the 16x1 transform coefficient vector is composed of a 0th element to a 15th element, the variance of the 0th element can be greater than that of the 15th element. That is, the element is located in the earlier position, the greater the variance of the element, and thus the greater the energy value.

[0198] If the inverse 16x16 non-separable transform is applied to the 16x1 transform coefficient vector (when the effects of quantization or integerization are ignored), the original 4x4 block signal can be reconstructed. If the forward 16x16 non-separable transform is an orthogonal transform, the inverse 16x16 transform can be obtained by transposing the matrix used for the forward 16x16 transform. In short, the data in the form of a 16x1 vector can be obtained by multiplying the inverse 16x16 non-separable transform matrix by the 16x1 transform coefficient vector, and if it is sorted in the row-major or column-major order as it was applied first, the 4x4 block signal can be reconstructed.

[0199] As described above, the elements of the 16x1 transform coefficient vector can each have different statistical properties. As in the example described above, if the transform coefficients located in the front (close to the zeroth element) have greater energy, a signal very close to the original signal can be reconstructed by applying the inverse transform to some transform coefficients that appear first without even using all the transform coefficients. For example, when the inverse 16x16 non-separable transform is composed of 16 column basis vectors, only L column basis vectors are left to configure a 16xL matrix, and among the transform coefficients, only the more important L transform coefficients (Lx1 vector, which can appear first in the example described above) are left, and then the 16xL matrix is multiplied by the Lx1 vector, thereby enabling the reconstruction of a 16x1 vector that is not much different from the original 16x1 vector data. As a result, only L coefficients are involved in data reconstruction. Therefore, when the transform coefficients are obtained, it is sufficient to obtain an Lx1 transform coefficient vector rather than a 16x1 transform coefficient vector. That is, L row direction transform vectors are selected from the forward 16x16 non-separable transform matrix to configure an Lx16 transform, and then it is multiplied by the 16x1 input vector, thereby obtaining L primary transform coefficients.

[0200] Embodiment 2: Configuration of the application area and the arrangement of transform coefficients for 4x4 RST

[0201] 4x4 RST can be applied as a two-dimensional transform, and in this case, can be applied twice to a block to which a primary transform such as DCT-type 2 has been applied. When the size of the block to which the primary transform is applied is NxN, it is generally larger than 4x4. Thus, when applying 4x4 RST to an NxN block, the following two methods can be considered.

[0202] 4x4 RST can be applied to some of the NxN regions, not all of the NxN regions. For example, 4x4 RST can be applied only to the upper left MxM region (M<=N).

[0203] The region to which the secondary transform is applied can be divided into 4x4 blocks, and 4x4 RST can be applied to each block.

[0204] Methods 1) and 2) can be mixed. For example, only the upper left MxM region can be divided into 4x4 blocks, and then 4x4 RST can be applied.

[0205] In a specific embodiment, a secondary transform can be applied only to the upper left 8x8 region. If the NxN block is equal to or larger than 8x8, 8x8 RS can be applied, and if the NxN block is smaller than 8x8 (4x4, 8x4, or 4x8), it can be divided into 4x4 blocks, and then 4x4 RST can be applied as in 2) above.

[0206] If L transform coefficients (1<=L<=16) are generated after applying 4x4 RST, there is some degree of freedom in how to arrange the L transform coefficients. However, since there can be a certain order in reading and processing transform coefficients in the residual coding section, the coding performance can vary depending on how the L transform coefficients are arranged in the two-dimensional block. In the High Efficiency Video Coding (HEVC) standard, residual coding starts from the position farthest from the DC position, and this is to take advantage of the fact that the farther from the DC position, the more the coefficient value experienced quantization is 0 or close to 0. Thus, in terms of coding performance, it can be advantageous to place the more critical and higher energy coefficients among the L transform coefficients later in the coding order.

[0207] Figure 16 Three forward scanning orders of transform coefficients or transform coefficient blocks applied in the HEVC standard are illustrated, in which (a) illustrates a diagonal scan, (b) illustrates a horizontal scan, and (c) illustrates a vertical scan.

[0208] Figure 16Three forward scanning orders of transform coefficients or transform coefficient blocks (4x4 blocks, coefficient groups (CGs)) applied in the HEVC standard are exemplified. Residual coding is performed in the order opposite to the scanning order of (a), (b), or (c) (i.e., in the order from 16 to 1). The three scanning orders shown in (a), (b), and (c) are selected according to the intra prediction mode. Thus, likewise, for L transform coefficients, the scanning order can be determined according to the intra prediction mode.

[0209] L belongs to the range 1 <= L < 16. Generally, L transform basis vectors can be selected from 16 transform basis vectors by any method. However, as in the example proposed above, from the perspective of encoding and decoding, it can be advantageous in terms of coding efficiency to select transform basis vectors having higher importance in terms of energy.

[0210] Figure 17 The positions of transform coefficients when applying the forward diagonal scanning in case of applying 4x4 RST to a 4x8 block according to an embodiment of the present disclosure are exemplified, and Figure 18 An example of merging the significant transform coefficients of two 4x4 blocks into a single block according to an embodiment of the present disclosure is exemplified.

[0211] If L is 8 when the upper left 4x8 block is split into 4x4 blocks according to the diagonal scanning order of (a) and 4x4 RST is applied (i.e., if only 8 transform coefficients are left among 16 transform coefficients), the transform coefficients can be positioned as shown in Figure 17 where only half of each 4x4 block can have transform coefficients, and the positions marked with X can be filled with 0 by default. Thus, according to the scanning order proposed in (a), L transform coefficients are arranged in each 4x4 block, and residual coding (e.g., residual coding in HEVC) can be applied assuming that the remaining (16-L) positions in each 4x4 block are filled with 0.

[0212] In addition, the L transform coefficients already arranged in two 4x4 blocks as shown in Figure 18 can be configured in one block. Specifically, since one 4x4 block is completely filled with transform coefficients of two 4x4 blocks when L is 8, no transform coefficients are left in the other block. Thus, since residual coding for a 4x4 block having no transform coefficients is not needed, in the case of HEVC, a flag (coded_sub_block_flag) indicating whether residual coding is applied to a block can be coded as 0. There can be various schemes of combining the positions of transform coefficients of two 4x4 blocks. For example, the positions can be combined according to any order, and the following method can also apply.

[0213] 1) The transform coefficients of two 4x4 blocks are alternately combined in scan order. That is, when the transform coefficients of the upper block are and the transform coefficients of the lower block are they can be alternately combined one by one as In addition, and can be exchanged in order (i.e., the can come first).

[0214] 2) The transform coefficients of the first 4x4 block can be arranged first, and then the transform coefficients of the second 4x4 block can be arranged. That is, they can be connected and arranged as Of course, the order can be changed as

[0215] Embodiment 3: Coding method of NSST (Non-Separable Secondary Transform) index for 4x4 RST

[0216] If the 4x4 RST is applied as shown in Figure 17 , the L+1th position to the 16th position can be filled with 0 according to the transform coefficient scan order of each 4x4 block. Therefore, if there is a non-zero value in the L+1th position to the 16th position in either of the two 4x4 blocks, it is inferred that the 4x4 RST is not applied. If the 4x4 RST has a structure in which a transform selected from a transform set prepared as the Joint Exploration Model (JEM) NSST is applied, the index of the transform to be applied can be signaled.

[0217] In some decoders, the NSST index can be known via bitstream parsing, and the bitstream parsing can be performed after residual decoding. In this case, if non-zero transform coefficients exist between the L+1th position and the 16th position by residual decoding, the decoder can avoid parsing the NSST index because it is confirmed that the 4x4 RST is not applied. Therefore, the signaling cost can be reduced only when necessary by optionally parsing the NSST index.

[0218] If the 4x4 RST is applied as shown in Figure 17 ​If multiple 4×4 blocks in a specific region are to be 4×4RST (at this time, the same or different 4×4RSTs can be applied), then the (same or different) 4×4RSTs applied to all 4×4 blocks can be specified via an NSST index. Since the 4×4RST and whether to apply 4×4RST are determined for all 4×4 blocks via an NSST index, if the result of checking for the presence of non-zero transform coefficients at positions L+1 to 16 for all 4×4 blocks is that non-zero transform coefficients exist at unallowed positions (positions L+1 to 16) during the residual decoding process, then the encoding device 100 can be configured not to encode the NSST index.

[0219] The encoding device 100 can separately signal the corresponding NSST indexes for the luma block and chroma block, and can separately signal the corresponding NSST indexes for the Cb component and Cr component. In the case of the chroma block, a common NSST index can be used. When using a single NSST index, the signaling of the NSST index is performed only once. If the Cb component and Cr component share a single NSST index, a 4×4 RST indicated by the same NSST index can be applied. In this case, the 4×4 RSTs for the Cb component and Cr component can be the same, or, although using the same NSST index, independent 4×4 RSTs can be set for the Cb component and Cr component. When using a shared NSST index for the Cb component and Cr component, it checks whether there are non-zero transform coefficients at positions L+1 to 6 for all 4×4 blocks of the Cb component and Cr component. If non-zero transform coefficients are found at positions L+1 to 16, the signaling of the NSST index can be skipped.

[0220] Even when such Figure 18 When the transform coefficients of two 4×4 blocks are merged into one 4×4 block, the encoding device 100 can check whether a non-zero transform coefficient appears at a position where no valid transform coefficients exist when applying 4×4RST, and then determine whether to signal the NSST index. Specifically, when L is 8 and therefore when 4×4RST is applied, in such a position... Figure 18 A 4×4 block as shown ( Figure 18In the case where there is no valid transform coefficient in the position marked with X in (b) of the block, the flag (coded_sub_block_flag) on whether to apply residual coding to the block can be checked, and if 1, the NSST index can not be signaled. As described above, although the NSST is described as an example non-separable transform below, other known terms (e.g., LFNST) can be used for the non-separable transform. For example, the NSST set and the NSST index can be used interchangeably with the LFNST set and the LFNST index, respectively. In addition, the RST as described herein is an example of a non-separable transform (e.g., LFNST) using a non-square transform matrix having a reduced output length and / or a reduced input length in a square non-separable transform matrix applied to at least a certain region (the upper left 4x4, 8x8 region or the remaining region except for the lower right 4x4 region in the 8x8 block) in the transform block, and the RST can be used interchangeably with the LFNST.

[0221] Embodiment 4: Optimization method in case of coding of 4x4 index before residual coding

[0222] In the case where the encoding of the NSST index is performed before the residual coding, it is predetermined whether to apply the 4x4 RST. Accordingly, the residual coding on the position where the transform coefficient is padded with 0 can be omitted. Here, whether to apply the 4x4 RST can be determined via the NSST index (e.g., if the NSST index is 0, the 4x4 RST is not applied), otherwise, whether to apply the 4x4 RST can be signaled via a separate syntax element (e.g., NSST flag). For example, if the separate syntax element is the NSST flag, the decoding apparatus 200 first parses the NSST flag, thereby determining whether to apply the 4x4 RST. Then, if the NSST flag is 1, the residual coding (decoding) on the position where there can be no valid transform coefficient can be omitted as described above.

[0223] In the case of HEVC, at the time of residual coding, the encoding is first performed on the last non-zero coefficient position in the TU. If the encoding of the NSST index is performed after the encoding of the last non-zero coefficient position, and the last non-zero coefficient position is a position where there is no non-zero coefficient in the case where the 4x4 RST is assumed to be applied, the decoding apparatus 200 can be configured not to apply the 4x4 RST without decoding the NSST index. For example, since there is no valid transform coefficient in the position marked with X (padded with 0) when the 4x4 RST is applied, if the last non-zero coefficient is in the region marked with X, the decoding apparatus 200 can skip the encoding of the NSST index. If the last non-zero coefficient is not in the region marked with X, the decoding apparatus 200 can perform the encoding of the NSST index. Figure 17 ​

[0224] If we know whether to apply 4×4 RST by conditionally encoding the NSST index after encoding the non-zero coefficient positions, then the remaining residual encoding can be handled in the following two ways:

[0225] 1) Perform regular residual coding without applying 4×4RST. That is, perform coding assuming that there may be non-zero transform coefficients at any position from the last non-zero coefficient position to DC.

[0226] 2) When applying 4×4RST, at specific locations or specific 4×4 blocks (e.g., those that are padded with 0 by default), Figure 17 At position X in the code, there are no transform coefficients. Therefore, the residual encoding for that position or block can be omitted. For example, in the code based on... Figure 17 When scanning in a specific order and reaching the position marked X, the encoding of the HEVC standard's flag (sig_coeff_flag) indicating the presence of a non-zero coefficient at that position can be omitted. In cases such as... Figure 18 In the case where the transform coefficients of the two blocks shown are merged into one block, the encoding of the flag indicating whether to apply residual coding to the 4×4 block filled with 0s (e.g., coded_sub_block_flag in the HEVC standard) can be omitted, and the value can be 0, and the 4×4 block can be filled with 0s without separate encoding.

[0227] When encoding the NSST index after encoding the last non-zero coefficient position, if the x-position (Px) and y-position (Py) of the last non-zero coefficient are less than Tx and Ty, respectively, the encoding of the NSST index is omitted, and 4×4 RST can be omitted. For example, if Tx = 1, Ty = 1, and there is a last non-zero coefficient at the DC position, the NSST index encoding is omitted. This scheme of determining whether to perform NSST index encoding by comparison with a threshold can be applied differently to the luma and chroma components. For example, different Tx and Ty can be applied to the corresponding one of the luma and chroma components, and the threshold can be applied to the luma component but not the chroma component. Conversely, the threshold can be applied to the chroma component but not the luma component.

[0228] Both methods can be applied simultaneously (NSST index coding is omitted if the last non-zero coefficient is in a region where no valid transform coefficients exist, and NSST index coding is omitted when both the X and Y coordinates of the last non-zero coefficient are less than a threshold). For example, first identify the threshold comparison of the position coordinates of the last non-zero coefficient, then check whether the last non-zero coefficient is in a region where no valid transform coefficients exist, and these two methods can be interchanged in order.

[0229] The method proposed in embodiment 4) can also be applied to 8x8 RST. That is, if the last non-zero coefficient is in a region other than the top-left 4x4 region in the top-left 8x8 region, the NSST index coding can be omitted, otherwise, the NSST index coding can be performed. In addition, if both the X and Y coordinates of the position of the last non-zero coefficient are smaller than a certain threshold, the NSST index coding can be omitted. These two methods can be performed simultaneously.

[0230] Embodiment 5: Application of different coding and residual coding of NSST index to each of the luminance component and the chrominance component when applying RST Embodiment 6

[0231] The schemes described above in connection with embodiments 3 and 4 can be applied differently to the luma component and the chroma component. That is, different NSST index coding and residual coding schemes can be applied to the luma component and the chroma component. For example, the scheme described above in connection with embodiment 4 can be applied to the luma component, and the scheme described above in connection with embodiment 3 can be applied to the chroma component. Furthermore, the conditional NSST index coding proposed in embodiments 3 or 4 can be applied to the luma component, while the conditional NSST index coding can not be applied to the chroma component, or vice versa (the conditional NSST index coding is applied to the chroma component but not to the luma component).

[0232] Embodiment 7

[0233] According to embodiments of the present disclosure, a mixed NSST transform set (MNTS) applying various NSST conditions during the process of applying NSST and a method of configuring the MNTS are provided.

[0234] According to JEM, depending on the pre-selected low block size, the 4x4 NSST set contains only 4x4 kernels, and the 8x8 NSST set contains only 8x8 kernels. According to embodiments of the present disclosure, a method of configuring a mixed NSST set is also proposed as follows.

[0235] The NSST set can include NSST kernels available in the NSST set and have one or more variable sizes, rather than fixed sizes (for example, both 4x4 NSST kernels and 8x8 NSST kernels are included in one NSST set).

[0236] The number of NSST kernels available in the NSST set can not be fixed, but can vary (for example, the first set includes three kernels, and the second set includes four kernels).

[0237] - The order of the NSST kernels can be variable, not fixed, according to the NSST set (e.g., in a first set, NSST kernels 1, 2 and 3 are mapped to NSST indices 1, 2 and 3, respectively, but in a second set, NSST kernels 3, 2 and 1 are mapped to NSST indices 1, 2 and 3, respectively).

[0238] More specifically, the following is an example method of configuring a hybrid NSST transform set.

[0239] - The priority of the NSST kernels available in the NSST transform set can be determined according to the NSST kernel size (e.g., 4x4 NSST and 8x8 NSST).

[0240] For example, if the block is large, the 8x8 NSST kernel can be more important than the 4x4 NSST kernel. Thus, a small value of the NSST index is assigned to the 8x8 NSST kernel.

[0241] - The priority of the NSST kernels available in the NSST transform set can be determined according to the order of the NSST kernels.

[0242] For example, a given 4x4 NSST first kernel can be prioritized over a 4x4 NSST second kernel.

[0243] Since the NSST index is encoded and transmitted, a higher priority (smaller index) can be assigned to a more frequent NSST kernel so that the NSST index can be signaled with fewer bits.

[0244] The following Tables 1 and 2 represent example hybrid NSST sets proposed according to the present embodiment.

[0245] [Table 1]

[0246]

[0247] [Table 2]

[0248]

[0249] Figure 19

[0250] According to embodiments of the present disclosure, a method of determining a NSST set is proposed that takes into account the block size and the intra prediction mode during the process of determining the secondary transform set.

[0251] The method proposed in the present embodiment configures the transform set adapted to the intra prediction mode associated with embodiment 6, thus allowing to configure and apply to the block kernels of various sizes.

[0252] Figure 19An example method of configuring a hybrid NSST set according to each intra prediction mode is exemplified.

[0253] Figure 19 An example table according to application of the method proposed in Embodiment 2 in association with Embodiment 6 is exemplified. In other words, as Figure 19 As shown in Embodiment 6, an index indicating whether each intra prediction mode follows the conventional NSST set configuration method or the other NSST set configuration method (‘hybrid type’) can be defined.

[0254] More specifically, in the case of the intra prediction mode whose index (‘hybrid type’) is defined as ‘1’, the NSST set configuration method of JEM is not followed, but the NSST set is configured using the NSST set configuration method defined in the system. Here, the NSST set configuration method defined in the system can mean the hybrid NSST set proposed in Embodiment 6. Figure 19

[0255] As another embodiment, although two types of transform set configuration methods based on the hybrid type information (flag) related to the intra prediction mode (the hybrid type NSST set configuration method according to the embodiments of the disclosure and the NSST set configuration based on JEM) are described in association with the table of Figure 20

[0256] In another embodiment, both the intra prediction mode and the transform block size can be considered to determine whether to configure the transform set suitable for the current block in the hybrid type. For example, if the mode type corresponding to the intra prediction mode is 0, the NSST set configuration of JEM is followed, otherwise (mode type == 1), the NSST set of various hybrid types can be determined according to the transform block size.

[0257] Embodiment 8 An example method of selecting the NSST set (or kernel) according to the size of the transform block and the intra prediction mode according to the embodiments of the disclosure is exemplified.

[0258] When determining the transform set, the decoding apparatus 200 can use the NSST index information to determine the used NSST kernel.

[0259] Embodiment 9: Simplified transform

[0260] ​​According to embodiments of the present disclosure, there are provided methods of efficiently coding NSST indices considering the statistical distribution change of the NSST indices transmitted after encoding when configuring the transform set considering both the intra prediction mode and the block size during the process of applying the secondary transform. According to embodiments of the present disclosure, there are provided methods of selecting a kernel to be applied using syntax indicating the kernel size.

[0261] According to embodiments of the present disclosure, since the number of available NSST kernels is different for each transform set, there are also provided a truncated unary binarization method as shown in Table 3 below depending on the maximum NSST index value available for efficient binarization for each set.

[0262] [Table 3]

[0263]

[0264] Table 3 represents the binarization of the NSST indices. Since the number of available NSST kernels is different for each transform set, the NSST indices can be binarized according to the maximum NSST index value.

[0265] Figure 21a

[0266] Due to complexity issues (e.g., large block transform or non-separable transform), there are provided simplified transforms applicable to core transforms (e.g., DCT or DST) and secondary transforms (e.g., NSST).

[0267] The main idea of the simplified transform is to map an N-dimensional vector to an R-dimensional vector in another space, where R / N (R < N) is the simplification factor. The simplified transform is an R x M matrix as represented in Equation 3 below.

[0268] [Equation 3]

[0269]

[0270] In Equation 1, the R rows of the transform are the R bases in the new N-dimensional space. Thus, the simplified transform is named as such because the number of elements of the vector output by the transform is smaller than the number of elements of the input vector (R < N). The inverse transform matrix for the simplified transform is the transpose of the forward transform. Below, forward and inverse simplified transforms are described with reference to Figure 21b and Figure 21a The forward and inverse simplified transforms according to embodiments of the present disclosure are exemplified.

[0271] Figure 21b and Figure 22 The forward and inverse simplified transforms according to embodiments of the present disclosure are exemplified.

[0272] The number of elements in the simplified transform is R x N, which is R / N as small as the full matrix size (N x N), which means the required memory is R / N of the full matrix.

[0273] In addition, the number of required products is R x N, which is R / N as small as the original N x N.

[0274] If X is an N-dimensional vector, R coefficients are obtained after applying the simplified transform, which means that it is sufficient to transmit only R values instead of the N coefficients originally intended to be transmitted.

[0275] Figure 22 is a flowchart illustrating an example of decoding using a simplified transform according to an embodiment of the disclosure.

[0276] As shown in Figure 23 As shown in

[0277] As shown in Table 4 and Table 5, operators used in the disclosure below are defined.

[0278] [Table 4]

[0279]

[0280] [Table 5]

[0281]

[0282] Figure 23 is a flowchart illustrating an example of applying a conditional simplified transform according to an embodiment of the disclosure. Figure 23 The operations of can be performed by the inverse quantizer 140 and the inverse transformer 150 of the decoding apparatus 200.

[0283] According to an embodiment, when a certain condition is satisfied, a simplified transform can be used. For example, a simplified transform can be applied to a block greater than a predetermined size as follows.

[0284] - width > TH && height > HT (where TH is a predefined value (e.g., 4))

[0285] Alternatively,

[0286] - width x height > K && MIN(width, height) > TH (K and TH are predefined values)

[0287] That is, when the width of the current block is greater than a predefined value (TH) and the height of the current block is greater than a predefined value (TH) as in the above conditions, the simplified transform can be applied. Or, when the product of the width and the height of the current block is greater than a predetermined value (K) and the smaller one of the width and the height of the current block is greater than a predefined value (TH), the simplified transform can be applied.

[0288] The simplified transform can be applied to a set of predetermined blocks as follows.

[0289] - width == TH && height == TH

[0290] Or,

[0291] - width == height

[0292] That is, if the width and the height of the current block are each identical to a predetermined value (TH) or the width and the height of the current block are identical (when the current block is a square block), the simplified transform can be applied.

[0293] Unless the condition for using the simplified transform is satisfied, a regular transform can be applied. The regular transform can be a transform that is predefined and available in a video encoding system. An example of the regular transform is as follows.

[0294] - DCT-2, DCT-4, DCT-5, DCT-7, DCT-8

[0295] Or,

[0296] - DST-1, DST-4, DST-7

[0297] Or,

[0298] - Non-separable transform

[0299] Or,

[0300] - JEM-NSST (HyGT)

[0301] As shown in Figure 23 , the simplified transform can rely on an index (Transform_idx) indicating which transform (e.g., DCT-4 or DST-1) will be used or which kernel will be applied (when multiple kernels are available). Specifically, Transmission_idx can be transmitted twice. Once is an index (Transform_idx_h) indicating a horizontal transform, and the other is an index (Transform_idx_v) indicating a vertical transform.

[0302] More specifically, referring to Figure 24, the decoding apparatus 200 performs inverse quantization on the input bitstream (S2305). Thereafter, the decoding apparatus 200 determines whether to apply a transform (S2310). The decoding apparatus 200 can determine whether to apply the transform via a flag indicating whether to skip the transform.

[0303] In a case where the transform is applied, the decoding apparatus 200 parses a transform index (Transform_idx) indicating that the transform is to be applied (S2315). Alternatively, the decoding apparatus 200 can select a transform kernel (S2330). For example, the decoding apparatus 200 can select a transform kernel corresponding to the transform index (Transform_idx). Also, the decoding apparatus 200 can select the transform kernel in consideration of a block size (width, height), an intra prediction mode, or CIdx (luma, chroma).

[0304] The decoding apparatus 200 determines whether a condition for applying a simplified transform is satisfied (S2320). The condition for applying the simplified transform can include the above-described condition. When the simplified transform is not applied, the decoding apparatus 200 can apply a regular inverse transform (S2325). For example, in step S2330, the decoding apparatus 200 can determine an inverse transform matrix from the selected transform kernel, and can apply the determined inverse transform matrix to the current block including the transform coefficients.

[0305] When the simplified transform is applied, the decoding apparatus 200 can apply a simplified inverse transform (S2335). For example, in step S2330, the decoding apparatus 200 can determine a simplified inverse transform matrix from the selected transform kernel in consideration of a simplification factor, and can apply the simplified inverse transform matrix to the current block including the transform coefficients.

[0306] Figure 24 is a flowchart illustrating a decoding example of a secondary inverse transform for applying a conditional simplified transform according to an embodiment of the disclosure. Figure 24 The operations of can be performed by the inverse transformer 230 of the decoding apparatus 200.

[0307] According to an embodiment, a simplified transform can be applied to a secondary transform, as shown in Figure 24 If the NSST index is parsed, the simplified transform can be applied.

[0308] Referring to Embodiment 10: Simplified transform as a secondary transform with different block sizes , the decoding apparatus 200 performs inverse quantization (S2405). The decoding apparatus 200 determines whether to apply NSST to the transform coefficients generated via the inverse quantization (S2410). That is, the decoding apparatus 200 determines whether to need to parse the NSST index (NSST_indx) according to whether the NSST is to be applied.

[0309] When the NSST is applied, the decoding apparatus 200 parses the NSST index (S2415) and determines whether the NSST index is greater than 0 (S2420). The NSST index can be reconstructed by the entropy decoder 210 via a scheme such as CABAC. When the NSST index is 0, the decoding apparatus 200 can omit the secondary inverse transform and apply the core inverse transform or the main inverse transform (S2445).

[0310] In addition, when the NSST is applied, the decoding apparatus 200 selects a transform kernel of the secondary inverse transform (S2435). For example, the decoding apparatus 200 can select a transform kernel corresponding to the NSST index (NSST_idx). In addition, the decoding apparatus 200 can select the transform kernel in consideration of a block size (width, height), an intra prediction mode, or CIdx (luma, chroma).

[0311] When the NSST index is greater than 0, the decoding apparatus 200 determines whether a condition for applying a simplified transform is satisfied (S2425). The condition for applying the simplified transform can include the above-described condition. When the simplified transform is not applied, the decoding apparatus 200 can apply a regular secondary inverse transform (S2430). For example, in step S2435, the decoding apparatus 200 can determine a secondary inverse transform matrix from the selected transform kernel, and can apply the determined secondary inverse transform matrix to the current block including the transform coefficients.

[0312] When the simplified transform is applied, the decoding apparatus 200 can apply a simplified secondary inverse transform (S2440). For example, in step S2335, the decoding apparatus 200 can determine a simplified inverse transform matrix from the selected transform kernel in consideration of a simplification factor, and can apply the simplified inverse transform matrix to the current block including the transform coefficients. Thereafter, the decoding apparatus 200 applies the core inverse transform or the main inverse transform (S2445).

[0313] Figure 25a

[0314] Figure 25b 、 Figure 26a 、 Figure 26b and Figure 25a Examples of the simplified transform and the simplified inverse transform according to the embodiments of the disclosure are illustrated.

[0315] According to the embodiments of the disclosure, for different block sizes such as 4x4, 8x8, or 16x16, the simplified transform can be used as the secondary transform and the secondary inverse transform in a video codec. For example, for an 8x8 block size and a downsizing factor R=16, the secondary transform and the secondary inverse transform can be set as shown in Figure 25b and Figures 26a to 26b .

[0316] may be set as shown in Embodiment 11 : Simplified transform as a secondary transform with non-rectangular shapePseudo code for the simplified transform and the simplified inverse transform as shown in FIG. 4.

[0317] Figure 27

[0318] Figure 27 An example region to which a simplified secondary transform is applied according to an embodiment of the disclosure is illustrated.

[0319] As described above, due to complexity issues, the secondary transform can be applied to the corner of 4x4 and 8x8. The simplified transform can be applied to a non-square shape.

[0320] As Figure 27 As shown in FIG. 5, the RST can be applied only to a certain region (a shaded region) of a block. In Embodiment 12: Simplification factor In FIG. 5, each square represents a 4x4 region, and the RST can be applied to 10 4x4 pixels (i.e., 160 pixels). In the case of a simplified factor R=16, the entire RST matrix is a 16x16 matrix, and this can be an acceptable amount of calculation.

[0321] In another example, in the case of applying the RST to an 8x8 block, the non-separable transform (RST) can be applied only to the remaining three 4x4 blocks (a total of 48 transform coefficients) of the upper left, upper right, and lower left except for the lower right 4x4 block.

[0322] Figure 28

[0323] Embodiment 13: High level syntax A simplified transform according to a simplified factor according to an embodiment of the disclosure is illustrated.

[0324] A change in the simplified factor causes a change in memory and multiplication complexity. As described above, due to the change in the simplified factor, the memory and multiplication complexity can be reduced by a factor of R / N. For example, in the case of R=16 for an 8x8 NSST, the memory and multiplication complexity can be reduced by 1 / 4.

[0325] Embodiment 14: Conditional application of 4x4 RST for worst case handling

[0326] The syntax elements represented in Table 6 below can be used in video encoding to process the RST. The semantics related to the simplified transform can exist in the sequence parameter set (SPS) or the slice header.

[0327] Reduced transform enabled flag is equal to 1 specifies that reduced transform is enabled and applied. Reduced transform enabled flag is equal to 0 specifies that reduced transform is not enabled. When reduced transform enabled flag is not present, it is inferred to be equal to 0.

[0328] Reduced transform factor specifies the number of reduced dimensions to be kept for the reduced transform. When reduced transform factor is not present, it is inferred to be equal to R.

[0329] Min reduced transform size specifies the minimum transform size for which the reduced transform is applied. When min reduced transform size is not present, it is inferred to be equal to 0.

[0330] Max reduced transform size specifies the maximum transform size for which the reduced transform is applied. When max reduced transform size is not present, it is inferred to be equal to 0.

[0331] Reduced transform factor specifies the number of reduced dimensions to be kept for the reduced transform. When reduced transform factor is not present, it is inferred to be equal to 0.

[0332] [Table 6]

[0333]

[0334]

[0335] Total multiplications

[0336] A non-separable quadratic transform (4x4 NSST) applicable to a 4x4 block is a 16x16 transform. The 4x4 NSST is applied to a block that has undergone a primary transform such as DCT-2, DST-7, or DCT-8. When the size of a block to which a primary transform is applied is NxM, the following methods can be considered when applying the 4x4 NSST to the NxM block.

[0337] 1) The following are conditions a) and b) for applying the 4x4 NSST to an NxM region.

[0338] a) N >= 4

[0339] b) M >= 4

[0340] 2) The 4x4 NSST can be applied to some (not all) NxM regions. For example, the 4x4 NSST can be applied only to a top-left KxJ region. The following a) and b) are conditions for this case.

[0341] a) K >= 4

[0342] b) J >= 4

[0343] 3) A region to which a secondary transform is to be applied can be divided into 4x4 blocks, and the 4x4 NSST can be applied to each block.

[0344] The computational complexity of the 4x4 NSST is a very critical consideration for an encoder and a decoder, and thus is analyzed in detail. Specifically, the computational complexity of the 4x4 NSST is analyzed based on the number of multiplications. In the case of a forward NSST, a 16x16 secondary transform is composed of 16 row direction transform basis vectors, and an inner product of a 16x1 vector and each transform basis vector results in a transform coefficient of the transform basis vector. The process of obtaining all transform coefficients of the 16 transform basis vectors is multiplying a 16x16 non-separable transform matrix by an input 16x1 vector. Thus, the total number of multiplications required for the 4x4 forward NSST is 256.

[0345] When the inverse 16x16 non-separable transform is applied to the 16x1 transform coefficients in the decoder (when effects such as quantization and integerization are ignored), the coefficients of the original 4x4 primary transform block can be reconstructed. In other words, data in the form of a 16x1 vector can be obtained by multiplying the inverse 16x16 non-separable transform matrix by the 16x1 transform coefficient vector, and if the data is ordered in the row-major or column-major order as it was applied first, the 4x4 block signal (primary transform coefficients) can be reconstructed. Accordingly, the total number of multiplications required for the 4x4 inverse NSST is 256.

[0346] As described above, the number of multiplications required for each sample unit when the 4x4 NSST is applied is 16. This is a value obtained by dividing the total number of multiplications 256 obtained during a process of performing an inner product of each transform basis vector and a 16x1 vector (a process of performing the 4x4 NSST) by the total number of samples 16. The number of multiplications required for both the forward 4x4 NSST and the inverse 4x4 NSST is 16.

[0347] In the case of an 8x8 block, the number of multiplications per sample required when the 4x4 NSST is applied is determined according to the region to which the 4x4 NSST has been applied.

[0348] 1. In the case where the 4x4 NSST is applied only to the upper left 4x4 region: 256 (the number of multiplications necessary for the 4x4 NSST process) / 64 (the total number of samples in the 8x8 block) = 4 multiplications / sample

[0349] 2. In the case where the 4x4 NSST is applied to the upper left 4x4 region and the upper right 4x4 region: 512 (the number of multiplications necessary for two 4x4 NSSTs) / 64 (the total number of samples in the 8x8 block) = 8 multiplications / sample

[0350] 3. In the case where the 4x4 NSST is applied to all 4x4 regions in the 8x8 block: 1024 (the number of multiplications necessary for four 4x4 NSSTs) / 64 (the total number of samples in the 8x8 block) = 16 multiplications / sample

[0351] As described above, if the block size is large, the range to which the 4x4 NSST is applied can be reduced in order to reduce the number of multiplications required per sample end in the worst case.

[0352] Accordingly, if the 4x4 NSST is used, the worst case occurs when the TU size is 4x4. In this case, the following method can reduce the worst case complexity.

[0353] Method 1. The 4x4 NSST is not applied to a smaller TU (i.e., a 4x4 TU).

[0354] Method 2. Apply 4x4 RST instead of 4x4 NST to a 4x4 block (4x4 TU).

[0355] It is observed through experiments that Method 1 causes a serious degradation in coding performance because it does not apply 4x4 NSST. It is revealed that Method 2 is able to reconstruct a signal very close to the original signal by applying inverse transform to some transform coefficients located in the front even without using all transform coefficients in accordance with statistical properties of elements of a 16x1 transform coefficient vector, and thus is able to maintain most of coding performance.

[0356] Specifically, in the case of 4x4 RST, when the inverse (or forward) 16x16 non-separable transform is composed of 16 column basis vectors, only L column basis vectors are left, and a 16xL matrix is configured. Since only L more important transform coefficients are left among transform coefficients, the product of the 16xL matrix and the Lx1 vector can result in reconstruction of a 16x1 vector having a very small difference from the original 16x1 vector data.

[0357] As a result, only L coefficients are involved in data reconstruction. Therefore, in order to obtain transform coefficients, it is sufficient to obtain an Lx1 transform coefficient vector instead of a 16x1 transform coefficient vector. That is, an Lx16 transform matrix is configured by selecting L row direction transform vectors from the forward 16x16 non-separable transform matrix, and L transform coefficients are obtained by multiplying the Lx16 transform matrix with the 16x1 input vector.

[0358] L belongs to the range 1<=L<16. In general, L transform basis vectors can be selected from 16 transform basis vectors by any method. However, as described above, in view of encoding and decoding, it can be advantageous in terms of coding efficiency to select transform basis vectors having higher importance in terms of signal energy. The worst case number of multiplications per sample in a 4x4 block according to the transform for the L value is shown in Table 7 below.

[0359] [Table 7]

[0360] L Multiplication per pixel Embodiment 15: Conditional application of 8x8 RST for worst case handling 16 256 16 8 128 8 4 64 4 2 32 2

[0361] As described above, 4x4 NSST and 4x4 RST can be used in a comprehensive manner as shown in Table 8 below in order to reduce the worst case multiplication complexity. (However, the following example describes a condition for applying 4x4 NSST and 4x4 RST under the condition that 4x4 NSST is applied (i.e., when both the width and height of the current block are equal to or greater than 4)).

[0362] As described above, the 4x4 NSST for a 4x4 block is a square (16x16) transform matrix that receives 16 data and outputs 16 data, and the 4x4 RST means a non-square (8x16) transform matrix that receives 16 data and outputs R (e.g., 8) data less than 16 with respect to the encoder side. The 4x4 RST means a non-square (16x8) transform matrix that receives R (e.g., 8) data less than 16 and outputs 16 data with respect to the decoder side.

[0363] [Table 8]

[0364]

[0365] Referring to Table 8, when the width and height of the current block are 4, the 4x4 RST based on the 8x16 matrix is applied to the current block, otherwise (if the width or height of the current block is not 4), the 4x4 NSST can be applied to the top-left 4x4 region of the current block. More specifically, if the size of the current block is 4x4, the non-separable transform with an input length of 16 and an output length of 8 can be applied. In the case of the inverse non-separable transform, the non-separable transform with an input length of 8 and an output length of 16 can be applied.

[0366] As described above, the 4x4 NSST and the 4x4 RST can be used in combination as shown in Table 9 below in order to reduce the worst case multiplication complexity.

[0367] [Table 9]

[0368]

[0369] Referring to Table 9, when the width and height of the current block are each 4, the 4x4 RST based on the 8x16 matrix is applied, and if the product of the width and height of the current block is less than a threshold (TH), the 4x4 NSST is applied to the top-left 4x4 region of the current block, and if the width of the current block is equal to or greater than the height, the 4x4 NSST is applied to the top-left 4x4 region of the current block and a 4x4 region located to the right of the top-left 4x4 region, and for the remaining regions (when the product of the width and height of the current block is equal to or greater than the threshold and the width of the current block is less than the height), the 4x4 NSST is applied to the top-left 4x4 region of the current block and a 4x4 region located below the top-left 4x4 region.

[0370] As a result, the 4x4 RST (e.g., 8x16 matrix) can be applied to the 4x4 block instead of the 4x4 NSST in order to reduce the computational complexity of the worst case multiplication.

[0371] Total multiplications

[0372] A non-separable secondary transform (8x8 NSST) suitable for an 8x8 block is a 64x64 transform. The 8x8 NSST is applied to a block which has undergone a primary transform such as DCT-2, DST-7, or DCT-8. When the size of a block to which a primary transform is applied is NxM, the following methods can be considered in applying the 8x8 NSST to the NxM block.

[0373] 1) The following are conditions c) and d) for applying the 8x8 NSST to an NxM region.

[0374] c) N >= 8

[0375] d) M >= 8

[0376] 2) The 8x8 NSST can be applied to some (not all) NxM regions. For example, the 8x8 NSST can be applied only to a left upper KxJ region. The following c) and d) are conditions for this case.

[0377] c) K >= 8

[0378] d) J >= 8

[0379] 3) A region to which a secondary transform is to be applied can be divided into 8x8 blocks, and the 8x8 NSST can be applied to each block.

[0380] The computational complexity of the 8x8 NSST is a very critical consideration for an encoder and a decoder, and thus is analyzed in detail. Specifically, the computational complexity of the 8x8 NSST is analyzed based on the number of multiplications. In the case of a forward NSST, a 64x64 secondary transform is composed of 64 row direction transform basis vectors, and the inner product of a 64x1 vector and each transform basis vector results in a transform coefficient of the transform basis vector. The process of obtaining all transform coefficients of the 64 transform basis vectors is multiplying a 64x64 non-separable transform matrix by an input 64x1 vector. Thus, the total number of multiplications required for the 8x8 forward NSST is 4,096.

[0381] When an inverse 64x64 non-separable inverse transform is applied to 64x1 transform coefficients in a decoder (when effects such as quantization and integerization are ignored), the coefficients of the original 8x8 primary transform block can be reconstructed. In other words, data in the form of a 64x1 vector can be obtained by multiplying a 64x64 non-separable transform matrix by a 64x1 transform coefficient vector, and if the data is ordered in the row-major or column-major order as it was applied first, an 8x8 block signal (primary transform coefficients) can be reconstructed. Thus, the total number of multiplications required for the 8x8 inverse NSST is 4,096.

[0382] As described above, the number of multiplications required per sample unit when applying 8x8 NSST is 64. This is a value obtained by dividing the total number of multiplications 4,096, which is obtained during a process of performing an inner product of each transform basis vector and a 64x1 vector (a process of performing 8x8 NSST), by the total number of samples 64. The number of multiplications required for both the forward 8x8 NSST and the inverse 8x8 NSST is 64.

[0383] In the case of a 16x16 block, the number of multiplications per sample required when applying 8x8 NSST is determined according to the region to which 8x8 NSST has been applied.

[0384] 1. In the case of applying 8x8 NSST to only the upper left 8x8 region: 4096 (the number of multiplications necessary for 8x8 NSST processing) / 256 (the total number of samples in a 16x16 block) = 16 multiplications / sample

[0385] 2. In the case of applying 8x8 NSST to the upper left 8x8 region and the upper right 8x8 region: 8192 (the number of multiplications necessary for two 8x8 NSSTs) / 256 (the total number of samples in a 16x16 block) = 32 multiplications / sample

[0386] 3. In the case of applying 8x8 NSST to all 8x8 regions in a 16x16 block: 16384 (the number of multiplications necessary for four 8x8 NSSTs) / 256 (the total number of samples in a 16x16 block) = 64 multiplications / sample

[0387] As described above, if the block size is large, the range to which 8x8 NSST is applied can be reduced to reduce the number of multiplications required per sample end in the worst case.

[0388] In the case of applying 8x8 NSST, since the 8x8 block is the smallest TU to which 8x8 NSST is applicable, the case where the TU size is 8x8 is the worst case in terms of the number of multiplications required from each sample. In this case, the following methods can reduce the worst case complexity.

[0389] Method 1. Do not apply 8x8 NSST to smaller TUs (i.e., 8x8 TUs).

[0390] Method 2. Apply 8x8 RST instead of 8x8 NSST to 8x8 blocks (8x8 TUs).

[0391] It is observed through experiments that Method 1 causes a serious degradation in coding performance because it does not apply 8x8 NSST. It is disclosed that Method 2 is able to reconstruct a signal very close to the original signal by applying inverse transform to some transform coefficients located in the front even without using all transform coefficients in accordance with statistical properties of elements of a 64x1 transform coefficient vector, and thus is able to maintain most of coding performance.

[0392] Specifically, in the case of 8x8 RST, when the inverse (or forward) 64x64 non-separable transform is composed of 16 column basis vectors, only L column basis vectors are left, and a 64xL matrix is configured. Since only L more critical transform coefficients are left among transform coefficients, the product of the 64xL matrix and the Lx1 vector can result in reconstruction of a 64x1 vector having a very small difference from the original 64x1 vector data.

[0393] In addition, as described in Embodiment 11, RST can not be applied to all 64 transform coefficients included in the 8x8 block, but can be applied to a partial region (i.e., a remaining region except for a right-bottom 4x4 region in the 8x8 block).

[0394] As a result, only L coefficients are involved in data reconstruction. Therefore, in order to obtain transform coefficients, it is sufficient to obtain an Lx1 transform coefficient vector instead of a 64x1 transform coefficient vector. That is, an Lx64 transform matrix is configured by selecting L row direction transform vectors from a forward 64x64 non-separable transform matrix, and L transform coefficients are obtained by multiplying the Lx64 transform matrix with a 64x1 input vector.

[0395] The range of the L value can be 1<=L<64, and generally, L vectors can be selected among 64 transform basis vectors in any method, but selecting transform basis vectors having high signal energy importance in encoding and decoding as described above can be beneficial in terms of coding efficiency. The number of multiplications required per sample in an 8x8 block depending on the change of the L value in the worst case is shown in Table 10 below.

[0396] [Table 10]

[0397] L Multiplication per pixel Figure 29 64 4096 64 32 2048 32 16 1024 16 8 512 8 4 256 4

[0398] As described above, 8x8 RST having different L values can be used in a full comprehensive manner as shown in Table 11 below in order to reduce the worst case multiplication complexity. (However, the following example describes a condition for applying 8x8 RST under the condition that 8x8 NSST is applied (i.e., when both the width and height of the current block are equal to or greater than 8)).

[0399] [Table 11]

[0400]

[0401] Referring to Table 11, when the width and height of the current block are each 8, an 8x8 RST based on an 8x64 matrix is applied to the current block, otherwise (if the width or height of the current block is not 8), an 8x8 RST based on a 16x64 matrix can be applied to the current block. More specifically, when the size of the current block is 8x8, an invertible transform with an input length of 64 and an output length of 8 can be applied, otherwise, an invertible transform with an input length of 64 and an output length of 16 can be applied. In the case of inverse invertible transform, when the current block is 8x8, an invertible transform with an input length of 8 and an output length of 64 can be applied, otherwise, an invertible transform with an input length of 16 and an output length of 64 can be applied.

[0402] In addition, as described in Embodiment 11, since, for example, in the case where the RST is applied to the remaining area except for the right-bottom 4x4 area in the 8x8 block, the RST can be applied only to the partial area, not the entire 8x8 block, thus an 8x8 RST based on an 8x48 or 16x48 matrix can be applied. That is, in the case where each of the width and height corresponds to 8, an 8x8 RST based on an 8x48 matrix can be applied, otherwise (in the case where the width and height of the current block are not 8), an 8x8 RST based on a 16x48 matrix can be applied.

[0403] For the forward invertible transform, in the case where the current block is 8x8, an invertible transform with an input length of 48 and an output length of 8 can be applied, otherwise, an invertible transform with an input length of 48 and an output length of 16 can be applied.

[0404] For the backward invertible transform, in the case where the current block is 8x8, an invertible transform with an input length of 8 and an output length of 48 can be applied, otherwise, an invertible transform with an input length of 16 and an output length of 48 can be applied.

[0405] Therefore, in the case where the RST is applied to a block larger than 8x8 based on the encoder side, in the case where each of the width and height of the block corresponds to 8, an invertible transform matrix with an input length of 64 or less (e.g., 48 or 64) and an output length less than 64 (e.g., 8) (8x48 or 8x64 matrix) can be applied. In the case where the width or height of the block does not correspond to 8, an invertible transform matrix with an input length of 64 or less (e.g., 48 or 64) and an output length less than 64 (e.g., 16) (16x48 or 16x64 matrix) can be applied.

[0406] In addition, in a case where RST is applied to a block larger than 8x8 based on a decoder side, in a case where each of a width and a height of the block corresponds to 8, an inseparable transform matrix (48x8 or 64x8 matrix) having an input length smaller than 64 (for example, 8) and an output length of 64 or less (for example, 48 or 64) can be applied. In a case where the width or the height of the block does not correspond to 8, an inseparable transform matrix (48x16 or 64x16 matrix) having an input length smaller than 64 (for example, 16) and an output length of 64 or less (for example, 48 or 64) can be applied.

[0407] Table 12 represents examples of various 8x8 RST applications in a case where 8x8 NSST is applied (i.e., in a case where the width and the height of the block are greater than or equal to 8).

[0408] [Table 12]

[0409]

[0410]

[0411] Referring to Table 12, in a case where each of the width and the height of the current block is 8, 8x8 RST based on an 8x64 matrix (or an 8x48 matrix) can be applied, in a case where the product of the width and the height of the current block is smaller than a threshold (TH), 8x8 RST based on a 16x64 matrix (or a 16x48 matrix) is applied to the top-left 8x8 region of the current block, and otherwise (in a case where the width or the height of the current block is not 8 and the product of the width and the height of the current block is greater than or equal to the threshold), 8x8 RST based on a 32x64 matrix (or a 32x48 matrix) is applied to the top-left 8x8 region.

[0412] Figure 30 An example of an encoding flowchart to which a transform according to an embodiment of the disclosure is applied is illustrated.

[0413] The encoding apparatus 100 performs primary transform on the residual block (step, S2910). The primary transform can also be referred to as core transform. As an embodiment, the encoding apparatus 100 can perform the primary transform by using the MTS described above. In addition, the encoding apparatus 100 can transmit an MTS index indicating a specific MTS among MTS candidates to the decoding apparatus 200. In this case, the MTS candidates can be constructed based on an intra prediction mode of the current block.

[0414] The encoding apparatus 100 determines whether to apply secondary transform (step, S2920). As an example, the encoding apparatus 100 can determine whether to apply secondary transform based on residual transform coefficients according to the primary transform. For example, the secondary transform can be NSST or RST.

[0415] The encoding apparatus 100 determines to perform secondary transform (step, S2930). In this case, the encoding apparatus 100 can determine to perform secondary transform based on the NSST (or RST) transform set specified according to the intra prediction mode.

[0416] In addition, as an example, before step S2930, the encoding apparatus 100 can determine a region to which secondary transform is applied based on the size of the current block.

[0417] The encoding apparatus 100 performs secondary transform by using the secondary transform determined in step S2930 (step S2940).

[0418] Figure 30 An example of a decoding flowchart that performs transform to which the embodiments of the present disclosure are applied is exemplified.

[0419] The decoding apparatus 200 determines whether to apply secondary inverse transform (step, S3010). For example, the secondary inverse transform can be NSST or RST. As an example, the decoding apparatus 200 can determine whether to apply secondary inverse transform based on the secondary transform flag received from the encoding apparatus 100.

[0420] The decoding apparatus 200 determines to perform secondary inverse transform (step, S3020). In this case, the decoding apparatus 200 can determine to perform secondary inverse transform applied to the current block based on the NSST (or RST) transform set specified according to the above-described intra prediction mode.

[0421] In addition, as an example, before step S3020, the decoding apparatus 200 can determine a region to which secondary inverse transform is applied based on the size of the current block.

[0422] The decoding apparatus 200 performs secondary inverse transform on the dequantized residual block by using the secondary inverse transform determined in step S3020 (step, S3030).

[0423] The decoding apparatus 200 performs primary inverse transform on the residual block on which secondary inverse transform is performed. The primary inverse transform can be referred to as core inverse transform. As an embodiment, the decoding apparatus 200 can perform primary inverse transform by using the above-described MTS. In addition, as an example, before step S3040, the decoding apparatus 200 can determine whether to apply MTS to the current block. In this case, the step of determining whether to apply MTS can also be included in the decoding flowchart of FIG. 29. Figure 30

[0424] As an example, in the case where MTS is applied to the current block (i.e., cu_mts_flag = 1), the decoding apparatus 200 can construct an MTS candidate based on the intra prediction mode of the current block. In this case, the step of constructing an MTS candidate can also be included in the decoding flowchart of FIG. 29.​Figure 31 The decoding apparatus 200 can determine whether to perform the primary inverse transform applied to the current block by using the mts_idx indicating the specific MTS among the constructed MTS candidates.

[0425] Figure 32 An example of a detailed block diagram of the transformer 120 in the encoding apparatus 100 to which embodiments of the present disclosure are applied is exemplified.

[0426] The encoding apparatus 100 to which embodiments of the present disclosure are applied can include a primary transformer 3110, a secondary transform application determination unit 3120, a secondary transform determination unit 3130, and a secondary transformer 3140.

[0427] The primary transformer 3110 can perform the primary transform on the residual block. The primary transform can also be referred to as a core transform. As an embodiment, the primary transformer 3110 can perform the primary transform by using the MTS described above. In addition, the primary transformer 3110 can transmit the MTS index indicating the specific MTS among the MTS candidates to the decoding apparatus 200. In this case, the MTS candidates can be constructed based on the intra prediction mode of the current block.

[0428] The secondary transform application determination unit 3120 determines the secondary transform. As an example, the secondary transform application determination unit 3120 can determine whether to apply the secondary transform based on the residual transform coefficient according to the primary transform. For example, the secondary transform can be the NSST or the RST.

[0429] The secondary transform determination unit 3130 determines to perform the secondary transform. In this case, the secondary transform determination unit 3130 can determine to perform the secondary transform based on the NSST (or RST) transform set specified according to the intra prediction mode.

[0430] In addition, as an example, the secondary transform determination unit 3130 can determine the region to which the secondary transform is applied based on the size of the current block.

[0431] The secondary transformer 3140 can perform the secondary transform by using the determined secondary transform.

[0432] Figure 33 An example of a detailed block diagram of the inverse transformer 230 in the decoding apparatus to which embodiments of the present disclosure are applied is exemplified.

[0433] The decoding apparatus 200 to which embodiments of the present disclosure are applied includes a secondary inverse transform application determination unit 3210, a secondary inverse transform determination unit 3220, a secondary inverse transformer 3230, and a primary inverse transformer 3240.

[0434] The secondary inverse transform application determination unit 3210 determines whether to apply a secondary inverse transform. For example, the secondary inverse transform can be NSST or RST. As an example, the secondary inverse transform application determination unit 3210 can determine whether to apply a secondary inverse transform based on a secondary transform flag received from the encoding apparatus 100. As another example, the secondary inverse transform application determination unit 3210 can also determine whether to apply a secondary inverse transform based on transform coefficients of a residual block.

[0435] The secondary inverse transform determination unit 3220 can determine a secondary inverse transform. In this case, the secondary inverse transform determination unit 3220 can determine to perform a secondary inverse transform applied to a current block based on an NSST (or RST) transform set specified according to the above-described intra prediction mode.

[0436] In addition, as an example, the secondary inverse transform determination unit 3220 can determine a region to which a secondary inverse transform is applied based on a size of the current block.

[0437] Further, as an example, the secondary inverse transformer 3230 can perform a secondary inverse transform on a residual block after dequantization by using the determined secondary inverse transform.

[0438] The primary inverse transformer 3240 can perform a primary inverse transform on a residual block on which a secondary inverse transform is performed. As an implementation, the primary inverse transformer 3240 can perform a primary inverse transform by using the above-described MTS. In addition, as an example, the primary inverse transformer 3240 can determine whether to apply the MTS to the current block.

[0439] As an example, in the case where the MTS is applied to the current block (i.e., cu_mts_flag = 1), the primary inverse transformer 3240 can construct an MTS candidate based on an intra prediction mode of the current block. In addition, the primary inverse transformer 3240 can determine a primary inverse transform applied to the current block by using mts_idx indicating a specific MTS among the constructed MTS candidates.

[0440] Figure 33 An example of a decoding flowchart to which a transform is applied according to an embodiment of the disclosure is illustrated. Figure 16 The operations of FIG. 1 can be performed by the inverse transformer 230 of the decoding apparatus 200.

[0441] In step S3305, the decoding device 200 determines an input length and an output length of the non-separable transform based on the height and the width of the current block. Here, each of the width and the height of the block corresponds to 8, the input length of the non-separable transform can be determined as 8, and the output length can be determined as a value larger than the input length and smaller than or equal to 64 (for example, 48 or 64). For example, in a case where the non-separable transform is applied to all of the transform coefficients of an 8x8 block at the encoder side, the output length can be determined as 64, and in a case where the non-separable transform is applied to part (for example, part other than the right lower 4x4 block in the 8x8 block) of the transform coefficients of an 8x8 block at the encoder side, the output length can be determined as 48.

[0442] In step S3310, the decoding device 200 determines a non-separable transform matrix corresponding to the input length and the output length of the non-separable transform. For example, in a case where the input length of the non-separable transform is 8 and the output length thereof is 48 or 64 (in a case where the size of the current block is 4x4), a 48x8 or 64x8 matrix derived from a transform kernel can be determined as the non-separable transform, and in a case where the input length of the non-separable transform is 16 and the output length thereof is 48 or 64 (in a case where the size of the current block is smaller than 8x8 but not 4x4), a 48x16 or 64x16 matrix can be determined as the non-separable transform.

[0443] According to the embodiment of the present disclosure, the decoding device 200 can determine a non-separable transform set index (for example, NSST index) based on the intra prediction mode of the current block, determine a non-separable transform kernel corresponding to a non-separable transform index in a non-separable transform set included in the non-separable transform set index, and determine a non-separable transform matrix from the non-separable transform kernel based on the input length and the output length determined in step S3305.

[0444] In step S3315, the decoding device 200 applies the non-separable transform matrix determined for the current block to as many coefficients as the input length (8 or 16) determined for the current block. For example, in a case where the input length of the non-separable transform is 8 and the output length thereof is 48 or 64, a 48x8 or 64x8 matrix derived from a transform kernel can be applied to 8 coefficients included in the current block, and in a case where the input length of the non-separable transform is 16 and the output length thereof is 48 or 64, a 48x16 or 64x16 matrix derived from a transform kernel can be applied to 16 coefficients in the upper left 4x4 region of the current block. Here, the coefficients to which the non-separable transform is applied are coefficients from the DC position of the current block along the path of the predetermined scan order (for example, the path of (a), (b), or (c) of Figure 34

[0445] ​In addition, for each of the cases where the width and height of the current block do not correspond to 8, the decoding apparatus 200 can apply a non-separable transform matrix (48x16 or 64x16) that outputs as many transformed coefficients as the output length (e.g., 48 or 64) with 16 coefficients in the top-left 4x4 region of the current block as input, in the case where the product of the width and height of the current block is less than a threshold, and a non-separable transform matrix (48x32 or 64x32) that outputs as many transformed coefficients as the output length (e.g., 48 or 64) with 32 coefficients of the current block as input, in the case where the product of the width and height of the current block is greater than or equal to the threshold.

[0446] In the case where the output length is 64, 64 transformed data (transformed coefficients) by applying the non-separable transform matrix to the 8x8 block are set, and in the case where the output length is 48, 48 transformed data (transformed coefficients) by applying the non-separable transform matrix to the remaining region except for the right-bottom 4x4 region in the 8x8 block are set.

[0447] Figure 34 A block diagram example of a device for processing a video signal to which embodiments of the disclosure are applied is exemplified. Figure 1 The video signal processing device 3400 can correspond to Figure 2 the encoding apparatus 100 or Figure 35 the decoding apparatus 200.

[0448] The video signal processing device 3400 includes a memory 3420 for storing an image signal and a processor 3410 for processing the image signal while being coupled with the memory.

[0449] The processor 3410 according to the embodiments of the disclosure can include at least one processing circuit for processing an image signal, and process the image signal by executing a command for encoding or decoding. That is, the processor 3410 can encode original image data or decode encoded image signals by executing the above-described encoding or decoding method.

[0450] Figure 36 An example video encoding system according to the embodiments of the disclosure is exemplified.

[0451] The video encoding system can include a source device and a receiving device. The source device can deliver encoded video / image information or data in the form of a file or a stream to the receiving device via a digital storage medium or a network.

[0452] The source device can include a video source, an encoding device, and a transmitter. The reception device can include a receiver, a decoding device, and a renderer. The encoding device can be referred to as a video / image encoding device, and the decoding device can be referred to as a video / image decoding device. The transmitter can be included in the encoding device. The receiver can be included in the decoding device. The renderer can include a display unit, and the display unit can be configured as a separate device or an external component.

[0453] The video source can obtain a video / image by capturing, synthesizing, or generating a video / image. The video source can include a video / image capturing device and / or a video / image generating device. The video / image capturing device can include, for example, a video / image archive including previously captured videos / images and one or more cameras. The video / image generating device can include, for example, a computer, a tablet PC, or a smartphone, and can (electronically) generate a video / image. For example, a virtual video / image can be generated via, for example, a computer, in which case a process for generating its related data can replace a video / image capturing process.

[0454] The encoding device can encode an input video / image. The encoding device can perform a series of processes such as prediction, transformation, and quantization for compression and encoding efficiency. The encoded data (encoded video / image information) can be output in the form of a bitstream.

[0455] The transmitter can transmit the encoded video / image information or data, which has been output in the form of a bitstream, to the receiver of the reception device in the form of a file or streaming via a digital storage medium or a network. The digital storage medium can include various types of storage media such as USB, SD, CD, DVD, Blu-ray, HDD, or SSD. The transmitter can include an element for generating a media file in a predetermined file format and an element for transmission through a broadcasting / communication network. The receiver can extract a bitstream and transmit the bitstream to the decoding device.

[0456] The decoding device can perform a series of processes such as dequantization, inverse transformation, and prediction corresponding to the operations of the encoding device, and decode a video / image.

[0457] The renderer can render the decoded video / image. The rendered video / image can be displayed on a display unit.

[0458] ​ FIG. 1 is a view illustrating a structure of a content streaming system according to an embodiment of the disclosure.

[0459] The content streaming system to which the disclosure is applied can mainly include an encoding server, a streaming server, a web server, a media storage, a user device, and a multimedia input device.

[0460] The encoding server can compress content input from a multimedia input device such as a smartphone, a camera, or a camcorder into digital data, generate a bitstream, and transmit the bitstream to the streaming server. As an example, when a multimedia input device such as a smartphone, a camera, or a camcorder itself generates a bitstream, the encoding server can be omitted.

[0461] A bitstream can be generated by applying the encoding or bitstream generation method of the present disclosure, and the streaming server can temporarily store the bitstream while transmitting or receiving the bitstream.

[0462] The streaming server can transmit multimedia data to a user device based on a request made by the user through a web server, and the web server serves as an agent that informs the user of what service will be provided. If the user transmits a request for a desired service to the web server, the web server transmits the request to the streaming server, and the streaming server transmits multimedia data to the user. The content streaming system can include a separate control server, in which case the control server controls commands / responses between devices in the content streaming system.

[0463] The streaming server can receive content from a media storage device and / or an encoding server. For example, when receiving content from an encoding server, the content can be received in real time. In this case, in order to provide a seamless service, the streaming server can store a bitstream for a predetermined time.

[0464] Examples of the user device can include a mobile phone, a smartphone, a laptop computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation device, a touch screen PC, a tablet PC, an ultrabook, a wearable device such as a smart watch, smart glasses, or a head-mounted display (HMD), a digital TV, a desktop computer, or a digital signage device.

[0465] In the content streaming system, the server can be a distributed server, in which case data received by each server can be distributed and processed.

[0466] Further, the processing method according to the present disclosure can be implemented in the form of a program executed by a computer and stored in a computer-readable recording medium. Multimedia data having a data structure according to the present disclosure can also be stored in a computer-readable recording medium. The computer-readable recording medium includes all types of storage devices and distribution devices in which data readable by a computer is stored. The computer-readable recording medium can include, for example, a Blu-ray (BD), a universal serial bus (USB), a ROM, a PROM, an EEPROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device. Further, the computer-readable recording medium includes a medium implemented in the form of a carrier wave (for example, transmission over the Internet). Further, a bitstream generated by an encoding method can be stored in a computer-readable recording medium or can be transmitted through a wired / wireless communication network.

[0467] Further, the embodiments of the present disclosure can be implemented as a computer program product according to program codes, and the program codes can be executed in a computer according to the embodiments of the present disclosure. The program codes can be stored on a computer-readable carrier.

[0468] As described above, the embodiments of the present disclosure can be implemented and executed on a processor, a microprocessor, a controller, or a chip. For example, the functional units shown in each drawing can be implemented and executed on a computer, a processor, a microprocessor, a controller, or a chip.

[0469] Further, the decoder and the encoder according to the present disclosure can be included in a multimedia broadcast transmission / reception device, a mobile communication terminal, a home theater video system, a digital theater video system, a surveillance camera, a video conversion device, a real-time communication device such as a video communication, a mobile streaming device, a storage medium, a camcorder, a video on demand (VoD) service providing device, an over the top video (OTT) video system, an Internet streaming service providing device, a 3D video system, a video phone video system, a medical video system, etc., and can be used to process a video signal or a data signal. For example, the OTT video system can include a game console, a Blu-ray player, an Internet access TV, a home theater system, a smart phone, a tablet PC, a digital video recorder (DVR), etc.

[0470] Further, the processing method according to the disclosure can be manufactured in the form of a program executed by a computer and stored in a computer-readable recording medium. Multimedia data having a data structure according to the disclosure can also be stored in a computer-readable recording medium. The computer-readable recording medium includes all types of storage devices and distribution devices in which computer-readable data are stored. The computer-readable recording medium can include, for example, Blu-ray (BD), Universal Serial Bus (USB), ROM, PROM, EEPROM, RAM, CD-ROM, magnetic tapes, floppy disks, and optical data storage devices. Further, the computer-readable recording medium includes a medium implemented in the form of a carrier wave (e.g., transmission through the Internet). Further, a bitstream generated by an encoding method can be stored in a computer-readable recording medium or can be transmitted through a wired / wireless communication network.

[0471] Further, the embodiments of the disclosure can be implemented as a computer program product according to program codes, and the program codes can be executed in a computer according to the embodiments of the disclosure. The program codes can be stored on a computer-readable carrier.

[0472] The above-described embodiments are combinations of elements and features of the present disclosure. The elements or features can be considered selective unless otherwise mentioned. Each element or feature can be practiced without being combined with other elements or features. Also, embodiments of the present disclosure can be constructed by combining elements and / or features. The order of operations described in embodiments of the present disclosure can be re-arranged. Some constructions of any one embodiment can be included in another embodiment and can be replaced with a corresponding construction of another embodiment. It is clear that the scope of the claims of the present disclosure is not limited by the claims explicitly cited and combinations thereof.

[0473] The embodiments of the disclosure can be implemented by various means, for example, hardware, firmware, software, or a combination thereof. In a hardware configuration, the method according to the embodiments of the disclosure can be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), processors, controllers, micro-controllers, microprocessors, etc.

[0474] In a firmware or software configuration, the embodiments of the disclosure can be implemented in the form of a module, a procedure, a function, etc. Software codes can be stored in a memory and executed by a processor. The memory can be located inside or outside the processor and can transmit data to and receive data from the processor via various known means.

[0475] Those skilled in the art will appreciate that the disclosure can be embodied in other specific ways without departing from the spirit and essential characteristics of the disclosure. The above embodiments are therefore to be construed in all aspects as illustrative and not restrictive. The scope of the disclosure should be determined by the appended claims and their legal equivalents, rather than by the description of the above embodiments, and all changes that fall within the meaning and equivalents of the appended claims should be embraced therein.

[0476] Industrial applicability

[0477] While example aspects of the disclosure have been described with respect to the specific embodiments thereof, it is to be understood that various changes and modifications can be made to the disclosure by those skilled in the art without departing from the spirit and scope of the disclosure.

Claims

1. A method of decoding an image signal by a device, the method comprising the steps of: determining an input length and an output length of a non-separable transform based on a height and a width of a current block; determining a non-separable transform matrix for the current block, the size of the non-separable transform matrix being determined based on the input length and the output length of the non-separable transform; applying the non-separable transform matrix to coefficients of the current block, the number of the coefficients being related to the input length of the non-separable transform; and performing a primary inverse transform on the coefficients to which the non-separable transform is applied, wherein the input length and the output length of the non-separable transform are determined separately, wherein based on the height and the width of the current block, the size of the non-separable transform matrix is determined to be one of four predefined sizes, wherein based on the height and the width of the current block both being greater than 8, the input length of the non-separable transform is determined to be 16 and the output length of the non-separable transform is greater than the input length of the non-separable transform, wherein based on the height and the width of the current block both being equal to 8, the input length of the non-separable transform is determined to be 8 and the output length of the non-separable transform is greater than the input length of the non-separable transform, and wherein the output length is determined to be 48 or 64.

2. The method according to claim 1, wherein the step of applying the non-separable transform matrix comprises applying the non-separable transform matrix to a top-left 4x4 region of the current block based on the height and the width of the current block both being greater than 8 and a product of the width and the height being less than a threshold value.

3. The method according to claim 1, wherein the step of determining the non-separable transform matrix comprises the steps of: determining a non-separable transform set index based on an intra prediction mode of the current block; determining a non-separable transform kernel related to a non-separable transform index in a non-separable transform set included in the non-separable transform set index; and determining the non-separable transform matrix from the non-separable transform kernel based on the input length and the output length.

4. A method of encoding an image signal by a device, the method comprising the steps of: performing a primary transform on a current block; determining an input length and an output length of a non-separable transform based on a height and a width of the current block; determining a non-separable transform matrix for the current block, the size of the non-separable transform matrix being determined based on the input length and the output length of the non-separable transform; applying the non-separable transform matrix to coefficients of the current block after the primary transform, the number of the coefficients being related to the input length of the non-separable transform; and encoding non-separable transform index information for the non-separable transform matrix for the current block, wherein the input length and the output length of the non-separable transform are determined separately, wherein based on the height and the width of the current block, the size of the non-separable transform matrix is determined to be one of four predefined sizes, wherein based on the height and the width of the current block both being greater than 8, the input length of the non-separable transform is determined to be 16 and the output length of the non-separable transform is greater than the input length of the non-separable transform, wherein based on the height and the width of the current block both being equal to 8, the input length of the non-separable transform is determined to be 8 and the output length of the non-separable transform is greater than the input length of the non-separable transform, and wherein the output length is determined to be 48 or 64. wherein, based on the height and the width of the current block both being greater than 8, the output length of the non-separable transform is determined to be 16 and the input length of the non-separable transform is greater than the output length of the non-separable transform, wherein, based on the height and the width of the current block both being equal to 8, the output length of the non-separable transform is determined to be 8 and the input length of the non-separable transform is greater than the output length of the non-separable transform, and wherein the input length is determined to be 48 or 64.

5. A method of transmitting a bitstream generated by a method of encoding an image signal, the method of encoding the image signal comprising the steps of: performing a primary transform on a current block; determining an input length and an output length of a non-separable transform based on a height and a width of the current block; determining a non-separable transform matrix for the current block, the size of the non-separable transform matrix being determined based on the input length and the output length of the non-separable transform; applying the non-separable transform matrix to coefficients of the current block after the primary transform, the number of the coefficients being related to the input length of the non-separable transform; and encoding non-separable transform index information for the non-separable transform matrix for the current block, wherein the input length and the output length of the non-separable transform are determined separately, wherein, based on the height and the width of the current block, the size of the non-separable transform matrix is determined to be one of four predefined sizes, wherein, based on the height and the width of the current block both being greater than 8, the output length of the non-separable transform is determined to be 16 and the input length of the non-separable transform is greater than the output length of the non-separable transform, wherein, based on the height and the width of the current block both being equal to 8, the output length of the non-separable transform is determined to be 8 and the input length of the non-separable transform is greater than the output length of the non-separable transform, and wherein the input length is determined to be 48 or 64. ​

Images