Image encoding / decoding method, apparatus, and recording medium for storing bitstreams based on non-separated linear transformation.

Abstract

An image encoding / decoding method and apparatus are provided. The image decoding method according to the present disclosure includes the steps of: generating a residual block of a current block by performing an inverse transform on the current block based on a predetermined non-separable linear transform matrix; and reconstructing the current block based on the residual block, where the non-separable linear transform matrix can be applied to all transform coefficients of the current block regardless of the size of the current block.

Description

[Technical Field]

[0001] This disclosure relates to an image encoding / decoding method, apparatus, and recording medium for storing a bitstream, and more particularly to an image encoding / decoding method, apparatus, and recording medium for storing a bitstream generated by the image encoding method / apparatus of this disclosure. [Background technology]

[0002] Recently, the demand for high-resolution, high-quality images, such as HD (High Definition) and UHD (Ultra High Definition) images, has been increasing in various fields. As image data becomes higher resolution and higher quality, the amount of information or bits transmitted increases relatively compared to conventional image data. This increase in the amount of information or bits transmitted leads to increased transmission and storage costs.

[0003] This necessitates highly efficient image compression technology to effectively transmit, store, and reproduce high-resolution, high-quality image information. [Overview of the project] [Problems that the invention aims to solve]

[0004] The purpose of this disclosure is to provide an image coding / decoding method and apparatus with improved coding / decoding efficiency.

[0005] Furthermore, this disclosure aims to provide an image coding / decoding method and apparatus that performs non-separated first-order transformation.

[0006] Furthermore, this disclosure aims to provide an image coding / decoding method and apparatus that performs unseparated primary transformation based on the MTS scheme.

[0007] Furthermore, this disclosure aims to provide an image coding / decoding method and apparatus that performs non-separated primary transformation on a subblock basis.

[0008] Furthermore, this disclosure aims to provide an encoding / decoding method and apparatus that performs an unseparated linear transformation based on the symmetry between blocks.

[0009] Furthermore, this disclosure aims to provide a non-temporary computer-readable recording medium for storing a bitstream generated by the image encoding method or apparatus described herein.

[0010] Furthermore, this disclosure aims to provide a non-temporary computer-readable recording medium for storing a bitstream that is received by the image decoding device provided herein, decoded, and used to restore an image.

[0011] Furthermore, this disclosure aims to provide a method for transmitting a bitstream generated by an image encoding method or apparatus according to this disclosure.

[0012] The technical problems that this disclosure seeks to solve are not limited to those described above, and other technical problems not mentioned above will be clearly understood by a person with ordinary skill in the art to which this disclosure pertains from the following description. [Means for solving the problem]

[0013] An image decoding method according to one aspect of the present disclosure includes the steps of generating a residual block of the current block by performing an inverse transform on the current block based on a predetermined non-separable linear transformation matrix, and restoring the current block based on the residual block, wherein the non-separable linear transformation matrix can be applied to all transformation coefficients of the current block, regardless of the size of the current block.

[0014] An image decoding apparatus according to another aspect of the present disclosure includes a memory and at least one processor, the at least one processor generating a residual block of the current block by performing an inverse transform on the current block based on an inseparable linear transform matrix, and restoring the current block based on the residual block, wherein the inseparable linear transform matrix can be applied to all transformation coefficients of the current block, regardless of the size of the current block.

[0015] An image encoding method according to another aspect of the present disclosure includes the steps of generating a transformation coefficient block of a current block by performing a transformation on the current block based on a predetermined non-separable linear transformation matrix, and encoding the current block based on the transformation coefficient block, wherein the non-separable linear transformation matrix can be applied to all residual samples of the current block, regardless of the size of the current block.

[0016] A computer-readable recording medium according to another aspect of the present disclosure can store a bitstream generated by an image encoding method or image encoding apparatus of the present disclosure.

[0017] Another aspect of the transmission method of the present disclosure can transmit a bitstream generated by an image encoding device or image encoding method of the present disclosure.

[0018] The features described above, which are a brief summary of this disclosure, are merely illustrative examples of the detailed description of this disclosure described below and do not limit the scope of this disclosure. [Effects of the Invention]

[0019] According to this disclosure, an image encoding / decoding method and apparatus with improved encoding / decoding efficiency can be provided.

[0020] Furthermore, according to this disclosure, an image encoding / decoding method and apparatus that perform non-separated first-order transformation can be provided.

[0021] Also, according to the present disclosure, an image encoding / decoding method and apparatus for performing non-separable primary conversion based on the MTS scheme can be provided.

[0022] Also, according to the present disclosure, an image encoding / decoding method and apparatus for performing non-separable primary conversion based on a sub-block basis can be provided.

[0023] Also, according to the present disclosure, an image encoding / decoding method and apparatus for performing non-separable primary conversion based on the symmetry between blocks can be provided.

[0024] Also, according to the present disclosure, a non-transitory computer-readable recording medium for storing a bitstream generated by the image encoding method or apparatus according to the present disclosure can be provided.

[0025] Also, according to the present disclosure, a non-transitory computer-readable recording medium for storing a bitstream received by the image decoding apparatus according to the present disclosure, decoded, and used for restoring an image can be provided.

[0026] Also, according to the present disclosure, a method for transmitting a bitstream generated by the image encoding method or apparatus according to the present disclosure can be provided.

[0027] The effects obtained in the present disclosure are not limited to the above-described effects, and other effects not described above will be clearly understood by those of ordinary skill in the technical field to which the present disclosure pertains from the following description.

Brief Description of the Drawings

[0028] [Figure 1] It is a diagram schematically showing a video coding system to which an embodiment according to the present disclosure can be applied.

[0029] [Figure 2] It is a diagram schematically showing an image encoding apparatus to which an embodiment according to the present disclosure can be applied.

[0030] [Figure 3] This figure schematically shows an image decoding apparatus to which the embodiments of this disclosure can be applied.

[0031] [Figure 4] This is a diagram illustrating the LFNST application method.

[0032] [Figure 5] This is a flowchart showing the encoding process in which AMT is performed.

[0033] [Figure 6] This is a flowchart showing the decoding process in which AMT is performed.

[0034] [Figure 7] This is a flowchart showing the encoding process performed by NSST.

[0035] [Figure 8] This is a flowchart showing the decoding process during NSST.

[0036] [Figure 9-10] This is a diagram to explain how to run NSST.

[0037] [Figure 11-12] This is a diagram to explain how to execute RST.

[0038] [Figure 13] This figure illustrates the transformation and inverse transformation processes according to one embodiment of the present disclosure.

[0039] [Figures 14a-14d] This is a diagram illustrating the process of non-separable quadratic transformation.

[0040] [Figures 15a-15d] This figure illustrates a non-separable first-order transformation process according to one embodiment of the present disclosure.

[0041] [Figures 16a-16d] This figure illustrates a non-separable first-order transformation process according to other embodiments of the present disclosure.

[0042] [Figure 17] This flowchart shows a conversion method according to one embodiment of the present disclosure.

[0043] [Figure 18] This flowchart shows a method for inverse transformation according to one embodiment of the present disclosure.

[0044] [Figure 19] This flowchart shows a subblock non-separable linear / inverse transform method according to one embodiment of the present disclosure.

[0045] [Figures 20a-20b] This diagram illustrates the non-separable first-order transformation process of a subblock substrate.

[0046] [Figure 21-24] This figure illustrates a method for determining an unseparable linear transformation set according to one embodiment of the present disclosure.

[0047] [Figure 25] This flowchart shows an image encoding method according to one embodiment of the present disclosure.

[0048] [Figure 26] This is a flowchart showing an image decoding method according to one embodiment of the present disclosure.

[0049] [Figure 27] This figure illustrates a content streaming system to which the embodiments described herein can be applied. [Modes for carrying out the invention]

[0050] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings, so that they can be easily implemented by a person with ordinary skill in the art to which the present disclosure pertains. However, the present disclosure can be implemented in a variety of different forms and is not limited to the embodiments described herein.

[0051] In describing embodiments of this disclosure, if it is determined that a specific description of a known configuration or function would obscure the gist of this disclosure, such detailed description will be omitted. In the drawings, parts unrelated to the description of this disclosure will be omitted, and similar parts will be denoted by the same reference numerals.

[0052] In this disclosure, when one component is described as being “connected,” “joined,” or “linked” to another component, this can include not only direct connections but also indirect connections where another component exists between them. Furthermore, when one component is described as “containing” or “having” another component, this means, unless otherwise stated to the contrary, that it may include another component rather than excluding it.

[0053] In this disclosure, terms such as "first," "second," etc., are used solely for the purpose of distinguishing one component from another, and do not limit the order or importance of the components unless otherwise specified. Therefore, within the scope of this disclosure, a first component in one embodiment may be called a second component in another embodiment, and similarly, a second component in one embodiment may be called a first component in another embodiment.

[0054] In this disclosure, components that are distinguished from each other are used to clearly describe their respective characteristics and do not necessarily mean that the components are separate. In other words, multiple components may be integrated to constitute a single hardware or software unit, or a single component may be distributed to constitute multiple hardware or software units. Therefore, such integrated or distributed embodiments are also included in the scope of this disclosure, without needing to be specifically mentioned.

[0055] In this disclosure, the components described in various embodiments are not necessarily essential components, and some may be optional components. Therefore, embodiments consisting of a subset of the components described in one embodiment are also included in the scope of this disclosure. Furthermore, embodiments that include additional components in addition to the components described in various embodiments are also included in the scope of this disclosure.

[0056] This disclosure relates to the encoding and decoding of images, and the terms used in this disclosure may have their ordinary meanings in the art to which this disclosure pertains, unless otherwise defined herein.

[0057] In this disclosure, "video" can mean a collection of images in a sequence of time.

[0058] In this disclosure, "picture" generally means a unit representing any one image within a specific time period, and "slice / tile" is an encoding unit that constitutes part of a picture, and a single picture can consist of one or more slices / tiles. Furthermore, a slice / tile may contain one or more CTUs (coding tree units).

[0059] In this disclosure, “pixel” or “pel” may mean the smallest unit that constitutes a picture (or image). The term “sample” may also be used as a counterpart to pixel. A sample may generally represent a pixel or a pixel value, or it may represent only the pixel / pixel value of the luma component, or only the pixel / pixel value of the chroma component.

[0060] In this disclosure, “unit” can refer to a basic unit of image processing. A unit may include at least one of a specific region of a picture and information associated with that region. A unit may be used interchangeably with terms such as “sample array,” “block,” or “area,” as it may be used. Generally, an M×N block may include a set (or array) of samples (or sample arrays) or transform coefficients consisting of M columns and N rows.

[0061] In this disclosure, “current block” can mean any one of the following: “current coding block,” “current coding unit,” “block to encode,” “block to decode,” or “block to process.” If prediction is performed, “current block” can mean “current prediction block” or “block to predict.” If transformation (inverse transformation) / quantization (inverse quantization) is performed, “current block” can mean “current transformation block” or “block to transform.” If filtering is performed, “current block” can mean “block to filter.”

[0062] Furthermore, in this disclosure, "current block" may mean the block containing all of the rumor component blocks and chroma component blocks, or the "rumor block of the current block," unless there is an explicit mention of a chroma block. The rumor component block of the current block may be expressed with an explicit mention of a rumor component block, such as "rumor block" or "current rumor block." Similarly, the chroma component block of the current block may be expressed with an explicit mention of a chroma component block, such as "chroma block" or "current chroma block."

[0063] In this disclosure, " / " and "," may be interpreted as "and / or." For example, "A / B" and "A, B" may be interpreted as "A and / or B." Also, "A / B / C" and "A, B, C" may mean "at least one of A, B and / or C."

[0064] In this disclosure, “or” may be interpreted as “and / or.” For example, “A or B” may mean 1) “A” only, 2) “B” only, or 3) “A and B.” Alternatively, in this disclosure, “or” may mean “additionally or alternatively.”

[0065] In this disclosure, “at least one A, B, and C” may mean “A only,” “B only,” “C only,” or “any combination of A, B, and C.” Also, “at least one A, B, or C” or “at least one A, B, and / or C” may mean “at least one A, B, and C.”

[0066] The parentheses used in this disclosure may mean "for example." For example, when "Prediction (Intra Prediction)" is shown, "Intra Prediction" may be proposed as an example of "Prediction." In other words, "Prediction" in this disclosure is not limited to "Intra Prediction," and "Intra Prediction" may be proposed as an example of "Prediction." Also, when "Prediction (i.e., Intra Prediction)" is shown, "Intra Prediction" may be proposed as an example of "Prediction."

[0067] Overview of the video coding system

[0068] Figure 1 is a schematic diagram showing a video coding system to which the embodiments of this disclosure can be applied.

[0069] A video coding system according to one embodiment may include an encoding device 10 and a decoding device 20. The encoding device 10 can transmit encoded video and / or image information or data to the decoding device 20 via a digital storage medium or network in file or streaming format.

[0070] An encoding device 10 according to one embodiment may include a video source generation unit 11, an encoding unit 12, and a transmission unit 13. A decoding device 20 according to one embodiment may include a receiving unit 21, a decoding unit 22, and a rendering unit 23. The encoding unit 12 may be called a video / image encoding unit, and the decoding unit 22 may be called a video / image decoding unit. The transmission unit 13 may be included in the encoding unit 12. The receiving unit 21 may be included in the decoding unit 22. The rendering unit 23 may also include a display unit, which may be configured as a separate device or external component.

[0071] The video source generation unit 11 can acquire video / images through processes such as video / image capture, synthesis, or generation. The video source generation unit 11 may include a video / image capture device and / or a video / image generation device. The video / image capture device may include, for example, one or more cameras, or a video / image archive containing previously captured video / images. The video / image generation device may include, for example, a computer, tablet, and smartphone, and may generate video / images (electronically). For example, virtual video / images may be generated via a computer, in which case the video / image capture process may be replaced by a process in which the relevant data is generated.

[0072] The encoding unit 12 can encode the input video / image. The encoding unit 12 can perform a series of steps such as prediction, transformation, and quantization for compression and encoding efficiency. The encoding unit 12 can output the encoded data (encoded video / image information) in bitstream format.

[0073] The transmission unit 13 can transmit encoded video / image information or data, output in bitstream format, to the receiving unit 21 of the decoding device 20 via a digital storage medium or network in file or streaming format. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. The transmission unit 13 may include elements for generating media files via a predetermined file format and elements for transmission via a broadcast / communication network. The receiving unit 21 can extract / receive the bitstream from the storage medium or network and transmit it to the decoding unit 22.

[0074] The decoding unit 22 can decode the video / image by performing a series of steps such as inverse quantization, inverse transform, and prediction, corresponding to the operation of the encoding unit 12.

[0075] The rendering unit 23 can render the decoded video / image. The rendered video / image can be displayed via the display unit.

[0076] Overview of Image Encoding Devices

[0077] Figure 2 is a schematic diagram showing an image encoding device to which the embodiments of this disclosure can be applied.

[0078] As shown in Figure 2, the image coding device 100 may include an image splitting unit 110, a subtraction unit 115, a transformation unit 120, a quantization unit 130, an inverse quantization unit 140, an inverse transformation unit 150, an addition unit 155, a filtering unit 160, a memory 170, an inter-prediction unit 180, an intra-prediction unit 185, and an entropy coding unit 190. The inter-prediction unit 180 and the intra-prediction unit 185 can together be called the "prediction unit". The transformation unit 120, the quantization unit 130, the inverse quantization unit 140, and the inverse transformation unit 150 may be included in a residual processing unit. The residual processing unit may further include a subtraction unit 115.

[0079] All or at least some of the multiple components constituting the image encoding device 100 can be implemented by a single hardware component (e.g., an encoder or processor) depending on the embodiment. Furthermore, the memory 170 may include a DPB (decoded picture buffer) and can be implemented by a digital storage medium.

[0080] The image splitting unit 110 can split an input image (or picture, frame) input to the image encoding device 100 into one or more processing units. For example, the processing units may be called coding units (CUs). Coding units can be obtained by recursively splitting a coding tree unit (CTU) or the largest coding unit (LCU) using a QT / BT / TT (Quad-tree / binary-tree / ternary-tree) structure. For example, a single coding unit can be split into multiple coding units of deeper depth based on a quad-tree structure, a binary-tree structure and / or a ternary-tree structure. For the splitting of coding units, a quad-tree structure may be applied first, followed by a binary-tree structure and / or a ternary-tree structure. Based on the final coding unit that cannot be further split, the coding procedure according to this disclosure can be performed. The largest coding unit can be used as the final coding unit, or a lower-depth coding unit obtained by dividing the largest coding unit can be used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and / or restoration, as described later. As another example, the processing units of the coding procedure may be prediction units (PU) or transformation units (TU). The prediction unit and the transformation unit may be divided or partitioned from the final coding unit, respectively. The prediction unit may be a unit of sample prediction, and the transformation unit may be a unit that derives transformation coefficients and / or a unit that derives a residual signal from transformation coefficients.

[0081] The prediction unit (inter-prediction unit 180 or intra-prediction unit 185) can make predictions for the block to be processed (current block) and generate a predicted block that includes prediction samples for the current block. The prediction unit can determine whether intra-prediction or inter-prediction is applied to the current block or on a CU basis. The prediction unit can generate various information regarding the prediction of the current block and transmit it to the entropy coding unit 190. The prediction information can be encoded by the entropy coding unit 190 and output in bitstream format.

[0082] The intra-prediction unit 185 can predict the current block by referring to a sample in the current picture. The referenced sample may be located in the vicinity (neighbor) or at a distance from the current block, according to the intra-prediction mode and / or intra-prediction technique. The intra-prediction mode may include multiple non-directional modes and multiple directional modes. The non-directional modes may include, for example, a DC mode and a Planar mode. The directional modes may include, for example, 33 or 65 directional prediction modes, depending on the degree of fineness of the prediction direction. However, this is merely an example, and more or fewer directional prediction modes may be used depending on the settings. The intra-prediction unit 185 may also determine the prediction mode to be applied to the current block using the prediction modes applied to the surrounding blocks.

[0083] The interprediction unit 180 can derive a predicted block relative to the current block based on a reference block (reference sample array) identified by motion vectors on the reference picture. In this case, in order to reduce the amount of motion information transmitted in interprediction mode, motion information can be predicted in units of blocks, subblocks, or samples based on the correlation of motion information between the surrounding blocks and the current block. The motion information may include motion vectors and reference picture indices. The motion information may further include interprediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of interprediction, the surrounding blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. The reference picture containing the reference block and the reference picture containing the temporal neighboring block may be the same or different from each other. The temporal neighboring block may be called a collocated reference block, collocated CU (colCU), etc. The reference picture containing the temporal neighboring block may be called a collocated picture (colPic). For example, the interpretation unit 180 can construct a motion information candidate list based on surrounding blocks and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Interpretation can be performed based on various prediction modes; for example, in skip mode and merge mode, the interpretation unit 180 can use the motion information of surrounding blocks as the motion information of the current block. In skip mode, unlike merge mode, the residual signal may not be transmitted.In motion vector prediction (MVP) mode, the motion vector of the surrounding block is used as the motion vector predictor, and the motion vector of the current block can be signaled by encoding the motion vector difference and an indicator for the motion vector predictor. The motion vector difference can represent the difference between the motion vector of the current block and the motion vector predictor.

[0084] The prediction unit can generate a prediction signal based on various prediction methods and / or techniques described later. For example, the prediction unit can apply intra-prediction or inter-prediction to predict the current block, and can also apply intra-prediction and inter-prediction simultaneously. A prediction method that applies intra-prediction and inter-prediction simultaneously to predict the current block can be called CIIP (combined inter and intra prediction). The prediction unit can also perform intra-block copy (IBC) to predict the current block. Intra-block copy can be used for content image / video coding such as in games, for example, in SCC (screen content coding). IBC is a method of predicting the current block using a reference block that has already been restored in the current picture at a predetermined distance from the current block. When IBC is applied, the position of the reference block in the current picture can be encoded as a vector (block vector) corresponding to the predetermined distance. IBC basically performs prediction within the current picture, but it can be performed similarly to inter-prediction in that it derives the reference block within the current picture. In other words, IBC can use at least one of the interpretation techniques described in this disclosure.

[0085] The predicted signal generated by the prediction unit can be used to generate a reconstructed signal or a residual signal. The subtraction unit 115 can generate a residual signal (residual block, residual sample array) by subtracting the predicted signal output from the prediction unit (predicted block, predicted sample array) from the input image signal (original block, original sample array). The generated residual signal can be transmitted to the conversion unit 120.

[0086] The transformation unit 120 can generate transformation coefficients by applying transformation techniques to the residual signal. For example, the transformation techniques may include at least one of the following: DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), KLT (Karhunen-Loeve Transform), GBT (Graph-Based Transform), or CNT (Conditionally Non-linear Transform). Here, GBT refers to a transformation obtained from a graph when the relationship information between pixels is represented by this graph. CNT refers to a transformation obtained by generating a prediction signal using all previously reconstructed pixels. The transformation process can be applied to pixel blocks of the same size and square shape, or to non-square, variable-sized blocks.

[0087] The quantization unit 130 can quantize the conversion coefficients and transmit them to the entropy coding unit 190. The entropy coding unit 190 can encode the quantized signal (information about the quantized conversion coefficients) and output it in bitstream format. The information about the quantized conversion coefficients can be called residual information. The quantization unit 130 can rearrange the block-form quantized conversion coefficients into a one-dimensional vector format based on the coefficient scan order, and can also generate information about the quantized conversion coefficients based on the one-dimensional vector format of the quantized conversion coefficients.

[0088] The entropy coding unit 190 can perform various coding methods, such as exponential Golomb, CAVLC (context-adaptive variable length coding), and CABAC (context-adaptive binary arithmetic coding). In addition to the quantized conversion coefficients, the entropy coding unit 190 can also encode information necessary for video / image restoration (e.g., the values ​​of syntax elements) together or separately. The encoded information (e.g., encoded video / image information) can be transmitted or stored in bitstream format in units of NAL (network abstraction layer) units. The video / image information may further include information about various parameter sets, such as adaptive parameter sets (APS), picture parameter sets (PPS), sequence parameter sets (SPS), or video parameter sets (VPS). The video / image information may also further include general constraint information. The signaling information, transmitted information and / or syntax elements referred to in this disclosure may be encoded via the encoding procedure described above and included in the bitstream.

[0089] The bitstream can be transmitted over a network or stored on a digital storage medium. Here, the network may include broadcast networks and / or communication networks, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. A transmission unit (not shown) for transmitting the signal output from the entropy encoding unit 190 and / or a storage unit (not shown) for storing it may be provided as an internal / external element of the image encoding device 100, or the transmission unit may be provided as a component of the entropy encoding unit 190.

[0090] The quantized conversion coefficients output from the quantization unit 130 can be used to generate a residual signal. For example, by applying inverse quantization and inverse transformation to the quantized conversion coefficients via the inverse quantization unit 140 and the inverse transformation unit 150, a residual signal (residual block or residual sample) can be reconstructed.

[0091] The adder 155 can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the reconstructed residual signal to the prediction signal output from the inter-prediction unit 180 or the intra-prediction unit 185. If there is no residual for the block to be processed, such as when skip mode is applied, the predicted block can be used as the reconstructed block. The adder 155 may be called the reconstruction unit or the reconstructed block generation unit. The generated reconstructed signal can be used for intra-prediction of the next block to be processed in the current picture, or, as described later, for inter-prediction of the next picture after filtering.

[0092] On the other hand, LMCS (luma mapping with chroma scaling) can also be applied during the picture encoding and / or restoration process.

[0093] The filtering unit 160 can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit 160 can apply various filtering methods to the restored picture to generate a modified restored picture, and the modified restored picture can be stored in the memory 170, specifically in the DPB of the memory 170. The various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, and bilateral filter. The filtering unit 160 can generate various filtering-related information, as will be described later in the explanation of each filtering method, and transmit it to the entropy coding unit 190. The filtering-related information can be encoded by the entropy coding unit 190 and output in bitstream format.

[0094] The corrected restored picture transmitted to memory 170 can be used as a reference picture in the interpretation unit 180. When interpretation is applied via this, the image encoding device 100 can avoid prediction mismatches between the image encoding device 100 and the image decoding device, and can also improve encoding efficiency.

[0095] The DPB in memory 170 can store the modified restored picture for use as a reference picture in the inter-prediction unit 180. Memory 170 can store motion information of blocks from which motion information in the current picture has been derived (or encoded) and / or motion information of blocks in the picture that have already been restored. The stored motion information can be transmitted to the inter-prediction unit 180 for use as motion information of spatially surrounding blocks or motion information of temporally surrounding blocks. Memory 170 can store restored samples of restored blocks in the current picture and transmit them to the intra-prediction unit 185.

[0096] Overview of the image decoding device

[0097] Figure 3 is a schematic diagram showing an image decoding apparatus to which the embodiments of this disclosure can be applied.

[0098] As shown in Figure 3, the image decoding device 200 can be configured to include an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an additive unit 235, a filtering unit 240, a memory 250, an inter-prediction unit 260, and an intra-prediction unit 265. The inter-prediction unit 260 and the intra-prediction unit 265 can together be called the "prediction unit". The inverse quantization unit 220 and the inverse transform unit 230 can be included in the residual processing unit.

[0099] All or at least some of the multiple components constituting the image decoding device 200 can be implemented by a single hardware component (e.g., a decoder or processor) according to the embodiment. Furthermore, the memory 170 may include a DPB and can be implemented by a digital storage medium.

[0100] An image decoding device 200, having received a bitstream containing video / image information, can restore the image by executing a process corresponding to the process performed in the image encoding device 100 in Figure 2. For example, the image decoding device 200 can perform decoding using the processing unit applied in the image encoding device. Therefore, the decoding processing unit can be, for example, a coding unit. The coding unit can be obtained by dividing a coding tree unit or a maximum coding unit. The restored image signal decoded and output via the image decoding device 200 can then be reproduced via a playback device (not shown).

[0101] The image decoding device 200 can receive the signal output from the image encoding device 2 in bitstream format. The received signal can be decoded via the entropy decoding unit 210. For example, the entropy decoding unit 210 can parse the bitstream to derive information necessary for image restoration (or picture restoration) (e.g., video / image information). The video / image information may further include information about various parameter sets, such as adaptive parameter set (APS), picture parameter set (PPS), sequence parameter set (SPS), or video parameter set (VPS). The video / image information may also further include general constraint information. The image decoding device may further use the parameter set information and / or the general constraint information to decode the image. The signaling information, received information, and / or syntax elements referred to in this disclosure can be obtained from the bitstream by decoding via the decoding procedure. For example, the entropy decoding unit 210 can decode information in the bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and output the values ​​of syntax elements necessary for image reconstruction and the quantized values ​​of conversion coefficients related to the residual. More specifically, the CABAC entropy decoding method receives bins corresponding to each syntax element from the bitstream, determines a context model using the syntax element information to be decoded, the decoding information of the surrounding blocks and the blocks to be decoded, or the symbol / bin information decoded in a previous step, predicts the probability of bin occurrence based on the determined context model, and performs arithmetic decoding of the bins to generate symbols corresponding to the values ​​of each syntax element. At this time, after determining the context model, the CABAC entropy decoding method can update the context model using the decoded symbol / bin information for the context model of the next symbol / bin.Of the information decoded by the entropy decoding unit 210, information related to prediction is provided to the prediction unit (inter-prediction unit 260 and intra-prediction unit 265), and the residual values ​​that have undergone entropy decoding in the entropy decoding unit 210, i.e., quantized conversion coefficients and related parameter information, can be input to the inverse quantization unit 220. In addition, of the information decoded by the entropy decoding unit 210, information related to filtering can be provided to the filtering unit 240. On the other hand, a receiving unit (not shown) that receives signals output from the image coding device may be further provided as an internal / external element of the image decoding device 200, or the receiving unit may be provided as a component of the entropy decoding unit 210.

[0102] On the other hand, the image decoding device according to this disclosure may be called a video / image / picture decoding device. The image decoding device may also include an information decoder (video / image / picture information decoder) and / or a sample decoder (video / image / picture sample decoder). The information decoder may include an entropy decoding unit 210, and the sample decoder may include at least one of an inverse quantization unit 220, an inverse transform unit 230, an adder unit 235, a filtering unit 240, a memory 250, an inter-prediction unit 260, and an intra-prediction unit 265.

[0103] The inverse quantization unit 220 can inverse quantize the quantized transformation coefficients and output the transformation coefficients. The inverse quantization unit 220 can rearrange the quantized transformation coefficients in a two-dimensional block format. In this case, the rearrangement can be performed based on the coefficient scan order performed by the image encoding device. The inverse quantization unit 220 can perform inverse quantization on the quantized transformation coefficients using quantization parameters (e.g., quantization step size information) to obtain the transformation coefficients.

[0104] The inverse conversion unit 230 can inversely convert the conversion coefficients to obtain residual signals (residual blocks, residual sample arrays).

[0105] The prediction unit can make predictions for the current block and generate a predicted block containing prediction samples for the current block. Based on the prediction information output from the entropy decoding unit 210, the prediction unit can determine whether intra-prediction or inter-prediction is applied to the current block and can determine a specific intra / inter-prediction mode (prediction technique).

[0106] As described in the explanation of the prediction unit of the image coding device 100, the prediction unit can generate prediction signals based on various prediction methods (techniques) described later.

[0107] The intra-prediction unit 265 can predict the current block by referring to the samples in the current picture. The description of the intra-prediction unit 185 can also be applied to the intra-prediction unit 265.

[0108] The interprediction unit 260 can derive a predicted block relative to the current block based on a reference block (reference sample array) identified by motion vectors on a reference picture. In this case, to reduce the amount of motion information transmitted in interprediction mode, motion information can be predicted in block, sub-block, or sample units based on the correlation of motion information between surrounding blocks and the current block. The motion information may include motion vectors and reference picture indices. The motion information may further include interprediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In interprediction, surrounding blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. For example, the interprediction unit 260 can construct a motion information candidate list based on surrounding blocks and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. Interprediction can be performed based on various prediction modes (techniques), and the prediction information may include information indicating the mode (technique) of interprediction for the current block.

[0109] The adder 235 can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the acquired residual signal to the predicted signal (predicted block, predicted sample array) output from the prediction unit (including the inter-prediction unit 260 and / or intra-prediction unit 265). If there is no residual for the block to be processed, such as when skip mode is applied, the predicted block can be used as the reconstructed block. The description of the adder 155 also applies to the adder 235. The adder 235 is sometimes called the reconstruction unit or reconstructed block generation unit. The generated reconstructed signal can be used for intra-prediction of the next block to be processed in the current picture, or for inter-prediction of the next picture via filtering, as described later.

[0110] The filtering unit 240 can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit 240 can apply various filtering methods to the restored picture to generate a modified restored picture, and the modified restored picture can be stored in the memory 250, specifically in the DPB of the memory 250. The various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, and bilateral filter.

[0111] The restored picture stored (modified) in the DPB of memory 250 can be used as a reference picture in the inter-prediction unit 260. Memory 250 can store motion information of blocks from which motion information in the current picture has been derived (or decoded) and / or motion information of blocks in the picture that have already been restored. The stored motion information can be transmitted to the inter-prediction unit 260 for use as motion information of spatially surrounding blocks or motion information of temporally surrounding blocks. Memory 250 can store restored samples of restored blocks in the current picture and transmit them to the intra-prediction unit 265.

[0112] In this specification, the embodiments described for the filtering unit 160, inter-prediction unit 180, and intra-prediction unit 185 of the image coding device 100 can be applied similarly or in a corresponding manner to the filtering unit 240, inter-prediction unit 260, and intra-prediction unit 265 of the image decoding device 200, respectively.

[0113] Overview of Conversion / Inverse Conversion

[0114] As described above, the encoding device can derive residual blocks (residual samples) based on blocks (predicted samples) predicted by intra / inter / IBC prediction, and can derive quantized transformation coefficients by applying transformation and quantization to the derived residual samples. Information for the quantized transformation coefficients (residual information) can be encoded in the residual coding syntax and output in bitstream format. The decoding device can obtain the information for the (quantized) transformation coefficients (residual information) from the bitstream, decode it, and derive quantized transformation coefficients. The decoding device can derive residual samples via inverse quantization / inverse transformation based on the quantized transformation coefficients. As described above, at least one of the quantization / inverse quantization and / or transformation / inverse transformation is optional. If the quantization / inverse quantization is omitted, the quantized transformation coefficients may be called transformation coefficients. If the transformation / inverse transformation is omitted, the transformation coefficients may be called coefficients or residual coefficients, or may still be called transformation coefficients for consistency of expression. Whether the aforementioned transformation / inverse transformation can be omitted can be signaled based on the transform_skip_flag.

[0115] Furthermore, in this disclosure, quantized transformation coefficients and transformation coefficients may be referred to as transformation coefficients and scaled transformation coefficients, respectively. In this case, residual information may include information about the transformation coefficients, and such information about the transformation coefficients may be signaled via residual coding syntax. Transformation coefficients may be derived based on the residual information (or information about the transformation coefficients), and scaled transformation coefficients may be derived via an inverse transformation (scaling) of the transformation coefficients. Residual samples may be derived based on an inverse transformation (transformation) of the scaled transformation coefficients. This may be similarly applied / expressed in other parts of this disclosure.

[0116] The aforementioned transformations / inverse transformations can be performed based on transformation kernels. For example, according to this disclosure, a multiple transform selection (MTS) scheme can be applied. In this case, a subset of transformation kernels can be selected from a set of many and applied to the current block. Transformation kernels can be referred to by various terms such as transformation matrices or transformation types. For example, a set of transformation kernels can represent a combination of vertical transformation kernels and horizontal transformation kernels.

[0117] For example, MTS index information (or the mts_idx syntax element) can be generated / encoded by the encoding device and signaled to the decoding device to indicate one of the conversion kernel sets. For example, the conversion kernel sets based on the value of the MTS index information can be derived as shown in Table 1.

[0118] [Table 1]

[0119] Table 1 shows the tyTypeHor and trTypeVer values ​​for tu_mts_idx[x0][y0].

[0120] The aforementioned set of conversion kernels can also be determined, for example, based on cu_sbt_horizontal_flag and cu_sbt_pos_flag, as shown in Table 2.

[0121] [Table 2]

[0122] Table 2 shows the tyTyperHor and trTypeVer values ​​for cu_sbt_horizontal_flag and cu_sbt_pos_flag. Here, a cu_sbt_horizontal_flag such as 1 can indicate that the coding unit is currently horizontally divided into two transformation blocks. Conversely, a cu_sbt_horizontal_flag such as 0 can indicate that the coding unit is currently vertically divided into two transformation blocks. Also, a cu_sbt_pos_flag such as 1 can indicate that the syntax elements tu_cbf_luma, tu_cbf_cb and tu_cbf_cr of the first transformation unit in the coding unit are not present in the bitstream. Conversely, a cu_sbt_pos_flag such as 0 can indicate that the syntax elements tu_cbf_luma, tu_cbf_cb and tu_cbf_cr of the second transformation unit in the coding unit are not present in the bitstream.

[0123] On the other hand, in Tables 1 and 2, trTypeHor can represent a horizontal transformation kernel, and trTypeVer can represent a vertical transformation kernel. A trTypeHor / trTypeVer value of 0 can represent DCT2, a trTypeHor / trTypeVer value of 1 can represent DST7, and a trTypeHor / trTypeVer value of 2 can represent DCT8. However, this is illustrative, and other values ​​may be mapped to other DCTs / DSTs by convention.

[0124] Table 3 shows illustrative basis functions for DCT2, DCT8, and DST7 as described above.

[0125] [Table 3]

[0126] In this disclosure, the transformation of the MTS base is applied as a primary transform, and a secondary transform may be applied further. The secondary transform may be applied only to the coefficients in the upper left w × h region of the coefficient block to which the primary transform was applied, and may be called an RST (reduced secondary transform). For example, w and / or h may be 4 or 8. In the transformation, the primary and secondary transforms may be applied sequentially to the residual block, and in the inverse transform, the inverse quadratic transform and inverse primary transform may be applied sequentially to the transformed coefficients. The secondary transform (RST transform) may be called a low frequency coefficients transform (LFCT) or a low frequency non-seperable transform (LFNST). The inverse quadratic transform may be called an inverse LFCT or inverse LFNST.

[0127] Figure 4 is a diagram illustrating the LFNST application method.

[0128] Referring to Figure 4, LFNST can be applied between the forward linear transformation 411 and quantization 413 in the encoder stage, and between the inverse quantization 421 and the inverse linear transformation (or inverse linear transformation) 423 in the decoder stage.

[0129] In LFNST, a 4x4 inseparable transform or an 8x8 inseparable transform can be applied (selectively) depending on the size of the block. For example, a 4x4 LFNST can be applied to relatively small blocks (i.e., min(width, height) < 8), and an 8x8 LFNST can be applied to relatively large blocks (i.e., min(width, height) > 4). Figure 4 illustrates that a 4x4 forward LFNST can be applied to 16 input coefficients, and an 8x8 forward LFNST can be applied to 64 input coefficients. Also, Figure 4 illustrates that a 4x4 reverse LFNST can be applied to 8 input coefficients, and an 8x8 reverse LFNST can be applied to 16 input coefficients.

[0130] LFNST can use a total of four transformation sets, and for each transformation set, two inseparable transformation matrices (kernels). The mapping from the intra-prediction mode to the transformation sets can be predefined as shown in Table 4.

[0131] [Table 4]

[0132] Referring to Table 4, if three CCLM modes are used for the current block, each having a predicted mode number between 81 and 83 (i.e., 81 ≤ IntraPredMode ≤ 83), then transformation set 0 can be selected for the current chroma block. For each transformation set, a specified non-separated quadratic transformation candidate can be further specified by an explicitly signaled LFNST index. This index can be signaled in the bitstream once per IntraCU after the transformation coefficient.

[0133] On the other hand, the conversion / inverse conversion can be performed in units of CU or TU. That is, the conversion / inverse conversion can be applied to residual samples within a CU or residual samples within a TU. The CU size and TU size may be the same, or there may be multiple TUs within the CU region. On the other hand, the CU size can generally represent the lumens component (sample) CB size. The TU size can generally represent the lumens component (sample) TB size. The chromens component (sample) CB or TB size can be derived based on the lumens component (sample) CB or TB size according to the component ratio of the color format (chromens format, e.g., 4:4:4, 4:2:2, 4:2:0, etc.). The TU size can be derived based on maxTbSize. For example, if the CU size is larger than the maxTBSize, multiple TUs (TBs) of maxTbSize can be derived from the CU, and the conversion / inverse conversion can be performed in units of the TUs (TBs). The maxTbSize can be considered when determining whether various intra-prediction types such as ISP are applicable. The information for maxTbSize may be predetermined, or it may be generated and encoded by an encoding device and signaled to a decoding device.

[0134] As mentioned above, transformations can be applied to residual blocks. This is to decorrelate the residual blocks as much as possible, concentrate the coefficients at low frequencies, and create a zero tail at the edge of the block. In the JEM software, the transformation part includes two main functions: core transformations and quadratic transformations. Core transformations consist of the DCT (discrete cosine transform) and DST (discrete sine transform) families, which are applied to all rows and columns of the residual block. Subsequently, quadratic transformations can be applied to the upper left corner of the output of the core transformation. Similarly, inverse transformations can be applied in the order of quadratic inverse and core inverse. Specifically, the quadratic inverse can be applied to the upper left corner of the coefficient block. Subsequently, core inverse transformations are applied to the rows and columns of the output of the quadratic inverse. Core transformations / inverse transformations are sometimes referred to as linear transformations / inverse transformations.

[0135] Overview of AMT (Adaptive Multiple Core Transform)

[0136] In addition to the existing DCT-2 and 4×4DST-7, adaptive (or explicit) multiple transform (AMT or EMT) techniques are available for residual coding of inter- and intra-coded blocks. AMT and EMT may be used in combination below. AMT allows the use of multiple transformations selected from the DCT / DST family in addition to existing transformations. The transformation matrices newly introduced in JEM are DST-7, DCT-8, DST-1, and DCT-5. The DST / DCT basis functions used in AMT are shown in Table 5.

[0137] [Table 5]

[0138] EMT can be applied to CUs with widths and heights equal to or smaller than 64, and whether or not EMT is applied can be controlled by a CU level flag. For example, if the CU level flag is 0, DCT-2 is applied to the CU to encode the resist. For luma coding blocks within CUs to which EMT is applied, two additional flags are signaled to identify the horizontal and vertical transforms to which EMT is used. In JEM, the manual of a block can be coded in transform skip mode. In intra-residual coding, a mode-dependent transform candidate selection process is used, based on other resistive statistics of other intra-prediction modes. For example, three transform subsets are defined as shown in Table 6, and a transform subset is selected based on the intra-prediction mode as shown in Table 7.

[0139] [Table 6]

[0140] [Table 7]

[0141] Along with the subset concept, the transformation subset is initially identified based on Table 6 using the intra-prediction mode for CUs where the CU-level EMT_CU_flag is 1. Thereafter, for each of the horizontal (EMT_TU_horizontal_flag) and vertical (EMT_TU_vertical_flag) transformations, one of the two transformation candidates within the identified transformation subset is selected based on explicit signaling using flags according to Table 7.

[0142] [Table 8]

[0143] Table 8 shows the transform configuration groups to which AMT is applied.

[0144] Referring to Table 8, the transformation setting groups are determined based on the prediction mode, and there can be a total of six groups (G0 to G5). G0 to G4 correspond to cases where intra-prediction is applied, and G5 indicates the transformation combination (or transformation set, transformation combination set) applied to the residual block generated by inter-prediction.

[0145] A single transformation combination can consist of a horizontal transform (or row transform) applied to the rows of the 2D block, and a vertical transform (or column transform) applied to the columns.

[0146] Here, every conversion setting group can have four conversion combination candidates. The four conversion combination candidates can be selected or determined via a conversion combination index from 0 to 3, and the conversion combination index can be encoded and transmitted from the encoder to the decoder.

[0147] For example, residual data (or residual signals) obtained by intra-prediction may have different statistical characteristics depending on the intra-prediction mode. Therefore, as shown in Table 8, other transformations may be applied to each intra-prediction method instead of the general cosine transform. In this disclosure, the transformation types may be denoted, for example, as DCT-Type2, DCT-II, and DCT-2.

[0148] Table 8 shows the cases when using 35 intra-prediction modes and when using 67 intra-prediction modes. Multiple transformation combinations can be applied to each transformation setting group separated within each intra-prediction mode column. For example, multiple transformation combinations can consist of four (row-direction transformation, column-direction transformation) combinations. As a specific example, in group 0, DST-7 and DCT-5 can be applied to both the row (horizontal) and column (vertical) directions, so a total of four combinations are possible.

[0149] Since a total of four transformation kernel combinations can be applied to each intra-prediction mode, a transformation combination index for selecting one of them can be transmitted for each transformation unit. In this disclosure, the transformation combination index may be referred to as the AMT index and can be represented by amt_idx.

[0150] Furthermore, in addition to the conversion kernels presented in Table 8, there may be cases where DCT-2 is optimal for both the row and column directions due to the characteristics of the residual signals. Therefore, by defining an AMT flag for each coding unit, the conversion can be applied adaptively. Here, if the AMT flag is 0, DCT-2 is applied for both the row and column directions, and if the AMT flag is 1, one of four combinations can be selected or determined via the AMT index.

[0151] For example, if the AMT flag is 0, and the number of conversion coefficients for a single conversion unit is less than 3, the conversion kernel in Table 8 will not be applied, and the DST-7 codec can be applied to both the row and column directions.

[0152] For example, if the conversion coefficient values ​​are parsed first and the number of conversion coefficients is less than 3, the AMT index will not be parsed and DST-7 will be applied, thereby reducing the amount of additional information transmitted.

[0153] For example, AMT can only be applied when both the width and height of the conversion unit are 32 or less.

[0154] As an example, Table 8 can be pre-configured via offline training.

[0155] For example, an AMT index can be defined as a single index that can simultaneously point to a combination of horizontal and vertical transformations. Alternatively, an AMT index can be defined as separate horizontal and vertical transformation indices.

[0156] Figure 5 is a flowchart showing the encoding process in which AMT is performed.

[0157] AMT can be applied regardless of whether it is a primary or secondary transformation. That is, there is no constraint that it must be applied to only one or the other; both are applicable. Here, a primary transformation can mean a transformation for initially transforming a residual block, and a secondary transformation can mean a transformation for applying a transformation to a block produced as a result of a primary transformation.

[0158] First, the encoding device can determine the transformation group corresponding to the current block (S510). Here, the transformation group may refer to the transformation groups mentioned above with reference to Table 8, but is not limited to these, and may consist of other transformation combinations.

[0159] The encoding device can perform conversions on the available candidate conversion combinations within the conversion group (S520).

[0160] As a result of the above conversion, the encoding device can determine or select the conversion combination with the lowest RD (Rate Distortion) cost (S530).

[0161] The encoding device can encode a transformation combination index corresponding to the selected transformation combination (S540).

[0162] Figure 6 is a flowchart showing the decoding process in which AMT is performed.

[0163] First, the decryption device can determine the conversion group for the current block (step S610).

[0164] The decoding device can parse the transformation combination index (S620). Here, the transformation combination index can correspond to any one of the multiple transformation combinations within the transformation group. The steps of determining the transformation group (S610) and parsing the transformation combination index (S620) may be performed simultaneously.

[0165] The decoding device can derive a conversion combination corresponding to the conversion combination index (S630). Here, the conversion combination can refer to the conversion combinations mentioned above with reference to Table 8, but is not limited to these. In other words, configurations using other conversion combinations are also possible.

[0166] The decoding device can perform an inverse transformation on the current block based on the transformation combination (S640). If the transformation combination consists of a row transformation and a column transformation, the row transformation can be applied first, followed by the column transformation. However, this process is not limited to this, and the order may be reversed.

[0167] Overview of Secondary Transformation and NSST Index Coding

[0168] For quadratic / inverse transformations, a mode-dependent non-separable secondary transform (MDNSST) can be applied. To maintain low complexity, MDNSST can only be applied to low-frequency coefficients after a linear transformation. If both the width (W) and height (H) of the transformation coefficient block are equal to or greater than 8, an 8x8 non-separable secondary transform is applied to the upper-left 8x8 region of the transformation coefficient block. Conversely, if either the width or height is less than 8, a 4x4 non-separable secondary transform is applied, and this 4x4 non-separable secondary transform can be performed on the upper-left min(8,W) x min(8,H) region of the transformation coefficient block, where min(A,B) is a function that outputs the smaller of A and B.

[0169] For both 4x4 and 8x8 block sizes, there can be a total of 35x3 non-separable quadratic transformations. Here, 35 represents the number of transformation sets identified by the intra-prediction mode, and 3 can represent the number of NSST candidates for each intra-prediction mode. The mapping from intra-prediction modes to transformation sets can be defined as shown in Table 9.

[0170] [Table 9]

[0171] Within a set of translations, an NSST index (NSST idx) can be coded to indicate a translation kernel. If NSST is not applied, an NSST index with a value of 0 can be signaled.

[0172] Secondary transformations (e.g., MDNSST) are not applied to blocks coded in transformation-skip mode. If the MDNSST index is signaled to the CU and is not 0, MDNSST is not used for blocks of components coded in transformation-skip mode within the CU. The general coding structure, including coefficient coding and NSST index coding, is shown in Figures 6 and 7. Additionally, a CBF (coded block flag) is encoded to determine whether coefficient coding and NSST coding are applied. In Figures 6 and 7, the CBF flag can represent a luma-block cbf flag (cbf_luma flag) or a chroma-block cbf flag (cbf_cb flag or cbf_cr flag). When the CBF flag is 1, transformation coefficients are coded.

[0173] Figure 7 is a flowchart showing the encoding process in which NSST is performed.

[0174] Referring to Figure 7, the encoding device checks whether the CBF flag is 1 or not (S710). If the CBF flag is 0 ("NO" in S710), the encoding device does not perform transformation coefficient coding or NSST index coding. Conversely, if the CBF flag is 1 ("YES" in S710), the encoding device performs coding on the transformation coefficients (S720). After that, the encoding device decides whether or not NSST index coding is applied (S730) and performs NSST index coding (S740). If NSST index coding is not applied ("NO" in S730), the encoding device terminates the transformation procedure without applying NSST and can proceed to subsequent steps (e.g., quantization).

[0175] Figure 8 is a flowchart showing the decoding process in which NSST is performed.

[0176] Referring to Figure 8, the decoder checks whether the CBF flag is 1 or not (S810). If the CBF flag is 0 ("NO" in S810), the decoder does not perform conversion coefficient decoding or NSST index decoding. Conversely, if the CBF flag is 1 ("YES" in S810), the decoder performs decoding on the conversion coefficients (S820). After that, the decoder determines whether or not NSST index coding is applied (S830) and parses the NSST index (S840).

[0177] NSST can only be applied to the upper left 8x8 or 4x4 region of a block to which a linear transformation has been applied, rather than to the entire TU. For example, if the block size is 8x8 or larger, 8x8NSST can be applied, and if the block size is less than 8x8, 4x4NSST can be applied. Also, if 8x8NSST is applied, 4x4NSST can be applied to each 4x4 block. Both 8x8NSST and 4x4NSST follow the transformation set configuration described above; 8x8NSST can have 64 input data and 64 output data, while 4x4NSST can have 16 inputs and 16 outputs.

[0178] Figures 9 and 10 are diagrams illustrating the NSST execution method. Figure 9 shows Givens Rotation, and Figure 10 shows a 4x4 NSST Round configuration consisting of a Givens Rotation layer and Permutation.

[0179] Both 8x8 NSST and 4x4 NSST can be constructed as hierarchical combinations of Givens rotations. The matrix corresponding to one Givens rotation is given by Equation 1, and the matrix product can be represented graphically as shown in Figure 9.

[0180]

number

[0181] As shown in Figure 9, two input data x m and x n Applying the matrix from Equation 1 to the two output data t m and t n It can be obtained.

[0182] Since one Givens rotation rotates two data points, processing 64 data points (in the case of 8x8 NSST) or 16 data points (in the case of 4x4 NSST) requires 32 or 8 Givens rotations, respectively. Therefore, a Givens rotation layer can be constructed by grouping 32 or 8 Givens rotations.

[0183] As shown in Figure 10, the output data for one Givens rotation layer is transmitted to the input data for the next Givens rotation layer via substitution (or shuffling). The substitution patterns, as shown in Figure 10, are defined regularly, and in the case of 4x4NSST, four Givens rotation layers and their corresponding substitutions form one round. 4x4NSST is performed in two rounds, and 8x8NSST is performed in four rounds. Different rounds use the same substitution pattern, but the Givens rotation angles applied may differ. Therefore, it is necessary to save the angle data for all Givens rotations that constitute each transformation.

[0184] As a final step, the data output after passing through the Givens rotation layer undergoes one final substitution, and information about this substitution is stored separately for each transformation. In the case of forward NSST, this substitution is performed last, and in the case of reverse NSST, the reverse process of this substitution (i.e., reverse substitution or inverse substitution) is performed first.

[0185] In the case of reverse NSST, the substitution is applied in the reverse order of the Givens rotation layer applied in forward NSST, and a negative (-) value is added to each Givens rotation angle.

[0186] Overview of RST (Reduced Secondary Transform)

[0187] Figures 11 and 12 are diagrams for explaining the RST execution method.

[0188] Assuming that an orthogonal matrix representing one transformation has an N×N form, the RT (reduced transform) leaves only R (R < N) out of the N transformation basis vectors. The matrix for the forward RT that generates the transform coefficients can be defined as in Equation 2.

[0189]

Number

[0190] Since the matrix for the inverse RT is the transpose matrix of the forward RT matrix, if the application of the forward RT and the inverse RT is formulated, it is as shown in (a) and (b) of FIG. 11.

[0191] The RT applied to the 8x8 block at the top left of a transformation coefficient block to which a linear transformation has been applied is sometimes referred to as 8x8RST. When the value of R is 16 in Equation 2, the forward 8x8RST has a 16x64 matrix form, and the reverse 8x8RST has a 64x16 matrix form. The transformation set configurations shown in Table 9 can also be applied to 8x8RST. That is, 8x8RST can be determined by the transformation set in Table 9. Since one transformation set consists of two or three transformations depending on the intra-prediction mode, one of up to four transformations can be selected, including the case where a quadratic transformation is not applied (one transformation can correspond to the identity matrix). Assuming that indices 0, 1, 2, and 3 are assigned to the four transformations respectively (for example, index 0 can be assigned to the identity matrix, i.e., when a quadratic transformation is not applied), the transformation to be applied can be specified by signaling the syntax element corresponding to the NSST index for each transformation coefficient block. In other words, for the top-left corner block of an 8x8 grid, an 8x8NSST can be specified in the case of NSST, and an 8x8RST can be specified in the case of RST, via the NSST index.

[0192] Applying the forward 8x8RST as shown in equation 2 above generates 16 effective conversion coefficients. Therefore, the 64 input data points constituting the 8x8 region can be considered to be reduced to 16 output data points. From a two-dimensional perspective, the effective conversion coefficients fill only 1 / 4 of the region. Consequently, the 16 output data points obtained by applying the forward 8x8RST fill the upper left region in Figure 12.

[0193] Figure 12 shows the process of performing reverse scans from the 64th to the 17th scan in the reverse scan order.

[0194] In Figure 12, the 4x4 region in the upper left corner becomes the ROI (region of interest) region where valid conversion coefficients are found, and the remaining region is empty. The empty region can be filled with a default value of 0. If a non-zero valid conversion coefficient is found in a region other than the ROI region in Figure 12, it is certain that 8x8RST has not been applied, and NSST index coding can be omitted. Conversely, if no non-zero conversion coefficient is found in a region other than the ROI region in Figure 12 (i.e., the region other than the ROI is filled with 0), it is possible that 8x8RST has been applied, and the NSST index can be coded. This type of conditional NSST index coding requires checking for the presence or absence of non-zero conversion coefficients, so it can be performed after the residual coding process.

[0195] In this disclosure, NSST / RT / RST may be collectively referred to as LFNST, and the NSST index or (R)ST index may be collectively referred to as the LFNST index. LFNST can be applied to low-frequency conversion coefficients located in the upper left region of the conversion coefficient block in a non-separable conversion format based on a conversion kernel (conversion matrix or conversion matrix kernel).

[0196] The embodiments of this disclosure will be described in detail below with reference to the attached drawings.

[0197] Figure 13 is a diagram illustrating the conversion and inverse conversion processes according to one embodiment of the present disclosure. In Figure 13, the conversion unit 1310 can correspond to the conversion unit 120 in Figure 2, and the inverse conversion unit 1320 can correspond to the inverse conversion unit 150 in Figure 2 or the inverse conversion unit 230 in Figure 3.

[0198] Referring to Figure 13, the conversion unit 1310 can include a primary conversion unit 1311 and a secondary conversion unit 1312.

[0199] The primary transformation unit 1311 can apply a primary transformation to a residual sample (A) to generate a (primary) transformation coefficient (B). In this disclosure, the primary transformation may be referred to as a core transform.

[0200] The first-order conversion can be performed based on the MTS scheme. When an existing MTS is applied, a spatial-domain to frequency-domain conversion can be applied to the residual signal (or residual block) based on DCT type 2, DST type 7, and DCT type 8, etc., to generate conversion coefficients (or first-order conversion coefficients). Here, DCT type 2, DST type 7, and DCT type 8, etc., are sometimes referred to as conversion types, conversion kernels, or conversion cores. Examples of basis functions for DCT type 2, DST type 7, and DCT type 8 are as described above with reference to Table 3. However, this is illustrative, and embodiments of this disclosure may also apply to cases where the configuration of the existing MTS kernel is different, i.e., when other types of DCT / DST or conversion skips are included.

[0201] Existing MTS (Multi-Transition System) is a separable transform system that applies one kernel horizontally and another kernel vertically. While it is generally known that non-separable transform kernels offer even higher encoding / decoding efficiency than separable transform kernels, conventional linear transforms do not utilize non-separable transform methods.

[0202] Therefore, according to the embodiments of this disclosure, a primary transform can be performed based on a non-separable transform kernel. In this disclosure, a primary transform based on a non-separable transform kernel may be referred to as a non-separable primary transform or a non-separable core transform.

[0203] The unseparable linear transform can replace at least one of the existing MTS candidates or be added as a new MTS candidate. For example, only DCT type 2 and the unseparable linear transform may be used as MTS candidates, or the unseparable linear transform may be used in addition to DCT type 2, DST type 7, and DCT type 8 as MTS candidates.

[0204] Because the non-separable linear transformation is included in the MTS candidates, the MTS index table in Table 1 (for example, tu_mts_idx[x0][y0]) can be modified as shown in Table 10 or Table 11, for example.

[0205] [Table 10]

[0206] [Table 11]

[0207] In Tables 10 and 11, trTypeHor can represent a horizontal transformation kernel, and trTypeVer can represent a vertical transformation kernel. In Table 10, a trTypeHor / trTypeVer value of 0 represents DCT type 2, a trTypeHor / trTypeVer value of 1 represents DST type 7, and a trTypeHor / trTypeVer value of 2 represents an unseparable linear transformation. In Table 11, a trTypeHor / trTypeVer value of 0 represents DST type 2, a trTypeHor / trTypeVer value of 1 represents DST type 7, and a trTypeHor / trTypeVer value of 2 represents DST type 8. Furthermore, a trTypeHor / trTypeVer value of 3 can represent an unseparable linear transformation. However, this is merely an example, and the embodiments of this disclosure are not limited thereto.

[0208] Since an inseparable linear transformation has the attribute that horizontal and vertical transformations are not separated, the transformation kernel for an inseparable linear transformation must always be the same for both the horizontal and vertical directions. Therefore, according to one embodiment of this disclosure, if trTypeHor has a value indicating an inseparable linear transformation, the trTypeVer value can also be constrained to have a value indicating an inseparable linear transformation. For example, in Table 11, if the trTypeHor value is 3, the trTypeVer value must not be between 0 and 2, but can be constrained to only 3.

[0209] On the other hand, in some embodiments, an inseparable linear transformation may be added as an option separate from the MTS scheme described above. For example, an inseparable linear transformation may not be included in the MTS candidates and may be used as an independent transformation candidate. In this case, a predetermined first flag (e.g., nspt_flag) may be signaled to indicate whether or not an inseparable linear transformation is applied, and a second flag (e.g., mts_flag) indicating whether or not MTS is applied may be signaled only when the first flag is 0 (i.e., indicating that an inseparable linear transformation is not applied).

[0210] An unseparable linear transform can be performed, for example, with a 4x4 block as input, as follows. An example of a 4x4 input block X is given by equation 3.

[0211]

number

[0212] When the input block X is expressed in vector form, it can be shown as in equation 4.

[0213]

number

[0214] In this case, the inseparable linear transformation can be calculated as shown in equation 5.

[0215]

number

[0216] Here, The JPEG0007879170000017.jpg11105 transformation coefficient vector is shown. The JPEG0007879170000018.jpg file shows a 1011516×16 non-separable transformation matrix. JPEG0007879170000019.jpg8100 This represents the multiplication of a matrix and a vector.

[0217] Equation 5 allows us to derive a 16 × 1 transformation coefficient vector, JPEG0007879170000020.jpg12123 can be reconstructed into 4x4 blocks according to the scan order (e.g., horizontal, vertical, diagonal, or a predetermined / storage scan order). However, this is merely an example, and various optimized inseparable transform calculation methods may be used to reduce the computational complexity of the inseparable linear transform.

[0218] Thus, according to the embodiments of this disclosure, the primary conversion unit 1311 can generate (primary) conversion coefficients B by applying an unseparated primary conversion to the residual sample A.

[0219] The secondary transformation unit 1312 can generate (secondary) transformation coefficients C by applying a secondary transformation to (primary) transformation coefficients B. In one example, the aforementioned LFNST can be applied as the secondary transformation. The (secondary) transformation coefficients C can then be encoded by quantization and entropy coding processes and used to generate a bitstream.

[0220] Next, the inverse conversion unit 1320 may include an (inverse)secondary conversion unit 1321 and an (inverse)first-order conversion unit 1322.

[0221] The (inverse) quadratic transformation unit 1321 can generate (first-order) (inverse) transformation coefficients B' by applying an (inverse) quadratic transformation to the inversely quantized (second-order) transformation coefficients C''. Here, the (inverse) quadratic transformation can correspond to the inverse process of the quadratic transformation performed by the transformation unit 1310.

[0222] The (inverse) linear transformation unit 1322 can generate a residual sample A' by applying the (inverse) linear transformation to the (linear) (inverse) transformation coefficient B'.

[0223] According to embodiments of this disclosure, the (inverse) linear transform may include an inseparable linear inverse transform. The inseparable linear inverse transform may be included as an MTS candidate or provided as an independent inverse transform candidate. The inseparable linear inverse transform corresponds to the inverse process of the inseparable linear transform, and its specific details are as described above in relation to the inseparable linear transform.

[0224] On the other hand, inseparable linear transformations are fundamentally the same as or similar to inseparable quadratic transformations in that they apply the transformation simultaneously without separating the horizontal and vertical transformations. However, they have different characteristics in specific transformation methods, such as the target of the transformation, the transformation matrix, and the zero-out range. The differences between inseparable linear transformations and inseparable quadratic transformations will be explained in detail below with reference to Figures 14a to 16d.

[0225] Figures 14a to 14d are diagrams illustrating the non-separable quadratic transformation process.

[0226] Figures 14a to 14d show the unseparated quadratic conversion process in the encoder stage, i.e., the forward unseparated quadratic conversion process.

[0227] Figure 14a shows the case where the input block size is 4x4, and Figure 14b shows the case where the input block size is 8x4 or 4x8. Furthermore, Figure 14c shows the case where the input block size is 8x8, and Figure 14d shows the case where the input block size is 16x8 or 8x16.

[0228] In Figures 14a to 14d, the areas shown with thick lines indicate regions to which the non-separable quadratic transform is applied.

[0229] First, referring to Figure 14a, the unseparated quadratic transform can be applied to the entire 4x4 input block.

[0230] Specifically, a 16x8 inseparable quadratic transformation matrix can be applied to the 16 (linear) transformation coefficients of a 4x4 input block to generate 8 (quadratic) transformation coefficients. The 16x8 matrix is ​​used to reduce the multiplication complexity in the worst case. Such an inseparable quadratic transformation is sometimes called an RST (reduced secondary transform) because the number of (forward-referenced) output coefficients is less than the number of input coefficients.

[0231] The eight output coefficients can be placed in the top-left region (shaded area) of the 4x4 output block according to the diagonal scan order. The remaining areas in the 4x4 output block where the output coefficients are not filled can be filled with zero values ​​(i.e., zero out). In this case, the zero out area becomes the bottom-right region of the 4x4 output block.

[0232] Next, referring to Figure 14b(a), the unseparated quadratic transform can only be applied to the left 4x4 region of the 8x4 input block. This is because the unseparated quadratic transform is performed in units of 4x4 or 8x8 regions.

[0233] Specifically, 16 (quadratic) transformation coefficients can be generated by applying a 16x16 inseparable quadratic transformation matrix to the 16 (linear) transformation coefficients in the left-hand 4x4 region. The 16x16 matrix is ​​used to reduce the multiplication complexity in the worst-case scenario.

[0234] The 16 output coefficients can be placed in the left 4x4 region (shaded region) of the 8x4 output block according to the diagonal scan order. Since the area targeted by the inseparable quadratic transform (i.e., the left 4x4 region) is completely filled with output coefficients, zero-out may not occur. The remaining area in the 8x4 output block to which the inseparable quadratic transform has not been applied still contains the (linear) transform coefficients.

[0235] Referring to Figure 14b(b), the inseparable quadratic transform can only be applied to the upper 4x4 region of the 4x8 input block. As with Figure 14b(a), this is because the inseparable quadratic transform is performed in 4x4 or 8x8 region units.

[0236] Specifically, 16 (quadratic) transformation coefficients can be generated by applying a 16x16 inseparable quadratic transformation matrix to the 16 (linear) transformation coefficients in the upper 4x4 region.

[0237] The 16 output coefficients can be placed in the upper 4x4 region (shaded region) of the 4x8 output block according to the diagonal scan order. Since the area targeted by the inseparable quadratic transform (i.e., the upper 4x4 region) is completely filled with output coefficients, zero-out may not occur. Then, in the remaining area of ​​the 4x8 output block to which the inseparable quadratic transform has not been applied, the (linear) transform coefficients remain as they are.

[0238] Next, referring to Figure 14c, the unseparated quadratic transform can be applied to the entire 8x8 input block.

[0239] Specifically, by applying a 48x8 inseparable quadratic transformation matrix to the 48 (linear) transformation coefficients of an 8x8 input block, eight (quadratic) transformation coefficients can be generated. The 48x8 matrix is ​​used to reduce the multiplication complexity in the worst case. The forward inseparable quadratic transformation uses a 48x1 or 16x1 vector as the input vector, and only the (linear) transformation coefficients in the 4x4 regions at the top left, top right, and bottom left of the 8x8 input block can be used as input values ​​for the inseparable quadratic transformation. In other words, the (linear) transformation coefficients in the 4x4 region at the bottom right of the 8x8 input block are not used as input values ​​for the inseparable quadratic transformation.

[0240] The eight output coefficients can be placed in the upper left region (shaded region) of the 8x8 output block according to the diagonal scan order. Then, the remaining region in the 8x8 output block where the output coefficients are not filled can be filled with zero values. In this case, the 4x4 region at the lower right of the 8x8 output block is not used as an input value for the unseparated quadratic transform, so zero-out is not performed for that region, and the (linear) transform coefficients remain as they are.

[0241] Next, referring to Figure 14d(a), the unseparated quadratic transform can only be applied to the left 8x8 region of the 16x8 input block. This is because the unseparated quadratic transform is performed in units of 4x4 or 8x8 regions.

[0242] Specifically, by applying a 48x16 inseparable quadratic transformation matrix to the 48 (linear) transformation coefficients in the left 8x8 region, 16 (quadratic) transformation coefficients can be generated. The 48x16 matrix is ​​used to reduce the multiplication complexity in the worst case. Since the forward inseparable quadratic transformation uses a 48x1 or 16x1 vector as the input vector, the (linear) transformation coefficients in the 4x4 region at the bottom right of the left 8x8 region are no longer used as input values ​​for the inseparable quadratic transformation.

[0243] The 16 output coefficients can be placed in the 4x4 region (shaded area) at the top left of the 16x8 output block according to the diagonal scan order. Then, the remaining areas in the 16x8 output block where the output coefficients are not filled can be filled with zero values. In this case, the 4x4 region at the bottom right of the left 8x8 area is not used as an input value for the unseparated quadratic transform, so zero-out is not performed for that area, and the (linear) transform coefficients remain as they are.

[0244] Referring to Figure 14d(b), the unseparated quadratic transform can only be applied to the upper 8x8 region of the 8x16 input block. As with Figure 14d(a), this is because the unseparated quadratic transform is performed in 4x4 or 8x8 region units.

[0245] Specifically, by applying a 48x16 inseparable quadratic transformation matrix to the 48 (linear) transformation coefficients in the upper 8x8 region, 16 (quadratic) transformation coefficients can be generated. The 48x16 matrix is ​​used to reduce the multiplication complexity in the worst case. Since the forward inseparable quadratic transformation uses a 48x1 or 16x1 vector as the input vector, the (linear) transformation coefficients in the 4x4 region at the bottom right of the upper 8x8 region are no longer used as input values ​​for the inseparable quadratic transformation.

[0246] The 16 output coefficients can be placed in the 4x4 region (shaded area) at the top left of the 8x16 output block according to the diagonal scan order. Then, the remaining areas in the 8x16 output block where the output coefficients are not filled can be filled with zero values. In this case, the 4x4 region at the bottom right of the upper 8x8 area is not used as an input value for the unseparated quadratic transform, so zero-out is not performed for that area, and the (linear) transform coefficients remain as they are.

[0247] As described above with reference to Figures 14a to 14d, the unseparable quadratic transform can only be applied to the upper left region of the input block. Furthermore, the zero-out region can be limited to the region where the (quadratic) transform coefficients are not satisfied within the region to which the unseparable quadratic transform was actually applied.

[0248] Figures 15a to 15d are diagrams for explaining a non-separable primary conversion process according to an embodiment of the present disclosure.

[0249] Figures 15a to 15d show a non-separable primary conversion in the encoder stage, that is, a forward non-separable primary conversion process.

[0250] Figure 15a shows the case where the size of the input block is 4×4, and Figure 15b shows the case where the size of the input block is 8×4 or 4×8. Also, Figure 15c shows the case where the size of the input block is 8×8, and Figure 15d shows the case where the size of the input block is 16×8 or 8×16.

[0251] In Figures 15a to 15d, the regions indicated by thick lines show the regions to which the non-separable primary conversion is applied.

[0252] First, referring to Figure 15a, the non-separable primary conversion can be applied to the entire 4×4 input block. Specifically, 16 (primary) conversion coefficients can be generated by applying a 16×16 non-separable primary conversion matrix to the 16 residual samples of the 4×4 input block.

[0253] Next, referring to (a) of Figure 15b, the non-separable primary conversion can be applied to the entire 8×4 input block. Specifically, 32 (primary) conversion coefficients can be generated by applying a 32×32 non-separable primary conversion matrix to the 32 residual samples of the 8×4 input block. Also, referring to (b) of Figure 15b, the non-separable primary conversion can be applied to the entire 4×8 input block. Specifically, 32 (primary) conversion coefficients can be generated by applying a 32×32 non-separable primary conversion matrix to the 32 residual samples of the 4×8 input block.

[0254] Next, referring to Figure 15c, the inseparable linear transformation can be applied to the entire 8x8 input block. Specifically, by applying a 64x64 inseparable linear transformation matrix to the 64 residual samples of the 8x8 input block, 64 (linear) transformation coefficients can be generated.

[0255] Next, referring to Figure 15d(a), the inseparable linear transformation can be applied to the entire 16×8 input block. Specifically, by applying a 128×128 inseparable linear transformation matrix to the 128 residual samples of the 16×8 input block, 128 (linear) transformation coefficients can be generated. Also, referring to Figure 15d(b), the inseparable linear transformation can be applied to the entire 8×16 input block. Specifically, by applying a 128×128 inseparable linear transformation matrix to the 128 residual samples of the 8×16 input block, 128 (linear) transformation coefficients can be generated.

[0256] Thus, the unseparated linear transformation according to the embodiments of this disclosure can be applied to the entire region of the input block. Furthermore, since all residual samples in the input block are (linear) transformed by the unseparated linear transformation, zero-outs are eliminated. In this respect, the unseparated linear transformation has characteristics that differ from the unseparated quadratic transformation.

[0257] On the other hand, Figures 15a to 15d show the case where the number of input samples and the number of output coefficients of the inseparable linear transform are the same. However, depending on the embodiment, an RT (reduced transform) form of inseparable linear transform may be applied in which the number of output coefficients of the inseparable linear transform is smaller than the number of input samples.

[0258] Figures 16a to 16d illustrate the process of an unseparable linear transformation according to another embodiment of the present disclosure. The unseparable linear transformations in Figures 16a to 16d are similar to those in Figures 15a to 15d in that the unseparable linear transformation is applied to the entire input block. The following explanation will focus on the differences between the two, omitting redundant explanations.

[0259] First, referring to Figure 16a, a 16x8 inseparable linear transformation matrix can be applied to 16 residual samples in a 4x4 input block to generate 8 (linear) transformation coefficients. Such an inseparable linear transformation is sometimes called an RPT (reduced primary transform) or RCT (reduced core transform) because the number of (forward-referenced) output coefficients resulting from the inseparable linear transformation is less than the number of input samples.

[0260] The eight output coefficients can be placed in the upper left region (shaded area) of the 4x4 output block according to the diagonal scan order. Then, all remaining areas in the 4x4 output block where the output coefficients are not filled can be filled with zero values ​​(i.e., zero out). This corrects all residual sample values.

[0261] Next, referring to Figure 16b(a), a 32x16 non-separable linear transformation matrix can be applied to the 32 residual samples in the 8x4 input block to generate 16 (linear) transformation coefficients. The 16 output coefficients can be placed in the left 4x4 region (shaded region) of the 8x4 output block according to the diagonal scan order. Then, all remaining regions in the 8x8 output block that do not have output coefficients can be filled with zero values. In this way, all residual sample values ​​can be corrected.

[0262] Referring to (b) in Figure 16b, a 32x16 non-separable linear transformation matrix can be applied to the 32 residual samples in the 4x8 input block to generate 16 (linear) transformation coefficients. The 16 output coefficients can be placed in the upper 4x4 region (shaded region) of the 4x8 output block according to the diagonal scan order. Then, all remaining regions in the 8x8 output block where the output coefficients are not filled can be filled with zero values. In this way, all residual sample values ​​can be corrected.

[0263] Next, referring to Figure 16c, a 64x8 non-separable linear transformation matrix can be applied to the 64 residual samples of the 8x8 input block to generate 8 (linear) transformation coefficients. The 8 output coefficients can be placed in the upper left region (shaded region) of the 8x8 output block according to the diagonal scan order. Then, all remaining regions in the 8x8 output block that do not have output coefficients can be filled with zero values. In this way, all residual sample values ​​can be corrected.

[0264] Next, referring to Figure 16d(a), a 128x16 non-separable quadratic transformation matrix can be applied to the 128 residual samples of the 16x8 input block to generate 16 (linear) transformation coefficients. The 16 output coefficients can be placed in the 4x4 region (shaded area) at the top left of the 16x8 output block according to the diagonal scan order. Then, all remaining areas in the 16x8 output block that do not have output coefficients can be filled with zero values. In this way, all residual sample values ​​can be corrected.

[0265] Referring to Figure 16d(b), a 128x16 non-separable quadratic transformation matrix can be applied to the 128 residual samples of the 8x16 input block to generate 16 (linear) transformation coefficients. The 16 output coefficients can be placed in the 4x4 region (shaded area) at the top left of the 16x8 output block according to the diagonal scan order. Then, all remaining areas in the 16x8 output block that do not have output coefficients can be filled with zero values. In this way, all residual sample values ​​can be corrected.

[0266] Thus, the unseparated linear transform according to the embodiments of this disclosure can be applied to the entire region of the input block. Furthermore, zero-out can be performed on all remaining regions in the output block where the (linear) transform coefficients are not satisfied. This allows all input values ​​(i.e., residual sample values) of the unseparated linear transform to be corrected. In this respect, the unseparated linear transform has characteristics that differ from the unseparated quadratic transform.

[0267] Figure 17 is a flowchart showing a conversion method according to one embodiment of the present disclosure.

[0268] The conversion method shown in Figure 17 can be performed by the image encoding device shown in Figure 2. For example, steps S1710 to S1730 can be performed by the conversion unit 120.

[0269] Referring to Figure 17, the image encoding device can generate (linear) transformation coefficients by applying an unseparated linear transformation to the residual samples (S1710). The unseparated linear transformation can modify all residual samples within a residual block. In one embodiment, the unseparated linear transformation may have a reduced transform (RT) form in which the number of output coefficients is smaller than the number of input samples. In this case, zero-out can be performed for all regions in which no output coefficients have been generated.

[0270] The image encoding device can determine whether to apply a secondary transform to the (primary) transform coefficients (S1720). The secondary transform may be a non-separable secondary transform, for example, NSST or RST. In one embodiment, the image encoding device can determine whether to apply the secondary transform based on the primary-transformed residual transform coefficients. For example, when the number of non-zero residual transform coefficients included in the secondary-transform application target region is greater than or equal to a predetermined threshold, the image encoding device can determine that the secondary transform is to be applied. In contrast, when the number of non-zero residual transform coefficients included in the secondary-transform application target region is less than the predetermined threshold, the image encoding device can determine that the secondary transform is not applied. Information regarding whether the secondary transform is applied can be encoded as a predetermined syntax element (e.g., sps_lfnst_enabled_flag, lfnst_idx, etc.).

[0271] When it is determined that the secondary transform is to be applied (``YES'' in S1720), the image encoding device can apply the secondary transform to the (primary) transform coefficients to generate (secondary) transform coefficients (S1730). In this case, a bitstream can be generated based on the (secondary) transform coefficients.

[0272] In contrast, when it is determined that the secondary transform is not applied (``NO'' in S1720), the image encoding device may not perform the secondary transform on the (primary) transform coefficients. In this case, a bitstream can be generated based on the (primary) transform coefficients.

[0273] FIG. 18 is a flowchart showing a reverse transform method according to an embodiment of the present disclosure.

[0274] The reverse transform method of FIG. 18 can be performed by the image encoding device of FIG. 2 or the image decoding device of FIG. 3. For example, steps S1810 to S1830 can be performed by the reverse transform unit 150 of FIG. 2 or the reverse transform unit 230 of FIG. 3. Hereinafter, for convenience of explanation, the image decoding device will be described as a reference.

[0275] Referring to Figure 18, the image decoding device can determine whether or not to apply a quadratic inverse transform to the transformation coefficients obtained from the bitstream (S1810). The quadratic inverse transform may be an unseparated quadratic inverse transform, such as NSST or RST. In one embodiment, the image decoding device can determine whether or not to apply a quadratic inverse transform based on predetermined syntax elements obtained from the bitstream (e.g., sps_lfnst_enabled_flag, lfnst_idx, etc.). For example, if lfnst_idx has a first value (e.g., 0), the image decoding device can determine that a quadratic inverse transform is not applied. Conversely, if lfnst_idx has a value other than the first value (e.g., 0), the image decoding device can determine that a quadratic inverse transform is applied.

[0276] If it is determined that a quadratic inverse transform is to be applied (YES in S1810), the image decoding device can apply a quadratic inverse transform to the transform coefficients obtained from the bitstream to generate (first-order) transform coefficients (S1820). In this case, the transform coefficients obtained from the bitstream can correspond to (second-order) transform coefficients.

[0277] In contrast, if it is determined that a quadratic inverse transform is not applied (NO in S1810), the image decoder does not need to perform a quadratic inverse transform on the transformation coefficients obtained from the bitstream. In this case, the transformation coefficients obtained from the bitstream can correspond to (first-order) transformation coefficients.

[0278] The image decoding device can generate residual samples by applying an unseparated inverse linear transform to the (linear) transformation coefficients (S1830). All (linear) transformation coefficients can be corrected by the unseparated inverse linear transform. In one embodiment, the unseparated inverse linear transform may have a reduced transform (RT) form in which the number of output coefficients is greater than the number of input coefficients.

[0279] Non-separable primary transformation of subblock substrate

[0280] In one embodiment, instead of applying an inseparable linear transformation to a relatively large input block that matches the width and height of the block, the block can be divided into subblocks, and then an inseparable linear transformation can be applied using an inseparable transformation matrix that matches the width and height of each subblock. For example, when applying an inseparable linear transformation to a 4x8 block, the 4x8 block can be horizontally divided into two 4x4 subblocks in the spatial domain, and an inseparable linear transformation can be applied to each 4x4 subblock in 4x4 block units. Alternatively, when applying an inseparable linear transformation to a 16x8 block, the 16x8 block can be vertically divided into two 8x8 subblocks in the spatial domain, and an inseparable linear transformation can be applied to each 8x8 subblock in 8x8 block units.

[0281] Figure 19 is a flowchart showing a method for non-separable linear / inverse transformation of subblocks according to one embodiment of the present disclosure.

[0282] The conversion method in Figure 19 can be performed by the image encoding device in Figure 2. For example, steps S1910 to S1930 can be performed by the conversion unit 120. The inverse conversion method in Figure 19 can be performed by the image encoding device in Figure 2 or the image decoding device in Figure 3. For example, steps S1910 to S1930 can be performed by the inverse conversion unit 150 in Figure 2 or the inverse conversion unit 230 in Figure 3.

[0283] Referring to Figure 19, the image coding / decoding device can determine whether predetermined subblock transformation / inverse transformation conditions are met (S1910). In one embodiment, the image coding / decoding device can determine whether the subblock transformation / inverse transformation conditions are met based on the result of comparing the size of the input block with a predetermined threshold. Here, the threshold can include a first threshold of size 4×4 and a second threshold of size 8×8. Specifically, if the size of the input block is greater than the first threshold and less than the second threshold, the image coding / decoding device can determine that the subblock transformation / inverse transformation conditions are met. Also, if the size of the input block is greater than the second threshold, the image coding / decoding device can determine that the subblock transformation / inverse transformation conditions are met.

[0284] If the subblock conversion condition is met (YES in S1910), the image encoding / decoding device can divide the input block to obtain multiple subblocks (S1920). For example, the image encoding / decoding device can vertically divide an 8x4 block to obtain two 4x4 subblocks. Alternatively, the image encoding / decoding device can horizontally divide an 8x16 block to obtain two 8x8 subblocks.

[0285] In contrast, if the subblock conversion condition is not met ("NO" in S1910), the image encoding / decoding device may decide not to divide the input block and proceed to step S1930.

[0286] The image encoding / decoding device can then apply an inseparable linear transformation / inverse transformation to the input block or to each of its subblocks (S1930). When the inseparable linear transformation / inverse transformation is applied to the entire input block, the inseparable transformation matrix can be determined based on the width and height of the input block. In contrast, when the inseparable linear transformation / inverse transformation is applied to each of its subblocks, the inseparable transformation matrix can be determined based on the width and height of each subblock.

[0287] Figures 20a and 20b are diagrams illustrating the non-separable first-order transformation process of the subblock substrate.

[0288] Figures 20a and 20b show the unseparated linear transformation in the encoder stage, i.e., the forward unseparated linear transformation process.

[0289] Figure 20a shows the case where the input block size is 8×4 or 4×8, and Figure 20b shows the case where the input block size is 16×8 or 8×16.

[0290] In Figures 20a and 20b, the areas shown with thick lines indicate regions to which the unseparable linear transform is applied.

[0291] First, referring to (a) in Figure 20a, the 8×4 input block can be divided into two 4×4 subblocks Sb1 and Sb2. Then, a 4×4 inseparable linear transformation can be applied to each of the subblocks Sb1 and Sb2. Specifically, by applying a 16×16 inseparable linear transformation matrix to the first subblock Sb1, 16 (linear) transformation coefficients can be generated. Similarly, by applying a 16×16 inseparable linear transformation matrix to the second subblock Sb2, 16 (linear) transformation coefficients can be generated.

[0292] Referring to Figure 20a(b), the 4×8 input block can be divided into two 4×4 subblocks Sb3 and Sb4. A 4×4 inseparable linear transformation can then be applied to each of the subblocks Sb3 and Sb4. Specifically, by applying a 16×16 inseparable linear transformation matrix to the third subblock Sb3, 16 (linear) transformation coefficients can be generated. Similarly, by applying a 16×16 inseparable linear transformation matrix to the fourth subblock Sb4, 16 (linear) transformation coefficients can be generated.

[0293] Next, referring to Figure 20b(a), the 16×8 input block can be divided into two 8×8 subblocks Sb1 and Sb2. Then, an 8×8 inseparable linear transformation can be applied to each of the subblocks Sb1 and Sb2. Specifically, by applying a 64×64 inseparable linear transformation matrix to the first subblock Sb1, 64 (linear) transformation coefficients can be generated. Similarly, by applying a 64×64 inseparable linear transformation matrix to the second subblock Sb2, 64 (linear) transformation coefficients can be generated.

[0294] Referring to (b) in Figure 20b, the 8×16 input block can be divided into two 8×8 subblocks Sb3 and Sb4. Then, an 8×8 inseparable linear transformation can be applied to each of the subblocks Sb3 and Sb4. Specifically, by applying a 64×64 inseparable linear transformation matrix to the third subblock Sb3, 16 (linear) transformation coefficients can be generated. Similarly, by applying a 64×64 inseparable linear transformation matrix to the fourth subblock Sb4, 16 (linear) transformation coefficients can be generated.

[0295] On the other hand, depending on the embodiment, different inseparable linear transformation matrices may be applied to each subblock. For example, in Figure 2a, a 16×16 inseparable linear transformation matrix may be applied to the first subblock Sb1, and a 16×8 inseparable linear transformation matrix may be applied to the second subblock Sb2. In this case, the region in the second subblock Sb2 where no (linear) transformation coefficients have been generated can be filled with zero values ​​(i.e., zero out).

[0296] Unseparable linear transformation set and kernel determination method

[0297] According to embodiments of the present disclosure, an unseparated linear transformation set and / or kernel can be configured in a variety of ways based on at least one of the following: prediction mode (e.g., intra-prediction mode, inter-prediction mode, etc.), input block width / height, number of pixels in the input block, position of subblocks within the input block, explicitly signaled syntax elements, statistical properties of surrounding pixels, whether or not a quadratic transformation is applied, or a quantization parameter (QP).

[0298] In one embodiment, if the current block's prediction mode is intra-prediction mode, the intra-prediction modes can be grouped into N sets, and each set can contain k transformation kernels. In this case, the number of intra-prediction modes and the grouping method can be varied depending on the embodiment.

[0299] The following describes in detail a method for determining an unseparated linear transformation set based on an intra-prediction mode according to one embodiment of the present disclosure.

[0300] Figures 21 to 24 illustrate a method for determining an unseparable linear transformation set according to one embodiment of the present disclosure.

[0301] The intra-prediction mode may include two non-directional intra-prediction modes and 65 directional intra-prediction modes. The non-directional intra-prediction mode may include Planar intra-prediction mode (number 0) and DC intra-prediction mode (number 1), and the directional intra-prediction mode may include 65 intra-prediction modes (numbers 2 through 66). However, this is illustrative and the embodiments of this disclosure are not limited thereto.

[0302] By applying wide-angle intra prediction (WAIP), the directional intra prediction modes can further include intra prediction modes -14 to 1 and intra prediction modes 67 to 80. Figure 22 illustrates the extended intra prediction modes and their prediction directions in consideration of WAIP.

[0303] Referring to Figure 21, modes -14 to -1 and 2 to 33, and modes 35 to 80 are symmetrical with respect to the prediction direction with respect to mode 34. For example, modes 10 and 58 are symmetrical with respect to the direction corresponding to mode 34, and mode -1 is symmetrical with mode 67. Therefore, according to one embodiment of this disclosure, when constructing a transformation set for a linear transformation, the input data can be transposed and used for vertical modes that are symmetrical with respect to mode 34. Here, transposing the input data can be interpreted as constructing N×M data where rows become columns and columns become rows in a two-dimensional block data M×N.

[0304] In one example, when a 4x4 block is used as input data, the 16 data points constituting the 4x4 region can be appropriately arranged to form a 16x1 vector for an inseparable linear transformation. The 16x1 vector can be arranged in a row-major order, as shown in Figure 22a. This can represent the order in which two-dimensional data is arranged in one dimension for a forward inseparable linear transformation. It can also represent the order in which the transformation coefficients generated by the inverse inseparable linear transformation are arranged in two dimensions.

[0305] On the other hand, as mentioned above, directional modes -14 to -1 and 2 to 80 are symmetrically arranged around mode 34. Therefore, if the data arrangement order for constructing a 16×1 input vector for modes -14 to -1 and 2 to 33 is column-major, then for modes 35 to 80, the input vector can be constructed in the order shown in Figure 22b. The data arrangement order in Figure 23b is column-major. This can mean the order in which two-dimensional data is arranged in one dimension for a forward inseparable linear transformation. This can also mean the order in which the transformation coefficients generated by the inverse inseparable linear transformation are arranged in two dimensions.

[0306] On the other hand, mode 34 can be considered neither strictly horizontal nor vertical, but from the perspective of the data arrangement scheme of this disclosure, it is classified as belonging to the horizontal direction. That is, for modes -14 to -1 and 2 to 33, the input data sorting scheme for the horizontal mode, i.e., row-first sorting, can be used, and the input data can be transposed and used for the vertical mode which is symmetrical with respect to mode 34.

[0307] On the other hand, in the case of non-square blocks, it is not possible to utilize the symmetry in the positive direction block (i.e., the symmetry between the P mode and the 68-P mode (2≦P≦33) or the symmetry between the Q mode and the 66-Q mode (-14≦Q≦-1) in an N×N block). Therefore, according to another embodiment of the present invention, instead of utilizing the symmetry based solely on the intra-prediction mode described above, the symmetry between block configurations that are in a transpose relationship with each other, i.e., the symmetry between a K×L block and an L×K block, can be utilized together with the intra-prediction mode.

[0308] Specifically, referring to Figure 23, a symmetric relationship can exist between the K×L block predicted to be mode P and the L×K block predicted to be mode 68-P (2≦P≦33). Furthermore, a symmetric relationship can exist between the K×L block predicted to be mode Q and the L×K block predicted to be mode 66-Q (-14≦Q≦-1).

[0309] For example, as shown in Figure 24, a K×L block with mode 2 and an L×L block with mode 66 are symmetrical to each other, and as a result, the same transformation kernel can be applied to both blocks. If the transformation sets for intra-prediction modes are mapped based on the K×L block (i.e., there is a table for which transformation set to apply to each prediction mode), then in order to apply a transformation to an L×K block, instead of mode P applied to the L×K block, we can have mode 68-P (2≦P≦33) (or instead of mode Q applied to the L×K block, we can have mode 66-Q (-14≦Q≦-1)), and obtain the transformation set via the mapping table based on the K×L block. This allows us to select a transformation set based on mode 2 instead of mode 66 in the example in Figure 25 in order to apply a transformation to an L×K block. Furthermore, for K×L blocks, after scanning the input data in a predetermined order (e.g., row-major order, column-major order) to construct a one-dimensional vector, a forward inseparable linear transformation can be applied. Furthermore, for L×K blocks, the input data can be scanned according to its transposed order (i.e., if a K×L block is scanned in row-major order, it is scanned in column-major order; if a K×L block is scanned in column-major order, it is scanned in row-major order) to construct a one-dimensional vector, and then a forward inseparable linear transformation can be applied.

[0310] On the other hand, the intra-prediction mode in Figure 21 is illustrative, so even if the intra-prediction mode is set in a different manner, the inseparable linear transformation can be applied based on the intra-prediction mode and / or the symmetry between blocks as described above. Also, in the case of mode 34, when applied to a K×L block, the transformation set can be determined using mode 34 based on the K×L block, the input data can be scanned in a predetermined order to construct a one-dimensional vector, and then the inseparable linear transformation can be applied. Similarly, when applied to an L×K block, the transformation set can be determined using mode 34, the input data can be scanned in transposed order to construct a one-dimensional vector, and then the inseparable linear transformation can be applied.

[0311] The above describes a method of deriving a transformation set via intra-prediction mode based on a K×L block and constructing the input data. However, it is also possible to construct the input data based on an L×K block. In this case, the same symmetry described above can be used to apply the inseparable linear transformation to the K×L block. Furthermore, when a K×L block is used as the basis, it is possible to construct the data so that the relationship K > L is always satisfied. Additionally, when applying the inverse inseparable linear transformation, the same symmetry between the K×L block and the L×K block described above can be used to determine the transformation set, and the one-dimensional output data derived via the inverse transformation can be placed in a two-dimensional block. On the other hand, in the case of non-square blocks, it is possible to use a different number of transformation sets than in the case of square blocks without utilizing symmetry, and to select the transformation set using a different mapping table than in the case of square blocks.

[0312] Specific examples of mapping tables for selecting conversion sets are shown in Tables 12 through 14.

[0313] [Table 12]

[0314] Table 12 illustrates how to assign each transformation set to an intra-prediction mode when five transformation sets exist. In Table 12, the predModeIntra value represents the intra-prediction mode value modified to account for WAIP, and TrSetIdx represents the index value indicating a specific non-separable primary transformation set.

[0315] Referring to Table 12, it can be seen that the intra-prediction mode applies the same unseparated set of linear transformations to modes located in symmetrical directions. On the other hand, the mapping table in Table 12, which uses five transformation sets, is merely an example, and the embodiments of this disclosure are not limited thereto. For example, a mapping table may be constructed using six or more transformation sets, different from those in Table 12.

[0316] As another example in Table 12, to improve compression performance, either unseparated first-order transformations are not applied to WAIP (as in Table 13), or a separate transformation set is not configured for WAIP, and the mapping table is configured to use the same transformation set mapped to adjacent intra-predictive modes for WAIP (as in Table 14).

[0317] [Table 13]

[0318] [Table 14]

[0319] On the other hand, in one embodiment, the number of transformation sets for non-separable linear transformations and the number of transformation kernels within each set can be determined differently based on the width and / or height of the (input) block. For example, for a 4x4 block, n1 transformation sets and k1 transformation kernels within each set can be configured. In contrast, for a 4x8 block, n2 transformation sets and k2 transformation kernels within each set can be configured.

[0320] In one embodiment, the number of transformation sets for non-separable linear transformations and the number of transformation kernels within each set can be determined differently based on the product of the width and height of the (input) blocks. For example, if the product of the width and height of the (input) blocks is 256 or greater (or greater), n3 transformation sets and k3 transformation kernels within each set can be configured. Otherwise, n4 transformation sets and k4 transformation kernels within each set can be configured.

[0321] In one embodiment, the number of transformation sets and the number of transformation kernels within each set for an inseparable linear transformation can be determined differently depending on the position of each subblock currently contained within the transformation block. For example, if a 4x8 or 8x4 block is divided into two 4x4 subblocks and an inseparable linear transformation is applied to each subblock, then n5 transformation sets and k5 transformation kernels within each set can be configured for the inseparable linear transformation applied to the upper left 4x4 subblock. Conversely, for the inseparable linear transformation applied to the remaining 4x4 subblocks, n6 transformation sets and k6 transformation kernels within each set can be configured.

[0322] In one embodiment, a predetermined syntax element indicating information regarding non-separable primary transformation can also be explicitly signaled. For example, when three types of non-separable primary transformation configurations are supported (e.g., n7 transformation sets and k7 transformation kernels in each set, n8 transformation sets and k8 transformation kernels in each set, n9 transformation sets and k9 transformation kernels in each set), the syntax element can have any one value of 0, 1, or 2. At this time, the syntax elements with different values can indicate different non-separable primary transformation configurations.

[0323] In one embodiment, the configuration of non-separable primary transformation can be determined differently based on whether secondary transformation / inverse transformation is applied and / or the type of the secondary transformation / inverse transformation. For example, when secondary transformation is not applied, for non-separable primary transformation, n 10 transformation sets and k 10 transformation kernels in each set can be configured. In contrast, when secondary transformation is applied, n 11 transformation sets and k 11 transformation kernels in each set can be configured.

[0324] In one embodiment, the configuration of non-separable primary transformation can be determined differently based on the quantization parameter (QP) value and / or the range of QP values. For example, when the QP value is relatively small and large, a first configuration including n 12 transformation sets and k 12 transformation kernels in each set, and n 13 transformation sets and k 13A second configuration including a conversion kernel can be applied. In this case, the magnitude of the QP value can be determined based on a predetermined threshold (e.g., 32). For example, if the QP value is 32 or less (or less than 32), the QP value can be classified as relatively small. Conversely, if the QP value is greater than 32 (or greater than or equal to 32), the QP value can be classified as relatively large. Depending on the embodiment, it is also possible to divide the range of QP values ​​into three or more ranges and apply a different configuration to each range.

[0325] The image encoding / decoding method according to one embodiment of this disclosure will be described in detail below with reference to Figures 25 and 26.

[0326] Figure 25 is a flowchart showing an image encoding method according to one embodiment of the present disclosure.

[0327] The image encoding method shown in Figure 25 can be performed by the image encoding device shown in Figure 2. For example, steps S2510 and S2520 can be performed by the conversion unit 120.

[0328] Referring to Figure 25, the image encoding device can generate a transformation coefficient block for the current block by performing a transformation on the current block based on a predetermined non-separable linear transformation matrix (S2510).

[0329] In one embodiment, the transformation can be performed based on multiple transform selection (MTS).

[0330] In one embodiment, zeroing out of the transformation coefficient block can be selectively performed based on the size of the inseparable linear transformation matrix. For example, zeroing out may not be performed if the inseparable linear transformation matrix has a square size. Alternatively, zeroing out may be performed on at least one residual sample within the residual block if the inseparable linear transformation matrix has a non-square size.

[0331] In one embodiment, the non-separable linear transformation matrix can be determined from a predetermined set of non-separable transformations. In this case, the set of non-separable transformations can be configured differently based on at least one of the following: the prediction mode of the current block, the size, the number of pixels, the transformation set configuration information, whether or not a quadratic transformation is performed, or the quantization parameters.

[0332] In one embodiment, the conversion can be performed based on subblocks. Specifically, the image encoding device can determine whether the current block satisfies predetermined subblock conversion conditions, and if the subblock conversion conditions are met, it can divide the current block to obtain a plurality of subblocks. The image encoding device can perform the conversion on each of the plurality of subblocks.

[0333] The image encoding device can then encode the current block based on the conversion coefficient block (S2520).

[0334] In one embodiment, the non-separable linear transformation matrix can be applied to all residual samples of the current block, regardless of the size of the current block.

[0335] Figure 26 is a flowchart showing an image decoding method according to one embodiment of the present disclosure.

[0336] The image decoding method shown in Figure 26 can be performed using the image decoding device shown in Figure 3. For example, steps S2610 and S2620 can be performed using the inverse transform unit 230 shown in Figure 3.

[0337] Referring to Figure 26, the image decoding device can generate a residual block of the current block by performing an inverse transform on the current block based on a predetermined non-separable linear transformation matrix (S2610).

[0338] In one embodiment, the inverse transformation can be performed based on multiple transform selection (MTS).

[0339] In one embodiment, zeroing out of the transformation coefficient block can be selectively performed based on the size of the inseparable linear transformation matrix. For example, zeroing out may not be performed based on the inseparable linear transformation matrix having a square size. Alternatively, zeroing out may be performed on at least one residual sample within the residual block based on the inseparable linear transformation matrix having a non-square size.

[0340] In one embodiment, the non-separable linear transformation matrix can be determined from a predetermined set of non-separable transformations. In this case, the set of non-separable transformations can be configured differently based on at least one of the following: the prediction mode of the current block, the size, the number of pixels, the transformation set configuration information, whether or not a quadratic inverse transformation is performed, or the quantization parameters.

[0341] In one embodiment, the transformation can be performed based on subblocks. Specifically, the image encoding device can determine whether the current block satisfies predetermined subblock transformation conditions, and based on whether the subblock transformation conditions are met, it can divide the current block to obtain a plurality of subblocks. The image encoding device can perform the transformation on each of the plurality of subblocks. In this case, the subblock inverse transformation conditions may include a first condition regarding whether the size of the current block is greater than a first threshold, and a second condition regarding whether the size of the current block is greater than a second threshold.

[0342] In one embodiment, the non-separable linear transformation matrix can be determined to be the same matrix as the transformation matrix for a first block that is symmetrical with the current block in at least one of the intra-prediction mode and block configuration.

[0343] The image decoding device can then reconstruct the current block based on the residual block (S2620).

[0344] In one embodiment, the non-separable linear transformation matrix can be applied to all transformation coefficients of the current block, regardless of the size of the current block.

[0345] As described above, according to embodiments of this disclosure, an unseparated transform that does not separate horizontal and vertical transforms can be used as a linear transform. This can further improve the transform / inverse transform efficiency. The unseparated linear transform can be applied to the entire input block and can change the values ​​of all samples / coefficients in the input block. Depending on the embodiment, the unseparated linear transform can have an RT (reduced transform) form. The transform set mapping table for the unseparated linear transform can be configured based on the symmetry of the intra-prediction mode. In this case, the transform set used for the unseparated linear transform can be selected from the mapping table based on the symmetry of the intra-prediction mode and / or the symmetry between block forms.

[0346] The exemplary methods in this disclosure are presented as a series of actions for clarity of explanation, but this is not intended to restrict the order in which the steps are performed, and each step may be performed simultaneously or in a different order, if necessary. To implement the methods according to this disclosure, the exemplary steps may be further expanded to include other steps, or some steps may be expanded to include the remaining steps, or some steps may be expanded to include additional steps.

[0347] In this disclosure, an image encoding device or image decoding device that performs a predetermined operation (step) may perform an operation (step) to confirm the conditions or status of the execution of said operation (step). For example, if it is stated that a predetermined operation is performed when a predetermined condition is satisfied, the image encoding device or image decoding device may perform an operation to confirm whether or not the predetermined condition is satisfied, and then perform the predetermined operation.

[0348] The various embodiments of this disclosure are not intended to list all possible combinations, but rather to illustrate representative aspects of this disclosure. The matters described in the various embodiments may be applied independently or in combination of two or more.

[0349] Furthermore, various embodiments of this disclosure can be implemented by hardware, firmware, software, or a combination thereof. In the case of hardware implementation, it can be implemented by one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), general processors, controllers, microcontrollers, microprocessors, etc.

[0350] Furthermore, the image decoding and image encoding devices to which the embodiments of this disclosure are applied can be included in multimedia broadcasting transceivers, mobile communication terminals, home cinema video equipment, digital cinema video equipment, surveillance cameras, video conferencing equipment, real-time communication equipment such as video communications, mobile streaming equipment, storage media, camcorders, video-on-demand (VoD) service providers, over-the-top (OTT) video equipment, internet streaming service providers, three-dimensional (3D) video equipment, image-phone video equipment, and medical video equipment, and can be used to process video signals or data signals. For example, over-the-top (OTT) video equipment can include game consoles, Blu-ray players, internet-connected TVs, home theater systems, smartphones, tablet PCs, and digital video recorders (DVRs).

[0351] Figure 27 illustrates a content streaming system to which the embodiments of this disclosure can be applied.

[0352] As shown in Figure 27, a content streaming system to which an embodiment of the present disclosure is applied may broadly include an encoding server, a streaming server, a web server, media storage, user equipment, and multimedia input devices.

[0353] The encoding server is responsible for compressing content input from multimedia input devices such as smartphones, cameras, and camcorders into digital data to generate a bitstream, and transmitting this bitstream to the streaming server. In other cases, if a multimedia input device such as a smartphone, camera, or video camera directly generates the bitstream, the encoding server can be omitted.

[0354] The bitstream can be generated by an image encoding method and / or image encoding apparatus to which an embodiment of the present disclosure is applied, and the streaming server can temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[0355] The streaming server transmits multimedia data to the user's device based on the user's request via a web server, and the web server can act as an intermediary to inform the user of available services. When a user requests a desired service from the web server, the web server transmits this to the streaming server, and the streaming server can transmit multimedia data to the user. In this case, the content streaming system may include a separate control server, in which case the control server can play a role in controlling the commands and responses between the devices within the content streaming system.

[0356] The streaming server can receive content from media storage and / or encoding servers. For example, when receiving content from the encoding server, the content can be received in real time. In this case, in order to provide a smooth streaming service, the streaming server can store the bitstream for a certain period of time.

[0357] Examples of user devices include mobile phones, smartphones, laptop computers, digital broadcasting terminals, PDAs (personal digital assistants), PMPs (portable multimedia players), navigation systems, slate PCs, tablet PCs, ultrabooks, wearable devices such as smartwatches, smart glasses, HMDs (head-mounted displays), digital TVs, desktop computers, and digital signage.

[0358] Each server within the aforementioned content streaming system can be operated as a distributed server, in which case the data received from each server can be processed in a distributed manner.

[0359] The scope of this disclosure includes software or machine-executable commands (e.g., operating systems, applications, firmware, programs, etc.) that enable the operation of various embodiments to be performed on a device or computer, and non-transitory computer-readable medium on which such software or commands etc. are stored and can be executed on a device or computer. [Industrial applicability]

[0360] The embodiments described herein can be used for image encoding / decoding.

Claims

1. An image decoding method performed by an image decoding device, A step of generating a residual sample of the current block by performing an inverse transform on the current block based on a selection between a predetermined non-separable linear transformation matrix and a predetermined set of separable linear transformation matrices including a horizontal transformation matrix and a vertical transformation matrix, The steps include generating a reconstructed sample based on a residual sample, An image decoding method wherein the non-separable linear transformation matrix is ​​applied to all transformation coefficients of the current block, regardless of the size of the current block.

2. The image decoding method according to claim 1, wherein the inverse transformation is performed based on MTS (multiple transform selection).

3. The aforementioned inseparable linear transformation matrix is ​​determined from a predetermined set of inseparable transformations, The image decoding method according to claim 1, wherein the non-separated transform sets are configured to be different from each other based on at least one of the prediction mode, size, number of pixels, transform set configuration information, whether a quadratic inverse transform is performed, or quantization parameters of the current block.

4. The steps include determining whether the current block satisfies a predetermined subblock inverse transformation condition, Based on the condition that the inverse subblock transformation conditions are met, the step of obtaining multiple subblocks by dividing the current block, The image decoding method according to claim 1, further comprising the step of performing the inverse transform for each of the plurality of subblocks.

5. The image decoding method according to claim 4, wherein the subblock inverse transformation condition includes a first condition relating to whether the size of the current block is greater than a first threshold, and a second condition relating to whether the size of the current block is greater than a second threshold.

6. The image decoding method according to claim 1, wherein the non-separable linear transformation matrix is ​​equal to the transformation matrix for a first block having symmetry with the current block in block form.

7. An image encoding method performed by an image encoding device, A step of generating transformation coefficients for the current block by performing a transformation on the current block based on a selection between a predetermined non-separable linear transformation matrix and a predetermined set of separable linear transformation matrices including a horizontal transformation matrix and a vertical transformation matrix, The step of encoding residual information based on the conversion coefficient of the current block, An image encoding method in which the non-separable linear transformation matrix is ​​applied to all residual samples of the current block, regardless of the size of the current block.

8. The image encoding method according to claim 7, wherein the conversion is performed based on MTS (multiple transform selection).

9. The aforementioned inseparable linear transformation matrix is ​​determined from a predetermined set of inseparable transformations, The image coding method according to claim 7, wherein the non-separated transform sets are configured to be different from each other based on at least one of the prediction mode, size, number of pixels, transform set configuration information, whether a quadratic inverse transform is performed, or quantization parameters of the current block.

10. The steps include determining whether the current block satisfies a predetermined subblock inverse transformation condition, Based on the condition that the inverse subblock transformation conditions are met, the step of obtaining multiple subblocks by dividing the current block, The image encoding method according to claim 7, further comprising the step of performing the conversion for each of the plurality of subblocks.

11. A method relating to a bitstream of image information, A step of generating transformation coefficients for the current block by performing a transformation on the current block based on a selection between a predetermined non-separable linear transformation matrix and a predetermined set of separable linear transformation matrices including a horizontal transformation matrix and a vertical transformation matrix, The steps of generating the bitstream by encoding the residual information based on the conversion coefficient of the current block, The step of transmitting the data of the bitstream, A method in which the non-separable linear transformation matrix is ​​applied to all residual samples of the current block, regardless of the size of the current block.

Images