Video coding method and apparatus based on secondary conversion

Abstract

To relate to a video coding method.SOLUTION: A video coding method comprises: a step for deriving transform coefficients through inverse quantization on the basis of quantized transform coefficients for a target block; a step for deriving modified transform coefficients on the basis of an inverse reduced secondary transform (RST) of the transform coefficients; and a step for generating a reconstructed picture on the basis of residual samples for the target block on the basis of an inverse primary transform of the modified transform coefficients. The inverse RST using a transform kernel matrix is performed on transform coefficients of the upper-left 4×4 region of an 8×8 region of the target block, and the modified transform coefficients of the upper-left 4×4 region, upper-right 4×4 region, and lower-left 4×4 region of the 8×8 region are derived through the inverse RST.SELECTED DRAWING: Figure 10

Description

[Technical Field]

[0001] This document relates to video coding technology, and more specifically, to a video coding method and apparatus based on transformation in a video coding system. [Background technology]

[0002] Recently, demand for high-resolution, high-quality video / video, such as 4K or 8K or higher UHD (Ultra High Definition) video / video, has been increasing in various fields. As video / video data becomes higher resolution and higher quality, the amount of information or bits transmitted relative to existing video / video data increases. Therefore, when transmitting video / video data using existing wired or wireless broadband lines, or storing video / video data using existing storage media, transmission and storage costs increase.

[0003] Furthermore, interest in and demand for immersive media such as VR (Virtual Reality), AR (Artificial Reality), and holograms have recently increased, and there has been a rise in broadcasting of video content with different visual characteristics from real-world footage, such as game footage.

[0004] Therefore, highly efficient video compression technology is required to effectively compress, transmit, store, and play back high-resolution, high-quality video information with the diverse characteristics described above. [Overview of the project] [Problems that the invention aims to solve]

[0005] The technical objective of this document is to provide a method and apparatus for improving the efficiency of video coding.

[0006] Another technical objective of this paper is to provide methods and apparatus for improving conversion efficiency.

[0007] Another technical objective of this paper is to provide a video coding method and apparatus based on RST (reduced secondary transform).

[0008] Another technical objective of this paper is to provide a method and apparatus for improving the efficiency of quadratic transformation by differentiating the array of transformation coefficients using an intra-predictive mode.

[0009] Another technical objective of this paper is to provide a method and apparatus for optimizing the transformation kernel matrix applied to quadratic transformations to improve the efficiency of the quadratic transformation.

[0010] Another technical objective of this paper is to provide a video coding method and apparatus based on a set of transformations that can increase coding efficiency. [Means for solving the problem]

[0011] According to one embodiment of this document, a video decoding method is provided that is performed by a decoding device. The method includes the steps of: deriving a transformation coefficient via inverse quantization based on a quantized transformation coefficient for a target block; deriving a modified transformation coefficient based on an inverse RST (reduced secondary transform) using a pre-set transformation kernel matrix for the transformation coefficient; deriving a residual sample for the target block based on an inverse linear transformation of the modified transformation coefficient; and generating a restored picture based on the residual sample for the target block, wherein the step of deriving the modified transformation coefficient is performed by applying the transformation kernel matrix to the transformation coefficient of the upper left 4x4 region of the 8x8 region of the target block, thereby deriving the modified transformation coefficients for the upper left 4x4 region, the upper right 4x4 region, and the lower left 4x4 region of the 8x8 region.

[0012] When performing matrix operations between the transformation coefficients of the upper left 4x4 region of the 8x8 region and the transformation kernel matrix, the transformation coefficients of the upper left 4x4 region of the 8x8 region are arranged in a one-dimensional array according to the forward diagonal scanning order.

[0013] The transformation coefficients of the one-dimensional array are arranged in two dimensions within the 8x8 region, specifically the upper left 4x4 region, the upper right 4x4 region, and the lower left 4x4 region, in either a row-first or column-first order, by an intra-prediction mode applied to the target block after matrix operations with the transformation kernel matrix.

[0014] Another embodiment of this document provides a decoding device for performing video decoding. The decoding device includes an entropy decoding unit that derives information for quantized transformation coefficients and predictions for a target block from a bitstream; a prediction unit that generates prediction samples for the target block based on the prediction information; an inverse transformation unit that includes an inverse RST (reduced secondary transform) unit that derives modified transformation coefficients based on an inverse RST (reduced secondary transform) of the transformation coefficients and an inverse linear transformation unit that derives residual samples for the target block based on an inverse linear transformation of the modified transformation coefficients; and an additive unit that generates restored samples based on the residual samples and the prediction samples, wherein the inverse RST unit derives modified transformation coefficients for the upper left 4x4 region, the upper right 4x4 region, and the lower left 4x4 region of the 8x8 region by applying the transformation kernel matrix to the transformation coefficients of the upper left 4x4 region of the 8x8 region of the target block.

[0015] According to one embodiment of this document, a video encoding method executed by an encoding device is provided. The method includes steps of deriving a prediction sample based on an intra prediction mode applied to a target block, deriving a residual sample for the target block based on the prediction sample, deriving a transform coefficient for the target block based on a first-order transform of the residual sample, deriving a modified transform coefficient based on an RST (reduced secondary transform) for the transform coefficient, and performing quantization based on the modified transform coefficient to derive a quantized transform coefficient. The step of deriving the modified transform coefficient is characterized by applying the transform kernel matrix to the transform coefficients of the upper left 4×4 region, the upper right 4×4 region, and the lower left 4×4 region of the 8×8 region of the target block to derive a modified transform coefficient corresponding to the upper left 4×4 region of the 8×8 region.

[0016] According to another embodiment of this document, a digital storage medium storing video data including encoded video information and a bitstream generated by a video encoding method executed by an encoding device is provided.

[0017] According to another embodiment of this document, a digital storage medium storing video data including encoded video information and a bitstream for causing a decoding device to execute the video decoding method is provided.

Advantages of the Invention

[0018] According to this document, the overall video / video compression efficiency can be improved.

[0019] According to this document, the efficiency of the secondary transform can be improved by making the array of transform coefficients different according to the intra prediction mode.

[0020] According to this document, video coding can be performed based on a conversion set to improve video coding efficiency.

[0021] According to this paper, the efficiency of quadratic transformations can be improved by optimizing the transformation kernel matrix applied to them. [Brief explanation of the drawing]

[0022] [Figure 1] This document outlines some examples of video / image coding systems to which it can be applied. [Figure 2] This diagram schematically illustrates the configuration of a video / image encoding device to which this document can be applied. [Figure 3] This diagram schematically illustrates the configuration of a video / image decoding device to which this document can be applied. [Figure 4] A schematic representation of a multiplexing technique according to one embodiment of this document is provided below. [Figure 5] Sixty-five intra-directional modes for prediction directions are shown as examples. [Figure 6] This is a diagram illustrating the RST according to one embodiment of this document. [Figure 7] The scanning order of the conversion coefficients according to one embodiment of this document is shown below. [Figure 8] This flowchart shows the reverse RST process according to one embodiment of this document. [Figure 9] This is a flowchart illustrating the operation of a video decoding device according to one embodiment of this document. [Figure 10] This is a control flowchart illustrating the inverse RST according to one embodiment of this document. [Figure 11] This is a flowchart illustrating the operation of a video encoding device according to one embodiment described herein. [Figure 12] This is a control flowchart illustrating the RST according to one embodiment of this document. [Figure 13] An illustrative diagram of a content streaming system structure to which this document applies is shown below. [Modes for carrying out the invention]

[0023] This document is subject to various modifications and may have multiple embodiments, with specific embodiments illustrated and described in detail in the drawings. However, this does not limit this document to specific embodiments. In this specification, commonly used terms are used solely to describe specific embodiments and are not intended to limit the technical ideas of this document. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as “includes” or “has” should be understood to indicate the existence of features, figures, steps, actions, components, parts, or combinations thereof described in the specification, without prejudice to the existence or possibility of adding one or more other features, figures, steps, actions, components, parts, or combinations thereof.

[0024] On the other hand, each configuration shown in the diagrams described in this document is shown independently for the convenience of explaining its distinct characteristic functions, and does not mean that each configuration is embodied in separate hardware or separate software. For example, two or more of the configurations may be combined to form a single configuration, and a single configuration may be divided into multiple configurations. Embodiments in which each configuration is integrated and / or separated are also included within the scope of the rights of this document, as long as they do not deviate from the essence of this document.

[0025] The preferred embodiments of this document will be described in more detail below with reference to the attached drawings. The same reference numerals will be used for the same components in the drawings, and redundant descriptions of the same components will be omitted.

[0026] This document relates to video / image coding. For example, the methods / examples disclosed in this document may be related to the VVC (Versatile Video Coding) standard (ITU-T Rec.H.266), next-generation video / image coding standards after VVC, or other video coding-related standards (e.g., HEVC (High Efficiency Video Coding) standard (ITU-T Rec.H.265), EVC (essential video coding) standard, AVS2 standard, etc.).

[0027] This document presents various implementations of video / image coding, and unless otherwise noted, these implementations can be performed in combination with each other.

[0028] In this document, "video" can mean a collection of images over time. "Picture" generally refers to a single image representing a specific time period, while "slice" or "tile" is a unit that constitutes part of a picture in coding. A slice or tile can contain one or more CTUs (coding tree units). A single picture can consist of one or more slices or tiles. A single picture can consist of one or more tile groups. A tile group can contain one or more tiles.

[0029] A pixel or pel can refer to the smallest unit that makes up a picture (or image). Alternatively, the term 'sample' can be used as a counterpart to pixel. A sample can generally represent a pixel or a pixel value, or it can represent only the luma component pixel / pixel value, or only the chroma component pixel / pixel value. Alternatively, a sample can refer to a pixel value in the spatial domain, and if such a pixel value is converted to the frequency domain, it can also refer to the conversion coefficient in the frequency domain.

[0030] A unit can represent a basic unit of image processing. A unit can contain at least one of the following: a specific region of a picture and information associated with that region. A unit can contain one luma block and two chroma (e.g., cb, cr) blocks. The term unit may sometimes be used interchangeably with terms such as block or area. In general, an M×N block can contain a set (or array) of samples (or sample arrays) or transform coefficients consisting of M columns and N rows.

[0031] In this document, the terms " / " and "," are interpreted as "and / or." For example, "A / B" is interpreted as "A and / or B," and "A, B" is interpreted as "A and / or B." Additionally, "A / B / C" means "at least one of A, B, and / or C." Similarly, "A, B, C" also means "at least one of A, B, and / or C."

[0032] In addition, in this document, "or" is interpreted as "and / or." For example, "A or B" can mean 1) only "A," 2) only "B," or 3) both "A and B." Alternatively, "or" in this document can mean "additionally or alternatively."

[0033] Figure 1 schematically shows an example of a video / image coding system to which this document can be applied.

[0034] Referring to Figure 1, a video / image coding system can include a source device and a receiving device. The source device can transmit encoded video / image information or data to the receiving device in file or streaming form via a digital storage medium or network.

[0035] The source device may include a video source, an encoding device, and a transmitter. The receiving device may include a receiver, a decoding device, and a renderer. The encoding device may be called a video / image encoding device, and the decoding device may be called a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display unit, which may consist of a separate device or external component.

[0036] A video source can acquire video / images through processes such as video / image capture, synthesis, or generation. A video source may include video / image capture devices and / or video / image generation devices. Video / image capture devices may include, for example, one or more cameras, or video / image archives containing previously captured video / images. Video / image generation devices may include, for example, computers, tablets, and smartphones, and can generate video / images (electronically). For example, virtual video / images can be generated via a computer, in which case the video / image capture process can be replaced as the process by which the associated data is generated.

[0037] An encoding device can encode input video / image data. For compression and coding efficiency, the encoding device can perform a series of steps, including prediction, transformation, and quantization. The encoded data (encoded video / image information) can be output in bitstream format.

[0038] The transmitting unit can transmit encoded video / image information or data output in bitstream format to the receiving unit of a receiving device via a digital storage medium or network in file or streaming format. The digital storage medium can include a variety of storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. The transmitting unit may include elements for generating media files via a predetermined file format and may include elements for transmission via a broadcast / communication network. The receiving unit can receive / extract the bitstream and transmit it to a decoding device.

[0039] A decoding device can decode video / images by performing a series of steps, such as inverse quantization, inverse transformation, and prediction, corresponding to the operation of an encoding device.

[0040] The renderer can render the decoded video / image. The rendered video / image can be displayed via the display unit.

[0041] Figure 2 is a schematic diagram illustrating the configuration of a video / image encoding device to which this document can be applied. Hereinafter, the term "video encoding device" may include image encoding devices.

[0042] Referring to Figure 2, the encoding device 200 may be configured to include an image partitioner 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 may include an inter-predictor 221 and an intra-predictor 222. The residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234, and an inverse transformer 235. The residual processor 230 may further include a subtractor 231. The adder 250 may be called a reconstructor or a reconstructed block generator. The aforementioned video splitting unit 210, prediction unit 220, residual processing unit 230, entropy encoding unit 240, addition unit 250, and filtering unit 260 can be configured by one or more hardware components (e.g., an encoder chipset or processor) depending on the embodiment. The memory 270 may also include a DPB (decoded picture buffer) and may be configured by a digital storage medium. The hardware components may further include the memory 270 as an internal / external component.

[0043] The video splitting unit 210 can split the input video (or picture, frame) input to the encoding device 200 into one or more processing units. For example, the processing units may be called coding units (CUs). In this case, coding units can be recursively split from a coding tree unit (CTU) or the largest coding unit (LCU) using a QTBTTT (Quad-tree binary-tree ternary-tree) structure. For example, a single coding unit can be split into multiple coding units of deeper depth based on a quad-tree structure, a binary-tree structure, and / or a ternary structure. In this case, for example, the quad-tree structure may be applied first, followed by the binary-tree structure and / or the ternary structure. Alternatively, the binary-tree structure may be applied first. The coding procedure according to this document can be executed based on the final coding unit that cannot be further split. In this case, based on coding efficiency due to video characteristics, the largest coding unit can be used as the final coding unit, or, if necessary, the coding unit can be recursively divided into lower-depth coding units so that the optimally sized coding unit is used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and restoration, which will be described later. As another example, the processing unit may further include a prediction unit (PU) or a transformation unit (TU). In this case, the prediction unit and the transformation unit can each be separated or partitioned from the final coding unit described above.The prediction unit is a unit of sample prediction, and the conversion unit is a unit that derives a conversion coefficient and / or a unit that derives a residual signal from the conversion coefficient.

[0044] The term "unit" can sometimes be used interchangeably with terms such as "block" or "area." Generally, an M×N block can represent a set of samples or transform coefficients consisting of M columns and N rows. A sample can generally represent a pixel or a pixel value, and may represent only the luminance (luma) component pixel / pixel value, or only the chroma component pixel / pixel value. A sample can be used as the term corresponding to a single picture (or image) pixel or pel.

[0045] The subtraction unit 231 can generate a residual signal (residual block, residual sample, or residual sample array) by subtracting the predicted signal (predicted block, predicted sample, or predicted sample array) output from the prediction unit 220 from the input video signal (original block, original sample, or original sample array), and the generated residual signal is transmitted to the conversion unit 232. The prediction unit 220 can perform a prediction on the block to be processed (hereinafter referred to as the current block) and generate a predicted block that includes a predicted sample for the current block. The prediction unit 220 can determine whether intra-prediction or inter-prediction is applied on a current block or CU basis. As will be described later in the explanation of each prediction mode, the prediction unit can generate various information related to prediction, such as prediction mode information, and transmit it to the entropy encoding unit 240. The information related to prediction can be encoded by the entropy encoding unit 240 and output in bitstream form.

[0046] The intra-prediction unit 222 can predict the current block by referring to a sample in the current picture. The referenced sample may be located adjacent to the current block or at a distance, depending on the prediction mode. In intra-prediction, the prediction mode may include multiple non-directional modes and multiple directional modes. Non-directional modes may include, for example, DC mode and planar mode. Directional modes may include, for example, 33 directional prediction modes or 65 directional prediction modes, depending on the degree of fineness of the prediction direction. However, this is merely an example, and more or fewer directional prediction modes may be used depending on the settings. The intra-prediction unit 222 can also use the prediction mode applied to the adjacent block to determine the prediction mode to be applied to the current block.

[0047] The interprediction unit 221 can derive a predicted block relative to the current block based on a reference block (reference sample array) identified by motion vectors on the reference picture. In this case, in order to reduce the amount of motion information transmitted in interprediction mode, motion information can be predicted in units of blocks, subblocks, or samples based on the correlation of motion information between adjacent blocks and the current block. The motion information may include motion vectors and reference picture indices. The motion information may further include interprediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of interprediction, adjacent blocks may include spatially adjacent blocks existing in the current picture and temporally adjacent blocks existing in the reference picture. The reference picture containing the reference block and the reference picture containing the temporally adjacent block may be the same or different. The temporally adjacent block may also be called a collocated reference block or colCU, and the reference picture containing the temporally adjacent block is also called a collocated picture (colPic). For example, the interpretation unit 221 can construct a motion information candidate list based on adjacent blocks and generate information indicating which candidates are used to derive the motion vector and / or reference picture index of the current block. Interpretation can be performed based on various prediction modes; for example, in skip mode and merge mode, the interpretation unit 221 can use the motion information of adjacent blocks as the motion information of the current block. In skip mode, unlike merge mode, a residual signal may not be transmitted.In motion vector prediction (MVP) mode, the motion vector of an adjacent block is used as a motion vector predictor, and the motion vector difference is signaled to indicate the motion vector of the current block.

[0048] The prediction unit 220 can generate prediction signals based on various prediction methods described later. For example, the prediction unit can apply intra-prediction or inter-prediction for a single block, and can also apply intra-prediction and inter-prediction simultaneously. This can be called combined inter and intra-prediction (CIIP). The prediction unit can also perform intra-block copy (IBC) for predictions on blocks. The intra-block copy can be used for content video / moving video coding such as in games, for example, as in SCC (screen content coding). IBC basically performs prediction within the current picture, but can be performed in a manner similar to inter-prediction in that it derives a reference block within the current picture. That is, IBC can utilize at least one of the inter-prediction techniques described in this document.

[0049] The predicted signals generated via the interpretation unit 221 and / or the intrapretation unit 222 can be used to generate a reconstructed signal or a residual signal. The transformation unit 232 can generate transform coefficients by applying a transformation technique to the residual signal. For example, the transformation technique may include DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), GBT (Graph-Based Transform), or CNT (Conditionally Non-linear Transform). Here, GBT refers to a transformation obtained from a graph, assuming that the relationship information between pixels is represented by this graph. CNT refers to a transformation obtained by generating a predicted signal using all previously reconstructed pixels and based on that. The transformation process can also be applied to pixel blocks of the same size that are square, or to non-square blocks of variable size.

[0050] The quantization unit 233 quantizes the conversion coefficients and transmits them to the entropy encoding unit 240, which can encode the quantized signal (information about the quantized conversion coefficients) and output it as a bitstream. The information about the quantized conversion coefficients can be called residual information. The quantization unit 233 can rearrange the block-form quantized conversion coefficients into a one-dimensional vector form based on the coefficient scan order, and can also generate information about the quantized conversion coefficients based on the one-dimensional vector form of the quantized conversion coefficients. The entropy encoding unit 240 can perform various encoding methods, such as exponential Golomb, CAVLC (context-adaptive variable length coding), and CABAC (context-adaptive binary arithmetic coding). In addition to the quantized conversion coefficients, the entropy encoding unit 240 can also encode information necessary for video / image restoration (e.g., the values ​​of syntax elements) together with or separately from the quantized conversion coefficients. Encoded information (e.g., encoded video / image information) can be transmitted or stored in bitstream form in units of network abstraction layer (NAL) units. The video / image information may further include information about various parameter sets, such as adaptation parameter sets (APS), picture parameter sets (PPS), sequence parameter sets (SPS), or video parameter sets (VPS). The video / image information may also further include general constraint information. Signaling / transmitted information and / or syntax elements described later in this document can be encoded via the encoding procedure described above and included in the bitstream.The bitstream can be transmitted over a network or stored in a digital storage medium. Here, the network may include broadcast networks and / or communication networks, and the digital storage medium may include a variety of storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The signal output from the entropy encoding unit 240 can be transmitted by a transmitting unit (not shown) and / or stored by a storage unit (not shown) which are configured as internal / external elements of the encoding device 200, or the transmitting unit may be included in the entropy encoding unit 240.

[0051] The quantized conversion coefficients output from the quantization unit 233 can be used to generate a prediction signal. For example, a residual signal (residual block or residual sample) can be reconstructed by applying inverse quantization and inverse transformation to the quantized conversion coefficients via the inverse quantization unit 234 and the inverse transformation unit 235. The adder unit 250 can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample, or reconstructed sample array) by adding the reconstructed residual signal to the prediction signal output from the prediction unit 220. If there is no residual for the block to be processed, such as when skip mode is applied, the predicted block can be used as the reconstructed block. The generated reconstructed signal can be used for intra-prediction of the next block to be processed in the current picture, and can also be used for inter-prediction of the next picture after filtering, as described later.

[0052] On the other hand, LMCS (luma mapping with chroma scaling) can also be applied during the picture encoding and / or restoration process.

[0053] The filtering unit 260 can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit 260 can apply various filtering methods to the restored picture to generate a modified restored picture, and the modified restored picture can be stored in the memory 270, specifically in the DPB of the memory 270. The various filtering methods can include, for example, deblocking filtering, sample adaptive offset (SAO), adaptive loop filter, and bilateral filter. The filtering unit 260 can generate various filtering information and transmit it to the entropy encoding unit 240, as will be described later in the explanation of each filtering method. The filtering information can be encoded by the entropy encoding unit 240 and output in bitstream format.

[0054] The corrected restored picture sent to memory 270 can be used as a reference picture in the interpretation unit 221. When interpretation is applied via this, the encoding device can avoid prediction mismatches between the encoding device 200 and the decoding device, and can also improve encoding efficiency.

[0055] The DPB in memory 270 can store the corrected restored picture for use as a reference picture in the inter-prediction unit 221. Memory 270 can store motion information of blocks from which motion information has been derived (or encoded) in the current picture and / or motion information of blocks in the picture that have already been restored. The stored motion information can be transmitted to the inter-prediction unit 221 for use as motion information of spatially adjacent blocks or motion information of temporally adjacent blocks. Memory 270 can store restored samples of restored blocks in the current picture and transmit them to the intra-prediction unit 222.

[0056] Figure 3 is a schematic diagram illustrating the configuration of a video / image decoding device to which this document can be applied.

[0057] Referring to Figure 3, the decoding device 300 can be configured to include an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 may include an inter-predictor 332 and an intra-predictor 331. The residual processor 320 may include a dequantizer 321 and an inverse transformer 322. The aforementioned entropy decoder 310, residual processor 320, predictor 330, adder 340, and filtering device 350 can be configured by a single hardware component (e.g., a decoder chipset or processor) depending on the embodiment. The memory 360 may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The aforementioned hardware component may further include memory 360 as an internal / external component.

[0058] When a bitstream containing video / image information is input, the decoding device 300 can reconstruct the image in accordance with the process by which the video / image information was processed in the encoding device shown in Figure 2. For example, the decoding device 300 can derive units / blocks based on block division-related information obtained from the bitstream. The decoding device 300 can perform decoding using the processing units applied in the encoding device. Thus, the decoding processing units are, for example, coding units, which can be divided from a coding tree unit or a maximum coding unit into a quad-tree structure, a binary tree structure, and / or a terminally tree structure. One or more conversion units can be derived from the coding unit. The reconstructed video signal decoded and output via the decoding device 300 can then be played back via a playback device.

[0059] The decoding device 300 can receive the signal output from the encoding device shown in Figure 2 in bitstream form, and the received signal can be decoded via the entropy decoding unit 310. For example, the entropy decoding unit 310 can parse the bitstream to derive information necessary for video restoration (or picture restoration) (e.g., video / image information). The video / image information may further include information about various parameter sets, such as the adaptation parameter set (APS), picture parameter set (PPS), sequence parameter set (SPS), or video parameter set (VPS). The video / image information may also further include general constraint information. The decoding device can decode the picture based on the parameter set information and / or the general constraint information. The signaling / received information and / or syntax elements described later in this document can be decoded via the decoding procedure and obtained from the bitstream. For example, the entropy decoding unit 310 can decode information in the bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and output the values ​​of syntax elements necessary for image restoration and the quantized values ​​of conversion coefficients related to the residual. More specifically, the CABAC entropy decoding method receives bins corresponding to each syntax element in the bitstream, determines a context model using the syntax element information to be decoded and the decoding information of adjacent and decoded blocks or the symbol / bin information decoded in a previous step, predicts the probability of bin occurrence based on the determined context model, and performs arithmetic decoding of the bins to generate symbols corresponding to the values ​​of each syntax element.In this case, the CABAC entropy decoding method can update the context model after determining the context model by utilizing the information of the decoded symbol / bin for the context model of the next symbol / bin. Of the information decoded by the entropy decoding unit 310, information related to prediction is provided to the prediction unit 330, and information for the residual on which entropy decoding was performed by the entropy decoding unit 310, i.e., quantized conversion coefficients and related parameter information, can be input to the inverse quantization unit 321. In addition, of the information decoded by the entropy decoding unit 310, information related to filtering can be provided to the filtering unit 350. On the other hand, a receiving unit (not shown) that receives the signal output from the encoding device can be further configured as an internal / external element of the decoding device 300, or the receiving unit may be a component of the entropy decoding unit 310. On the other hand, the decoding device described in this document may be called a video / image / picture decoding device, and the decoding device may also be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include the entropy decoding unit 310, and the sample decoder may include at least one of the inverse quantization unit 321, inverse transformation unit 322, prediction unit 330, addition unit 340, filtering unit 350, and memory 360.

[0060] The inverse quantization unit 321 can inverse quantize the quantized transformation coefficients and output the transformation coefficients. The inverse quantization unit 321 can rearrange the quantized transformation coefficients in a two-dimensional block form. In this case, the rearrangement can be performed based on the coefficient scan order performed by the encoding device. The inverse quantization unit 321 can perform inverse quantization on the quantized transformation coefficients using quantization parameters (e.g., quantization step size information) and obtain the transformation coefficients.

[0061] The inverse conversion unit 322 performs an inverse conversion on the conversion coefficients to obtain a residual signal (residual block, residual sample array).

[0062] The prediction unit can perform a prediction on the current block and generate a predicted block containing prediction samples for the current block. Based on the prediction information output from the entropy decoding unit 310, the prediction unit can determine whether intra-prediction or inter-prediction is applied to the current block, and can determine a specific intra / inter-prediction mode.

[0063] The prediction unit can generate prediction signals based on various prediction methods described later. For example, the prediction unit can apply intra-prediction or inter-prediction for a single block, and can also apply intra-prediction and inter-prediction simultaneously. This can be called combined inter and intra-prediction (CIIP). The prediction unit can also perform intra-block copying (IBC) for predictions on blocks. This intra-block copying can be used for content video / moving video coding, such as in SCC (screen content coding), for games. IBC basically performs prediction within the current picture, but can be performed in a manner similar to inter-prediction in that it derives a reference block within the current picture. That is, IBC can utilize at least one of the inter-prediction techniques described in this document.

[0064] The intra-prediction unit 331 can predict the current block by referring to a sample in the current picture. The referenced sample may be located adjacent to the current block or at a distance, depending on the prediction mode. In intra-prediction, the prediction mode may include multiple non-directional modes and multiple directional modes. The intra-prediction unit 331 can also use the prediction mode applied to the adjacent block to determine the prediction mode to be applied to the current block.

[0065] The interprediction unit 332 can derive a predicted block for the current block based on a reference block (reference sample array) identified by motion vectors on the reference picture. In this case, to reduce the amount of motion information transmitted in interprediction mode, motion information can be predicted on a block, subblock, or sample basis based on the correlation of motion information between adjacent blocks and the current block. The motion information may include motion vectors and reference picture indices. The motion information may further include interprediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of interprediction, adjacent blocks may include spatially adjacent blocks present in the current picture and temporally adjacent blocks present in the reference picture. For example, the interprediction unit 332 can construct a motion information candidate list based on adjacent blocks and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. Interprediction can be performed based on various prediction modes, and the prediction information may include information indicating the mode of interprediction for the current block.

[0066] The summing unit 340 can generate a restored signal (restored picture, restored block, restored sample array) by adding the acquired residual signal to the predicted signal (predicted block, predicted sample array) output from the prediction unit 330. If there is no residual for the block to be processed, such as when skip mode is applied, the predicted block can be used as the restored block.

[0067] The addition unit 340 may be called the restoration unit or restoration block generation unit. The generated restoration signal can be used for intra-prediction of the next block to be processed in the current picture, and can be output after filtering as described later, or it can be used for intra-prediction of the next picture.

[0068] On the other hand, LMCS (luma mapping with chroma scaling) can also be applied during the picture decoding process.

[0069] The filtering unit 350 can apply filtering to the restored signal to improve subjective / objective image quality. For example, the filtering unit 350 can apply various filtering methods to the restored picture to generate a modified restored picture, and can transmit the modified restored picture to the memory 360, specifically to the DPB of the memory 360. The various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, and bilateral filter.

[0070] The (modified) restored picture stored in the DPB of memory 360 can be used as a reference picture by the inter-prediction unit 332. Memory 360 can store motion information of blocks from which motion information in the current picture has been derived (or decoded) and / or motion information of blocks in the picture that have already been restored. The stored motion information can be transmitted to the inter-prediction unit 332 for use as motion information of spatially adjacent blocks or motion information of temporally adjacent blocks. Memory 360 can store restored samples of restored blocks in the current picture and transmit them to the intra-prediction unit 331.

[0071] In this specification, the embodiments described for the prediction unit 330, inverse quantization unit 321, inverse conversion unit 322, and filtering unit 350 of the decoding device 300 can also be applied to the prediction unit 220, inverse quantization unit 234, inverse conversion unit 235, and filtering unit 260 of the encoding device 200, respectively, in the same manner or in a corresponding manner.

[0072] As mentioned above, prediction is performed to improve compression efficiency when performing video coding. This allows for the generation of a predicted block containing predicted samples for the current block, which is the block to be coded. Here, the predicted block contains predicted samples in the spatial domain (or pixel domain). The predicted block is derived in both the encoding and decoding devices, and the encoding device can improve video coding efficiency by signaling the decoding device with information (residual information) about the residual between the original block and the predicted block, which is not the original sample value of the original block itself. The decoding device can derive a residual block containing residual samples based on the residual information, and can combine the residual block and the predicted block to generate a restored block containing restored samples, thereby generating a restored picture containing the restored block.

[0073] The residual information can be generated through transformation and quantization procedures. For example, an encoding device can signal the relevant residual information (via a bitstream) to a decoding device by deriving a residual block between the original block and the predicted block, performing a transformation procedure on the residual samples (residual sample array) contained in the residual block to derive transformation coefficients, and performing a quantization procedure on the transformation coefficients to derive quantized transformation coefficients. Here, the residual information may include information such as the value information, position information, transformation technique, transformation kernel, and quantization parameters of the quantized transformation coefficients. The decoding device can perform an inverse quantization / inverse transformation procedure based on the residual information to derive a residual sample (or residual block). The decoding device can generate a reconstructed picture based on the predicted block and the residual block. The encoding device can also derive a residual block by inverse quantization / inverse transformation of the quantized transformation coefficients for reference for subsequent interpretation of the picture, and generate a reconstructed picture based on this.

[0074] Figure 4 schematically illustrates the multiplexing technique described in this document.

[0075] Referring to Figure 4, the conversion unit can correspond to the conversion unit in the encoding device shown in Figure 2, and the inverse conversion unit can correspond to the inverse conversion unit in the encoding device shown in Figure 2 or the inverse conversion unit in the decoding device shown in Figure 3.

[0076] The transformation unit can perform a linear transformation based on the residual samples (residual sample array) in the residual block to derive (linear) transformation coefficients (S410). Such a linear transformation can be called a core transformation. Here, the linear transformation can be based on Multiple Transform Selection (MTS), and when multiple transformations are applied to the linear transformation, it can be called a multiple core transformation.

[0077] Multiple core transforms can represent a transformation method that additionally uses a Discrete Cosine Transform (DCT) type 2 and a Discrete Sine Transform (DST) type 7, a Discrete Sine Transform (DST) type 8, and / or a Discrete Sine Transform (DST) type 1. That is, the multiple core transforms can represent a transformation method that transforms a spatial domain residual signal (or residual block) into frequency domain transformation coefficients (or first-order transformation coefficients) based on a plurality of transformation kernels selected from the DCT type 2, the DST type 7, the DCT type 8, and the DST type 1. Here, the first-order transformation coefficients can be called temporary transformation coefficients from the perspective of the transformer.

[0078] That is, when an existing conversion method is applied, a spatial-domain to frequency-domain conversion of a residual signal (or residual block) can be applied based on DCT type 2 to generate conversion coefficients. In contrast, when the multiple core conversion is applied, a spatial-domain to frequency-domain conversion of a residual signal (or residual block) can be applied based on DCT type 2, DST type 7, DCT type 8, and / or DST type 1, etc., to generate conversion coefficients (or first-order conversion coefficients). Here, DCT type 2, DST type 7, DCT type 8, and DST type 1, etc., can be called conversion types, conversion kernels, or conversion cores.

[0079] For reference, the DCT / DST conversion type can be defined based on a basis set, which is shown in the table below.

[0080] [Table 1]

[0081] When the multi-core transformation is performed, a vertical transformation kernel and a horizontal transformation kernel can be selected from the transformation kernels for the target block, and a vertical transformation can be performed on the target block based on the vertical transformation kernel, and a horizontal transformation can be performed on the target block based on the horizontal transformation kernel. Here, the horizontal transformation may represent a transformation to the horizontal component of the target block, and the vertical transformation may represent a transformation to the vertical component of the target block. The vertical transformation kernel / horizontal transformation kernel can be adaptively determined based on the prediction mode and / or transformation index of the target block (CU or subblock) including the residual block.

[0082] Furthermore, for example, when applying MTS to perform a linear transformation, a specific basis function can be set to a predetermined value, and the mapping relationship to the transformation kernel can be set by combining which basis function is applied when it is a vertical or horizontal transformation. For example, if the horizontal transformation kernel is represented by trTypeHor and the vertical transformation kernel is represented by trTypeVer, then a trTypeHor or trTypeVer value of 0 can be set to DCT2, a trTypeHor or trTypeVer value of 1 can be set to DST7, and a trTypeHor or trTypeVer value of 2 can be set to DCT8.

[0083] In this case, the MTS index information can be encoded and signaled to the decoding device to indicate one of several sets of conversion kernels. For example, if the MTS index is 0, it can indicate that both trTypeHor and trTypeVer values ​​are 0; if the MTS index is 1, it can indicate that both trTypeHor and trTypeVer values ​​are 1; if the MTS index is 2, it can indicate that the trTypeHor value is 2 and the trTypeVer value is 1; if the MTS index is 3, it can indicate that the trTypeHor value is 1 and the trTypeVer value is 2; and if the MTS index is 4, it can indicate that both trTypeHor and trTypeVer values ​​are 2.

[0084] The conversion unit can perform a quadratic transformation based on the (primary) transformation coefficients to derive modified (secondary) transformation coefficients (S420). The primary transformation is a transformation from the spatial domain to the frequency domain, and the secondary transformation means a transformation to a more compressed representation by utilizing the correlations that exist between the (primary) transformation coefficients. The secondary transformation may include a non-separable transform. In this case, the secondary transformation may be called a non-separable secondary transform (NSST) or MDNSST (mode-dependent non-separable secondary transform). The non-separable secondary transform may represent a transformation that generates modified transformation coefficients (or secondary transformation coefficients) for the residual signal by performing a quadratic transformation on the (primary) transformation coefficients derived via the primary transformation based on a non-separable transform matrix. Here, the transformation can be applied at once to the (primary) transformation coefficients without separately applying the vertical and horizontal transformations (or applying the horizontal and vertical transformations independently) based on the non-separable transform matrix. That is, the non-separable quadratic transform can represent a transform method that, without separating the vertical and horizontal components of the (primary) transform coefficients, rearranges a two-dimensional signal (transformation coefficient) into a one-dimensional signal via a specifically determined direction (e.g., row-first or column-first direction), and then generates a transformed coefficient (or quadratic transform coefficient) modified based on the non-separable transform matrix. For example, row-first ordering arranges the first row, second row, ..., Nth row in a single column for an M×N block, and column-first ordering arranges the first column, second column, ..., Mth column in a single column for an M×N block. The non-separable quadratic transform can be applied to the top-left region of a block composed of (primary) transform coefficients (hereinafter referred to as a transform coefficient block).For example, if both the width (W) and height (H) of the conversion coefficient block are 8 or greater, an 8x8 inseparable quadratic transformation can be applied to the upper left 8x8 region of the conversion coefficient block. Also, if both the width (W) and height (H) of the conversion coefficient block are 4 or greater, and either the width (W) or height (H) of the conversion coefficient block is less than 8, a 4x4 inseparable quadratic transformation can be applied to the upper left min(8,W) x min(8,H) region of the conversion coefficient block. However, the embodiments are not limited to these, and for example, even if only the condition that both the width (W) or height (H) of the conversion coefficient block are 4 or greater is met, a 4x4 inseparable quadratic transformation can also be applied to the upper left min(8,W) x min(8,H) region of the conversion coefficient block.

[0085] Specifically, for example, when a 4x4 input block is used, the unseparated quadratic transform can be performed as follows:

[0086] The aforementioned 4x4 input block X is shown as follows:

[0087]

number

[0088] When X is expressed in vector form, JPEG0007893952000003.jpg84 is shown as follows:

[0089]

number

[0090] As shown in equation 2, vector JPEG0007893952000005.jpg84 rearranges the 2D block of X in equation 1 into a 1D vector using row-first order.

[0091] In this case, the quadratic inseparable transform can be calculated as follows.

[0092]

number

[0093] Here, JPEG0007893952000007.jpg75 shows the transformation coefficient vector, and T shows the 16×16 (non-separable) transformation matrix.

[0094] 16 × 1 transformation coefficient vector via the above formula 3 JPEG0007893952000008.jpg75 can be derived, and the above JPEG0007893952000009.jpg75 can be reorganized into 4x4 blocks via a scan order (horizontal, vertical, diagonal, etc.). However, the calculations described above are merely illustrative, and methods such as HyGT (Hypercube-Givens Transform) can also be used to calculate inseparable quadratic transforms in order to reduce the computational complexity of such transformations.

[0095] On the other hand, the unseparated quadratic transform can have a mode-dependent transform kernel (or transform core, transform type) selected. Here, the mode may include intra-predictive mode and / or inter-predictive mode.

[0096] As described above, the non-separable quadratic transformation can be performed based on an 8x8 transformation or a 4x4 transformation determined based on the width (W) and height (H) of the transformation coefficient block. An 8x8 transformation refers to a transformation that can be applied to an 8x8 region contained within the transformation coefficient block when both W and H are equal to or greater than 8, and the 8x8 region is the upper left 8x8 region within the transformation coefficient block. Similarly, a 4x4 transformation refers to a transformation that can be applied to a 4x4 region contained within the transformation coefficient block when both W and H are equal to or greater than 4, and the 4x4 region is the upper left 4x4 region within the transformation coefficient block. For example, an 8x8 transformation kernel matrix can be a 64x64 / 16x64 matrix, and a 4x4 transformation kernel matrix can be a 16x16 / 8x16 matrix.

[0097] In this case, for mode-based conversion kernel selection, three non-separable quadratic conversion kernels can be configured for each conversion set for both 8x8 and 4x4 conversions, resulting in 35 conversion sets. That is, 35 conversion sets can be configured for 8x8 conversions, and 35 conversion sets can be configured for 4x4 conversions. In this case, each of the 35 conversion sets for 8x8 conversions can contain three 8x8 conversion kernels, and each of the 35 conversion sets for 4x4 conversions can contain three 4x4 conversion kernels. However, the size of the conversions, the number of sets, and the number of conversion kernels in each set are merely examples, and sizes other than 8x8 or 4x4 can be used, or n sets can be configured, with k conversion kernels in each set.

[0098] The aforementioned set of transformations may be called an NSST set, and the transformation kernels within the NSST set may be called NSST kernels. The selection of a particular set from among the aforementioned set of transformations may be performed, for example, based on the intra-prediction mode of the target block (CU or subblock).

[0099] For reference, for example, an intraprediction mode may include two non-directinoal (or non-angular) intraprediction modes and 65 directional (or angular) intraprediction modes. The non-directinoal intraprediction mode may include a planar intraprediction mode (number 0) and a DC intraprediction mode (number 1), and the directional intraprediction mode may include 65 intraprediction modes (numbers 2 through 66). However, this is merely an example, and this document may also apply to cases with a different number of intraprediction modes. On the other hand, a 67th intraprediction mode may be used in some cases, and this 67th intraprediction mode may represent a linear model (LM) mode.

[0100] Figure 5 illustrates 65 intradirectional modes for predicting directions.

[0101] Referring to Figure 5, we can distinguish between intra-prediction modes with horizontal directionality and intra-prediction modes with vertical directionality, centered around intra-prediction mode 34, which has a diagonal prediction direction pointing upward to the left. In Figure 5, H and V represent horizontal and vertical directionality, respectively, and the numbers -32 to 32 indicate a displacement of 1 / 32 units on the sample grid position. This can be used to indicate an offset relative to the mode index value. Intra-prediction modes 2 through 33 have horizontal directionality, while intra-prediction modes 34 through 66 have vertical directionality. On the other hand, intra-prediction mode 34 can be seen as having neither horizontal nor vertical directionality strictly speaking, but it can be classified as belonging to the horizontal directionality in terms of determining the transformation set of the quadratic transformation. This is because the input data is transposed for the vertical directionality modes which are symmetrical with respect to intra-prediction mode 34, while the input data alignment method for the horizontal directionality modes is used for intra-prediction mode 34. Transposing input data means that for 2D block data M×N, rows become columns and columns become rows to form N×M data. Intra-prediction modes 18 and 50 represent the horizontal intra-prediction mode and the vertical intra-prediction mode, respectively. Intra-prediction mode 2 can be called the right-upward diagonal intra-prediction mode because it predicts in the right-upward direction with a left-side reference pixel. Similarly, intra-prediction mode 34 can be called the right-downward diagonal intra-prediction mode, and intra-prediction mode 66 can be called the left-downward diagonal intra-prediction mode.

[0102] In this case, the mapping between the 35 transformation sets and the intra-prediction mode is shown, for example, in the table below. For reference, when the LM mode is applied to the target block, the quadratic transformation is not applied to the target block.

[0103] [Table 2]

[0104] On the other hand, once it is determined that a specific set is to be used, one of the k transformation kernels within that specific set can be selected via an inseparable quadratic transformation index. The encoding device can derive an inseparable quadratic transformation index that points to a specific transformation kernel based on an RD (rate-distortion) check, and can signal the decoding device to the inseparable quadratic transformation index. The decoding device can select one of the k transformation kernels within the specific set based on the inseparable quadratic transformation index. For example, NSST index value 0 may point to the first inseparable quadratic transformation kernel, NSST index value 1 may point to the second inseparable quadratic transformation kernel, and NSST index value 2 may point to the third inseparable quadratic transformation kernel. Alternatively, NSST index value 0 may indicate that the first inseparable quadratic transformation is not applied to the target block, and NSST index values ​​1 to 3 may point to the three transformation kernels.

[0105] Referring again to Figure 4, the transformation unit can perform the non-separable quadratic transformation based on the selected transformation kernel and obtain the modified (quadratic) transformation coefficients. The modified transformation coefficients can be derived as quantized transformation coefficients via the quantization unit, as described above, and can be encoded and signaled to the decoding unit and transmitted to the inverse quantization / inverse transformation unit within the encoding unit.

[0106] On the other hand, as mentioned above, if the quadratic transformation is omitted, the (primary) transformation coefficients, which are the output of the primary (separated) transformation, can be derived as quantized transformation coefficients via the quantization unit, as described above, and can be encoded and transmitted to the decoding device for signaling and to the inverse quantization / inverse transformation unit within the encoding device.

[0107] The inverse transform unit can perform a series of steps in the reverse order of the steps performed by the transform unit described above. The inverse transform unit can receive the (inversely quantized) transform coefficients, perform a quadratic (inverse) transform to derive (primary) transform coefficients (S450), and perform a primary (inverse) transform on the (primary) transform coefficients to obtain a residual block (residual sample) (S460). Here, the primary transform coefficients can be called modified transform coefficients from the perspective of the inverse transform unit. As described above, the encoding and decoding devices can generate a restored block based on the residual block and the predicted block, and generate a restored picture based on this.

[0108] On the other hand, the decoding device may further include a quadratic inverse transform applicability determination unit (or an element that determines whether a quadratic inverse transform is applicable) and a quadratic inverse transform determination unit (or an element that determines the quadratic inverse transform). The quadratic inverse transform applicability determination unit can determine whether a quadratic inverse transform is applicable. For example, the quadratic inverse transform may be NSST or RST, and the quadratic inverse transform applicability determination unit can determine whether a quadratic inverse transform is applicable based on a quadratic transform flag parsed from the bitstream. As another example, the quadratic inverse transform applicability determination unit may also determine whether a quadratic inverse transform is applicable based on the transformation coefficients of the residual block.

[0109] The quadratic inverse transform determination unit can determine the quadratic inverse transform. In this case, the quadratic inverse transform determination unit can determine the quadratic inverse transform to be applied to the current block based on the NSST (or RST) transform set specified by the intra-prediction mode. In one embodiment, the quadratic transform determination method can be determined depending on the linear transform determination method. Various combinations of linear and quadratic transforms can be determined by the intra-prediction mode. Also, as an example, the quadratic inverse transform determination unit can determine the region to which the quadratic inverse transform is applied based on the size of the current block.

[0110] On the other hand, as mentioned above, if the quadratic (inverse) transformation is omitted (inverse quantization), a residual block (residual sample) can be obtained by receiving the transformation coefficients and performing the primary (separated) inverse transformation. As mentioned above, the encoding and decoding devices can generate a reconstructed block based on the residual block and the predicted block, and generate a reconstructed picture based on this.

[0111] On the other hand, in this document, in order to reduce the computational complexity and memory requirements of unseparable quadratic transforms, we can apply RST (reduced secondary transform), which is a reduced version of the NSST concept in which the size of the transformation matrix (kernel) is reduced.

[0112] On the other hand, the conversion kernel, conversion matrix, and coefficients constituting the conversion kernel matrix described in this document—that is, kernel coefficients or matrix coefficients—can be represented using 8 bits. This is one condition for implementation in decoding and encoding devices, and it reduces the memory requirements for storing the conversion kernel, along with a reasonably acceptable performance degradation compared to existing 9-bit or 10-bit representations. Furthermore, representing the kernel matrix with 8 bits allows for the use of smaller multipliers, making it more compatible with SIMD (Single Instruction Multiple Data) instructions used for optimal software implementation.

[0113] In this specification, RST can mean a transformation performed on a residual sample of a target block based on a transform matrix whose size has been reduced by a simplification factor. When a simplification transformation is performed, the amount of computation required during the transformation can be reduced by reducing the size of the transform matrix. That is, RST can be used to resolve the complexity problem that arises when transforming large blocks or during non-separable transformations.

[0114] RST can also be referred to by a variety of terms, including reduced transform, reduced secondary transform, reduction transform, simplified transform, and simple transform, and the names used to refer to RST are not limited to the examples listed. Alternatively, because RST primarily occurs in the low-frequency region, which includes non-zero coefficients in the transform block, it is also called LFNST (Low-Frequency Non-Separable Transform).

[0115] On the other hand, when the quadratic inverse transformation is performed based on RST, the inverse transformation unit 235 of the encoding device 200 and the inverse transformation unit 322 of the decoding device 300 may include an inverse RST unit that derives corrected transformation coefficients based on the inverse RST of the transformation coefficients, and an inverse linear transformation unit that derives a residual sample for the target block based on the inverse linear transformation of the corrected transformation coefficients. The inverse linear transformation means the inverse transformation of the linear transformation applied to the residual. In this document, deriving transformation coefficients based on a transformation may mean deriving transformation coefficients by applying the transformation in question.

[0116] Figure 6 is a diagram illustrating an RST according to one embodiment described in this document.

[0117] In this specification, “target block” may mean the current block or residual block on which coding is performed.

[0118] In one embodiment of the RST, an N-dimensional vector can be mapped to an R-dimensional vector located in another space, thereby determining a reduced transformation matrix, where R is less than N. N can represent the square of the side length of the block to which the transformation is applied, or the total number of transformation coefficients corresponding to the block to which the transformation is applied, and the simplified factor can represent the R / N value. The simplified factor can also be called by various terms such as reduced factor, reduction factor, simplified factor, or simple factor. On the other hand, R can be called the reduced coefficient, but in some cases the simplified factor can also represent R. Also, in some cases the simplified factor can represent the N / R value.

[0119] In one embodiment, the simplification factor or simplification coefficient may be signaled via a bitstream, but the embodiment is not limited thereto. For example, predefined values ​​for the simplification factor or simplification coefficient may be stored in each encoding device 200 and decoding device 300, in which case the simplification factor or simplification coefficient may not be signaled separately.

[0120] The size of the simplified transformation matrix according to one embodiment is R×N, which is smaller than the size N×N of a normal transformation matrix, and can be defined as shown in the following equation 4.

[0121]

number

[0122] The matrix T within the Reduced Transform block shown in Figure 6(a) is the matrix T in Equation 4. R×N This can mean the following: As shown in Figure 6(a), the simplified transformation matrix T is used for the residual sample for the target block. R×N When multiplication occurs, a conversion coefficient for the target block can be derived.

[0123] In one embodiment, if the size of the block to which the transformation is applied is 8x8 and R=16 (i.e., R / N=16 / 64=1 / 4), then the RST shown in Figure 6(a) can be expressed by a matrix operation as shown in Equation 5 below. In this case, memory and multiplication operations can be reduced to approximately 1 / 4 by a simplification factor.

[0124] In this document, matrix operations can be understood as operations in which a matrix is ​​placed to the left of a column vector and multiplied by the column vector to obtain a column vector.

[0125]

number

[0126] In equation 5, r1 to r 64 This can represent the residual sample for the target block, and more specifically, it is the transformation coefficient generated by applying a linear transformation. The result of the calculation in Equation 5 is the transformation coefficient c for the target block. i This can be derived, and c i The derivation process is as shown in equation 6.

[0127]

number

[0128] The result of the calculation in formula 6, and the conversion coefficients c1 to c for the target block. R The following can be derived: That is, when R=16, the conversion coefficients c1 to c for the target block are 16 The following can be derived. If a regular transformation instead of RST were applied, and a transformation matrix of size 64×64 (N×N) was multiplied by a residual sample of size 64×1 (N×1), 64 (N) transformation coefficients would be derived for the target block. However, because RST was applied, only 16 (R) transformation coefficients are derived for the target block. As the total number of transformation coefficients for the target block decreases from N to R, the amount of data that the encoding device 200 sends to the decoding device 300 decreases, thus increasing the transmission efficiency between the encoding device 200 and the decoding device 300.

[0129] From the perspective of the size of the transformation matrix, the size of a normal transformation matrix is ​​64 × 64 (N × N), while the size of a simplified transformation matrix is ​​reduced to 16 × 64 (R × N). Therefore, when executing RST, memory usage can be reduced by an R / N ratio compared to when performing a normal transformation. Also, compared to the number of multiplication operations N × N when using a normal transformation matrix, the number of multiplication operations can be reduced by an R / N ratio (R × N) when using a simplified transformation matrix.

[0130] In one embodiment, the conversion unit 232 of the encoding device 200 can derive conversion coefficients for a target block by performing a primary conversion and an RST-based secondary conversion on the residual samples for the target block. Such conversion coefficients can be transmitted to the inverse conversion unit of the decoding device 300. The inverse conversion unit 322 of the decoding device 300 can derive modified conversion coefficients based on an inverse RST (reduced secondary transform) for the conversion coefficients, and can derive residual samples for the target block based on an inverse primary conversion for the modified conversion coefficients.

[0131] Inverse RST matrix T according to one embodiment N×R has a size of N×R, which is smaller than the size N×N of the normal inverse conversion matrix, and is in a transpose relationship with the simplified conversion matrix T shown in Equation 4. R×N

[0132] The matrix T in the Reduced Inv.Transform block shown in (b) of FIG. 6 t can mean the inverse RST matrix T R×N T (the superscript T means transpose). When the inverse RST matrix T R×N T is multiplied by the conversion coefficients for the target block as shown in (b) of FIG. 6, modified conversion coefficients for the target block or residual samples for the target block can be derived. The inverse RST matrix T R×N T can also be expressed as (T [[ID=2S]] R×N ) T N×R .

[0133] More specifically, when inverse RST is applied in the secondary inverse conversion, the inverse RST matrix T R×N TWhen multiplied, the modified transformation coefficients for the target block can be derived. On the other hand, the inverse RST can be applied in the inverse linear transformation, in which case, when the transformation coefficients for the target block are multiplied by the inverse RST matrix TR × NT, the residual samples for the target block can be derived.

[0134] In one embodiment, when the size of the block to which the inverse transform is applied is 8x8 and R=16 (i.e., R / N=16 / 64=1 / 4), the RST shown in Figure 6(b) can be expressed by a matrix operation as shown in the following equation 7.

[0135]

number

[0136] In formula 7, c1 to c 16 This can show the conversion coefficient for the target block. The result of the calculation in Equation 7, the modified conversion coefficient for the target block, or r showing the residual sample for the target block. j This can be derived, and r j The derivation process is as shown in equation 8.

[0137]

number

[0138] The result of the calculation in Equation 8, the modified conversion coefficient for the target block, or r1 to r representing the residual sample for the target block. NThis can be derived. Considering the size of the inverse transform matrix, the size of a normal inverse transform matrix is ​​64 × 64 (N × N), while the size of the simplified inverse transform matrix is ​​reduced to 64 × 16 (N × R). Therefore, when performing the inverse RST, memory usage can be reduced by an R / N ratio compared to when performing a normal inverse transform. Also, compared to the number of multiplication operations N × N when using a normal inverse transform matrix, the number of multiplication operations can be reduced by an R / N ratio (N × R) when using the simplified inverse transform matrix.

[0139] On the other hand, the transformation set configuration shown in Table 2 can also be applied to 8x8 RST. That is, the corresponding 8x8 RST can be applied using the transformation set in Table 2. Since one transformation set consists of two or three transformations (kernels) depending on the prediction mode on the screen, it can be configured to select one from a maximum of four transformations, including the case where a quadratic transformation is not applied. When a quadratic transformation is not applied, it can be considered as if the identity matrix is ​​applied. If we assign indices 0, 1, 2, and 3 to the four transformations respectively (for example, index 0 can be assigned to the identity matrix, i.e., when a quadratic transformation is not applied), the transformation to be applied can be specified by signaling each transformation coefficient block with a syntax element called an NSST index. That is, an 8x8 NSST can be specified for the 8x8 upper left block via the NSST index, and an 8x8 RST can be specified in the RST configuration. 8×8 NSST and 8×8 RST refer to transformations that can be applied to an 8×8 area contained within a transformation coefficient block when both W and H of the target block to be transformed are equal to or greater than 8, and the relevant 8×8 area is the upper left 8×8 area within the transformation coefficient block. Similarly, 4×4 NSST and 4×4 RST refer to transformations that can be applied to a 4×4 area contained within a transformation coefficient block when both W and H of the target block are equal to or greater than 4, and the relevant 4×4 area is the upper left 4×4 area within the transformation coefficient block.

[0140] On the other hand, when applying an (forward) 8x8 RST like in equation 4, 16 effective conversion coefficients are generated. This means that the 64 input data constituting the 8x8 region are reduced to 16 output data. From a two-dimensional perspective, this means that only about 1 / 4 of the region is filled with effective conversion coefficients. Therefore, the 16 output data obtained by applying the forward 8x8 RST can be seen, for example, in the upper left region of the block in Figure 7 (conversion coefficients 1 through 16, i.e., c1, c2, ..., c obtained via equation 6). 16 This can be satisfied by a diagonal scanning sequence from number 1 to number 16.

[0141] Figure 7 shows the scanning sequence of conversion coefficients according to one embodiment of this document. As mentioned above, if the forward scan sequence starts from number 1, the reverse scan can be performed from the 64th to the 17th forward scan in the direction and order of the arrows shown in Figure 7.

[0142] In Figure 7, the upper left 4x4 region is the ROI (Region of Interest) region where the valid transformation coefficients are satisfied, while the remaining region is empty. The empty region can be filled with a value of 0 by default.

[0143] That is, when an 8x8 RST with a forward transformation matrix of 16x64 is applied to an 8x8 region, the output transformation coefficients are placed in the upper left 4x4 region, and regions where no output transformation coefficients exist can be filled with 0 (from the 64th to the 17th) following the scan order in Figure 7.

[0144] If a valid non-zero conversion coefficient is found outside the ROI region in Figure 7, it is certain that the 8x8 RST will not be applied, and the corresponding NSST index coding can be omitted. On the other hand, if no non-zero conversion coefficient is found outside the ROI region in Figure 7 (for example, when the conversion coefficient for regions other than the ROI is set to 0 when the 8x8 RST is applied), it is possible that the 8x8 RST was applied, and therefore the NSST index can be coded. This type of conditional NSST index coding requires checking for the presence or absence of non-zero conversion coefficients, and can therefore be performed after the residual coding process.

[0145] This document deals with the design and related optimization methods of RSTs that can be applied to 4x4 blocks from the RST structure described in this embodiment. Naturally, some concepts can be applied not only to 4x4 RSTs but also to 8x8 RSTs or other forms of conversion.

[0146] Figure 8 is a flowchart showing the reverse RST process according to one embodiment of this document.

[0147] Each step disclosed in Figure 8 can be performed by the decoding device 300 disclosed in Figure 3. More specifically, S800 can be performed by the inverse quantization unit 321 disclosed in Figure 3, and S810 and S820 can be performed by the inverse transform unit 322 disclosed in Figure 3. Therefore, specific details that overlap with those described above in Figure 3 will be omitted or simplified in their explanation. On the other hand, in this document, RST can mean a transform applied in the forward direction, and inverse RST can mean a transform applied in the in-bus direction.

[0148] In one embodiment, the detailed operation by the inverse RST is the exact opposite in order to the detailed operation by the RST, and the detailed operation by the RST and the detailed operation by the inverse RST are substantially similar. Therefore, a person ordinary in the art will readily understand that the descriptions of the inverse RST described below in S800 to S820 are applicable to the RST in the same or similar manner.

[0149] The decoding device 300 according to one embodiment can derive conversion coefficients by performing inverse quantization on the quantized conversion coefficients for the target block (S800).

[0150] On the other hand, the decoding device 300 can determine whether or not to apply an inverse quadratic transformation after the inverse linear transformation and before the inverse quadratic transformation. For example, the inverse quadratic transformation may be NSST or RST. As one example, the decoding device can determine whether or not to apply an inverse quadratic transformation based on a quadratic transformation flag parsed from the bitstream. As another example, the decoding device can also determine whether or not to apply an inverse quadratic transformation based on the transformation coefficients of the residual block.

[0151] Furthermore, the decoding device 300 can determine the inverse quadratic transformation. In this case, the decoding device 300 can also determine the inverse quadratic transformation to be applied to the current block based on the NSST (or RST) transformation set specified by the intra-prediction mode. In one embodiment, the quadratic transformation determination method can be determined depending on the primary transformation determination method. For example, it can be determined that RST or LFNST is applied only when DCT-2 is applied as the transformation kernel in the primary transformation. Alternatively, various combinations of primary and quadratic transformations can be determined by the intra-prediction mode.

[0152] As an example, the decoding device 300 can also determine the region to which the inverse quadratic transformation is applied based on the size of the current block before the step of determining the inverse quadratic transformation.

[0153] In one embodiment, the decoding device 300 can select a transform kernel (S810). More specifically, the decoding device 300 can select a transform kernel based on at least one of the following: the transform index, the width and height of the region to which the transform is applied, the intra-prediction mode used in video decoding, and information regarding the color component of the target block. However, the embodiment is not limited thereto, and for example, the transform kernel may be a predefined one, and no additional information for selecting a transform kernel may be signaled.

[0154] In one example, information regarding the hue component of the target block can be indicated via CIdx. If the target block is a luma block, CIdx can indicate 0, and if the target block is a chroma block, such as a Cb block or a Cr block, CIdx can indicate a non-zero value (e.g., 1).

[0155] The decoding device 300 according to one embodiment can apply the inverse RST to the conversion coefficients based on the selected conversion kernel and reduced factor (S820).

[0156] Below, we propose a method for determining a secondary NSST set, i.e., a secondary transformation set or transformation set, by considering the intra-prediction mode and block size, based on one embodiment of this document.

[0157] As one embodiment, by configuring a set for the current transformation block based on the intra prediction mode described above, a transformation set composed of transformation kernels of various sizes can be applied to the transformation block. When the transformation sets in Table 3 are represented from 0 to 3, they are shown in Table 4.

[0158] [Table 3]

[0159] [Table 4]

[0160] The indices 0, 2, 18, and 34 shown in Table 3 correspond to 0, 1, 2, and 3 in Table 4, respectively. In Tables 3 and 4, only four conversion sets are used instead of 35, which significantly reduces the memory space required.

[0161] Furthermore, the diverse number of transformation kernel matrices that can be included in each transformation set can be set as shown in the table below.

[0162] [Table 5]

[0163] [Table 6]

[0164] [Table 7]

[0165] Table 5 shows that two available translation kernels are used for each translation set, resulting in translation indices ranging from 0 to 2.

[0166] According to Table 6, two available conversion kernels are used for conversion set 0, i.e., the conversion set for the DC mode and planar mode among the intra-prediction modes, while one conversion kernel is used for each of the remaining conversion sets. In this case, the available conversion index for conversion set 1 changes from 0 to 2, and the conversion index for the remaining conversion sets 1 through 3 changes from 0 to 1.

[0167] Table 7 shows that one available translation kernel is used for each translation set, resulting in translation indices ranging from 0 to 1.

[0168] On the other hand, in the transformation set mapping in Table 3, four transformation sets can be used in all cases, and these four transformation sets can be rearranged as shown in Table 4 so that they are divided into indices 0, 1, 2, and 3. Tables 8 and 9 below exemplify four transformation sets that can be used for quadratic transformations, with Table 8 showing a transformation kernel matrix that can be applied to an 8x8 block, and Table 9 showing a transformation kernel matrix that can be applied to a 4x4 block. Tables 8 and 9 consist of two transformation kernel matrices per transformation set, and two transformation kernel matrices can be applied to all intra-prediction modes as shown in Table 5.

[0169] [Table 8-1]

[0170] [Table 8-2]

[0171] [Table 8-3]

[0172] [Table 8-4]

[0173] [Table 8-5]

[0174] [Table 8-6]

[0175] Table 8-7

[0176] Table 8-8

[0177] Table 9-1

[0178] Table 9-2

[0179] Table 9-3

[0180] Table 9-4

[0181] Table 9-5

[0182] Table 9-6

[0183] Table 9-7

[0184] Table 9-8

[0185] The transformation kernel matrix examples presented in Table 8 are all transformation kernel matrices multiplied by 128 as a scaling value. In the g_aiNsst8×8[N1][N2]

[16]

[64] array that appears in the matrix array of Table 8, N1 indicates the number of transformation sets (N1 is divided into 4 or 35, indices 0, 1, ..., N1-1), N2 indicates the number of transformation kernel matrices that make up each transformation set (1 or 2), and

[16]

[64] indicates a 16×64 Reduced Secondary Transform (RST).

[0186] As shown in Tables 3 and 4, if a set of transformations is composed of a single transformation kernel matrix, then in Table 8, either the first or second transformation kernel matrix can be used for that set of transformations.

[0187] Applying the relevant RST outputs 16 transformation coefficients, but if only the m×64 portion of the 16×64 matrix is ​​applied, it can be configured to output only m transformation coefficients. For example, by setting m=8 and multiplying only the 8×64 matrix from the top to output only 8 transformation coefficients, the computational complexity can be halved. To reduce the computational complexity in the worst case, an 8×64 matrix can be applied to an 8×8 transformation unit (TU).

[0188] Thus, an m×64 transformation matrix (m≦16, e.g., the transformation kernel matrix in Table 8) that can be applied to an 8×8 area receives 64 data inputs and generates m coefficients. That is, as shown in Equation 5, if the 64 data points form a 64×1 vector, the m×64 matrix and the 64×1 vector are multiplied in order to generate an m×1 vector. At this time, the 64 data points forming the 8×8 area can be appropriately arranged to construct a 64×1 vector, and as an example, the data can be arranged in the order of the indices displayed at each position in the 8×8 area, as shown in Table 10 below.

[0189] [Table 10]

[0190] As shown in Table 10, the data array in the 8x8 region for the quadratic transformation is in row-major direction order. This refers to the order in which two-dimensional data is arranged in one dimension for a quadratic transformation, specifically RST or LFNST, and this can be applied to the forward quadratic transformation performed by the encoding device. Therefore, in the reverse quadratic transformation performed in the inverse transformation section of the encoding device or the inverse transformation section of the decoding device, the resulting transformation coefficients, i.e., the linear transformation coefficients, can be arranged in two dimensions as shown in Table 10.

[0191] On the other hand, when the prediction modes on the screen consist of 67 modes as shown in Figure 5, all directional modes (2nd to 66th) are configured symmetrically with respect to mode 34. That is, mode (2+n) is symmetrical with mode (66-n) with respect to mode 34 on the prediction direction side (0≦n≦31). Therefore, if the data array order for constructing a 64×1 input vector for mode (2+n), i.e., modes 2 through 33, is row-major as shown in Table 10, then for mode (66-n), a 64×1 input vector can be constructed in the order shown in Table 11.

[0192] [Table 11]

[0193] As shown in Table 11, the data array in the 8x8 region for the quadratic transformation is in column-major direction order. This refers to the order in which two-dimensional data is arranged in one dimension for a quadratic transformation, specifically RST or LFNST, and this can be applied to the forward quadratic transformation performed by the encoding device. Therefore, in the reverse quadratic transformation performed in the inverse transform section of the encoding device or the inverse transform section of the decoding device, the resulting transformation coefficients, i.e., the linear transformation coefficients, can be arranged in two dimensions as shown in Table 11.

[0194] Table 11 shows that for prediction modes (66-n) on the screen, i.e., modes 35 through 66, a 64 × 1 input vector can be constructed using a column-first direction order.

[0195] To summarize, for the (2+n)th mode, the input data can be arranged symmetrically in row-major order, and for the (66-n)th mode (0≦n≦31), the input data can be arranged in column-major order, while applying the same transformation kernel matrix as shown in Table 8. Examples of which transformation kernel matrices to apply to each mode are shown in Tables 5 to 7. In this case, for intra-prediction mode 0 (planar mode), intra-prediction mode 1 (DC mode), and intra-prediction modes 34, either the arrangement order in Table 10 or Table 11 can be applied. For example, for intra-prediction mode 34, the input data can be arranged in row-major order as shown in Table 10.

[0196] As another example shows, the transformation kernel matrix examples presented in Table 9, which can be applied to a 4x4 region, are all transformation kernel matrices multiplied by 128 as a scaling value. In the g_aiNsst4×4[N1][N2]

[16]

[64] array that appears in the matrix array of Table 9, N1 indicates the number of transform sets (N1 is divided into 4 or 35, indices 0, 1, ..., N1-1), N2 indicates the number of transformation kernel matrices that make up each transformation set (1 or 2), and

[16]

[16] indicates a 16x16 transformation.

[0197] As shown in Tables 3 and 4, if a set of transformations is composed of a single transformation kernel matrix, then in Table 9, either the first or second transformation kernel matrix can be used for that set of transformations.

[0198] Similar to the 8x8 RST case, by using only the mx16 portion of the 16x16 matrix, it is possible to configure it so that only m transformation coefficients are output. For example, by setting m=8 and multiplying only the 8x16 matrix from the top to output only 8 transformation coefficients, the computational complexity can be halved. To reduce the computational complexity in the worst case, an 8x16 matrix can be applied to a 4x4 transformation unit (TU).

[0199] Basically, the transformation kernel matrices that can be applied to the 4x4 region presented in Table 9 are applicable to 4x4 TU, 4xM TU, and Mx4 TU (for M>4, 4xM TU and Mx4 TU, the 4x4 region is divided and the specified transformation kernel matrix is ​​applied to each, or it can be applied only to the upper left 4x8 or 8x4 region), or it can be applied only to the upper left 4x4 region. If the quadratic transformation is configured to be applied only to the upper left 4x4 region, the transformation kernel matrix that can be applied to the 8x8 region presented in Table 8 becomes unnecessary.

[0200] Thus, an m×64 transformation matrix (m≦16, e.g., the transformation kernel matrix in Table 9) that can be applied to a 4×4 region receives 16 data inputs and generates m coefficients. That is, if the 16 data points form a 16×1 vector, the m×16 matrix and the 16×1 vector are multiplied in order to generate an m×1 vector. At this time, the 16 data points forming the 4×4 region can be appropriately arranged to construct a 16×1 vector, and as an example, the data can be arranged in the order of the indices displayed at each position in the 4×4 region, as shown in Table 12 below.

[0201] [Table 12]

[0202] As shown in Table 12, the data arrangement in the 4x4 region for the quadratic transformation is row-major direction order. This refers to the order in which two-dimensional data is arranged in one dimension for a quadratic transformation, specifically RST or LFNST, and this can be applied to the forward quadratic transformation performed by the encoding device. Therefore, in the reverse quadratic transformation performed in the inverse transformation section of the encoding device or the inverse transformation section of the decoding device, the resulting transformation coefficients, i.e., the linear transformation coefficients, can be arranged in two dimensions as shown in Table 12.

[0203] On the other hand, when the prediction modes on the screen consist of 67 modes as shown in Figure 5, all directional modes (2nd to 66th) are configured symmetrically around mode 34. That is, mode (2+n) is symmetrical with mode (66-n) on the prediction direction side (0≦n≦31) around mode 34. Therefore, if the data array order for constructing a 16×1 input vector for mode (2+n), i.e., modes 2 through 33, is row-major as shown in Table 12, then for mode (66-n), a 16×1 input vector can be constructed in the order shown in Table 13.

[0204] [Table 13]

[0205] As shown in Table 13, the data arrangement in the 4x4 region for the quadratic transformation is column-major directional order. This refers to the order in which two-dimensional data is arranged in one dimension for a quadratic transformation, specifically RST or LFNST, and this can be applied to the forward quadratic transformation performed by the encoding device. Therefore, in the reverse quadratic transformation performed in the inverse transform section of the encoding device or the inverse transform section of the decoding device, the resulting transformation coefficients, i.e., the linear transformation coefficients, can be arranged in two dimensions as shown in Table 13.

[0206] Table 13 shows that for prediction modes (66-n) on the screen, i.e., modes 35 through 66, a 16×1 input vector can be constructed using a column-first direction order.

[0207] To summarize, for (2+n) modes, the input data can be arranged symmetrically in row-major order, and for (66-n) modes (0≦n≦31), the input data can be arranged in column-major order, while applying the same transformation kernel matrix as shown in Table 9. Examples of which transformation kernel matrices to apply to each mode are shown in Tables 5 to 7. In this case, for intra-prediction mode 0 (planar mode), intra-prediction mode 1 (DC mode), and intra-prediction modes 34, either the arrangement order in Table 12 or Table 13 can be applied. For example, for intra-prediction mode 34, the input data can be arranged in row-major order as shown in Table 12.

[0208] On the other hand, other embodiments of this document allow for the selection of only 48 data points from 64 data points constituting an 8x8 region, which are not represented by the maximum 16x64 transformation kernel matrix shown in Tables 8 and 9, and the application of the maximum 16x48 transformation kernel matrix. Here, "maximum" means that for an mx48 transformation kernel matrix capable of generating m coefficients, the maximum value of m is 16.

[0209] The 16x48 transformation kernel matrix according to this embodiment can be shown in Table 14.

[0210] [Table 14-1]

[0211] [Table 14-2]

[0212] [Table 14-3]

[0213] [Table 14-4]

[0214] [Table 14-5]

[0215] [Table 14-6]

[0216] [Table 14-7]

[0217] [Table 14-8]

[0218] When performing RST by applying an m×48 conversion kernel matrix (m≤16) to an 8×8 area, it is possible to receive an input of 48 pieces of data and generate m coefficients. Table 14 shows an example of the conversion kernel matrix when m is 16, and receives an input of 48 pieces of data to generate 16 coefficients. That is, when 48 pieces of data form a 48×1 vector, a 16×1 vector can be generated by multiplying the 16×48 matrix and the 48×1 vector in order. At this time, 48 pieces of data forming an 8×8 area can be appropriately arranged to form a 48×1 vector, and the input data can be arranged in the following order.

[0219]

Table 15

[0220] During RST, when performing matrix operations by applying a maximum 16×48 conversion kernel matrix as shown in Table 14, 16 corrected conversion coefficients are generated, and the 16 corrected conversion coefficients can be arranged in the upper left 4×4 area in scanning order, and the upper right 4×4 area and the lower left 4×4 area can be filled with 0s. Table 16 shows an example of the arrangement order of the 16 corrected conversion coefficients generated through matrix operations.

[0221]

Table 16

[0222] As shown in Table 16, the modified transformation coefficients generated when applying a maximum 16x48 transformation kernel matrix can fill the upper left 4x4 region according to the scanning order. In this case, the numbers at each position in the upper left 4x4 region indicate the scanning order. Typically, the coefficient generated from the dot product operation between the top row of the 16x48 transformation kernel matrix and the 48x1 input column vector is the first in the scanning order. In this case, the scanning order can coincide with the direction of descending to the lower rows. For example, the coefficient generated from the dot product operation between the 48x1 input column vector and the nth row from the top is the nth in the scanning order.

[0223] In the case of a maximum 16x48 transformation kernel matrix, the 4x4 area in the lower right of Table 16 is an area to which no quadratic transformation is applied, so the original input data (linear transformation coefficients) is preserved as is, and the 4x4 areas in the upper right and lower left become filled with 0.

[0224] Furthermore, according to other embodiments, other scanning sequences can be applied besides those presented in Table 16. For example, row-first or column-first scanning sequences can be applied.

[0225] Furthermore, applying a 16x64 transformation kernel matrix as shown in Table 8 also generates 16 transformation coefficients. These 16 transformation coefficients can be arranged in the scanning order presented in Table 16. When applying the 16x64 transformation kernel matrix, all 64 input data points (not 48) are used for matrix operations, resulting in all 4x4 regions except the top-left 4x4 region being filled with 0. In this case as well, the scanning order can be the diagonal scanning order shown in Table 16, or other different scanning orders such as row-major or column-major.

[0226] On the other hand, when inverse RST or LFNST is performed as the inverse transformation process executed by the decoding device, the input coefficient data to which inverse RST is applied consists of one-dimensional vectors following the arrangement order in Table 16. The corrected coefficient vectors obtained by multiplying the one-dimensional vectors by the corresponding inverse RST matrix on the left side can be arranged in two-dimensional blocks according to the arrangement order in Table 15.

[0227] The decoding device can arrange the information for the previously received conversion coefficients in reverse scanning order, starting from number 64 in Figure 7, in diagonal scanning order, in order to derive the conversion coefficients.

[0228] Subsequently, the inverse transformation unit 322 of the decoding device can apply the transformation kernel matrix to the transformation coefficients arranged in one dimension according to the scanning order in Table 16. That is, 48 ​​modified transformation coefficients can be derived through matrix operations between the one-dimensional transformation coefficients arranged according to the scanning order in Table 16 and the transformation kernel matrix based on the transformation kernel matrix in Table 14. In other words, the one-dimensional transformation coefficients can be derived as 48 modified transformation coefficients through matrix operations with a matrix obtained by transposing the transformation kernel matrix in Table 14.

[0229] The 48 modified transformation coefficients derived in this way can be arranged in two dimensions for the inverse linear transformation, as shown in Table 15.

[0230] To summarize, when RST or LFNST is applied to an 8x8 region during the transformation process, a matrix operation is performed between the 48 transformation coefficients from the upper left, upper right, and lower left regions of the 8x8 region (excluding the lower right region) and a 16x48 transformation kernel matrix. For the matrix operation, the 48 transformation coefficients are input in a one-dimensional array in the order shown in Table 15. After such a matrix operation is performed, 16 modified transformation coefficients are derived, and these modified transformation coefficients can be arranged in the upper left region of the 8x8 region in the form shown in Table 16.

[0231] Conversely, when inverse RST or LFNST is applied to an 8x8 region during the inverse transformation process, the 16 transformation coefficients corresponding to the upper left of the 8x8 region can be input in a one-dimensional array form according to the scanning order shown in Table 16 and performed matrix operations with the 48x16 transformation kernel matrix. That is, the matrix operation in this case can be expressed as (48x16 matrix) * (16x1 transformation coefficient vector) = (48x1 modified transformation coefficient vector). Here, an nx1 vector can be interpreted as having the same meaning as an nx1 matrix, and can therefore also be expressed as an nx1 column vector. Also, * signifies matrix multiplication. When such a matrix operation is performed, 48 modified transformation coefficients can be derived, and these 48 modified transformation coefficients can be arranged in the upper left, upper right, and lower left regions of the 8x8 region, excluding the lower right region, as shown in Table 15.

[0232] On the other hand, as shown in Table 15 in one embodiment, the data array in the 8x8 region for the quadratic transformation is in row-first direction order. On the other hand, when the prediction modes in the screen consist of 67 as shown in Figure 5, all direction modes (2nd to 66th) are configured symmetrically with respect to mode 34. That is, mode (2+n) is symmetric with mode (66-n) with respect to mode 34 on the (0≦n≦31) prediction direction side. Therefore, if the data array order for constructing a 48×1 input vector for mode (2+n), i.e., modes 2 to 33, is in row-first direction as shown in Table 15, then for mode (66-n), a 48×1 input vector can be constructed in order as shown in Table 17.

[0233] [Table 17]

[0234] As shown in Table 17, the data array in the 8×8 region for the second conversion is in column - first order. Table 17 shows that for the prediction modes (66 - n) in the screen, that is, for the 35th to 66th modes, a 64×1 input vector can be constructed in column - first order.

[0235] After sorting, for the (2 + n)th mode, the input data can be symmetrically arranged in row - first order, and for the (66 - n)th mode (0≦n≦31), the input data can be symmetrically arranged in column - first order, while applying the same conversion kernel matrix as shown in Table 14. Examples of which conversion kernel matrix to apply for each mode are shown in Tables 5 to 7. [[ID=​​​​​​​​​Therefore, if there is a non-zero conversion coefficient between the (m+1)th and 16th positions in the scanning order, or in the upper right 4x4 region or the lower left 4x4 region, it can be determined that the m x 48 quadratic transformation is not applicable (m ≤ 16). In such cases, the index for the quadratic transformation is not signaled. The decoding device first parses the conversion coefficients to check whether the relevant condition (i.e., whether a non-zero conversion coefficient exists in a region where the conversion coefficient should be 0 in the quadratic transformation) is met. If it is met, the index for the relevant quadratic transformation can be guided to 0 without parsing. For example, when m = 16, the applicability of the quadratic transformation and whether the index for the quadratic transformation can be parsed can be determined by checking whether there is a non-zero coefficient in the upper right 4x4 region or the lower left 4x4 region.

[0239] Table 18, on the other hand, shows other examples of transformation kernel matrices that can be applied to a 4x4 region.

[0240] [Table 18-1]

[0241] [Table 18-2]

[0242] [Table 18-3]

[0243] [Table 18-4]

[0244] On the other hand, in order to reduce the computational complexity in the worst case, the following implementation can be proposed. In this document, a matrix consisting of M rows and N columns is represented as an M×N matrix, where the M×N matrix represents the transformation matrix applied when performing a forward transformation, i.e., a transformation (RST) in the encoding device. Therefore, in the inverse transformation (inverse RST) performed in the decoding device, an N×M matrix obtained by transposing the M×N matrix can be used. Furthermore, although the following description applies to the case where an m×64 transformation kernel matrix (m≦16) is applied as the transformation matrix for an 8×8 region, it can also be applied to the case where the input vector is 48×1 and an m×48 transformation kernel matrix (m≦16) is applied. That is, 16×64 (or m×64) can be replaced with 16×48 (or m×48).

[0245] 1) For a block (e.g., a transformation unit) with width W and height H, if W ≥ 8 and H ≥ 8, a transformation kernel matrix that can be applied to an 8x8 region is applied to the upper left 8x8 region of the block. If W = 8 and H = 8, only the 8x64 portion of the 16x64 matrix can be applied. That is, 8 transformation coefficients can be generated. Alternatively, only the 8x48 portion of the 16x48 matrix can be applied. That is, 8 transformation coefficients can be generated.

[0246] 2) For a block (e.g., a transformation unit) with width W and height H, if one of W or H is less than 8, i.e., if one of W or H is 4, then a transformation kernel matrix that can be applied to a 4x4 region is applied to the upper left of the block. For the case where W=4 and H=4, only the 8x16 portion of the 16x16 matrix can be applied, and in this case, 8 transformation coefficients are generated.

[0247] If (W,H) = (4,8) or (8,4), the quadratic transformation is applied only to the upper left 4x4 region. If W or H is greater than 8, i.e., W or H is equal to or greater than 16, and the other is 4, the quadratic transformation is applied only to the two 4x4 blocks in the upper left. That is, the specified transformation kernel matrix can be applied to a maximum of two 4x4 blocks up to a 4x8 or 8x4 region in the upper left.

[0248] 3) For blocks with width W and height H (e.g., transformation units), the quadratic transformation is not applied if both W and H are 4.

[0249] 4) For a block with width W and height H (e.g., a transformation unit), the number of coefficients generated by applying a quadratic transformation can be configured to be kept at 1 / 4 or less of the area of ​​the transformation unit (i.e., the total number of pixels constituting the transformation unit = W × H). For example, when both W and H are 4, the top 4 × 16 matrix of a 16 × 16 matrix can be applied so that 4 transformation coefficients are generated.

[0250] When a quadratic transformation is applied only to the largest 8x8 region in the upper left of the overall transformation unit (TU), the 4x8 or 8x4 transformation unit must generate 8 or fewer coefficients. Therefore, the configuration can be made to apply the topmost 8x16 matrix of a 16x16 matrix to the 4x4 region in the upper left. For the 8x8 transformation unit, up to a 16x64 matrix (or a 16x48 matrix) can be applied (up to 16 coefficients can be generated). For 4xN or Nx4 (N≧16) transformation units, a 16x16 matrix can be applied to the 4x4 block in the upper left, or the topmost 8x16 matrix of a 16x16 matrix can be applied to the two 4x4 blocks located in the upper left. In a similar manner, for the 4x8 or 8x4 transformation unit, the topmost 4x16 matrix of a 16x16 matrix can be applied to each of the two 4x4 blocks located in the upper left to generate 8 transformation coefficients in all cases.

[0251] 5) The maximum size of a quadratic transformation applied to a 4x4 region can be limited to 8x16. In this case, the amount of memory required to store the transformation kernel matrix applied to the 4x4 region can be reduced to half compared to a 16x16 matrix.

[0252] For example, for all the transformation kernel matrices presented in Table 9 or Table 18, only the top 8x16 matrix can be extracted from each 16x16 matrix, limiting the maximum size to 8x16. This can then be implemented in an actual video coding system to store only the corresponding 8x16 matrix of the transformation kernel matrix.

[0253] If the maximum applicable transformation size is 8x16, and the maximum number of multiplications required to generate a single coefficient is limited to 8, then in the case of a 4x4 block, a maximum of 8x16 matrices can be applied, and for 4xN blocks and Nx4 blocks (N≧8, N=2n, n≧3), a maximum of 8x16 matrices can be applied to each of the two 4x4 blocks in the upper left corner that make up the interior. For example, for 4xN blocks and Nx4 blocks (N≧8, N=2n, n≧3), an 8x16 matrix can be applied to one 4x4 block in the upper left corner.

[0254] In one embodiment, when coding an index that specifies the quadratic transformation to be applied to the Luma component, more specifically, if a set of transformations consists of two transformation kernel matrices, it is necessary to specify whether to apply the quadratic transformation and, if so, which transformation kernel matrices to apply. For example, if the quadratic transformation is not applied, the transformation index can be coded as 0, and if it is applied, the transformation indices for the two sets of transformations can be coded as 1 and 2, respectively.

[0255] In this case, when coding the conversion index, truncated unary coding can be used. For example, conversion indexes 0, 1, and 2 can be coded by assigning the binary codes 0, 10, and 11 respectively.

[0256] Furthermore, when coding using the truncated unary method, a different CABAC context can be assigned to each bin, and when coding conversion indices 0, 10, and 11 as in the example above, two CABAC contexts can be used.

[0257] On the other hand, when coding a transformation index that specifies the quadratic transformation to be applied to the color difference component, more specifically, if a set of transformations consists of two transformation kernel matrices, it is necessary to specify whether or not to apply the quadratic transformation, and if so, which transformation kernel matrix to apply, similar to when coding the transformation index for the quadratic transformation to the luma component. For example, if the quadratic transformation is not applied, the transformation index can be coded as 0, and if it is applied, the transformation index for the two transformation sets can be coded as 1 and 2, respectively.

[0258] In this case, when coding the conversion index, truncated unary coding can be used. For example, conversion indexes 0, 1, and 2 can be coded by assigning the binary codes 0, 10, and 11 respectively.

[0259] Furthermore, when coding using the truncated unary method, a different CABAC context can be assigned to each bin, and when coding conversion indices 0, 10, and 11 as in the example above, two CABAC contexts can be used.

[0260] Furthermore, in one embodiment, different CABAC context sets can be assigned depending on the chromatic intra-prediction mode. For example, if the modes are divided into non-directional modes, such as planar mode or DC mode, and other directional modes (i.e., divided into two groups), then, as in the example above, when coding 0, 10, and 11, the corresponding CABAC context sets (consisting of two contexts) can be assigned to each group.

[0261] When dividing the chroma intra-prediction mode into several groups and assigning the corresponding CABAC context set accordingly, the chroma intra-prediction mode value must be known before coding the conversion index for the quadratic transformation. However, in the case of chroma direct mode (DM), the luma intra-prediction mode value is used directly, so the intra-prediction mode value for the luma component must also be known. Therefore, when coding information for the color difference component, a data dependency on the luma component information can arise. In the case of chroma DM mode, when performing conversion index coding for a quadratic transformation without information on the intra-prediction mode, it is possible to map to a specific group to eliminate the aforementioned data dependency. For example, if the chroma intra-prediction mode is chroma DM mode, it can be considered as planar mode or DC mode and the corresponding CABAC context set can be used to perform the conversion index coding, or it can be considered as another directional mode and the corresponding CABAC context set can be applied.

[0262] Figure 9 is a flowchart illustrating the operation of a video decoding device according to one embodiment described in this document.

[0263] Each step disclosed in Figure 9 can be performed by the decoding device 300 disclosed in Figure 3. More specifically, S910 can be performed by the entropy decoding unit 310 disclosed in Figure 3, S920 can be performed by the inverse quantization unit 321 disclosed in Figure 3, S930 and S940 can be performed by the inverse transform unit 322 disclosed in Figure 3, and S950 can be performed by the adder unit 340 disclosed in Figure 3. In addition, the operation by S910 to S950 is based in part on the content described above in Figures 4 to 8. Therefore, specific details that overlap with the content described above in Figures 3 to 8 will be omitted or simplified in explanation.

[0264] A decoding device 300 according to one embodiment can derive quantized transformation coefficients for a target block from a bitstream (S910). More specifically, the decoding device 300 can decode information regarding quantized transformation coefficients for a target block from a bitstream, and can derive quantized transformation coefficients for a target block based on the information regarding quantized transformation coefficients for a target block. The information regarding quantized transformation coefficients for a target block can be contained in an SPS (Sequence Parameter Set) or a slice header, and may include at least one of the following: information on whether a simplified transformation (RST) is applied, information on the simplified factor, information on the minimum transformation size to apply the simplified transformation, information on the maximum transformation size to apply the simplified transformation, the inverse simplified transformation size, and information on a transformation index that points to one of the transformation kernel matrices contained in the transformation set.

[0265] The decoding device 300 according to one embodiment can derive the conversion coefficients by performing inverse quantization on the quantized conversion coefficients for the target block (S920).

[0266] The derived transformation coefficients can be arranged in 4x4 block units according to the reverse diagonal scan order, and the transformation coefficients within each 4x4 block can also be arranged according to the reverse diagonal scan order. In other words, the transformation coefficients after inverse quantization can be arranged following the reverse scan order applied in video codecs such as VVC and HEVC.

[0267] The decoding device 300 according to one embodiment can derive a modified conversion coefficient based on the inverse RST (reduced secondary transform) of the conversion coefficient (S930).

[0268] In one example, the inverse RST can be performed based on an inverse RST matrix, which is a non-square matrix with fewer columns than rows.

[0269] In one embodiment, S930 may include the steps of decoding a transformation index, determining whether the conditions for applying an inverse RST based on the transformation index are met, selecting a transformation kernel matrix, and, if the conditions for applying an inverse RST are met, applying an inverse RST to the transformation coefficients based on the selected transformation kernel matrix and / or simplification factor. In this case, the size of the simplification inverse transformation matrix may be determined based on the simplification factor.

[0270] The decoding device 300 according to one embodiment can derive a residual sample for the target block based on the inverse transformation of the modified transformation coefficient (S940).

[0271] The decoding device 300 can perform an inverse linear transformation on the modified transformation coefficients for the target block, in which case a simplified inverse transformation may be applied, or a normal separation transformation may be used.

[0272] The decoding device 300 according to one embodiment can generate a reconstructed sample based on a residual sample for the target block and a predicted sample for the target block (S950).

[0273] Referring to S930, it can be confirmed that the residual sample for the target block is derived based on the inverse RST for the conversion coefficients for the target block. Considering the size of the inverse transformation matrix, the size of a normal inverse transformation matrix is ​​N×N, while the size of the inverse RST matrix is ​​reduced to N×R. Therefore, compared to performing a normal transformation, memory usage can be reduced by R / N when performing an inverse RST. Also, compared to the number of multiplication operations N×N when using a normal inverse transformation matrix, the number of multiplication operations can be reduced by R / N (N×R) when using an inverse RST matrix. In addition, when applying an inverse RST, only R conversion coefficients need to be decoded. Compared to when a normal inverse transformation is applied, where N conversion coefficients need to be decoded, the total number of conversion coefficients for the target block decreases from N to R, thus increasing the decoding efficiency. In summary, according to S930, the (inverse) transformation efficiency and decoding efficiency of the decoding device 300 can be increased via the inverse RST.

[0274] Figure 10 is a control flowchart illustrating an inverse RST according to one embodiment described in this document.

[0275] The decoding device 300 receives information from the bitstream regarding the quantized conversion coefficients, intra-prediction mode, and conversion index (S1000).

[0276] The quantized conversion coefficients received from the bitstream are derived as conversion coefficients via inverse quantization, as shown in Figure 9, S920.

[0277] To apply the inverse RST to the inversely quantized transformation coefficients, the transformation set and transformation kernel matrix applied to the target block are derived (S1010).

[0278] For example, a transformation set is derived from the mapping relationship of the intra-prediction modes of the target block, and multiple intra-prediction modes can be mapped to a single transformation set. Furthermore, each of a single transformation set can contain multiple transformation kernel matrices. A transformation index can indicate any one of the multiple transformation kernel matrices; for example, if a single transformation set consists of two transformation kernel matrices, the transformation index can indicate any one of the two transformation kernel matrices.

[0279] In one embodiment, the syntax element of the conversion index can indicate whether the inverse RST is applied and one of the conversion kernel matrices included in the conversion set. If the conversion set includes two conversion kernel matrices, there are three possible values ​​for the syntax element of the conversion index.

[0280] That is, in one embodiment, the value of the syntax element for a transformation index may include 0, indicating that the inverse RST is not applied to the target block; 1, indicating the first transformation kernel matrix among the transformation kernel matrices; and 2, indicating the second transformation kernel matrix among the transformation kernel matrices. Such information is received as syntax information, which is received as a binary-evolved bin string containing 0 and 1.

[0281] The transformation kernel matrix, as an example, can be applied to a specific area in the upper left of the target block, such as an 8x8 or 4x4 area, depending on the reduction or simplification size of the quadratic transformation. The size of the modified transformation coefficients output by applying the transformation kernel matrix, i.e., the number of transformation coefficients, can be derived based on the transformation index, the intra-prediction mode, and the size of the target block to which the quadratic transformation is applied.

[0282] For example, when an inverse quadratic transformation is applied to the transformation coefficients of a region of a target block, i.e., an 8x8 region or a 4x4 region, the inverse quadratic transformation can be applied to only a portion of the transformation coefficients contained within the 8x8 region or 4x4 region. If only 48 transformation coefficients from the 8x8 region are input for the inverse quadratic transformation, the 64xm transformation kernel matrix applied to the 8x8 region can be further reduced to a 48xm transformation kernel matrix.

[0283] For example, m is 16, and the 48x16 transformation kernel matrix is ​​a transformation kernel matrix based on Table 14, i.e., a matrix transposed to the matrix in Table 14. If there are four transformation sets, and each transformation set contains two transformation kernel matrices, the transformation index, which indicates whether the inverse quadratic transformation is applied and which of the transformation kernel matrices included in the transformation set is used, can be 0, 1, or 2. If the transformation index is 0, it indicates that the inverse quadratic transformation is not applied, so if there are four transformation sets, all eight transformation kernel matrices can be used for the inverse quadratic transformation.

[0284] As shown in Equation 7, the one-dimensional array of transformation coefficients derived through inverse quantization can be derived as modified transformation coefficients having a two-dimensional array through matrix operations with the transformation kernel matrix.

[0285] The inverse transformation unit 322 in this embodiment can derive modified transformation coefficients for the upper left 4x4 region, the upper right 4x4 region, and the lower left 4x4 region of the 8x8 region of the target block by applying a transformation kernel matrix to the transformation coefficients of the upper left 4x4 region of the 8x8 region of the target block (S1020).

[0286] For example, when performing matrix operations between the transformation coefficients of the upper left 4x4 region of an 8x8 area and the transformation kernel matrix, the transformation coefficients of the upper left 4x4 region of the 8x8 area are arranged in a one-dimensional array according to the forward diagonal scanning order as shown in Table 16. After matrix operations with the transformation kernel matrix, the transformation coefficients of the one-dimensional array can be arranged in two dimensions in the upper left 4x4 region, the upper right 4x4 region, and the lower left 4x4 region of the 8x8 area according to either the row-major or column-major order, as shown in Table 15 or Table 17, depending on the intra-prediction mode applied to the target block. That is, an inverse quadratic transformation can be applied to the 16 transformation coefficients of the upper left 4x4 region of the 8x8 area, and 48 modified transformation coefficients can be derived in the upper left 4x4 region, the upper right 4x4 region, and the lower left 4x4 region of the 8x8 area through operations with the transformation kernel matrix.

[0287] If the intra-prediction mode that can be applied to the target block is one of the 65 directional modes, and the intra-prediction modes are symmetrical with respect to intra-prediction mode 34 in the upper left diagonal direction, and the intra-prediction mode applied to the target block is one of modes 2 through 34 in the left direction relative to intra-prediction mode 34, then the modified conversion coefficients can be arranged in a two-dimensional array according to row-first direction order.

[0288] If the intra-prediction mode applied to the target block is intra-prediction mode 35 through 66, which are to the right of intra-prediction mode 34, the modified conversion coefficients can be arranged in a two-dimensional array according to the column-first direction order.

[0289] Furthermore, if the intra-prediction mode applied to the target block is planar mode or DC mode, the modified conversion coefficients can be arranged in a two-dimensional array according to row-first direction order.

[0290] The inverse transform unit 322 can apply the inverse RST to generate modified transformation coefficients in an 8x8 or 4x4 region in a 2D block, and subsequently, an inverse linear transform is applied to the modified transformation coefficients of the 2D block thus generated.

[0291] Figure 11 is a flowchart illustrating the operation of a video encoding device according to one embodiment described in this document.

[0292] Each step disclosed in Figure 11 can be performed by the encoding device 200 disclosed in Figure 2. More specifically, S1110 can be performed by the prediction unit 220 disclosed in Figure 2, S1120 can be performed by the subtraction unit 231 disclosed in Figure 2, S1130 and S1140 can be performed by the conversion unit 232 disclosed in Figure 2, and S1150 can be performed by the quantization unit 233 and entropy encoding unit 240 disclosed in Figure 2. In addition, the operation by S1110 to S1150 is based in part on the content described above in Figures 4 to 8. Therefore, specific details that overlap with the content described above in Figures 2 and 4 to 8 will be omitted or simplified in explanation.

[0293] The encoding device 200 according to one embodiment can derive predicted samples based on the intra-prediction mode applied to the target block (S1110).

[0294] The encoding device 200 according to one embodiment can derive a residual sample for the target block (S1120).

[0295] An encoding device 200 according to one embodiment can derive conversion coefficients for the target block based on a linear conversion of residual samples (S1130). The linear conversion can be performed via a plurality of conversion kernels, in which case the conversion kernel can be selected based on the intra-prediction mode.

[0296] The decoding device 300 can perform a quadratic transformation, specifically NSST, on the transformation coefficients for the target block. In this case, NSST can be performed based on a simplified transformation (RST) or not based on an RST. When NSST is performed based on an RST, it can correspond to the operation in S1140.

[0297] An encoding device 200 according to one embodiment can derive modified conversion coefficients for a target block based on an RST for conversion coefficients (S1140). In one example, the RST can be performed based on a simplified conversion matrix or a conversion kernel matrix, the simplified conversion matrix being a non-square matrix with fewer rows than columns.

[0298] In one embodiment, S1140 may include the steps of determining whether the conditions for applying RST are met, generating and encoding a transformation index based on the determination, selecting a transformation kernel matrix, and, if the conditions for applying RST are met, applying RST to the residual sample based on the selected transformation kernel matrix and / or simplification factor. In this case, the size of the simplification transformation kernel matrix may be determined based on the simplification factor.

[0299] An encoding device 200 according to one embodiment can perform quantization based on the modified transformation coefficients for the target block to derive quantized transformation coefficients, and encode information regarding the quantized transformation coefficients (S1150).

[0300] More specifically, the encoding device 200 can generate information about quantized transformation coefficients and encode the generated information about quantized transformation coefficients.

[0301] In one example, the information regarding the quantized transformation coefficients may include at least one of the following: information on whether RST is applied, information on the simplification factor, information on the minimum transformation size to which RST is applied, and information on the maximum transformation size to which RST is applied.

[0302] Referring to S1140, it can be confirmed that the conversion coefficients for the target block are derived based on the RST for the residual sample. Considering this from the perspective of the size of the conversion kernel matrix, the size of a normal conversion kernel matrix is ​​N×N, while the size of the simplified conversion matrix is ​​reduced to R×N. Therefore, when executing RST, memory usage can be reduced by an R / N ratio compared to when performing a normal conversion. Also, compared to the number of multiplication operations N×N when using a normal conversion kernel matrix, the number of multiplication operations can be reduced by an R / N ratio (R×N) when using a simplified conversion kernel matrix. Furthermore, when RST is applied, only R conversion coefficients are derived. Compared to when a normal conversion is applied and N conversion coefficients are derived, the total number of conversion coefficients for the target block decreases from N to R, thus reducing the amount of data that the encoding device 200 sends to the decoding device 300. In summary, according to S1140, the conversion efficiency and coding efficiency of the encoding device 200 can be increased via RST.

[0303] Figure 12 is a control flowchart illustrating an RST according to one embodiment described in this document.

[0304] First, the encoding device 200 can determine the conversion set based on the mapping relationship by the intra-prediction mode applied to the target block (S1200).

[0305] Subsequently, the conversion unit 232 can select one of the multiple conversion kernel matrices included in the conversion set (S1210).

[0306] For example, a transformation set is derived from the mapping relationship of the intra-prediction modes of the target block, and multiple intra-prediction modes can be mapped to a single transformation set. Furthermore, each of a single transformation set can contain multiple transformation kernel matrices. If a single transformation set consists of two transformation kernel matrices, a transformation index pointing to either of the two transformation kernel matrices can be encoded and signaled to the decoding device.

[0307] When two transformation processes are applied to a residual sample, if the residual sample undergoes a linear transformation, it can be called a transformation coefficient, and if a quadratic transformation such as RST is performed after the linear transformation, it can be called a modified transformation coefficient.

[0308] For example, when a quadratic transformation is applied to the transformation coefficients of a region within a target block, i.e., an 8x8 region or a 4x4 region, the quadratic transformation can be applied to only a portion of the transformation coefficients contained within the 8x8 region or 4x4 region. For instance, if a quadratic transformation is applied to only 48 of the transformation coefficients in an 8x8 region, the mx64 transformation kernel matrix applied to the 8x8 region can be reduced to an mx48 transformation kernel matrix.

[0309] For example, m is 16, and the 16x48 transformation kernel matrix is ​​shown in Table 14. If there are four transformation sets, and each transformation set contains two transformation kernel matrices, the transformation index, which indicates whether the inverse quadratic transformation is applied and which of the transformation kernel matrices included in the transformation set is used, can be 0, 1, or 2. If the transformation index is 0, it indicates that the quadratic transformation is not applied, so if there are four transformation sets, all eight transformation kernel matrices can be used for the quadratic transformation.

[0310] When the transformation unit 232 performs RST using the transformation kernel matrix on the transformation coefficients, it can arrange the transformation coefficients of the two-dimensional array that have undergone a linear transformation into a one-dimensional array in either a row-first or column-first order based on the intra-prediction mode applied to the target block. Specifically, as an example, the transformation unit 232 can derive modified transformation coefficients corresponding to the upper left 4x4 region of the 8x8 region of the target block by applying the transformation kernel matrix to the transformation coefficients of the upper left 4x4 region, the upper right 4x4 region, and the lower left 4x4 region of the 8x8 region (S1220).

[0311] The transformation kernel matrix can be applied to a specific area in the upper left of the target block, such as an 8x8 area, a 4x4 area, or a portion of an 8x8 area, depending on the size of the quadratic transformation (reduction or simplification). The magnitude of the modified transformation coefficients output by applying the transformation kernel matrix, i.e., the number of modified transformation coefficients, can be derived based on the size of the transformation kernel matrix, the intra-prediction mode, and the size of the target block to which the quadratic transformation is applied.

[0312] As shown in Equation 5, the two-dimensional transformation coefficients must be arranged in one dimension for matrix operations with the transformation kernel matrix, and a number of modified transformation coefficients less than the number of transformation coefficients can be derived through operations such as Equation 6.

[0313] In other words, the transformation coefficients of a two-dimensional array in a specific region are read into one dimension according to a fixed directional order and derived as modified transformation coefficients through matrix operations with the transformation kernel matrix.

[0314] For example, when performing matrix operations on the transformation coefficients and transformation kernel matrix of the upper left 4x4 region of an 8x8 area, the 48 transformation coefficients of the upper left 4x4 region, the upper right 4x4 region, and the lower left 4x4 region of the 8x8 area can be arranged in a one-dimensional array in either a row-major or column-major order, as shown in Table 15 or Table 17, depending on the intra-prediction mode applied to the target block, and the derived 16 modified transformation coefficients can be arranged in the diagonal scanning direction of the upper left 4x4 region of the 8x8 area, as shown in Table 16.

[0315] If the intra-prediction mode that can be applied to the target block is one of the 65 directional modes, and the intra-prediction mode is symmetrical with respect to intra-prediction mode 34 in the upper left diagonal direction, and the intra-prediction mode applied to the target block is intra-prediction mode 2 to 34 in the left direction relative to intra-prediction mode 34, then the conversion coefficients for the upper left 4x4 region, the upper right 4x4 region, and the lower left 4x4 region of the 8x8 region can be arranged in a one-dimensional array according to row-first direction order as shown in Table 15.

[0316] If the intra-prediction mode applied to the target block is intra-prediction mode 35 to 66, which are to the right of intra-prediction mode 34, the conversion coefficients for the upper left 4x4 region, the upper right 4x4 region, and the lower left 4x4 region of the 8x8 area can be arranged in a one-dimensional array according to the column-first direction order as shown in Table 17.

[0317] Furthermore, if the intra-prediction mode applied to the target block is planar mode or DC mode, the conversion coefficients for the upper left 4x4 region, the upper right 4x4 region, and the lower left 4x4 region of the 8x8 region can be arranged in a one-dimensional array according to row-first direction order.

[0318] When RST is executed in this manner, the information for RST can be encoded by the entropy encoding unit 240.

[0319] First, the entropy encoding unit 240 derives a syntax element value for a transformation index that points to one of the transformation kernel matrices included in the transformation set, then binaryizes the derived syntax element value for the transformation index, and then encodes the bin of the syntax element bin string based on context information for the bin string of the transformation index, i.e., the context model.

[0320] The encoded syntax element bin string can be output in bitstream form to the decoding device 300 or externally.

[0321] In the embodiments described above, the method is explained based on a flowchart in a series of steps or blocks, but this document is not limited to the order of the steps, and some steps may occur with other steps, in a different order, or simultaneously. Furthermore, those skilled in the art will understand that the steps shown in the flowchart are not exclusive, other steps may be included, or one or more steps in the flowchart may be deleted without affecting the scope of this document.

[0322] The method described in this document above can be implemented in software form, and the encoding and / or decoding devices described in this document can be included in devices that perform video processing, such as TVs, computers, smartphones, set-top boxes, and display devices.

[0323] In this document, when embodiments are embodied in software, the methods described above can be embodied in modules (processes, functions, etc.) that perform the functions described above. These modules are stored in memory and can be executed by a processor. The memory may be internal or external to the processor and may be connected to the processor by a variety of well-known means. The processor may include an ASIC (application-specific integrated circuit), other chipsets, logic circuits, and / or data processing devices. The memory may include ROM (read-only memory), RAM (random access memory), flash memory, memory cards, storage media, and / or other storage devices. In other words, the embodiments described in this document can be embodied and executed on a processor, microprocessor, controller, or chip. For example, the functional units shown in each drawing can be embodied and executed on a computer, processor, microprocessor, controller, or chip.

[0324] Furthermore, decoding and encoding devices to which this document applies may include multimedia broadcasting transceivers, mobile communication terminals, home cinema video equipment, digital cinema video equipment, surveillance cameras, video interaction equipment, real-time communication equipment such as video communications, mobile streaming equipment, storage media, camcorders, video-on-demand (VoD) service providers, over-the-top (OTT) video equipment, internet streaming service providers, 3D video equipment, image-phone video equipment, and medical video equipment, and can be used to process video signals or data signals. For example, over-the-top (OTT) video equipment may include game consoles, Blu-ray players, internet-connected TVs, home theater systems, smartphones, tablet PCs, and digital video recorders (DVRs).

[0325] Furthermore, the processing methods to which this document applies can be produced in the form of programs executed on a computer and stored on a computer-readable recording medium. Multimedia data having the data structure described in this document can also be stored on a computer-readable recording medium. The computer-readable recording medium includes all types of storage devices and distributed storage devices that store data readable by a computer. The computer-readable recording medium can include, for example, Blu-ray discs (BDs), general-purpose serial buses (USB), ROMs, PROMs, EPROMs, EEPROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, and optical data storage devices. The computer-readable recording medium also includes media embodied in the form of carrier waves (e.g., transmission over the Internet). Furthermore, bitstreams generated by encoding methods can be stored on a computer-readable recording medium or transmitted over a wireless network. The embodiments of this document can also be embodied in computer program products using program code, and the program code can be executed on a computer according to the embodiments of this document. The program code can be stored on a computer-readable carrier.

[0326] Figure 13 illustrates an example of a content streaming system structure to which this document applies.

[0327] Furthermore, the content streaming systems to which this document applies may broadly include encoding servers, streaming servers, web servers, media storage facilities, user equipment, and multimedia input devices.

[0328] The encoding server is responsible for compressing content input from multimedia input devices such as smartphones, cameras, and camcorders into digital data to generate a bitstream, and transmitting this bitstream to the streaming server. In other cases, if a multimedia input device such as a smartphone, camera, or camcorder directly generates the bitstream, the encoding server may be omitted. The bitstream can be generated by the encoding method or bitstream generation method to which this document applies, and the streaming server may temporarily store the bitstream during the process of transmitting or receiving it.

[0329] The streaming server transmits multimedia data to user devices based on user requests via a web server, and the web server acts as an intermediary to inform users about available services. When a user requests a desired service from the web server, the web server transmits this to the streaming server, and the streaming server transmits multimedia data to the user. In this case, the content streaming system may include a separate control server, in which case the control server controls the commands and responses between the devices within the content streaming system.

[0330] The streaming server can receive content from a media storage and / or encoding server. For example, if it starts receiving content from the encoding server, it can receive the content in real time. In this case, in order to provide a smooth streaming service, the streaming server can store the bitstream for a certain period of time.

[0331] Examples of user devices include mobile phones, smartphones, laptop computers, digital broadcasting terminals, PDAs (personal digital assistants), PMPs (portable multimedia players), navigation systems, slate PCs, tablet PCs, ultrabooks, wearable devices (such as smartwatches, smart glasses, HMDs (head-mounted displays)), digital TVs, desktop computers, and digital signage. Each server within the content streaming system can be operated as a distributed server, in which case the data received by each server can be processed in a distributed manner.

Claims

1. In a video decoding method performed by a decoding device, The steps include receiving a bitstream, The steps include obtaining a two-dimensional array of conversion coefficients based on the received bitstream, The steps include: deriving multiple first transformation coefficients from the upper left 4x4 region of the two-dimensional array of transformation coefficients; The steps include: deriving a plurality of second transformation coefficients based on the inverse quadratic transformation of the plurality of first transformation coefficients; A step of deriving a two-dimensional array of values ​​based on the plurality of second transformation coefficients, wherein the plurality of second transformation coefficients are mapped to the upper left 8x8 region of the two-dimensional array of values, excluding the lower right 4x4 region within the upper left 8x8 region. The steps include: deriving a resistive sample array based on the inverse linear transformation of the two-dimensional array of values; The steps include generating a reconstructed picture based on the sequence of the residual sample, The step of deriving the plurality of second transformation coefficients includes a step of performing a matrix operation between the transformation kernel matrix and the plurality of first transformation coefficients, wherein the plurality of first transformation coefficients are arranged in a forward diagonal scanning order of the upper left 4x4 region of the two-dimensional array of transformation coefficients, The matrix operation between the transformation kernel matrix and the plurality of first transformation coefficients in the upper left 4x4 region is (48x16 matrix) * (16x1 transformation coefficient vector), A method wherein the plurality of second transformation coefficients are arranged in two dimensions in the two-dimensional array of values ​​according to the row-first direction corresponding to the applied intra-prediction mode.

2. In a video encoding method performed by a video encoding device, The steps include: deriving prediction samples based on the intra prediction mode applied to the target block of the picture; The steps include: deriving the sequence of the residual sample based on the predicted sample; The steps include: deriving a two-dimensional array of values ​​based on a linear transformation of the array of residual samples; A step of deriving a plurality of second transformation coefficients from the two-dimensional array of values, wherein the plurality of second transformation coefficients are derived from the upper left 8x8 region of the two-dimensional array of values, excluding the lower right 4x4 region within the upper left 8x8 region. The steps include: deriving a plurality of first transformation coefficients based on the quadratic transformation of the plurality of second transformation coefficients; A step of deriving a two-dimensional array of transformation coefficients based on the plurality of first transformation coefficients, wherein the plurality of first transformation coefficients are mapped to the upper left 4x4 region of the two-dimensional array of transformation coefficients, The step of outputting a bitstream containing information derived based on the two-dimensional array of conversion coefficients, The step of deriving the plurality of first transformation coefficients includes the step of performing a matrix operation between the transformation kernel matrix and the plurality of second transformation coefficients, wherein the plurality of second transformation coefficients are arranged for the matrix operation, The matrix operation between the transformation kernel matrix and the plurality of second transformation coefficients in the upper left 4x4 region, upper right 4x4 region and lower left 4x4 region of the upper left 8x8 region is (16x48 matrix) * (48x1 transformation coefficient vector), The plurality of second transformation coefficients are derived from the two-dimensional array of values ​​according to the row-first direction corresponding to the applied intra-prediction mode, in a method.

3. Regarding methods for transmitting video data, A step of generating a bitstream relating to the video, wherein the bitstream is: The steps include: deriving prediction samples based on the intra prediction mode applied to the target block of the picture; The steps include: deriving the sequence of the residual sample based on the predicted sample; The steps include: deriving a two-dimensional array of values ​​based on a linear transformation of the array of residual samples; A step of deriving a plurality of second transformation coefficients from the two-dimensional array of values, wherein the plurality of second transformation coefficients are derived from the upper left 8x8 region of the two-dimensional array of values, excluding the lower right 4x4 region within the upper left 8x8 region. The steps include: deriving a plurality of first transformation coefficients based on the quadratic transformation of the plurality of second transformation coefficients; A step of deriving a two-dimensional array of transformation coefficients based on the plurality of first transformation coefficients, wherein the plurality of first transformation coefficients are mapped to the upper left 4x4 region of the two-dimensional array of transformation coefficients, A step of outputting the bitstream containing information derived based on the two-dimensional array of conversion coefficients, and a step of generating based on the above, The step of transmitting the data, which includes the bitstream, The step of deriving the plurality of first transformation coefficients includes the step of performing a matrix operation between the transformation kernel matrix and the plurality of second transformation coefficients, wherein the plurality of second transformation coefficients are arranged for the matrix operation, The matrix operation between the transformation kernel matrix and the plurality of second transformation coefficients in the upper left 4x4 region, upper right 4x4 region and lower left 4x4 region of the upper left 8x8 region is (16x48 matrix) * (48x1 transformation coefficient vector), The plurality of second transformation coefficients are derived from the two-dimensional array of values ​​according to the row-first direction corresponding to the applied intra-prediction mode, in a method.

Images