Signaling of information indicating transform kernel set in image coding

Abstract

The invention relates to signaling of information indicating a transform kernel set in image coding. An image decoding method according to the present document includes a step of generating a residual sample of a current block based on residual information including a multiple transform selection (MTS) index and information on transform coefficients, generating the residual sample from the transform coefficients according to the information on the transform coefficients by using a transform kernel set, and decoding the residual sample based on the residual sample. The transform kernel set is determined by the MTS index from transform kernel set candidates, at least one bin of a bin string of the MTS index is derived based on context coding, and the context coding is performed based on a value of the context index with respect to the MTS index.

Description

[0001] This application is a divisional application of a patent application with application number 202080052291.2 (PCT / KR2020 / 007597), which was submitted to the China Patent Office on January 19, 2022, with an international application date of June 11, 2020, and the invention name is “Signaling of information indicating a transform kernel set in image compilation”. Technical Field

[0002] The present disclosure relates to image coding techniques, and more particularly, to signaling of information indicating a transform kernel set in image coding. Background Art

[0003] Recently, there is an increasing demand for high-resolution, high-quality images / videos such as 4K or 8K ultra-high definition (UHD) images / videos in various fields. As the image / video resolution or quality becomes higher, relatively more information or bits are transmitted compared to conventional image / video data. Therefore, if the image / video data is transmitted via a medium such as an existing wired / wireless broadband line or is stored in a conventional storage medium, the cost of transmission and storage is likely to increase.

[0004] Furthermore, there is growing interest and demand for virtual reality (VR) and artificial reality (AR) content and immersive media such as holograms; and there is also growing broadcasting of images / videos that exhibit image / video characteristics different from actual images / videos (e.g., game images / videos).

[0005] Therefore, highly efficient image / video compression technology is required to effectively compress and transmit, store, or play high-resolution, high-quality images / videos showing various characteristics as described above. Summary of the invention

[0006] Technical Solution

[0007] According to an embodiment of this document, a method and apparatus for enhancing image / video coding efficiency are provided.

[0008] According to an embodiment of this document, a method and apparatus for signaling information indicating a transform kernel set in image coding are provided.

[0009] According to an embodiment of this document, a method and apparatus for signaling MTS index information are provided.

[0010] According to an embodiment of this document, a method and apparatus for compiling information indicating a transform kernel set are provided.

[0011] According to an embodiment of this document, a method and apparatus for signaling information representing a transform kernel set to be applied to a current block among a plurality of transform kernel sets is provided.

[0012] According to an embodiment of this document, a method and apparatus for context compilation or bypass compilation of bins relative to a bin string indexed by an MTS is provided.

[0013] According to an embodiment of this document, a video / image decoding method performed by a decoding device is provided.

[0014] According to an embodiment of this document, a decoding device for performing video / image decoding is provided.

[0015] According to an embodiment of this document, a video / image encoding method performed by an encoding device is provided.

[0016] According to an embodiment of this document, an encoding apparatus for performing video / image encoding is provided.

[0017] According to an embodiment of this document, there is provided a computer-readable digital storage medium storing encoded video / image information generated according to a video / image encoding method disclosed in at least one of the embodiments of this document.

[0018] According to an embodiment of this document, there is provided a computer-readable digital storage medium storing encoded information or encoded video / image information that causes a decoding device to perform a video / image decoding method disclosed in at least one of the embodiments of this document.

[0019] Beneficial Effects

[0020] According to this document, the overall image / video compression efficiency can be enhanced.

[0021] According to this document, information indicating a transform kernel set may be efficiently signaled.

[0022] According to this document, the complexity of the compilation system can be reduced by efficiently compiling information indicating a transformation core set.

[0023] The effects that can be obtained through the detailed examples of this document are not limited to the effects listed above. For example, there may be various technical effects that can be understood or derived from this document by ordinary technicians in the relevant field. Therefore, the detailed effects of this document are not limited to the effects clearly stated in this document, but may include various effects that can be understood or derived from the technical features of this document. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 An example of a video / image coding system to which this document is applicable is schematically illustrated.

[0025] Figure 2 is a diagram schematically illustrating the configuration of a video / image encoding device to which this document is applied.

[0026] Figure 3 is a diagram schematically illustrating the configuration of a video / image decoding device to which this document is applied.

[0027] Figure 4 The intra directional modes in 65 prediction directions are exemplarily illustrated.

[0028] Figure 5 Context-Adaptive Binary Arithmetic Coding (CABAC) for encoding syntax elements is exemplarily illustrated.

[0029] Figure 6 and Figure 7 A multiple transformation scheme according to an embodiment of this document is schematically shown.

[0030] Figure 8 and Fig. 9 is a diagram for explaining RST according to an embodiment of this document.

[0031] Fig.10 and Fig.11 is a diagram for explaining RST according to an embodiment of this document.

[0032] Fig.12 and Fig.13 An example of a video / image decoding method and related components according to embodiment(s) of this document is schematically illustrated.

[0033] Fig.14 An example of a content streaming system to which the embodiments disclosed in this document may be applied is shown. DETAILED DESCRIPTION

[0034] The present disclosure can be modified in various forms, and its specific embodiments will be described and illustrated in the accompanying drawings. However, these embodiments are not intended to limit the present disclosure. The terms used in the following description are only used to describe specific embodiments and are not intended to limit the present disclosure. Singular expressions include plural expressions as long as they are clearly interpreted differently. Terms such as "including" and "having" are intended to indicate the presence of features, quantities, steps, operations, elements, parts, or combinations thereof used in the following description, so it should be understood that the possibility of the presence or addition of one or more different features, quantities, steps, operations, elements, parts, or combinations thereof is not excluded.

[0035] In addition, the various configurations of the drawings described in this document are independent illustrations for the purpose of illustrating the functions of the features that are different from each other, and do not mean that the various configurations are implemented by different hardware or different software. For example, two or more configurations in the configuration can be combined to form one configuration, and one configuration can also be divided into multiple configurations. Without departing from the subject matter of this document, embodiments in which the configurations are combined and / or separated are included in the scope of the claims.

[0036] Hereinafter, an example of the present embodiment will be described in detail with reference to the accompanying drawings. In addition, throughout the drawings, like reference numerals are used to indicate like elements, and the same description of the like elements will be omitted.

[0037] In this document, video may refer to a series of images over time. A picture generally refers to a unit representing one image of a specific time frame, and a slice / tile refers to a unit that constitutes a part of a picture in terms of coding. A slice / tile may include one or more coding tree units (CTUs). A picture may consist of one or more slices / tiles. A picture may consist of one or more tile groups. A tile group may include one or more tiles.

[0038] A pixel or a picture element (pel) may refer to the smallest unit constituting a picture (or image). In addition, a "sample" may be used as a term corresponding to a pixel. A sample may generally represent a pixel or a pixel value, and may represent only a pixel / pixel value of a luminance component or only a pixel / pixel value of a chrominance component.

[0039] A unit may represent a basic unit of image processing. A unit may include at least one of a specific area of ​​a picture and information related to the area. A unit may include a luminance block and two chrominance (e.g., cb, cr) blocks. In some cases, a unit may be used interchangeably with terms such as a block or an area. In general, an M×N block may include M columns and N rows of samples (or sample arrays) or a set (or array) of transform coefficients. Alternatively, a sample may mean a pixel value in the spatial domain, and when such a pixel value is transformed to the frequency domain, it may mean a transform coefficient in the frequency domain.

[0040] In addition, brackets used in the present specification may mean "for example". Specifically, in the case of expressing "prediction (intra-frame prediction)", it may indicate that "intra-frame prediction" is proposed as an example of "prediction". In other words, the term "prediction" in the present specification is not limited to "intra-frame prediction", and it may indicate that "intra-frame prediction" is proposed as an example of "prediction". In addition, even in the case of expressing "prediction (ie, intra-frame prediction)", it may indicate that "intra-frame prediction" is proposed as an example of "prediction".

[0041] This document relates to video / image coding. For example, the methods / embodiments disclosed in this document may relate to the Versatile Video Coding (VVC) standard (ITU-T Rec. H.266), the next generation video / image coding standard after VVC, or other video coding related standards (e.g., High Efficiency Video Coding (HEVC) standard (ITU-T Rec. H.265), Basic Video Coding (EVC) standard, AVS2 standard, etc.).

[0042] In this document, the terms " / " and "," should be interpreted as indicating "and / or". For example, the expression "A / B" may mean "A and / or B". In addition, "A, B" may mean "A and / or B". In addition, "A / B / C" may mean "at least one of A, B, and / or C". In addition, "A / B / C" may mean "at least one of A, B, and / or C".

[0043] In addition, in this document, the term "or" should be interpreted as indicating "and / or". For example, the expression "A or B" may include 1) only A, 2) only B, and / or 3) both A and B. In other words, the term "or" in this document should be interpreted as indicating "additionally or alternatively".

[0044] In the present specification, "at least one of A and B" may mean "only A", "only B", or "both A and B". In addition, in the present specification, the expression "at least one of A or B" or "at least one of A and / or B" may be interpreted as being the same as "at least one of A and B".

[0045] In addition, in the present specification, "at least one of A, B, and C" may mean "only A", "only B", "only C", or "any combination of A, B, and C". In addition, "at least one of A, B, or C" or "at least one of A, B and / or C" may mean "at least one of A, B, and C".

[0046] In this specification, technical features described separately in one drawing may be implemented separately or may be implemented simultaneously.

[0047] This document proposes various embodiments of video / image coding, and unless otherwise mentioned, the above embodiments may also be performed in combination with each other.

[0048] Figure 1 An example of a video / image coding system to which the disclosure of this document may be applied is illustrated.

[0049] Reference Figure 1The video / image compilation system may include a source device and a receiving device. The source device may send the encoded video / image information or data to the receiving device in the form of a file or stream transmission via a digital storage medium or a network.

[0050] The source device may include a video source, an encoding device, and a transmitter. The receiving device may include a receiver, a decoding device, and a renderer. The encoding device may be referred to as a video / image encoding device, and the decoding device may be referred to as a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display, and the display may be configured as a separate device or an external component.

[0051] The video source may acquire the video / image by a process of capturing, synthesizing or generating the video / image. The video source may include a video / image capturing device, and / or a video / image generating device. For example, the video / image capturing device may include one or more cameras, a video / image archive including previously captured videos / images, etc. For example, the video / image generating device may include a computer, a tablet computer, and a smart phone, and may generate the video / image (electronically). For example, a virtual video / image may be generated by a computer, etc. In this case, the video / image capturing process may be replaced by a process of generating the relevant data.

[0052] The encoding device may encode the input video / image. For compression and coding efficiency, the encoding device may perform a series of processes such as prediction, transformation, and quantization. The encoded data (encoded video / image information) may be output in the form of a bitstream.

[0053] The transmitter may send the encoded image / image information or data output in the form of a bit stream to a receiver of a receiving device in the form of a file or stream via a digital storage medium or a network. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitter may include an element for generating a media file in a predetermined file format, and may include an element for transmission via a broadcast / communication network. The receiver may receive / extract a bit stream and send the received bit stream to a decoding device.

[0054] The decoding device may decode a video / image by performing a series of processes corresponding to the operations of the encoding device, such as dequantization, inverse transformation, and prediction.

[0055] The renderer may render the decoded video / image. The rendered video / image may be displayed through a display.

[0056] Figure 2is a diagram schematically illustrating a configuration of a video / image encoding device to which the disclosure of this document can be applied. Hereinafter, the so-called video encoding device may include an image encoding device.

[0057] Reference Figure 2 , the encoding device 200 includes an image segmenter 210, a predictor 220, a residual processor 230 and an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 may include an inter-frame predictor 221 and an intra-frame predictor 222. The residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234, and an inverse transformer 235. The residual processor 230 may also include a subtractor 231. The adder 250 may be referred to as a reconstructor or a reconstructed block generator. According to an embodiment, the image segmenter 210, the predictor 220, the residual processor 230, the entropy encoder 240, the adder 250, and the filter 260 may be configured by at least one hardware component (e.g., an encoder chipset or processor). In addition, the memory 270 may include a decoded picture buffer (DPB), or may be configured by a digital storage medium. The hardware component may also include the memory 270 as an internal / external component.

[0058] The image divider 210 may divide the input image (or picture or frame) input to the encoding device 200 into one or more processors. For example, the processor may be referred to as a coding unit (CU). In this case, the coding unit may be recursively divided from a coding tree unit (CTU) or a maximum coding unit (LCU) according to a quadtree binary tree ternary tree (QTBTTT) structure. For example, one coding unit may be divided into a plurality of coding units with a deeper depth based on a quadtree structure, a binary tree structure, and / or a ternary structure. In this case, for example, a quadtree structure may be applied first, and a binary tree structure and / or a ternary structure may be applied later. Alternatively, a binary tree structure may be applied first. The coding process according to the present disclosure may be performed based on a final coding unit that is no longer divided. In this case, the maximum coding unit may be used as the final coding unit based on coding efficiency, etc. according to image characteristics, or if necessary, the coding unit may be recursively divided into coding units with a deeper depth and a coding unit with an optimal size may be used as the final coding unit. Here, the coding process may include a process of prediction, transformation, and reconstruction (to be described later). As another example, the processor may further include a prediction unit (PU) or a transform unit (TU). In this case, the prediction unit and the transform unit may be split or partitioned from the above-mentioned final coding unit. The prediction unit may be a unit for sample prediction, and the transform unit may be a unit for deriving a transform coefficient and / or a unit for deriving a residual signal from a transform coefficient.

[0059] In some cases, unit can be used interchangeably with terms such as block or region. In general, an M×N block can represent a set of samples or transform coefficients consisting of M columns and N rows. Samples can generally represent pixels or pixel values, and can represent only the pixel / pixel value of the luma component or only the pixel / pixel value of the chroma component. Sample can be used as a term corresponding to one picture (or image) of a pixel or pixel.

[0060] The subtractor 231 can generate a residual signal (residual block, residual sample or residual sample array) by subtracting the prediction signal (prediction block, prediction sample or prediction sample array) output from the predictor 220 from the input image signal (original block, original sample or original sample array), and the generated residual signal is sent to the transformer 232. The predictor 220 can perform prediction on the processing target block (hereinafter, referred to as "current block") and generate a prediction block including the prediction sample of the current block. The predictor 220 can determine whether to apply intra prediction or inter prediction in units of the current block or CU. As described later in the description of each prediction mode, the predictor can generate various types of information related to the prediction, such as prediction mode information, and transmit the generated information to the entropy encoder 240. Information about the prediction can be encoded in the entropy encoder 240 and output in the form of a bitstream.

[0061] The intra-frame predictor 222 may predict the current block with reference to samples in the current picture. Depending on the prediction mode, the referenced samples may be located near the current block or may be spaced apart. In intra-frame prediction, the prediction mode may include multiple non-directional modes and multiple directional modes. For example, the non-directional mode may include a DC mode and a plane mode. For example, depending on the level of detail of the prediction direction, the directional mode may include 33 directional prediction modes or 65 directional prediction modes. However, this is only an example, and more or fewer directional prediction modes may be used depending on the settings. The intra-frame predictor 222 may use the prediction mode applied to the neighboring blocks to determine the prediction mode applied to the current block.

[0062] The inter-frame predictor 221 may derive a prediction block of the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. Here, in order to reduce the amount of motion information sent in the inter-frame prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter-frame prediction, the neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different. The temporal neighboring block may be referred to as a collocated reference block, a collocated CU (colCU), etc., and the reference picture including the temporal neighboring block may be referred to as a collocated picture (colPic). For example, the inter-frame predictor 221 may configure a motion information candidate list based on the neighboring blocks and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Inter-frame prediction may be performed based on various prediction modes. For example, in the case of skip mode and merge mode, the inter-frame predictor 221 may use motion information of a neighboring block as motion information of a current block. In skip mode, unlike merge mode, a residual signal may not be transmitted. In the case of motion vector prediction (MVP) mode, a motion vector of a neighboring block may be used as a motion vector predictor, and a motion vector of a current block may be indicated by signaling a motion vector difference.

[0063] The predictor 220 can generate a prediction signal based on various prediction methods described below. For example, the predictor can not only apply intra prediction or inter prediction to predict a block, but also apply both intra prediction and inter prediction at the same time. This can be called combined inter and intra prediction (CIIP). In addition, the predictor can perform intra block copy (IBC) for prediction of the block. Intra block copy can be used for content image / motion image coding of games, etc., for example, screen content coding (SCC). IBC basically performs prediction in the current picture, but can be performed similarly to inter prediction because a reference block is derived in the current picture. That is, IBC can use at least one of the inter prediction techniques described in this document.

[0064] The prediction signal generated by the inter-frame predictor 221 and / or the intra-frame predictor 222 can be used to generate a reconstruction signal or to generate a residual signal. The transformer 232 can generate a transform coefficient by applying a transform technique to the residual signal. For example, the transform technique may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a graph-based transform (GBT), or a conditional nonlinear transform (CNT). Here, GBT means a transform obtained from a graph when the relationship information between pixels is represented by a graph. CNT refers to a transform obtained based on a prediction signal generated using all previously reconstructed pixels. In addition, the transform process can be applied to square pixel blocks of the same size, or can be applied to blocks of variable size other than square.

[0065] The quantizer 233 may quantize the transform coefficients and send them to the entropy encoder 240, and the entropy encoder 240 may encode the quantized signal (information about the quantized transform coefficients) and output a bitstream. Information about the quantized transform coefficients may be referred to as residual information. The quantizer 233 may rearrange the quantized transform coefficients of the block type into a one-dimensional vector form based on the coefficient scanning order, and generate information about the quantized transform coefficients based on the quantized transform coefficients in the one-dimensional vector form. The entropy encoder 240 may perform various encoding methods such as (for example) exponential Golomb, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), etc. The entropy encoder 240 may encode the information necessary for video / image reconstruction together with or separately from the quantized transform coefficients (for example, the value of the syntax element, etc.). The encoded information (for example, the encoded video / image information) may be transmitted or stored in units of a network abstraction layer (NAL) in the form of a bitstream. The video / image information may also include information related to various parameter sets such as an adaptive parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may also include conventional constraint information. In this document, the information and / or syntax elements to be signaled / sent, which will be described later, may be encoded by the above-mentioned encoding process and included in the bitstream. The bitstream may be sent over a network or may be stored in a digital storage medium. Here, the network may include a broadcast network and / or a communication network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmitter (not shown) that transmits a signal output from the entropy encoder 240 or a storage unit (not shown) that stores the signal may be configured as an internal / external element of the encoding device 200, and alternatively, the transmitter may be included in the entropy encoder 240.

[0066] The quantized transform coefficients output from the quantizer 233 can be used to generate a prediction signal. For example, a residual signal (residual block or residual sample) can be reconstructed by applying dequantization and inverse transformation to the quantized transform coefficients via the dequantizer 234 and the inverse transformer 235. The adder 250 adds the reconstructed residual signal to the prediction signal output from the predictor 220 to generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample, or reconstructed sample array). If there is no residual of the processing target block (such as, when the jump mode is applied), the prediction block can be used as a reconstructed block. The generated reconstructed signal can be used for intra-frame prediction of the next processing target block in the current picture, and as described below, can be used for inter-frame prediction of the next picture performed by filtering.

[0067] Furthermore, luma mapping with chroma scaling (LMCS) may be applied during the picture encoding and / or reconstruction process.

[0068] The filter 260 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filter 260 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and store the modified reconstructed picture in the memory 270, specifically, in the DPB of the memory 270. Various filtering methods may include, for example, deblocking filtering, sample adaptive offset (SAO), adaptive loop filter, bilateral filter, etc. The filter 260 can generate various types of information related to filtering and transmit the generated information to the entropy encoder 290, as described in the subsequent description of each filtering method. The information related to filtering can be encoded by the entropy encoder 290 and output in the form of a bitstream.

[0069] The corrected reconstructed picture transmitted to the memory 270 may be used as a reference picture in the inter predictor 221. When inter prediction is applied by the encoding device, prediction mismatch between the encoding device 200 and the decoding device may be avoided, and encoding efficiency may be improved.

[0070] The DPB of the memory 270 may store the modified reconstructed picture for use as a reference picture in the inter-frame predictor 221. The memory 270 may store the motion information of the block from which the motion information in the current picture is derived (or encoded) and / or the motion information of the block in the reconstructed picture. The stored motion information may be transmitted to the inter-frame predictor 221 to be used as the motion information of the spatial neighboring block or the motion information of the temporal neighboring block. The memory 270 may store the reconstructed samples of the reconstructed block in the current picture and may transmit the reconstructed samples to the intra-frame predictor 222.

[0071] Figure 3 is a diagram schematically illustrating a configuration of a video / image decoding device to which the disclosure of this document can be applied.

[0072] Reference Figure 3 , the decoding device 300 may include an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 may include an inter-frame predictor 331 and an intra-frame predictor 332. The residual processor 320 may include a dequantizer 321 and an inverse transformer 321. According to an embodiment, the entropy decoding 310, the residual processor 320, the predictor 330, the adder 340, and the filter 350 may be configured by a hardware component (e.g., a decoder chipset or a processor). In addition, the memory 360 may include a decoded picture buffer (DPB) or may be configured by a digital storage medium. The hardware component may also include the memory 360 as an internal / external component.

[0073] When a bit stream including video / image information is input, the decoding apparatus 300 can reconstruct the bit stream corresponding to the bit stream in the video / image format. Figure 2 The image corresponding to the processing of the video / image information in the encoding device. For example, the decoding device 300 may derive the unit / block based on the block segmentation related information obtained from the bitstream. The decoding device 300 may use a processor applied in the encoding device to perform decoding. Therefore, for example, the decoding processor may be a coding unit, and the coding unit may be divided from the coding tree unit or the maximum coding unit according to a quadtree structure, a binary tree structure and / or a ternary tree structure. One or more transform units may be derived from the coding unit. The reconstructed image signal decoded and output by the decoding device 300 may be reproduced by a reproduction device.

[0074] The decoding device 300 may receive the Figure 2The received signal can be decoded by the entropy decoder 310. For example, the entropy decoder 310 can parse the bitstream to derive information (e.g., video / image information) necessary for image reconstruction (or picture reconstruction). The video / image information may also include information related to various parameter sets such as an adaptive parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may also include conventional constraint information. The decoding device may also decode the picture based on the information about the parameter set and / or the conventional constraint information. The information and / or syntax elements that are subsequently signaled / received and described in this document may be decoded and obtained from the bitstream through a decoding process. For example, the entropy decoder 310 may decode the information in the bitstream based on a coding method such as exponential Golomb coding, CABAC, or CAVLC, and output syntax elements required for image reconstruction and quantized values ​​of transform coefficients for the residual. More specifically, the CABAC entropy decoding method may receive a bin (binary bit) corresponding to each syntax element in the bitstream, determine a context model by using information of a symbol / bin decoded in the previous stage, decoding information of a decoding target block, or decoding target syntax element information, perform arithmetic decoding on the bin by predicting the probability of occurrence of the bin according to the determined context model, and generate a symbol corresponding to the value of each syntax element. In this case, the CABAC entropy decoding method may update the context model by using information of a symbol / bin decoded by a context model for the next symbol / bin after determining the context model. Information related to prediction among the information decoded by the entropy decoder 310 may be provided to the predictor 330, and information related to the residual for which entropy decoding has been performed in the entropy decoder 310 (ie, quantized transform coefficients and related parameter information) may be input to the dequantizer 321. In addition, information about filtering among the information decoded by the entropy decoder 310 may be provided to the filter 350. In addition, a receiver (not shown) for receiving a signal output from the encoding device may be additionally configured as an internal / external element of the decoding device 300, or the receiver may be a constituent element of the entropy decoder 310. In addition, the decoding device according to the present document may be referred to as a video / image / picture decoding device, and the decoding device may be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include the entropy decoder 310, and the sample decoder may include at least one of the dequantizer 321, the inverse transformer 322, the predictor 330, the adder 340, the filter 350, and the memory 360.

[0075] The dequantizer 321 may dequantize the quantized transform coefficient and output the transform coefficient. The dequantizer 321 may rearrange the quantized transform coefficient in a two-dimensional block form. In this case, the rearrangement may be performed based on a coefficient scanning order performed in the encoding device. The dequantizer 321 may perform dequantization on the quantized transform coefficient using a quantization parameter (e.g., quantization step size information) and obtain the transform coefficient.

[0076] The inverse transformer 322 inversely transforms the transform coefficients to obtain a residual signal (residual block, residual sample array).

[0077] The predictor 330 may perform prediction on the current block and generate a prediction block including prediction samples of the current block. The predictor may determine whether to apply intra prediction or inter prediction to the current block based on information on prediction output from the entropy decoder 310 and may determine a specific intra / inter prediction mode.

[0078] The predictor can generate a prediction signal based on the various prediction methods described below. For example, the predictor can not only apply intra prediction or inter prediction to predict a block, but also apply intra prediction and inter prediction at the same time. This can be called combined inter and intra prediction (CIIP). In addition, the predictor can perform intra block copy (IBC) for prediction of the block. Intra block copy can be used for content image / motion image coding of games, etc., for example, screen content coding (SCC). IBC basically performs prediction in the current picture, but can be performed similarly to inter prediction because the reference block is derived in the current picture. That is, IBC can use at least one of the inter prediction techniques described in this document.

[0079] The intra-frame predictor 332 can predict the current block by referencing samples in the current picture. Depending on the prediction mode, the referenced samples may be located in the neighborhood of the current block, or their location may be separated from the current block. In intra-frame prediction, the prediction mode may include a variety of non-directional modes and a variety of directional modes. The intra-frame predictor 332 may determine the prediction mode to be applied to the current block by using the prediction mode applied to the neighboring blocks.

[0080] The intra-frame predictor 331 can derive the prediction block of the current block based on the reference block (reference sample array) specified by the motion vector on the reference picture. In this case, in order to reduce the amount of motion information sent in the inter-frame prediction mode, the motion information can be predicted in units of blocks, sub-blocks or samples based on the motion information correlation between the neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may also include information about the inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter-frame prediction, the neighboring blocks may include spatial neighboring blocks existing in the current picture and temporal neighboring blocks existing in the reference picture. For example, the inter-frame predictor 331 may construct a motion information candidate list based on the neighboring blocks, and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. Inter-frame prediction may be performed based on various prediction modes, and the information about the prediction may include information indicating the mode of inter-frame prediction for the current block.

[0081] The adder 340 may generate a reconstructed signal (reconstructed picture, reconstructed block, or reconstructed sample array) by adding the obtained residual signal to the prediction signal (prediction block or prediction sample array) output from the predictor 330. If there is no residual of the processing target block (such as the case where the skip mode is applied), the prediction block may be used as the reconstructed block.

[0082] The adder 340 may be referred to as a reconstructor or a reconstructed block generator. The generated reconstructed signal may be used for intra prediction of the next block to be processed in the current picture, and as described later, may also be output through filtering or may also be used for inter prediction of the next picture.

[0083] In addition, luminance mapping with chroma scaling (LMCS) can also be applied to the picture decoding process.

[0084] The filter 350 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filter 350 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and store the modified reconstructed picture in the memory 360, specifically, in the DPB of the memory 360. The various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc.

[0085] The (corrected) reconstructed picture stored in the DPB of the memory 360 may be used as a reference picture in the inter-frame predictor 331. The memory 360 may store the motion information of the block from which the motion information in the current picture is derived (or decoded) and / or the motion information of the block in the reconstructed picture. The stored motion information may be transmitted to the inter-frame predictor 331 so as to be used as the motion information of the spatial neighboring block or the motion information of the temporal neighboring block. The memory 360 may store the reconstructed samples of the reconstructed block in the current picture and transmit the reconstructed samples to the intra-frame predictor 332.

[0086] In the present specification, the embodiments described in the predictor 330, dequantizer 321, inverse transformer 322, and filter 350 of the decoding device 300 may also be applied in the same manner as or corresponding to the predictor 220, dequantizer 234, inverse transformer 235, and filter 260 of the encoding device 200.

[0087] In addition, as described above, when performing video coding, prediction is performed to enhance compression efficiency. A prediction block including prediction samples of a current block (i.e., a target coding block) as a block to be coded can be generated by prediction. In this case, the prediction block includes prediction samples in the spatial domain (or pixel domain). The prediction block is derived identically in the encoding device and the decoding device. The encoding device can enhance image coding efficiency by notifying the decoding device with a signal of information (residual information) about the residual between the original block (rather than the original sample value of the original block itself) and the prediction block. The decoding device can derive a residual block including residual samples based on the residual information, can generate a reconstructed block including reconstructed samples by adding the residual block and the prediction block, and can generate a reconstructed picture including the reconstructed block.

[0088] The residual information may be generated through a transform process and a quantization process. For example, the encoding device may derive a residual block between the original block and the prediction block, may derive a transform coefficient by performing a transform process on the residual samples (residual sample array) included in the residual block, may derive a quantized transform coefficient by performing a quantization process on the transform coefficient, and may signal the relevant residual information (through a bitstream) to the decoding device. In this case, the residual information may include value information of quantized transform coefficients, position information, transform schemes, transform kernels, and quantization parameters, etc. The decoding device may perform a dequantization / inverse transform process based on the residual information and may derive residual samples (or residual blocks). The decoding device may generate a reconstructed picture based on the prediction block and the residual block. In addition, for the inter-frame prediction reference of subsequent pictures, the encoding device may derive a residual block by dequantizing / inverse transforming the quantized transform coefficients, and may generate a reconstructed picture based on this.

[0089] For example, the intra prediction mode may include two non-directional or non-angular intra prediction modes and 65 directional or angular intra prediction modes. The non-directional intra prediction mode may include a plane intra prediction mode No. 0 and a DC intra prediction mode No. 1, and the directional intra prediction mode may include 65 (No. 2 to No. 66) intra prediction modes. However, this is exemplary, and according to the embodiments of this document, it can even be applied to the case where a different number of intra prediction modes are provided. In addition, in some cases, intra prediction mode No. 67 may also be used, and intra prediction mode No. 67 may represent a linear model (LM) mode.

[0090] Figure 4 The intra directional modes in 65 prediction directions are exemplarily illustrated.

[0091] refer to Figure 4 , around intra prediction mode No. 34 having an upper left diagonal prediction direction, the modes can be divided into intra prediction modes having a horizontal directionality and intra prediction modes having a vertical directionality. Figure 4 In , H and V may refer to horizontal and vertical directions, respectively, and the numbers -32 to 32 may represent a displacement in units of 1 / 32 in the sample grid position. This may represent an offset in the mode index value.

[0092] For example, intra prediction modes No. 2 to No. 33 may have horizontal directivity, and intra prediction modes No. 34 to No. 66 may have vertical directivity. In addition, technically, intra prediction mode No. 34 may be considered to have neither horizontal directivity nor vertical directivity, but from the perspective of determining a transform set for a secondary transform, may be classified as belonging to horizontal directivity. This is because the input data is transposed and used for a vertical direction mode symmetrical around intra prediction mode No. 34, and an input data arrangement method for a horizontal direction mode is used for intra prediction mode No. 34. Here, transposition of input data may mean that for two-dimensional block data M×N, N×M data is constructed in such a way that rows become columns and columns become rows.

[0093] In addition, intra prediction mode No. 18 and intra prediction mode No. 50 may represent a horizontal intra prediction mode and a vertical intra prediction mode, respectively, and intra prediction mode No. 2 may be referred to as an upper right diagonal intra prediction mode because prediction is performed in an upper right direction with a left reference pixel. In the same context, intra prediction mode No. 34 may be referred to as a lower right diagonal intra prediction mode, and intra prediction mode No. 66 may be referred to as a lower left diagonal intra prediction mode.

[0094] Figure 5 Context-Adaptive Binary Arithmetic Coding (CABAC) for encoding syntax elements is exemplarily illustrated.

[0095] For example, in the CABAC coding process, if the input signal is a syntax element that is not a binary value, the value of the input signal can be transformed into a binary value by binarization. In addition, if the input signal is already a binary value (that is, if the value of the input signal is a binary value), binarization may not be performed, and the input signal may be used as is. Here, each bin0 or 1 constituting a binary value may be referred to as a bin. For example, if the binarized binary string is 110, each of 1, 1, and 0 may be represented as a bin. The bin of a syntax element may represent the value of the syntax element. Binarization may be based on various binarization methods such as truncated Rice binarization or fixed-length binarization, and the binarization method for the target syntax element may be predefined. The binarization process may be performed by a binarizer in an entropy encoder.

[0096] Thereafter, the binarized bin of the syntax element may be input to a conventional coding engine or a bypass coding engine. The conventional coding engine of the encoding device may assign a context model reflecting a probability value about the corresponding bin, and encode the corresponding bin based on the assigned context model. The conventional coding engine of the encoding device may update the context model of the corresponding bin after performing coding on the corresponding bin. The bin coded as the above content may be represented as a context coded bin.

[0097] In addition, in the case where the binarized bins of the syntax elements are input to the bypass coding engine, they can be coded as follows. For example, the bypass coding engine of the encoding device can omit the process of estimating the probability of the input bin and the process of updating the probability model applied to the coded bin. In the case of applying bypass coding, the encoding device can code the input bin by applying a regular probability distribution instead of assigning a context model, and by doing so, the coding speed can be increased. The bin coded as the above content can be represented as a bypass bin.

[0098] Entropy decoding may mean a process of performing the same process as the above-described entropy encoding in reverse order.

[0099] The decoding device (entropy decoder) can decode the encoded image / video information. The image / video information may include segmentation related information, prediction related information (e.g., inter / intra prediction partition information, intra prediction mode information, inter prediction mode information, etc.), residual information, or in-loop filtering related information, or may include various syntax elements therefor. Entropy coding may be performed in units of syntax elements.

[0100] The decoding device may perform binarization of the target syntax element. Here, the binarization may be based on various binarization methods such as truncated Rice binarization or fixed length binarization, and the binarization method for the target syntax element may be predefined. The decoding device may derive an available bin string (bin string candidate) of an available value of the target syntax element through the binarization process. The binarization process may be performed by a binarizer in an entropy decoder.

[0101] The decoding device can compare the derived bin string with the available bin string of the corresponding syntax element while sequentially decoding or parsing the corresponding bin of the target syntax element from the input bits in the bitstream. If the derived bin string is equal to one of the available bin strings, the value corresponding to the corresponding bin string is derived as the value of the corresponding syntax element. Otherwise, the decoding device can re-execute the above process after further parsing the next bit in the bitstream. Through such processing, variable length bits can be used to perform signaling of specific information (or specific syntax elements) in the bitstream even without using the start bit or end bit of the corresponding information. In this way, relatively fewer bits can be allocated to smaller values, so the overall coding efficiency can be enhanced.

[0102] The decoding apparatus may perform context model-based or bypass-based decoding of a corresponding bin from among the bin strings of the bitstream based on an entropy coding technique such as CABAC or CAVLC.

[0103] In the case of decoding a syntax element based on a context model, a decoding device may receive a bin corresponding to a syntax element through a bitstream, may determine a context model using decoding information of a decoding target block or a neighboring block and a syntax element or symbol / bin information decoded in a previous stage, and may perform arithmetic decoding of the bin by predicting the occurrence probability of the received bin according to the determined context model to derive a syntax element value. Thereafter, a context model of a next decoded bin may be updated based on the determined context model.

[0104] The context model can be allocated and updated by the bin of context coding (regular coding), and the context model can be indicated based on the context index (ctxIdx) or the context index increment (ctxInc). The ctxIdx can be derived based on ctxInc. Specifically, for example, the ctxIdx representing the context model of each bin of regular coding can be derived by the sum of ctxInc and the context index offset (ctxIdxOffset). For example, ctxInc can be derived differently by bin. ctxIdxOffset can be represented as the lowest value of ctxIdx. Generally, ctxIdxOffset can be a value used to distinguish from the context models of other syntax elements, and the context model of a syntax element can be divided or derived based on ctxInc.

[0105] During entropy coding, it may be determined whether coding is performed by a regular coding engine or a bypass coding engine, and thus, the coding path may be switched. Entropy decoding may perform the same process as entropy coding in reverse order.

[0106] In addition, for example, in the case where a syntax element is bypass-decoded, the decoding device may receive a bin corresponding to the syntax element through a bitstream, and may decode the input bin by applying a regular probability distribution. In this case, the decoding device may omit a process of deriving a context model of the syntax element and a process of updating the context model applied to the bin after decoding.

[0107] Figure 6 and Figure 7 The schematic diagram illustrates the multiple transformation technique according to the present document.

[0108] exist Figure 6 and Figure 7 In the example above, the converter may correspond to Figure 2 The converter in the encoding device of , and the inverse converter may correspond to the above-mentioned Figure 2 The inverse transformer in the encoding device or Figure 3 An inverse transformer in a decoding device.

[0109] The transformer may derive (primary) transform coefficients by performing a primary transform based on residual samples (residual sample arrays) in the residual block (S610). Such a primary transform may be referred to as a core transform. Here, the primary transform may be based on a multi-transform selection (MTS), and in the case where a multi-transform is applied as the primary transform, it may be referred to as a multi-core transform.

[0110] For example, the multi-core transform may represent a transform method by additionally using discrete cosine transform (DCT) type 2 (DCT-II), discrete sine transform (DST) type 7 (DST-VII), DCT type 8 (DCT-VIII), and / or DST type 1 (DST-I). That is, the multi-core transform may represent a transform method for transforming a residual signal (or residual block) in the spatial domain into a transform coefficient (or primary transform coefficient) in the frequency domain based on a plurality of transform kernels selected from DCT type 2, DST type 7, DCT type 8, and DST type 1. Here, the primary transform coefficient may be referred to as a temporary transform coefficient on the transformer side.

[0111] In other words, when an existing transform method is applied, a transform from the spatial domain to the frequency domain of a residual signal (or a residual block) can be applied based on DCT type 2, and a transform coefficient can be generated. However, in contrast, when a multi-core transform is applied, a transform from the spatial domain to the frequency domain of a residual signal (or a residual block) can be applied based on DCT type 2, DST type 7, DCT type 8, and / or DST type 1, and a transform coefficient (or a primary transform coefficient) can be generated. Here, DCT type 2, DST type 7, DCT type 8, and DST type 1 can be referred to as a transform type, a transform kernel, or a transform core. DCT / DST transform types can be defined based on basis functions.

[0112] In the case of performing a multi-core transform, a vertical transform core and / or a horizontal transform core for a target block may be selected from the transform cores, and a vertical transform for the target block may be performed based on the vertical transform core, and a horizontal transform for the target block may be performed based on the horizontal transform core. Here, the horizontal transform may represent a transform for a horizontal component of the target block, and the vertical transform may represent a transform for a vertical component of the target block. The vertical transform core / horizontal transform core may be adaptively determined based on a prediction mode and / or a transform index of a target block (CU or subblock) including a residual block.

[0113] In addition, for example, in the case of performing a primary transform by applying MTS, a specific basis function can be configured as a specified value, and in the case of a vertical transform or a horizontal transform, the mapping relationship of the transform kernel can be configured by combining the basis functions applied. For example, in the case where the horizontal direction transform kernel is represented by trTypeHor and the vertical direction transform kernel is represented by trTypeVer, a trTypeHor or trTypeVer with a value of 0 can be configured as DCT2, and a trTypeHor or trTypeVer with a value of 1 can be configured as DCT7. A trTypeHor or trTypeVer with a value of 2 can be configured as DCT8.

[0114] Alternatively, for example, in order to indicate any one of a plurality of transform core sets, the MTS index may be encoded, and the MTS index information may be signaled to the decoding device. Here, the MTS index may be represented as a tu_mts_idx syntax element or an mts_idx syntax element. For example, if the MTS index is 0, it may represent that both the TrTypeHor and TrTypeVer values ​​are 0, and (trTypeHor, trTypeVer) = (DCT2, DCT2). If the MTS index is 1, it may represent that both the TrTypeHor and TrTypeVer values ​​are 1, and (trTypeHor, trTypeVer) = (DST7, DST7). If the MTS index is 2, it may represent that the TrTypeHor value is 2, and the TrTypeVer value is 1, and (trTypeHor, trTypeVer) = (DCT8, DST7). If the MTS index is 3, it may represent that the TrTypeHor value is 1, and the TrTypeVer value is 2, and (trTypeHor, trTypeVer) = (DST7, DCT8). If the MTS index is 4, it can be indicated that the TrTypeHor and TrTypeVer values ​​are both 2, and (trTypeHor, trTypeVer) = (DCT8, DCT8). For example, the transform kernel set according to the MTS index can be shown in the following table.

[0115] [Table 1]

[0116]

[0117] The transformer may derive modified (secondary) transform coefficients by performing a secondary transform based on the (primary) transform coefficients (S620). The primary transform may be a transform from the spatial domain to the frequency domain, while the secondary transform may represent the transform in a more compressed form by using the correlation between the (primary) transform coefficients.

[0118] For example, the secondary transform may include an inseparable transform. In this case, the secondary transform may be referred to as an inseparable secondary transform (NSST) or a mode-dependent inseparable secondary transform (MDNSST). An inseparable secondary transform may represent a transform that generates a modified transform coefficient (or secondary transform coefficient) for a residual signal by performing a secondary transform on the (primary) transform coefficient derived by the primary transform based on an inseparable transform matrix. Here, the vertical transform and the horizontal transform cannot be applied separately (or independently) to the (primary) transform coefficients based on the inseparable transform matrix, but may be applied all at once.

[0119] In other words, the non-separable secondary transform can represent the following transformation method: without separating the (primary) transform coefficients into vertical components and horizontal components, for example, a two-dimensional signal (transform coefficient) is rearranged into a one-dimensional signal in a specially determined direction (for example, a row-first direction or a column-first direction), and then a modified transform coefficient (or secondary transform coefficient) is generated based on the non-separable transform matrix.

[0120] For example, the row-first direction (or order) may indicate that the M×N blocks are arranged in a row in the order of the first row to the Nth row, and the column-first direction (or order) may indicate that the M×N blocks are arranged in a row in the order of the first column to the Mth column. Here, M and N may indicate the width (W) and height (H) of the block, and may all be positive integers.

[0121] For example, an inseparable secondary transform may be applied to the upper left region of a block consisting of (primary) transform coefficients (hereinafter, a transform coefficient block). For example, if the width (W) and height (H) of the transform coefficient block are both equal to or greater than 8, an 8×8 inseparable secondary transform may be applied to the upper left 8×8 region of the transform coefficient block. In addition, if the width (W) and height (H) of the transform coefficient block are both equal to or greater than 4 and less than 8, a 4×4 inseparable secondary transform may be applied to the upper left min (8, W) × min (8, H) region of the transform coefficient block. However, the embodiment is not limited thereto, and for example, even if the condition that the width (W) and height (H) of the transform coefficient block are both equal to or greater than 4 is satisfied, a 4×4 inseparable secondary transform may be applied to the upper left min (8, W) × min (8, H) region of the transform coefficient block.

[0122] Specifically, for example, in the case of using a 4×4 input block, the inseparable secondary transform may be performed as follows.

[0123] A 4×4 input block X can be represented as follows.

[0124] [Formula 1]

[0125]

[0126] For example, the vector form of X can be expressed as follows.

[0127] [Formula 2]

[0128]

[0129] Referring to formula 2, A vector X may be represented, and a two-dimensional block of X in Equation 1 may be rearranged in a row-major order and represented as a one-dimensional vector.

[0130] In this case, the quadratic non-separable transform can be calculated as follows.

[0131] [Formula 3]

[0132]

[0133] here, may denote a transform coefficient vector, and T may denote a 16×16 (non-separable) transform matrix.

[0134] Based on Equation 3, we can derive a 16×1 , and can be divided into Reorganize into 4×4 blocks. However, the above calculation is exemplary, and in order to reduce the computational complexity of the non-separable quadratic transform, a hypercube-givens transform (HyGT) or the like may be used to calculate the non-separable quadratic transform.

[0135] Furthermore, in the inseparable secondary transform, a transform kernel (or transform core or transform type) may be selected in a mode-dependent manner. Here, the mode may include an intra prediction mode and / or an inter prediction mode.

[0136] For example, as described above, an indivisible secondary transform may be performed based on an 8×8 transform or a 4×4 transform determined based on the width (W) and height (H) of the transform coefficient block. For example, if both W and H are equal to or greater than 8, the 8×8 transform may represent a transform that may be applied to an 8×8 region included inside the corresponding transform coefficient block, and the 8×8 region may be the upper left 8×8 region inside the corresponding transform coefficient block. In addition, similarly, if both W and H are equal to or greater than 4, the 4×4 transform may represent a transform that may be applied to a 4×4 region included inside the corresponding transform coefficient block, and the 4×4 region may be the upper left 4×4 region inside the corresponding transform coefficient block. For example, the 8×8 transform kernel matrix may be a 64×64 / 16×64 matrix, and the 4×4 transform kernel matrix may be a 16×16 / 8×16 matrix.

[0137] In this case, for mode-based transform core selection, two indivisible secondary transform cores for each transform set of indivisible secondary transforms may be configured for all 8×8 transforms and 4×4 transforms, and four transform sets may be provided. That is, four transform sets may be configured for 8×8 transforms, and four transform sets may be configured for 4×4 transforms. In this case, each of the four transform sets for 8×8 transforms may include two 8×8 transform cores, and each of the four transform sets for 4×4 transforms may include two 4×4 transform cores.

[0138] However, the size of the transform, the number of sets, and the number of transform cores in the set are exemplary, and sizes other than 8×8 or 4×4 may be used, or n sets may be configured, or k transform cores may be included in each set. Here, n and k may be positive integers.

[0139] For example, a transform set may be referred to as an NSST set, and a transform core in the NSST set may be referred to as an NSST core.For example, selection of a specific set among transform sets may be performed based on an intra prediction mode of a target block (CU or subblock).

[0140] In addition, if it is determined that a specific set is used for an inseparable transform, one of the k transform cores in the specific set can be selected by an inseparable secondary transform index. For example, the encoding device can derive an inseparable secondary transform index representing a specific transform core based on a rate-distortion (RD) check, and can signal the inseparable secondary transform index to the decoding device. For example, the decoding device can select one of the k transform cores in a specific set based on the inseparable secondary transform index. For example, an NSST index with a value of 0 can represent a first inseparable secondary transform core, an NSST index with a value of 1 can represent a second inseparable secondary transform core, and an NSST index with a value of 2 can represent a third inseparable secondary transform core. Alternatively, an NSST index with a value of 0 can indicate that the first inseparable secondary transform is not applied to the target block, and NSST indexes with values ​​of 1 to 3 can indicate the three transform cores as above.

[0141] The transformer can perform an inseparable secondary transform based on the selected transform kernel, and can obtain a modified (secondary) transform coefficient. The modified transform coefficient can be derived as a quantized transform coefficient quantized by the above-mentioned quantizer, and can be encoded to be notified to the decoding device with a signal and can be transmitted to the dequantizer / inverse transformer in the encoding device.

[0142] In addition, if the secondary transform is omitted as described above, the (primary) transform coefficients as the output of the primary (separable) transform can be derived as quantized transform coefficients by a quantizer as described above, and can be encoded to notify the decoding device with a signal and can be transmitted to the dequantizer / inverse transformer in the encoding device.

[0143] refer to Figure 7, the inverse transformer may perform a series of processes in the reverse order of the processes performed by the above-mentioned transformer. The inverse transformer may receive the (dequantized) transform coefficients, derive the (primary) transform coefficients by performing a secondary (inverse) transform (S750), and obtain the residual block (residual sample) by performing a primary (inverse) transform with respect to the (primary) transform coefficients (S760). Here, on the inverse transformer side, the primary transform coefficients may be referred to as modified transform coefficients. As described above, the encoding device and / or the decoding device may generate a reconstructed block based on the residual block and the prediction block, and may generate a reconstructed picture based thereon.

[0144] In addition, the decoding device may also include a secondary inverse transform application / non-application determiner (or an element for determining whether to apply a secondary inverse transform) and a secondary inverse transform determiner (or an element for determining a secondary inverse transform). For example, the secondary inverse transform application / non-application determiner may determine whether to apply a secondary inverse transform. For example, the secondary inverse transform may be NSST or RST, and the secondary inverse transform application / non-application determiner may determine whether to apply a secondary inverse transform based on a secondary transform flag parsed or obtained from a bitstream. Alternatively, for example, the secondary inverse transform application / non-application determiner may determine whether to apply a secondary inverse transform based on a transform coefficient of a residual block.

[0145] The secondary inverse transform determiner may determine the secondary inverse transform. In this case, the secondary inverse transform determiner may determine the secondary inverse transform applied to the current block based on the NSST (or RST) transform set specified according to the intra prediction mode. Alternatively, the secondary transform determination method may be determined depending on the primary transform determination method. Alternatively, various combinations of the primary transform and the secondary transform may be determined according to the intra prediction mode. For example, the secondary inverse transform determiner may determine the area to which the secondary inverse transform is applied based on the size of the current block.

[0146] In addition, if the secondary (inverse) transform is omitted as described above, a residual block (residual sample) can be obtained by receiving the (dequantized) transform coefficients and performing a primary (separable) inverse transform. As described above, the encoding device and / or the decoding device can generate a reconstructed block based on the residual block and the prediction block, and can generate a reconstructed picture based thereon.

[0147] Furthermore, in this document, in order to reduce the computational and memory requirements associated with the non-separable quadratic transform, a reduced quadratic transform (RST) with a reduced transform matrix (kernel) size may be applied to the concept of NSST.

[0148] In this document, RST may mean a (simplified) transformation performed on residual samples of a target block based on a transformation matrix whose size is reduced by a simplification factor. When this is performed, the amount of calculation required during the transformation can be reduced because the size of the transformation matrix is ​​reduced. That is, RST can be used to solve the computational complexity problem that occurs during the transformation of large-sized blocks or inseparable transformations.

[0149] For example, RST may be referred to as various terms such as reduced transform, reduced secondary transform, reduced transform, simplified transform, or simple transform, and the name referring to RST is not limited to the listed examples. In addition, RST is mainly performed in the low frequency domain including non-zero coefficients in the primary transform block, and thus may be referred to as a low frequency non-separable transform (LFNST).

[0150] In addition, in the case where a secondary inverse transform is performed based on the RST, the inverse transformer 235 of the encoding device 200 and the inverse transformer 322 of the decoding device 300 may include an inverse RST unit that derives a modified transform coefficient based on the inverse RST for the transform coefficient and an inverse primary transformer that derives a residual sample of the target block based on the inverse primary transform for the modified transform coefficient. The inverse primary transform means an inverse transform of the primary transform that has been applied to the residual. In this document, deriving a transform coefficient based on a transform may mean deriving a transform coefficient by applying a corresponding transform.

[0151] Figure 8 and Fig. 9 is a diagram illustrating an RST according to an embodiment of this document.

[0152] For example, Figure 8 may be a diagram illustrating the application of a positive downscaling transform, and Fig. 9 It may be a diagram illustrating application of an inverse downscaling transform. In this document, a target block may refer to a current block, a residual block, or a transform block on which coding is performed.

[0153] For example, in RST, an N-dimensional vector can be mapped to an R-dimensional vector located in another space, and a reduced transformation matrix can be determined. Here, N and R can be positive integers, and R can be less than N. N can mean the square of the length of one side of the block to which the transformation is applied or the total number of transformation coefficients corresponding to the block to which the transformation is applied, and the simplification factor can mean the R / N value. The simplification factor can be referred to as various terms such as reduction factor, reduction factor, simplification factor or simple factor. In addition, R can be referred to as a reduction coefficient, and in some cases, the simplification factor can mean R. In addition, in some cases, the simplification factor can mean the N / R value.

[0154] For example, the simplification factor or reduction coefficient may be signaled through a bitstream, but is not limited thereto. For example, predefined values ​​of the simplification factor or reduction coefficient may be stored in the encoding device 200 and the decoding device 300, and in this case, the simplification factor or reduction coefficient may not be signaled separately.

[0155] For example, the size of the simplified transformation matrix (R×N) may be smaller than the size of the conventional transformation matrix ((N×N)), and may be defined as in the following equation.

[0156] [Formula 4]

[0157]

[0158] For example, Figure 8 The matrix T in the reduced transform block shown in the figure can be expressed as the matrix T of formula 4 R×N .like Figure 8 As shown, the residual samples of the target block are multiplied by the simplified transformation matrix T R×N In this case, the transform coefficients of the target block can be derived.

[0159] For example, in the case where the size of the block to which the transformation is applied is 8×8 and R is 16 (ie, R / N = 16 / 64 = 1 / 4), the matrix operation according to Equation 5 can be expressed as follows Figure 8 In this case, the memory and multiplication operations can be reduced by a simplification factor of about 1 / 4.

[0160] In this document, matrix operations may be understood as operations that obtain a column vector by placing a matrix to the left of a column vector and multiplying the matrix by the column vector.

[0161] [Formula 5]

[0162]

[0163] In formula 5, r1 to r 64 can represent the residual samples of the target block. Alternatively, for example, they can be transform coefficients generated by applying the primary transform. Based on the operation result of Formula 5, the transform coefficient c of the target block can be derived i .

[0164] For example, when R is 16, the transform coefficients c1 to c 16. If a transform matrix having a size of 64×64 (N×N) by applying a conventional transform instead of RST is multiplied by a residual sample having a size of 64×1 (N×1), 64 (N) transform coefficients of the target block can be derived, but since RST is applied, only 16 (R) transform coefficients of the target block can be derived. Since the total number of transform coefficients of the target block is reduced from N to R, the amount of data transmitted from the encoding device 200 to the decoding device 300 can be reduced, and thus the transmission efficiency between the encoding device 200 and the decoding device 300 can be increased.

[0165] Considering the size of the transformation matrix, since the size of the conventional transformation matrix is ​​64×64 (N×N), and the size of the simplified transformation matrix is ​​reduced to 16×64 (R×N), the memory usage when performing RST can be reduced by the R / N ratio compared to the case of performing the conventional transformation. In addition, the use of the simplified transformation matrix can reduce the number of multiplication operations (R×N) by the R / N ratio compared to the number of multiplication operations (N×N) when the conventional transformation matrix is ​​used.

[0166] In an embodiment, the transformer 232 of the encoding device 200 may derive the transform coefficient of the target block by performing the primary transform of the residual sample of the target block and the secondary transform based on the RST. The transform coefficient may be transmitted to the inverse transformer of the decoding device 300, and the inverse transformer 322 of the decoding device 300 may derive the modified transform coefficient based on the inverse reduced secondary transform (RST) of the transform coefficient, and may derive the residual sample of the target block based on the inverse primary transform of the modified transform coefficient.

[0167] The inverse RST matrix T according to an embodiment N×R The size of can be N×R which is smaller than the size of the conventional inverse transformation matrix N×N, and can be combined with the simplified transformation matrix T illustrated in Equation 4 R×N A transposed relationship.

[0168] Fig. 9 The matrix T in the reduced inverse transform block shown in the figure t The inverse RST matrix T can be expressed as R×N T Here, the superscript T can represent transposition. Fig. 9 As shown, the transform coefficients for the target block are multiplied by the inverse RST matrix T R×N T In the case of , the modified transform coefficients of the target block or the residual samples for the target block can be derived. The inverse RST matrix T R×N T It can be expressed as (T R×N ) T N×R .

[0169] More specifically, in the case where the inverse RST is applied as a secondary inverse transform, the transform coefficients of the target block can be multiplied by the inverse RST matrix T R×N T To derive the modified transform coefficients of the target block. In addition, the inverse RST can be applied as the inverse primary transform, and in this case, the transform coefficients of the target block can be multiplied by the inverse RST matrix T R×N T To derive the residual samples of the target block.

[0170] In an embodiment, when the size of the block to which the inverse transform is applied is 8×8 and R is 16 (ie, R / N = 16 / 64 = 1 / 4), the following matrix operation can be expressed according to Figure 7 RST.

[0171] [Formula 6]

[0172]

[0173] In formula 6, c1 to c 16 The transform coefficient of the target block can be represented by r. The modified transform coefficient of the target block or the residual sample of the target block can be derived based on the calculation result of equation 6. j That is, r1 to r2 representing the modified transform coefficients of the target block or the residual samples of the target block can be derived. N .

[0174] Considering the size of the inverse transformation matrix, since the size of the conventional inverse transformation matrix is ​​64×64 (N×N), and the size of the simplified inverse transformation matrix is ​​reduced to 64×16 (N×R), the memory usage when performing the inverse RST can be reduced by the R / N ratio compared to the case of performing the conventional inverse transformation. In addition, the use of the simplified inverse transformation matrix can reduce the number of multiplication operations (N×R) by the R / N ratio compared to the number of multiplication operations (N×N) when the conventional inverse transformation matrix is ​​used.

[0175] In addition, the transform set can be configured and applied even for 8×8 RST. That is, the corresponding 8×8 RST can be applied according to the transform set. Since one transform set is composed of two or three transform cores according to the intra prediction mode, it can be configured to select one of up to four transforms, even including the case where the secondary transform is not applied. In the transform when the secondary transform is not applied, it can be considered that the unit matrix is ​​applied. If it is assumed that the four transforms are assigned indexes 0, 1, 2, or 3 (for example, the index No. 0 can be assigned to the unit matrix, that is, the case where the secondary transform is not applied), the transform to be applied can be specified by signaling a syntax element that is the NSST index of each transform coefficient block. That is, through the NSST index, the 8×8 NSST can be specified for the 8×8 upper left block, and in the RST configuration, the 8×8 RST can be specified. 8×8 NSST and 8×8 RST can represent the transform that can be applied to the 8×8 area included in the corresponding transform coefficient block when the W and H of the target block that becomes the transform target are equal to or greater than 8, and the 8×8 area can be the upper left 8×8 area inside the corresponding transform coefficient block. Similarly, 4×4 NSST and 4×4 RST may represent a transform that can be applied to a 4×4 region included in a corresponding transform coefficient block when W and H of the target block are both equal to or greater than 4, and the 4×4 region may be the upper left 4×4 region within the corresponding transform coefficient block.

[0176] In addition, for example, the encoding device may obtain a bitstream by encoding the value of the syntax element or the quantized value of the transform coefficient of the residual based on various coding methods such as exponential Golomb coding, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), etc. In addition, the decoding device may derive the value of the syntax element or the quantized value of the transform coefficient of the residual based on various coding methods such as exponential Golomb coding, CAVLC, CABAC, etc.

[0177] In addition, an embodiment of the present document may propose a scheme for signaling an MTS index. Here, as described above, the MTS index may represent any one of a plurality of transform core sets, and the MTS index may be encoded, and the MTS index information may be signaled to the decoding device. The decoding device may obtain the MTS index by decoding the MTS index information, and may determine the transform core set to be applied based on the MTS index. The MTS index may be represented as a tu_mts_idx syntax element or an mts_idx syntax element. For example, the MTS index may be binarized using the Rice-Golmb parameter 0th order, but may be binarized based on truncated rice. In the case of binarization based on truncated rice, the input parameter cMax may have a value of 4, and cRiceParam may have a value of 0. For example, the encoding device may derive a bin for the MTS index by binarizing the MTS index, derive the MTS index information (bits for the MTS index) by encoding the derived bin, and signal the MTS index information to the decoding device. The decoding apparatus may derive a bin for the MTS index by decoding the MTS index information, and derive the MTS index by comparing the derived bin for the MTS index with candidate bins for the MTS index.

[0178] For example, the MTS index (e.g., tu_mts_idx syntax element or mts_idx syntax element) may be context coded based on a context model or context index relative to all bins. In this case, a context index increment (ctxInc) for context coding of the MTS index or ctxInc according to a bin position may be allocated or determined as in Table 2. In addition, as in Table 2, a context model is selected according to a bin position.

[0179] [Table 2]

[0180]

[0181] Referring to Table 2, ctxInc for bin numbered 0 (first bin) may be allocated based on cqtDepth. Here, cqtDepth may represent a quadtree depth for the current block, and may be derived as a value from 0 to 5. That is, ctxInc for bin numbered 0 may be allocated a value from 0 to 5 according to cqtDepth. In addition, ctxInc for bin numbered 1 (second bin) may be allocated 6, ctxInc for bin numbered 2 (third bin) may be allocated 7, and ctxInc for bin numbered 3 (fourth bin) may be allocated 8. That is, ctxInc having different values ​​may be allocated for bin numbered 0 to bin numbered 3. Here, different ctxInc values ​​may represent different context models, and in this case, 9 context models for MTS index coding may be provided.

[0182] In addition, for example, as in Table 3, with respect to all bins, the MTS index (eg, tu_mts_idx syntax element or mts_idx syntax element) may be bypass coded. In this case, a context model for MTS index coding may be provided.

[0183] [Table 3]

[0184]

[0185] In addition, for example, as in Table 4, the MTS index (e.g., tu_mts_idx syntax element or mts_idx syntax element) may be context coded based on a context model or a context index with respect to a bin numbered 0 (the first bin), and may be bypass coded with respect to the remaining bins. That is, 0 may be assigned to ctxInc for the bin numbered 0 (the first bin). In this case, one context model for MTS index coding may be provided.

[0186] [Table 4]

[0187]

[0188] In addition, for example, as in Table 5, the MTS index (e.g., tu_mts_idx syntax element or mts_idx syntax element) may be context coded based on a context model or a context index with respect to a bin numbered 0 (first bin) and a bin numbered 1 (second bin), and may be bypass coded with respect to the remaining bins. That is, 0 may be assigned to ctxInc of the bin numbered 0 (first bin), and 1 may be assigned to ctxInc of the bin numbered 1 (second bin). In this case, 2 context models for MTS index coding may be provided.

[0189] [Table 5]

[0190]

[0191] In addition, for example, as in Table 6, the MTS index (e.g., tu_mts_idx syntax element or mts_idx syntax element) may be context coded based on a context model or a context index with respect to all bins, and one ctxInc may be allocated with respect to each bin. That is, 0 may be allocated to the ctxInc of the bin numbered 0 (first bin), and 1 may be allocated to the ctxInc of the bin numbered 1 (second bin). 2 may be allocated to the ctxInc of the bin numbered 2 (third bin), and 2 may be allocated to the ctxInc of the bin numbered 3 (fourth bin). In this case, 4 context models for MTS index coding may be provided.

[0192] [Table 6]

[0193]

[0194] As described above, according to an embodiment, since the number of context models is reduced by applying bypass coding to all or some bins of the MTS index or applying a specific value to ctxInc (although context coding is applied), the complexity can be reduced and the output of the decoder can be increased. In addition, in an embodiment, in the case of using a context model as described above, the initial value and / or the multi-window size can be variable based on the occurrence statistics of the position of each bin.

[0195] Fig.10 and Fig.11 An example of a video / image encoding method and related components according to an embodiment of this document is schematically shown.

[0196] Fig.10 The method disclosed in can be Figure 2 or Fig.11 Specifically, for example, Fig.10 The S1000 to S1010 can be made of Fig.11 The residual processor 230 of the encoding device performs, and Fig.10 The S1020 can be Fig.11 The entropy encoder 240 of the encoding device performs. In addition, although Fig.10 Not shown, but can be Fig.11 The predictor 220 of the encoding device derives the predicted sample or the prediction related information, the residual processor 230 of the encoding device can derive the residual information from the original sample or the predicted sample, and the entropy encoder 240 of the encoding device can generate a bitstream from the residual information or the prediction related information. Fig. 9 The methods disclosed in may be included in the embodiments described above in this document.

[0197] refer to Fig.10 , the encoding device derives the residual sample of the current block (S1000). For example, the encoding device may derive the residual sample based on the predicted sample and the original sample. Fig.10 Not shown, in order to generate a prediction sample of the current block, the encoding apparatus may perform intra prediction or inter prediction with respect to the current block in consideration of rate-distortion (RD) cost, and may generate prediction-related information including prediction mode / type information.

[0198] The encoding device generates residual information of the current block based on the residual samples (S1010). For example, the encoding device may derive the transform coefficients of the current block based on the residual samples, and may generate the residual information based on the transform coefficients. For example, the encoding device may derive the transform coefficients by performing a transform relative to the residual samples. Here, the transform may be performed based on a transform kernel or a transform kernel set. For example, the transform kernel set may include a horizontal transform kernel and a vertical transform kernel. For example, the encoding device may derive the transform coefficients by performing a primary transform relative to the residual samples. In addition, for example, the encoding device may derive temporary transform coefficients by performing a primary transform relative to the residual samples, and may derive the transform coefficients by performing a secondary transform relative to the temporary transform coefficients. For example, the transform performed based on the transform kernel set may be represented as a primary transform.

[0199] For example, the residual information may include an MTS index and / or information about a transform coefficient. The encoding device may generate an MTS index and / or information about a transform coefficient based on the transform coefficient.

[0200] The MTS index may represent a transform core set (transform coefficient) applied to the current block among the transform core set candidates. Here, the MTS index may be represented as a tu_mts_idx syntax element or an mts_idx syntax element. As described above, the transform core set may include a horizontal transform core and a vertical transform core, and the horizontal transform core may be represented as trTypeHor, and the vertical transform core may be represented as trTypeVer.

[0201] For example, the trTypeHor and trTypeVer values ​​may be represented by a horizontal direction transform kernel and a vertical direction transform kernel applied to (transform coefficients) of the current block, and the MTS index may be represented by the trTypeHor and trTypeVer values ​​as one of candidates 0 to 4 inclusive.

[0202] For example, if the MTS index is 0, it can indicate that both trTypeHor and trTypeVer are 0. In addition, if the MTS index is 1, it can indicate that both trTypeHor and trTypeVer are 1. In addition, if the MTS index is 2, it can indicate that trTypeHor is 2 and trTypeVer is 1. If the MTS index is 3, it can indicate that trTypeHor is 1 and trTypeVer is 1. In addition, if the MTS index is 4, it can indicate that both trTypeHor and trTypeVer are 2. For example, if the trTypeHor or trTypeVer value is 0, it can indicate that DCT2 is applied to (transform coefficients) of the current block in the horizontal direction or the vertical direction, if it is 1, it can indicate that DST7 is applied, and if it is 2, it can indicate that DCT8 is applied. That is, based on the MTS index, each of the transform kernel to be applied in the horizontal direction and the transform kernel to be applied in the vertical direction can be represented as one of the candidates including DCT2, DST7 and DCT8.

[0203] The MTS index may be represented based on a bin of a bin string of the MTS index. In other words, the MTS index may be binarized and represented as a bin of a bin string of the MTS index, and the bin of the MTS index may be entropy encoded.

[0204] In other words, at least one of the bins of the bin string of the MTS index can be represented based on context coding. Here, context coding can be performed based on the value of the context index increment (ctxInc). In addition, context coding can be performed based on the context index (ctxIdx) or the context model. Here, the context index can be represented based on the value of the context index increment. In addition, the context index can be represented based on the value of the context index increment and the context index offset (ctxIdxOffset).

[0205] For example, all bins of the bin string of the MTS index may be represented based on context compilation. For example, the ctxInc of the first bin or the 0th bin (bin 0) among the bins of the bin string for the MTS index may be represented based on cqtDepth. Here, cqtDepth may represent the quadtree depth for the current block and may be represented as a value from 0 to 5. In addition, the ctxInc for the second bin or the bin numbered 1 (bin 1) may be represented as 6, the ctxInc for the third bin or the bin numbered 2 (bin 2) may be represented as 7, and the ctxInc for the fourth bin or the bin numbered 3 (bin 3) may be represented as 8. In addition, for example, the ctxInc of the first bin or the 0th bin (bin 0) among the bins of the bin string for the MTS index may be represented as 0, the ctxInc for the second bin or the bin numbered 1 (bin 1) may be represented as 1, the ctxInc for the third bin or the bin numbered 2 (bin 2) may be represented as 2, and the ctxInc for the fourth bin or the bin numbered 3 (bin 3) may be represented as 3. That is, the number of context index increment values ​​that can be used for context coding of the first bin among the bins of the bin string may be 1. Alternatively, the bin string for the MTS index may include up to four bins, and among the four bins, the number of context index increment values ​​allocated to the bin derived by context coding may be 1.

[0206] In addition, for example, some bins of the bin string of the MTS index can be represented based on context coding, and the remaining bins can be represented based on bypass coding. For example, the ctxInc of the first bin or the bin numbered 0 (bin 0) among the bins of the bin string for the MTS index can be represented as 0, and the remaining bins can be represented based on bypass coding. In addition, for example, the ctxInc of the first bin or the bin numbered 0 (bin 0) among the bins of the bin string for the MTS index can be represented as 0, the ctxInc for the second bin or the bin numbered 1 (bin 1) can be represented as 1, and the remaining bins can be represented based on bypass coding. That is, the number of context index increment values ​​that can be used for context coding of the first bin among the bins of the bin string can be 1.

[0207] In addition, all bins of the bin string of the MTS index may be represented based on bypass coding. Here, bypass coding may mean that context coding is performed based on uniform probability distribution, and since the update process of context coding is omitted, coding efficiency may be improved.

[0208] The information about the transform coefficients may indicate information used to derive the transform coefficients. In addition, for example, the information about the transform coefficients may include information about residual samples, transform-related information, and / or quantization-related information. For example, the information about the transform coefficients may include information about quantized transform coefficients.

[0209] The encoding device generates a bitstream by encoding the video information including the residual information (S1020). For example, the video information may be referred to as image information. For example, the video information may further include prediction related information. For example, the encoding device may encode the video information to output a bitstream. The bitstream may also be referred to as coded (image or video) information.

[0210] In addition, despite the Fig.10 Although not shown in FIG. 1 , for example, the encoding device may generate a reconstructed sample based on the residual sample and the prediction sample. In addition, a reconstructed block and a reconstructed picture may be derived based on the reconstructed sample.

[0211] For example, the encoding device may generate a bit stream or coding information by encoding the image information including all or some of the above information (or syntax elements). In addition, the information may be output in the form of a bit stream. In addition, the bit stream or coding information may be sent to the decoding device via a network or a storage medium. In addition, the bit stream or coding information may be stored in a computer-readable storage medium, and the bit stream or coding information may be generated by the above image coding method.

[0212] Fig.12 and Fig.13 An example of a video / image decoding method and related components according to an embodiment of this document is schematically shown.

[0213] Fig.12 The method disclosed in can be Figure 3 or Fig.13 Specifically, for example, Fig.12 The S1200 can be made by Fig.13 The entropy decoder 310 of the decoding device performs, and Fig.12 The S1210 can be Fig.13 The residual processor 320 of the decoding device performs, and Fig.12 The S1220 can be Fig.13 The adder 340 of the decoding device performs. In addition, although Fig.12 Not shown, but can be Fig.13 The entropy decoder 310 of the decoding device derives prediction related information or residual information from the bitstream, and the residual processor 320 of the decoding device can derive residual samples from the residual information. The predictor 330 of the decoding device can derive prediction samples from the prediction related information, and the adder 340 of the decoding device can derive a reconstructed block or a reconstructed picture from the residual samples or the prediction samples. Fig.12 The methods disclosed in may include the embodiments described above in this document.

[0214] Reference Fig.12 , the decoding device obtains residual information from the bitstream (S1200). For example, the decoding device may obtain the residual information by parsing or decoding the bitstream. Here, the bitstream may be referred to as coded (image) information. For example, the residual information may include an MTS index and / or information about a transform coefficient.

[0215] The MTS index may represent a transform core set to be applied to the current block among the transform core set candidates. Here, the MTS index may be represented as a tu_mts_idx syntax element or an mts_idx syntax element. In addition, the transform core set may include a transform core to be applied to the current block in the horizontal direction and a transform core to be applied to the current block in the vertical direction. Here, the transform core to be applied in the horizontal direction may be represented as trTypeHor, and the transform core to be applied in the vertical direction may be represented as trTypeVer.

[0216] For example, the MTS index may be derived as one of candidates including 0 to 4, and according to the MTS index, each of trTypeHor and trTypeVer may be derived as one of 0 to 2. For example, if the MTS index is 0, trTypeHor and trTypeVer may all be 0. Furthermore, if the MTS index is 1, trTypeHor and trTypeVer may all be 1. Furthermore, if the MTS index is 2, trTypeHor may be 2, and trTypeVer may be 1. If the MTS index is 3, trTypeHor may be 1, and trTypeVer may be 1. Furthermore, if the MTS index is 4, trTypeHor and trTypeVer may all be 2. For example, a trTypeHor or trTypeVer value may represent a transform kernel, and if 0, may represent DCT2, if 1, may represent DST7, and if 2, may represent DCT8. That is, each of a transform kernel to be applied in the horizontal direction and a transform kernel to be applied in the vertical direction may be derived as one of candidates including DCT2, DST7, and DCT8 based on the MTS index.

[0217] The MTS index may be derived based on a bin of a bin string of the MTS index. In other words, the MTS index information may be entropy decoded and derived as a binarized MTS index, and the binarized MTS index may be represented as a bin of a bin string of the MTS index.

[0218] In other words, at least one of the bins of the bin string of the MTS index can be derived based on context coding. Here, context coding can be performed based on the value of the context index increment (ctxInc). In addition, context coding can be performed based on the context index (ctxIdx) or the context model. Here, the context index can be derived based on the value of the context index increment. In addition, the context index can be derived based on the value of the context index increment and the context index offset (ctxIdxOffset).

[0219] For example, all bins of the bin string of the MTS index may be derived based on context compilation. For example, the ctxInc of the first bin or the 0th bin (bin 0) among the bins of the bin string of the MTS index may be assigned based on cqtDepth. Here, cqtDepth may represent the quadtree depth for the current block and may be derived as a value between 0 and 5. In addition, the ctxInc for the second bin or the bin numbered 1 (bin 1) may be assigned 6, the ctxInc for the third bin or the bin numbered 2 (bin2) may be assigned 7, and the ctxInc for the fourth bin or the bin numbered 3 (bin 3) may be assigned 8. In addition, for example, the ctxInc of the first bin or the 0th bin (bin 0) among the bins of the bin string for the MTS index may be assigned 0, the ctxInc for the second bin or the bin numbered 1 (bin 1) may be assigned 1, the ctxInc for the third bin or the bin numbered 2 (bin 2) may be assigned 2, and the ctxInc for the fourth bin or the bin numbered 3 (bin 3) may be assigned 3. That is, the number of context index increment values ​​that can be used for context coding of the first bin among the bins of the bin string may be 1. Alternatively, the bin string of the MTS index may include up to four bins, and the number of context index increment values ​​assigned to the bins derived by context coding among the four bins may be 1.

[0220] In addition, for example, some bins of the bin string of the MTS index can be derived based on context coding, and the remaining bins can be derived based on bypass coding. For example, the ctxInc of the first bin or the bin numbered 0 (bin 0) among the bins of the bin string for the MTS index can be assigned 0, and the remaining bins can be derived based on bypass coding. In addition, for example, the ctxInc of the first bin or the bin numbered 0 (bin 0) among the bins of the bin string for the MTS index can be assigned 0, the ctxInc of the second bin or the bin numbered 1 (bin 1) can be assigned 1, and the remaining bins can be derived based on bypass coding. That is, the number of context index increment values ​​that can be used for context coding of the first bin among the bins of the bin string can be 1.

[0221] In addition, all bins of the bin string of the MTS index can be derived based on bypass coding. Here, bypass coding can mean that context coding is performed based on uniform probability distribution, and since the update process of context coding is omitted, coding efficiency can be improved.

[0222] The information about the transform coefficients may indicate information used to derive the transform coefficients. In addition, the residual information may further include information used to derive residual samples. For example, the residual information may include information about residual samples, inverse transform related information and / or inverse quantization related information. For example, the residual information may include information about quantized transform coefficients.

[0223] The decoding device generates a residual sample of the current block based on the residual information (S1210). For example, the decoding device may derive a transform coefficient of the current block based on the residual information, and may generate a residual sample based on the transform coefficient. For example, the decoding device may derive a quantized transform coefficient of the current block based on information about the quantized transform coefficient included in the residual information. For example, the decoding device may derive the transform coefficient of the current block by performing dequantization with respect to the quantized transform coefficient.

[0224] For example, the decoding device may generate residual samples of the current block based on the MTS index included in the residual information and the transform coefficient according to the information about the transform coefficient included in the residual information (S1120). For example, the residual samples may be generated based on the transform core set represented by the transform coefficient and the MTS index. That is, the decoding device may generate residual samples from the transform coefficient by inverse transform using the transform core set represented by the MTS index. Here, the inverse transform using the transform core set represented by the MTS index may be included in the primary inverse transform. In addition, when generating residual samples from the transform coefficient, the decoding device may use not only the primary inverse transform but also the secondary inverse transform. In this case, the decoding device may derive the modified transform coefficient by performing a secondary inverse transform with respect to the transform coefficient, and may generate residual samples by performing a primary inverse transform with respect to the modified transform coefficient.

[0225] The decoding device generates a reconstructed sample of the current block based on the residual sample (S1220). For example, the decoding device may obtain prediction related information including a prediction mode / type from a bitstream, and may generate a prediction sample for the current block by performing intra prediction or inter prediction based on the prediction mode / type information. In addition, for example, the decoding device may generate a reconstructed sample based on the prediction sample and the residual sample. In addition, for example, a reconstructed block or a reconstructed picture may be derived based on the reconstructed sample.

[0226] For example, the decoding device can obtain image information including all or part of the above-mentioned multiple information (or syntax elements) by decoding the bit stream or coding information. In addition, the bit stream or coding information can be stored in a computer-readable storage medium, and can cause the above-mentioned decoding method to be executed.

[0227] Although the method has been described based on a flowchart that lists steps or blocks in sequence in the above-mentioned embodiments, the steps of this document are not limited to a specific order, and specific steps may be performed in different steps or in a different order or simultaneously relative to the above-mentioned steps. In addition, it will be understood by a person of ordinary skill in the art that the steps in the flowchart are not exclusive, and another step may be included therein, or one or more steps in the flowchart may be deleted without affecting the scope of the present disclosure.

[0228] The above-mentioned method according to the present disclosure may be in the form of software, and the encoding device and / or decoding device according to the present disclosure may be included in a device for performing image processing (e.g., TV, computer, smart phone, set-top box, display device, etc.).

[0229] When the embodiments of the present disclosure are implemented with software, the above-mentioned methods can be implemented with modules (processing or functions) that perform the above-mentioned functions. The modules can be stored in a memory and executed by a processor. The memory can be installed inside or outside the processor and can be connected to the processor via various well-known devices. The processor may include an application-specific integrated circuit (ASIC), other chipsets, logic circuits, and / or data processing devices. The memory may include a read-only memory (ROM), a random access memory (RAM), a flash memory, a memory card, a storage medium, and / or other storage devices. In other words, according to the embodiments of the present disclosure, it can be implemented and executed on a processor, a microprocessor, a controller, or a chip. For example, the functional units illustrated in the corresponding figures can be implemented and executed on a computer, a processor, a microprocessor, a controller, or a chip. In this case, information about the implementation (e.g., information about instructions) or an algorithm can be stored in a digital storage medium.

[0230] In addition, the decoding device and the encoding device of the embodiment of the present document can be included in a multimedia broadcast transceiver, a mobile communication terminal, a home theater video device, a digital theater video device, a surveillance camera, a video chat device, a real-time communication device such as video communication, a mobile streaming device, a storage medium, a portable camera, a video on demand (VoD) service provider, an over-the-top (OTT) video device, an Internet streaming service provider, a 3D video device, a virtual reality (VR) device, an augmented reality (AR) device, an image phone video device, a vehicle terminal (e.g., a vehicle (including an autonomous vehicle) terminal, an aircraft terminal, or a ship terminal) and a medical video device; and can be used to process image signals or data. For example, OTT video devices may include game consoles, Blueray players, networked TVs, home theater systems, smart phones, tablet PCs, and digital video recorders (DVRs).

[0231] In addition, the processing method of the embodiment of the present document can be generated in the form of a program executed by a computer and can be stored in a computer-readable recording medium. The multimedia data with a data structure according to the embodiment of the present document can also be stored in a computer-readable recording medium. Computer-readable recording media include all kinds of storage devices and distributed storage devices storing computer-readable data. Computer-readable recording media may include, for example, Blu-ray discs (BD), universal serial buses (USB), ROMs, PROMs, EPROMs, EEPROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, and optical data storage devices. Computer-readable recording media also include media implemented in the form of carrier waves (e.g., transmission on the Internet). In addition, the bit stream generated by the encoding method can be stored in a computer-readable recording medium, or can be transmitted through a wired or wireless communication network.

[0232] In addition, the embodiments of this document can be implemented as a computer program product based on a program code, and the program code can be executed on a computer according to the embodiments of this document. The program code can be stored on a computer readable carrier.

[0233] Fig.14 represents an example of a content streaming system that can be applied to the embodiments of this document.

[0234] refer to Fig.14 A content streaming system to which the embodiments of this document are applied may generally include an encoding server, a streaming server, a web server, a media storage device, a user device, and a multimedia input device.

[0235] The encoding server is used to compress the content input from the multimedia input device such as a smart phone, a camera, a camcorder, etc. into digital data, generate a bit stream, and transmit it to the streaming server. As another example, in the case where the multimedia input device such as a smart phone, a camera, a camcorder, etc. directly generates a bit stream, the encoding server can be omitted.

[0236] The bitstream may be generated by an encoding method or a bitstream generation method to which the embodiments of this document are applied, and the streaming server may temporarily store the bitstream in the process of sending or receiving the bitstream.

[0237] The streaming server transmits multimedia data to the user device through a network server based on the user's request, and the network server acts as a tool to inform the user of what services exist. When the user requests the service the user wants, the network server transfers the request to the streaming server, and the streaming server transmits the multimedia data to the user. In this regard, the content streaming system may include a separate control server, and in this case, the control server is used to control the command / response between the various devices in the content streaming system.

[0238] The streaming server may receive content from a media storage device and / or an encoding server. For example, in the case where the content is received from the encoding server, the content may be received in real time. In this case, the streaming server may store the bitstream for a predetermined period of time to smoothly provide a streaming service.

[0239] For example, the user device may include a mobile phone, a smart phone, a laptop computer, a digital broadcast terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation, a tablet PC, a tablet PC, an ultrabook, a wearable device (e.g., a watch-type terminal (smart watch), a glasses-type terminal (smart glasses), a head-mounted display (HMD)), a digital TV, a desktop computer, a digital signage, etc.

[0240] Each server in the content streaming system may be operated as a distributed server, and in this case, data received by each server may be processed in a distributed manner.

[0241] The claims in this specification can be combined in various ways. For example, the technical features in the method claims of this specification can be combined to be implemented or performed in a device, and the technical features in the device claims can be combined to be implemented or performed in a method. In addition, the technical features in the method claims and the device claims can be combined to be implemented or performed in a device. In addition, the technical features in the method claims and the device claims can be combined to be implemented or performed in a method.

Claims

1. A decoding device for image decoding, the decoding device comprising: Memory; as well as at least one processor, the at least one processor connected to the memory, the at least one processor configured to: Obtaining residual information from a bitstream; Generate a residual sample of the current block based on the residual information; and Generate a reconstructed sample of the current block based on the residual sample, The residual information includes a multiple transform selection (MTS) index and information about transform coefficients. wherein the residual samples are generated from the transform coefficients according to the information about the transform coefficients by using a transform kernel set, wherein, based on the MTS index, the transform core set is determined from the transform core set candidates, Wherein, based on context compilation, at least one bin among the bins of the bin string of the MTS index is derived, wherein the context coding is performed based on a value of a context index increment for the MTS index, and The bin string of the MTS index includes up to 4 bins, wherein the number of values ​​of the context index increment assigned to the bins derived by the context coding among the 4 bins is 1.

2. A coding device for image coding, the coding device comprising: Memory; as well as at least one processor, the at least one processor connected to the memory, the at least one processor configured to: deriving residual samples for the current block; generating residual information based on the residual samples for the current block; and encoding the image information including the residual information to generate a bit stream, The residual information includes a multiple transform selection (MTS) index and information about transform coefficients. wherein the information about the transform coefficients is generated from the residual samples by using a transform kernel set, The MTS index indicates the transform core set among the transform core set candidates. Wherein, based on context compilation, at least one bin among the bins of the bin string of the MTS index is derived, wherein the context coding is performed based on a value of a context index increment for the MTS index, and The bin string of the MTS index includes up to 4 bins, wherein the number of values ​​of the context index increment assigned to the bins derived by the context compilation among the 4 bins is 1.

3. A device for sending image data, the device comprising: at least one processor configured to obtain a bitstream for the image, wherein the bitstream is generated based on: deriving residual samples for a current block, generating residual information based on the residual samples for the current block, and encoding image information including the residual information to generate the bitstream; and a transmitter configured to transmit the data including the bit stream, The residual information includes a multiple transform selection (MTS) index and information about transform coefficients. wherein the information about the transform coefficients is generated from the residual samples by using a transform kernel set, The MTS index indicates the transform core set among the transform core set candidates. Wherein, based on context compilation, at least one bin among the bins of the bin string of the MTS index is derived, wherein the context coding is performed based on a value of a context index increment for the MTS index, and The bin string of the MTS index includes up to 4 bins, wherein the number of values ​​of the context index increment assigned to the bins derived by the context coding among the 4 bins is 1.

Images