Image coding method and device based on wide-angle intra-frame prediction and transformation

Abstract

The image decoding method according to the present document includes the following steps: obtaining intra prediction mode information and LFNST index; deriving the intra prediction mode of the current block based on the intra prediction mode information; when wide-angle intra prediction is applied to the current block, deriving a modified intra prediction mode by intra prediction mode remapping; deciding an LFNST set including an LFNST matrix based on the modified intra prediction mode; selecting one of the LFNST matrices based on the LFNST set and the LFNST index; and deriving a transform coefficient of the current block based on the selected LFNST matrix.

Description

Technical Field

[0001] The present disclosure relates to an image encoding technology, and more particularly, to a method and apparatus for encoding an image based on wide-angle intra prediction and transformation in an image encoding system. Background Art

[0002] Nowadays, the demand for high-resolution and high-quality images / videos such as 4K, 8K or higher ultra-high-definition (UHD) images / videos has been growing in various fields. As image / video data becomes higher resolution and higher quality, the amount of information or bit volume sent increases compared to traditional image data. Therefore, when using a medium such as a traditional wired / wireless broadband line to send image data or using an existing storage medium to store image / video data, its transmission cost and storage cost increase.

[0003] In addition, nowadays, interest and demand for immersive media such as virtual reality (VR) and artificial reality (AR) content or holograms are increasing, and broadcasting of images / videos having image characteristics different from real images such as game images is increasing.

[0004] Therefore, there is a need for an efficient image / video compression technology that effectively compresses and transmits or stores and reproduces information of high-resolution and high-quality images / videos having various characteristics as described above. Summary of the invention

[0005] Technical issues

[0006] A technical aspect of the present disclosure is to provide a method and an apparatus for increasing image encoding efficiency.

[0007] Another technical aspect of the present disclosure is to provide a method and apparatus for increasing the efficiency of transform index encoding.

[0008] Still another technical aspect of the present disclosure is to provide a method and apparatus for encoding a transform index in intra prediction to which wide-angle intra prediction is applied.

[0009] Technical Solution

[0010] According to an embodiment of the present disclosure, an image decoding method performed by a decoding device is provided. The method may include: obtaining intra prediction mode information and LFNST index from a bitstream; deriving an intra prediction mode of a current block based on the intra prediction mode information; when wide-angle intra prediction is applied to the current block, deriving a modified intra prediction mode by intra prediction mode remapping; determining an LFNST set including an LFNST matrix based on the modified intra prediction mode; selecting one of the LFNST matrices based on the LFNST set and the LFNST index; deriving a transform coefficient of the current block based on the selected LFNST matrix; and deriving a residual sample of the current block based on the transform coefficient.

[0011] Here, when the partitioning tree structure for the current block is a dual tree type, an LFNST index may be received for each of the luma block and the chroma block. When the intra prediction mode of the chroma block is a direct mode (DM) using the same mode as the intra prediction mode of the luma block, wide-angle intra prediction may be applied to the chroma block.

[0012] When any one of four LFNST sets is determined according to an intra prediction mode applied to a current block, an LFNST set in a case where the modified intra prediction mode is in a range from 67 to 80 and an LFNST set in a case where the modified intra prediction mode is in a range from -1 to -14 may be the same.

[0013] According to another embodiment of the present disclosure, there is provided an image encoding method performed by an encoding device. The method may include: deriving an intra prediction mode of a current block; deriving a modified intra prediction mode by intra prediction mode remapping based on whether wide-angle intra prediction is applied to the current block; deriving prediction samples based on the modified intra prediction mode; deriving residual samples of the current block based on the prediction samples; determining an LFNST set including an LFNST matrix based on the modified intra prediction mode; deriving transform coefficients of the current block based on the residual samples, the LFNST set and the LFNST matrix; and encoding image information including the intra prediction mode and an LFNST index indicating the LFNST matrix.

[0014] According to still another embodiment of the present disclosure, a digital storage medium storing image data including encoded image information and a bit stream generated according to an image encoding method performed by an encoding device may be provided.

[0015] According to yet another embodiment of the present disclosure, a digital storage medium storing image data including encoded image information and a bit stream so that a decoding device performs an image decoding method may be provided.

[0016] Technical Effects

[0017] According to the present disclosure, the overall image / video compression efficiency can be increased.

[0018] According to the present disclosure, the efficiency of transform index encoding can be increased.

[0019] Technical aspects of the present disclosure may provide a method and apparatus for encoding a transform index in intra prediction to which wide-angle intra prediction is applied.

[0020] The effects that can be obtained by the specific examples of the present disclosure are not limited to the effects listed above. For example, there may be various technical effects that can be understood by a person of ordinary skill in the relevant field or derived from the present disclosure. Therefore, the specific effects of the present disclosure are not limited to those explicitly described in the present disclosure, and may include various effects that can be understood or derived from the technical features of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 An example of a video / image encoding system to which the present disclosure can be applied is schematically illustrated.

[0022] Figure 2 is a diagram schematically illustrating a configuration of a video / image encoding device to which the present disclosure can be applied.

[0023] Figure 3 is a diagram schematically illustrating a configuration of a video / image decoding device to which the present disclosure can be applied.

[0024] Figure 4 is a diagram illustrating 67 intra prediction modes according to an embodiment of the present disclosure.

[0025] Figure 5 is a diagram illustrating a wide-angle intra prediction mode according to an embodiment of the present disclosure.

[0026] Figure 6 is a diagram illustrating intra prediction of a non-square block according to an embodiment of the present disclosure.

[0027] Figure 7 is a diagram illustrating a DM applicable when deriving an intra prediction mode of a chroma block according to an embodiment of the present disclosure.

[0028] Figure 8 is a diagram illustrating a CCLM applicable when deriving an intra prediction mode of a chroma block according to an embodiment.

[0029] Fig. 9 Multiple transformation schemes according to embodiments of the present disclosure are schematically illustrated.

[0030] Fig.10 is a diagram illustrating an RST according to an embodiment of the present disclosure.

[0031] Fig.11 is a flowchart illustrating the operation of a video decoding device according to an embodiment of the present disclosure.

[0032] Fig.12 is a flowchart illustrating the operation of a video encoding device according to an embodiment of the present disclosure.

[0033] Fig.13 The structure of a content streaming system to which the present disclosure is applied is illustrated. DETAILED DESCRIPTION

[0034] Although the present disclosure may be susceptible to various modifications and includes various embodiments, its specific embodiments have been shown by way of example in the accompanying drawings, and will now be described in detail. However, this is not intended to limit the present disclosure to the specific embodiments disclosed herein. The terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the technical ideas of the present disclosure. Unless the context clearly indicates otherwise, the singular form may include the plural form. Terms such as "including" and "having" are intended to indicate the presence of features, numbers, steps, operations, elements, components, or combinations thereof used in the following description, and therefore should not be understood as precluding the possibility of the presence or addition of one or more different features, numbers, steps, operations, elements, components, or combinations thereof.

[0035] In addition, for the convenience of describing the different characteristic functions, each component on the drawings described herein is illustrated independently, however, it is not intended that each component is implemented by separate hardware or software. For example, any two or more of these components can be combined to form a single component, and any single component can be divided into multiple components. Embodiments in which components are combined and / or divided will fall within the scope of the patent rights of the present disclosure as long as they do not depart from the essence of the present disclosure.

[0036] Hereinafter, preferred embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings. In addition, in the accompanying drawings, the same reference numerals are used for the same components, and repeated description of the same components will be omitted.

[0037] This document relates to video / image coding. For example, the methods / examples disclosed in this document may relate to the VVC (Versatile Video Coding) standard (ITU-T Rec. H.266), the next generation video / image coding standard after VVC, or other video coding related standards (e.g., HEVC (High Efficiency Video Coding) standard (ITU-T Rec. H.265), EVC (Basic Video Coding) standard, AVS2 standard, etc.).

[0038] In this document, various embodiments related to video / image encoding may be provided, and unless otherwise specified, the embodiments may be combined with each other and performed.

[0039] In this document, video can refer to a collection of a series of images over a period of time. Generally, a picture refers to a unit of an image representing a specific time region, and a slice / tile is a unit that constitutes a part of a picture. A slice / tile may include one or more coding tree units (CTUs). A picture may consist of one or more slices / tiles. A picture may consist of one or more tile groups. A tile group may include one or more tiles.

[0040] A pixel or a picture element (pel) may refer to the smallest unit constituting a picture (or image). In addition, a "sample" may be used as a term corresponding to a pixel. A sample may generally represent a pixel or a pixel value, and may represent only a pixel / pixel value of a luminance component or only a pixel / pixel value of a chrominance component. Alternatively, a sample may mean a pixel value in a spatial domain, or when the pixel value is transformed into a frequency domain, it may mean a transform coefficient in a frequency domain.

[0041] A unit may represent a basic unit of image processing. A unit may include at least one of a specific region and information related to the region. A unit may include a luminance block and two chrominance (e.g., cb, cr) blocks. Depending on the situation, terms such as unit, block, region, etc. may be used interchangeably. In general, an M×N block may include a set (or array) of samples (or sample arrays) or transform coefficients consisting of M columns and N rows.

[0042] In this document, the terms " / " and "," should be interpreted as indicating "and / or". For example, the expression "A / B" may mean "A and / or B". In addition, "A, B" may mean "A and / or B". In addition, "A / B / C" may mean "at least one of A, B, and / or C". In addition, "A / B / C" may mean "at least one of A, B, and / or C".

[0043] In addition, in this document, the term "or" should be interpreted as indicating "and / or". For example, the expression "A or B" may include 1) only A, 2) only B, and / or 3) both A and B. In other words, the term "or" in this document should be interpreted as indicating "additionally or alternatively".

[0044] In the present disclosure, "at least one of A and B" may mean "only A", "only B", or "both A and B". In addition, in the present disclosure, the expression "at least one of A or B" or "at least one of A and / or B" may be interpreted as "at least one of A and B".

[0045] Furthermore, in the present disclosure, “at least one of A, B, and C” may mean “only A”, “only B”, “only C”, or “any combination of A, B, and C”. Furthermore, “at least one of A, B, or C” or “at least one of A, B and / or C” may mean “at least one of A, B, and C”.

[0046] In addition, the brackets used in the present disclosure may mean "for example". Specifically, when the indication is "prediction (intra-frame prediction)", it may mean that "intra-frame prediction" is proposed as an example of "prediction". In other words, the "prediction" of the present disclosure is not limited to "intra-frame prediction", and "intra-frame prediction" is proposed as an example of "prediction". In addition, when the indication is "prediction (ie, intra-frame prediction)", this may also mean that "intra-frame prediction" is proposed as an example of "prediction".

[0047] Technical features described separately in one drawing in the present disclosure may be implemented separately or may be implemented simultaneously.

[0048] Figure 1 An example of a video / image encoding system to which the present disclosure can be applied is schematically illustrated.

[0049] Reference Figure 1 The video / image coding system may include a first device (source device) and a second device (receiving device). The source device may transmit the coded video / image information or data to the receiving device in the form of a file or stream via a digital storage medium or a network.

[0050] The source device may include a video source, an encoding device, and a transmitter. The receiving device may include a receiver, a decoding device, and a renderer. The encoding device may be referred to as a video / image encoding device, and the decoding device may be referred to as a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display, and the display may be configured as a separate device or an external component.

[0051] The video source can obtain the video / image by capturing, synthesizing or generating the process of the video / image. The video source may include a video / image capturing device and / or a video / image generating device. The video / image capturing device may include, for example, one or more cameras, a video / image archive including previously captured videos / images, etc. The video / image generating device may include, for example, a computer, a tablet computer, and a smart phone, and may (electronically) generate the video / image. For example, a virtual video / image may be generated by a computer, etc. In this case, the video / image capturing process may be replaced by the process of generating the relevant data.

[0052] The encoding device can encode the input video / image. The encoding device can perform a series of processes such as prediction, transformation and quantization for compression and coding efficiency. The encoded data (encoded video / image information) can be output in the form of a bitstream.

[0053] The transmitter may transmit the encoded video / image information or data output in the form of a bit stream to a receiver of a receiving device in the form of a file or stream through a digital storage medium or a network. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitter may include an element for generating a media file in a predetermined file format, and may include an element for transmitting through a broadcast / communication network. The receiver may receive / extract a bit stream and transmit the received / extracted bit stream to a decoding device.

[0054] The decoding device may decode a video / image by performing a series of processes such as dequantization, inverse transformation, prediction, etc. corresponding to the operation of the encoding device.

[0055] The renderer can render the decoded video / image, and the rendered video / image can be displayed through a display.

[0056] Figure 2 is a diagram schematically illustrating a configuration of a video / image encoding device to which the present disclosure can be applied. Hereinafter, the so-called video encoding device may include an image encoding device.

[0057] Reference Figure 2 , the encoding device 200 may include an image divider 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 may include an inter-frame predictor 221 and an intra-frame predictor 222. The residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234, and an inverse transformer 235. The residual processor 230 may further include a subtractor 231. The adder 250 may be referred to as a reconstructor or a reconstructed block generator. According to an embodiment, the image divider 210, the predictor 220, the residual processor 230, the entropy encoder 240, the adder 250, and the filter 260 described above may be composed of one or more hardware components (e.g., an encoder chipset or processor). In addition, the memory 270 may include a decoded picture buffer (DPB) and may be composed of a digital storage medium. The hardware component may further include a memory 270 as an internal / external component.

[0058] The image divider 210 may divide the input image (or picture or frame) input to the encoding device 200 into one or more processing units. As an example, the processing unit may be referred to as a coding unit (CU). In this case, starting from a coding tree unit (CTU) or a maximum coding unit (LCU), the coding unit may be recursively divided according to a quadtree binary tree ternary tree (QTBTTT) structure. For example, based on a quadtree structure, a binary tree structure, and / or a ternary tree structure, a coding unit may be divided into a plurality of coding units with a deeper depth. In this case, for example, a quadtree structure may be applied first, and a binary tree structure and / or a ternary tree structure may be applied later. Alternatively, a binary tree structure may be applied first. The encoding process according to the present disclosure may be performed based on a final coding unit that is not further divided. In this case, based on the coding efficiency according to the image characteristics, the maximum coding unit may be directly used as the final coding unit. Alternatively, the coding unit may be recursively divided into coding units of a deeper depth as needed, whereby a coding unit of the best size may be used as the final coding unit. Here, the encoding process may include processes such as prediction, transformation, and reconstruction, which will be described later. As another example, the processing unit may further include a prediction unit (PU) or a transform unit (TU). In this case, the prediction unit and the transform unit may be separated or divided from the above-mentioned final coding unit. The prediction unit may be a unit for sample prediction, and the transform unit may be a unit for deriving a transform coefficient and / or a unit for deriving a residual signal from a transform coefficient.

[0059] Unit and terms such as block, region, etc. may be used to replace each other depending on the situation. In general, an M×N block may represent a set of samples or transform coefficients consisting of M columns and N rows. A sample may generally represent a pixel or a pixel value, and may represent only a pixel / pixel value of a luminance component, or only a pixel / pixel value of a chrominance component. A sample may be used as a term corresponding to a pixel or a pel of a picture (or image).

[0060] The subtractor 231 subtracts the prediction signal (prediction block, prediction sample array) output from the predictor 220 from the input image signal (original block, original sample array) to generate a residual signal (residual block, residual sample array), and the generated residual signal is sent to the transformer 232. The predictor 220 can perform prediction on the processing target block (hereinafter referred to as the "current block"), and can generate a prediction block including the prediction samples of the current block. The predictor 220 can determine whether to apply intra-frame prediction or inter-frame prediction based on the current block or CU. As discussed later in the description of each prediction mode, the predictor can generate various information related to the prediction, such as prediction mode information, and send the generated information to the entropy encoder 240. Information about the prediction can be encoded in the entropy encoder 240 and output in the form of a bitstream.

[0061] The intra-frame predictor 222 can predict the current block by referring to the samples in the current picture. Depending on the prediction mode, the reference sample can be located near the current block or separated from the current block. In intra-frame prediction, the prediction mode may include a variety of non-directional modes and a variety of directional modes. The non-directional mode may include, for example, a DC mode and a plane mode. Depending on the level of detail of the prediction direction, the directional mode may include, for example, 33 directional prediction modes or 65 directional prediction modes. However, this is only an example, and more or fewer directional prediction modes may be used depending on the setting. The intra-frame predictor 222 may determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring blocks.

[0062] The inter-frame predictor 221 may derive a prediction block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. At this time, in order to reduce the amount of motion information sent in the inter-frame prediction mode, the motion information may be predicted based on a block, a sub-block, or a sample based on the correlation of the motion information between the neighboring block and the current block. The motion information may include a motion vector and a reference picture index. The motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter-frame prediction, the neighboring blocks may include spatial neighboring blocks existing in the current picture and temporal neighboring blocks existing in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same as or different from each other. The temporal neighboring block may be referred to as a collocated reference block, a collocated CU (colCU), etc., and the reference picture including the temporal neighboring block may be referred to as a collocated picture (colPic). For example, the inter-frame predictor 221 may configure a motion information candidate list based on the neighboring blocks, and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Inter-frame prediction may be performed based on various prediction modes. For example, in the case of the skip mode and the merge mode, the inter-frame predictor 221 can use the motion information of the neighboring block as the motion information of the current block. In the skip mode, unlike the merge mode, the residual signal cannot be sent. In the case of the motion information prediction (motion vector prediction, MVP) mode, the motion vector of the neighboring block can be used as a motion vector predictor, and the motion vector of the current block can be indicated by signaling the motion vector difference.

[0063] The predictor 220 can generate a prediction signal based on various prediction methods. For example, the predictor can apply intra prediction or inter prediction to the prediction of a block, and can also apply intra prediction and inter prediction at the same time. This can be called combined inter and intra prediction (CIIP). In addition, the predictor can be based on an intra-block copy (IBC) prediction mode or a palette mode to perform prediction on the block. The IBC prediction mode or the palette mode can be used for content image / video encoding such as games such as screen content coding (SCC). Although IBC basically performs prediction in the current block, its execution method is similar to inter prediction in that it derives a reference block in the current block. That is, IBC can use at least one of the inter prediction techniques described in the present disclosure.

[0064] The prediction signal generated by the inter-frame predictor 221 and / or the intra-frame predictor 222 can be used to generate a reconstruction signal or to generate a residual signal. The transformer 232 can generate a transform coefficient by applying a transform technique to the residual signal. For example, the transform technique may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-Loève transform (KLT), a graph-based transform (GBT), or a conditional nonlinear transform (CNT). Here, GBT means a transform obtained from a curve graph when the relationship information between pixels is represented by a curve graph. CNT refers to a transform obtained based on a prediction signal generated using all previously reconstructed pixels. In addition, the transform process can be applied to square pixel blocks of the same size, or can be applied to blocks of variable size rather than square blocks.

[0065] The quantizer 233 may quantize the transform coefficients and send them to the entropy encoder 240, and the entropy encoder 240 may encode the quantized signal (information about the quantized transform coefficients) and output the encoded signal in the bitstream. The information about the quantized transform coefficients may be referred to as residual information. The quantizer 233 may rearrange the quantized transform coefficients of the block type into a one-dimensional vector form based on the coefficient scanning order, and generate information about the quantized transform coefficients based on the quantized transform coefficients in the one-dimensional vector form. The entropy encoder 240 may perform various encoding methods such as, for example, exponential Golomb, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), etc. The entropy encoder 240 may encode the information required for video / image reconstruction other than the quantized transform coefficients (e.g., the value of the syntax element, etc.) together or separately. The encoded information (e.g., the encoded video / image information) may be transmitted or stored on a unit basis of a network abstraction layer (NAL) in the form of a bitstream. The video / image information may also include information about various parameter sets such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), a video parameter set (VPS), etc. In addition, the video / image information may also include conventional constraint information. In the present disclosure, information and / or syntax elements sent from an encoding device to / signaled to a decoding device may be included in the video / image information. The video / image information may be encoded by the above-mentioned encoding process and included in a bitstream. The bitstream may be transmitted over a network or stored in a digital storage medium. Here, the network may include a broadcast network, a communication network, and / or the like, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmitter (not shown) that transmits a signal output from the entropy encoder 240 or a memory (not shown) that stores it may be configured as an internal / external element of the encoding device 200, or the transmitter may be included in the entropy encoder 240.

[0066] The quantized transform coefficients output from the quantizer 233 can be used to generate a prediction signal. For example, by applying dequantization and inverse transformation to the quantized transform coefficients using the dequantizer 234 and the inverse transformer 235, the residual signal (residual block or residual sample) can be reconstructed. The adder 155 adds the reconstructed residual signal to the prediction signal output from the inter-frame predictor 221 or the intra-frame predictor 222, so that a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) can be generated. When there is no residual for processing the current block as in the case of applying the jump mode, the prediction block can be used as a reconstructed block. The adder 250 can be referred to as a reconstructor or a reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next processed current block in the current picture, and as described later, can be used for inter-frame prediction of the next picture performed by filtering.

[0067] Furthermore, in the picture encoding and / or reconstruction process, luminance mapping with chroma scaling (LMCS) may be applied.

[0068] The filter 260 can improve the subjective / objective video quality by applying filtering to the reconstructed signal. For example, the filter 260 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and the modified reconstructed picture can be stored in the memory 270, especially in the DPB of the memory 270. Various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive ring filter, bilateral filter, etc. As discussed later in the description of each filtering method, the filter 260 can generate various information related to filtering and send the generated information to the entropy encoder 240. Information about filtering can be encoded in the entropy encoder 240 and output in the form of a bitstream.

[0069] The modified reconstructed picture transmitted to the memory 270 may be used as a reference picture in the inter-frame predictor 221. Accordingly, the encoding device can avoid prediction mismatch in the encoding device 100 and the decoding device when applying inter-frame prediction, and can also improve encoding efficiency.

[0070] The memory 270DPB may store the modified reconstructed picture so as to use it as a reference picture in the inter-frame predictor 221. The memory 270 may store the motion information of the block in the current picture from which the motion information has been derived (or encoded) and / or the motion information of the block in the reconstructed picture. The stored motion information may be sent to the inter-frame predictor 221 to be used as the motion information of the neighboring block or the motion information of the temporal neighboring block. The memory 270 may store the reconstructed samples of the reconstructed block in the current picture and send them to the intra-frame predictor 222.

[0071] Figure 3is a diagram schematically illustrating a configuration of a video / image decoding device to which the present disclosure can be applied.

[0072] Reference Figure 3 , the video decoding device 300 may include an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 may include an inter-frame predictor 332 and an intra-frame predictor 331. The residual processor 320 may include a dequantizer 321 and an inverse transformer 322. According to an embodiment, the entropy decoder 310, the residual processor 320, the predictor 330, the adder 340, and the filter 350 described above may be composed of one or more hardware components (e.g., a decoder chipset or a processor). In addition, the memory 360 may include a decoded picture buffer (DPB) and may be composed of a digital storage medium. The hardware component may also include a memory 360 as an internal / external component.

[0073] When a bit stream including video / image information is input, the decoding device 300 can Figure 2 The image is reconstructed correspondingly to the processing of the video / image information in the encoding device. For example, the decoding device 300 can derive the unit / block based on the information related to the block segmentation obtained from the bit stream. The decoding device 300 can perform decoding by using the processing unit applied in the encoding device. Therefore, the decoding processing unit can be, for example, a coding unit, which can be divided into a quadtree structure, a binary tree structure and / or a ternary tree structure using a coding tree unit or a maximum coding unit. One or more transformation units can be derived from the coding unit. And, the reconstructed image signal decoded and output by the decoding device 300 can be reproduced by a reproducer.

[0074] The decoding device 300 may receive the Figure 2The signal output by the encoding device of the encoding device, and the received signal can be decoded by the entropy decoder 310. For example, the entropy decoder 310 can parse the bitstream to derive the information (e.g., video / image information) required for image reconstruction (or picture reconstruction). The video / image information may also include information about various parameter sets such as adaptive parameter set (APS), picture parameter set (PPS), sequence parameter set (SPS), video parameter set (VPS), etc. In addition, the video / image information may also include conventional constraint information. The decoding device may further decode the picture based on the information about the parameter set and / or the conventional constraint information. In the present disclosure, the signaling / receiving information and / or syntax elements to be described later can be decoded and obtained from the bitstream through the decoding process. For example, the entropy decoder 310 can decode the information in the bitstream based on a coding method such as exponential Golomb coding, CAVLC, CABAC, etc., and can output the values ​​of the syntax elements required for image reconstruction and the quantized values ​​of the transform coefficients of the residual. More specifically, the CABAC entropy decoding method can receive the bin corresponding to each syntax element in the bitstream, use the decoding target syntax element information and the decoding information of the neighboring and decoding target blocks or the information of the symbol / bin decoded in the previous step to determine the context model, predict the bin generation probability according to the determined context model and perform arithmetic decoding on the bin to generate the symbol corresponding to each syntax element value. Here, the CABAC entropy decoding method can update the context model using the information of the symbol / bin decoded by the context model for the next symbol / bin after determining the context model. The information about prediction among the information decoded in the entropy decoder 310 can be provided to the predictor (inter-frame predictor 332 and intra-frame predictor 331), and the residual value (ie, quantized transform coefficient) and the associated parameter information that have been entropy decoded in the entropy decoder 310 can be input to the residual processor 320. The residual processor 320 can derive a residual signal (residual block, residual sample, residual sample array). In addition, the information about filtering among the information decoded in the entropy decoder 310 can be provided to the filter 350. In addition, a receiver (not shown) receiving a signal output from the encoding device may also constitute the decoding device 300 as an internal / external element, and the receiver may be a component of the entropy decoder 310. In addition, the decoding device according to the present disclosure may be referred to as a video / image / picture encoding device, and the decoding device may be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include the entropy decoder 310, and the sample decoder may include at least one of the dequantizer 321, the inverse transformer 322, the adder 340, the filter 350, the memory 360, the inter-frame predictor 332, and the intra-frame predictor 331.

[0075] The dequantizer 321 may output the transform coefficient by dequantizing the quantized transform coefficient. The dequantizer 321 may rearrange the quantized transform coefficient into a two-dimensional block form. In this case, the rearrangement may be performed based on the order of coefficient scanning that has been performed in the encoding device. The dequantizer 321 may perform dequantization on the quantized transform coefficient using a quantization parameter (e.g., quantization step size information) and obtain the transform coefficient.

[0076] The inverse transformer 322 obtains a residual signal (residual block, residual sample array) by performing inverse transformation on the transformation coefficients.

[0077] The predictor may perform prediction on the current block and generate a prediction block including prediction samples for the current block. The predictor may determine whether to apply intra prediction or inter prediction to the current block based on information on prediction output from the entropy decoder 310, and may specifically determine an intra / inter prediction mode.

[0078] The predictor can generate a prediction signal based on various prediction methods. For example, the predictor can apply intra prediction or inter prediction to the prediction of a block, and can also apply intra prediction and inter prediction at the same time. This can be called combined inter and intra prediction (CIIP). In addition, the predictor can perform intra-block copying (IBC) for the prediction of the block. Intra-block copying can be used for content image / video coding such as games such as screen content coding (SCC). Although IBC basically performs prediction in the current block, its execution method is similar to inter prediction in that it derives a reference block in the current block. That is, IBC can use at least one of the inter prediction techniques described in the present disclosure.

[0079] The intra-frame predictor 331 can predict the current block by referring to samples in the current picture. Depending on the prediction mode, the reference sample can be located near the current block or separated from the current block. In intra-frame prediction, the prediction mode can include multiple non-directional modes and multiple directional modes. The intra-frame predictor 331 can determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring block.

[0080] The inter-frame predictor 332 may derive a prediction block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. At this time, in order to reduce the amount of motion information sent in the inter-frame prediction mode, the motion information may be predicted based on a block, a sub-block, or a sample based on the correlation of the motion information between the neighboring block and the current block. The motion information may include a motion vector and a reference picture index. The motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter-frame prediction, the neighboring blocks may include spatial neighboring blocks existing in the current picture and temporal neighboring blocks existing in the reference picture. For example, the inter-frame predictor 332 may configure a motion information candidate list based on neighboring blocks, and derive a motion vector and / or a reference picture index of the current block based on the received candidate selection information. Inter-frame prediction may be performed based on various prediction modes, and information about the prediction may include information indicating a mode of inter-frame prediction for the current block.

[0081] The adder 340 can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the obtained residual signal to the prediction signal (prediction block, prediction sample array) output from the predictor 330. When there is no residual for the processing target block as in the case of applying the skip mode, the prediction block can be used as the reconstructed block.

[0082] The adder 340 may be referred to as a reconstructor or a reconstructed block generator. The generated reconstructed signal may be used for intra prediction of the next processing target block in the current block, and as described later, may be output through filtering or used for inter prediction of the next picture.

[0083] Furthermore, in the picture decoding process, luma mapping with chroma scaling (LMCS) may be applied.

[0084] The filter 350 can improve the subjective / objective video quality by applying filtering to the reconstructed signal. For example, the filter 350 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and the modified reconstructed picture can be sent to the memory 360, especially to the DPB of the memory 360. The various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive ring filter, bilateral filter, etc.

[0085] The (modified) reconstructed picture stored in the DPB of the memory 360 may be used as a reference picture in the inter-frame predictor 332. The memory 360 may store the motion information of the block in the current picture from which the motion information has been derived (or decoded) and / or the motion information of the block in the reconstructed picture. The stored motion information may be sent to the inter-frame predictor 332 to be used as the motion information of the neighboring block or the motion information of the temporal neighboring block. The memory 360 may store the reconstructed samples of the reconstructed block in the current picture and send them to the intra-frame predictor 331.

[0086] In the present specification, the examples described in the predictor 330, the dequantizer 321, the inverse transformer 322, and the filter 350 of the decoding device 300 may be similarly or correspondingly applied to the predictor 220, the dequantizer 234, the inverse transformer 235, and the filter 260 of the encoding device 200, respectively.

[0087] As described above, prediction is performed in order to improve the compression efficiency when performing video encoding. Accordingly, a prediction block including prediction samples for a current block as a coding target block can be generated. Here, the prediction block includes prediction samples in a spatial domain (or a pixel domain). The prediction block can be derived identically in an encoding device and a decoding device, and the encoding device can improve the image coding efficiency by signaling to the decoding device information (residual information) about the residual between the original block and the prediction block rather than the original sample value of the original block itself. The decoding device can derive a residual block including residual samples based on the residual information, generate a reconstructed block including reconstructed samples by adding the residual block to the prediction block, and generate a reconstructed picture including the reconstructed block.

[0088] The residual information may be generated by a transform process and a quantization process. For example, the encoding device may derive a residual block between the original block and the prediction block, derive a transform coefficient by performing a transform process on the residual samples (residual sample array) included in the residual block, and derive a quantized transform coefficient by performing a quantization process on the transform coefficient, so that it can signal the associated residual information to the decoding device (through a bitstream). Here, the residual information may include value information, position information, transform technology, transform kernel, quantization parameter, etc. of the quantized transform coefficient. The decoding device may perform a quantization / dequantization process based on the residual information and derive a residual sample (or residual sample block). The decoding device may generate a reconstructed block based on the prediction block and the residual block. The encoding device may derive a residual block by dequantizing / inverse transforming the quantized transform coefficient so as to serve as a reference for inter-frame prediction of the next picture, and may generate a reconstructed picture based on this.

[0089] Figure 4 is a diagram illustrating 67 intra prediction modes according to an embodiment of the present disclosure.

[0090] In the intra prediction according to the embodiment of the present disclosure, the Figure 4 And the 67 intra prediction modes in Table 1.

[0091] [Table 1]

[0092]

[0093] This is an extension of the existing 35 angle modes to 67 angle modes for intra-frame coding and more accurate prediction of high-resolution images. Figure 4 The arrows indicated by dotted lines in indicate 32 angle modes newly added among the 35 angle modes.

[0094] The intra plane (INTRA_PLANAR) mode and the intra DC (INTRA_DC) mode are the same as the existing intra plane mode and the existing intra DC mode. The added 32 angle modes can be applied to all block sizes and can be applied to both intra encoding and decoding of luma and chroma components.

[0095] Reference Figure 4 , based on the intra prediction mode 34 having the upper left diagonal prediction direction, the intra prediction mode can be divided into an intra prediction mode having a horizontal directionality and an intra prediction mode having a vertical directionality. Figure 4 In , H and V indicate horizontal directivity and vertical directivity, respectively, and numbers -32 to 32 indicate displacements of 1 / 32 units on sample grid positions. These numbers may represent offsets for mode index values. Intra-prediction modes 2 to 33 have horizontal directivity, and intra-prediction modes 34 to 66 have vertical directivity. Strictly speaking, intra-prediction mode 34 may be considered to be neither horizontal nor vertical, but may be classified as belonging to horizontal directivity when determining a transform set for a secondary transform. This is because the input data is transposed for a vertical orientation mode symmetrical based on intra-prediction mode 34, and an input data alignment method for a horizontal mode is used for intra-prediction mode 34. Transposing the input data means switching rows and columns of two-dimensional M×N block data to N×M data. Intra-prediction mode 18 and intra-prediction mode 50 may represent a horizontal intra-prediction mode and a vertical intra-prediction mode, respectively, and intra-prediction mode 2 may be referred to as an upper right diagonal intra-prediction mode because intra-prediction mode 2 has a left reference pixel and performs prediction in the upper right direction. Similarly, intra-prediction mode 34 may be referred to as a bottom-right diagonal intra-prediction mode, and intra-prediction mode 66 may be referred to as a bottom-left diagonal intra-prediction mode.

[0096] As described above, the prediction direction of the intra prediction may be defined as 45 degrees to -135 degrees in the clockwise direction. However, when the current block is a non-square block, some existing directional intra prediction modes may be adaptively replaced with a wide angle intra prediction (WAIP) mode.

[0097] Figure 5 is a diagram illustrating a wide-angle intra prediction mode according to an embodiment of the present disclosure.

[0098] When wide-angle intra prediction is applied, information about the existing intra prediction may be signaled and may be parsed and then remapped to the index of the wide-angle intra prediction mode. Therefore, the total number of intra prediction modes for a specific block (e.g., a non-square block of a specific size) may not be changed, that is, the total number of intra prediction modes is 67, and the intra prediction mode encoding for the specific block may not be changed.

[0099] Will refer to Figure 6 To describe why the foregoing wide-angle intra prediction method increases the coding efficiency of an image under certain circumstances. Figure 6 is a diagram illustrating intra prediction of a non-square block according to an embodiment of the present disclosure.

[0100] like Figure 6 As shown in (a) of FIG. 1 , when the width of the current prediction block is greater than its height, the upper reference pixel is usually close to the position inside the block to be predicted. Therefore, prediction can be performed more accurately in the lower left direction than in the upper right direction. On the contrary, when the height of the block is greater than its width, as shown in FIG. Figure 6 As shown in (b), the left reference pixel is usually close to the position inside the block to be predicted. Therefore, prediction can be performed more accurately in the upper right direction than in the lower left direction. Therefore, it can be advantageous to apply remapping to the index of the wide-angle intra prediction mode (i.e., mode index transformation).

[0101] Table 2 illustrates a process for deriving a modified intra mode by remapping the intra prediction mode to the wide-angle intra prediction mode.

[0102] [Table 2]

[0103]

[0104] Referring to Table 2, the variable predModeIntra specifying the intra prediction mode, the height and width of the transform block, etc. are used as input values ​​for the wide-angle intra prediction mode mapping process, and the output value is the modified intra prediction mode (predModeIntra). The height and width of the transform block or the coding block may be the height and width of the current block for remapping the intra prediction mode. Here, the variable whRatio reflecting the ratio between the width and the width may be set to Abs(log2(nW / nH)).

[0105] For non-square blocks, the intra prediction mode can be modified according to two separate cases.

[0106] First, if all conditions are met (1) the width of the current block is greater than the height, (2) the intra prediction mode before modification is equal to or greater than 2, and (3) the intra prediction mode is less than a value derived as (8+2*whRatio) when whRatio is greater than 1 and a value derived as 8 when whRatio is less than or equal to 1 (predModeIntra is less than (whRatio>1)? (8+2*whRatio): 8), the intra prediction mode is set to a value equal to predModeIntra+65.

[0107] In another case, if all conditions are met (1) the height of the current block is greater than the width, (2) the intra-frame prediction mode before modification is less than or equal to 66, and (3) the intra-frame prediction mode is greater than the value derived as (60-2*whRatio) when whRatio is greater than 1 and the value derived as 60 when whRatio is less than or equal to 1 (predModeIntra is greater than (whRatio>1)? (60-2*whRatio):60), the intra-frame prediction mode is set to be equal to the value of predModeIntra-67.

[0108] When the wide-angle intra prediction mode is applied with reference to the process shown in Table 2, the modes to be transformed according to the mode index are listed with respect to 67 modes as shown in Table 3.

[0109] [Table 3]

[0110] condition Alternative intra prediction modes W / H==2 Mode 2, 3, 4, 5, 6, 7 W / H>2 Mode 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 W / H==1 none H / W==1 / 2 Mode 61, 62, 63, 64, 65, 66 H / W<1 / 2 Mode 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66

[0111] The mode index for wide-angle intra prediction may be encoded as a mode index before transformation. For example, even when mode 2 is transformed to mode 67 in a 67-mode configuration, the previous mode index value 2 may be encoded.

[0112] When intra prediction is performed on the current block, prediction of the luminance component block (luminance block) of the current block and prediction of the chrominance component block (chrominance block) thereof may be performed. In this case, the intra prediction mode for the chrominance component (chrominance block) may be set separately from the intra prediction mode for the luminance component (luminance block).

[0113] In this specification, "chroma block", "chroma image", etc. may refer to the same meaning as chroma block, chroma image, etc., and thus chroma and chroma may be used interchangeably. Similarly, "luminance block", "luminance image", etc. may refer to the same meaning as luma block, luma image, etc., and thus luminance and luma may be used interchangeably.

[0114] In this specification, a "current chroma block" may refer to a chroma component block of a current block as a current coding unit, and a "current luminance block" may refer to a luminance component block of a current block as a current coding unit. Therefore, the current luminance block and the current chroma block correspond to each other. However, the current luminance block and the current chroma block do not always have the same block shape and the same number of blocks, and may have different block shapes and different numbers of blocks depending on the situation. In some cases, the current chroma block may correspond to a current luminance region, in which case the current luminance region may include at least one luminance block.

[0115] The intra prediction mode of the chroma component may be indicated based on the intra chroma prediction mode information, and the intra chroma prediction mode information may be signaled in the form of an intra_chroma_pred_mode syntax element. For example, the intra chroma prediction mode information may indicate one of a planar mode, a DC mode, a vertical mode, a horizontal mode, a derived mode (DM), and a CCLM mode. Here, when using Figure 4 Of the 67 intra prediction modes shown, planar mode may indicate intra prediction mode 0, DC mode may indicate intra prediction mode 1, vertical mode may indicate intra prediction mode 50, and horizontal mode may indicate intra prediction mode 18. DM may also be referred to as direct mode. CCLM may be referred to as LM.

[0116] DM and CCLM are related intra prediction modes for predicting chroma blocks using information about luma blocks. DM may refer to a mode in which the same intra prediction mode as that of the luma component is applied to the intra prediction mode of the chroma component. CCLM may refer to an intra prediction mode in which the reconstructed samples of the luma block are downsampled in the process of generating a prediction block for the chroma block, and then the samples derived by applying CCLM parameters α and β to the downsampled samples are used as prediction samples for the chroma block.

[0117] Figure 7is a diagram illustrating a DM applicable when deriving an intra prediction mode of a chroma block according to an embodiment of the present disclosure.

[0118] As described above, DM can be defined as a prediction mode that uses the intra-prediction direction of the luma block at the same position as the current chroma block as is. In this specification, a chroma intra-frame mode that directly uses the intra-frame prediction mode of the luma block (luma block) to derive a chroma intra-frame mode or a chroma intra-frame prediction mode that can be derived using a luma mode or based on a luma mode can be referred to as DM.

[0119] In the Joint Exploration Test Model (JEM) used in the Joint Video Exploration Team (JVET), DM, which is an existing single mode, is extended to multiple modes for use.

[0120] That is, when configuring the intra prediction mode for chroma images, you can choose to override Figure 7 The prediction mode of the block at the position shown is used as multiple DM modes and is used for intra prediction. Using one DM selected from multiple DMs for the luma block as a candidate for the intra prediction mode for the chroma block may be referred to as multi-DM signaling (MDMS).

[0121] The process for deriving DM may be as follows.

[0122] (1) Intra prediction modes for the center (Cr), top left (TL), top and bottom (TR), bottom left (BL), and bottom right (BR) positions of the luma block at the same position as the chroma block

[0123] (2) Intra prediction modes for the left (L), above (A), BL, upper right (AR), and upper left (AL) blocks that are neighbors of the current chroma block

[0124] (3) Planar mode, DC mode

[0125] (4) The previously selected directional mode (the directional mode selected in (1) and (2)) is -1 or +1.

[0126] (5) Vertical mode, horizontal mode, mode 2, 34, 66, 10, 26

[0127] (6) When the five prediction modes are not selected through the above process, the previously selected mode is copied and selected.

[0128] When there are multiple directional modes in (4), the first directional mode of -1 or +1 can be selected, and then the second directional mode of -1 or +1 can be selected. In this way, multiple directional modes of -1 or +1 can be selected in sequence, and five prediction modes can be finally selected as candidate modes through (6).

[0129] When there are 67 intra prediction modes, modes 10 and 26 that can be selected in (5) correspond to the prediction mode numbers.

[0130] Figure 8 is a diagram illustrating a CCLM applicable when deriving an intra prediction mode of a chroma block according to an embodiment.

[0131] In the present specification, a “reference sample template” may refer to a set of neighboring reference samples of a current chroma block used to predict a current chroma block. The reference sample template may be predefined, and information about the reference sample template may be signaled from the encoding device 200 to the decoding device 300.

[0132] Reference Figure 8 , a group of samples shaded in a line around the 4×4 block that is the current chroma block indicates the reference sample template. Figure 8 As shown in FIG. 1 , the reference sample template is arranged in reference samples of one line, and the reference sample area in the brightness area corresponding to the reference sample template is arranged in two lines.

[0133] In an embodiment, when performing intra-frame coding of a chrominance image in a joint exploration test model (JEM) used in the joint video exploration team (JVET), a cross-component linear model (CCLM) may be used. The CCLM is a method for predicting pixel values ​​of a chrominance image from pixel values ​​of a reconstructed luminance image and is based on the characteristic of high correlation between luminance and chrominance images.

[0134] The CCLM prediction of Cb and Cr chrominance images can be based on the following equation.

[0135] [Formula 1]

[0136] Pred C (i,j)=α·Rec′ L (i,j)+β

[0137] Here, pred C (i, j) represents the Cb or Cr chrominance image to be predicted, Rec L '(i, j) represents the reconstructed luminance image adjusted to the chrominance block size, and (i, j) represents the coordinates of the pixel. In the 4:2:0 color format, since the size of the luminance image is twice the size of the chrominance image, it is necessary to generate a Rec with the chrominance block size by downsampling. L ', so we can consider Rec L (2i, 2j) and neighboring pixels are used to use the chrominance image pred C The pixel of the brightness image of (i, j). Rec L'(i, j) can be called the downsampled luma sample.

[0138] For example, Rec can be derived using six neighboring pixels as shown below L '(i, j).

[0139] [Formula 2]

[0140] Rec′ L (x, y) = (2×Rec L (2x,2y)+2×Rec L (2x,2y+1)+Rec L (2x-1,2y)+Rec L (2x+1,2y)

[0141] +Rec L (2x-1,2y+1)+Rec L (2x+1,2y+1)+4)>>3

[0142] α and β represent the adjacent templates of Cb or Cr chrominance blocks and Figure 8 The cross-correlation and average difference between adjacent templates of the luminance block in the shadow area of ​​. For example, α and β are expressed by Equation 3.

[0143] [Formula 3]

[0144]

[0145] L(n) represents the adjacent reference samples and / or left adjacent samples of the luminance block corresponding to the current chrominance image, C(n) represents the adjacent reference samples and / or left adjacent samples of the current chrominance block to which encoding is currently applied, and (i, j) represents the pixel position. In addition, L(n) may represent the downsampled upper adjacent samples and / or left adjacent samples of the current luminance block. N may represent the total number of pixel pairs (luminance and chrominance) values ​​used for calculation of CCLM parameters, and may indicate a value that is twice the smaller of the width and height of the current chrominance block.

[0146] A picture may be divided into a sequence of coding tree units (CTUs). A CTU may correspond to a coding tree block (CTB). Alternatively, a CTU may include a coding tree block for luma samples and two coding tree blocks for corresponding chroma samples. Depending on whether the luma block and the corresponding chroma block have separate partitioning structures, the tree type may be classified as a single tree (SINGLE_TREE) or a dual tree (DUAL_TREE). A single tree may indicate that the chroma block has the same partitioning structure as the luma block, while a dual tree may indicate that the chroma component block has a partitioning structure different from that of the luma block.

[0147] Hereinafter, a transformation process related to an image encoding or decoding process of the present disclosure will be described.

[0148] Fig. 9 The multi-conversion technology according to the embodiment of the present disclosure is schematically illustrated.

[0149] Reference Fig. 9 , the converter can correspond to the aforementioned Figure 2 The converter in the encoding device, and the inverse converter may correspond to the aforementioned Figure 2 an inverse transformer in an encoding device, or Figure 3 An inverse transformer in a decoding device.

[0150] The transformer may derive (primary) transform coefficients by performing a primary transform based on the residual samples (residual sample array) in the residual block (S910). The primary transform may be referred to as a core transform. In this document, a primary transform may be based on a multi-transform selection (MTS), and when a multi-transform is used as a primary transform, it may be referred to as a multi-core transform.

[0151] The multi-core transform may refer to a method of performing transform using discrete cosine transform (DCT) type 2 and discrete sine transform (DST) type 7, DCT type 8, and / or DST type 1 in addition. That is, the multi-core transform may refer to a transform method of transforming a residual signal (or residual block) in the spatial domain into a transform coefficient (or primary transform coefficient) in the frequency domain based on a plurality of transform kernels selected from DCT type 2, DST type 7, DCT type 8, and DST type 1. In this article, the primary transform coefficient may be referred to as a temporary transform coefficient from the perspective of the transformer.

[0152] In other words, when a conventional transform method is applied, a transform coefficient can be generated by applying a transform from the spatial domain to the frequency domain to a residual signal (or residual block) based on DCT type 2. In contrast, when a multi-core transform is applied, a transform coefficient (or a primary transform coefficient) can be generated by applying a transform from the spatial domain to the frequency domain to a residual signal (or residual block) based on DCT type 2, DST type 7, DCT type 8, and / or DST type 1. In this document, DCT type 2, DST type 7, DCT type 8, and DST type 1 may be referred to as a transform type, a transform kernel, or a transform core. These DCT / DST transform types may be defined based on basis functions.

[0153] When performing multi-core transformation, a vertical transformation core and a horizontal transformation core for the current block may be selected from the transformation cores, a vertical transformation may be performed on the current block based on the vertical transformation core, and a horizontal transformation may be performed on the current block based on the horizontal transformation core. Here, the horizontal transformation may indicate a transformation of the horizontal component of the current block, and the vertical transformation may indicate a transformation of the vertical component of the current block. The vertical transformation core / horizontal transformation core may be adaptively determined based on the prediction mode and / or transformation index of the current block (CU or subblock) including the residual block.

[0154] In addition, according to the example, if a transformation is performed once by applying MTS, the mapping relationship of the transformation kernel can be set by setting a specific basis function to a predetermined value and combining the basis functions to be applied in the vertical transformation or the horizontal transformation. For example, when the horizontal transformation kernel is represented as trTypeHor and the vertical direction transformation kernel is represented as trTypeVer, trTypeHor or trTypeVer with a value of 0 can be set to DCT2, trTypeHor or trTypeVer with a value of 1 can be set to DST7, and trTypeHor or trTypeVer with a value of 2 can be set to DCT8.

[0155] In this case, the MTS index information may be encoded and signaled to the decoding device to indicate any one of the plurality of transform kernel sets. For example, an MTS index of 0 may indicate that both trTypeHor and trTypeVer values ​​are 0, an MTS index of 1 may indicate that both trTypeHor and trTypeVer values ​​are 1, an MTS index of 2 may indicate that both trTypeHor and trTypeVer values ​​are 2, an MTS index of 3 may indicate that both trTypeHor and trTypeVer values ​​are 1, and an MTS index of 4 may indicate that both trTypeHor and trTypeVer values ​​are 2.

[0156] In one example, the transformation core set according to the MTS index information is shown in the following table.

[0157] [Table 4]

[0158] tu_mts_idx[x0][y0] 0 1 2 3 4 trTypeHor 0 1 2 1 2 trTypeVer 0 1 1 2 2

[0159] The transformer may derive modified (secondary) transform coefficients (S920) by performing a secondary transform based on the (primary) transform coefficients. A primary transform is a transform from the spatial domain to the frequency domain, while a secondary transform refers to a transform into a more compressive representation by using the correlation between the (primary) transform coefficients. The secondary transform may include an inseparable transform. In this case, the secondary transform may be referred to as an inseparable secondary transform (NSST) or a pattern-dependent inseparable secondary transform (MDNSST). An inseparable secondary transform may represent a transform that generates a modified transform coefficient (or secondary transform coefficient) for a residual signal by performing a secondary transform on the (primary) transform coefficients derived by the primary transform based on an inseparable transform matrix. At this time, the vertical transform and the horizontal transform may not be applied separately to the (primary) transform coefficients (or the horizontal transform and the vertical transform may not be applied independently), but the transform matrix may be applied once based on the inseparable transform. In other words, the inseparable secondary transform may represent a transform method in which the vertical component and the horizontal component of the (primary) transform coefficient are not separated, and, for example, the two-dimensional signal (transform coefficient) is rearranged into a one-dimensional signal through a certain determined direction (e.g., a row first direction or a column first direction), and then the modified transform coefficient (or secondary transform coefficient) is generated based on the inseparable transform matrix. For example, according to the row priority order, the M×N blocks are arranged in a row in the order of the first row, the second row, ... and the Nth row. According to the column priority order, the M×N blocks are arranged in a row in the order of the first column, the second column, ... and the Nth column. The inseparable secondary transform may be applied to the upper left region of a block (hereinafter, referred to as a transform coefficient block) configured with the (primary) transform coefficient. For example, if the width (W) and height (H) of the transform coefficient block are both equal to or greater than 8, an 8×8 inseparable secondary transform may be applied to the upper left 8×8 region of the transform coefficient block. In addition, if the width (W) and height (H) of the transform coefficient block are both equal to or greater than 4, and the width (W) or height (H) of the transform coefficient block is less than 8, a 4×4 inseparable secondary transform may be applied to the upper left min(8, W)×min(8, H) region of the transform coefficient block. However, the embodiment is not limited thereto, and for example, even if only the condition that the width (W) or height (H) of the transform coefficient block is equal to or greater than 4 is satisfied, a 4×4 inseparable secondary transform may be applied to the upper left min(8, W)×min(8, H) region of the transform coefficient block.

[0160] Specifically, for example, if a 4×4 input block is used, the non-separable secondary transform may be performed as follows.

[0161] A 4×4 input block X can be represented as follows.

[0162] [Formula 4]

[0163]

[0164] If X is represented as a vector, then the vector It can be expressed as follows.

[0165] [Formula 5]

[0166]

[0167] In Equation 5, the vector is a one-dimensional vector obtained by rearranging the two-dimensional block X of Equation 4 according to the row-major order.

[0168] In this case, the non-separable quadratic transform can be calculated as follows.

[0169] [Formula 6]

[0170]

[0171] In this formula, denotes a transform coefficient vector and T denotes a 16×16 (non-separable) transform matrix.

[0172] By using the above formula 6, the 16×1 transform coefficient vector can be derived And the vector can be divided into two parts by scanning order (horizontally, vertically, diagonally, etc.) Reorganized into 4×4 blocks. However, the above calculation is an example, and Hypercube-Givens Transform (HyGT) or the like may also be used for the calculation of the inseparable secondary transform in order to reduce the calculation complexity of the inseparable secondary transform.

[0173] Furthermore, in the inseparable secondary transform, the transform kernel (or transform core, transform type) may be selected to be mode-dependent. In this case, the mode may include an intra prediction mode and / or an inter prediction mode.

[0174] As described above, an inseparable secondary transform may be performed based on an 8×8 transform or a 4×4 transform determined based on the width (W) and height (H) of the transform coefficient block. An 8×8 transform refers to a transform that can be applied to an 8×8 region included in a transform coefficient block when both W and H are equal to or greater than 8, and the 8×8 region may be the upper left 8×8 region in the transform coefficient block. Similarly, a 4×4 transform refers to a transform that can be applied to a 4×4 region included in a transform coefficient block when both W and H are equal to or greater than 4, and the 4×4 region may be the upper left 4×4 region in the transform coefficient block. For example, an 8×8 transform kernel matrix may be a 64×64 / 16×64 matrix, and a 4×4 transform kernel matrix may be a 16×16 / 8×16 matrix.

[0175] Here, in order to select the mode-dependent transform cores, two inseparable secondary transform cores may be configured for each transform set of inseparable secondary transforms for both the 8×8 transform and the 4×4 transform, and there may be four transform sets. That is, four transform sets may be configured for the 8×8 transform, and four transform sets may be configured for the 4×4 transform. In this case, each of the four transform sets for the 8×8 transform may include two 8×8 transform cores, and each of the four transform sets for the 4×4 transform may include two 4×4 transform cores.

[0176] However, as the size of the transform (ie, the size of the region to which the transform is applied) may be, for example, a size other than 8×8 or 4×4, the number of sets may be n, and the number of transform kernels in each set may be k.

[0177] The transform set may be referred to as an NSST set or a LFNST set. A specific set among the transform sets may be selected, for example, based on the intra prediction mode of the current block (CU or subblock). A low frequency non-separable transform (LFNST) may be an example of a reduced non-separable transform, which will be described later and represents a non-separable transform for a low frequency component.

[0178] According to an example, four transform sets according to intra prediction modes may be mapped, for example, as shown in the following table.

[0179] [Table 5]

[0180] lfnstPredModeIntra lfnstTrSetIdx lfnstPredModeIntra<0 1 0<=lfnstPredModeIntra<=1 0 2<=lfnstPredModeIntra<=12 1 13<=lfnstPredModeIntra<=23 2 24<=lfnstPredModeIntra<=44 3 45<=lfnstPredModeIntra<=55 2 56<=lfnstPredModeIntra<=80 1 81<=lfnstPredModeIntra<=83 0

[0181] As shown in Table 5, any one of four transform sets, ie, lfnstTrSetIdx, may be mapped to any one of four indexes (ie, 0 to 3) according to the intra prediction mode.

[0182] When it is determined that a specific set is used for an inseparable transform, one of the k transform cores in the specific set can be selected by an inseparable secondary transform index. The encoding device can derive an inseparable secondary transform index indicating a specific transform core based on a rate-distortion (RD) check, and can signal the inseparable secondary transform index to the decoding device. The decoding device can select one of the k transform cores in a specific set based on the inseparable secondary transform index. For example, lfnst index value 0 can refer to a first inseparable secondary transform core, lfnst index value 1 can refer to a second inseparable secondary transform core, and lfnst index value 2 can refer to a third inseparable secondary transform core. Alternatively, lfnst index value 0 can indicate that the first inseparable secondary transform is not applied to the current block, and lfnst index values ​​1 to 3 can indicate three transform cores.

[0183] The transformer can perform an inseparable secondary transform based on the selected transform kernel, and can obtain modified (secondary) transform coefficients. As described above, the modified transform coefficients can be derived as transform coefficients quantized by a quantizer, and can be encoded and signaled to a decoding device, and transmitted to a dequantizer / inverse transformer in an encoding device.

[0184] In addition, as described above, if the secondary transform is omitted, the (primary) transform coefficients that are the output of the primary (separable) transform can be derived as transform coefficients quantized by the quantizer as described above, and can be encoded and signaled to the decoding device and transmitted to the dequantizer / inverse transformer in the encoding device.

[0185] The inverse transformer may perform a series of processes in the reverse order of the order already performed in the above-mentioned transformer. The inverse transformer may receive the (dequantized) transform coefficients, and derive the (primary) transform coefficients by performing a secondary (inverse) transform (S950), and may obtain the residual block (residual sample) by performing a (inverse) transform on the (primary) transform coefficients (S960). In this regard, from the perspective of the inverse transformer, the primary transform coefficients may be referred to as modified transform coefficients. As described above, the encoding device and the decoding device may generate a reconstructed block based on the residual block and the prediction block, and may generate a reconstructed picture based on the reconstructed block.

[0186] The decoding device may further include a secondary inverse transform application determiner (or an element for determining whether to apply a secondary inverse transform) and a secondary inverse transform determiner (or an element for determining a secondary inverse transform). The secondary inverse transform application determiner may determine whether to apply a secondary inverse transform. For example, the secondary inverse transform may be NSST, RST, or LFNST, and the secondary inverse transform application determiner may determine whether to apply a secondary inverse transform based on a secondary transform flag obtained by parsing the bitstream. In another example, the secondary inverse transform application determiner may determine whether to apply a secondary inverse transform based on a transform coefficient of a residual block.

[0187] The secondary inverse transform determiner may determine the secondary inverse transform. In this case, the secondary inverse transform determiner may determine the secondary inverse transform applied to the current block based on the LFNST (NSST or RST) transform set specified according to the intra prediction mode. In an embodiment, the secondary transform determination method may be determined depending on the primary transform determination method. Various combinations of the primary transform and the secondary transform may be determined according to the intra prediction mode. In addition, in an example, the secondary inverse transform determiner may determine the area to which the secondary inverse transform is applied based on the size of the current block.

[0188] In addition, as described above, if the secondary (inverse) transform is omitted, the (dequantized) transform coefficients can be received, a (separable) inverse transform can be performed, and a residual block (residual sample) can be obtained. As described above, the encoding device and the decoding device can generate a reconstructed block based on the residual block and the prediction block, and can generate a reconstructed picture based on the reconstructed block.

[0189] Furthermore, in the present disclosure, a reduced secondary transform (RST) in which the size of a transformation matrix (kernel) is reduced may be applied in the concept of NSST in order to reduce the amount of calculation and storage required for an inseparable secondary transform.

[0190] In addition, the transformation kernel, transformation matrix, and coefficients constituting the transformation kernel matrix described in the present disclosure, that is, the kernel coefficients or matrix coefficients, can be represented by 8 bits. This can be a condition for implementation in a decoding device and an encoding device, and compared with the existing 9 bits or 10 bits, the amount of storage required to store the transformation kernel can be reduced, and performance degradation can be reasonably adapted. In addition, representing the kernel matrix with 8 bits can allow the use of small multipliers and can be more suitable for single instruction multiple data (SIMD) instructions for optimal software implementation.

[0191] In the present specification, the term "RST" may refer to a transformation performed on residual samples of a current block based on a transformation matrix whose size is reduced according to a reduction factor. In the case of performing a reduced transformation, the amount of calculation required for the transformation can be reduced due to the reduction in the size of the transformation matrix. That is, RST can be used to solve the computational complexity problem that occurs when transforming a block of large size or an inseparable transform.

[0192] RST may be referred to as various terms such as reduced transform, reduced secondary transform, downscaling transform, simplified transform, and simple transform, and the names that RST may be referred to as are not limited to the listed examples. Alternatively, as described above, since RST is mainly performed in a low-frequency region including non-zero coefficients in a transform block, it may be referred to as a low-frequency non-separable transform (LFNST).

[0193] In addition, when performing a secondary inverse transform based on the RST, the inverse transformer 235 of the encoding device 200 and the inverse transformer 322 of the decoding device 300 may include: an inverse reduction secondary transformer that derives a modified transform coefficient based on an inverse RST of the transform coefficient; and an inverse primary transformer that derives a residual sample of the current block based on an inverse primary transform of the modified transform coefficient. The inverse primary transform refers to an inverse transform of a primary transform applied to the residual. In the present disclosure, deriving a transform coefficient based on a transform may refer to deriving a transform coefficient by applying a transform.

[0194] Fig.10 is a diagram illustrating an RST according to an embodiment of the present disclosure.

[0195] In the present disclosure, “current block” may refer to a current block to be encoded, a residual block, or a transform block.

[0196] In the RST according to the example, an N-dimensional vector can be mapped to an R-dimensional vector located in another space, so that a reduction transformation matrix can be determined, where R is less than N. N may refer to the square of the length of the side of the block to which the transformation is applied, or the total number of transformation coefficients corresponding to the block to which the transformation is applied, and the reduction factor may refer to an R / N value. The reduction factor may be referred to as a reduction factor, a reduction factor, a simplification factor, a simple factor, or other various terms. In addition, R may be referred to as a reduction coefficient, but depending on the situation, the reduction factor may refer to R. In addition, depending on the situation, the reduction factor may refer to an N / R value.

[0197] In the example, the reduction factor or reduction coefficient may be signaled through a bitstream, but the example is not limited thereto. For example, a predetermined value for the reduction factor or reduction coefficient may be stored in each of the encoding device 200 and the decoding device 300, and in this case, the reduction factor or reduction coefficient may not be signaled separately.

[0198] The size of the reduced transform matrix according to an example may be R×N which is smaller than N×N (the size of a conventional transform matrix), and may be defined as in Equation 7 below.

[0199] [Formula 7]

[0200]

[0201] Fig.10 The matrix T in the reduced transform block shown in (a) may refer to the matrix T of Formula 7 R×N .like Fig.10 As shown in (a), when the reduction transformation matrix T R×N When multiplied by the residual samples of the current block, the transform coefficients of the current block can be derived.

[0202] In the example, if the size of the block to which the transform is applied is 8×8 and R=16 (ie, R / N=16 / 64=1 / 4), then according to Fig.10 The RST of (a) can be expressed as a matrix operation shown in the following Equation 8. In this case, the storage and multiplication calculations can be reduced to about 1 / 4 by a reduction factor.

[0203] In the present disclosure, a matrix operation may be understood as an operation of obtaining a column vector by multiplying a column vector by a matrix provided on the left side of the column vector.

[0204] [Formula 8]

[0205]

[0206] In formula 8, r1 to r 64 may represent the residual sample of the current block, and specifically may be a transform coefficient generated by applying a transform. As a result of the calculation of Equation 8, the transform coefficient c of the current block may be derived. i , and derive c i The process can be shown as formula 9.

[0207] [Formula 9]

[0208]

[0209] As a result of the calculation of Equation 9, the transform coefficients c1 to c2 of the current block can be derived. R That is, when R=16, the transform coefficients c1 to c 16 . If a normal transform is applied instead of RST, and a transform matrix of 64×64 (N×N) size is multiplied by a residual sample of 64×1 (N×1) size, only 16 (R) transform coefficients are derived for the current block because RST is applied, although 64 (N) transform coefficients are derived for the current block. Since the total number of transform coefficients for the current block is reduced from N to R, the amount of data transmitted by the encoding device 200 to the decoding device 300 is reduced, and thus the transmission efficiency between the encoding device 200 and the decoding device 300 can be improved.

[0210] When considered from the perspective of the size of the transformation matrix, the size of the conventional transformation matrix is ​​64×64 (N×N), but the size of the reduced transformation matrix is ​​reduced to 16×64 (R×N), so the storage usage in the case of performing RST can be reduced by the R / N ratio compared to the case of performing the conventional transformation. In addition, when compared with the number of multiplication calculations N×N in the case of using the conventional transformation matrix, the use of the reduced transformation matrix can reduce the number of multiplication calculations (R×N) by the R / N ratio.

[0211] In an example, the transformer 232 of the encoding device 200 may derive the transform coefficient of the current block by performing a primary transform and a secondary transform based on RST on the residual samples of the current block. These transform coefficients may be transmitted to the inverse transformer of the decoding device 300, and the inverse transformer 322 of the decoding device 300 may derive the modified transform coefficient based on the inverse reduced secondary transform (RST) for the transform coefficient, and may derive the residual sample of the current block based on the inverse primary transform for the modified transform coefficient.

[0212] According to the example, the inverse RST matrix T N×R The size of is N×R which is smaller than the size of the conventional inverse transform matrix N×N, and is similar to the reduced transform matrix T shown in Equation 7. R×NHas a transposition relationship.

[0213] Fig.10 The matrix T in the reduced inverse transform block shown in (b) t It can refer to the inverse RST matrix T N×R T (The superscript T refers to transposition). Fig.10 As shown in (b), when the inverse RST matrix T N×R T When multiplied by the transform coefficients of the current block, the modified transform coefficients of the current block or the residual samples of the current block can be derived. R×N T It can be expressed as (T R×N T ) N×R .

[0214] More specifically, when the inverse RST is used as the secondary inverse transform, when the inverse RST matrix T N×R T When multiplied by the transform coefficients of the current block, the modified transform coefficients of the current block can be derived. In addition, the inverse RST can be used as an inverse primary transform, and in this case, when the inverse RST matrix T N×R T When multiplied by the transform coefficients of the current block, the residual samples of the current block can be derived.

[0215] In the example, if the size of the block to which the inverse transform is applied is 8×8 and R=16 (ie, R / N=16 / 64=1 / 4), then according to Fig.10 The RST of (b) can be expressed as the matrix operation shown in the following equation 10.

[0216] [Formula 10]

[0217]

[0218] In formula 10, c1 to c 16 As a result of the calculation of Equation 10, r representing the modified transform coefficient of the current block or the residual sample of the current block may be derived. j , and derive r j The process can be shown as formula 11.

[0219] [Formula 11]

[0220]

[0221] As a result of the calculation of Equation 11, r1 to r2 representing the modified transform coefficients of the current block or the residual samples of the current block can be derived. N. From the perspective of the size of the inverse transformation matrix, the size of the conventional inverse transformation matrix is ​​64×64 (N×N), but the size of the inverse reduction transformation matrix is ​​reduced to 64×16 (R×N), so the storage usage rate in the case of performing the inverse RST can be reduced by the R / N ratio compared to the case of performing the conventional inverse transformation. In addition, when compared with the number of multiplication calculations N×N in the case of using the conventional inverse transformation matrix, the use of the inverse reduction transformation matrix can reduce the number of multiplication calculations (N×R) by the R / N ratio.

[0222] The transform set configuration shown in Table 5 can also be applied to 8×8 RST. That is, 8×8 RST can be applied according to the transform set in Table 5. Since a transform set includes two or three transforms (kernels) according to the intra prediction mode, it can be configured to select one of up to four transforms including the case where the secondary transform is not applied. In the transform where the secondary transform is not applied, the application of the identity matrix can be considered. Assuming that indexes 0, 1, 2, and 3 are assigned to the four transforms respectively (for example, index 0 can be assigned to the case where the identity matrix is ​​applied, that is, the case where the secondary transform is not applied), the transform index or lfnst index as a syntax element can be signaled for each transform coefficient block, thereby specifying the transform to be applied. That is, for the upper left 8×8 block, the 8×8 NSST in the RST configuration can be specified by the transform index, or the 8×8 lfnst can be specified when LFNST is applied. 8×8lfnst and 8×8RST refer to transforms that can be applied to an 8×8 region included in a transform coefficient block when W and H of the current block to be transformed are both equal to or greater than 8, and the 8×8 region may be the upper left 8×8 region in the transform coefficient block. Similarly, 4×4lfnst and 4×4RST refer to transforms that can be applied to a 4×4 region included in a transform coefficient block when W and H of the current block are both equal to or greater than 4, and the 4×4 region may be the upper left 4×4 region in the transform coefficient block.

[0223] According to an embodiment of the present disclosure, for the transformation in the encoding process, only 48 pieces of data can be selected, and a maximum 16×48 transform kernel matrix can be applied thereto, instead of applying a 16×64 transform kernel matrix to 64 pieces of data forming an 8×8 region. Here, "maximum" means that m has a maximum value of 16 in the m×48 transform kernel matrix for generating m coefficients. That is, when RST is performed by applying an m×48 transform kernel matrix (m≤16) to an 8×8 region, 48 pieces of data are input, and m coefficients are generated. When m is 16, 48 pieces of data are input and 16 coefficients are generated. That is, assuming that 48 pieces of data form a 48×1 vector, the 16×48 matrix and the 48×1 vector are multiplied in sequence, thereby generating a 16×1 vector. Here, the 48 pieces of data forming an 8×8 region can be appropriately arranged, thereby forming a 48×1 vector. For example, 48 pieces of data present in an area other than the lower right 4×4 area in an 8×8 region can form a 48×1 vector. Here, when the matrix operation is performed by applying the maximum 16×48 transform kernel matrix, 16 modified transform coefficients are generated and the 16 modified transform coefficients can be arranged in the upper left 4×4 region according to the scanning order, and the upper right 4×4 region and the lower left 4×4 region can be filled with zeros.

[0224] For the inverse transform in the decoding process, the transposed matrix of the aforementioned transform kernel matrix can be used. That is, when the inverse RST or LFNST is performed in the inverse transform process performed by the decoding device, the input coefficient data to which the inverse RST is applied is configured in a one-dimensional vector according to a predetermined arrangement order, and the modified coefficient vector obtained by multiplying the one-dimensional vector with the corresponding inverse RST matrix on the left side of the one-dimensional vector can be arranged in a two-dimensional block according to a predetermined arrangement order.

[0225] In summary, in the transformation process, when RST or LFNST is applied to an 8×8 region, 48 transform coefficients in the upper left region, the upper right region, and the lower left region except the lower right region of the 8×8 region are matrix-operated with a 16×48 transform kernel matrix. For the matrix operation, the 48 transform coefficients are input in a one-dimensional array. When the matrix operation is performed, 16 modified transform coefficients are derived, and the modified transform coefficients can be arranged in the upper left region of the 8×8 region.

[0226] On the contrary, in the inverse transform process, when the inverse RST or LFNST is applied to the 8×8 area, 16 transform coefficients corresponding to the upper left area of ​​the 8×8 area among the transform coefficients in the 8×8 area can be input in a one-dimensional array according to the scanning order, and can undergo a matrix operation with a 48×16 transform kernel matrix. That is, the matrix operation can be expressed as (48×16 matrix)*(16×1 transform coefficient vector)=(48×1 modified transform coefficient vector). Here, the n×1 vector can be interpreted as having the same meaning as the n×1 matrix, and can therefore be expressed as an n×1 column vector. In addition, * represents matrix multiplication. When the matrix operation is performed, 48 modified transform coefficients can be derived, and the 48 modified transform coefficients can be arranged in the upper left area, the upper right area, and the lower left area of ​​the 8×8 area except the lower right area.

[0227] When the secondary inverse transform is based on RST, the inverse transformer 235 of the encoding device 200 and the inverse transformer 322 of the decoding device 300 may include an inverse reduction secondary transformer for deriving modified transform coefficients based on the inverse RST of the transform coefficients and an inverse primary transformer for deriving residual samples of the current block based on the inverse primary transform of the modified transform coefficients. The inverse primary transform refers to an inverse transform of a primary transform applied to the residual. In the present disclosure, deriving transform coefficients based on a transform may refer to deriving transform coefficients by applying a transform.

[0228] When wide-angle intra prediction is applied in intra prediction and thus the intra prediction mode is reset, re-indexed, or remapped, a method for configuring an intra prediction mode for an inseparable secondary transform (ie, LFNST) may be discussed.

[0229] According to an example, an inseparable secondary transform may be applied based on a new or modified intra prediction mode, in which case there may be various examples for setting variables for applying the modified intra prediction mode to the transform set of Table 5.

[0230] According to an example, when an intra prediction mode (predModeIntra) is derived for intra prediction and wide-angle intra prediction is applied to a current block, the derived intra prediction mode may be updated or remapped corresponding to the wide-angle intra prediction. The intra mode modified by remapping may be set to a variable predModeIntra, which is the same as the intra mode before modification. The variable predModeIntra for the final intra prediction mode may be used as the intra prediction mode of the transform set for LFNST.

[0231] According to another example, the variable predModeIntra for the final intra prediction mode may be reset to the intra prediction mode (lfnstPredModeIntra) of the transform set for LFNST.

[0232] Alternatively, according to an example, when LFNST is applied instead of the intra prediction mode derived from the intra prediction, a new variable lfnstPredModeIntra may be set by applying wide-angle intra prediction. The image processing process according to the present embodiment may be shown in Tables 6 to 22 below.

[0233] [Table 6]

[0234]

[0235] Table 6 shows the signaling of the syntax element sps_lfnst_enabled_flag indicating whether the inseparable secondary transform is applied (ie, whether there is a transform index, ie, lfnst_idx, indicating whether the inseparable secondary transform is applied) in the image decoding process. lfnst_idx may be signaled in the residual coding syntax or the coding unit syntax.

[0236] [Table 7]

[0237]

[0238] Table 7 shows signaling of lfnst_idx in coding unit syntax. lfnst_idx may indicate any one of two transform kernel matrices included in a transform set, and lfnst_idx of 0 may indicate that an inseparable secondary transform (ie, LFNST) is not applied.

[0239] [Table 8]

[0240]

[0241] Table 8 shows that the intra prediction mode for applying wide-angle intra prediction is newly set in the transformation process. As shown in Table 8, when lfnst_idx of Table 7 is not 0, various variables for LFNST are derived. In the first step, when the current block is non-square, the variable whRatio representing the ratio between the height and width of the current block is set to (Abs(Log2(nTbW / nTbH))), and the intra prediction mode (lfnstPredModeIntra) for secondary transformation is set to the intra prediction mode IntraPredMode of the current block.

[0242] In the second step, when the current block is non-square, lfnstPredModeIntra is remapped to a new intra prediction mode. Since the process is the same as described with reference to Table 2, redundant description is omitted.

[0243] When the intra prediction mode (lfnstPredModeIntra) reflecting the wide-angle intra prediction is newly derived, the inseparable secondary transform (LFNST) may be performed based on the variable lfnstPredModeIntra.

[0244] [Table 9]

[0245]

[0246] Table 9 shows a list y[i] of transformed samples dequantized by a transform kernel matrix (low frequency inseparable transform process), where lfnstPredModeIntra corresponding to the index (lfnstPredModeIntra) for selecting a transform set is used as an input value. A transform set can be selected based on LfnstPredModeIntra, and various transform kernel matrices can be applied according to a transform index indicating any one of the transform sets. Tables 10 to 22 show transform kernel matrices (lowFreqTransMatrix) according to transform indices for 4×4 LFNST and 8×8 LFNST.

[0247] [Table 10]

[0248]

[0249] Table 10 shows a transform set index (lfnstTrSetIdx) according to an intra prediction mode (lfnstPredModeIntra) reflecting a wide-angle intra prediction mode. According to Table 10, in a prediction mode corresponding to wide-angle intra prediction, that is, if lfnstPredModeIntra is less than 0 (lfnstPredModeIntra<0) or lfnstPredModeIntra ranges from 56 to 80 (56<=lfnstPredModeIntra<=80), the transform set index (lfnstTrSetIdx) may be derived as 1.

[0250] Tables 11 to 14 sequentially show two transform kernel matrices that may be included in each set when the transform set index ranges from 0 to 3 in 4×4 LFNST. The transform kernel matrices in Tables 11 to 14 may be derived based on nTrS indicating the size of the transform coefficient output after LFNST, the transform set index (lfnstTrSetIdx), and the transform index (lfnstIdx). According to an example, the size nTrS of the transform coefficient in 4×4 LFNST may be 16.

[0251] [Table 11]

[0252]

[0253] [Table 12]

[0254]

[0255] [Table 13]

[0256]

[0257] [Table 14]

[0258]

[0259] Tables 15 to 22 show, in sequence, two transform kernel matrices that can be included in each set when the transform set index ranges from 0 to 3 in 8×8 LFNST. The transform kernel matrices in Tables 15 to 22 can be derived based on nTrS indicating the size of the transform coefficient output after LFNST, the transform set index (lfnstTrSetIdx), and the transform index (lfnstIdx). According to an example, the size nTrS of the transform coefficient in 8×8 LFNST may be 48. Since there are 48 output transform coefficients, the transform kernel matrix may be a 48×16 matrix, and is shown by dividing 48 matrix coefficients of one row by 16 strips. Table 15 shows the transform kernel matrix when the transform set index (lfnstTrSetIdx) is 0 and the transform index (lfnstIdx) is 1, and Table 16 shows the transform kernel matrix when the transform set index (lfnstTrSetIdx) is 0 and the transform index (lfnstIdx) is 2. Similarly, Table 17 and Table 18 respectively show the transform core matrices when the transform set index (lfnstTrSetIdx) is 1 and the transform index (lfnstIdx) is 1, and when the transform set index (lfnstTrSetIdx) is 1 and the transform index (lfnstIdx) is 2. Similarly, Table 19 and Table 20 respectively show the transform core matrices when the transform set index (lfnstTrSetIdx) is 2 and the transform index (lfnstIdx) is 1, and when the transform set index (lfnstTrSetIdx) is 2 and the transform index (LFnstIdx) is 2. Similarly, Table 21 and Table 22 respectively show the transform core matrices when the transform set index (lfnstTrSetIdx) is 3 and the transform index (lfnstIdx) is 1, and when the transform set index (lfnstTrSetIdx) is 3 and the transform index (lfnstIdx) is 2.

[0260] [Table 15]

[0261]

[0262] [Table 16]

[0263]

[0264] [Table 17]

[0265]

[0266] [Table 18]

[0267]

[0268] [Table 19]

[0269]

[0270] [Table 20]

[0271]

[0272] [Table 21]

[0273]

[0274] [Table 22]

[0275]

[0276] According to another example, the variable predModeIntra derived from intra prediction can be sent through input values ​​and output values ​​in subsequent encoding and decoding processes. Finally, LFNST can be applied using the variable wideAngPredModeIntra input in the inseparable transform (LFNST) process. The process is shown in the following table.

[0277] [Table 23]

[0278]

[0279] [Table 24]

[0280]

[0281] Table 23 shows that the variable wideAngPredModeIntra is derived as an output value in the sub-image processing step (the intra-frame sample prediction process specified in Clause 8.4.5.2 and the scaling and transformation process specified in Clause 8.7.2), while Table 24 shows that the variable wideAngPredModeIntra is derived as an output value in the intra-frame sample prediction step (intra-frame sample prediction in 8.4.5.2).

[0282] [Table 25]

[0283]

[0284] Table 25 shows the sub-steps of Table 24, where the variable wideAngPredModeIntra is derived as an output value. The variable wideAngPredModeIntra derived from Table 25 is the result of mode remapping for non-square blocks applying wide angle intra prediction, and is the same as described with reference to Table 2.

[0285] [Table 26]

[0286]

[0287] Table 26 shows that the variable wideAngPredModeIntra set in Table 25 is used as an input value in the transform step of image processing.

[0288] [Table 27]

[0289]

[0290] Table 27 shows that wideAngPredModeIntra is applied to the non-separable secondary transform (ie, LFNST). The transform kernel matrix according to Table 27 may be the same as the transform kernel matrix in Tables 11 to 22 above, and the index signaling of Tables 6 and 7 may also be applied to this embodiment.

[0291] The following figures are provided to describe specific examples of the present disclosure. Since the specific names of the devices shown in the figures or the names of specific signals / messages / fields are provided for illustration, the technical features of the present disclosure are not limited to the specific names used in the following figures.

[0292] Fig.11 is a flowchart illustrating the operation of a video decoding device according to an embodiment of the present disclosure.

[0293] Fig.11 Each operation shown can be performed by Figure 3 Specifically, S1100 may be performed by the decoding device 300 shown in FIG. Figure 3 The entropy decoder 310 shown in FIG. 1 is executed, and S1110 and S1120 can be performed by Figure 3 The intra-frame predictor 331 shown in FIG. 1100 is executed, and S1130 to S1160 may be executed by the inverse transformer 322. The operations according to S1100 to S1160 are based on the reference Figures 4 to 10 Therefore, some of the aforementioned details explained in the above reference will be omitted or briefly described. Figures 3 to 10 Explain the description of the specific details of those details that overlap.

[0294] The decoding apparatus 300 according to the embodiment may obtain intra prediction mode information and a LFNST index from a bitstream ( S1100 ).

[0295] The intra-frame prediction mode information may include a most probable mode (MPM) index indicating one of the MPM candidates in the MPM list and remaining intra-frame prediction mode information indicating one of the remaining intra-frame prediction modes not included in the MPM candidates, and the MPM list is derived based on the intra-frame prediction modes of neighboring blocks (e.g., left neighboring blocks and / or upper neighboring blocks) for the current block and additional intra-frame prediction modes.

[0296] In addition, the decoding device 300 may decode information about the quantized transform coefficient of the current block from the bitstream, and may derive the quantized transform coefficient of the current block based on the information about the quantized transform coefficient of the current block. The information about the quantized transform coefficient of the current block may be included in a sequence parameter set (SPS) or a slice header, and may include information about whether a reduced transform (RST) is applied, information about a minimum transform size for applying the reduced transform, information about a maximum transform size for applying the reduced transform, a reduced inverse transform size, and at least one of information about a transform index (LFNST index) indicating any one of transform kernel matrices (LFNST matrices) included in a transform set (LFNST set).

[0297] When the partition tree structure for the current block is a dual tree type, a LFNST index may be received for each of a luma block and a chroma block.

[0298] The decoding device 300 may also receive MTS index information indicating a transform kernel of an inverse primary transform.

[0299] The decoding device 300 according to the embodiment can derive the transform coefficient by dequantizing the quantized transform coefficient of the current block. According to the example, when performing the inverse secondary transform and the inverse primary transform, the sample on which the dequantization has been performed can be referred to as the transform coefficient, and the sample on which the inverse secondary transform has been performed can be referred to as the modified transform coefficient. If necessary, the two samples can be collectively referred to as the transform coefficient. When performing the inverse primary transform, the residual sample is derived.

[0300] The decoding apparatus 300 derives an intra prediction mode of a current block based on the intra prediction mode information ( S1110 ).

[0301] For example, the decoding device 300 may generate an MPM list based on prediction modes of neighboring blocks of the current block, and may derive the intra prediction mode of the current block based on the received MPM index or the remaining intra prediction mode information.

[0302] The decoding apparatus 300 may determine whether wide-angle intra prediction is applied to the current block before deriving the intra prediction sample, and may derive a modified intra prediction mode through intra prediction mode remapping when wide-angle intra prediction is applied to the current block ( S1120 ).

[0303] The current block may be a transform block or a coding block based on whether intra partition prediction is applied, and wide-angle intra prediction may be applied when the height and width of the current block are not the same, ie, when the current block is non-square.

[0304] For non-square blocks, the intra prediction mode may be modified according to two separate cases, and the modified intra prediction mode may be derived based on which of the height and width of the current block is larger, the ratio between the height and width of the current block, and the intra prediction mode.

[0305] First, in the first case, if all conditions are met (1) the width of the current block is greater than the height, (2) the intra-frame prediction mode before modification is equal to or greater than 2, and (3) the intra-frame prediction mode is less than the value derived as (8+2*whRatio) when whRatio is greater than 1 and the value derived as 8 when whRatio is less than or equal to 1, the intra-frame prediction mode is set to be equal to the value of predModeIntra+65.

[0306] In the second case, the intra prediction mode is set equal to the value of predModeIntra-67 if all conditions are met: (1) the height of the current block is greater than the width, (2) the intra prediction mode before modification is less than or equal to 66, and (3) the intra prediction mode is greater than the value derived as (60-2*whRatio) when whRatio is greater than 1 and the value derived as 60 when whRatio is less than or equal to 1.

[0307] For example, when the intra prediction mode information indicates the planar mode (0), the DC mode (1), and Figure 4 When any one of the directional modes 2 to 66 is selected, if the width of the current block is greater than the height of the current block, the modified intra prediction mode can be derived from Figure 5 If the height of the current block is greater than the width of the current block, the modified intra prediction mode may be derived from Figure 5 The range is -1 to -14.

[0308] When the intra prediction mode of the chroma block is DM using the same mode as the intra prediction mode of the luma block, wide-angle intra prediction may be applied to the chroma block.

[0309] The inverse transformer 322 of the decoding apparatus 300 may determine a transform set (ie, LFNST set) based on a mapping relationship according to a modified intra prediction mode applied to the current block (S1130), and may select any one of a plurality of LFNST matrices based on the LFNST set and the LFNST index (S1140).

[0310] According to an example, as shown in Table 5, when any one of four LFNST sets is determined according to the intra prediction mode of the current block, the same LFNST set may be determined in all cases where wide-angle intra prediction is applied. That is, the LFNST set in the case where the modified intra prediction mode is in the range from 67 to 80 and the LFNST set in the case where the modified intra prediction mode is in the range from -1 to -14 may be the same.

[0311] Subsequently, the decoding apparatus 300 may perform an inverse RST (eg, inverse LFNST) by applying the LFNST matrix to the dequantized transform coefficient, thereby deriving a modified transform coefficient of the current block ( S1150 ).

[0312] As described above, any one of a plurality of transform sets may be determined according to an intra prediction mode of a transform block to be transformed, and inverse RST may be performed based on any one of LFNST matrices included in the LFNST set indicated by an LFNST index.

[0313] The LFNST matrix may be a non-square matrix in which the number of columns is less than the number of rows.

[0314] Referring to S1150, it can be identified that the residual samples of the current block are derived based on the inverse RST of the transform coefficients of the current block. Regarding the size of the inverse transform matrix, the size of the general inverse transform matrix is ​​N×N, while the size of the inverse RST matrix is ​​reduced to N×R, so compared with when performing a general transform, the memory occupancy can be reduced by the R / N ratio when performing the inverse RST. In addition, compared with the number of multiplication operations N×N when using the general inverse transform matrix, when using the inverse RST matrix, the number of multiplication operations can be reduced by the R / N ratio (to N×R). In addition, since only R transform coefficients need to be decoded when applying the inverse RST, the total number of transform coefficients of the current block can be reduced from N to R compared to applying a general inverse transform in which N transform coefficients need to be decoded, thereby increasing the decoding efficiency. That is, according to S1150, the (inverse) transform efficiency and decoding efficiency of the decoding device 300 can be increased by the inverse RST.

[0315] The decoding apparatus 300 according to the embodiment may induce residual samples of the current block based on inverse transform of the modified transform coefficient ( S1160 ).

[0316] The decoding apparatus 300 may perform an inverse primary transform on the modified transform coefficients of the current block, in which case a simplified inverse transform may be applied or a conventional separable transform may be used as the inverse primary transform.

[0317] Subsequently, the decoding apparatus 300 may generate a reconstructed sample based on the residual sample of the current block and the prediction sample of the current block.

[0318] The following figures are provided to describe specific examples of the present disclosure. Since the specific names of the devices shown in the figures or the names of specific signals / messages / fields are provided for illustration, the technical features of the present disclosure are not limited to the specific names used in the following figures.

[0319] Fig.12 is a flowchart illustrating the operation of a video encoding device according to an embodiment of the present disclosure.

[0320] Fig.12 Each operation shown can be performed by Figure 2 Specifically, S1200 to S1220 may be performed by Figure 2 The predictor shown in FIG. 12 is executed, and S1230 can be performed by Figure 2 The subtractor 231 shown in FIG. 1 is executed, and S1240 and S1250 can be performed by Figure 2 The converter 232 shown in FIG. 1 is executed, and S1260 can be performed by Figure 2 The entropy encoder 240 shown in FIG. 1 is executed. The operations according to S1200 to S1260 are based on Figures 4 to 10 Therefore, the contents described in the above reference will be omitted or briefly described. Figure 2 and Figures 4 to 10 Explain the description of the specific details of those details that overlap.

[0321] The encoding apparatus 200 according to an embodiment may derive an intra prediction mode of a current block ( S1200 ).

[0322] The encoding apparatus 200 may configure an MPM list of a current block and may determine an optimal intra prediction mode based on rate-distortion optimization (RDO).

[0323] The MPM list may include candidate intra prediction modes (MPM candidates) that are highly likely to be applied to the current block. For example, the MPM list may include intra prediction modes of neighboring blocks and may also include a specific intra prediction mode according to a predetermined method.

[0324] The encoding device 200 may perform prediction based on various intra prediction modes, and may determine an optimal intra prediction mode based on rate-distortion optimization (RDO) thereon. The encoding device 200 may determine the optimal intra prediction mode using only the MPM candidates included in the MPM list, or may determine the optimal intra prediction mode using the remaining intra prediction modes in addition to the MPM candidates included in the MPM list.

[0325] When wide-angle intra prediction is applied to the current block, the encoding apparatus 200 may derive a modified intra prediction mode through intra prediction mode remapping ( S1210 ).

[0326] The current block may be a transform block or a coding block based on whether intra partition prediction is applied, and wide-angle intra prediction may be applied when the height and width of the current block are not the same, ie, when the current block is non-square.

[0327] For a non-square block, the intra prediction mode may be modified according to two separate cases, and the modified intra prediction mode may be derived based on which of the height and width of the current block is larger, the ratio between the height and width of the current block, and the intra prediction mode. For example, when the intra prediction mode information indicates the planar mode (0), the DC mode (1), and the Figure 4 When any one of the directional modes 2 to 66 is selected, if the width of the current block is greater than the height of the current block, the modified intra prediction mode can be derived from Figure 5 If the height of the current block is greater than the width of the current block, the modified intra prediction mode may be derived from Figure 5 The range is -1 to -14.

[0328] When the intra prediction mode of the chroma block is DM using the same mode as the intra prediction mode of the luma block, wide-angle intra prediction may be applied to the chroma block.

[0329] The encoding apparatus 200 may induce a prediction sample based on the modified intra prediction mode applied to the current block ( S1220 ), and may induce a residual sample of the current block based on the prediction sample ( S1230 ).

[0330] The encoding apparatus 200 according to an embodiment may determine a LFNST set including a LFNST matrix based on the modified intra prediction mode ( S1240 ).

[0331] The encoding apparatus 200 may perform an MTS-based primary transform before an LFNST set-based inseparable transform, and may generate an MTS index indicating which MTS is used.

[0332] According to an example, as shown in Table 5, when any one of four LFNST sets is determined according to the intra prediction mode of the current block, the same LFNST set may be determined in all cases where wide-angle intra prediction is applied. That is, the LFNST set in the case where the modified intra prediction mode is in the range from 67 to 80 and the LFNST set in the case where the modified intra prediction mode is in the range from -1 to -14 may be the same.

[0333] The encoding apparatus 200 may induce a transformation coefficient of the current block based on the residual sample, the LFNST set, and the LFNST matrix (S1250).

[0334] As described above, the transform coefficient of the current block may be derived based on one transform of the residual sample, and one transform may be performed by a plurality of transform kernels. In this case, the transform kernel may be selected based on the intra prediction mode.

[0335] Although residual samples after performing one transform may be referred to as transform coefficients, and samples after performing LFNST may be referred to as modified transform coefficients, these two types of samples may be collectively referred to as transform coefficients.

[0336] As described above, any one of a plurality of transform sets may be determined according to an intra prediction mode of a transform block to be transformed, and inverse RST may be performed based on any one of LFNST matrices included in the LFNST set.

[0337] Each LFNST set may include two LFNST matrices, and the LFNST matrices may be non-square matrices in which the number of columns is less than the number of rows.

[0338] Referring to S1250, the transform coefficients of the current block are derived based on the RST of the residual samples. Regarding the size of the transform kernel matrix, the size of the general transform kernel matrix is ​​N×N, while the size of the simplified transform matrix is ​​reduced to R×N, so the memory occupancy can be reduced by the R / N ratio when performing the RST compared to when performing the general transform. In addition, compared with the number of multiplication operations N×N when the general transform matrix is ​​used, the number of multiplication operations can be reduced by the R / N ratio (to N×R) when the simplified transform kernel matrix is ​​used. In addition, since only R transform coefficients need to be derived when applying RST, the total number of transform coefficients of the current block can be reduced from N to R compared to applying a general transform in which N transform coefficients are derived, thereby reducing the amount of data sent by the encoding device 200 to the decoding device 300. That is, according to S1250, the transformation efficiency and encoding efficiency of the encoding device 200 can be increased by RST.

[0339] The encoding apparatus 200 according to an embodiment may induce a quantized transform coefficient by performing quantization based on the modified transform coefficient of the current block and may encode information about the quantized transform coefficient.

[0340] The encoding apparatus 200 may encode and generate image information including an intra prediction mode and a LFNST index indicating a LFNST matrix in addition to information about the quantized transform coefficient (S1260).

[0341] The syntax element of the LFNST index according to this embodiment may indicate whether (inverse) LFNST is applied and any one of the LFNST matrices included in the LFNST set. When the LFNST set includes two transform kernel matrices, the syntax element of the transform index may have three values.

[0342] According to an example, when the partition tree structure for the current block is a dual tree type, a LFNST index may be encoded for each of a luma block and a chroma block.

[0343] Depending on an embodiment, the syntax element of the transform index may be derived as a value of 0 indicating that no (inverse) RST is applied to the current block, a value of 1 indicating the first transform core matrix among the transform core matrices, or a value of 2 indicating the second transform core matrix among the transform core matrices.

[0344] In the present disclosure, at least one of quantization / dequantization and / or transform / inverse transform may be omitted. When quantization / dequantization is omitted, the quantized transform coefficient may be referred to as a transform coefficient. When transform / inverse transform is omitted, the transform coefficient may be referred to as a coefficient or a residual coefficient, or may still be referred to as a transform coefficient for consistency of expression.

[0345] In addition, in the present disclosure, the quantized transform coefficient and the transform coefficient may be referred to as a transform coefficient and a scaled transform coefficient, respectively. In this case, the residual information may include information about the transform coefficient, and the information about the transform coefficient may be signaled by the residual coding syntax. The transform coefficient may be derived based on the residual information (or information about the transform coefficient), and the scaled transform coefficient may be derived by an inverse transform (scaling) of the transform coefficient. The residual sample may be derived based on an inverse transform (transform) of the scaled transform coefficient. These details may also be applied / expressed in other parts of the present disclosure.

[0346] In the above-mentioned embodiments, the method is explained based on the flowchart with the aid of a series of steps or boxes, but the present disclosure is not limited to the order of the steps, and a certain step may be performed in an order or step different from the above order or step, or a certain step may be performed concurrently with other steps. In addition, it can be understood by those of ordinary skill in the art that the steps shown in the flowchart are not exclusive, and another step may be incorporated or one or more steps in the flowchart may be deleted without affecting the scope of the present disclosure.

[0347] The above method according to the present disclosure may be implemented in a software form, and the encoding device and / or decoding device according to the present disclosure may be included in a device for image processing such as a television, a computer, a smart phone, a set-top box, a display device, and the like.

[0348] When the embodiments in the present disclosure are implemented by software, the above methods can be implemented as modules (steps, functions, etc.) for performing the above functions. These modules can be stored in a memory and can be executed by a processor. The memory can be inside or outside the processor and can be connected to the processor in various well-known ways. The processor may include an application-specific integrated circuit (ASIC), other chipsets, logic circuits and / or data processing devices. The memory may include a read-only memory (ROM), a random access memory (RAM), a flash memory, a memory card, a storage medium and / or other storage devices. In other words, the embodiments described in the present disclosure can be implemented and executed on a processor, a microprocessor, a controller or a chip. For example, the functional units shown in each of the accompanying drawings can be implemented and executed on a computer, a processor, a microprocessor, a controller or a chip.

[0349] In addition, the decoding device and encoding device of the present disclosure can be included in a multimedia broadcast transceiver, a mobile communication terminal, a home theater video device, a digital theater video device, a surveillance camera, a video chat device, a real-time communication device (such as video communication), a mobile streaming device, a storage medium, a portable camera, a video on demand (VoD) service provider, an over-the-top (OTT) video device, an Internet streaming service provider, a three-dimensional (3D) video device, a video phone video device, and a medical video device, and can be used to process a video signal or a data signal. For example, an over-the-top (OTT) video device can include a game console, a Blu-ray player, an Internet access TV, a home theater system, a smart phone, a tablet PC, a digital video recorder (DVR), etc.

[0350] In addition, the processing method of the present disclosure can be produced in the form of a program executed by a computer and can be stored in a computer-readable recording medium. Multimedia data with a data structure according to the present disclosure can also be stored in a computer-readable recording medium. Computer-readable recording media include various storage devices and distributed storage devices that store computer-readable data. Computer-readable recording media may include, for example, a Blu-ray disc (BD), a universal serial bus (USB), a ROM, a PROM, an EPROM, an EEPROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device. In addition, the computer-readable recording medium includes a medium implemented in the form of a carrier (e.g., transmission on the Internet). In addition, the bit stream generated by the encoding method can be stored in a computer-readable recording medium or transmitted via a wired or wireless communication network. In addition, the embodiments of the present disclosure can be implemented as a computer program product through a program code, and the program code can be executed on a computer according to the embodiments of the present disclosure. The program code can be stored on a computer-readable carrier.

[0351] Fig.13The structure of the content streaming system to which the present disclosure is applied is illustrated.

[0352] Furthermore, a content streaming system to which the present disclosure is applied may generally include an encoding server, a streaming server, a web server, a media storage device, a user device, and a multimedia input device.

[0353] The encoding server is used to compress the content input from a multimedia input device such as a smart phone, a camera, a camcorder, etc. into digital data to generate a bitstream, and send it to a streaming server. As another example, in the case where a multimedia input device such as a smart phone, a camera, a camcorder, etc. directly generates a bitstream, the encoding server can be omitted. The bitstream can be generated by applying the encoding method or the bitstream generation method disclosed in the present invention. And the streaming server can temporarily store the bitstream during the process of sending or receiving the bitstream.

[0354] The streaming server sends multimedia data to the user device through the web server based on the user's request, and the web server serves as a tool to inform the user of what services exist. When the user requests the service the user wants, the web server transmits the request to the streaming server, and the streaming server sends the multimedia data to the user. In this regard, the content streaming system may include a separate control server, and in this case, the control server is used to control the command / response between the corresponding devices in the content streaming system.

[0355] The streaming server can receive content from a media storage device and / or an encoding server. For example, in the case of receiving content from an encoding server, the content can be received in real time. In this case, in order to smoothly provide a streaming service, the streaming server can store the bit stream for a predetermined time.

[0356] For example, user devices may include mobile phones, smart phones, laptop computers, digital broadcast terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigators, tablet PCs, tablet PCs, ultrabooks, wearable devices (e.g., watch-type terminals (smart watches), glasses-type terminals (smart glasses), head-mounted displays (HMDs)), digital TVs, desktop computers, digital signage, etc. Each server in the content streaming system may operate as a distributed server, and in this case, data received by each server may be processed in a distributed manner.

[0357] The claims disclosed herein may be combined in various ways. For example, the technical features of the method claims of the present disclosure may be combined to be implemented or performed in a device, and the technical features of the device claims may be combined to be implemented or performed in a method. In addition, the technical features of the method claims and the device claims may be combined to be implemented or performed in a device, and the technical features of the method claims and the device claims may be combined to be implemented or performed in a method.

Claims

1. An image decoding method performed by a decoding device, the image decoding method comprising the following steps: Obtain intra prediction mode information and LFNST index from the bitstream; deriving an intra prediction mode of a current block based on the intra prediction mode information; Based on wide-angle intra prediction being applied to the current block, deriving a modified intra prediction mode by intra prediction mode remapping; determining a LFNST set including a LFNST matrix based on the modified intra prediction mode; selecting one of the LFNST matrices based on the LFNST set and the LFNST index; deriving transform coefficients of the current block based on the selected LFNST matrix; as well as deriving residual samples of the current block based on the transform coefficients; Wherein, when the partition tree structure for the current block is a dual tree type, the LFNST index is received for each of the luma block and the chroma block.

2. The image decoding method according to claim 1, wherein: The intra prediction mode based on the chroma block is a direct mode DM using the same mode as the intra prediction mode of the luminance block, and the wide-angle intra prediction is applied to the chroma block.

3. The image decoding method according to claim 1, wherein: The current block includes a transform block or a coding block, and Wherein, based on the fact that the height and width of the current block are different, the wide-angle intra prediction is applied to the current block.

4. The image decoding method according to claim 3, wherein: The modified intra prediction mode is derived based on which of the height and the width of the current block is greater, a ratio of the height to the width of the current block, and the intra prediction mode.

5. The image decoding method according to claim 4, wherein: The intra prediction mode information indicates any one of planar mode 0, DC mode 1, and directional modes 2 to 66, wherein, based on the width of the current block being greater than the height of the current block, the modified intra prediction mode is derived to be in a range from 67 to 80, and Wherein, based on the fact that the height of the current block is greater than the width of the current block, the modified intra prediction mode is derived to be in the range from -1 to -14.

6. The image decoding method according to claim 5, wherein: determining any one of four LFNST sets according to the intra prediction mode applied to the current block, and Here, the LFNST set in the case where the modified intra prediction mode is in the range from 67 to 80 is the same as the LFNST set in the case where the modified intra prediction mode is in the range from -1 to -14.

7. The image decoding method according to claim 5, wherein: The LFNST index indicates whether LFNST is applied and any one of the LFNST matrices included in the LFNST set.

8. An image encoding method performed by an image encoding device, the image encoding method comprising the following steps: Derivation of intra prediction mode for current block; deriving a modified intra prediction mode by intra prediction mode remapping based on whether wide-angle intra prediction is applied to the current block; deriving prediction samples based on the modified intra prediction mode; deriving residual samples of the current block based on the prediction samples; determining a LFNST set including a LFNST matrix based on the modified intra prediction mode; deriving transform coefficients of the current block based on the residual samples, the LFNST set, and the LFNST matrix; as well as encoding image information including the intra prediction mode and an LFNST index indicating the LFNST matrix; Wherein, when the partition tree structure for the current block is a dual tree type, the LFNST index is received for each of the luma block and the chroma block.

9. The image encoding method according to claim 8, wherein: The intra prediction mode based on the chroma block is a direct mode DM using the same mode as the intra prediction mode of the luminance block, and the wide-angle intra prediction is applied to the chroma block.

10. The image encoding method according to claim 8, wherein: The modified intra prediction mode is derived based on which of the height and the width of the current block is greater, a ratio of the height to the width of the current block, and the intra prediction mode.

11. The image encoding method according to claim 10, wherein: The intra prediction mode information indicates any one of planar mode 0, DC mode 1, and directional modes 2 to 66, wherein, based on the width of the current block being greater than the height of the current block, the modified intra prediction mode is derived to be in a range from 67 to 80, and Wherein, based on the fact that the height of the current block is greater than the width of the current block, the modified intra prediction mode is derived to be in the range from -1 to -14.

12. The image encoding method according to claim 11, wherein: determining any one of four LFNST sets according to the intra prediction mode applied to the current block, and Here, the LFNST set in the case where the modified intra prediction mode is in the range from 67 to 80 is the same as the LFNST set in the case where the modified intra prediction mode is in the range from -1 to -14.

13. A computer-readable digital storage medium storing computer instructions, wherein the method implemented when the instructions are executed by a processor comprises the following steps: deriving an intra prediction mode for the current block; deriving a modified intra prediction mode by intra prediction mode remapping based on whether wide-angle intra prediction is applied to the current block; deriving prediction samples based on the modified intra prediction mode; deriving residual samples of the current block based on the prediction samples; determining a LFNST set including a LFNST matrix based on the modified intra prediction mode; deriving transform coefficients of the current block based on the residual samples, the LFNST set, and the LFNST matrix; as well as encoding image information including the intra prediction mode and an LFNST index indicating the LFNST matrix to generate a bitstream; Wherein, when the partition tree structure for the current block is a dual tree type, the LFNST index is received for each of the luma block and the chroma block.

14. A method comprising the steps of: Obtaining a bitstream, wherein the bitstream is generated by: deriving an intra prediction mode of a current block; deriving a modified intra prediction mode through intra prediction mode remapping based on whether wide-angle intra prediction is applied to the current block; deriving prediction samples based on the modified intra prediction mode; deriving residual samples of the current block based on the prediction samples; determining an LFNST set including an LFNST matrix based on the modified intra prediction mode; deriving transform coefficients of the current block based on the residual samples, the LFNST set, and the LFNST matrix; and encoding image information including the intra prediction mode and an LFNST index indicating the LFNST matrix to generate the bitstream; and sending the bit stream; Wherein, when the partition tree structure for the current block is a dual tree type, the LFNST index is received for each of the luma block and the chroma block.

Images