Encoding method, decoding method, computer-readable storage medium, and transmission method

Abstract

The present disclosure relates to methods for encoding / decoding image information, a computer-readable storage medium for storing a bitstream, and a method for transmitting a bitstream.

Description

Encoding method, decoding method, computer-readable storage medium and transmission method

[0001] The present disclosure relates to a method for encoding / decoding video information, a computer-readable storage medium for storing a bitstream, and a method for transmitting the bitstream.

[0002] Recently, the demand for high-resolution, high-quality images, such as HD (High Definition) images and UHD (Ultra High Definition) images, is increasing in various application fields, and accordingly, high-efficiency image compression technologies are being discussed.

[0003] There are various technologies such as inter prediction technology that predicts pixel values ​​included in the current picture from pictures before or after the current picture, intra prediction technology that predicts pixel values ​​included in the current picture using pixel information within the current picture, and entropy coding technology that assigns short codes to values ​​with high frequency of appearance and long codes to values ​​with low frequency of appearance, and these video compression technologies can be used to effectively compress and transmit or store video data.

[0004] Accordingly, a highly efficient image compression technology is required to effectively transmit, store, and play high-resolution, high-quality image information.

[0005] The present disclosure provides a decoding method, an encoding method, a computer-readable recording medium, and a transmission method capable of improving coding efficiency by configuring a plurality of intra prediction mode candidates for a current block during an encoding / decoding process within a screen of a still image or a moving image, generating a blending candidate based on the intra prediction mode candidates, and then selecting the optimal blending candidate based on an error value with respect to a restored sample of a template area.

[0006] In other words, the present invention proposes an intra prediction mode based on a template-based blending method to minimize unnecessary signaling overhead while embracing the respective strengths of DIMD and TIMD. The number of intra modes to combine and the weighting values ​​to apply are too diverse. Transmitting these combinations to the decoder via signaling can incur significant signaling overhead. Therefore, we propose a method that finds the optimal combination in the template region and performs blending.

[0007] A decoding method according to one embodiment comprises the steps of: deriving intra prediction mode candidates; deriving at least one blending candidate based on the intra prediction mode candidates; deriving a prediction sample based on any one of the intra prediction mode candidates or the at least one blending candidate; and generating a reconstructed sample based on the prediction sample, wherein the prediction sample can be derived based on an error value between reconstructed samples of a surrounding template area of ​​a current block and prediction samples predicted based on any one of the intra prediction mode candidates or the at least one blending candidate.

[0008] An encoding method according to one embodiment is a method for encoding image information, comprising: deriving intra prediction mode candidates; deriving at least one blending candidate based on the intra prediction mode candidates; deriving a prediction sample based on any one of the intra prediction mode candidates or the at least one blending candidate; and deriving a residual sample based on the prediction sample, wherein the prediction sample can be derived based on an error value between restoration samples of a surrounding template area of ​​a current block and prediction samples predicted based on any one of the intra prediction mode candidates or the at least one blending candidate.

[0009] A computer-readable storage medium storing a bitstream generated by an encoding method according to one embodiment, the encoding method comprising: deriving intra prediction mode candidates; deriving at least one blending candidate based on the intra prediction mode candidates; deriving a prediction sample based on any one of the intra prediction mode candidates or the at least one blending candidate; deriving a residual sample based on the prediction sample; and storing a bitstream based on the residual sample; wherein the prediction sample can be derived based on an error value between restoration samples of a surrounding template area of ​​a current block and prediction samples predicted based on any one of the intra prediction mode candidates or the at least one blending candidate.

[0010] A method for transmitting data for an image according to one embodiment comprises: deriving intra prediction mode candidates; deriving at least one blending candidate based on the intra prediction mode candidates; deriving a prediction sample based on any one of the intra prediction mode candidates or the at least one blending candidate; deriving a residual sample based on the prediction sample; generating a bitstream based on the residual sample; and transmitting the data including the bitstream; wherein the prediction sample may be derived based on an error value between restoration samples of a surrounding template area of ​​a current block and prediction samples predicted based on any one of the intra prediction mode candidates or the at least one blending candidate.

[0011] According to the disclosed embodiment, in a process of encoding / decoding within a screen of a still image or a moving image, a plurality of intra prediction mode candidates for a current block are configured, a blending candidate is generated based on the intra prediction mode candidates, and then an optimal blending candidate is selected based on an error value with respect to a restored sample of a template area, thereby minimizing unnecessary signaling overhead and improving coding efficiency.

[0012] The effects that can be obtained from the present disclosure are not limited to the effects mentioned above, and other effects that are not mentioned will be clearly understood by a person having ordinary skill in the art to which the present disclosure pertains from the description below.

[0013] FIG. 1 illustrates a video / image coding system according to one embodiment.

[0014] FIG. 2 is a schematic block diagram of an encoding device to which the disclosed embodiment can be applied and in which encoding of a video / image signal is performed.

[0015] FIG. 3 is a schematic block diagram of a decoding device to which the disclosed embodiment can be applied and in which decoding of a video / image signal is performed.

[0016] FIG. 4 illustrates an example of a video / image decoding method to which the disclosed embodiment can be applied.

[0017] FIG. 5 illustrates an example of a video / image encoding method to which the disclosed embodiment can be applied.

[0018] Figure 6 illustrates an example hierarchical structure for a coded image according to one embodiment.

[0019] FIG. 7 is a flowchart illustrating a method of encoding an image based on intra prediction in an encoding method according to one embodiment.

[0020] Fig. 8 is a drawing showing the configuration of an encoding device used to perform the method of Fig. 7.

[0021] FIG. 9 is a flowchart illustrating a method of decoding an image based on intra prediction in a decoding method according to one embodiment.

[0022] Fig. 10 is a diagram showing the configuration of a decoding device used to perform the method of Fig. 9.

[0023] FIG. 11 is a flowchart illustrating an example of a method for signaling information about an intra prediction mode in an encoding method according to one embodiment.

[0024] FIG. 12 is a flowchart illustrating an example of a method for deriving an intra prediction mode in a decoding method according to one embodiment.

[0025] FIG. 13 is a flowchart illustrating in more detail an example of a method for deriving an intra prediction mode in a decoding method according to one embodiment.

[0026] Figure 14 is a drawing showing a template and reference sample used in TIMD mode.

[0027] Figure 15 is a drawing showing a configuration method of HoG used in DIMD mode.

[0028] Figure 16 is a diagram showing the prediction block configuration in DIMD mode.

[0029] Figure 17 is a diagram showing a peripheral restoration sample used in DIMD chroma mode.

[0030] Figure 18 is a flowchart showing an example of a processing flow for generating an intra prediction block.

[0031] Figure 19 is a diagram showing an example of a template area for obtaining an error value of a blending candidate.

[0032] Figure 20 is a flowchart illustrating an example of a decoding method according to one embodiment.

[0033] Figure 21 is a flowchart illustrating an example of an encoding method according to one embodiment.

[0034] FIG. 22 is a diagram illustrating an example of a content streaming system to which the disclosed embodiment can be applied.

[0035]

[0036] The present disclosure may be modified in various ways and encompasses numerous embodiments. Specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present disclosure to specific embodiments, but rather to encompass all modifications, equivalents, and alternatives falling within the spirit and technical scope of the present disclosure. Similar reference numerals have been used to designate similar components throughout the description of each drawing.

[0037] While terms such as "first" and "second" may be used to describe various components, these components should not be limited by these terms. These terms are used solely to distinguish one component from another. For example, without departing from the scope of the present disclosure, a first component could be referred to as a "second component," and similarly, a second component could also be referred to as a "first component." The term "and / or" includes a combination of multiple related items described herein or any of multiple related items described herein.

[0038] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components intervening. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components intervening.

[0039] The terminology used in this application is only used to describe specific embodiments and is not intended to limit the present disclosure. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, it should be understood that the terms "comprise" or "have" indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but do not preclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0040] The flowchart described in this disclosure merely illustrates operations in order for convenience of explanation, and the embodiments are not limited to the order shown in the flowchart. The disclosed operations may be performed in a different order than that shown in the flowchart, and some operations may be performed simultaneously.

[0041] The present disclosure relates to video / image coding. For example, the methods / embodiments disclosed in this specification can be applied to methods disclosed in the versatile video coding (VVC) standard. In addition, the methods / embodiments disclosed in this specification can be applied to methods disclosed in the essential video coding (EVC) standard, the AOMedia Video 1 (AV1) standard, the second generation of audio video coding standard (AVS2), or the next generation of video / image coding standards (e.g., H.267 or H.268).

[0042] This specification presents various embodiments of video / image coding, and unless otherwise stated, the embodiments may be performed in combination with each other.

[0043] In this specification, video may refer to a set of images over time. A picture generally refers to a unit representing one image at a specific time point, and a slice / tile is a unit that constitutes part of a picture in coding. A slice / tile may include one or more coding tree units (CTUs). A picture may be composed of one or more slices / tiles. A tile is a rectangular area consisting of multiple CTUs within a specific tile column and a specific tile row of a picture. A tile column is a rectangular area of ​​CTUs that has a height equal to the height of the picture and a width specified by the syntax requirements of the picture parameter set. A tile row is a rectangular area of ​​CTUs that has a height specified by the picture parameter set and a width equal to the width of the picture. CTUs within a tile are arranged consecutively according to the CTU raster scan, while tiles within a picture may be arranged consecutively according to the tile raster scan. A slice may contain an integer number of complete tiles or an integer number of contiguous complete CTU rows within a picture, which may be exclusively contained within a single NAL unit. Meanwhile, a picture may be divided into two or more subpictures. A subpicture may be a rectangular region of one or more slices within a picture.

[0044] A pixel, or pel, can refer to the smallest unit that constitutes a picture (or image). Additionally, the term "sample" can be used as a counterpart to a pixel. A sample can generally represent a pixel or a pixel value, and can represent only the pixel / pixel value of the luminance component, or only the pixel / pixel value of the chrominance component.

[0045] A unit may represent a basic unit of image processing. A unit may include at least one of a specific region of a picture and information related to the region. One unit may include one luma block and two chroma (e.g., cb, cr) blocks. In some cases, the term "unit" may be used interchangeably with terms such as "block" or "area." In general, an MxN block may include a set (or array) of samples (or sample array) or transform coefficients consisting of M columns and N rows.

[0046] In this specification, “A or B” can mean “only A,” “only B,” or “both A and B.” In other words, “A or B” in this specification can be interpreted as “A and / or B.” For example, “A, B or C” in this specification can mean “only A,” “only B,” “only C,” or “any combination of A, B, and C.”

[0047] As used herein, a slash ( / ) or a comma can mean "and / or." For example, "A / B" can mean "A and / or B." Accordingly, "A / B" can mean "only A," "only B," or "both A and B." For example, "A, B, C" can mean "A, B, or C."

[0048] In this specification, “at least one of A and B” may mean “only A,” “only B,” or “both A and B.” Additionally, in this specification, the expressions “at least one of A or B” or “at least one of A and / or B” may be interpreted identically to “at least one of A and B.”

[0049] Additionally, in this specification, “at least one of A, B and C” can mean “only A,” “only B,” “only C,” or “any combination of A, B and C.” Additionally, “at least one of A, B or C” or “at least one of A, B and / or C” can mean “at least one of A, B and C.”

[0050] Additionally, parentheses used herein may mean "for example." Specifically, when "prediction (intra-prediction)" is indicated, "intra-prediction" may be suggested as an example of "prediction." In other words, "prediction" in this specification is not limited to "intra-prediction," and "intra-prediction" may be suggested as an example of "prediction." Furthermore, even when "prediction (i.e., intra-prediction)" is indicated, "intra-prediction" may be suggested as an example of "prediction."

[0051] Technical features individually described in a single drawing in this specification may be implemented individually or simultaneously.

[0052] FIG. 1 illustrates a video / image coding system according to one embodiment.

[0053] Referring to FIG. 1, a video / image coding system may include a first device (source device) and a second device (receiving device).

[0054] A source device can transmit encoded video / image information or data to a receiving device via a digital storage medium or a network in the form of a file or streaming. The source device may include a video source, an encoding device, and a transmitting device. The receiving device may include a receiving device, a decoding device, and a renderer. The encoding device may be referred to as a video / image encoding device, and the decoding device may be referred to as a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display unit, and the display unit may be configured as a separate device or an external component.

[0055] A video source may obtain video / images through a process of capturing, synthesizing, or generating video / images. The video source may include a video / image capture device and / or a video / image generation device. The video / image capture device may include one or more cameras, a video / image archive containing previously captured video / images, etc. The video / image generation device may include a computer, a tablet, a smartphone, etc., and may (electronically) generate video / images. For example, a virtual video / image may be generated through a computer, etc., in which case the video / image capture process may be replaced by a process of generating related data.

[0056] An encoding device can encode input video / images. The encoding device can perform a series of procedures, such as prediction, transformation, and quantization, to improve compression and coding efficiency. The encoded data (encoded video / image information) can be output in the form of a bitstream.

[0057] The transmission unit can transmit encoded video / image information or data output in the form of a bitstream to the receiving unit of a receiving device via a digital storage medium or a network in the form of a file or streaming. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The storage medium can be a computer-readable storage medium. The transmission unit can include an element for generating a media file via a predetermined file format and an element for transmission via a broadcasting / communication network. The receiving unit can receive / extract the bitstream and transmit it to a decoding device.

[0058] The decoding device can decode the video / image by performing a series of procedures such as inverse quantization, inverse transformation, and prediction corresponding to the operation of the encoding device.

[0059] The renderer can render decoded video / images. The rendered video / images can be displayed through the display unit.

[0060] FIG. 2 is a schematic block diagram of an encoding device to which the disclosed embodiment can be applied and in which encoding of a video / image signal is performed.

[0061] Referring to FIG. 2, the encoding device (200) may be configured to include an image partitioner (210), a prediction unit (predictor) 220, a residual processor (residual processor) 230, an entropy encoder (entropy encoder) 240, an adder (adder) 250, a filter (filter) 260, and a memory (memory) 270. The prediction unit (220) may include an inter prediction unit (221) and an intra prediction unit (222). The residual processor (230) may include a transformer (transformer) 232, a quantizer (quantizer) 233, a dequantizer (dequantizer) 234, and an inverse transformer (inverse transformer) 235. The residual processing unit (230) may further include a subtractor (231). The addition unit (250) may be called a reconstructor or a recontructed block generator. The image segmentation unit (210), the prediction unit (220), the residual processing unit (230), the entropy encoding unit (240), the addition unit (250), and the filtering unit (260) described above may be configured by one or more hardware components (e.g., an encoding device chipset or processor) according to an embodiment. In addition, the memory (270) may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The hardware component may further include the memory (270) as an internal / external component.

[0062] The image segmentation unit (210) can segment an input image (or picture, frame) input to the encoding device (200) into one or more processing units (PUs). For example, the processing units may be called coding units (CUs). In this case, the coding units may be recursively segmented from a coding tree unit (CTU) or a largest coding unit (LCU) according to a QTBTTT (Quad-Tree Binary-Tree Ternary-Tree) structure.

[0063] For example, a single coding unit may be split into multiple coding units with deeper depths based on a quad-tree structure, a binary tree structure, and / or a ternary structure. In this case, for example, the quad-tree structure may be applied first, and the binary tree structure and / or the ternary structure may be applied later. Alternatively, the binary tree structure may be applied before the quad-tree structure. The coding procedure according to the present specification may be performed based on the final coding unit that is no longer split. In this case, based on coding efficiency according to image characteristics, etc., the largest coding unit may be used directly as the final coding unit, or, if necessary, the coding unit may be recursively split into coding units of lower depths, and the coding unit with the optimal size may be used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and restoration, which will be described later.

[0064] As another example, the processing unit may further include a prediction unit (PU) or a transform unit (TU). In this case, the prediction unit and the transform unit may each be split or partitioned from the final coding unit described above. The prediction unit may be a unit of sample prediction, and the transform unit may be a unit for deriving a transform coefficient and / or a unit for deriving a residual signal from a transform coefficient.

[0065] The term "unit" may be used interchangeably with terms such as "block" or "area" depending on the case. In general, an MxN block can represent a set of samples or transform coefficients consisting of M columns and N rows. A sample can generally represent a pixel or a pixel value, and can represent only a pixel / pixel value of a luminance component or only a pixel / pixel value of a chrominance component. A sample can be used as a term corresponding to a pixel or pel of a picture (or image).

[0066] The encoding device (200) can generate a residual signal (residual block, residual sample array) by subtracting a prediction signal (prediction block, prediction sample array) output from an inter prediction unit (221) or an intra prediction unit (222) from an input video signal (original block, original sample array), and the generated residual signal is transmitted to a conversion unit (232). In this case, a unit that subtracts a prediction signal (prediction block, prediction sample array) from an input video signal (original block, original sample array) within the encoding device (200) may be called a subtraction unit (231).

[0067] The prediction unit (220) can perform a prediction on a block to be processed (hereinafter, referred to as a current block) and generate a predicted block including prediction samples for the current block. The prediction unit (220) can determine whether intra prediction or inter prediction is applied on a current block or CU basis. The prediction unit (220) can generate various information related to prediction, such as prediction mode information, as described later in the description of each prediction mode, and transmit the information to the entropy encoding unit (240). The information related to prediction can be encoded by the entropy encoding unit (240) and output in the form of a bitstream.

[0068] The intra prediction unit (222) can predict the current block by referring to samples in the current picture. The referenced samples, i.e., the reference samples, may be located in the neighborhood of the current block or may be located a certain distance away from the current block depending on the prediction mode. In intra prediction, the prediction modes may include one or more non-directional modes and multiple directional modes. The non-directional mode may include at least one of the DC mode or the planar mode. The directional mode may include 33 directional modes or 65 directional modes depending on the degree of detail in the prediction direction. However, this is only an example, and a greater or lesser number of directional modes may be used depending on the settings. The intra prediction unit (222) may also determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring blocks.

[0069] The inter prediction unit (221) can derive a prediction block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. At this time, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information can be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the neighboring blocks and the current block. The motion information can include a motion vector and a reference picture index. The motion information can further include inter prediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter prediction, the neighboring block can include a spatial neighboring block existing in the current picture and a temporal neighboring block existing in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different. Temporal neighboring blocks may be called collocated reference blocks, collocated CUs (colCUs), etc., and reference pictures including temporal neighboring blocks may be called collocated pictures (colPic). For example, the inter prediction unit (221) may construct a motion information candidate list based on neighboring blocks, and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Inter prediction may be performed based on various prediction modes, and for example, in the case of skip mode and merge mode, the inter prediction unit (221) may use the motion information of neighboring blocks as the motion information of the current block. In the case of skip mode, unlike the merge mode, a residual signal may not be transmitted.In the motion vector prediction (MVP) mode, the motion vector of the surrounding blocks is used as a motion vector predictor, and the motion vector of the current block can be indicated by signaling the motion vector difference.

[0070] The prediction unit (220) can generate a prediction signal based on various prediction methods described below. For example, the prediction unit can apply intra prediction or inter prediction to predict a single block, and can also apply intra prediction and inter prediction simultaneously. This can be called combined inter and intra prediction (CIIP) mode. In addition, the prediction unit can perform an intra block copy (IBC) prediction mode to predict a block. The IBC prediction mode can be used for content image / video coding such as games, such as screen content coding (SCC). IBC basically performs prediction within the current picture, but can be performed similarly to inter prediction in that it derives a reference block within the current picture. That is, IBC can use at least one of the inter prediction techniques described herein. The prediction signal generated through the prediction unit (220) can be used to generate a restored signal or a residual signal.

[0071] The transform unit (232) can apply a transform technique to the residual signal to generate transform coefficients. For example, the transform technique can include at least one of a Discrete Cosine Transform (DCT), a Discrete Sine Transform (DST), a Karhunen-Loeve Transform (KLT), a Graph-Based Transform (GBT), or a Conditionally Non-linear Transform (CNT). Here, GBT refers to a transform obtained from a graph when the relationship information between pixels is expressed as a graph. CNT refers to a transform obtained based on generating a prediction signal using all previously restored pixels. In addition, the transform process can be applied to a pixel block having a square size and the same size, or can be applied to a block of a non-square variable size.

[0072] The quantization unit (233) quantizes the transform coefficients and transmits them to the entropy encoding unit (240), and the entropy encoding unit (240) can encode the quantized signal (information about the quantized transform coefficients) and output it as a bitstream. The information about the quantized transform coefficients can be called residual information. The quantization unit (233) can rearrange the quantized transform coefficients in a block form into a one-dimensional vector form based on the coefficient scan order, and can also generate information about the quantized transform coefficients based on the quantized transform coefficients in the one-dimensional vector form.

[0073] The entropy encoding unit (240) can perform various encoding methods, such as, for example, exponential Golomb, context-adaptive variable length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), etc. The entropy encoding unit (240) can also encode information necessary for video / image restoration (e.g., values ​​of syntax elements, etc.) together or separately from quantized transform coefficients.

[0074] Encoded information (e.g., encoded video / image information) can be transmitted or stored in the form of a bitstream in units of NAL (network abstraction layer) units. The video / image information may further include information on various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may further include general constraint information. In the present specification, information and / or syntax elements transmitted / signaled from an encoding device to a decoding device may be included in the video / image information. The video / image information may be encoded through the above-described encoding procedure and included in the bitstream. The bitstream may be transmitted via a network or stored in a digital storage medium. Here, the network may include a broadcasting network and / or a communication network, and the digital storage medium may include various storage media, such as a USB, SD, CD, DVD, Blu-ray, HDD, or SSD. For example, the storage medium may be a medium that stores the bitstream non-statutory.

[0075] A transmission unit (not shown) for transmitting a signal output from an entropy encoding unit (240) and / or a storage unit (not shown) for storing the signal may be configured as an internal / external element of the encoding device (200), or the transmission unit may be included in the entropy encoding unit (240).

[0076] The quantized transform coefficients output from the quantization unit (233) can be used to generate a prediction signal. For example, by applying inverse quantization and inverse transformation to the quantized transform coefficients through the inverse quantization unit (234) and the inverse transform unit (235), a residual signal (residual block or residual samples) can be reconstructed. The addition unit (250) can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the reconstructed residual signal to the prediction signal output from the inter prediction unit (221) or the intra prediction unit (222). When there is no residual for the block to be processed, such as when skip mode is applied, the predicted block can be used as a reconstructed block. The addition unit (250) may be called a reconstructor or a reconstructed block generation unit. The generated restoration signal can be used for intra prediction of the next processing target block within the current picture, and can also be used for inter prediction of the next picture after filtering as described below. Meanwhile, LMCS (luma mapping with chroma scaling) may be applied during the picture encoding and / or restoration process.

[0077] The filtering unit (260) can improve subjective / objective picture quality by applying filtering to the restoration signal. For example, the filtering unit (260) can apply various filtering methods to the restoration picture to generate a modified restoration picture, and store the modified restoration picture in the memory (270), specifically, in the DPB of the memory (270). The various filtering methods may include deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc. The filtering unit (260) can generate various information regarding filtering and transmit the information to the entropy encoding unit (240). The information regarding filtering may be encoded by the entropy encoding unit (240) and output in the form of a bitstream.

[0078] The modified restored picture transmitted to the memory (270) can be used as a reference picture in the inter prediction unit (221). Through this, when inter prediction is applied, the encoding device can avoid prediction mismatch between the encoding device (200) and the decoding device, and can also improve encoding efficiency.

[0079] The DPB of the memory (270) can store the modified restored picture to be used as a reference picture in the inter prediction unit (221). The memory (270) can store motion information of a block from which motion information in the current picture is derived (or encoded) and / or motion information of blocks in a picture that has already been restored. The stored motion information can be transferred to the inter prediction unit (221) to be used as motion information of a spatial neighboring block or motion information of a temporal neighboring block. The memory (270) can store restored samples of restored blocks in the current picture and transfer them to the intra prediction unit (222).

[0080] Image information output in the form of a bitstream from the encoding device (200) can be transmitted to the decoding device (300) through the transmission unit.

[0081] FIG. 3 is a schematic block diagram of a decoding device to which the disclosed embodiment can be applied and in which decoding of a video / image signal is performed.

[0082] Image information transmitted in the form of a bitstream from the encoding device (200) can be received by the decoding device (300).

[0083] Referring to FIG. 3, the decoding device (300) may be configured to include an entropy decoder (310), a residual processor (320), a predictor (330), an adder (340), a filter (350), and a memory (360). The predictor (330) may include an inter-prediction unit (332) and an intra-prediction unit (331). The residual processor (320) may include a dequantizer (321) and an inverse transformer (321).

[0084] The entropy decoding unit (310), residual processing unit (320), prediction unit (330), addition unit (340), and filtering unit (350) described above may be configured by a single hardware component (e.g., a decoding device chipset or processor) depending on the embodiment. In addition, the memory (360) may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The hardware component may further include the memory (360) as an internal / external component.

[0085] When a bitstream including video / image information is input, the decoding device (300) can restore the image corresponding to the process in which the video / image information is processed in the encoding device of FIG. 2. For example, the decoding device (300) can derive units / blocks based on block division-related information obtained from the bitstream. The decoding device (300) can perform decoding using a processing unit applied in the encoding device. Accordingly, the processing unit of decoding may be a coding unit, and the coding unit may be divided from a coding tree unit or a maximum coding unit according to a quad tree structure, a binary tree structure, and / or a ternary tree structure. One or more transform units may be derived from the coding unit. Then, the restored image signal decoded and output through the decoding device (300) can be reproduced through a reproduction device.

[0086] The decoding device (300) can receive a signal output from the encoding device of FIG. 2 in the form of a bitstream, and the received signal can be decoded through the entropy decoding unit (310). For example, the entropy decoding unit (310) can parse the bitstream to derive information (e.g., video / image information) necessary for image restoration (or picture restoration). The video / image information may further include information on various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may further include general constraint information. The decoding device can decode the picture further based on the information on the parameter set and / or the general constraint information. The signaling / received information and / or syntax elements described later in this specification can be decoded through the decoding procedure and obtained from the bitstream. For example, the entropy decoding unit (310) can decode information in a bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and output the values ​​of syntax elements required for image restoration and the quantized values ​​of transform coefficients for residuals. More specifically, the CABAC entropy decoding method receives a bin corresponding to each syntax element in the bitstream, determines a context model using information of the syntax element to be decoded and decoding information of the surrounding and decoding target blocks or information of symbols / bins decoded in the previous step, and predicts the occurrence probability of the bin according to the determined context model to perform arithmetic decoding of the bin to generate a symbol corresponding to the value of each syntax element.At this time, the CABAC entropy decoding method can update the context model using the information of the decoded symbol / bin for the context model of the next symbol / bin after determining the context model. Information regarding prediction among the information decoded by the entropy decoding unit (310) is provided to the prediction unit (inter prediction unit (332) and intra prediction unit (331)), and residual values ​​on which entropy decoding is performed by the entropy decoding unit (310), i.e., quantized transform coefficients and related parameter information, can be input to the residual processing unit (320). The residual processing unit (320) can derive a residual signal (residual block, residual samples, residual sample array). In addition, information regarding filtering among the information decoded by the entropy decoding unit (310) can be provided to the filtering unit (350). Meanwhile, a receiving unit (not shown) that receives a signal output from an encoding device may be further configured as an internal / external element of a decoding device (300), or the receiving unit may be a component of an entropy decoding unit (310).

[0087] Meanwhile, a decoding device according to the present specification may be called a video / video / picture decoding device, and the decoding device may be divided into an information decoding device (video / video / picture information decoding device) and a sample decoding device (video / video / picture sample decoding device). The information decoding device may include the entropy decoding unit (310), and the sample decoding device may include at least one of the inverse quantization unit (321), the inverse transformation unit (322), the addition unit (340), the filtering unit (350), the memory (360), the inter prediction unit (332), and the intra prediction unit (331).

[0088] The inverse quantization unit (321) can inverse quantize the quantized transform coefficients and output the transform coefficients. The inverse quantization unit (321) can rearrange the quantized transform coefficients into a two-dimensional block form. In this case, the rearrangement can be performed based on the coefficient scanning order performed in the encoding device. The inverse quantization unit (321) can perform inverse quantization on the quantized transform coefficients using quantization parameters (e.g., quantization step size information) and obtain transform coefficients.

[0089] In the inverse transform unit (322), the transform coefficients are inversely transformed to obtain a residual signal (residual block, residual sample array).

[0090] The prediction unit (320) can perform a prediction on the current block and generate a predicted block including prediction samples for the current block. The prediction unit (320) can determine whether intra-prediction or inter-prediction is applied to the current block based on the information regarding the prediction output from the entropy decoding unit (310), and can determine a specific intra / inter-prediction mode.

[0091] The prediction unit (320) can generate a prediction signal based on various prediction methods described below. For example, the prediction unit (320) can apply intra prediction or inter prediction for prediction of a single block, and can also apply intra prediction and inter prediction simultaneously. This can be called combined inter and intra prediction (CIIP) mode. In addition, the prediction unit can be based on an intra block copy (IBC) prediction mode or a palette mode for prediction of a block. The IBC prediction mode or palette mode can be used for content image / video coding such as games, such as screen content coding (SCC). IBC basically performs prediction within the current picture, but can be performed similarly to inter prediction in that it derives a reference block within the current picture. That is, IBC can utilize at least one of the inter prediction techniques described herein. The palette mode can be viewed as an example of intra coding or intra prediction. When palette mode is applied, information about the palette table and palette index may be signaled and included in the video / image information.

[0092] The intra prediction unit (331) can predict the current block by referring to samples within the current picture. The referenced samples may be located in the neighborhood of the current block, or may be located a certain distance away from the current block, depending on the prediction mode. In intra prediction, the prediction modes may include one or more non-directional modes and multiple directional modes. The intra prediction unit (331) may also determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring blocks.

[0093] The inter prediction unit (332) can derive a prediction block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. At this time, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information can be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the neighboring blocks and the current block. The motion information can include a motion vector and a reference picture index. The motion information can further include inter prediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter prediction, the neighboring blocks can include spatial neighboring blocks existing in the current picture and temporal neighboring blocks existing in the reference picture. For example, the inter prediction unit (332) can construct a motion information candidate list based on the neighboring blocks, and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. Inter prediction can be performed based on various prediction modes, and information about the prediction can include information indicating an inter prediction mode for the current block.

[0094] The addition unit (340) can generate a restoration signal (restored picture, restoration block, restoration sample array) by adding the acquired residual signal to the prediction signal (prediction block, prediction sample array) output from the prediction unit (including the inter-prediction unit (332) and / or intra-prediction unit (331)). When there is no residual for the block to be processed, such as when skip mode is applied, the prediction block can be used as the restoration block.

[0095] The addition unit (340) may be referred to as a restoration unit or restoration block generation unit. The generated restoration signal may be used for intra prediction of the next processing target block within the current picture, may be output after filtering as described below, or may be used for inter prediction of the next picture. Meanwhile, LMCS (luma mapping with chroma scaling) may be applied during the picture decoding process.

[0096] The filtering unit (350) can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit (350) can apply various filtering methods to the restored picture to generate a modified restored picture, and transmit the modified restored picture to the memory (360), specifically, to the DPB of the memory (360). The various filtering methods can include deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc.

[0097] The (modified) reconstructed picture stored in the DPB of the memory (360) can be used as a reference picture in the inter prediction unit (332). The memory (360) can store motion information of a block from which motion information in the current picture is derived (or decoded) and / or motion information of blocks in an already reconstructed picture. The stored motion information can be transmitted to the inter prediction unit (332) to be used as motion information of a spatial neighboring block or motion information of a temporal neighboring block. The memory (360) can store reconstructed samples of reconstructed blocks in the current picture and transmit them to the intra prediction unit (331).

[0098] In this specification, the embodiments described in the filtering unit (260), the inter prediction unit (221), and the intra prediction unit (222) of the encoding device (200) can be applied to the filtering unit (350), the inter prediction unit (332), and the intra prediction unit (331) of the decoding device (300) in the same or corresponding manner, respectively.

[0099] FIG. 4 illustrates an example of a video / image decoding method to which the disclosed embodiment can be applied.

[0100] In image / video coding, the pictures that make up an image / video can be encoded / decoded according to a series of decoding orders. The picture order corresponding to the output order of the decoded pictures can be set differently from the decoding order, and based on this, not only forward prediction but also backward prediction can be performed during inter prediction.

[0101] In FIG. 4, S400 may be performed in the entropy decoding unit (310) of the aforementioned decoding device (300), S410 may be performed in the prediction unit (330), S420 may be performed in the residual processing unit (320), S430 may be performed in the addition unit (340), and S440 may be performed in the filtering unit (350). S400 may include a decoding procedure according to the present disclosure, S410 may include an inter / intra prediction procedure according to the present disclosure, S420 may include a residual processing procedure according to the present disclosure, S430 may include a block / picture restoration procedure according to the present disclosure, and S440 may include an in-loop filtering procedure according to the present disclosure.

[0102] Referring to FIG. 4, the decoding device obtains image / video information from a bitstream (S400), performs prediction based on the obtained image / video information (S410), and restores a picture through residual processing (S420), inverse quantization for quantized transform coefficients, and inverse transformation (S430).

[0103] A modified restored picture can be generated by applying an in-loop filtering procedure (S440) to a restored picture generated through the above restoration procedure, and the modified restored picture can be output as a decoded picture and can be stored in a buffer or memory of a decoding device to be used as a reference picture in an inter prediction procedure when decoding a next picture. In some cases, the in-loop filtering procedure can be omitted, in which case the restored picture can be output as a decoded picture and can be stored in a buffer or memory of a decoding device to be used as a reference picture in an inter prediction procedure when decoding a next picture.

[0104] The in-loop filtering procedure (S440) may include a deblocking filtering procedure, a sample adaptive offset (SAO) procedure, an adaptive loop filter (ALF) procedure, and / or a bi-lateral filter procedure, and some or all of them may be omitted. In addition, one or some of the deblocking filtering procedure, the sample adaptive offset (SAO) procedure, the adaptive loop filter (ALF) procedure, and the bi-lateral filter procedure may be sequentially applied, or all of them may be sequentially applied. For example, the SAO procedure may be performed after the deblocking filtering procedure is applied to the restored picture. Or, for example, the ALF procedure may be performed after the deblocking filtering procedure is applied to the restored picture. This may also be performed in an encoding device.

[0105] FIG. 5 illustrates an example of a video / image encoding method to which the disclosed embodiment can be applied.

[0106] In FIG. 5, the prediction step (S500) may be performed in the prediction unit (220) of the encoding device (200) described above, residual processing (S510) based on the prediction result may be performed in the residual processing unit (230), and the step (S520) of encoding image information including prediction information and residual information may be performed in the entropy encoding unit (240). S500 may include an inter / intra prediction procedure according to the present disclosure, S510 may include a residual processing procedure according to the present disclosure, and S520 may include an encoding procedure according to the present disclosure.

[0107] The encoding procedure may optionally include a procedure for encoding information for picture restoration (e.g., prediction information, residual information, partitioning information, etc.) and outputting it in the form of a bitstream, as well as a procedure for generating a restored picture for the current picture and a procedure for applying in-loop filtering to the restored picture.

[0108] The encoding device (200) can derive (corrected) residual samples from the quantized transform coefficients through the inverse quantization unit (234) and the inverse transformation unit (235), and can generate a restored picture based on the prediction samples and (corrected) residual samples, which are outputs of S500. The restored picture generated in this way can be the same as the restored picture generated by the decoding device (300) described above. A modified restored picture can be generated through an in-loop filtering procedure for the restored picture, which can be stored in a buffer or memory, and, as in the case of the decoding device, can be used as a reference picture in the inter prediction procedure when encoding a subsequent picture.

[0109] As described above, some or all of the in-loop filtering procedure may be omitted in some cases. When the in-loop filtering procedure is performed, (in-loop) filtering-related information (parameters) may be encoded by the entropy encoding unit (240) and output in the form of a bitstream, and the decoding device (300) may perform the in-loop filtering procedure in the same manner as the encoding device based on the filtering-related information.

[0110] Through this in-loop filtering procedure, noise occurring during image / video coding, such as blocking artifacts and ringing artifacts, can be reduced, and subjective / objective image quality can be improved. In addition, by performing the in-loop filtering procedure in both the encoding device (200) and the decoding device (300), the same prediction results can be derived from the encoding device (200) and the decoding device (300), thereby increasing the reliability of picture coding and reducing the amount of data that must be transmitted for picture coding.

[0111] As described above, the picture restoration procedure can be performed not only in the decoding device (300) but also in the encoding device (200). A restoration block can be generated based on intra-prediction / inter-prediction for each block, and a restoration picture including the restoration blocks can be generated. If the current picture / slice / tile group is an I picture / slice / tile group, the blocks included in the current picture / slice / tile group can be restored based only on intra-prediction. On the other hand, if the current picture / slice / tile group is a P or B picture / slice / tile group, the blocks included in the current picture / slice / tile group can be restored based on intra-prediction or inter-prediction. In this case, inter-prediction may be applied to some blocks in the current picture / slice / tile group, and intra-prediction may be applied to some remaining blocks.

[0112] The color component of a picture may include a luma component and a chroma component, and embodiments according to the present disclosure may be applied to the luma component and the chroma component unless explicitly limited in the present disclosure.

[0113] Figure 6 illustrates an example hierarchical structure for a coded image according to one embodiment.

[0114] Referring to Figure 6, the coded image is divided into a Video Coding Layer (VCL) that handles the decoding processing of the image and the image itself, a subsystem that transmits and stores the coded information, and a Network Abstraction Layer (NAL) that exists between the VCL and the subsystem and is responsible for network adaptation functions.

[0115] In VCL, VCL data containing compressed image data (slice data) can be generated, or a parameter set containing information such as a picture parameter set (PPS), a sequence parameter set (SPS), a video parameter set (VPS), etc., or an SEI (Supplemental Enhancement Information) message additionally required for the image decoding process can be generated.

[0116] In NAL, a NAL unit can be created by adding header information (NAL unit header) to an RBSP (Raw Byte Sequence Payload) generated from a VCL. At this time, RBSP refers to slice data, parameter sets, SEI messages, etc. generated from a VCL. The NAL unit header can include NAL unit type information that is specific to the RBSP data included in the NAL unit.

[0117] As illustrated in Fig. 6, NAL units can be divided into VCL NAL units and non-VCL NAL units according to the RBSP generated from VCL. A VCL NAL unit can refer to a NAL unit that contains information about a video (slice data), and a non-VCL NAL unit can refer to a NAL unit that contains information necessary for decoding a video (parameter set or SEI message).

[0118] The above-described VCL NAL units and non-VCL NAL units can be transmitted over a network by attaching header information according to the data specifications of the lower system. For example, NAL units can be transformed into data formats of a certain standard, such as the H.266 / VVC file format, RTP (Real-time Transport Protocol), TS (Transport Stream), etc., and transmitted over various networks.

[0119] As described above, a NAL unit can have a NAL unit type specified according to the RBSP data structure included in the NAL unit, and information about the NAL unit type can be stored and signaled in the NAL unit header.

[0120] For example, depending on whether a NAL unit contains information about a picture (slice data), it can be broadly classified into VCL NAL unit types and Non-VCL NAL unit types. The VCL NAL unit type can be classified according to the nature and type of the picture contained in the VCL NAL unit, and the Non-VCL NAL unit type can be classified according to the type of parameter set, etc.

[0121] Below are examples of NAL unit types, specified by the type of parameter set included in the Non-VCL NAL unit type.

[0122] - APS (Adaptation Parameter Set) NAL unit: Type for NAL units containing APS

[0123] - DPS (Decoding Parameter Set) NAL unit: Type for NAL unit containing DPS

[0124] - VPS (Video Parameter Set) NAL unit: Type for NAL units containing VPS

[0125] - SPS (Sequence Parameter Set) NAL unit: Type for NAL units containing SPS

[0126] - PPS (Picture Parameter Set) NAL unit: Type for NAL units containing PPS

[0127] The above-described NAL unit types have syntax information for the NAL unit type, and the syntax information can be stored and signaled in the NAL unit header. For example, the syntax information can be nal_unit_type, and NAL unit types can be specified by the nal_unit_type value.

[0128] The slice header (slice header syntax, slice header information) may include information / parameters that are commonly applicable to the slices. The APS (APS syntax) or PPS (PPS syntax) may include information / parameters that are commonly applicable to one or more slices or pictures. The SPS (SPS syntax) may include information / parameters that are commonly applicable to one or more sequences. The VPS (VPS syntax) may include information / parameters that are commonly applicable to multiple layers. The DPS (DPS syntax) may include information / parameters that are commonly applicable to the entire video. The DPS may include information / parameters related to the concatenation of CVS (coded video sequence). In the present disclosure, the High Level Syntax (HLS) may include at least one of the APS syntax, PPS syntax, SPS syntax, VPS syntax, DPS syntax, or slice header syntax.

[0129] In the present disclosure, image / video information encoded in an image encoding device and signaled in the form of a bitstream may include information related to partitioning within a picture, intra / inter prediction information, residual information, in-loop filtering information, etc., and may also include information included in the slice header, information included in the APS, information included in the PPS, information included in the SPS, information included in the VPS, and / or information included in the DPS.

[0130] Meanwhile, the prediction unit (220, 330) of the encoding device (200) / decoding device (300) can derive a reference sample according to the intra prediction mode of the current block among the surrounding samples of the current block, and can generate a prediction sample of the current block based on the reference sample. Intra prediction can refer to a prediction that generates prediction samples for the current block based on reference samples in a picture to which the current block belongs (hereinafter, the current picture). When intra prediction is applied to the current block, surrounding reference samples to be used for intra prediction of the current block can be derived. The surrounding reference samples of the current block can include a sample adjacent to the left boundary of the current block of a size nWxnH and a total of 2xnH samples adjacent to the bottom-left, a sample adjacent to the top boundary of the current block and a total of 2xnW samples adjacent to the top-right, and one sample adjacent to the top-left of the current block. Alternatively, the peripheral reference samples of the current block may include upper peripheral samples of multiple columns and left peripheral samples of multiple rows. In addition, the peripheral reference samples of the current block may include a total of nH samples adjacent to the right boundary of the current block of size nWxnH, a total of nW samples adjacent to the bottom boundary of the current block, and one sample adjacent to the bottom-right of the current block.

[0131] However, some of the surrounding reference samples of the current block may not yet have been decoded or may not be available. In this case, the decoding device (300) may construct surrounding reference samples to be used for prediction by substituting the unavailable samples with available samples. Alternatively, the surrounding reference samples to be used for prediction may be constructed through interpolation of available samples.

[0132] When neighboring reference samples are derived, (i) a prediction sample can be derived based on an average or interpolation of neighboring reference samples of the current block, and (ii) the prediction sample can be derived based on a reference sample existing in a specific (prediction) direction with respect to the prediction sample among the neighboring reference samples of the current block. The case of (i) can be called a non-directional mode or a non-angular mode, and the case of (ii) can be called a directional mode or an angular mode.

[0133] Additionally, linear interpolation intra prediction (LIP) may be applied to perform intra prediction on the current block by linearly interpolating prediction sample values ​​generated based on the intra prediction mode of the current block.

[0134] Additionally, a temporary prediction sample of the current block may be derived based on filtered peripheral reference samples, and a prediction sample of the current block may be derived by weighting at least one reference sample derived according to an intra prediction mode among existing peripheral reference samples, i.e., unfiltered peripheral reference samples, and the temporary prediction sample. Such prediction may be referred to as Position Dependent Intra Prediction Combination (PDPC).

[0135] In addition, intra prediction encoding can be performed by selecting a reference sample line with the highest prediction accuracy among the surrounding multiple reference sample lines of the current block, deriving a prediction sample using the reference sample located in the prediction direction of the selected line, and then instructing (signaling) the used reference sample line to the decoding device. This case can be referred to as multi-reference line intra prediction (MRL) or MRL-based intra prediction.

[0136] Additionally, the current block can be divided into vertical or horizontal subpartitions, and intra prediction can be performed based on the same intra prediction mode, while peripheral reference samples can be derived and utilized for each subpartition. In other words, in this case, the intra prediction mode for the current block is applied equally to the subpartitions, but peripheral reference samples can be derived and utilized for each subpartition, thereby improving intra prediction performance in some cases. This prediction method can be called intra subpartitions (ISP) or ISP-based intra prediction.

[0137] Additionally, if the prediction direction based on the prediction sample points between surrounding reference samples, i.e., if the prediction direction points to a fractional sample location, the value of the prediction sample can be derived through interpolation of multiple reference samples located around the prediction direction (around the fractional sample location).

[0138] Information about the intra prediction mode may be included in the prediction information encoded by the encoding device (200) and included in the bitstream and transmitted to the decoding device (300). Information about the intra prediction mode may be implemented and transmitted in various forms, such as flag information indicating whether each intra prediction mode is applied or index information indicating one of several intra prediction modes.

[0139] The intra prediction methods described above may be referred to as intra prediction types, to distinguish them from intra prediction modes. Furthermore, the intra prediction types may be referred to by various terms, such as intra prediction techniques, intra prediction tools, or additional intra prediction modes. For example, the intra prediction types (or additional intra prediction modes, etc.) may include at least one of the LIP, PDPC, MRL, and ISP described above. A general intra prediction method, excluding specific intra prediction types such as LIP, PDPC, MRL, and ISP, may be referred to as a normal intra prediction type. The normal intra prediction type may be generally applied when the specific intra prediction types described above are not applied, and prediction may be performed based on the intra prediction modes described above. Meanwhile, post-processing filtering may be performed on the derived prediction samples, if necessary.

[0140] The intra prediction process may include an intra prediction mode / type determination step, a surrounding reference sample derivation step, and an intra prediction mode / type-based prediction sample derivation step. Additionally, a post-processing filtering step may be performed on the derived prediction samples, as needed. This will be described in detail below.

[0141] FIG. 7 is a flowchart showing a method of encoding an image based on intra prediction in an encoding method according to one embodiment, and FIG. 8 is a diagram showing the configuration of an encoding device used to perform the method of FIG. 7.

[0142] Referring to FIGS. 7 and 8 together, step S1000 may be performed by the intra prediction unit (222) of the encoding device (200), and step S1010 may be performed by the residual processing unit (230) of the encoding device. Specifically, step S1010 may be performed by the subtraction unit (115) of the encoding device (200). In step S1020, prediction information may be derived by the intra prediction unit (222) and encoded by the entropy encoding unit (240). In step S1020, residual information may be derived by the residual processing unit (230) and encoded by the entropy encoding unit (240). The residual information is information about the residual samples. The residual information may include information about quantized transform coefficients for the residual samples. As described above, the residual samples are derived as transform coefficients through the transform unit (232) of the encoding device (200), and the transform coefficients can be derived as quantized transform coefficients through the quantization unit (233). Information about the quantized transform coefficients can be encoded in the entropy encoding unit (240) through a residual coding procedure.

[0143] The encoding device (200) can perform intra prediction on the current block (S1000). The encoding device (200) can derive an intra prediction mode / type for the current block, derive surrounding reference samples of the current block, and generate prediction samples within the current block based on the intra prediction mode / type and the surrounding reference samples. The prediction samples can constitute a prediction block. Here, the intra prediction mode / type determination, surrounding reference sample derivation, and prediction sample generation procedures may be performed simultaneously, or one procedure may be performed before the other. For example, the intra prediction unit (222) of the encoding device (200) may include an intra prediction mode / type determination unit (222a), a reference sample derivation unit (222b), and a prediction sample derivation unit (222c). The intra prediction mode / type determination unit (222a) may determine the intra prediction mode / type for the current block, the reference sample derivation unit (222b) may derive surrounding reference samples of the current block, and the prediction sample derivation unit (222c) may derive prediction samples of the current block. Meanwhile, although not shown, when the prediction sample filtering procedure described below is performed, the intra prediction unit (222) may further include a prediction sample filtering unit (not shown). The encoding device (200) may determine a mode / type to be applied to the current block among a plurality of intra prediction modes / types. The encoding device (200) may compare RD costs for the intra prediction modes / types to determine the optimal intra prediction mode / type for the current block.

[0144] Meanwhile, the encoding device (200) may also perform a prediction sample filtering procedure. The prediction sample filtering procedure may be referred to as post-filtering. Some or all of the prediction samples may be filtered through the prediction sample filtering procedure. In some cases, the prediction sample filtering procedure may be omitted.

[0145] The encoding device (200) performs residual processing on the current block based on (filtered) prediction samples (S1010). Specifically, the encoding device (200) can compare the prediction samples with the original samples of the current block based on phase and derive residual samples.

[0146] The encoding device (200) can encode image information including information regarding the intra prediction (prediction information) and residual information regarding the residual samples (S1020). The prediction information can include the intra prediction mode information and / or the intra prediction type information. The encoding device (200) can output the encoded image information in the form of a bitstream. The output bitstream can be transmitted to the decoding device (300) via a storage medium or a network.

[0147] The residual information may include the residual coding syntax described below. The encoding device (200) may transform / quantize the residual samples to derive quantized transform coefficients. The residual information may include information about the quantized transform coefficients.

[0148] Meanwhile, as described above, the encoding device (200) can generate a restored picture (including restored samples and restored blocks). To this end, the encoding device (200) can inversely quantize / inversely transform the quantized transform coefficients to derive (corrected) residual samples. The reason for performing inverse quantization / inverse transformation on the residual samples after transforming / quantizing them in this way is to derive residual samples that are identical to the residual samples derived from the decoding device (300) as described above. The encoding device (200) can generate a restored block including restored samples for the current block based on the predicted samples and the (corrected) residual samples. A restored picture for the current picture can be generated based on the restored block. As described above, an in-loop filtering procedure, etc. can be further applied to the restored picture.

[0149] FIG. 9 is a flowchart showing a method of decoding an image based on intra prediction in a decoding method according to one embodiment, and FIG. 10 is a diagram showing the configuration of a decoding device used to perform the method of FIG. 9.

[0150] The decoding device (300) can perform an operation corresponding to the operation performed in the encoding device (200).

[0151] Steps S1100 to S1120 may be performed by the intra prediction unit (331) of the decoding device (300), and the prediction information of step S1100 and the residual information of step S1130 may be obtained from the bitstream by the entropy decoding unit (310) of the decoding device (300). The residual processing unit (320) of the decoding device (300) may derive residual samples for the current block based on the residual information. Specifically, the inverse quantization unit (321) of the residual processing unit (320) may perform inverse quantization based on the quantized transform coefficients derived based on the residual information to derive transform coefficients, and the inverse transform unit (322) of the residual processing unit (320) may perform inverse transformation on the transform coefficients to derive residual samples for the current block. Step 1140 may be performed by the addition unit (340) or restoration unit of the decoding device (300).

[0152] Specifically, the decoding device (300) can derive an intra prediction mode / type for the current block based on the received prediction information (intra prediction mode / type information) (S1100). The decoding device (300) can derive surrounding reference samples of the current block (S1110). The decoding device (300) generates prediction samples within the current block based on the intra prediction mode / type and the surrounding reference samples (S1120). In this case, the decoding device (300) can perform a prediction sample filtering procedure. The prediction sample filtering procedure may be referred to as post-filtering. Some or all of the prediction samples may be filtered by the prediction sample filtering procedure. In some cases, the prediction sample filtering procedure may be omitted.

[0153] The decoding device (300) generates residual samples for the current block based on the received residual information. The decoding device (300) generates reconstructed samples for the current block based on the prediction samples and the residual samples, and can derive a reconstructed block including the reconstructed samples (S1130). A reconstructed picture for the current picture can be generated based on the reconstructed block. As described above, an in-loop filtering procedure, etc., can be further applied to the reconstructed picture.

[0154] Here, the intra prediction unit (331) of the decoding device (300) may include an intra prediction mode / type determination unit (331a), a reference sample derivation unit (331b), and a prediction sample derivation unit (331c). The intra prediction mode / type determination unit (331a) determines the intra prediction mode / type for the current block based on the intra prediction mode / type information generated and signaled by the intra prediction mode / type determination unit (222a) of the encoding device (200), the reference sample derivation unit (331b) may derive surrounding reference samples of the current block, and the prediction sample derivation unit (331c) may derive prediction samples of the current block. Meanwhile, although not illustrated, when the above-described prediction sample filtering procedure is performed, the intra prediction unit (331) may further include a prediction sample filtering unit (not illustrated).

[0155] The intra prediction mode information may include flag information (e.g., intra_luma_mpm_flag) indicating, for example, whether the most probable mode (MPM) is applied to the current block or the remaining mode is applied, and if the MPM is applied to the current block, the intra prediction mode information may further include index information (e.g., intra_luma_mpm_idx) indicating one of the intra prediction mode candidates (MPM candidates). The intra prediction mode candidates (MPM candidates) may be composed of an MPM list. A list composed of MPM candidates may be referred to as an MPM list or an MPM candidate list.

[0156] Additionally, if MPM is not applied to the current block, the intra prediction mode information may further include remaining mode information (e.g., intra_luma_mpm_remainder) indicating one of the remaining intra prediction modes excluding the MPM candidates. The decoding device (300) may determine the intra prediction mode of the current block based on the intra prediction mode information.

[0157] In addition, the intra prediction type information can be implemented in various forms. For example, the intra prediction type information can include intra prediction type index information indicating one of the intra prediction types. As another example, the intra prediction type information can include at least one of reference sample line information (e.g., intra_luma_ref_idx) indicating whether MRL is applied to the current block and, if so, which reference sample line is used, ISP flag information (e.g., intra_subpartitions_mode_flag) indicating whether ISP is applied to the current block, ISP type information (e.g., intra_subpartitions_split_flag) indicating the split type of subpartitions if ISP is applied, flag information indicating whether PDCP is applied, or flag information indicating whether LIP is applied. In addition, the intra prediction type information can include a MIP flag indicating whether MIP (Matrix-based Intra Prediction) is applied to the current block.

[0158] Intra prediction mode information and / or intra prediction type information may be encoded / decoded using the coding method described in the disclosed embodiment. For example, intra prediction mode information and / or intra prediction type information may be encoded / decoded using entropy coding (e.g., CABAC, CAVLC) based on a truncated (rice) binary code.

[0159] Below, we describe in more detail how to determine the intra prediction mode.

[0160] When intra prediction is applied, the intra prediction mode applied to the current block can be determined using the intra prediction mode of the surrounding blocks. For example, the decoding device (300) can select one of the MPM candidates within the MPM list derived based on the intra prediction mode of the surrounding blocks of the current block (e.g., the left and / or upper surrounding blocks) and additional candidate modes, based on the received MPM index.

[0161] Alternatively, one of the remaining intra prediction modes that are not included in the above mpm candidates (and planar modes) may be selected based on the remaining intra prediction mode information. The mpm list may be configured to include or not include a planar mode as a candidate. For example, if the mpm list includes a planar mode as a candidate, the mpm list may have six candidates, and if the mpm list does not include a planar mode as a candidate, the mpm list may have three candidates. If the mpm list does not include a planar mode as a candidate, a not planar flag (e.g., intra_luma_not_planar_flag) indicating whether the intra prediction mode of the current block is not a planar mode may be signaled. For example, the mpm flag may be signaled first, and then the mpm index and the not planar flag may be signaled if the value of the mpm flag is 1. Additionally, the mpm index may be signaled if the value of the not planar flag is 1. Here, the reason the mpm list is configured not to include planar mode as a candidate is not because planar mode is not mpm, but rather because planar mode is always considered as mpm, so a flag (not planar flag) is first signaled to check whether it is planar mode or not.

[0162] For example, whether the intra prediction mode applied to the current block is among the mpm candidates (and planar mode) or among the remaining mode can be indicated based on the mpm flag (e.g., intra_luma_mpm_flag). A value of 1 of the mpm flag can indicate that the intra prediction mode for the current block is among the mpm candidates (and planar mode), and a value of 0 of the mpm flag can indicate that the intra prediction mode for the current block is not among the mpm candidates (and planar mode). A value of 0 of the not planar flag (e.g., intra_luma_not_planar_flag) can indicate that the intra prediction mode for the current block is the planar mode, and a value of 1 of the not planar flag can indicate that the intra prediction mode for the current block is not the planar mode. The mpm index can be signaled in the form of mpm_idx or intra_luma_mpm_idx syntax element, and the remaining intra prediction mode information can be signaled in the form of rem_intra_luma_pred_mode or intra_luma_mpm_remainder syntax element. For example, the remaining intra prediction mode information can point to one of the remaining intra prediction modes that are not included in the mpm candidates (and planar modes) among all intra prediction modes by indexing them in the order of the prediction mode number. The intra prediction mode can be an intra prediction mode for the luma component (sample). Hereinafter, the intra prediction mode information may include at least one of an mpm flag (e.g., intra_luma_mpm_flag), a not planar flag (e.g., intra_luma_not_planar_flag), an mpm index (e.g., mpm_idx or intra_luma_mpm_idx), and remaining intra prediction mode information (rem_intra_luma_pred_mode or intra_luma_mpm_remainder).In the disclosed embodiment, the MPM list may be referred to by various terms such as MPM candidate list, candModeList, etc. When MIP is applied to the current block, a separate mpm flag for MIP (e.g. intra_mip_mpm_flag), mpm index (e.g. intra_mip_mpm_idx), and remaining intra prediction mode information (e.g. intra_mip_mpm_remainder) may be signaled, and the aforementioned not planar flag is not signaled.

[0163] The signaling procedure for intra prediction mode information in the encoding device (200) and the intra prediction mode determination procedure in the decoding device (300) can be performed, for example, as follows.

[0164] FIG. 11 is a flowchart illustrating an example of a method for signaling information about an intra prediction mode in an encoding method according to one embodiment.

[0165] Referring to FIG. 11, the encoding device (200) constructs an MPM list for the current block (S1300). As described above, the MPM list may include candidate intra-prediction modes (MPM candidates) likely to be applied to the current block. The MPM list may include intra-prediction modes of surrounding blocks, or may further include specific intra-prediction modes according to a predetermined method. A specific method for constructing the MPM list will be described later.

[0166] The encoding device (200) determines the intra prediction mode of the current block (S1310). The encoding device (200) can perform prediction based on various intra prediction modes, and can determine the optimal intra prediction mode based on rate-distortion optimization (RDO) based thereon. In this case, the encoding device (200) may determine the optimal intra prediction mode using only the MPM candidates and the planar mode included in the MPM list, or may determine the optimal intra prediction mode using not only the MPM candidates and the planar mode included in the MPM list but also the remaining intra prediction modes. As a specific example, if the intra prediction type of the current block is a specific type (e.g., LIP, MRL, or ISP) rather than the normal intra prediction type, the encoding device (200) may consider only the MPM candidates and the planar mode as intra prediction mode candidates for the current block to determine the optimal intra prediction mode. That is, in this case, the intra prediction mode for the current block can only be determined among the MPM candidates and the planar mode, and in this case, the MPM flag may not be encoded / signaled. In this case, the decoding device (300) can assume that the MPM flag is 1 without being separately signaled.

[0167] The encoding device (200) generates intra-prediction mode information based on the determined intra-prediction mode (S1320). The intra-prediction mode information may indicate information regarding the intra-prediction mode determined for the current block. For example, the intra-prediction mode information may include at least one of information indicating whether the intra-prediction mode applied to the current block is included in the MPM list (e.g., mpm flag), information indicating whether the planar mode is applied to the current block (e.g., not planar flag), an mpm index indicating the intra-prediction mode applied to the current block among the MPM list, or remaining intra-prediction mode information.

[0168] Specifically, if the intra prediction mode of the current block is not a planar mode but one of the MPM candidates in the MPM list, the encoding device (200) generates an MPM index (e.g., mpm idx) that points to one of the MPM candidates. If the intra prediction mode of the current block is not in the MPM list either, the encoding device generates remaining intra prediction mode information that points to a mode that is the same as the intra prediction mode of the current block among the remaining intra prediction modes that are not included in the MPM list (and the planar mode).

[0169] The encoding device (200) can encode intra prediction mode information (S1330) and output it in the form of a bitstream. In general, the mpm index and the remaining intra prediction mode information are not signaled simultaneously when indicating the intra prediction mode for one block because they have an alternative relationship. That is, the mpm flag value 1 and the not planar flag or the mpm index are signaled together, or the mpm flag value 0 and the remaining intra prediction mode information are signaled together. However, as described above, when a specific intra prediction type is applied to the current block, the mpm flag may not be signaled, and only the not planar flag and / or the mpm index may be signaled. That is, in this case, the intra prediction mode information may include only the not planar flag and / or the mpm index.

[0170] FIG. 12 is a flowchart illustrating an example of a method for deriving an intra prediction mode in a decoding method according to one embodiment.

[0171] The decoding device (300) can determine the intra prediction mode in response to the intra prediction mode information determined and signaled by the encoding device (200).

[0172] Referring to FIG. 12, the decoding device (300) obtains intra prediction mode information from the bitstream (S1400). The intra prediction mode information may include at least one of an MPM flag, a not-planar flag, an MPM index, or a remaining intra prediction mode, as described above.

[0173] The decoding device (300) constructs an MPM list (S1410). The MPM list is constructed in the same manner as the MPM list constructed in the encoding device (200) described above. That is, the MPM list may include intra prediction modes of surrounding blocks, or may further include specific intra prediction modes according to a predetermined method. A specific PM list construction method is described below.

[0174] Although step S1410 is shown as being performed after step S1400, this is an example, and step S1410 may be performed before step S1400 or may be performed simultaneously.

[0175] The decoding device (300) derives the intra prediction mode of the current block based on the MPM list and the intra prediction mode information (S1420). If the value of the mpm flag is 1, the decoding device (300) may derive the planar mode as the intra prediction mode of the current block (based on the not planar flag) or derive the candidate indicated by the mpm index among the MPM candidates in the MPM list as the intra prediction mode of the current block. If the value of the mpm flag is 0, the decoding device (300) may derive the intra prediction mode indicated by the remaining intra prediction mode information among the remaining intra prediction modes that are not included in the MPM list and the planar mode as the intra prediction mode of the current block. Meanwhile, if the intra prediction type of the current block is a specific type (e.g., LIP, MRL, or ISP, etc.), the decoding device (300) may also derive the candidate indicated by the mpm index in the planar mode or the MPM list as the intra prediction mode of the current block without checking the mpm flag. A specific intra prediction mode derivation procedure like this can be represented as in Fig. 13.

[0176] FIG. 13 is a flowchart illustrating in more detail an example of a method for deriving an intra prediction mode in a decoding method according to one embodiment.

[0177] The description of the step (S1500) in which the decoding device (300) obtains intra prediction mode information from the bitstream and the step (S1510) in which the decoding device (300) constructs the MPM list is the same as described above.

[0178] In deriving the intra prediction mode, if the mpm flag is 1, or a specific intra prediction type such as LIP, MRL, ISP, etc. is applied (example of S1521), and the not planar flag is 1 (example of S1522), the intra prediction mode of the current block can be determined based on the MPM list and mpm index (S1523).

[0179] If the not planar flag is not 1 (NO in S1522), the planar mode can be determined as the intra prediction mode of the current block (S1524).

[0180] If the mpm flag is not 1 (NO in S1521), the intra prediction mode of the current block can be determined from among the remaining intra prediction modes not included in the MPM list based on the remaining intra prediction mode information (S1525).

[0181] The not planar flag may be signaled when MRL does not apply to the current block (i.e., intra_luma_ref_idx = = 0), and may be omitted when MRL applies to the current block (i.e., intra_luma_ref_idx != 0). If the not planar flag is omitted, its value may be assumed to be 1 by the decoding device (300).

[0182] Meanwhile, the intra prediction modes can include two directional intra prediction modes and 65 directional intra prediction modes. The non-directional intra prediction modes can include the planar intra prediction mode and the DC intra prediction mode, and the directional intra prediction modes can include intra prediction modes 2 to 66. The extended directional intra prediction modes can be applied to blocks of all sizes and can be applied to both luma and chroma components.

[0183] In addition to the intra prediction modes described above, intra prediction modes may further include a cross-component linear model (CCLM) mode for chroma samples. CCLM modes can be categorized into LT_CCLM, L_CCLM, and T_CCLM, depending on whether left-side samples, upper-side samples, or both are considered for deriving LM parameters, and can only be applied to chroma components.

[0184] Intra prediction modes can be indexed, for example, as shown in Table 1 below.

[0185] [Table 1]

[0186]

[0187] Meanwhile, the intra prediction type (or additional intra prediction mode, etc.) may include at least one of the aforementioned LIP, PDPC, MRL, ISP, and MIP. The intra prediction type may be indicated based on intra prediction type information, and the intra prediction type information may be implemented in various forms. For example, the intra prediction type information may include intra prediction type index information indicating one of the intra prediction types. As another example, the intra prediction type information may include at least one of reference sample line information (e.g., intra_luma_ref_idx) indicating whether the MRL is applied to the current block and, if so, which reference sample line is used, ISP flag information (e.g., intra_subpartitions_mode_flag) indicating whether the ISP is applied to the current block, ISP type information (e.g., intra_subpartitions_split_flag) indicating a split type of subpartitions if the ISP is applied, flag information indicating whether PDCP is applied, or flag information indicating whether LIP is applied. Additionally, the intra prediction type information may include a MIP flag (or may be called intra_mip_flag) indicating whether MIP is applied to the current block.

[0188] Information regarding the intra prediction mode / type of the current block can be coded and signaled at the CU (CU syntax) level, or implicitly determined based on conditions. In this case, some modes / types may be explicitly signaled, while others may be implicitly derived.

[0189] Below, we describe the specific method of constructing the MPM list.

[0190] Typically, when an image is divided into multiple blocks, the current block to be coded and the neighboring blocks have similar image characteristics. Therefore, the current block and the neighboring blocks are likely to have identical or similar intra-prediction modes. Accordingly, the encoding device (200) can utilize the intra-prediction modes of the neighboring blocks to encode the intra-prediction mode of the current block.

[0191] For example, the encoding device (200) / decoding device (300) can configure an MPM list for the current block. Here, MPM may refer to a mode used to improve coding efficiency by considering the similarity between the current block and surrounding blocks during intra prediction mode coding. As described above, the MPM list may be configured to include a planar mode, or may be configured to exclude a planar mode. For example, if the MPM list includes a planar mode, the number of candidates in the MPM list may be 6. And, if the MPM list does not include a planar mode, the number of candidates in the MPM list may be 5.

[0192] The encoding device (200) / decoding device (300) can configure an MPM list including six MPM candidates.

[0193] The following types of modes can be considered to construct the MPM list.

[0194] - Default intra modes

[0195] - Neighbor intra modes

[0196] - Intra modes derived from surrounding inter modes (IPM modes)

[0197] - DIMD (Decoder-side intra mode derivation) modes

[0198] - Derived intra modes

[0199] For the above peripheral intra modes, peripheral blocks, such as the left peripheral block, the upper peripheral block, the lower left peripheral block, the upper right peripheral block, and the upper left peripheral block, may be considered. When utilizing the peripheral intra modes with MPM, the input order may vary depending on the size information of the current block. For example, if the height of a block is greater than or equal to the width of the block, the intra mode of the upper peripheral block may be considered first, and then the intra mode of the left peripheral block may be considered.

[0200] Even when neighboring blocks are coded in inter mode rather than intra mode, intra mode information can be obtained from the IPM buffer. If the position indicated by the motion vector of a neighboring inter block is in intra mode, i.e., if the block at that position is coded in intra mode, the corresponding intra mode can be stored in the IPM buffer. The intra mode stored in the IPM buffer can be used as the MPM mode of the neighboring block.

[0201] Additionally, it is possible to construct an MPM list using the DIMD described below. If the current block is not in DIMD mode, the intra mode derived from DIMD can be used as an MPM candidate.

[0202] As described above, if the MPM list does not include a planar mode, the planar mode is excluded from the MPM list, and the number of candidates in the MPM list can be set to 5.

[0203] The above-described MPM list construction method can be used when MIP is not applied to the current block. For example, the above-described MPM list construction method can be used to derive intra prediction modes used in LIP, PDPC, MRL, ISP intra prediction, or normal intra prediction.

[0204] Meanwhile, the left peripheral block or the upper peripheral block can be coded based on the above-described MIP. That is, MIP can be applied when coding the left peripheral block or the upper peripheral block. In this case, it is not appropriate to use the MIP intra prediction mode number of the peripheral block (left peripheral block / upper peripheral block) to which the MIP is applied as it is in the MPM list for the current block to which the MIP is not applied. For example, the intra prediction mode of the peripheral block (left peripheral block / upper peripheral block) to which the MIP is applied can be considered as DC or planar mode. As another example, the intra prediction mode of the peripheral block (left peripheral block / upper peripheral block) to which the MIP is applied can be mapped to a general intra prediction mode based on a mapping table and used to construct the MPM list. In this case, the mapping can be performed based on the block size type of the current block. For example, the mapping table can be represented as shown in Table 3 below.

[0205] [Table 2]

[0206]

[0207] Here, MIP IntraPredMode[ xNbX ][ yNbX ] represents the MIP intra prediction mode of the surrounding block (left surrounding block / upper surrounding block), and the block size type MipSizeId represents the block size type of the surrounding block or the current block. The numbers under the block size type values ​​0, 1, and 2 represent the general intra prediction mode to which the MIP intra prediction mode is mapped for each block size type.

[0208] For example, if the block size type of the current block is 0 and the MIP intra prediction mode number of the surrounding block is 10, the mapped general intra prediction mode number may be 18. However, the above mapping relationship is an example and may be changed.

[0209] When MIP is applied to the current block, a separate MPM list for the current block to which MIP is applied can be configured. The MPM list can be called by various names, such as MIP MPM list (or MPM list for MIP, candMipModeList), to distinguish it from the MPM list for cases where MIP is not applied to the current block. Hereinafter, for the sake of distinction, it is referred to as MIP MPM list, but of course, it can also be called MPM list.

[0210] The above MIP MPM candidate list may include n candidates, for example, n may be 3. The above MIP MPM candidate list may be constructed based on a left peripheral block and an upper peripheral block of the current block. Here, the left peripheral block may refer to the uppermost block among the peripheral blocks adjacent to the left boundary of the current block. In addition, the upper peripheral block may refer to the leftmost block among the peripheral blocks adjacent to the upper boundary of the current block.

[0211] For example, when MIP is applied to the left neighboring block (and the block size type of the left neighboring block is the same as the block size type of the current block), the first candidate intra prediction mode (or candMipModeA) may be set to be the same as the MIP intra prediction mode of the left neighboring block. Also, for example, when MIP is applied to the upper neighboring block (and the block size type of the upper neighboring block is the same as the block size type of the current block), the second candidate intra prediction mode (or cand-ipModeB) may be set to be the same as the MIP intra prediction mode of the upper neighboring block.

[0212] Meanwhile, the left neighboring block or the upper neighboring block may be coded based on intra prediction rather than MIP. That is, an intra prediction type other than MIP may be applied when coding the left neighboring block or the upper neighboring block. In this case, it is not appropriate to use the normal intra prediction mode number of the neighboring block (left neighboring block / upper neighboring block) to which MIP is not applied as a candidate intra mode for the current block to which MIP is applied. For example, the MIP intra prediction mode of the neighboring block (left neighboring block / upper neighboring block) to which MIP is not applied may be regarded as a MIP intra prediction mode of a specific value (e.g., 0, 1, or 2). As another example, the normal intra prediction mode of the neighboring block (left neighboring block / upper neighboring block) to which MIP is not applied may be mapped to the MIP intra prediction mode based on a mapping table and used to construct a MIP MPM candidate list. In this case, the mapping may be performed based on the block size type of the current block. For example, the mapping table may be represented as in Table 4 below.

[0213] [Table 3]

[0214]

[0215] Here, IntraPredModeY[ xNbX ][ yNbX ] represents the intra prediction mode of the surrounding block (left surrounding block / upper surrounding block). Here, the intra prediction mode of the surrounding block may be an intra prediction mode for the luma component (sample), i.e., a luma intra prediction mode. The block size type MipSizeId represents the block size type of the surrounding block or the current block. The numbers under the block size type values ​​0, 1, and 2 represent the MIP intra prediction mode to which the general intra prediction mode is mapped for each block size type.

[0216] Additionally, the above-mentioned surrounding blocks (e.g., left surrounding block / upper surrounding block) may not be available (e.g., located outside the current picture, located outside the current tile / slice, etc.), or even if MIP is applied, a MIP intra prediction mode that is not available to the current block may be used depending on the block size type. In this case, a specific MIP intra prediction mode predefined for the first candidate and / or the second candidate may be used as the first candidate intra prediction mode or the second candidate intra prediction mode. Additionally, a specific MIP intra prediction mode predefined for the third candidate may be used as the third candidate intra prediction mode.

[0217] For example, the above-described specific MIP intra prediction mode can be represented as shown in Table 5 below. The above-described specific MIP intra prediction mode can be called MIP default candidate modes.

[0218] [Table 4]

[0219]

[0220] The MIP MPM list can be constructed based on the first candidate intra prediction mode and the second candidate intra prediction mode. For example, when the first candidate intra prediction mode and the second candidate intra prediction mode are different, the first candidate intra prediction mode can be placed as the 0th candidate (ex. candMipModeList[0]) of the MIP MPM list, and the second candidate intra prediction mode can be placed as the 1st candidate (ex. candMipModeList[1]) of the MIP MPM list. The second candidate (ex. candMipModeList[2]) of the MIP MPM list can use the above-described predefined specific MIP intra prediction mode.

[0221] Alternatively, when the first candidate intra prediction mode and the second candidate intra prediction mode are the same, one of the first candidate intra prediction mode and the second candidate intra prediction mode can be inserted as the 0th candidate (ex. candMipModeList[0]) of the MIP MPM list, and the first candidate (ex. candMipModeList[1]) of the MIP MPM list and the second candidate (ex. candMipModeList[2]) of the MIP MPM list can use the above-described predefined specific MIP intra prediction modes.

[0222] As described above, the MIP intra prediction mode of the current block can be derived based on the MIP MPM list. In this case, the MPM flag that can be included in the intra prediction mode information for the MIP as described above can be called intra_mip_mpm_flag, the MPM index can be called intra_mip_mpm_idx, and the remaining intra prediction mode information can be called intra_mip_mpm_remainder. The procedure for deriving the MIP intra prediction mode from the MIP MPM list can be performed as described above.

[0223] Figure 14 is a drawing showing a template and reference sample used in TIMD mode.

[0224] Referring to FIG. 14, in TIMD (Template based intra mode derivation), the Sum of Absolute Transformed Difference (SATD) between the predicted block predicted from the template region and the actual restored sample is calculated for the IPM intra modes of the surrounding adjacent intra blocks and inter blocks, and the mode with the smallest SATD can be selected as the intra mode of the current block.

[0225] Alternatively, the two modes with the lowest SATD can be selected, and the prediction blocks for the two prediction modes can be blended using a weighted sum method to use them as the prediction block for the current block.

[0226] Blending two modes can be applied when the following conditions are met:

[0227] costMode2 < 2*costMode1.

[0228] If the above conditions are satisfied, the prediction block is generated by blending two modes, otherwise only one mode with the minimum SATD value can be selected.

[0229] The ratio of weights weight1 and weight 2 when blending two prediction blocks is as follows.

[0230] weight1 = costMode2 / (costMode1+ costMode2)

[0231] weight2 = 1 - weight1

[0232]

[0233] Previously, we explained how to construct the MPM list. Below, we explain how to construct the SMPM (Secondary MPM) candidate list.

[0234] The encoding device (200) / decoding device (300) may also construct an SMPM list for the current block. The SMPM list may also be referred to as an SMPM list. In addition, the SMPM list may be constructed by excluding intra-modes that overlap with the MPM list. For example, if the MPM list includes mode 50, the SMPM list may not include mode 50.

[0235] For example, the encoding device (200) / decoding device (300) can compose a list with M SMPM candidates, where M can be 16. However, of course, the number of SMPM candidates can vary.

[0236] The following types of modes can be considered to construct the SMPM list.

[0237] - Default intra modes

[0238] - Neighbor intra modes

[0239] - Intra modes derived from surrounding inter modes (IPM modes)

[0240] - DIMD (Decoder-side intra mode derivation) modes

[0241] - Derived intra modes

[0242] For the above peripheral intra modes, peripheral blocks, such as the left peripheral block, the upper peripheral block, the lower left peripheral block, the upper right peripheral block, and the upper left peripheral block, may be considered. When utilizing the peripheral intra modes as SMPM candidates, the input order may vary depending on the size information of the current block. For example, if the block height is greater than or equal to the block width, the intra mode of the upper peripheral block may be considered first, and then the intra mode of the left peripheral block may be considered.

[0243] Even when neighboring blocks are coded in inter mode rather than intra mode, intra mode information can be obtained through the IPM buffer. If the intra mode is applied to the position indicated by the motion vector of a neighboring inter block, the corresponding intra mode can be stored in the IPM buffer. The intra mode stored in the IPM buffer can be used as a SMPM candidate for the neighboring block.

[0244] When intra prediction is applied to a current block, peripheral reference samples to be used for intra prediction of the current block can be derived. The peripheral reference samples of the current block may include a total of 2 x nH samples adjacent to the left boundary and bottom-left neighbors of the current block of a size nW x nH, a total of 2 x nW samples adjacent to the top boundary and top-right neighbors of the current block, and one sample adjacent to the top-left of the current block. Alternatively, the peripheral reference samples of the current block may include upper peripheral samples of multiple columns and left peripheral samples of multiple rows. In addition, the peripheral reference samples of the current block may include a total of nH samples adjacent to the right boundary of the current block of a size nW x nH, a total of nW samples adjacent to the bottom boundary of the current block, and one sample adjacent to the bottom-right of the current block.

[0245] Meanwhile, some of the surrounding reference samples of the current block may not yet be decoded or available. In this case, the decoding device (300) can construct surrounding reference samples to be used for prediction through interpolation of available samples.

[0246] Alternatively, if some of the surrounding reference samples of the current block have not yet been decoded or are not available, the decoding device (300) may construct surrounding reference samples to be used for prediction through extrapolation of the available samples. This may be done by substituting or padding pixels that have not yet been decoded or are not available with the last available sample while updating the referenceable samples with the latest samples, starting from the lower left and reaching the upper right reference sample.

[0247] As described above, when the intra prediction mode to be applied to the current block and the surrounding reference samples are derived, the prediction unit of the encoding device (200) / decoding device (300) can derive a reference sample according to the intra prediction mode of the current block among the surrounding reference samples of the current block, and can generate a prediction sample of the current block based on the reference sample.

[0248] As described above, (i) the prediction sample can be derived based on the average or interpolation of neighboring reference samples of the current block, and (ii) the prediction sample can also be derived based on reference samples existing in a specific (prediction) direction with respect to the prediction sample among the neighboring reference samples of the current block. The case of (i) can be called a non-directional mode or a non-angular mode, and the case of (ii) can be called a directional mode or an angular mode.

[0249] Additionally, a prediction sample may be generated by interpolating a second surrounding sample and a first surrounding sample, which are located in the opposite direction of the prediction direction of the intra prediction mode of the current block based on the prediction sample of the current block among surrounding reference samples. This method may be referred to as linear interpolation intra prediction (LIP).

[0250] Additionally, a temporary prediction sample of the current block can be derived based on filtered peripheral reference samples, and a prediction sample of the current block can be derived by weighting at least one reference sample derived according to the intra prediction mode among existing peripheral reference samples, i.e., unfiltered peripheral reference samples, and the temporary prediction sample. This method can be called Position Dependent Intra Prediction (PDPC).

[0251] In addition, intra prediction encoding can be performed by selecting a reference sample line with the highest prediction accuracy among the surrounding multiple reference sample lines of the current block, deriving a prediction sample using the reference sample located in the prediction direction of the selected line, and instructing (signaling) the reference sample line used at this time to a decoding device. This method may be called multi-reference line intra prediction (MRL) or MRL-based intra prediction.

[0252] In addition, the current block can be divided into vertical or horizontal subpartitions, and intra prediction can be performed based on the same intra prediction mode, while peripheral reference samples can be derived and utilized for each subpartition. That is, in this case, the intra prediction mode for the current block is applied equally to the subpartitions, and peripheral reference samples can be derived and utilized for each subpartition, thereby improving intra prediction performance in some cases. This prediction method can be called intra subpartitions (ISP) or ISP-based intra prediction.

[0253] If the prediction direction based on the prediction sample points between surrounding reference samples, that is, if the prediction direction points to a fractional sample location, the value of the prediction sample can also be derived through interpolation of multiple reference samples located around the prediction direction (around the fractional sample location).

[0254] As mentioned above, the intra prediction methods described above may be referred to as intra prediction types to distinguish them from the normal intra prediction mode or regular intra prediction mode described above. In addition, the intra prediction type may be referred to by various terms such as intra prediction technique, intra prediction tool, or additional intra prediction mode. For example, the intra prediction type (or additional intra prediction mode, etc.) may include at least one of the LIP, PDPC, MRL, and ISP described above. Information about the intra prediction type may be encoded in an encoding device and included in a bitstream to be signaled to a decoding device. In addition, information about the intra prediction type may be implemented in various forms such as flag information indicating whether each intra prediction type is applied or index information indicating one of multiple intra prediction types.

[0255] The MPM list for deriving the intra prediction mode described above may be configured differently depending on the intra prediction type. Alternatively, the MPM list may be configured in a common manner regardless of the intra prediction type.

[0256] Fig. 15 is a drawing showing a configuration method of HoG used in DIMD mode, and Fig. 16 is a drawing showing a configuration of a prediction block in DIMD mode.

[0257] The DIMD mode can be derived and used by the encoder and decoder without directly transmitting intra prediction mode information. First, horizontal and vertical gradients are obtained from the second neighboring sample column and row, and a Histogram of Gradients (HoG) can be constructed from them.

[0258] Referring to Figure 15, the HoG can be obtained by applying a Sobel filter using L-shaped rows and columns of 3 pixels around the current block. If the block boundaries exist in different CTUs, they are not used for texture analysis.

[0259] Afterwards, up to five intra modes with the largest histogram amplitudes are selected, and the predicted blocks predicted using these modes are blended with the planar mode to form the final predicted block. The weights can be derived from the histogram amplitudes. Additionally, a DIMD flag is transmitted on a block-by-block basis to determine whether DIMD is being used.

[0260] Figure 16 is an example of selecting two intra modes with the largest histogram amplitudes, and then blending the predicted blocks predicted using these modes with the planar mode to construct a final predicted block.

[0261] For example, for a block of size WХH, the weights for each of the five derived modes are adjusted if the magnitude of the upper histogram is twice that of the left histogram, or vice versa. In these cases, the weights vary depending on the position and are calculated as follows:

[0262] If the upper histogram is twice as large as the left histogram,

[0263] [Formula 1]

[0264]

[0265] If the left histogram is twice as large as the upper histogram,

[0266] [Formula 2]

[0267]

[0268] Here, wDimd_i represents the unmodified uniform weight for the selected DIMD, and △_i is a predefined value that can be set to 10.

[0269] Figure 17 is a diagram showing a peripheral restoration sample used in DIMD chroma mode.

[0270] Referring to FIG. 17, the DIMD chroma mode uses the DIMD derivation method to derive the chroma intra prediction mode of the current block based on the reconstructed Y, Cb, and Cr samples located in the adjacent second row and column.

[0271] Specifically, a horizontal gradient and a vertical gradient are calculated for each of the restored luma samples corresponding to the current chroma block and each of the restored Cb and Cr samples, thereby constructing a gradient histogram.

[0272] Afterwards, the intra prediction mode with the largest amplitude value of the histogram is used to perform chroma intra prediction of the current chroma block.

[0273] If the intra prediction mode derived from the DIMD chroma mode is identical to the intra prediction mode derived from the DM mode, the intra prediction mode with the second largest histogram amplitude value is used as the DIMD chroma mode. A CU-level flag is transmitted in the syntax to indicate whether the proposed DIMD chroma mode is applied.

[0274] Finally, for the WХH pair of the chroma CB to be predicted, pairs of vertical gradients and horizontal gradients are extracted from the second and third lines of the luma CB instead of the usual DIMD decoding reference sample set to construct a gradient histogram corresponding to the luma CB.

[0275] Another example of an intra prediction type applicable to one embodiment is intra prediction fusion. Intra prediction fusion combines multiple predictors generated from different reference lines using a weighted averaging method to derive a predicted sample. In this process, multiple intra predictors are generated and then combined through weighted averaging. The process for deriving the predictors to be combined is as follows.

[0276] For angular intra prediction modes including single modes of TIMD and DIMD, the present method derives intra prediction by applying weights in the form of p_fusion = w_0·p_line + w_1·p_(line+1) using p_line, which is an intra prediction from a default reference line, and p_(line+1), which is a prediction from a line directly above the default reference line. Here, the weights can be set to w_0 = 3 / 4 and w_1 = 1 / 4.

[0277] For TIMD modes with blending applied, p_line is used in the first mode (w_0 = 1, w_1 = 0), and p_(line+1) is used in the second mode (w_0 = 0, w_1 = 1).

[0278] For DIMD mode with blending applied, the number of predictors used in the weighted average increases from 3 to 6.

[0279] Intra prediction combination can be applied to luma blocks with a block size greater than 16 when the angular intra mode has a non-integer slope (i.e., reference sample interpolation is required). This method is used with Multiple Reference Lines (MRL) and is not applicable to blocks encoded with Intra Sub-Partitions (ISP). Predictor Dependent Prediction Combination (PDPC) can be applied for intra prediction modes that use the reference line closest to the current block.

[0280] When intra prediction is performed on the current block, prediction for the luma component block (luma block) of the current block and prediction for the chroma component block (chroma block) can be performed, and in this case, the intra prediction mode for the chroma component (chroma block) can be set separately from the intra prediction mode for the luma component (luma block).

[0281] For example, an intra prediction mode for a chroma component may be indicated based on intra chroma prediction mode information, and the intra chroma prediction mode information may be signaled in the form of an intra_chroma_pred_mode syntax element. As an example, the intra chroma prediction mode information may indicate one of candidate modes including at least one of a planar mode, a DC mode, a vertical mode, a horizontal mode, a DM, an L_CCLM, a T_CCLM, and a LT_CCLM mode. DM may be referred to as a direct mode, and CCLM may be referred to as LM.

[0282] Meanwhile, DM and CCLM are dependent intra prediction modes that use information from a luma block to predict a chroma block. DM may represent a mode in which the same intra prediction mode for the luma component is applied as the intra prediction mode for the chroma component. Furthermore, CCLM may represent an intra prediction mode in which, in the process of generating a prediction block for a chroma block, reconstructed samples of the luma block are subsampled, and then the CCLM parameters α and β are applied to the subsampled samples to use the resulting samples as prediction samples for the chroma block.

[0283] For example, Multiple Direct Modes (MDM) can be applied to the current chroma block.

[0284] Multiple Direct mode extends the existing single-mode DM mode to multiple modes. That is, when configuring the intra-prediction mode for a chrominance image, multiple DM modes are selected as follows:

[0285] - Intra prediction mode of CR, TL, TR, BL, BR of the same position luminance block (see Fig. 1.5.2-1)

[0286] - Intra prediction mode of L, A, BL, AR, AL blocks, which are blocks surrounding the current color block.

[0287] - Planner, DC mode

[0288] - -1 or +1 angular mode to previously selected angular mode

[0289] - Vertical, Horizontal, 2, 34, 66, 10, 26 modes (in case of 65 direction mode)

[0290] - If 5 prediction modes are not selected, copy the previously selected mode and select it.

[0291] In addition, the CCLM mode may be applied to the current chroma block. The CCLM mode is an intra prediction mode that utilizes the correlation between a luma block and a chroma block corresponding to the luma block, and may represent a mode in which a linear model may be derived based on surrounding samples of the luma block and surrounding samples of the chroma block, and prediction samples of the chroma block may be derived based on the linear model and the reconstructed samples of the luma block. Specifically, when the CCLM mode is applied to the current chroma block, parameters for the linear model may be derived based on surrounding samples used for intra prediction of the current chroma block and surrounding samples used for intra prediction of the current luma block.

[0292] To reduce cross-component redundancy, the CCLM prediction mode is used, which predicts chroma samples based on reconstructed luma samples of the same CU using the following linear model:

[0293] For example, the above linear model can be expressed based on the following equation 3.

[0294] [Formula 3]

[0295]

[0296] Here, predc(i,j) may represent a prediction sample at the (i,j) coordinate of the current chroma block within the current CU, and recL′(i,j) may represent a reconstruction sample at the (i,j) coordinate of the current luma block within the CU. In addition, recL′(i,j) may represent a down-sampled reconstruction sample of the current luma block.

[0297] The CCLM parameters (α and β) are derived using up to four adjacent chroma samples and their corresponding downsampled luma samples. Assuming the current chroma block size is WХH, W′ and H′ are set as follows:

[0298] - When LM mode is applied: W′= W, H′ = H

[0299] - When LM-A mode is applied: W′= W + H

[0300] - When LM-L mode is applied: H′= H+ W

[0301] The upper adjacent positions are denoted as S[0, -1]…S[W′-1, -1], and the left adjacent positions are denoted as S[-1, 0]…S[-1, H′-1]. Then, four samples are selected as follows:

[0302] - When LM mode is applied and both top and left adjacent samples are available:

[0303] S[W′ / 4, -1], S[3W′ / 4, -1], S[-1, H′ / 4], S[-1, 3H′ / 4]

[0304] - When LM-A mode is applied or only top adjacent samples are available:

[0305] S[W′ / 8, -1], S[3W′ / 8, -1], S[5W′ / 8, -1], S[7W′ / 8, -1]

[0306] - When LM-L mode is applied or only left adjacent samples are available:

[0307] S[-1, H′ / 8], S[-1, 3H′ / 8], S[-1, 5H′ / 8], S[-1, 7H′ / 8]

[0308] The four adjacent luma samples corresponding to the selected location are downsampled and compared, and the two smaller values ​​are x 0 A and x 1 In A, the two larger values ​​are x 0 B and x 1 is set to B. The corresponding chroma sample values ​​are y 0 A, y 1 A, y 0 B, y 1 It is represented by B. After that, x A , x B , y A , y B is derived as follows: Equation 4:

[0309] [Formula 4]

[0310]

[0311] The linear model parameters α and β are calculated according to the following equation 5:

[0312] [Formula 5]

[0313]

[0314] The above template and left template described above can be used to compute linear model coefficients together, but can also be used individually in two other LM modes, LM_A mode and LM_L mode.

[0315] In LM_A mode, only the upper template is used to compute the linear model coefficients. To obtain more samples, the upper template is extended to (W+H).

[0316] In LM_L mode, only the left template is used to compute the linear model coefficients. To obtain more samples, the left template is extended to (H+W).

[0317] For non-square blocks, the top template extends to W+W, and the left template extends to H+H.

[0318] To match the chroma sample positions in a 4:2:0 video sequence, luma samples are subjected to two types of downsampling filters to achieve a 2:1 downsampling ratio in both the horizontal and vertical directions. The choice of downsampling filter is specified by a flag at the SPS level.

[0319] For example, if the top reference line is located at the CTU boundary, only one luma line (the normal line buffer in intra prediction) is used to generate the downsampled luma samples.

[0320] This parameter calculation is performed as part of the decoding process and is not handled by a simple encoder search operation. Consequently, no syntax is used to pass the values ​​of α and β to the decoder.

[0321] For chroma intra-mode coding, a total of eight intra-modes are allowed. These modes include five traditional intra-modes and three cross-component linear model modes (CCLM, LM_A, and LM_L). The signaling and derivation processes for the chroma modes are shown in Tables 1 and 2 below.

[0322] Chroma mode encoding directly depends on the intra prediction mode of the corresponding luma block. Since I-slices allow separate block partitioning structures for luma and chroma components, a single chroma block can correspond to multiple luma blocks. Therefore, for chroma DM mode, the intra prediction mode of the corresponding luma block, including the center position of the current chroma block, can be directly inherited.

[0323] Table 6 described below shows a mapping table for deriving an intra chroma prediction mode when CCLM is not available, and Table 7 shows a mapping table for deriving an intra prediction mode when CCLM is available. As shown in the tables, the intra chroma prediction mode can be determined based on the intra luma prediction mode for a luma block covering the center lower right sample of the current block or chroma block (e.g., when DUAL_TREE is applied) and the value of the signaled intra chroma prediction mode (intra_chroma_pred_mode) information. The indices of IntraPredModeC[ xCb ][ yCb ] derived in the tables below can correspond to the indices of the intra prediction modes described above.

[0324] [Table 5]

[0325]

[0326] [Table 6]

[0327]

[0328] Figure 18 is a flowchart showing an example of a processing flow for generating an intra prediction block.

[0329] DIMD and TIMD are methods for weighted-summing (hereinafter, blending) multiple intra modes, and this method can improve coding efficiency. However, the combinations of weighted-summing multiple intra modes can vary. For example, DIMD generates a prediction block by weighted-summing up to five non-directional modes and directional modes. TIMD generates a prediction block by weighted-summing up to two non-directional modes and directional modes. Although DIMD and TIMD have many similarities in that they derive intra modes, weight-sum multiple prediction blocks, and derive weight values ​​for the weighted sum, they are applied separately as different modes, and completely different prediction blocks can be generated depending on how multiple modes are combined with which weights.

[0330] The present invention proposes an intra prediction mode based on template-based blending to minimize unnecessary signaling overhead while embracing the respective strengths of DIMD and TIMD. The number of intra modes to combine and the weighting values ​​to apply are too diverse. Transmitting these combinations to the decoder via signaling can incur significant signaling overhead. Therefore, we propose a method that finds the optimal combination in the template region and performs blending.

[0331] The method of the present embodiment defines separate syntax information in high-level parameters such as VPS, SPS, APS, PPS, Picture Header, or Slice Header, and whether to use the information can be determined when the information has a specific value. For example, if the value is 1, it can be determined to use the template-based blending method.

[0332] The flowchart of this embodiment may be as illustrated in Fig. 18. Each sequence may be changed or omitted.

[0333] - Intra-mode candidate configuration: This is the step of configuring intra-mode candidates. The method of configuring intra-mode candidates can be defined through prior agreement between the encoder and decoder.

[0334] for example,

[0335] (Method 1) Differential filtering is performed on the surrounding restoration area of ​​the current block, and a Histogram of gradients (HoG) can be constructed based on the gradient and amplitude (magnitude of change, e.g., the sum of the horizontal change magnitude and the vertical change magnitude) of the pixel values ​​obtained based on the ratio of horizontal change magnitude and vertical change magnitude, and intra mode candidates can be constructed in order of increasing amplitude. This method may be the same as or similar to the above DIMD.

[0336] (Method 2) The error values ​​for the reconstructed samples around the current block can be calculated for the intra modes stored in adjacent / non-adjacent intra / inter blocks around the current block (e.g., the above 1.2.2 MPM, the above 1.2.5 secondary MPM candidate), or / and the intra modes predefined between the encoder and decoder (e.g., Planar, DC, horizontal direction, vertical direction mode), and the intra mode candidates can be configured in order of the lowest error values. This method can be the same as or similar to the above TIMD.

[0337] This is one example, and other intra-mode candidates can be configured by pre-defining between the in / decoder.

[0338] To obtain the error value, error calculation methods such as SAD (Sum of Absolute difference), SATD (Sum of transformed Absolute difference), SSE (Sum of squared error), MR-SAD (Mean-removed sum of Absolute difference), MR-SSE (Mean-removed sum of squared error), and MR-SATD (Mean-removed sum of Absolute transformed difference) can be used.

[0339] (Method 3) Intra mode candidates can be configured for predefined default modes. For example, the default mode can be defined as { DC mode, Vertical mode, Horizontal mode, Vertical mode - 4, Vertical mode + 4, Horizontal mode - 4, Horizontal mode + 4, Vertical mode - 8, Vertical mode + 8, Horizontal mode - 8, Horizontal mode + 8, Vertical mode - 12, Vertical mode + 12, Horizontal mode - 12, Horizontal mode + 12, 34, 66, 2, Vertical mode - 2, Vertical mode + 2, Horizontal mode -2, Horizontal mode +2 …}. This is an example, and the default mode can be defined in a predefined intra mode order between the encoder and decoder.

[0340] Using the above methods, up to M intra mode candidates can be constructed. M can be, for example, 6. M is a positive integer, and can be determined to a different value by defining it between the encoder and decoder.

[0341] Alternatively, an intra-mode candidate can be constructed using a combination of the above methods. For example, if the size of the current block (width x height) is less than or equal to 256, an intra-mode candidate can be constructed using method 1, otherwise, an intra-mode candidate can be constructed using method 2. Conversely, if the size of the current block is less than or equal to 256, an intra-mode candidate can be constructed using method 2, otherwise, an intra-mode candidate can be constructed using method 1.

[0342] Alternatively, P candidates can be constructed using Method 1, Q candidates can be constructed using Method 2, and a total of P+Q(=M) intra-mode candidates can be constructed. For example, P can be 4 and Q can be 2. Alternatively, if M intra-mode candidates are not constructed using Method 1, intra-mode candidates can be constructed using Method 2 until M are filled. The above-mentioned case is an example of constructing intra-mode candidates by combining different methods, and can be confirmed and used with different values ​​and / or different conditions. The above-mentioned case is an example of combining Method 1 and Method 2, but Method 3 can also be applied and combined. That is, a combination of Method 1 and Method 2, a combination of Method 1 and Method 3, and a combination of Method 2 and Method 3 are all possible.

[0343] - Template-based mode candidate and weight determination: This is a step of generating a prediction block of blending candidates for a template area defined around the current block, calculating an error value with respect to the restored sample, and determining a blending candidate with a small error value. For example, when M is 6, the number of blending candidates for generating the final prediction block can be 1 to 6. In addition, even if the number of blending candidates is the same, the combinations can be diverse, such as (candidate 1, candidate 2), (candidate 1, candidate 3), …, (candidate 5, candidate 6). By calculating an error value for various candidates, a blending candidate can be determined.

[0344] For example, the blending candidates may be as follows: The M intra-mode candidates may be defined as M1 to M6.

[0345] - M1 (single mode, not blending)

[0346] - M2 (single mode, not blending)

[0347] - M3 (single mode, not blending)

[0348] - M4 (single mode, not blending)

[0349] - M5 (single mode, not blending)

[0350] - M6 (single mode, not blending)

[0351] - M1, M2

[0352] - M1, M2, M3

[0353] - M1, M2, M3, M4

[0354] - M1, M2, M3, M4, M5

[0355] - M1, M2, M3, M4, M5, M6

[0356] - M1, Planar mode

[0357] - M2, Planar mode

[0358] - M3, Planar mode

[0359] - M4, Planar mode

[0360] - M5, Planar mode

[0361] - M6, Planar mode

[0362] - M1, M2, Planar mode

[0363] - M1, M2, M3, Planar mode

[0364] - M1, M2, M3, M4, Planar mode

[0365] - M1, M2, M3, M4, M5, Planar mode

[0366] This is one example of a combination of M1 to M6 and a combination with the omnidirectional mode, and other combinations and / or partial combinations of the above examples may be determined by an inter-input / decoder agreement.

[0367] In order to obtain the error value for the above blending candidates, it is necessary to define weights for each intra mode.

[0368] For example, weights can be defined as follows:

[0369] If the method of obtaining HoG by performing differential filtering in the above intra mode candidate configuration step is used, the weight value can be determined based on the amplitude of the HoG. That is, the weight can be determined to be proportionally larger when the amplitude of the corresponding intra mode is large. For example, for the (M1, M2) blending candidates, if each amplitude is (a1, a2), the weight of M1 can be determined to have a ratio of a1 / (a1+a2), and the weight of M2 can be determined to have a ratio of a2 / (a1+a2). This can determine the weights in a similar / same way as DIMD.

[0370] Alternatively, when a method similar to TIMD is used in the intra mode candidate configuration step, the weight value can be determined based on the error value of the intra mode. That is, the weight can be determined to be inversely larger when the error value of the corresponding intra mode is small. For example, for the (M1, M2) blending candidates, when each error value is (c1, c2), the weight of M1 can be determined to have a ratio of c2 / (c1+c2), and the weight of M2 can be determined to have a ratio of c1 / (c1+c2). This allows the weights to be determined in a method similar to / same as TIMD.

[0371] Alternatively, the weight values ​​can be predefined as follows:

[0372] For two blending candidates, the weight ratio is:

[0373] (1 / 4, 3 / 4),

[0374] (2 / 4, 2 / 4),

[0375] (3 / 4, 1 / 4),

[0376] For the three blending candidates, the weight ratio is:

[0377] (2 / 4, 1 / 4, 1 / 4),

[0378] (1 / 4, 2 / 4, 1 / 4),

[0379] (1 / 4, 1 / 4, 2 / 4),

[0380] For the four blending candidates, the weight ratios are:

[0381] (4 / 16, 4 / 16, 4 / 16, 4 / 16),

[0382] (8 / 16, 4 / 16, 2 / 16, 2 / 16),

[0383] (12 / 16, 2 / 16, 1 / 16, 1 / 16),

[0384] This is an example of defining weights for two to four blending candidates, and weights can be defined for five or more blending candidates in the same way. Other weight combinations and / or subcombinations of the above examples can be determined by agreement between the encoder and decoder.

[0385] The error value in the template region can be calculated for the above blending candidates and weight candidates. The formula for calculating the error value (hereinafter, "Er") for a single blending candidate is as follows.

[0386] [Formula 6]

[0387]

[0388] ReconSamples: Reconstruction samples of the current block and the adjacent template area.

[0389] K: K is a positive integer and is the number of blending candidates.

[0390] : A predicted block generated with the i-th intra mode of the intra mode candidate in the template area adjacent to the current block.

[0391] : Weight ratio to be applied to the i-th prediction block ( )

[0392] w is a real number

[0393] Although the above formula explains that the error value is obtained using SAD, this is just one example, and to obtain the error value, error calculation methods such as SAD (Sum of Absolute Difference), SATD (Sum of Absolute Transformed Difference), SSE (Sum of Squared Error), MR-SAD (Mean-Removed Sum of Absolute difference), MR-SSE (Mean-Removed Sum of Squared Error), and MR-SATD (Mean-Removed Sum of Absolute Transformed Difference) can be used.

[0394] The above step may be omitted. For example, this may be the case when it is difficult to obtain sufficient intra-mode candidates from the current block. For example, this may be the case when at least N candidates are not filled in the intra-mode candidate construction step. For example, N may be 2. N may be a positive integer and may be a value predefined between the encoder and decoder. Alternatively, this may be the case when the y position of the current block is 0, the x position is 0, or the current block is located at the upper left of the current picture.

[0395] - Prediction block generation: An error value is calculated for the above blending candidates, and a prediction block can be generated using a blending candidate with a small error value.

[0396] For example, the above-described template-based blending method can be included in the mode of 1.4.10 DIMD and operate. That is, rather than inducing an intra mode like the current DIMD and always blending with a combination of the planar mode and up to P (e.g., P is 5), the method can find an optimal blending candidate based on a template and generate a prediction block.

[0397] For example, the above-described template-based blending method can be included in the mode of 1.2.3 TIMD and operate. That is, rather than inducing an intra mode like the current TIMD and always performing blending with a combination of non-directional modes and up to P (e.g., P is 2), the method can find an optimal blending candidate based on a template and generate a prediction block.

[0398] For example, the above-described template-based blending method can operate in a mode separate from the DIMD and TIMD modes. When the mode is called EIMD (Enhanced intra-mode derivation), EIMD can be signaled as a separate mode.

[0399] At this time, EIMD mode can be used as a mode to replace DIMD / TIMD. That is, EIMD mode and DIMD or / and TIMD mode cannot be enabled at the same time, and when DIMD or / and TIMD mode is enabled, syntax information related to EIMD mode may not be transmitted. Conversely, when EIMD mode is enabled, syntax information related to DIMD or / and TIMD mode may not be transmitted.

[0400] The intra mode candidates constituting the EIMD mode may be similar to the intra modes considered in DIMD or / and TIMD. Therefore, when the EIMD mode and the DIMD or / and TIMD modes operate simultaneously as separate modes, the intra mode candidates used in each technology can be configured so as not to overlap. For example, the intra mode candidates considered in DIMD or / and TIMD can be excluded from the intra mode candidates constituting the EIMD mode. The above example is when DIMD or / and TIMD are considered before EIMD in terms of sequence, and in the opposite case, the intra mode candidates constituting the EIMD mode can be excluded from the intra mode candidates constituting the DIMD or / and TIMD modes.

[0401] As another example, EIMD can be combined with the above modes. For example, if the size of the current block (width x height) is less than or equal to 256, EIMD may be signaled, and DIMD and / or TIMD may not be signaled. Conversely, if the size of the current block is greater than 256, EIMD may not be signaled. The above-mentioned cases are examples in which signaling is determined in relation to the block size, and may be determined and used with other values ​​and / or other conditions.

[0402] Figure 19 is a diagram showing an example of a template area for obtaining an error value of a blending candidate.

[0403] In the above embodiment, a template area (hereinafter, template area) is defined to obtain an error value of a blending candidate in the embodiment described in FIG. 18.

[0404] For example, the template area for obtaining the above error value can be defined as follows.

[0405] As illustrated in Figure 19, the adjacent upper A region of the current block can be used as a template area. The adjacent left L region of the current block can be used as a template area. The adjacent upper left AL region of the current block can be used as a template area. The upper right AR region of the current block can be used as a template area. The lower left LB region of the current block can be used as a template area. Furthermore, a partial or complete combination of the above regions can be used as a template area.

[0406] W, H: Width and height of the current block

[0407] : A is the width of the template area, which can be equal to or less than W.

[0408] : The width of the AR area template area, which can be equal to or less than W.

[0409] : AL, A, AR is the height of the template area, for example, it can be 4.

[0410] : AL, L, LB is the width of the template area, which can be 4, for example.

[0411] : L is the height of the template area, and can be equal to or less than H.

[0412] : LB area The height of the template area, which can be equal to or less than H.

[0413]

[0414] Template areas can be defined in the en / decoder dictionary without signaling, or a specific template area can be specified with signaling.

[0415] In one embodiment, the size of the template area may vary depending on the size (width x height) of the current block. A small block may use a small template area, and a large block may use a large template area. For example, if the size of the current block is 64 or less, the AL, A, and L areas may be used. Other blocks may use the A and L areas. Determining the template area according to the block size is defined so that a relatively small area of ​​the template area can be used for a large block, since the computational complexity required in the process of obtaining the error value is directly related to the size of the template area. In order to reduce the computational complexity required per pixel (pixel), the template area can be defined to be relatively small.

[0416] In one embodiment, template areas that contain unavailable samples beyond the boundaries of picture, slice, tile, CTU, and virtual boundaries may not be used. Alternatively, only some samples that are unavailable in the template area may be unused. In other words, even if unavailable samples are included in the template area, only available samples may be used.

[0417] In one embodiment, the sizes of the A, AR, L, LB, and AL regions are determined by considering the length of the current block (i.e., the length of the width or height). class The size of may be determined differently. For example, if the width is less than or equal to 8, can use the value of 2. Otherwise, The value of 4 can be used. Although width is explained as an example, the same can be applied depending on the length of height. The above-mentioned case is an example in which the size of the area is determined in relation to the length of the block, and can be determined and used with other values ​​and / or other conditions.

[0418] In one embodiment, the sizes of the A, AR, L, LB, and AL regions are determined by considering the shape of the block (i.e., the ratio of width and height). class The size of the block can be determined differently. For example, in the case of a 32x4 block, to minimize the computational complexity and memory increase required for calculating the error value, Use relatively small values The value can be a relatively large value. For example, is 2, The value of 4 can be used. The above-mentioned case is an example where the size of the area is determined in relation to the block shape, and can be determined and used with other values ​​and / or other conditions.

[0419] In one embodiment, the sizes of the A, AR, L, LB, and AL areas are determined based on the block size (the product of the block width and height). class The size of the block can be determined differently. For example, if the block size is 256 or more, is 4, can use the value of 4. Or, if the block size is 64 or less, is 2, The value of 2 can be used. The above-mentioned case is an example in which the size of the area is determined in relation to the block size, and can be determined and used with other values ​​and / or other conditions.

[0420] Additionally, it is obvious that the above embodiments can be combined. For example, if the block size is 16, only the AL, A, and L areas are used. is 2, The value of 2 can be used. Or, if the block size is 256, the A and L areas are used. is 4, can use the value of 4. Or, for 4x16 blocks, use the AL, A, L areas, considering the length of the block. is 4, The value of 2 can be used.

[0421] In addition, in the above embodiment and FIG. 19, the heights of AL, A, and AR are , and the width of AL, L, LB is Although only the same case is shown, the scope of the present invention is not limited thereto, and AL, A, and AR may have different heights, and AL, L, and LB may also have different widths.

[0422] Figure 20 is a flowchart illustrating an example of a decoding method according to one embodiment.

[0423] A decoding method according to one embodiment can be performed by the decoding device (300) described above. In the embodiment described below, descriptions are made focusing on contents not described above to avoid redundant descriptions, and the description described below does not solely support the embodiment of the decoding method. Descriptions regarding the operation of the decoding device (300) described above and descriptions regarding the decoding method (e.g., FIGS. 4, 9, 12, 13, etc.) and descriptions regarding various prediction modes or prediction types can be equally applied to the embodiment of FIG. 20 as long as they do not conflict with each other.

[0424] Referring to FIG. 20, a decoding method according to an embodiment includes a step of deriving intra prediction mode candidates; (S1800), a step of deriving at least one blending candidate based on the intra prediction mode candidates; (S1810), a step of deriving a prediction sample based on any one of the intra prediction mode candidates or the at least one blending candidate; (S1820), and a step of generating a reconstructed sample based on the prediction sample; (S1830), wherein the prediction sample is derived based on an error value between reconstructed samples of a surrounding template area of ​​a current block and prediction samples predicted based on any one of the intra prediction mode candidates or the at least one blending candidate.

[0425] Additionally, in the step of deriving intra prediction mode candidates (S1800), the method of configuring intra mode candidates can be defined by prior agreement between the encoder and decoder.

[0426] For example, by performing differential filtering on the surrounding reconstructed area of ​​the current block, a Histogram of gradients (HoG) can be constructed based on the gradient and amplitude (magnitude of change, e.g., the sum of the horizontal change magnitude and the vertical change magnitude) of the pixel values ​​obtained based on the ratio of horizontal and vertical changes, and intra mode candidates can be constructed in order of increasing amplitude (Method 1). This method can be the same as or similar to the above DIMD.

[0427] As another example, the error values ​​for the reconstructed samples around the current block can be calculated for intra prediction modes stored in adjacent / non-adjacent intra / inter blocks around the current block (e.g., the MPM, the SMPM (Secondary MPM) candidate), and / or intra modes predefined between the encoder and decoder (e.g., Planar, DC, horizontal direction, vertical direction mode), and intra prediction mode candidates can be organized in order of low error values ​​(Method 2). This method can be the same as or similar to the TIMD.

[0428] This is one example, and other intra-mode candidates can be configured by pre-defining between the encoder and decoder.

[0429] To obtain the error value, error calculation methods such as SAD (Sum of Absolute difference), SATD (Sum of transformed Absolute difference), SSE (Sum of squared error), MR-SAD (Mean-removed sum of Absolute difference), MR-SSE (Mean-removed sum of squared error), and MR-SATD (Mean-removed sum of Absolute transformed difference) can be used.

[0430] As another example, intra mode candidates can be constructed for predefined default modes. For example, the default mode can be defined as {DC mode, Vertical mode, Horizontal mode, Vertical mode - 4, Vertical mode + 4, Horizontal mode - 4, Horizontal mode + 4, Vertical mode - 8, Vertical mode + 8, Horizontal mode - 8, Horizontal mode + 8, Vertical mode - 12, Vertical mode + 12, Horizontal mode - 12, Horizontal mode + 12, 34, 66, 2, Vertical mode - 2, Vertical mode + 2, Horizontal mode -2, Horizontal mode +2 …} (Method 3). This is an example, and the default mode can be defined in a predefined intra mode order between the encoder and decoder.

[0431] Using the above methods, up to M intra mode candidates can be constructed. M can be, for example, 6. M is a positive integer, and can be determined to a different value by defining it between the encoder and decoder.

[0432] Alternatively, an intra-mode candidate can be constructed using a combination of the above methods. For example, if the size of the current block (width x height) is less than or equal to 256, an intra-mode candidate can be constructed using method 1, otherwise, an intra-mode candidate can be constructed using method 2. Conversely, if the size of the current block is less than or equal to 256, an intra-mode candidate can be constructed using method 2, otherwise, an intra-mode candidate can be constructed using method 1.

[0433] Alternatively, we can construct P candidates in Method 1, Q candidates in Method 2, and a total of P+Q(=M) intra-mode candidates. For example, P can be 4 and Q can be 2. Alternatively, if Method 1 fails to construct M intra-mode candidates, Method 2 can be used to construct intra-mode candidates until M are filled.

[0434] The above-mentioned cases are examples of constructing intra-mode candidates by combining different methods, and can be determined and used with different values ​​and / or conditions. While the above-mentioned cases exemplify a combination of Methods 1 and 2, Method 3 can also be applied and combined. In other words, combinations of Methods 1 and 2, Methods 1 and 3, and Methods 2 and 3 are all possible.

[0435] Meanwhile, the plurality of intra prediction mode candidates may include candidates of the Most Probable Mode (MPM) list for the current block.

[0436] The same explanation of how to construct an MPM list as described above applies to the candidates on the MPM list. To avoid duplication, the same explanation may be omitted here.

[0437] The MPM list may include a Primary MPM list (PMPM list), or may include both a PMPM list and a SMPM candidate list (Secondary MPM list).

[0438] For example, the size of the PMPM list may be 6, including the planar mode, and the size of the SMPM list may be 16. Accordingly, the size of the MPM list including both the PMPM list and the SMPM list may be 22. The size of each list may be a value defined in advance between the encoding device (200) and the decoding device (300).

[0439] In addition, the step (S1810) of deriving at least one blending candidate based on the intra prediction mode candidates is a step of generating prediction samples of blending candidates for a template area defined around the current block, calculating an error value with respect to a restored sample, and determining a blending candidate with a small error value.

[0440] For example, in the step (S1810) of deriving at least one blending candidate based on the intra prediction mode candidates, when M is 6, the number of blending candidates for generating the final prediction sample may be 1 to 6. In addition, even if the number of blending candidates is the same, the combinations may be diverse, such as (candidate 1, candidate 2), (candidate 1, candidate 3), …, (candidate 5, candidate 6). In this way, an error value can be obtained for various candidates, and the blending candidate can be determined based on the error value.

[0441] For example, the blending candidates may be as follows: The M intra-mode candidates may be defined as M1 to M6.

[0442] - M1 (single mode, not blending)

[0443] - M2 (single mode, not blending)

[0444] - M3 (single mode, not blending)

[0445] - M4 (single mode, not blending)

[0446] - M5 (single mode, not blending)

[0447] - M6 (single mode, not blending)

[0448] - M1, M2

[0449] - M1, M2, M3

[0450] - M1, M2, M3, M4

[0451] - M1, M2, M3, M4, M5

[0452] - M1, M2, M3, M4, M5, M6

[0453] - M1, Planar mode

[0454] - M2, Planar mode

[0455] - M3, Planar mode

[0456] - M4, Planar mode

[0457] - M5, Planar mode

[0458] - M6, Planar mode

[0459] - M1, M2, Planar mode

[0460] - M1, M2, M3, Planar mode

[0461] - M1, M2, M3, M4, Planar mode

[0462] - M1, M2, M3, M4, M5, Planar mode

[0463] This is one example of a combination of M1 to M6 and a combination with the omnidirectional mode, and other combinations or partial combinations of the above examples can be determined by agreement between the encoder and decoder.

[0464] That is, the at least one blending candidate can be derived by combining one or more of the plurality of intra prediction mode candidates or by combining at least one of the plurality of intra prediction mode candidates with a Planar mode.

[0465] Additionally, a prediction sample can be derived based on any one of the intra prediction mode candidates or at least one blending candidate (S1820).

[0466] Specifically, the prediction sample may be derived based on an error value between the predicted samples of the surrounding template area of ​​the current block and the predicted samples based on one of the intra prediction mode candidates or the at least one blending candidate.

[0467] The above error value may be calculated based on a weighted sum calculated according to weights applied to prediction samples of the surrounding template area predicted based on at least one blending candidate, and a difference between the restored samples of the template area. If there are two or more blending candidates, the sum of the weights applied to each blending candidate may be 1.

[0468] At this time, the formula for calculating the error value (hereinafter, Err) for one blending candidate is as follows.

[0469] [Formula 7]

[0470]

[0471] ReconSamples: Reconstruction samples of the current block and the adjacent template area.

[0472] K: K is a positive integer and is the number of blending candidates.

[0473] : A predicted block generated with the i-th intra mode of the intra mode candidate in the template area adjacent to the current block.

[0474] : Weight ratio to be applied to the i-th prediction block ( )

[0475] w is a real number

[0476] In the formula for calculating the above error value, if K is 1, that is, if the intra prediction mode candidate is a single mode rather than blending, =1 can be applied to calculate the error value.

[0477] In one embodiment, when the number of blending candidates is two, the error value may be a value obtained by multiplying a prediction block generated in the first intra mode of the intra mode candidate in the template area adjacent to the current block by a weight ratio to be applied to the first prediction block, a value obtained by adding a value obtained by multiplying a prediction block generated in the second intra mode of the intra mode candidate in the template area adjacent to the current block by a weight ratio to be applied to the second prediction block, and an absolute difference between the restoration samples in the template area adjacent to the current block.

[0478] At this time, the sum of the weight ratio to be applied to the first prediction block and the weight ratio to be applied to the second prediction block may be 1.

[0479] Although the error value is calculated using SAD in the above formula, this is just one example, and to calculate the error value, error calculation methods such as SAD (Sum of Absolute Difference), SATD (Sum of Absolute Transformed Difference), SSE (Sum of Squared Error), MR-SAD (Mean-Removed Sum of Absolute Difference), MR-SSE (Mean-Removed Sum of Squared Error), and MR-SATD (Mean-Removed Sum of Absolute Transformed Difference) can be used.

[0480] The step of calculating an error value between the predicted samples based on the restoration samples of the surrounding template area of ​​the current block and one of the intra prediction mode candidates or the at least one blending candidate may be omitted when it is difficult to obtain sufficient intra mode candidates in the current block.

[0481] For example, in the intra mode candidate configuration step, if at least N candidates are not filled in, the step can be omitted. For example, N can be 2. N is a positive integer and can be a value defined in advance between the encoder and decoder. Alternatively, the step can be omitted if the y position of the current block is 0, the x position is 0, or the position of the current block is located at the upper left of the current picture.

[0482] Meanwhile, in order to obtain an error value for the above blending candidate, a weight for each intra mode can be defined.

[0483] For example, weights can be defined as follows:

[0484] If the method of obtaining HoG by performing differential filtering in the above intra mode candidate configuration step is used, the weight value can be determined based on the amplitude of the HoG. That is, the weight can be determined to be proportionally larger when the amplitude of the corresponding intra mode is large. For example, for the (M1, M2) blending candidates, if each amplitude is (a1, a2), the weight of M1 can be determined to have a ratio of a1 / (a1+a2), and the weight of M2 can be determined to have a ratio of a2 / (a1+a2). This can determine the weights in a similar / same way as DIMD.

[0485] Alternatively, if a method similar to TIMD is used in the intra mode candidate configuration step, the weight value can be determined based on the error value of the intra mode.

[0486] That is, the weights can be determined to be inversely larger when the error value of the corresponding intra mode is small. For example, for the blending candidates (M1, M2), if the respective error values ​​are (c1, c2), the weight of M1 can be determined to have a ratio of c2 / (c1+c2), and the weight of M2 can be determined to have a ratio of c1 / (c1+c2). This can determine the weights in a similar / same way as TIMD.

[0487] Alternatively, the weight values ​​can be predefined as follows:

[0488] For two blending candidates, the weight ratio is:

[0489] (1 / 4, 3 / 4),

[0490] (2 / 4, 2 / 4),

[0491] (3 / 4, 1 / 4),

[0492] For the three blending candidates, the weight ratio is:

[0493] (2 / 4, 1 / 4, 1 / 4),

[0494] (1 / 4, 2 / 4, 1 / 4),

[0495] (1 / 4, 1 / 4, 2 / 4),

[0496] For the four blending candidates, the weight ratios are:

[0497] (4 / 16, 4 / 16, 4 / 16, 4 / 16),

[0498] (8 / 16, 4 / 16, 2 / 16, 2 / 16),

[0499] It can be like (12 / 16, 2 / 16, 1 / 16, 1 / 16).

[0500] This is an example of defining weights for two to four blending candidates, and weights can be defined for five or more blending candidates in the same way. Other weight combinations and / or subcombinations of the above examples can be determined by agreement between the encoder and decoder.

[0501] That is, the weight applied to the prediction samples generated based on the at least one blending candidate may be determined based on at least one of a method similar to or the same as TIMD (Template-based Intra Mode Derivation), a method similar to or the same as DIMD (Decoder-side Intra Mode Derivation), and a predetermined weight.

[0502] In the step (S1820) of deriving a prediction sample based on any one of the intra prediction mode candidates or at least one blending candidate, the prediction sample can be derived using a blending candidate with a small error value.

[0503] In addition, the above-described template-based blending method can be included in the mode of DIMD and operate. That is, rather than inducing an intra mode like the current DIMD and always blending with a combination of the planar mode and up to P (e.g., P is 5), the method can find an optimal blending candidate based on a template and generate a prediction sample.

[0504] For example, the above-described template-based blending method can be incorporated into the TIMD mode and operate. That is, rather than inducing an intra mode like the current TIMD and always blending with a combination of non-directional modes and up to P (e.g., P is 2), the method can find an optimal blending candidate based on a template and generate a prediction sample.

[0505] For example, the above-described template-based blending method can operate in a mode separate from the DIMD and TIMD modes. When the mode is called EIMD (Enhanced intra-mode derivation), EIMD can be signaled as a separate mode.

[0506] At this time, EIMD mode can be used as a mode to replace DIMD / TIMD. That is, EIMD mode and DIMD or / and TIMD mode cannot be enabled at the same time, and when DIMD or / and TIMD mode is enabled, syntax information related to EIMD mode may not be transmitted. Conversely, when EIMD mode is enabled, syntax information related to DIMD or / and TIMD mode may not be transmitted.

[0507] The intra-mode candidates that constitute the EIMD mode may be similar to the intra-modes considered in DIMD or / and TIMD. Therefore, when the EIMD mode and the DIMD or / and TIMD modes operate simultaneously as separate modes, the intra-mode candidates used in each technology can be configured so as not to overlap.

[0508] For example, an intra-mode candidate considered in DIMD or / and TIMD may be excluded from the intra-mode candidates constituting the EIMD mode. The above example is when DIMD or / and TIMD are considered before EIMD in terms of order, and in the opposite case, an intra-mode candidate constituting the EIMD mode may be excluded from the intra-mode candidates constituting the DIMD or / and TIMD mode.

[0509] As another example, EIMD can be combined with the above modes. For example, if the size of the current block (width x height) is less than or equal to 256, EIMD may be signaled, and DIMD and / or TIMD may not be signaled. Conversely, if the size of the current block is greater than 256, EIMD may not be signaled. The above-mentioned cases are examples in which signaling is determined in relation to the block size, and may be determined and used with other values ​​and / or other conditions.

[0510] Additionally, the present embodiment may include a step (S1830) of generating a restoration sample based on the derived prediction sample.

[0511] As described above, the method of the present embodiment defines separate syntax information in high-level parameters such as VPS, SPS, APS, PPS, Picture Header, or Slice Header, and whether to use the information can be determined when the information has a specific value. For example, if the value is 1, it can be determined to use the template-based blending method.

[0512] Figure 21 is a flowchart illustrating an example of an encoding method according to one embodiment.

[0513] An encoding method according to one embodiment can be performed by the encoding device (200) described above. In the embodiment described below, descriptions are made focusing on contents not described above to avoid redundant descriptions, and the description described below does not solely support the embodiment of the encoding method. The descriptions regarding the operation of the encoding device (200) described above, the descriptions regarding the encoding method (e.g., the descriptions regarding FIGS. 5, 7, 11, etc.), and the descriptions regarding various prediction modes or prediction types can be equally applied to the example of FIG. 21 as long as they do not conflict with each other.

[0514] Referring to FIG. 21, a decoding method according to an embodiment includes a step of deriving intra prediction mode candidates; (S2100), a step of deriving at least one blending candidate based on the intra prediction mode candidates; (S2110), a step of deriving a prediction sample based on any one of the intra prediction mode candidates or the at least one blending candidate; (S2120), and a step of deriving a residual sample based on the prediction sample (S2130).

[0515] The description in the embodiment of the decoding method described above can be equally applied to the embodiment of the encoding method.

[0516] Specifically, in the step of deriving intra prediction mode candidates (S2100), the method of configuring intra mode candidates can be defined by agreement between the encoder and the decoder in advance.

[0517] For example, by performing differential filtering on the surrounding reconstructed area of ​​the current block, a Histogram of gradients (HoG) can be constructed based on the gradient and amplitude (magnitude of change, e.g., the sum of the horizontal change magnitude and the vertical change magnitude) of the pixel values ​​obtained based on the ratio of horizontal and vertical changes, and intra mode candidates can be constructed in order of increasing amplitude (Method 1). This method can be the same as or similar to the above DIMD.

[0518] As another example, the error values ​​for the reconstructed samples around the current block can be calculated for intra prediction modes stored in adjacent / non-adjacent intra / inter blocks around the current block (e.g., the MPM, the SMPM (Secondary MPM) candidate), and / or intra modes predefined between the encoder and decoder (e.g., Planar, DC, horizontal direction, vertical direction mode), and intra prediction mode candidates can be organized in order of low error values ​​(Method 2). This method can be the same as or similar to the TIMD.

[0519] This is one example, and other intra-mode candidates can be configured by pre-defining between the encoder and decoder.

[0520] To obtain the error value, error calculation methods such as SAD (Sum of Absolute difference), SATD (Sum of transformed Absolute difference), SSE (Sum of squared error), MR-SAD (Mean-removed sum of Absolute difference), MR-SSE (Mean-removed sum of squared error), and MR-SATD (Mean-removed sum of Absolute transformed difference) can be used.

[0521] As another example, intra mode candidates can be constructed for predefined default modes. For example, the default mode can be defined as {DC mode, Vertical mode, Horizontal mode, Vertical mode - 4, Vertical mode + 4, Horizontal mode - 4, Horizontal mode + 4, Vertical mode - 8, Vertical mode + 8, Horizontal mode - 8, Horizontal mode + 8, Vertical mode - 12, Vertical mode + 12, Horizontal mode - 12, Horizontal mode + 12, 34, 66, 2, Vertical mode - 2, Vertical mode + 2, Horizontal mode -2, Horizontal mode +2 …} (Method 3). This is an example, and the default mode can be defined in a predefined intra mode order between the encoder and decoder.

[0522] Using the above methods, up to M intra mode candidates can be constructed. M can be, for example, 6. M is a positive integer, and can be determined to a different value by defining it between the encoder and decoder.

[0523] Alternatively, an intra-mode candidate can be constructed using a combination of the above methods. For example, if the size of the current block (width x height) is less than or equal to 256, an intra-mode candidate can be constructed using method 1, otherwise, an intra-mode candidate can be constructed using method 2. Conversely, if the size of the current block is less than or equal to 256, an intra-mode candidate can be constructed using method 2, otherwise, an intra-mode candidate can be constructed using method 1.

[0524] Alternatively, we can construct P candidates in Method 1, Q candidates in Method 2, and a total of P+Q(=M) intra-mode candidates. For example, P can be 4 and Q can be 2. Alternatively, if Method 1 fails to construct M intra-mode candidates, Method 2 can be used to construct intra-mode candidates until M are filled.

[0525] The above-mentioned cases are examples of constructing intra-mode candidates by combining different methods, and can be determined and used with different values ​​and / or conditions. While the above-mentioned cases exemplify a combination of Methods 1 and 2, Method 3 can also be applied and combined. In other words, combinations of Methods 1 and 2, Methods 1 and 3, and Methods 2 and 3 are all possible.

[0526] Meanwhile, the plurality of intra prediction mode candidates may include candidates of the Most Probable Mode (MPM) list for the current block.

[0527] The same explanation of how to construct an MPM list as described above applies to the candidates on the MPM list. To avoid duplication, the same explanation may be omitted here.

[0528] The MPM list may include a Primary MPM list (PMPM list), or may include both a PMPM list and a SMPM candidate list (Secondary MPM list).

[0529] For example, the size of the PMPM list may be 6, including the planar mode, and the size of the SMPM list may be 16. Accordingly, the size of the MPM list including both the PMPM list and the SMPM list may be 22. The size of each list may be a value defined in advance between the encoding device (200) and the decoding device (300).

[0530] In addition, the step (S2110) of deriving at least one blending candidate based on the intra prediction mode candidates is a step of generating prediction samples of blending candidates for a template area defined around the current block, calculating an error value with respect to a restored sample, and determining a blending candidate with a small error value.

[0531] For example, in the step (S2110) of deriving at least one blending candidate based on the intra prediction mode candidates, when M is 6, the number of blending candidates for generating the final prediction sample may be 1 to 6. In addition, even if the number of blending candidates is the same, the combinations may be diverse, such as (candidate 1, candidate 2), (candidate 1, candidate 3), …, (candidate 5, candidate 6). In this way, an error value can be obtained for various candidates, and the blending candidate can be determined based on the error value.

[0532] For example, the blending candidates may be as follows: The M intra-mode candidates may be defined as M1 to M6.

[0533] - M1 (single mode, not blending)

[0534] - M2 (single mode, not blending)

[0535] - M3 (single mode, not blending)

[0536] - M4 (single mode, not blending)

[0537] - M5 (single mode, not blending)

[0538] - M6 (single mode, not blending)

[0539] - M1, M2

[0540] - M1, M2, M3

[0541] - M1, M2, M3, M4

[0542] - M1, M2, M3, M4, M5

[0543] - M1, M2, M3, M4, M5, M6

[0544] - M1, Planar mode

[0545] - M2, Planar mode

[0546] - M3, Planar mode

[0547] - M4, Planar mode

[0548] - M5, Planar mode

[0549] - M6, Planar mode

[0550] - M1, M2, Planar mode

[0551] - M1, M2, M3, Planar mode

[0552] - M1, M2, M3, M4, Planar mode

[0553] - M1, M2, M3, M4, M5, Planar mode

[0554] This is one example of a combination of M1 to M6 and a combination with the omnidirectional mode, and other combinations or partial combinations of the above examples can be determined by agreement between the encoder and decoder.

[0555] That is, the at least one blending candidate can be derived by combining one or more of the plurality of intra prediction mode candidates or by combining at least one of the plurality of intra prediction mode candidates with a Planar mode.

[0556] Additionally, a prediction sample can be derived based on any one of the intra prediction mode candidates or at least one blending candidate (S2120).

[0557] Specifically, the prediction sample may be derived based on an error value between the predicted samples of the surrounding template area of ​​the current block and the predicted samples based on one of the intra prediction mode candidates or the at least one blending candidate.

[0558] The above error value may be calculated based on a weighted sum calculated according to weights applied to prediction samples of the surrounding template area predicted based on at least one blending candidate, and a difference between the restored samples of the template area. If there are two or more blending candidates, the sum of the weights applied to each blending candidate may be 1.

[0559] At this time, the formula for calculating the error value (hereinafter, Err) for one blending candidate is as follows.

[0560] [Formula 8]

[0561]

[0562] ReconSamples: Reconstruction samples of the current block and the adjacent template area.

[0563] K: K is a positive integer and is the number of blending candidates.

[0564] : A predicted block generated with the i-th intra mode of the intra mode candidate in the template area adjacent to the current block.

[0565] : Weight ratio to be applied to the i-th prediction block ( )

[0566] w is a real number

[0567] In the formula for calculating the above error value, if K is 1, that is, if the intra prediction mode candidate is a single mode rather than blending, =1 can be applied to calculate the error value.

[0568] In one embodiment, when the number of blending candidates is two, the error value may be a value obtained by multiplying a prediction block generated in the first intra mode of the intra mode candidate in the template area adjacent to the current block by a weight ratio to be applied to the first prediction block, a value obtained by adding a value obtained by multiplying a prediction block generated in the second intra mode of the intra mode candidate in the template area adjacent to the current block by a weight ratio to be applied to the second prediction block, and an absolute difference between the restoration samples in the template area adjacent to the current block.

[0569] At this time, the sum of the weight ratio to be applied to the first prediction block and the weight ratio to be applied to the second prediction block may be 1.

[0570] Although the error value is calculated using SAD in the above formula, this is just one example, and to calculate the error value, error calculation methods such as SAD (Sum of Absolute Difference), SATD (Sum of Absolute Transformed Difference), SSE (Sum of Squared Error), MR-SAD (Mean-Removed Sum of Absolute Difference), MR-SSE (Mean-Removed Sum of Squared Error), and MR-SATD (Mean-Removed Sum of Absolute Transformed Difference) can be used.

[0571] The step of calculating an error value between the predicted samples based on the restoration samples of the surrounding template area of ​​the current block and one of the intra prediction mode candidates or the at least one blending candidate may be omitted when it is difficult to obtain sufficient intra mode candidates in the current block.

[0572] For example, in the intra mode candidate configuration step, if at least N candidates are not filled in, the step can be omitted. For example, N can be 2. N is a positive integer and can be a value defined in advance between the encoder and decoder. Alternatively, the step can be omitted if the y position of the current block is 0, the x position is 0, or the position of the current block is located at the upper left of the current picture.

[0573] Meanwhile, in order to obtain an error value for the above blending candidate, a weight for each intra mode can be defined.

[0574] For example, weights can be defined as follows:

[0575] If the method of obtaining HoG by performing differential filtering in the above intra mode candidate configuration step is used, the weight value can be determined based on the amplitude of the HoG. That is, the weight can be determined to be proportionally larger when the amplitude of the corresponding intra mode is large. For example, for the (M1, M2) blending candidates, if each amplitude is (a1, a2), the weight of M1 can be determined to have a ratio of a1 / (a1+a2), and the weight of M2 can be determined to have a ratio of a2 / (a1+a2). This can determine the weights in a similar / same way as DIMD.

[0576] Alternatively, if a method similar to TIMD is used in the intra mode candidate configuration step, the weight value can be determined based on the error value of the intra mode.

[0577] That is, the weights can be determined to be inversely larger when the error value of the corresponding intra mode is small. For example, for the blending candidates (M1, M2), if the respective error values ​​are (c1, c2), the weight of M1 can be determined to have a ratio of c2 / (c1+c2), and the weight of M2 can be determined to have a ratio of c1 / (c1+c2). This can determine the weights in a similar / same way as TIMD.

[0578] Alternatively, the weight values ​​can be predefined as follows:

[0579] For two blending candidates, the weight ratio is:

[0580] (1 / 4, 3 / 4),

[0581] (2 / 4, 2 / 4),

[0582] (3 / 4, 1 / 4),

[0583] For the three blending candidates, the weight ratio is:

[0584] (2 / 4, 1 / 4, 1 / 4),

[0585] (1 / 4, 2 / 4, 1 / 4),

[0586] (1 / 4, 1 / 4, 2 / 4),

[0587] For the four blending candidates, the weight ratios are:

[0588] (4 / 16, 4 / 16, 4 / 16, 4 / 16),

[0589] (8 / 16, 4 / 16, 2 / 16, 2 / 16),

[0590] It can be like (12 / 16, 2 / 16, 1 / 16, 1 / 16).

[0591] This is an example of defining weights for two to four blending candidates, and weights can be defined for five or more blending candidates in the same way. Other weight combinations and / or subcombinations of the above examples can be determined by agreement between the encoder and decoder.

[0592] That is, the weight applied to the prediction samples generated based on the at least one blending candidate may be determined based on at least one of a method similar to or the same as TIMD (Template-based Intra Mode Derivation), a method similar to or the same as DIMD (Decoder-side Intra Mode Derivation), and a predetermined weight.

[0593] In the step (S2120) of deriving a prediction sample based on any one of the intra prediction mode candidates or at least one blending candidate, the prediction sample can be derived using a blending candidate with a small error value.

[0594] In addition, the above-described template-based blending method can be included in the mode of DIMD and operate. That is, rather than inducing an intra mode like the current DIMD and always blending with a combination of the planar mode and up to P (e.g., P is 5), the method can find an optimal blending candidate based on a template and generate a prediction sample.

[0595] For example, the above-described template-based blending method can be incorporated into the TIMD mode and operate. That is, rather than inducing an intra mode like the current TIMD and always blending with a combination of non-directional modes and up to P (e.g., P is 2), the method can find an optimal blending candidate based on a template and generate a prediction sample.

[0596] For example, the above-described template-based blending method can operate in a mode separate from the DIMD and TIMD modes. When the mode is called EIMD (Enhanced intra-mode derivation), EIMD can be signaled as a separate mode.

[0597] At this time, EIMD mode can be used as a mode to replace DIMD / TIMD. That is, EIMD mode and DIMD or / and TIMD mode cannot be enabled at the same time, and when DIMD or / and TIMD mode is enabled, syntax information related to EIMD mode may not be transmitted. Conversely, when EIMD mode is enabled, syntax information related to DIMD or / and TIMD mode may not be transmitted.

[0598] The intra-mode candidates that constitute the EIMD mode may be similar to the intra-modes considered in DIMD or / and TIMD. Therefore, when the EIMD mode and the DIMD or / and TIMD modes operate simultaneously as separate modes, the intra-mode candidates used in each technology can be configured so as not to overlap.

[0599] For example, an intra-mode candidate considered in DIMD or / and TIMD may be excluded from the intra-mode candidates constituting the EIMD mode. The above example is when DIMD or / and TIMD are considered before EIMD in terms of order, and in the opposite case, an intra-mode candidate constituting the EIMD mode may be excluded from the intra-mode candidates constituting the DIMD or / and TIMD mode.

[0600] As another example, EIMD can be combined with the above modes. For example, if the size of the current block (width x height) is less than or equal to 256, EIMD may be signaled, and DIMD and / or TIMD may not be signaled. Conversely, if the size of the current block is greater than 256, EIMD may not be signaled. The above-mentioned cases are examples in which signaling is determined in relation to the block size, and may be determined and used with other values ​​and / or other conditions.

[0601] Additionally, the present embodiment may include a step (S2130) of deriving a residual sample based on the predicted sample.

[0602] As described above, the method of the present embodiment defines separate syntax information in high-level parameters such as VPS, SPS, APS, PPS, Picture Header, or Slice Header, and whether to use the information can be determined when the information has a specific value. For example, if the value is 1, it can be determined to use the template-based blending method.

[0603] Up to now, the embodiments have been described separately for convenience of explanation, but as mentioned above, a combination of two or more embodiments is possible, and changes required by the combination of embodiments may also be included in the scope of the disclosed invention or disclosed embodiments.

[0604] Meanwhile, a computer-readable storage medium storing a bitstream generated by the encoding method described above may also be included in the scope of the disclosed embodiment.

[0605] Additionally, a method for transmitting a bitstream generated by the encoding method described above may also be included in the scope of the disclosed embodiment, and a transmission device for implementing such a transmission method may include at least one processor for generating a bitstream by the encoding method and a transmitter for transmitting the generated bitstream.

[0606] The embodiments described herein may be implemented and performed on a processor, microprocessor, controller, or chip. For example, the functional units depicted in each drawing may be implemented and performed on a computer, processor, microprocessor, controller, or chip. In this case, information for implementation (e.g., information on instructions) or algorithms may be stored on a digital storage medium.

[0607] In addition, the decoding device and encoding device to which the present invention is applied may be included in a multimedia broadcasting transmitting and receiving device, a mobile communication terminal, a home cinema video device, a digital cinema video device, a surveillance camera, a video conversation device, a real-time communication device such as a video communication, a mobile streaming device, a storage medium, a camcorder, a video-on-demand (VoD) service providing device, an OTT (Over the top video) device, an Internet streaming service providing device, a three-dimensional (3D) video device, a VR (virtual reality) device, an AR (argumente reality) device, a video phone video device, a transportation terminal (ex. a vehicle (including an autonomous vehicle) terminal, an airplane terminal, a ship terminal, etc.), and a medical video device, and may be used to process a video signal or a data signal. For example, the OTT (Over the top video) device may include a game console, a Blu-ray player, an Internet-connected TV, a home theater system, a smartphone, a tablet PC, a DVR (Digital Video Recorder), etc.

[0608] In addition, the processing method to which the present invention is applied can be produced in the form of a program executed by a computer and can be stored in a computer-readable recording medium. Multimedia data having a data structure according to the present invention can also be stored in a computer-readable recording medium. The computer-readable recording medium includes all types of storage devices and distributed storage devices in which computer-readable data is stored. The computer-readable recording medium may include, for example, a Blu-ray disc (BD), a universal serial bus (USB), a ROM, a PROM, an EPROM, an EEPROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device. In addition, the computer-readable recording medium includes media implemented in the form of a carrier wave (e.g., transmission via the Internet). In addition, a bitstream generated by an encoding method can be stored in a computer-readable recording medium or transmitted via a wired or wireless communication network.

[0609] Additionally, embodiments of the present invention may be implemented as a computer program product using program code, and the program code may be executed on a computer according to embodiments of the present invention. The program code may be stored on a computer-readable carrier.

[0610] FIG. 22 is a diagram illustrating an example of a content streaming system to which the disclosed embodiment can be applied.

[0611] Referring to FIG. 22, a content streaming system to which the embodiment(s) of the present specification are applied may largely include an encoding server, a streaming server, a web server, a media storage, a user device, and a multimedia input device.

[0612] The encoding server compresses content input from multimedia input devices such as smartphones, cameras, and camcorders into digital data, generates a bitstream, and transmits it to the streaming server. Alternatively, if multimedia input devices such as smartphones, cameras, and camcorders directly generate bitstreams, the encoding server may be omitted.

[0613] The above bitstream can be generated by an encoding method or a bitstream generation method to which the embodiment(s) of the present specification are applied, and the streaming server can temporarily store the bitstream during the process of transmitting or receiving the bitstream.

[0614] The streaming server transmits multimedia data to a user device based on a user request via a web server, and the web server acts as an intermediary to inform the user of available services. When a user requests a desired service from the web server, the web server transmits the request to the streaming server, and the streaming server transmits the multimedia data to the user. At this time, the content streaming system may include a separate control server, in which case the control server controls commands / responses between each device within the content streaming system.

[0615] The streaming server can receive content from a media repository and / or an encoding server. For example, when receiving content from the encoding server, the content can be received in real time. In this case, to provide a smooth streaming service, the streaming server can store the bitstream for a certain period of time.

[0616] Examples of the user devices may include mobile phones, smart phones, laptop computers, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation devices, slate PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, HMDs), digital TVs, desktop computers, digital signage, etc.

[0617] Each server within the above content streaming system can be operated as a distributed server, in which case data received from each server can be processed in a distributed manner.

[0618] The claims set forth in this specification may be combined in various ways. For example, the technical features of the method claims of this specification may be combined and implemented as a device, and the technical features of the device claims of this specification may be combined and implemented as a method. Furthermore, the technical features of the method claims and the technical features of the device claims of this specification may be combined and implemented as a device, and the technical features of the method claims and the technical features of the device claims of this specification may be combined and implemented as a method.

[0619]

[0620] Embodiments according to the present disclosure can be used to encode / decode images.

Claims

1. In a method for decoding video information, A step of deriving intra prediction mode candidates; A step of deriving at least one blending candidate based on the intra prediction mode candidates; A step of deriving a prediction sample based on any one of the intra prediction mode candidates or at least one blending candidate; and comprising a step of generating a restoration sample based on the above prediction sample, A method wherein the above prediction sample is derived based on an error value between prediction samples predicted based on one of the intra prediction mode candidates or at least one blending candidate and restoration samples of the surrounding template area of ​​the current block.

2. In paragraph 1, A method wherein the intra prediction mode candidates include candidates of a Most Probable Mode (MPM) list for the current block.

3. In paragraph 1, The above intra prediction mode candidates are: A method derived based on at least one of the TIMD (Template-based Intra Mode Derivation) method, the DIMD (Decoder-side Intra Mode Derivation) method, and a plurality of predetermined intra prediction modes.

4. In paragraph 1, At least one of the blending candidates is, A method derived by combining one or more of the intra prediction mode candidates or by combining at least one of the intra prediction mode candidates with a Planar mode.

5. In paragraph 1, The above error value is, A method, wherein the weighted sum is calculated based on the weights applied to the predicted samples of the surrounding template region predicted based on any one of the intra prediction mode candidates or at least one blending candidate, and the difference between the restored samples of the template region.

6. In paragraph 5, A method comprising, when there are two or more blending candidates, ensuring that the sum of the weights applied to the prediction samples is 1.

7. In paragraph 5, A method, wherein the weights applied to the prediction samples generated based on the at least one blending candidate are determined based on at least one of a Template-based Intra Mode Derivation (TIMD) method, a Decoder-side Intra Mode Derivation (DIMD) method, and a predetermined weight.

8. In paragraph 1, The surrounding template area of ​​the current block above is A method comprising: defining a template region based on a portion of a surrounding area of ​​a current block, wherein the size of the template region is determined based on the size of the current block.

9. In a method for encoding image information, A step of deriving intra prediction mode candidates; A step of deriving at least one blending candidate based on the intra prediction mode candidates; A step of deriving a prediction sample based on any one of the intra prediction mode candidates or at least one blending candidate; and A step of deriving a residual sample based on the above predicted sample is included, A method wherein the above prediction sample is derived based on an error value between prediction samples predicted based on one of the intra prediction mode candidates or at least one blending candidate and restoration samples of the surrounding template area of ​​the current block.

10. In paragraph 9, A method wherein the intra prediction mode candidates include candidates of a Most Probable Mode (MPM) list for the current block.

11. In paragraph 9, The above intra prediction mode candidates are: A method derived based on at least one of the TIMD (Template-based Intra Mode Derivation) method, the DIMD (Decoder-side Intra Mode Derivation) method, and a plurality of predetermined intra prediction modes.

12. In paragraph 9, At least one of the blending candidates is, A method derived by combining one or more of the intra prediction mode candidates or by combining at least one of the intra prediction mode candidates with a Planar mode.

13. In paragraph 9, The above error value is, A method, wherein the weighted sum is calculated based on the weights applied to the predicted samples of the surrounding template region predicted based on any one of the intra prediction mode candidates or at least one blending candidate, and the difference between the restored samples of the template region.

14. In paragraph 13, A method comprising, when there are two or more blending candidates, ensuring that the sum of the weights applied to the prediction samples is 1.

15. In paragraph 13, A method, wherein the weights applied to the prediction samples generated based on the at least one blending candidate are determined based on at least one of a Template-based Intra Mode Derivation (TIMD) method, a Decoder-side Intra Mode Derivation (DIMD) method, and a predetermined weight.

16. In paragraph 9, The surrounding template area of ​​the current block above is A method comprising: defining a template region based on a portion of a surrounding area of ​​a current block, wherein the size of the template region is determined based on the size of the current block.

17. In a computer-readable storage medium storing a bitstream generated by an encoding method, The above encoding method is, A step of deriving intra prediction mode candidates; A step of deriving at least one blending candidate based on the intra prediction mode candidates; A step of deriving a prediction sample based on any one of the intra prediction mode candidates or at least one blending candidate; A step of deriving a residual sample based on the above predicted sample; and A step of storing a bitstream based on the residual sample; A computer-readable storage medium, wherein the above prediction sample is derived based on an error value between prediction samples predicted based on one of the intra prediction mode candidates or the at least one blending candidate and restoration samples of the surrounding template area of ​​the current block.

18. In paragraph 17, A method wherein the intra prediction mode candidates include candidates of a Most Probable Mode (MPM) list for the current block.

19. In the method of transmitting data for video, A step of deriving intra prediction mode candidates; A step of deriving at least one blending candidate based on the intra prediction mode candidates; A step of deriving a prediction sample based on any one of the intra prediction mode candidates or at least one blending candidate; A step of deriving a residual sample based on the above prediction sample; A step of generating a bitstream based on the residual sample; and A step of transmitting the data including the bitstream; A transmission method, wherein the above prediction sample is derived based on an error value between prediction samples predicted based on one of the intra prediction mode candidates or the at least one blending candidate and restoration samples of the surrounding template area of ​​the current block.

20. In paragraph 19, A method wherein the intra prediction mode candidates include candidates of a Most Probable Mode (MPM) list for the current block.

Images