Image coding method using template-based intra prediction and device therefor

Abstract

An image decoding method according to an embodiment of the present disclosure includes the steps of: acquiring prediction-related information through a bitstream; deriving a prediction type for the current block on the basis of the prediction-related information; deriving a template for the current block on the basis of the prediction type; deriving n intra prediction modes for the current block on the basis of the template for the current block; generating n prediction blocks on the basis of the n intra prediction modes; and generating a prediction block for the current block on the basis of the n prediction blocks.

Description

Image coding method and device using template-based intra prediction

[0001] The present disclosure relates to a method for image / video coding and a device therefor.

[0002] Image / video coding is used in various applications such as digital storage media, television broadcasting, video streaming services, and real-time communications, and the demand for high-resolution, high-quality images / videos is increasing in various fields.

[0003] As the image / video becomes higher resolution and higher quality, the data size of the image / video increases, and the amount of information or bits transmitted increases relatively. Therefore, when transmitting image data using media such as existing wired or wireless broadband lines or storing image / video data using existing storage media, the transmission and storage costs increase.

[0004] In addition, interest in and demand for immersive media such as VR (virtual reality), AR (artificial reality), MR (mixed reality) content and holograms have been increasing recently, and attempts to provide immersive experiences using immersive media in games, education, medicine, real estate, marketing, etc. are increasing.

[0005] Accordingly, a highly efficient image / video compression technology is required to effectively compress, transmit, store, and play high-resolution, high-quality image / video information having various characteristics as described above.

[0006] According to one embodiment of the present disclosure, a method and device for improving video / image coding efficiency are provided.

[0007] According to one embodiment of the present disclosure, an intra prediction based video / image coding method and device are provided.

[0008] According to one embodiment of the present disclosure, a video decoding method performed by a decoding device is provided. The method includes a step of obtaining prediction-related information through a bitstream, a step of deriving a prediction type for the current block based on the prediction-related information, a step of deriving a template for the current block based on the prediction type, a step of deriving n intra-prediction modes for the current block based on the template for the current block, and a step of generating a prediction block for the current block based on the n intra-prediction modes, wherein a size of the template is determined based on a size of the current block.

[0009] According to one embodiment of the present disclosure, a video encoding method performed by an encoding device is provided. The method includes a step of determining a prediction type for a current block, a step of deriving a template for the current block based on the prediction type, a step of deriving n intra prediction modes for the current block based on the template for the current block, a step of generating a prediction block for the current block based on the n intra prediction modes, a step of generating prediction-related information based on the determined prediction type, and a step of encoding video information including the prediction-related information, wherein a size of the template is determined based on a size of the current block.

[0010] According to one embodiment of the present disclosure, a decoding device for video decoding is provided. The decoding device includes a memory and at least one processor connected to the memory, and the at least one processor is configured to perform the steps of: obtaining prediction-related information through a bitstream; deriving a prediction type for the current block based on the prediction-related information; deriving a template for the current block based on the prediction type; deriving n intra-prediction modes for the current block based on the template for the current block; and generating a prediction block for the current block based on the n intra-prediction modes, wherein a size of the template is determined based on a size of the current block.

[0011] According to one embodiment of the present disclosure, an encoding device for video encoding is provided. The encoding device includes a memory and at least one processor connected to the memory, and the at least one processor is configured to perform the steps of determining a prediction type for a current block, deriving a template for the current block based on the prediction type, deriving n intra prediction modes for the current block based on the template for the current block, generating a prediction block for the current block based on the n intra prediction modes, generating prediction-related information based on the determined prediction type, and encoding video information including the prediction-related information, wherein a size of the template is determined based on a size of the current block.

[0012] According to one embodiment of the present disclosure, a method is provided for transmitting video / video data including a bitstream generated according to a video / video encoding method according to at least one of the embodiments of the present disclosure.

[0013] According to one embodiment of the present disclosure, a device is provided for transmitting video / video data including a bitstream generated according to a video / video encoding method according to at least one of the embodiments of the present disclosure.

[0014] According to one embodiment of the present disclosure, a computer-readable storage medium storing a program for performing a method according to at least one of the embodiments of the present disclosure may be provided.

[0015] According to one embodiment of the present disclosure, a computer-readable digital storage medium storing encoded video / video information generated by a video / video encoding method according to at least one of the embodiments of the present disclosure is provided.

[0016] According to one embodiment of the present disclosure, there is provided a computer-readable digital storage medium storing encoded information or encoded video / image information that causes a decoding device to perform a video / image decoding method according to at least one of the embodiments of the present disclosure.

[0017] According to one embodiment of the present disclosure, the overall video / image compression efficiency can be improved.

[0018] According to one embodiment of the present disclosure, prediction performance for a current block can be improved.

[0019] According to one embodiment of the present disclosure, intra prediction of a current block can be performed based on a template.

[0020] According to one embodiment of the present disclosure, intra prediction performance for a current block can be improved while reducing side information for intra prediction mode signaling.

[0021] According to one embodiment of the present disclosure, intra prediction modes can be derived based on a template, and a prediction block for a current block can be generated through fusion or weighted sum of prediction blocks using the intra prediction modes, thereby increasing intra prediction efficiency.

[0022] According to one embodiment of the present disclosure, it is possible to efficiently determine an intra prediction mode stored for a current block for which intra prediction is performed based on a template and / or an intra prediction mode used for selecting a transformation kernel of the current block.

[0023] FIG. 1 schematically illustrates an example of a video / image coding system to which embodiments of the present disclosure may be applied.

[0024] FIG. 2 is a drawing schematically illustrating the configuration of a video / image encoding device to which embodiments of the present disclosure can be applied.

[0025] FIG. 3 is a drawing schematically illustrating the configuration of a video / image decoding device to which embodiments of the present disclosure can be applied.

[0026] Figure 4 illustrates an intra prediction procedure as an example.

[0027] Figure 5 shows examples of intra prediction based video / image encoding methods.

[0028] Figure 6 shows examples of intra prediction based video / image decoding methods.

[0029] Figure 7 shows examples of directional intra prediction modes.

[0030] Figure 8 shows an example of template-based HoG calculation in DIMD.

[0031] Figure 9 shows examples of peripheral blocks for deriving an MPM list.

[0032] Figure 10 illustrates DIMD-based modes and intra prediction modes derived from surrounding blocks.

[0033] Figure 11 illustrates intra prediction modes of previous blocks within a certain area.

[0034] Figure 12 shows an example of a planar horizontal mode-based prediction and an example of a planar vertical mode-based prediction.

[0035] Figure 13 shows an example of planar diagonal mode-based prediction.

[0036] Figure 14 illustrates an example of deriving a transformation kernel for the directional planar mode.

[0037] Figure 15 is an example of TIMD application when the current block is a square block.

[0038] Figure 16 is an example of TIMD application when the current block is a non-square block.

[0039] FIG. 17 schematically illustrates a video / image encoding method according to an embodiment(s) of the present disclosure.

[0040] FIG. 18 schematically illustrates a video / image decoding method according to an embodiment(s) of the present disclosure.

[0041] This disclosure may have various modifications and embodiments, and thus specific embodiments will be illustrated and described in detail in the drawings. However, this is not intended to limit the embodiments of the present disclosure to the specific embodiments. The terminology used herein is only used to describe specific embodiments and is not intended to limit the technical spirit of the present disclosure. The singular forms used herein are intended to include the plural forms as well, unless the context clearly indicates otherwise. The term "and / or" as used herein includes any one or a combination of two or more of the associated listed items. The terms "comprises," "comprises," and "contains" as used herein specify the presence of stated features, numbers, operations, elements, components, and / or combinations thereof, but do not preclude the presence or addition of one or more other features, numbers, operations, elements, components, and / or combinations thereof. The use of the term "can" in connection with an example or embodiment (e.g., what the example or embodiment can include or implement) in this disclosure means that there is at least one example or embodiment that includes or implements such feature, but not all examples are limited thereto and such feature or configuration may be omitted.

[0042] Meanwhile, each component in the drawings described in this disclosure is depicted independently for the convenience of explaining different characteristic functions. This does not imply that each component is implemented with separate hardware or software. For example, two or more components may be combined to form a single component, or a single component may be divided into multiple components. Embodiments in which each component is integrated and / or separated are also included within the scope of the present disclosure, as long as they do not deviate from the essence of the present disclosure.

[0043] In this disclosure, "A or B" can mean "only A," "only B," or "both A and B." In other words, "A or B" in this disclosure can be interpreted as "A and / or B." For example, "A, B or C" in this disclosure can mean "only A," "only B," "only C," or "any combination of A, B and C."

[0044] As used herein, a slash ( / ) or a comma may mean "and / or." For example, "A / B" may mean "A and / or B." Accordingly, "A / B" may mean "only A," "only B," or "both A and B." For example, "A, B, C" may mean "A, B, or C."

[0045] In the present disclosure, “at least one of A and B” may mean “only A,” “only B,” or “both A and B.” Additionally, in the present disclosure, the expressions “at least one of A or B” or “at least one of A and / or B” may be interpreted identically to “at least one of A and B.”

[0046] Additionally, in the present disclosure, “at least one of A, B and C” can mean “only A,” “only B,” “only C,” or “any combination of A, B and C.” Additionally, “at least one of A, B or C” or “at least one of A, B and / or C” can mean “at least one of A, B and C.”

[0047] Additionally, parentheses used in the present disclosure may mean "for example." Specifically, when "prediction (intra-prediction)" is indicated, "intra-prediction" may be suggested as an example of "prediction." In other words, "prediction" in the present disclosure is not limited to "intra-prediction," and "intra-prediction" may be suggested as an example of "prediction." Furthermore, even when indicated as "prediction (i.e., intra-prediction)," "intra-prediction" may be suggested as an example of "prediction."

[0048] Technical features individually described in one drawing in this disclosure may be implemented individually or simultaneously.

[0049] The present disclosure relates to video / image coding. For example, the methods / embodiments described in this disclosure may be applied to methods disclosed in the enhanced compression model (ECM) or H.267 standards. Furthermore, the methods / embodiments disclosed in this disclosure may be applied to methods disclosed in the AV2 (AOMedia Video 2) standard or next-generation video / image coding standards (e.g., H.268, H.269, etc.).

[0050] In the present disclosure, coding may include encoding and / or decoding. In the present disclosure, image coding may be used interchangeably with video coding.

[0051] In the present disclosure, a video may refer to a set of images over time. A picture generally refers to a unit representing one image at a specific time point, and a slice / tile is a unit that constitutes a part of a picture in coding. A slice / tile may include one or more CTUs (coding tree units). A picture may be composed of one or more slices / tiles. A tile may represent a rectangular area of ​​CTUs within a specific tile row and a specific tile column within a picture.

[0052] Meanwhile, a single picture may be divided into two or more subpictures. A subpicture may be a rectangular region of one or more slices within a picture.

[0053] A pixel or pel can refer to the smallest unit that constitutes a picture (or image). Additionally, the term "sample" can be used as a counterpart to a pixel. A sample can generally represent a pixel or a pixel value, and can also represent only the pixel / pixel value of the luma component or only the pixel / pixel value of the chroma component.

[0054] A unit may represent a basic unit of image processing. A unit may include at least one of a specific region of a picture and information related to the region. One unit may include one luma block and two chroma (e.g., cb, cr) blocks. In some cases, the term "unit" may be used interchangeably with terms such as "block" or "area." In general, an MxN block may include a set (or array) of samples (or sample array) or transform coefficients consisting of M columns and N rows.

[0055] Hereinafter, embodiments of the present disclosure will be described in more detail with reference to the attached drawings. Hereinafter, identical reference numerals may be used for identical components in the drawings, and redundant descriptions of identical components may be omitted.

[0056] FIG. 1 schematically illustrates an example of a video / image coding system to which embodiments of the present disclosure may be applied.

[0057] Referring to FIG. 1, a video / image coding system may include a first device (encoding device) and a second device (decoding device). The first device may transmit encoded video / image information or data to the second device via a digital storage medium or a network in the form of a file or streaming.

[0058] The video / image coding system may further include a video / image acquisition device and a video / image renderer. The video / image acquisition device may be included in the encoding device, or may be configured as a separate device or external component. The video / image renderer may be included in the decoding device, or may be configured as a separate device or external component.

[0059] The first device may include the transmission unit as an internal component, or as a separate device or external component.

[0060] The second device may include the receiver as an internal component, or as a separate device or external component.

[0061] An encoder may be referred to as an encoding device, and a decoder may be referred to as a decoding device. A transmitting unit may be included in an encoding device. A receiving unit may be included in a decoding device. A renderer may include a display unit, and the display unit may be comprised of a separate device or an external component.

[0062] The decoding device and encoding device to which the embodiment(s) of the present disclosure are applied may be included in a multimedia broadcasting transmitting and receiving device, a mobile communication terminal, a home cinema video device, a digital cinema video device, a surveillance camera, a video conversation device, a real-time communication device such as a video communication, a mobile streaming device, a storage medium, a camcorder, a video-on-demand (VoD) service providing device, an OTT (Over the top video) device, an Internet streaming service providing device, a three-dimensional (3D) video device, a VR (virtual reality) device, an AR (agumented reality) device, a video phone video device, a transportation terminal (e.g., a vehicle (including an autonomous vehicle) terminal, an airplane terminal, a ship terminal, etc.), and a medical video device, and may be used to process a video signal or a data signal. For example, the OTT (Over the top video) device may include a game console, a Blu-ray player, an Internet-connected TV, a home theater system, a smartphone, a tablet PC, a DVR (Digital Video Recorder), etc.

[0063] A video / image capture device can capture a video / image source. The video / image capture device can capture the video / image through a process of capturing, synthesizing, or generating the video / image. The video / image capture device can include a video / image capture device and / or a video / image generation device. The video / image capture device can include, for example, one or more cameras, a video / image archive containing previously captured video / images, etc. The video / image generation device can include, for example, a camcorder, a computer, a tablet, a smartphone, etc., and can (electronically) generate the video / image. For example, a virtual video / image can be generated through a computer, etc., in which case the video / image capture process can be replaced by a process of generating related data. The video / image source can also perform a video / image preprocessing process to input optimized video / image to an encoder.

[0064] An encoding device can encode input video / images. The encoding device can perform a series of procedures, such as prediction, transformation, and quantization, to improve compression and coding efficiency. The encoded data (encoded video / image information) can be output in the form of a bitstream.

[0065] The transmission unit can transmit encoded video / image information or data output in bitstream form to the reception unit of the receiving device through the network in the form of a file or streaming. The encoded video / image information or data output in bitstream form can also be transmitted to the reception unit through a streaming server. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmission unit can include an element for generating a media file through a predetermined file format and an element for transmission through a broadcasting / communication network. The reception unit can receive / extract the bitstream and transmit it to a decoding device.

[0066] The streaming server may temporarily store the bitstream during the process of transmitting or receiving the bitstream. The streaming server transmits multimedia data to a user device based on a user request via a web server, and the web server acts as a medium that informs the user of available services. When a user requests a desired service from the web server, the web server transmits it to the streaming server, and the streaming server transmits multimedia data to the user. At this time, the content streaming system may include a separate control server, and in this case, the control server serves to control commands / responses between each device within the content streaming system.

[0067] The streaming server can receive content from a media storage device and / or an encoding device. For example, when receiving content from the encoding device, the content can be received in real time. In this case, to provide a smooth streaming service, the streaming server can store the bitstream for a certain period of time.

[0068] The decoding device can decode the video / image by performing a series of procedures such as inverse quantization, inverse transformation, and prediction corresponding to the operation of the encoding device.

[0069] The renderer can render decoded video / images. The rendered video / images can be displayed through the display unit.

[0070] FIG. 2 is a diagram schematically illustrating the configuration of a video / image encoding device to which embodiments of the present disclosure may be applied. The term "encoding device" hereinafter may include an image encoding device and / or a video encoding device.

[0071] Referring to FIG. 2, the encoding device (200) may be configured to include an image partitioner (210), a prediction unit (predictor) 220, a residual processor (residual processor) 230, an entropy encoder (entropy encoder) 240, an adder (adder) 250, a filter (filter) 260, and a memory (memory) 270. The prediction unit (220) may include an inter prediction unit and an intra prediction unit. The residual processor (230) may include a transformer (transformer) 232, a quantizer (quantizer) 233, a dequantizer (dequantizer) 234, and an inverse transformer (inverse transformer) 235. The residual processor (230) may further include a subtractor (subtractor) 231. The addition unit (250) may be called a reconstruction unit or a reconstructed block generator. The image segmentation unit (210), prediction unit (220), residual processing unit (230), entropy encoding unit (240), addition unit (250), and filtering unit (260) described above may be configured by one or more hardware components (e.g., an encoder chipset or processor) depending on the embodiment. In addition, the memory (270) may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The hardware component may further include the memory (270) as an internal / external component.

[0072] The image segmentation unit (210) can segment an input image (or picture, frame) input to the encoding device (200) into one or more processing units. For example, the processing units may be referred to as coding units (CUs). In this case, the coding units may be recursively segmented from a coding tree unit (CTU) or a largest coding unit (LCU) according to a Quad-tree binary-tree ternary-tree (QTBTTT) structure. For example, one coding unit may be segmented into a plurality of coding units of deeper depth based on a quad-tree structure, a binary tree structure, and / or a ternary structure. In this case, for example, the quad-tree structure may be applied first, and the binary tree structure and / or the ternary structure may be applied later. Alternatively, the binary tree structure may be applied first. The coding procedure according to the present disclosure may be performed based on the final coding unit that is no longer segmented. In this case, based on coding efficiency according to image characteristics, etc., the maximum coding unit can be used as the final coding unit, or, if necessary, the coding unit can be recursively divided into coding units of lower depths, and the coding unit of the optimal size can be used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and restoration described below. As another example, the processing unit may further include a prediction unit (PU) or a transformation unit (TU). In this case, the prediction unit and the transformation unit may each be divided or partitioned from the final coding unit described above.The above prediction unit may be a unit for sample prediction, and the above transformation unit may be a unit for deriving a transformation coefficient and / or a unit for deriving a residual signal from a transformation coefficient.

[0073] The term "unit" may be used interchangeably with terms such as "block" or "area" depending on the case. In general, an MxN block can represent a set of samples or transform coefficients consisting of M columns and N rows. A sample can generally represent a pixel or a pixel value, and can represent only the pixel / pixel value of the luminance component, or only the pixel / pixel value of the chroma component. A sample can be used as a term corresponding to a pixel or pel in a picture (or image).

[0074] The encoding device (200) can generate a residual signal (residual block, residual sample array) by subtracting a prediction signal (predicted block, prediction sample array) output from a prediction unit from an input video signal (original block, original sample array), and the generated residual signal is transmitted to a conversion unit (232). In this case, as illustrated, a unit that subtracts a prediction signal (predicted block, prediction sample array) from an input video signal (original block, original sample array) within the encoder (200) may be called a subtraction unit (231). The prediction unit can perform prediction on a block to be processed (hereinafter, referred to as a current block) and generate a predicted block including prediction samples for the current block. The prediction unit can determine whether intra prediction or inter prediction is applied on a current block or CU basis. The prediction unit can generate various information regarding prediction, such as prediction mode information, as described later in the description of each prediction mode, and transmit the information to the entropy encoding unit (240). The information regarding prediction can be encoded in the entropy encoding unit (240) and output in the form of a bitstream.

[0075] An intra prediction unit can predict a current block by referring to samples within a current picture. The referenced samples may be located in the neighborhood of the current block or may be located away from the current block depending on the prediction mode. In intra prediction, prediction modes may include multiple non-directional modes and multiple directional modes. Non-directional modes may include, for example, a DC mode and a planar mode. Directional modes may include, for example, 33 directional prediction modes or 65 directional prediction modes depending on the degree of detail in the prediction direction. However, this is only an example, and more or less directional prediction modes may be used depending on the settings. The intra prediction unit may also determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring blocks.

[0076] An inter prediction unit can derive a predicted block for a current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. At this time, in order to reduce the amount of motion information transmitted in an inter prediction mode, motion information can be predicted in units of blocks, subblocks, or samples based on the correlation of motion information between neighboring blocks and the current block. The motion information can include a motion vector and a reference picture index. The motion information can further include information on an inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter prediction, neighboring blocks can include spatial neighboring blocks existing in the current picture and temporal neighboring blocks existing in the reference picture. A reference picture including the reference block and a reference picture including the temporal neighboring blocks may be the same or different. The above temporal neighboring blocks may be called collocated reference blocks, collocated CUs (colCUs), etc., and a reference picture including the temporal neighboring blocks may be called a collocated picture (colPic). For example, the inter prediction unit may construct a motion information candidate list based on the neighboring blocks, and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Inter prediction may be performed based on various prediction modes, and for example, in the case of skip mode and merge mode, the inter prediction unit may use the motion information of the neighboring blocks as the motion information of the current block. In the case of skip mode, unlike the merge mode, a residual signal may not be transmitted.In the motion vector prediction (MVP) mode, the motion vector of the surrounding blocks is used as a motion vector predictor, and the motion vector of the current block can be indicated by signaling the motion vector difference.

[0077] The prediction unit (220) can generate a prediction signal based on various prediction methods described below. For example, the prediction unit can apply intra prediction or inter prediction for prediction of a single block, and can also apply intra prediction and inter prediction simultaneously. This can be called combined inter and intra prediction (CIIP). In addition, the prediction unit can be based on an intra block copy (IBC) prediction mode or a palette mode for prediction of a block. The IBC prediction mode or palette mode can be used for screen content coding (SCC), for example. IBC basically performs prediction within the current picture, but can be performed similarly to inter prediction in that it derives a reference block based on a block vector within the current picture. That is, IBC can utilize at least one of the inter prediction techniques described in the present disclosure.

[0078] The prediction signal generated through the prediction unit (220) can be used to generate a restoration signal or a residual signal. The transformation unit (232) can apply a transformation technique to the residual signal to generate transform coefficients. For example, the transformation technique can include at least one of a Discrete Cosine Transform (DCT), a Discrete Sine Transform (DST), a Karhunen-Loe've Transform (KLT), a Graph-Based Transform (GBT), or a Conditionally Non-linear Transform (CNT).

[0079] The quantization unit (233) quantizes the transform coefficients and transmits them to the entropy encoding unit (240), and the entropy encoding unit (240) can encode the quantized signal (information about the quantized transform coefficients) and output it as a bitstream. The information about the quantized transform coefficients may be called residual information. The quantization unit (233) can rearrange the quantized transform coefficients in a block form into a one-dimensional vector form based on a coefficient scan order, and can also generate information about the quantized transform coefficients based on the quantized transform coefficients in the one-dimensional vector form. The entropy encoding unit (240) can perform various encoding methods, such as, for example, exponential Golomb, context-adaptive variable length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), etc. The entropy encoding unit (240) may encode information necessary for video / image restoration (e.g., values ​​of syntax elements, etc.) together or separately from the quantized transform coefficients. The encoded information (e.g., encoded video / image information) may be transmitted or stored in the form of a bitstream in units of NAL (network abstraction layer) units. The video / image information may further include information regarding various parameter sets such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may further include general constraint information. In the present disclosure, information and / or syntax elements transmitted / signaled from an encoding device to a decoding device may be included in the video / image information. The video / image information may be encoded through the above-described encoding procedure and included in the bitstream.The above bitstream may be transmitted through a network or stored in a digital storage medium. Here, the network may include a broadcasting network and / or a communication network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The signal output from the entropy encoding unit (240) may be configured as an internal / external element of the encoding device (200) by a transmitting unit (not shown) and / or a storing unit (not shown), or the transmitting unit may be included in the entropy encoding unit (240).

[0080] The quantized transform coefficients output from the quantization unit (233) can be used to generate a prediction signal. For example, by applying inverse quantization and inverse transformation to the quantized transform coefficients through the inverse quantization unit (234) and the inverse transform unit (235), a residual signal (residual block or residual samples) can be reconstructed. The addition unit (250) can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the reconstructed residual signal to the prediction signal output from the prediction unit. When there is no residual for the target block to be processed, such as when skip mode is applied, the predicted block can be used as the reconstructed block. The addition unit (250) may be called a reconstructor or a reconstructed block generation unit. The generated reconstructed signal can be used for intra prediction of the next target block to be processed within the current picture, and can also be used for inter prediction of the next picture after filtering as described below.

[0081] Meanwhile, LMCS (luma mapping with chroma scaling) may be applied during the picture encoding and / or restoration process.

[0082] The filtering unit (260) can improve subjective / objective picture quality by applying filtering to the restoration signal. For example, the filtering unit (260) can apply various filtering methods to the restoration picture to generate a modified restoration picture, and store the modified restoration picture in the memory (270), specifically, in the DPB of the memory (270). The various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc. The filtering unit (260) can generate information regarding filtering and transmit it to the entropy encoding unit (240). The information regarding filtering can be encoded by the entropy encoding unit (240) and output in the form of a bitstream.

[0083] The modified restored picture transmitted to the memory (270) can be used as a reference picture in the inter prediction unit. Through this, the encoding device can avoid prediction mismatch between the encoding device (200) and the decoding device when inter prediction is applied, and can also improve encoding efficiency.

[0084] The memory (270) DPB can store the modified reconstructed picture to be used as a reference picture in the inter prediction unit. The memory (270) can store motion information of a block from which motion information is derived (or encoded) within the current picture and / or motion information of blocks within a picture that has already been reconstructed. The stored motion information can be transferred to the inter prediction unit to be used as motion information of a spatial neighboring block or motion information of a temporal neighboring block. The memory (270) can store reconstructed samples of reconstructed blocks within the current picture and transfer them to the intra prediction unit.

[0085] FIG. 3 is a diagram schematically illustrating the configuration of a video / image decoding device to which embodiments of the present disclosure may be applied. The term "decoding device" hereinafter may include an image decoding device and / or a video decoding device.

[0086] Referring to FIG. 3, the decoding device (300) may be configured to include an entropy decoder (310), a residual processor (320), a predictor (330), an adder (340), a filter (350), and a memory (360). The predictor (330) may include an inter-prediction unit and an intra-prediction unit. The residual processor (320) may include a dequantizer (321) and an inverse transformer (321). The entropy decoding unit (310), residual processing unit (320), prediction unit (330), addition unit (340), and filtering unit (350) described above may be configured by a single hardware component (e.g., decoder chipset or processor) depending on the embodiment. In addition, the memory (360) may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The hardware component may further include the memory (360) as an internal / external component.

[0087] When a bitstream including video / image information is input, the decoding device (300) can restore the image corresponding to the process in which the video / image information is processed in the encoding device of FIG. 2. For example, the decoding device (300) can derive units / blocks based on block division-related information obtained from the bitstream. The decoding device (300) can perform decoding using a processing unit applied in the encoding device. Therefore, the processing unit of decoding may be, for example, a coding unit, and the coding unit may be divided from a coding tree unit or a maximum coding unit according to a quad tree structure, a binary tree structure, and / or a ternary tree structure. One or more transform units may be derived from the coding unit. Then, the restored image signal decoded and output through the decoding device (300) can be reproduced through a reproduction device.

[0088] The decoding device (300) can receive a signal output from the encoding device in the form of a bitstream, and the received signal can be decoded through the entropy decoding unit (310). For example, the entropy decoding unit (310) can parse the bitstream to derive information (e.g., video / image information) necessary for image restoration (or picture restoration). The video / image information may further include information on various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may further include general constraint information. The decoding device can decode the picture further based on the information on the parameter set and / or the general constraint information. The signaling / received information and / or syntax elements described later in the present disclosure can be obtained from the bitstream by being decoded through the decoding procedure. For example, the entropy decoding unit (310) can decode information in a bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and output the values ​​of syntax elements required for image restoration and the quantized values ​​of transform coefficients for residuals. More specifically, the CABAC entropy decoding method receives a bin corresponding to each syntax element in the bitstream, determines a context model using information of a syntax element to be decoded and decoding information of surrounding and decoding target blocks or information of symbols / bins decoded in the previous step, and predicts the occurrence probability of a bin according to the determined context model to perform arithmetic decoding of the bin to generate a symbol corresponding to the value of each syntax element.At this time, the CABAC entropy decoding method can update the context model using the information of the decoded symbol / bin for the context model of the next symbol / bin after determining the context model. Information regarding prediction among the information decoded by the entropy decoding unit (310) is provided to the prediction unit (330), and residual values ​​on which entropy decoding is performed by the entropy decoding unit (310), i.e., quantized transform coefficients and related parameter information, can be input to the residual processing unit (320). The residual processing unit (320) can derive a residual signal (residual block, residual samples, residual sample array). In addition, information regarding filtering among the information decoded by the entropy decoding unit (310) can be provided to the filtering unit (350). Meanwhile, a receiving unit (not shown) that receives a signal output from an encoding device may be further configured as an internal / external element of the decoding device (300), or the receiving unit may be a component of the entropy decoding unit (310). Meanwhile, the decoding device according to the present disclosure may be called a video / video / picture decoding device, and the decoding device may be divided into an information decoder (video / video / picture information decoder) and a sample decoder (video / video / picture sample decoder). The information decoder may include the entropy decoding unit (310), and the sample decoder may include at least one of the inverse quantization unit (321), the inverse transformation unit (322), the addition unit (340), the filtering unit (350), the memory (360), and the prediction unit (330).

[0089] The inverse quantization unit (321) can inverse quantize the quantized transform coefficients and output the transform coefficients. The inverse quantization unit (321) can rearrange the quantized transform coefficients into a two-dimensional block form. In this case, the rearrangement can be performed based on the coefficient scanning order performed in the encoding device. The inverse quantization unit (321) can perform inverse quantization on the quantized transform coefficients using quantization parameters (e.g., quantization step size information) and obtain transform coefficients.

[0090] In the inverse transform unit (322), the transform coefficients are inversely transformed to obtain a residual signal (residual block, residual sample array).

[0091] The prediction unit can perform a prediction on the current block and generate a predicted block containing prediction samples for the current block. Based on the information regarding the prediction output from the entropy decoding unit (310), the prediction unit can determine whether intra-prediction or inter-prediction is applied to the current block, and can determine a specific intra / inter-prediction mode.

[0092] The prediction unit (330) can generate a prediction signal based on various prediction methods described below. For example, the prediction unit can apply intra prediction or inter prediction for prediction of a single block, and can also apply intra prediction and inter prediction simultaneously. This can be called combined inter and intra prediction (CIIP). In addition, the prediction unit can be based on an intra block copy (IBC) prediction mode or a palette mode for prediction of a block. The IBC prediction mode or palette mode can be used for screen content coding (SCC), for example. IBC basically performs prediction within the current picture, but can be performed similarly to inter prediction in that it derives a reference block based on a block vector within the current picture. That is, IBC can utilize at least one of the inter prediction techniques described in the present disclosure.

[0093] The intra prediction unit can predict the current block by referencing samples within the current picture. The referenced samples may be located in the neighborhood of the current block or may be located away from the current block, depending on the prediction mode. In intra prediction, the prediction modes may include multiple non-directional modes and multiple directional modes. The intra prediction unit can also determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring blocks.

[0094] The inter prediction unit can derive a predicted block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. At this time, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information can be predicted in units of blocks, subblocks, or samples based on the correlation of the motion information between the neighboring blocks and the current block. The motion information can include a motion vector and a reference picture index. The motion information can further include information on the inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter prediction, the neighboring blocks can include spatial neighboring blocks existing in the current picture and temporal neighboring blocks existing in the reference picture. For example, the inter prediction unit (332) can construct a motion information candidate list based on the neighboring blocks, and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. Inter prediction can be performed based on various prediction modes, and the information about the prediction can include information indicating the mode of inter prediction for the current block.

[0095] The addition unit (340) can generate a restoration signal (restored picture, restoration block, restoration sample array) by adding the acquired residual signal to the prediction signal (predicted block, prediction sample array) output from the prediction unit (330). In cases where there is no residual for the block to be processed, such as when skip mode is applied, the predicted block can be used as the restoration block.

[0096] The addition unit (340) may be referred to as a restoration unit or restoration block generation unit. The generated restoration signal may be used for intra prediction of the next processing target block within the current picture, may be output after filtering as described below, or may be used for inter prediction of the next picture.

[0097] Meanwhile, LMCS (luma mapping with chroma scaling) may be applied during the picture decoding process.

[0098] The filtering unit (350) can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit (350) can apply various filtering methods to the restored picture to generate a modified restored picture, and transmit the modified restored picture to the memory (360), specifically, to the DPB of the memory (360). The various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc.

[0099] The (modified) reconstructed picture stored in the DPB of the memory (360) can be used as a reference picture in the inter prediction unit. The memory (360) can store motion information of a block from which motion information is derived (or decoded) within the current picture and / or motion information of blocks within a picture that has already been reconstructed. The stored motion information can be transmitted to the inter prediction unit to be used as motion information of a spatial neighboring block or motion information of a temporal neighboring block. The memory (360) can store reconstructed samples of reconstructed blocks within the current picture and transmit them to the intra prediction unit.

[0100] In this specification, the embodiments described in the filtering unit (260) and the prediction unit (220) of the encoding device (200) can be applied to the filtering unit (350) and the prediction unit (330) of the decoding device (300) in the same or corresponding manner, respectively.

[0101] As described above, prediction is performed to increase compression efficiency when performing video coding. Through this, a predicted block including prediction samples for a current block, which is a coding target block, can be generated. Here, the predicted block includes prediction samples in a spatial domain (or pixel domain). The predicted block is derived identically from an encoding device and a decoding device, and the encoding device can increase video coding efficiency by signaling information (residual information) about the residual between the original block and the predicted block, rather than the original sample value of the original block itself, to a decoding device. The decoding device can derive a residual block including residual samples based on the residual information, and generate a reconstructed block including reconstructed samples by combining the residual block and the predicted block, and can generate a reconstructed picture including the reconstructed blocks.

[0102] The residual information may be generated through a transformation and quantization procedure. For example, the encoding device may derive a residual block between the original block and the predicted block, perform a transformation procedure on residual samples (a residual sample array) included in the residual block to derive transform coefficients, and perform a quantization procedure on the transform coefficients to derive quantized transform coefficients, thereby signaling the related residual information to a decoding device (via a bitstream). Here, the residual information may include information such as value information, position information, a transformation technique, a transformation kernel, and quantization parameters of the quantized transform coefficients. The decoding device may perform an inverse quantization / inverse transformation procedure based on the residual information to derive residual samples (or residual blocks). The decoding device may generate a reconstructed picture based on the predicted block and the residual block. The encoding device can also inversely quantize / inversely transform the quantized transform coefficients to derive a residual block for reference in inter prediction of a subsequent picture, and generate a restored picture based on the residual block.

[0103] In the present disclosure, at least one of quantization / dequantization and / or transformation / inverse transformation may be omitted. If the quantization / dequantization is omitted, the quantized transform coefficient may be referred to as a transform coefficient. If the transformation / inverse transformation is omitted, the transform coefficient may be referred to as a coefficient or a residual coefficient, or may still be referred to as a transform coefficient for consistency of expression.

[0104] In addition, in the present disclosure, the quantized transform coefficients and transform coefficients may be referred to as transform coefficients and scaled transform coefficients, respectively. In this case, the residual information may include information about the transform coefficient(s), and the information about the transform coefficient(s) may be signaled via residual coding syntax. Transform coefficients may be derived based on the residual information (or information about the transform coefficient(s)), and scaled transform coefficients may be derived through inverse transformation (scaling) of the transform coefficients. Residual samples may be derived based on inverse transformation (transformation) of the scaled transform coefficients. This may be similarly applied / expressed in other parts of the present disclosure.

[0105] Intra prediction may refer to a prediction that generates prediction samples for a current block based on reference samples within a picture to which the current block belongs (hereinafter, referred to as the current picture). When intra prediction is applied to a current block, peripheral reference samples to be used for intra prediction of the current block may be derived. The peripheral reference samples of the current block may include H+W samples located to the left of a current block of size WХH, W+H samples located at the top of the current block, and at least one sample neighboring the top-left of the current block. Alternatively, the peripheral reference samples of the current block may include upper peripheral samples of multiple rows and left peripheral samples of multiple columns.

[0106] Some of the surrounding reference samples of the current block may not yet be decoded or available. In this case, the decoder can construct surrounding reference samples to be used for prediction by padding or substituting the unavailable samples with available samples.

[0107] When peripheral reference samples are derived, prediction samples of the current block can be derived based on the peripheral reference samples and intra prediction mode / type information. Here, the intra prediction mode can indicate one of non-directional prediction modes and directional prediction modes that indicate spatial correlation for intra prediction. Here, the directional prediction mode can be called an angular prediction mode, and the non-directional prediction mode can be called a non-angular prediction mode. The intra prediction type can indicate various prediction types for performing intra prediction. Intra prediction types may include, for example, multi-reference line (MRL), intra sub-partitions (ISP), Position dependent intra prediction (PDPC), matrix weighted intra prediction (MIP) or matrix based intra prediction, cross-component linear model (CCLM), multi-model linear model (MMLM), Decoder side intra mode derivation (DIMD), fusion of chroma intra prediction modes, intra template matching, fusion for template-based intra mode derivation (TIMD), intra prediction fusion, cross-component convolutional model (CCCM), cross-component prediction (CCP), spatial geometric partitioning mode (SGPM), etc. In some cases, intra prediction modes and / or intra prediction types may be used to perform intra prediction.

[0108] Specifically, the intra prediction procedure may include an intra prediction mode / type determination step, a reference sample derivation step, and an intra prediction mode / type-based prediction sample derivation step. Additionally, a post-processing filtering step may be performed on the derived prediction samples, if necessary.

[0109] Figure 4 illustrates an intra prediction procedure as an example.

[0110] Referring to FIG. 4, the intra prediction procedure as described above may include an intra prediction mode / type determination step, a reference sample derivation step, and an intra prediction performance (prediction sample generation) step. The intra prediction procedure may be performed in an encoding device and a decoding device as described above.

[0111] The coding device determines the intra prediction mode / type (S400). The coding device may include an encoding device and / or a decoding device as described above.

[0112] An encoding device can determine an intra prediction mode / type applied to the current block from among various intra prediction modes / types described in the present disclosure, and can generate prediction-related information. The prediction-related information can include intra prediction mode information indicating an intra prediction mode applied to the current block and / or intra prediction type information indicating an intra prediction type applied to the current block. A decoding device can determine an intra prediction mode / type applied to the current block based on the prediction-related information.

[0113] For example, when intra prediction is applied, the intra prediction mode to be applied to the current block can be determined using the intra prediction mode of the surrounding block. For example, the coding device can select one of the MPM candidates in the MPM list derived based on the intra prediction mode of the surrounding blocks of the current block (e.g., the left and / or upper surrounding blocks) and / or additional candidate modes based on the received index information, or can select one of the remaining intra prediction modes not included in the MPM candidates based on MPM reminder information (remaining intra prediction mode information). The MPM list can be configured to include or not include the planar mode as a candidate.

[0114] The coding device can construct a list of most probable modes (MPMs) for the current block. The MPM list can also be referred to as an MPM candidate list. Here, the MPM can refer to a mode used to improve coding efficiency by considering the similarity between the current block and surrounding blocks during intra prediction mode coding.

[0115] The encoding device can perform prediction based on various intra prediction modes, and determine an optimal intra prediction mode based on rate-distortion optimization (RDO) based on the prediction. In this case, the encoding device can determine the optimal intra prediction mode using MPM candidates configured in the MPM list, or can determine the optimal intra prediction mode using intra prediction modes other than the MPM candidates configured in the MPM list. Specifically, for example, if the intra prediction type of the current block is not a normal intra prediction type but a specific type (e.g., DIMD, TIMD, MRL, or ISP), the encoding device can determine the optimal intra prediction mode by considering only the MPM candidates as intra prediction mode candidates for the current block. That is, in this case, the intra prediction mode for the current block can be determined only from among the MPM candidates, and in this case, the MPM flag may not be encoded / signaled. In this case, the decoding device can infer that the MPM flag is 1 without being separately signaled with the MPM flag.

[0116] Meanwhile, in general, if the intra prediction mode of the current block is not a planar mode but one of the MPM candidates in the MPM list, the encoding device generates an mpm index (mpm idx) pointing to one of the MPM candidates. If the intra prediction mode of the current block is not in the MPM list either, the encoding device generates MPM reminder information (remaining intra prediction mode information) pointing to a mode that is the same as the intra prediction mode of the current block among the remaining intra prediction modes that are not included in the MPM list (and the planar mode). The MPM reminder information may include, for example, an intra_luma_mpm_remainder syntax element.

[0117] A decoding device obtains intra prediction mode information from a bitstream. The intra prediction mode information may include at least one of an MPM flag, an MPM index, and MPM reminder information (remaining intra prediction mode information) as described above. The decoding device may configure an MPM list. The MPM list is configured in the same manner as the MPM list configured in the encoding device. That is, the MPM list may include intra prediction modes of neighboring blocks, and may further include specific intra prediction modes according to a predetermined method.

[0118] The decoding device can determine the intra prediction mode of the current block based on the MPM list and the intra prediction mode information. For example, if the value of the MPM flag is 1, the decoding device can derive the candidate indicated by the MPM index among the MPM candidates in the MPM list as the intra prediction mode of the current block.

[0119] As another example, when the value of the MPM flag is 0, the decoding device can derive the intra prediction mode indicated by the remaining intra prediction mode information (which may be referred to as MPM remainder information) from among the remaining intra prediction modes as the intra prediction mode of the current block.

[0120] The coding device derives reference samples of the current block (S410). The reference samples may include peripheral reference samples of the current block. The peripheral reference samples of the current block may include H+W samples located to the left of the current block of size WХH, W+H samples located to the top of the current block, and at least one sample neighboring the top-left of the current block. Alternatively, the peripheral reference samples of the current block may include upper peripheral samples of multiple rows and left peripheral samples of multiple columns.

[0121] The coding device performs intra prediction on the current block to derive prediction samples (S1220). The coding device can derive the prediction samples based on the intra prediction mode / type and the reference samples. The coding device can derive a reference sample according to the intra prediction mode of the current block among the reference samples of the current block, and can derive a prediction sample of the current block based on the reference sample.

[0122] An encoding procedure based on intra prediction may roughly include, for example:

[0123] Figure 5 shows examples of intra prediction based video / image encoding methods.

[0124] Referring to FIG. 5, S500 may be performed by a prediction unit of an encoding device, S505 may be performed by a residual processing unit of the encoding device, and S510 or S515 may be performed by an entropy encoding unit of the encoding device. Specifically, the prediction-related information may be derived by the prediction unit and encoded by the entropy encoding unit. The residual information may be derived by the residual processing unit and encoded by the entropy encoding unit. The residual information is information about the residual samples. The residual information may include information about quantized transform coefficients for the residual samples. As described above, the residual samples may be derived as transform coefficients through a transform unit of the encoding device, and the transform coefficients may be derived as quantized transform coefficients through a quantization unit. The information about the quantized transform coefficients may be encoded in the entropy encoding unit through a residual coding procedure.

[0125] An encoding device performs intra prediction on a current block (S500). The encoding device can derive an intra prediction mode / type for the current block, derive reference samples of the current block, and generate prediction samples within the current block based on the intra prediction mode / type and the reference samples. Here, the intra prediction mode / type determination, surrounding reference sample derivation, and prediction sample generation procedures may be performed simultaneously, or one procedure may be performed before the other. The encoding device can determine a mode / type to be applied to the current block among a plurality of intra prediction modes / types. The encoding device can compare RD costs for the intra prediction modes / types and determine an optimal intra prediction mode / type for the current block.

[0126] Meanwhile, the encoding device may also perform a predictive sample filtering procedure. Predictive sample filtering may be referred to as post-filtering. Some or all of the predictive samples may be filtered through the predictive sample filtering procedure. In some cases, the predictive sample filtering procedure may be omitted.

[0127] The encoding device generates residual samples for the current block based on the predicted samples (S505). The encoding device can compare the predicted samples with the original samples of the current block based on phase and derive the residual samples.

[0128] An encoding device may encode image / video information including information regarding the intra prediction (prediction-related information) and / or information regarding the residual samples (residual information) (S510 or S515). The prediction-related information may include intra prediction mode information and intra prediction type information. The encoding device may output the encoded image / video information in the form of a bitstream. The output bitstream may be transmitted to a decoding device via a storage medium or a network.

[0129] The residual information may include residual coding syntax elements. The encoding device may transform / quantize the residual samples to derive quantized transform coefficients. The residual information may include information about the quantized transform coefficients.

[0130] Meanwhile, as described above, the encoding device can generate a restored picture (including restored samples and restored blocks). To this end, the encoding device can dequantize / inversely transform the quantized transform coefficients again to derive (corrected) residual samples. The reason for performing dequantization / inversely transforming the residual samples after transforming / quantizing them in this way is to derive residual samples that are identical to the residual samples derived from the decoding device as described above. The encoding device can generate a restored block including restored samples for the current block based on the predicted samples and the (corrected) residual samples. A restored picture for the current picture can be generated based on the restored block. As described above, an in-loop filtering procedure, etc. can be further applied to the restored picture.

[0131] The decoding device can perform operations corresponding to those performed by the encoding device. A video / image decoding procedure based on intra prediction may include, for example, the following.

[0132] Figure 6 shows examples of intra prediction based video / image decoding methods.

[0133] Referring to FIG. 6, S600 may be performed by an entropy decoding unit of a decoding device, S610 may be performed by a prediction unit of the decoding device, S615 may be performed by a residual processing unit of the decoding device, and S620 may be performed by an adder or restoration unit of the decoding device.

[0134] Specifically, the decoding device obtains image / video information from the bitstream (S600). The image / video information may include prediction-related information and / or residual information.

[0135] The decoding device performs intra prediction based on prediction-related information (S610). The decoding device may derive an intra prediction mode / type for the current block based on the prediction-related information, derive reference samples of the current block, and generate prediction samples within the current block based on the intra prediction mode / type and the reference samples. In this case, the decoding device may perform a prediction sample filtering procedure. The prediction sample filtering procedure may be referred to as post-filtering. Some or all of the prediction samples may be filtered by the prediction sample filtering procedure. In some cases, the prediction sample filtering procedure may be omitted.

[0136] The decoding device performs residual processing based on the residual information (S615). The decoding device can derive residual samples for the current block based on the residual information. Specifically, the inverse quantization unit of the residual processing unit performs inverse quantization based on the quantized transform coefficients derived based on the residual information to derive transform coefficients, and the inverse transform unit of the residual processing unit performs inverse transformation on the transform coefficients to derive residual samples for the current block.

[0137] The decoding device generates a reconstructed block / picture (S620). The decoding device can generate reconstructed samples for the current block based on the prediction samples and / or the residual samples, and derive a reconstructed block including the reconstructed samples. A reconstructed picture for the current picture can be generated based on the reconstructed block. As described above, an in-loop filtering procedure, etc., can be further applied to the reconstructed picture.

[0138] The intra prediction mode information and / or the intra prediction type information may be encoded / decoded through the binarization and coding method described in the present disclosure. For example, the intra prediction mode information and / or the intra prediction type information may be binarized through fixed-length binarization, truncated Rice binarization, truncated unary binarization, etc. For example, the intra prediction mode information and / or the intra prediction type information may be encoded / decoded through entropy coding (e.g., CABAC, CAVLC) coding.

[0139] Figure 7 illustrates examples of directional intra prediction modes. Figure 7 may illustrate an example of a case including 65 directional intra prediction modes.

[0140] Referring to FIG. 7, the directional intra prediction modes may include directional intra prediction modes of mode #2 to mode #65. In this case, mode #50 may represent a vertical intra prediction mode, and mode #18 may represent a horizontal intra prediction mode. Meanwhile, this is merely an example, and the number and types of candidate intra prediction modes may be changed. Meanwhile, one or more non-directional intra prediction modes may be considered, for example, mode #0 may represent an intra planar prediction mode (planar mode), and mode #1 may represent an intra DC prediction mode (DC mode).

[0141] For intra prediction modes, mode numbers can be assigned, for example, as shown in the following table.

[0142] Intra prediction modeAssociated name0INTRA_PLANAR1INTRA_DC2...66INTRA_ANGULAR2...INTRA_ANGULAR6681...83INTRA_LT_CCLM, INTRA_L_CCLM, INTRA_T_CCLM

[0143] According to the present disclosure, various intra prediction modes can be used for intra prediction of the current block. For example, an intra prediction mode can be derived based on the Decoder-side intra mode derivation (DIMD) technique. DIMD can be referred to as a DIMD type.

[0144] Figure 8 shows an example of template-based HoG calculation in DIMD.

[0145] Referring to Fig. 8, DIMD can calculate a Histogram of Gradient (HoG) using a template that includes surrounding samples of the current block. For example, the HoG can be calculated using horizontal / vertical Sobel filters based on line templates of n (e.g., 3) surrounding samples. In this case, horizontal and vertical Sobel filters can be applied to calculate the HoG. If the template is located in a different CTU, the Sobel filter may not be applied to the upper CTU boundary, or the DIMD technique may not be applied to the current block.

[0146] Meanwhile, the filter applied for HoG calculation may be determined differently based on the block size. For example, if the block size is 4x4, 4x8, or 8x4, a 2x2 kernel filter may be applied instead of a 3x3 Sobel filter for HoG calculation.

[0147] Meanwhile, an adaptive number of (surrounding) reference samples can be used to compute the HoG of DIMD. In this case, the number of sample lines in the DIMD template can be four or more.

[0148] For candidate intra prediction modes, n intra prediction modes having the highest histogram values ​​can be selected / extracted through the HoG calculation. In this case, a prediction block for the current block can be derived using the n intra prediction modes. Furthermore, in this case, the prediction block can be derived using the n intra prediction modes and a non-directional mode (e.g., planar mode or DC mode). In this case, a predictor according to each intra prediction mode can be derived, and the prediction block can be derived by weighting / averaging them. In other words, through prediction fusion, the predictors of the n (e.g., 5) selected / extracted intra prediction modes and the predictor of the planar mode can be fused.

[0149] For example, the above n may be 5. In this case, five derived modes with the highest HoG may be derived. The derived modes may be called DIMD-based modes or DIMD derived modes. For example, the number of n may be determined differently based on the block size. For example, when the W*H of the block is 128 or more, n may be 7, and in other cases, n may be 5.

[0150] Meanwhile, for the calculation of the HoG of DIMD, a specific block vector may be used. For example, the block vector may be derived based on the motion vector or block vector of a surrounding block. A reference area of ​​a moved location may be derived using a specific block vector based on the position of the current block, and the HoG may be calculated using (restored) reference samples within the reference area. In this case, the template may be located in the reference area or may be located on the upper / left periphery of the reference area. In this case, the size of the reference area may be the same as the size of the current block. Meanwhile, the size of the reference area may be different from the size of the current block. For example, the width of the reference area may be half the width of the current block, and / or the height of the reference area may be half the height of the current block. In this case, the size of the template may also be changed within the width and height range of the reference area.

[0151] The n intra modes derived through the above DIMD may be called DIMD derived modes or DIMD-based modes. The DIMD derived modes may be included as candidates in the MPM list (e.g., PMPM list).

[0152] When the DIMD technique (or type) is applied to the current block, the first intra mode derived by the DIMD can be stored as the intra mode of the target block and can be referenced in configuring the MPM list of the subsequent block.

[0153] When the DIMD technique (or type) is applied to the current block, the first intra mode derived by the DIMD can be referred to as the mode for selecting a transformation kernel (or transformation set).

[0154] Meanwhile, for the above weighted sum / weighted average or prediction fusion, weights between predictors are required. According to an embodiment of the present disclosure, a look-up table (LUT) can be used to derive the weights. Using the existing full HoG, dimdMode i Calculate dimdMode by calculating Habove and Hleft separately i We can calculate whether the dependency is on the upper template or the left template, i.e. whether it is location-dependent.

[0155] If it is not position dependent, the weight (wDimd) is based on the HoG size as before. i , wPlanar) can be determined.

[0156] If the data is position-dependent, sample-based blending can be applied. In this case, weights can be applied differently on a sample-by-sample basis.

[0157] If the upper HoG (e.g. Habove) is equal to or greater than twice the left HoG (e.g. Hleft), the weights can be based on the following mathematical formula:

[0158]

[0159] Here Δ i can represent the difference or absolute value of the upper HoG and the left HoG.

[0160] If the left HoG (e.g. Hleft) is equal to or greater than twice the upper HoG (e.g. Habove), the weights can be based on the following mathematical formula:

[0161]

[0162] Here Δ i can represent the difference or absolute value of the left HoG and the upper HoG.

[0163] Meanwhile, additional W columns can be used for DIMD if the upper right side is available, and additional H rows can be used if the lower left side is available. That is, the region of the template containing the surrounding (reconstruction) samples for HOG computation can be based on the availability of the surrounding (reconstruction) samples.

[0164] Meanwhile, when n intra (prediction) modes are derived from DIMD, n intra mode predictors and planner predictors are combined, but if only one intra mode is derived from DIMD, only the one intra mode can be used. Alternatively, even if only one intra mode is derived from DIMD, it can be combined with the planner mode predictor. For example, if the HoG value of the intra mode with the highest HoG in DIMD is equal to or greater than a threshold, only the one intra mode can be derived. As another example, if the first HoG value of the intra mode with the highest HoG in DIMD is greater than the second HoG value of the intra mode with the second highest HoG by a predetermined value or more, only the one intra mode can be derived. In addition, n intra prediction modes having HoGs greater than a predetermined threshold value among the DIMD derived modes may be selected.

[0165] Meanwhile, for a block to which DIMD is applied, the first mode derived from DIMD can be stored as the intra prediction mode of the current block. The stored mode can be referenced as a mode for selecting a mode of a surrounding block and / or a transformation kernel (or transformation set) to be referenced for deriving the intra prediction mode of a subsequent block.

[0166] Meanwhile, there may be cases where the first mode derived by DIMD is not the same as the actual prediction mode of the current block (or the optimal prediction mode most correlated with the current block). Therefore, it is necessary to review modes that can be stored as the intra prediction mode of the current block other than the first intra prediction mode derived by DIMD. For example, if the HoG of the first mode derived by DIMD is greater than a certain threshold, the first mode can be stored as the intra prediction mode of the current block, and if not, the planar or DC mode can be stored as the intra prediction mode of the current block. Alternatively, if the first predictor (first prediction samples) of the current block generated by DIMD and the second predictor (second prediction samples) generated by the first mode derived by DIMD are compared and are within a certain SAD / SATD, the DIMD-derived first mode can be stored as the intra prediction mode of the current block, and if not, the planar or DC mode can be stored as the intra prediction mode of the current block.

[0167] As another example, a candidate list may be formed with m intra prediction modes derived by DIMD, and any one of the candidate lists may be stored. Here, m may be equal to n, or m may be 2 or 3. In this case, the first predictor (first prediction samples) of the current block generated by DIMD may be compared with predictors according to each of the m intra prediction modes, and the intra prediction mode having the lowest SAD / SATD among the m intra prediction modes may be stored as the intra prediction mode of the current block.

[0168] Alternatively, the first intra prediction mode derived based on the DIMD technique for the first predictor (first prediction samples) derived by the DIMD technique (i.e., the intra prediction mode (first intra prediction mode) with the highest HoG based on the Sobel filter-based HoG derived from the prediction samples) may be stored as the intra prediction mode of the current block. In this case, the first intra prediction mode derived based on the template of the current block and the first intra prediction mode derived based on the predictor of the current block may be different.

[0169] DIMD-related information may be signaled for the above DIMD. The DIMD-related information may include at least one of a DIMD availability flag, a DIMD application flag, and / or a DIMD flag.

[0170] The DIDM enabled flag (dimd_enabeld_flag) can be signaled in higher level syntax (e.g., SPS). When the value of the DIMD enabled flag is 1, the DIMD applied flag (dimd_applied_flag) and / or the DIMD flag (dimd_flag) can be signaled.

[0171] The above DIMD application flag can be signaled at the PPS, PH (picture header), SH (slice header), or CTU level. Even when DIMD is available, DIMD can be turned on / off at the picture, slice, or CTU level through the DIMD application flag. The DIMD flag can be signaled at the CU, PU, ​​or TU level. The DIMD flag can be signaled before the MPM flag (or PMPM flag).

[0172] Signaling of the above DIMD related information may include, for example:

[0173] The DIMD flag (dimd_flag) may be signaled before the PMPM flag (pmpm_flag). In this case, the PMPM flag may be signaled when the value of the DIMD flag is 0, and may be omitted when the value of the DIMD flag is 1. This can be expressed, for example, as shown in the following table.

[0174]

[0175] Meanwhile, the DIMD flag (dimd_flag) may be signaled before (in parsing order) the reference line index (intra_luma_ref_idx). The reference line index is information indicating one or more of a plurality of surrounding reference sample lines when the plurality of surrounding reference sample lines are used for intra prediction of the current block. The reference line index may be signaled when the DIMD flag is not 1. The PMPM flag may be signaled when the DIMD flag is not 1 and the reference sample line index indicates 0. This may be expressed, for example, as shown in the following table.

[0176]

[0177] Meanwhile, the DIMD flag (dimd_flag) can be signaled after the reference line index (in the parsing order) (in the intra_luma_ref_idx). The DIMD flag can be signaled when the value of the reference line index is 0. The PMPM flag can be signaled when the DIMD flag is not 1 and the reference sample line index points to 0. This can be expressed, for example, as shown in the following table.

[0178]

[0179] Meanwhile, as described above, an MPM list may be constructed to efficiently signal the intra prediction mode of the current block. In a method according to one embodiment of the present disclosure, multiple MPM lists may be constructed. The multiple MPM lists may include a first MPM list and a second MPM list. The first MPM list may be referred to as a PMPM (primary MPM) list, and the second MPM list may be referred to as an SMPM (secondary MPM) list.

[0180] The first MPM list and the second MPM list may be configured according to predetermined criteria. For example, a general MPM list including k candidates may be configured first, and the first n candidates of the general MPM list may be included in the first MPM list, and the remaining m candidates may be included in the second MPM list. For example, k may be 22, n may be 6, and / or m may be 16. For example, the first candidate (the candidate of the first entry) of the general MPM list may always be in planar mode. As another example, when the planar mode is signaled based on a separate planar flag (or not_planar_flag), k may be 21, n may be 5, and / or m may be 16.

[0181] At least one of the remaining candidates, excluding the planar mode, can be derived from the surrounding blocks.

[0182] Figure 9 shows examples of peripheral blocks for deriving an MPM list.

[0183] Referring to FIG. 9, the surrounding blocks may include at least one of a left surrounding block (L), a lower left surrounding block (BL), an upper surrounding block (A), an upper right surrounding block (AR), and / or an upper left surrounding block (AL) of the current block.

[0184] In addition, the remaining candidates may include DIMD-based modes (intra prediction modes derived based on DIMD). For example, intra prediction modes derived from surrounding blocks and DIMD-based modes (intra prediction modes derived based on DIMD) may be sorted in descending order of SAD cost based on SAD cost (hereinafter, referred to as sorted intra prediction modes) and added to the general MPM list or the first MPM list. In this case, only a predetermined number p of sorted intra prediction modes may be extracted and added to the general MPM list or the first MPM list. Here, the SAD cost may be calculated based on a predictor derived by applying the candidate mode to the restored samples and template of the template of the current block. Directional prediction modes among the sorted intra prediction modes may be referred to as sorted directional modes. The p may be, for example, 5 or 6. Alternatively, the p may be 7 or 8. In the above DIMD-based modes, the planar mode and / or the DC mode may be excluded. The same applies hereinafter. The DIMD-based mode may be referred to as the DIMD-derived mode.

[0185] For example, one can add the sorted prediction modes, modes with offsets added or subtracted from the (extracted) sorted directional modes, and certain default modes until all k entries are filled.

[0186] As another example, the sorting procedure may be omitted for some or all of the DIMD-based modes. This is because the DIMD-based modes have a high correlation with the current block. Therefore, the sorting procedure may be omitted for some or all of the DIMD-based modes, and they may be assigned to the front of the MPM list (e.g., the general MPM list or the first MPM list). In this case, the planar mode may be positioned at the front, and some or all of the DIMD-based modes may be assigned after the planar mode. Alternatively, if the planar mode is not included in the MPM list, some or all of the DIMD-based modes may be assigned to the front. Alternatively, the DIMD-based modes may be compared with intra-prediction modes derived from neighboring blocks, and overlapping modes may be assigned to the front of the MPM list because they have a high correlation with the current block. In other words, the pruning procedure and the sorting procedure may be combined. For example, according to the previous method, a pruning procedure and a sorting procedure must be performed respectively to remove duplicates between intra prediction modes derived from surrounding blocks (first group) and DIMD-based modes (second group), but according to the present method, the sorting procedure can be omitted by assigning priorities to duplicate modes while performing a duplicate check.

[0187] Figure 10 illustrates DIMD-based modes and intra prediction modes derived from surrounding blocks.

[0188] Referring to Fig. 10, if mode B and mode C overlap between two groups, mode B and mode C can be assigned priority in the MPM list. For example, in this case, mode B and mode C can be assigned to the front of the MPM list or (immediately) after the planner mode.

[0189] In this case, for example, the priority between Mode B and Mode C can be based on the priority between DIMD-based modes. For DIMD-based modes, the priority can be determined without a separate SAD-based sorting procedure because the priority has already been derived based on HoG.

[0190] Meanwhile, intra prediction modes used in coding previous blocks within a certain region based on history can be stored and reused. This can be called HIPM (history-based intra prediction mode). The HIPM can be called by various names such as HMPM (history-based MPM), history-based candidate mode, etc. The certain region can include, for example, a CTU, a CTU row, a plurality of CTU rows, a slice, a tile, etc. The plurality of CTU rows can include, for example, the nth CTU row and the n-1th CTU row where the current block is located. In this case, the HIPM list can perform reordering based on the frequency of the intra prediction mode. That is, an intra prediction mode with a high occurrence frequency within a certain region can have a higher priority than an intra prediction mode with a low occurrence frequency and can be assigned to a higher position in the list. When assigned to a higher position in the list, the mode can be indicated with a lower index value.

[0191] Figure 11 illustrates intra prediction modes of previous blocks within a certain area.

[0192] Referring to Figure 11, a HIPM list can be configured such that a specific mode (e.g., Mode #50) with the highest frequency is given a higher priority. This list can be referred to as a buffer. In this case, for example, Mode #28 can be assigned the next highest frequency position after Mode #50.

[0193] When constructing the HIPM list, the intra prediction mode of the previous block within the predetermined region may be unavailable. This may be, for example, when the previous block is coded based on inter prediction (intercoded). In this case, the intra prediction mode of the previous block may be considered as planar mode or DC mode. Alternatively, the intra prediction mode of the previous block may be omitted when constructing the HIPM list.

[0194] The above HIPM list can be used to construct the MPM list. For example, the MPM list can be constructed based on x candidates in order of priority among the candidates of the HIPM list. That is, the MPM list can include x candidates in order of priority among the candidates of the HIPM list. Alternatively, the candidates of the MPM list can be derived based on intra prediction modes derived from neighboring blocks (first group) and DIMD-based modes (second group) and / or candidates of the HIPM list (third group). In this case, a pruning and sorting procedure can be performed, and only the first p can be extracted through the sorting procedure.

[0195] Meanwhile, as described above, a general MPM list, a first MPM list, and / or a second MPM list may exist. The second MPM list may be divided into four groups. In this case, for example, the PMPM flag may be signaled first, and when the PMPM flag is 1, the PMPM index may be signaled. Here, the PMPM flag may indicate whether the intra prediction mode of the current block exists in the first MPM list. For example, the GMPM flag may be signaled before the PMPM flag. In this case, the PMPM flag and the SMPM flag described below may be signaled when the GMPM flag is 1. When the PMPM flag is 0, the SMPM flag may be signaled first, and when the SMPM flag is 1, the group index (SMPM group index) for the second MPM list may be parsed / signaled first, and the mode index (SMPM mode index) may be signaled / parsed later. Information about the intra prediction mode (e.g., GMPM flag, PMPM flag, PMPM index, SMPM flag, SMPM group index, and / or SMPM mode index) can be signaled via the CU syntax. If the GMPM flag is 0, the PMPM flag and SMPM flag can be derived as 0 without signaling / coding. The GMPM flag can also be simply called the MPM flag. The GMPM flag can be omitted in some cases.

[0196] Information about intra prediction can be signaled, for example, as follows. Information about intra prediction can include one or more syntax elements. The same applies hereinafter.

[0197]

[0198] In this case, for example, smpm_group_idx can be fixed-length binarized, and smpm_mode_idx can be fixed-length binarized.

[0199] As another example, smpm_group_idx can be truncated rice (or truncated unary) binarized. In this case, smpm_mode_idx can be fixed-length binarized. Alternatively, smpm_mode_idx can determine the binarization method depending on the smpm_group_idx value. For example, if the smpm_group_idx value is 0 or 1, it can be truncated rice (or truncated unary) binarized, and in other cases, it can be fixed-length binarized. smpm_mode_idx can be context-model-based coded, in which case the context model of smpm_mode_idx can be determined differently based on the size of the current block and / or the smpm_group_idx value. For example, the context model can be directed based on the context index increment (ctxInc), and the ctxInc for a bin of smpm_mode_idx can be determined differently based on the smpm_group_idx value, for example, as follows. As described below, smpm_group_idx can be replaced with smpm_group_flag.

[0200]

[0201]

[0202] The second MPM list can be divided into two groups. In this case, the number of candidates in the first group can be the same as or different from the number of candidates in the second group. For example, the number of candidates in the second MPM list can be 16, 12, or 8. In this case, the first group can contain 4 or 8 candidates, and the second group can contain 8 candidates.

[0203] For example, smpm_group_flag can be used instead of smpm_group_idx. smpm_group_flag can indicate whether the intra prediction mode of the current block is included in the second candidate group among the candidate groups of the second MPM list. smpm_mode_idx can determine the binarization method according to the smpm_group_flag value. For example, if the smpm_group_flag value points to the first group (e.g., value 1), truncated rice (or truncated unary) binarization is performed, and in other cases, fixed-length binarization can be performed.

[0204] In this case, for example, information about intra prediction can be signaled as follows:

[0205]

[0206] Meanwhile, smpm_idc may be used instead of smpm_flag and / or smpm_group_flag (or smpm_group_idx).

[0207] In this case, for example, information about intra prediction can be signaled as follows:

[0208]

[0209] smpm_idc can indicate whether the intra prediction mode of the current block is included in the second MPM list, and if so, to which group it belongs. smpm_idc can be binarized based on truncated rice (or truncated unary).

[0210] The following table shows an example of binarization of smpm_idc.

[0211] smpm_idcDescriptionBinarization0Not included01In First group102In Second group11

[0212] smpm_mode_idx can be called smpm_idx. smpm_idx can be binarized based on truncated rice (or truncated unary).

[0213] The following table shows an example of binarization of smpm_idx.

[0214] smpm_idxDescriptionBinarization0First candidate01Second candidate102Third candidate1103Fourth candidate111

[0215] Alternatively, the binarization of smpm_idx can be based on smpm groups. For example, the binarization of smpm_idx can be based on the smpm_idc value. For example, if the smpm_idc value is 1, smpm_idx can be binarized based on truncated rice (or truncated unary), and if the smpm_idc value is 2, smpm_idx can be binarized based on fixed length. In this case, for example, the number of candidates in the first group can be 4, and the number of candidates in the second group can be 8. This allows the maximum number of bins for the binarization to match the worst case.

[0216] smpm_idx (for first group)DescriptionBinarization0First candidate01Second candidate102Third candidate1103Fourth candidate111

[0217] smpm_idx (for non-first group)DescriptionBinarization0First candidate0001Second candidate0012Third candidate0103Fourth candidate0114Fifth candidate1005Sixth candidate1016Seventh candidate1107Eighth candidate11

[0218] As described above, for the first group of the second MPM list, variable-length binarization can be performed considering the best case since the correlation with the current block is high, and for the second group of the second MPM list, fixed-length binarization can be performed considering the worst case since the correlation with the current block is relatively low.

[0219] The above smpm_idx can be context-based coded (e.g., CABAC), in which case the context index (or context index increment) of bin 0 can be set differently depending on the case. The context model of smpm_idx can be determined differently based on the size of the current block and / or the smpm_idc value. For example, the context index (or context index increment) of bin 0 of smpm_idx can be set differently based on smpm_idc (or smpm_group_idx or smpm_group_flag).

[0220] For example, if the value of smpm_idc (or smpm_group_idx or smpm_group_flag) is greater than 1, the context index increment of bin 0 of smpm_idx can be 1. If the value of smpm_idc (or smpm_group_idx or smpm_group_flag) is 1, the context index increment of bin 0 of smpm_idx can be 0. Or vice versa. In the tables below, smpm_idc can be replaced with smpm_group_idx or smpm_group_flag. In this case, the value of smpm_group_idx or smpm_group_flag, which is the condition, can be set to 1 less than the value of smpm_idc.

[0221]

[0222]

[0223]

[0224]

[0225] Through context-based coding as described above, a context model can be adaptively allocated to intra prediction mode-related information, and entropy coding efficiency can be efficiently increased.

[0226] Meanwhile, according to one embodiment of the present disclosure, two or more directional planar modes may be added in addition to the existing planar mode. The directional planar modes may include a planar horizontal mode and a planar vertical mode.

[0227] Figure 12 illustrates examples of planar horizontal mode-based prediction and planar vertical mode-based prediction. In Figure 12, (a) illustrates an example of planar horizontal mode-based prediction, and (b) illustrates an example of planar vertical mode-based prediction.

[0228] Referring to Fig. 12, in the planar horizontal mode, a prediction sample is generated using the left reference sample of the target sample within the current block and the upper right reference sample (TR) of the current block. In this case, the prediction sample can be generated by performing horizontal linear interpolation using the left reference sample and the upper right reference sample (TR). The horizontal linear interpolation can be performed based on, for example, the following mathematical equation.

[0229]

[0230] Here, pred(x, y) represents the predicted sample value at the (x, y) coordinate, W represents the width of the current block, and TR represents the x-coordinate of the upper-right reference sample. rec(-1, y) may represent the value of the restored reference sample at the (-1, y) coordinate, and rec(TR, -1) may represent the value of the restored reference sample (i.e., the upper-right reference sample) at the (TR, -1) coordinate. For reference, the above coordinates may represent the value when the top-left sample position of the current block is (0, 0).

[0231] In planar vertical mode, prediction samples are generated using the upper reference sample of the target sample within the current block and the lower left reference sample of the current block. In this case, the prediction samples can be generated by performing vertical linear interpolation using the upper reference sample and the lower left reference sample (BL). The vertical linear interpolation can be performed based on, for example, the following mathematical equation.

[0232]

[0233] Here, pred(x, y) represents the predicted sample value at the (x, y) coordinate, H represents the height of the current block, and BL represents the y-coordinate of the lower-left reference sample. rec(x, -1) may represent the value of the restored reference sample at the (x, -1) coordinate, and rec(-1, BL) may represent the value of the restored reference sample (i.e., the lower-left reference sample) at the (-1, BL) coordinate. For reference, the above coordinates may represent the value when the upper-left sample position of the current block is (0, 0).

[0234] Meanwhile, the above directional planar mode may further include a planar diagonal mode.

[0235] Figure 13 shows an example of planar diagonal mode-based prediction.

[0236] Referring to Fig. 13, in the planar diagonal mode, prediction sample generation can be performed through linear interpolation of two diagonal reference samples. For example, prediction sample generation can be performed through linear interpolation of an upper-right diagonal reference sample and a lower-left diagonal reference sample of a target sample within the current block. In this case, for a target sample located in the lower-right direction within the current block (e.g., P3, P6, P9, P7, P10, P11, P12, P13, P14 or P15), the upper-right surrounding reference sample and the lower-left surrounding reference sample of the current block can be used. For example, TR and BL can be used to predict target samples P3, P6, P9 or P12. In this case, a prediction sample can be generated through linear interpolation of TR and BL.

[0237] The directional planar mode may be determined to be available based on high-level syntax and block size. For example, a directional planar mode enabled flag (e.g., dir_planar_enabled_flag) may be signaled through high-level syntax. Here, the high-level syntax may include PPS, SPS, picture header, slice header, etc. Meanwhile, a constraint_flag for the directional planar mode may be signaled in general constraint information. For example, the general constraint information may include no_dir_planar_contraint_flag indicating whether the directional planar mode is restricted.

[0238] Additionally, the directional planar mode may be allowed when the block size is, for example, width*height > 64, width <= 128, height <= 128. Alternatively, the directional planar mode may be allowed when the block size is width*height > 128, width <= 256, height <= 256. Alternatively, the directional planar mode may be allowed only when the current block is non-square. Alternatively, the planar horizontal mode and the planar horizontal mode may be allowed when the current block is non-square, and the planar diagonal mode may be allowed when the current block is square. Alternatively, the planar horizontal mode may be allowed when the width of the current block is greater than the height, and the planar vertical mode may be allowed when the height of the current block is greater than the width. Alternatively, the planar horizontal mode may be allowed when the width of the current block is smaller than the height, and the planar vertical mode may be allowed when the height of the current block is smaller than the width. Alternatively, the planar horizontal mode may not be allowed when the width of the current block is larger than the height by a certain threshold value or more, and the planar vertical mode may not be allowed when the height of the current block is larger than the width by a certain threshold value or more. Alternatively, the planar horizontal mode may not be allowed when the width of the current block is smaller than the height by a certain threshold value or more, and the planar vertical mode may not be allowed when the height of the current block is smaller than the width by a certain threshold value or more. Through this, signaling such as selection information (e.g., directional_idx) described below may be omitted depending on the condition.

[0239] Information about the directional planner mode described below can be signaled if the directional planner mode availability flag allows the directional planner mode and / or if conditions for the block size are satisfied.

[0240] The above directional planar mode may be allowed when the current block is non-square. For example, if the current block is square, the prediction-related information may not include the selection information. In this case, the selection information may be derived as a specific value without being encoded / decoded.

[0241] For example, the planar horizontal mode may not be allowed if the width of the current block is greater than its height. In this case, the candidate modes may not include the planar horizontal mode. That is, the selection information may select one of the candidate modes excluding the planar horizontal mode. Alternatively, the signaling of the selection information may be omitted, and it may be implicitly determined that the planar vertical mode is applied to the current block.

[0242] For example, the planar vertical mode may not be allowed if the width of the current block is greater than its height. In this case, the candidate modes may not include the planar vertical mode. That is, the selection information may select one of the candidate modes excluding the planar vertical mode. Alternatively, the signaling of the selection information may be omitted, and it may be implicitly determined that the planar horizontal mode is applied to the current block.

[0243] For example, the planar horizontal mode may not be allowed if the width of the current block is greater than the height by a first threshold value. In this case, the candidate modes may not include the planar horizontal mode. That is, the selection information may select one of the candidate modes excluding the planar horizontal mode. Alternatively, the signaling of the selection information may be omitted, and it may be implicitly determined that the planar vertical mode is applied to the current block.

[0244] The planar vertical mode may not be allowed if the width of the current block is greater than the height by a second threshold value. In this case, the candidate modes may not include the planar vertical mode. That is, the selection information may select one of the candidate modes excluding the planar vertical mode. Alternatively, the signaling of the selection information may be omitted, and it may be implicitly determined that the planar horizontal mode is applied to the current block. The first threshold value may be the same as the second threshold value. Alternatively, the first threshold value may be different from the second threshold value.

[0245] For example, the planar horizontal mode may not be allowed if the height of the current block is greater than its width. In this case, the candidate modes may not include the planar horizontal mode. That is, the selection information may select one of the candidate modes excluding the planar horizontal mode. Alternatively, the signaling of the selection information may be omitted, and it may be implicitly determined that the planar vertical mode is applied to the current block.

[0246] For example, the planar vertical mode may not be allowed if the height of the current block is greater than its width. In this case, the candidate modes may not include the planar vertical mode. That is, the selection information may select one of the candidate modes excluding the planar vertical mode. Alternatively, the signaling of the selection information may be omitted, and it may be implicitly determined that the planar horizontal mode is applied to the current block.

[0247] For example, the planar horizontal mode may not be allowed if the height of the current block is greater than the width by a first threshold value. In this case, the candidate modes may not include the planar horizontal mode. That is, the selection information may select one of the candidate modes excluding the planar horizontal mode. Alternatively, the signaling of the selection information may be omitted, and it may be implicitly determined that the planar vertical mode is applied to the current block.

[0248] The above planar vertical mode may not be allowed if the height of the current block is greater than the width by a second threshold value. In this case, the candidate modes may not include the planar vertical mode. That is, the selection information may select one of the candidate modes excluding the planar vertical mode. Alternatively, the signaling of the selection information may be omitted, and it may be implicitly determined that the planar horizontal mode is applied to the current block. The first threshold value may be the same as the second threshold value. Alternatively, the first threshold value may be different from the second threshold value.

[0249] Signaling for the above directional planar modes (e.g., planar horizontal mode and planar vertical mode, etc.) can be performed, for example, as follows.

[0250]

[0251] Signaling for the above planar horizontal mode and planar vertical mode can be performed in an additional mode when the planar mode is applied to the current block.

[0252] directional_flag is flag information indicating whether a directional planar mode (e.g. planar horizontal mode or planar vertical mode, etc.) is applied to the current block.

[0253] directional_idx is optional information indicating which of the candidate modes, including planar horizontal mode and planar vertical mode, is applied to the current block. directional_idx may be called, for example, planar_hor_flag or planar_ver_flag.

[0254] The above flag information may be coded based on a context model. For example, the context model may be determined based on the size of the current block, the DIMD-based mode of the current block, and / or the values ​​of flag information of neighboring blocks for the current block. For example, the context model may be determined based on a context index increment, and the context index increment for the flag information may be derived based on the values ​​of flag information of the left neighboring block and flag information of the upper neighboring block.

[0255] The above selection information may be coded based on a context model. For example, the context model may be determined based on the size of the current block, the DIMD-based mode of the current block, and / or the value of selection information of a neighboring block for the current block. For example, the context model may be determined based on a context index increment, and the context index increment may be determined based on the size of the current block, the DIMD-based mode of the current block, and / or the value of selection information of a neighboring block for the current block.

[0256] Meanwhile, the if (INTRA_PLANAR) check is a conditional statement that determines whether the planar mode is applied to the current block. In cases where signaling for the planar mode is performed separately, it can be replaced with if (!not_planar_flag), etc.

[0257]

[0258]

[0259] Here, mpm_flag can be replaced with pmpm_flag, and mpm_idx can be replaced with pmpm_idx.

[0260] The above directional_idx can be binarized, for example, based on truncated rice (or truncated unary).

[0261] Typedirectional_idxbinarizationType100Type2110Type3211

[0262] For example, type 1 may correspond to planar horizontal mode, type 2 may correspond to planar vertical mode, and type 3 may correspond to planar diagonal mode. However, this is an example, and index allocation and / or binarization for directional_idx of the current block may be adaptively performed based on the size of the current block, DIMD-based mode, etc. For example, when the height of the current block is greater than the width, type 1 may correspond to planar horizontal mode, type 2 may correspond to planar vertical mode, and type 3 may correspond to planar diagonal mode. When the width of the current block is greater than the height, type 1 may correspond to planar vertical mode, type 2 may correspond to planar horizontal mode, and type 3 may correspond to planar diagonal mode. If the current block is square, Type 1 may correspond to the planar diagonal mode, Type 2 may correspond to the planar horizontal mode, and Type 3 may correspond to the planar vertical mode. Alternatively, if the current block is square, Type 1 may correspond to the planar diagonal mode, Type 2 may correspond to the planar vertical mode, and Type 3 may correspond to the planar horizontal mode.

[0263] Meanwhile, the above directional_idx may be omitted, and instead, it may be determined which of the candidate modes, including the planar horizontal mode and the planar vertical mode, is applied based on TIMD or DIMD. For example, it may be implicitly determined whether the planar vertical mode or the planar horizontal mode is applied based on the HoG derived based on DIMD.

[0264] In addition, when determining the availability of the planar horizontal mode and the planar vertical mode by comparing the width and height of the current block as described above, it is possible to determine which mode among the planar horizontal mode and the planar vertical mode is applied without signaling dirctional_idx.

[0265] Alternatively, syntax elements such as dir_planar_idc may be used instead of directional_flag and directional_idx . dir_planar_idc may be called planar_directional_idc or directional_idc .

[0266]

[0267]

[0268] dir_planar_idc indicates whether the current block is in normal planar mode, planar horizontal mode, or planar vertical mode. The dir_planar_idc can be binarized based on truncated rice (or truncated unitary).

[0269] Typedir_planar_idcbinarizationPlanar00Planar_horizontal110Planar_vertical211

[0270] Meanwhile, when a directional planar mode (such as planar horizontal mode or planar vertical mode) is applied to the current block, the intra prediction stored for the current block can be determined based on a predetermined method.

[0271] For example, the intra prediction mode of the current block referred to in procedures such as deriving the intra prediction mode of the subsequent block and generating the MPM list may be the (normal) planar mode.

[0272] Meanwhile, when selecting a transformation kernel for the first / second (inverse) transformation of the current block, it may also be based on the intra prediction mode of the current block. For example, a transformation set including transformation kernels may be determined based on the intra prediction mode of the current block. In this case, the intra prediction mode to be referenced may be mapped to the horizontal intra prediction mode if the planar horizontal mode is applied to the current block, or to the vertical intra prediction mode if the planar vertical mode is applied to the current block, and used when selecting the transformation kernel. In addition, the scan order for scanning (quantized) transform coefficients may be determined based on the intra prediction mode of the current block. In this case, for example, for determining the scan order, if the planar horizontal mode is applied to the current block, it may be considered that the horizontal intra prediction mode is applied, and if the planar vertical mode is applied to the current block, it may be considered that the vertical intra prediction mode is applied, and thus the scan order may be determined. For example, if it is determined that the horizontal intra prediction mode is applied, the scanning order may be horizontal, and if it is determined that the vertical intra prediction mode is applied, the scanning order may be vertical.

[0273] Meanwhile, in this case, there is a problem that more than two intra prediction modes are stored for the current block, which becomes a problem of asymmetric memory buffer allocation in the case of planar horizontal mode or planar vertical mode and in the case of other than planar horizontal mode in implementation.

[0274] Therefore, a method is required to coordinate the intra prediction mode stored for reference in deriving the intra prediction mode of a subsequent block and the intra prediction mode stored for selecting the transformation kernel of the current block. Therefore, a method is required in which the prediction mode for deriving the intra prediction mode of the surrounding blocks is also stored in horizontal / vertical mode, or a planar mode is used as the prediction mode for deriving the first / second transformation kernels as before, but the transformation kernels corresponding to the planar mode within the first / second transformation sets are changed.

[0275] For example, the intra prediction mode of the current block referred to in the procedure of deriving the intra prediction mode of the block thereafter and generating the MPM list, etc., may be mapped to the horizontal intra prediction mode if the planar horizontal mode is applied to the current block, or to the vertical intra prediction mode if the planar vertical mode is applied to the current block. The following table exemplarily shows the mapping relationship when the directional planar mode is used for the current block.

[0276] Mode Planner Horizontal Mode for selecting the mode transformation kernel (or transformation set) to be referenced in deriving the intra prediction mode of the block after the applied modeVertical Intra Prediction ModeVertical Intra Prediction ModePlanner Vertical ModeHorizontal Intra Prediction ModeHorizontal Intra Prediction Mode

[0277] That is, in this case, the number of intra prediction modes that must be stored for the current block can be reduced to 1.

[0278] Alternatively, separate mode numbers # can be assigned to the planner horizontal mode and planner vertical mode as follows. The mode numbers # here are examples, and other reserved numbers can be assigned.

[0279] Intra prediction modeAssociated name0INTRA_PLANAR91, 92INTRA_PLANAR_HOR, INTRA_PLANAR_VER1INTRA_DC2...6INTRA_ANGULAR26...NTRA_ANGULAR6681...83INTRA_LT_CCLM, INTRA_L_CCLM, INTRA_T_CCLM

[0280] In this case, the number of intra prediction modes to be stored for the current block can be fixed to 1.

[0281] Meanwhile, if reference is made to derive the intra prediction mode of a subsequent block or to derive a transformation kernel, a separate prediction mode mapping procedure may be performed.

[0282] For example, the storage stores the mode numbers of the planner horizontal mode and the planner vertical mode, but in actual use, it can be mapped and utilized as follows.

[0283] Applied Modes Modes for selecting the mode transformation kernel (or transformation set) to be referenced in deriving the intra prediction mode of the subsequent block after the mode being saved Planner Horizontal Mode Planner Horizontal Mode (ex. Mode # 91) Planner Mode (ex. Mode # 0) Vertical Intra Prediction Mode (ex. Mode # 50) Planner Vertical Mode Planner Horizontal Mode (ex. Mode # 92) Planner Mode (ex. Mode # 0) Horizontal Intra Prediction Mode (ex. Mode # 18)

[0284] Meanwhile, if a directional planar mode is applied to the current block, the mode to be saved is the planar mode, but when deriving a mode to be referred to in deriving the intra prediction mode of a subsequent block and / or for selecting a transformation kernel (or transformation set), the directional planar selection information (directional_idx or dir_planar_idc) may be checked, and the referenced mode may be adaptively derived based on this.

[0285] For example, if the planner orientation mode (planner horizontal mode or planner vertical mode, etc.) is applied as shown in the table below, saving can be done in planner mode.

[0286] Applied ModesSaved ModesPlanner Horizontal ModePlanner Mode (ex. Mode # 0)Planner Vertical ModePlanner Mode (ex. Mode # 0)

[0287] Meanwhile, in this case, for example, in the transformation kernel derivation step, the value of the directional planner selection information can be referenced as shown in the following drawing.

[0288] Figure 14 illustrates an example of deriving a transformation kernel for the directional planar mode.

[0289] Referring to FIG. 14, if the intra prediction mode of the current block, predModeIntra, is equal to the intra planar mode and directional_flag is 1, predModeIntra can be modified. In this case, for example, if directional_idx is 0 or the planar horizontal mode is applied, predModeIntra can be mapped to the intra horizontal mode. For example, if directional_idx is 1 or the planar vertical mode is applied, predModeIntra can be mapped to the intra horizontal mode. For example, if directional_idx is 2 or the planar diagonal mode is applied, predModeIntra can be mapped to the intra planar mode.

[0290] When deriving the transformation kernel of the current block, the predModeIntra after mapping can be used to select a transformation set and / or derive the transformation kernel.

[0291] Through the above method, the problem of asymmetric memory buffer allocation for the directional planar mode can be solved, and the intra prediction mode stored for reference in deriving the intra prediction mode of a subsequent block based on one stored intra prediction mode and the intra prediction mode stored for selecting the transformation kernel of the current block can be coordinated.

[0292] Meanwhile, according to one embodiment of the present disclosure, a TIMD technique may be applied. The TIMD may mean template-based intra mode derivation or fusion of template-based intra mode derivation. The TIMD technique may be referred to as a TIMD type. In TIMD, a template including surrounding samples of a current block is derived, a predictor of the template according to a candidate prediction mode is derived based on reference samples of the template (i.e., prediction samples of the template are derived), and this is compared with a reconstructed template (i.e., reconstructed samples of the template), thereby deriving an optimal intra prediction mode.

[0293] Figure 15 is an example of TIMD application when the current block is a square block.

[0294] Referring to FIG. 15, the template size of the current block may be L. That is, the template of the current block may include a left template and an upper template, the width of the left template may be L, and the height of the upper template may be L. Reference samples for the template are derived, and the reference samples for the template include left reference samples and upper reference samples. The height of the left reference region where the left reference samples are located may be based on the height of the left template, and the width of the upper reference region where the upper reference samples are located may be based on the width of the upper template. The width of the left reference region may be equal to the height of the upper reference region. The width of the left reference region may be 1, and the height of the upper reference region may be 1. When the current block is a square block, the height of the left reference region may be equal to the width of the upper reference region. Alternatively, when the current block is a square block, the height of the left reference region may be equal to the width of the upper reference region minus 1. Alternatively, if the current block is a square block, the height of the left reference area may be equal to the width of the upper reference area + 1.

[0295] Figure 16 is an example of TIMD application when the current block is a non-square block.

[0296] Referring to Fig. 16, the template of the current block includes a left template and an upper template, and the sizes of the left template and the upper template may be different. For example, the width of the left template may be L1, and the height of the upper template may be L2. The L2 may be different from the L1. The height L2 of the upper template and the width L1 of the left template may be different depending on the size of the current block, etc. For example, if the current block is a non-square block whose width is greater than its height, L2 may be larger than L1. For example, if the current block is a non-square block whose height is greater than its width, L1 may be larger than L2. Reference samples for the template are derived, and the reference samples for the template may be located in the left, upper left, and upper regions of the template. The reference samples for the template include left reference samples and upper reference samples. The height of the left reference region where the left reference samples are located may be based on the height of the left template, and the width of the upper reference region where the upper reference samples are located may be based on the width of the upper template. The width of the left reference region may be equal to the height of the upper reference region. The width of the left reference region may be 1, and the height of the upper reference region may be 1. When the current block is a non-square block, the height of the left reference region may be different from the width of the upper reference region. The height of the left reference region may be based on the height (H) of the current block and / or the height (L2) of the upper template. The width of the upper reference region may be based on the width (W) of the current block and / or the width (L1) of the left template. For example, the height of the left reference region may be equal to 2(H+L2) or 2(H+L2)+1. For example, the width of the upper reference area may be equal to 2(W+L1) or 2(W+L1)+1.

[0297] TIMD candidate intra prediction modes can be determined based on various criteria. For example, TIMD candidate intra prediction modes can be limited to MPMs. For example, TIMD candidate intra prediction modes can be limited to candidate modes in the MPM (or PMPM) list. In this case, for example, the timd flag can be signaled after the mpm flag (or pmpm flag) signaling.

[0298] As another example, predetermined candidate intra prediction modes can be defined. The predetermined candidate intra prediction modes can include some or all of the above-described DIMD derivation modes (DIMD-based modes). The predetermined candidate intra prediction modes can include a default mode. The default mode can include, for example, at least one of a vertical mode, a horizontal mode, a left-down diagonal mode, a left-up diagonal mode, or a right-up diagonal mode. In this case, the TIMD operation can be performed before the MPM list is constructed. For example, in this case, the timd flag can be signaled before the mpm flag (or pmpm flag) signaling. In other words, the timd flag can be signaled after the dimd flag and before the mpm flag (or pmpm flag).

[0299] According to the TIMD technique, among the candidate intra prediction modes, n intra prediction modes with the minimum SAD cost or minimum SATD cost can be derived. Here, n can be 1 or 2 or more. For example, when n is 2, two TIMD derived modes can be weight-based fused after applying PDPC. The weights can be calculated based on the costs for the n modes.

[0300] The weights for TIMD can be calculated, for example, using the following mathematical formulas.

[0301]

[0302]

[0303] When deriving weights, division operations can be replaced by using LUTs.

[0304] Meanwhile, the TIMD derivation mode (TIMD-based mode) can derive only one intra prediction mode when certain conditions are satisfied. For example, if the cost of the intra prediction mode having the minimum SAD cost or minimum SATD cost calculated based on TIMD is less than a threshold, only one intra prediction mode can be derived. As another example, if the difference between the first SATD cost of the first intra prediction mode having the minimum SATD cost calculated based on TIMD and the second SATD cost of the second intra prediction mode having the second minimum SATD cost is greater than a threshold, only one intra prediction mode can be derived.

[0305] Meanwhile, although the above example shows an example of calculating the cost based on SAD or SATD, this is an example, and MR SAD (Mean-Removed Sum of Absolute Differences) may also be used. In this case, the area of ​​the second template area when calculating the cost based on MR SAD may be wider than the area of ​​the first template area when calculating the cost based on SAD or SATD. For example, the area of ​​the second template area may be four times the area of ​​the first template area. For example, the value of L2 of the second template area may be twice the value of L2 of the first template area, and the value of L1 of the second template area may be twice the value of L1 of the first template area.

[0306] Meanwhile, for a block to which TIMD is applied, a specific intra mode can be stored as the intra prediction mode of the current block. The stored mode can be referenced as a mode for selecting a mode of a surrounding block and / or a transformation kernel (or transformation set) to be referenced in deriving the intra prediction mode of a subsequent block.

[0307] For example, for a block to which TIMD is applied, the first TIMD derived mode can be stored as the intra prediction mode of the current block.

[0308] As another example, for a block to which TIMD is applied, the first mode among the DIMD derivation modes may be stored as the intra prediction mode of the current block.

[0309] As another example, a candidate list may be formed with m intra prediction modes derived by TIMD, and any one of the candidate lists may be stored. Here, m may be equal to n, or m may be 2 or 3. In this case, the first predictor (first prediction samples) of the current block generated by TIMD or the reconstructed block may be compared with the predictors according to each of the m intra prediction modes, and the intra prediction mode having the lowest SAD / SATD among the m intra prediction modes may be stored as the intra prediction mode of the current block.

[0310] As another example, for a block to which TIMD is applied, the first intra prediction mode derived based on the DIMD technique for the prediction samples of the current block (i.e., the intra prediction mode (the first intra prediction mode) with the highest HoG based on the Sobel filter-based HoG derived from the prediction samples) can be stored as the intra prediction mode of the current block. In this case, the first intra prediction mode derived based on the TIMD of the current block and the first intra prediction mode derived based on the predictor of the current block may be different.

[0311] The above TIMD related signaling may include, for example:

[0312] The following table shows examples of signaling of prediction-related information, including TIMD-related signaling.

[0313]

[0314] Meanwhile, in the case of the above signaling method, there is a bit loss because information indicating whether it is a regular MPM procedure (e.g., regular MPM flag) must be additionally signaled.

[0315]

[0316] As described above, by signaling timd_flag, which indicates whether the TIMD technique is applied to the current block, before MPM mode information (e.g., mpm flag or pmpm_flag), additional information can be reduced. In this case, as described above, predetermined candidate intra prediction modes can be defined instead of MPMs and used in the TIMD procedure.

[0317] Meanwhile, TIMD may not be used together with ISP or MRL. For example, if TIMD is applied, ISP may not be used, and if TIMD is applied, MRL may not be used. For example, the TIMD flag may be signaled before the ISP flag. In this case, if the TIMD flag is 1, the ISP flag may be implicitly derived as 0 (without signaling). For example, the TIMD flag may be signaled before the MRL index. In this case, if the TIMD flag is 1, the MRL index may be implicitly derived as 0 (without signaling). In another example, the ISP flag may be signaled before the TIMD flag. In this case, if the ISP flag is 1, the TIMD flag may be implicitly derived as 0 (without signaling). In another example, the MRL index may be signaled before the TIMD flag. In this case, if the MRL index is non-zero, the TIMD flag can be implicitly derived as 0 (without signaling).

[0318] Meanwhile, in order to reduce the computational complexity for TIMD and increase signaling efficiency, a TIMD merge mode may be used. In this case, the TIMD derivation mode(s) of the current block may be derived based on a previously TIMD coded block. For this purpose, adjacent and non-adjacent spatial neighboring blocks may be scanned. The spatial neighboring blocks may be located in at least one of the left, upper, lower left, upper right, and upper left directions of the current block. In addition, temporal neighboring blocks may be scanned for the TIMD merge mode. The temporal neighboring blocks may include at least one of the lower right block, the center lower right block, the right block, and the lower block of the co-located block of the reference picture (called picture). If a scanned block is coded in TIMD or TIMD merge mode, a TIMD merge candidate including TIMD information (e.g., prediction mode, fusion flag, fusion weight, etc.) of the scanned block can be added to a TIMD merge (candidate) list. The TIMD merge list can include n candidates, where n can be signaled via HLS, e.g., SPS, PPS, or can be determined as a specific value (e.g., 5, etc.).

[0319] Whether TIMD merge mode is applied can be signaled based on a flag (timd merge flag) or implicitly determined based on a predetermined condition. The predetermined condition may include whether the size of the current block is within a predetermined range. The predetermined condition may include whether the current block is a square block.

[0320] The above TIMD merge flag may be signaled after the TIMD flag signaling if the TIMD flag indicates true (1).

[0321] The above TIMD merge list can be structured, for example, as shown in the following table.

[0322] TIMD merge indexCandidateIntra modesFusion weights01(IPM 1,1 , IPM 1,2 , IPM 1,3 )(W 1,1 , W 1,2 , W 1,3 )12(IPM 2,1 , IPM 2,2 , IPM 2,3 )(W 2,1 , W 2,2 , W 2,3 )............n-1n(IPM n,1 , IPM n,2 , IPM n,3 )(W n,1 , W n,2 , W n,3 )

[0323] As described above, the TIMD merge list may include TIMD merge candidates. The TIMD merge candidates may include n intra prediction modes (IPMs) and weights.

[0324] Candidates in the above TIMD merge list can be sorted based on SAD cost. In this case, a predetermined template can be used. For example, the size of the template can be 1 and include at least one of the first left peripheral sample line and the first upper peripheral sample line of the current block. The first IPM of the TIMD candidate can be used to calculate the SAD cost. Through this, a candidate with a lower SAD can be sorted so that a lower index value is assigned. For example, if the SADs of two candidates are the same, the sorting can be based on the scan order. That is, if the SADs of two candidates are the same, a lower index value can be assigned to a candidate that is added to the list first due to a higher scan order. As another example, if the SADs of two candidates (e.g., the first SAD) are the same, the second IPM of the TIMD candidate can be used to additionally calculate a second SAD. In this case, the second SADs of the two candidates can be compared, and a higher priority can be assigned to the candidate with the lower SAD value. Here, assigning a higher priority to a specific candidate may mean assigning a lower index value. Meanwhile, if the second SADs of two candidates are the same, a third SAD can be additionally calculated using the third IPM of the two candidates and compared. Alternatively, if the second SADs of two candidates are the same, a higher priority can be assigned to the candidate with a higher scan order. As another example, if the SADs of two candidates (e.g., the first SAD) are the same and the two candidates have the highest priority, a candidate selection index (or flag) indicating either index 0 or index 1 can be signaled from the encoder to the decoder to indicate a specific candidate. This allows for selecting the optimal candidate while minimizing the transmission of side information. The candidate selection index (or flag) can be signaled when the TIMD merge mode is applied and a specific condition is satisfied.The above specific condition may be the SAD of the two highest priority candidates (i.e., if there are two or more candidates with the lowest SAD).

[0325] The above TIMD merge mode can be applied to a luma block. That is, the TIMD merge mode can be applied when the current block is a luma component block. The TIMD merge mode can determine whether to be applied based on the block size. For example, the TIMD merge mode may not be applied when the size of the current block is equal to or smaller than a specific threshold (e.g., 4x4 block size). For example, the TIMD merge mode may not be applied when the size of the current block is equal to or larger than a specific threshold (e.g., 36x36 or 64x64 size). For example, the TIMD merge mode can determine whether to be applied based on whether the current block is non-square. Specifically, for example, when the current block is non-square, the TIMD merge mode may not be applied. Or, for example, the TIMD merge mode may not be applied when the difference between the width and height of the current block is greater than a certain value (e.g., 4x32, 32x4, 8x64, 64x8, 4x64, 64x4, etc.).

[0326] According to the above-described embodiment(s), intra prediction of a current block can be performed based on a template, thereby reducing side information for intra prediction mode signaling and improving intra prediction performance for the current block. In addition, intra prediction modes can be derived based on a template, and a (final) prediction block for the current block can be generated through fusion or weighted sum of prediction blocks using the derived intra prediction modes, thereby improving intra prediction efficiency. In addition, an intra prediction mode stored for a current block for which intra prediction is performed based on a template and / or an intra prediction mode used for selecting a transform kernel of the current block can be efficiently determined.

[0327] FIG. 17 schematically illustrates a video / image encoding method according to an embodiment(s) of the present disclosure. The method disclosed in FIG. 17 may be performed by the encoding device disclosed in FIG. 2. Specifically, for example, S1700 to S1740 of FIG. 17 may be performed by the prediction unit (220) of the encoding device (200), and S1750 of FIG. 17 may be performed by the entropy encoding unit (240) of the encoding device (200). The method disclosed in FIG. 17 may include the embodiments described above in the present disclosure.

[0328] Referring to FIG. 17, the encoding device determines a prediction type for the current block (S1700). The prediction type includes one of the prediction types described above. The prediction type may include, for example, the TIMD type or TIMD merge type described above.

[0329] The encoding device derives a template for the current block (S1710). The template may include a left template and an upper template. The left template may be located at a left periphery of the current block, and the upper template may be located at an upper periphery of the current block. The n intra prediction modes may be derived based on the template and reference samples of the template. The reference samples of the template may include first reference samples of a left reference area and second reference samples of an upper reference area. The left reference area may be located at a left periphery of the left template, and the upper reference area may be located at an upper periphery of the upper template.

[0330] The width of the left template may be L1, and the height of the upper template may be L2. Based on the case where the current block is a non-square block, the L1 may be different from the L2. Based on the case where the current block is a non-square block, the height of the left reference area may be different from the width of the upper reference area. The height of the left reference area may be based on the height H of the current block and the height L2 of the upper template, and the width of the upper reference area may be based on the width W of the current block and the width L1 of the left template. For example, the height of the left reference area may be equal to 2(H+L2) or 2(H+L2)+1, and the width of the upper reference area may be equal to 2(W+L1) or 2(W+L1)+1.

[0331] The encoding device derives n intra prediction modes for the current block based on the template (S1720). The n intra prediction modes for the current block may be selected from candidate intra prediction modes related to the prediction type. In this case, a cost for each candidate intra prediction mode may be calculated based on the template, the reference samples of the template, and the candidate intra prediction modes, and the n intra prediction modes may be determined based on the cost for each candidate intra prediction mode. The cost for each candidate intra prediction mode may be calculated based on SAD, SATD, or MR SAD. If the n intra prediction modes having the lowest cost among the above candidate intra prediction modes are determined, and the first cost of the candidate intra prediction mode having the first lowest cost among the above candidate intra prediction modes is less than a predetermined threshold value, or the first cost of the candidate intra prediction mode having the first lowest cost is less than the second cost of the candidate intra prediction mode having the second lowest cost by a threshold value or more, then n may be determined to be 1.

[0332] The n intra prediction modes for the current block are selected from among candidate intra prediction modes related to the prediction type, and the candidate intra prediction modes may include at least one of candidate modes of a most probable mode (MPM) list or decoder side intra mode derivation (DIMD) derived modes.

[0333] The n intra prediction modes for the current block are selected from candidate intra prediction modes related to the prediction type, and the candidate intra prediction modes additionally or alternatively include a default mode, and the default mode may include at least one of a vertical mode, a horizontal mode, a left-down diagonal mode, a left-up diagonal mode, and a right-up diagonal mode.

[0334] When the TIMD merge type is applied to the current block, the above-described S1710 and / or S1720 may be omitted or replaced with another procedure. For example, n intra prediction modes and / or weights of the current block may be derived based on the TIMD merge list.

[0335] The encoding device generates a prediction block for the current block based on the n intra prediction modes (S1730). The encoding device can generate n prediction blocks based on the n intra prediction modes, and generate a (final) prediction block for the current block based on the n prediction blocks. The encoding device can generate the (final) prediction block for the current block based on a weighted sum or weight-based fusion of the n blocks.

[0336] An encoding device generates prediction-related information (S1740). The prediction-related information may include various pieces of information described above. The prediction-related information may include information disclosed in the tables described above. For example, the prediction-related information may include a TIMD (fusion of template-based intra mode derivation) flag. The TIMD flag may indicate whether the TIMD type is applied as a prediction type for the current block. For example, the TIMD flag may have a signaling order that precedes a MPM (most probable mode) flag. For example, the TIMD flag may have a later signaling order than the DIMD (decoder side intra mode derivation) flag. For example, the TIMD flag may have a later signaling order than the DIMD (decoder side intra mode derivation) flag and the MPM (most probable mode) flag. The TIMD flag may be signaled based on a case where the value of the ISP (intra sub-partitions_mode) flag is 0. Based on a case where the value of the ISP flag is 1, the signaling of the TIMD flag may be omitted, and the value of the TIMD flag may be implicitly indicated as 0. The TIMD flag may be signaled based on a case where the value of the MRL index is 0, and when the value of the MRL index is not 0, the signaling of the TIMD flag may be omitted, and the value of the TIMD flag may be implicitly indicated as 0.

[0337] Based on the case where the prediction type for the current block is the TIMD type, at least one of the intra prediction modes stored for the current block or the intra prediction modes for selecting a transformation set for the current block can be determined as a specific mode.

[0338] For example, the specific mode may be determined as the first DIMD (decoder side intra mode derivation) derivation mode among the DIMD derivation modes.

[0339] As another example, a candidate list may be formed with m intra prediction modes derived by TIMD, and any one of the candidate lists may be stored. Here, m may be equal to n, or m may be 2 or 3. In this case, the first predictor (first prediction samples) of the current block generated by TIMD or the reconstructed block may be compared with the predictors according to each of the m intra prediction modes, and the intra prediction mode having the lowest SAD / SATD among the m intra prediction modes may be determined as the specific mode.

[0340] As another example, for a block to which TIMD is applied, the first intra prediction mode derived based on the DIMD technique for the prediction samples of the current block (i.e., the intra prediction mode (the first intra prediction mode) with the highest HoG based on the Sobel filter-based HoG derived again for the prediction samples) may be determined as the specific mode. In this case, the first intra prediction mode derived based on the TIMD of the current block and the first intra prediction mode derived based on the predictor of the current block may be different.

[0341] The above prediction-related information may include a TIMD merge flag. The TIMD merge flag may indicate whether the prediction type of the current block is a TIMD merge type.

[0342] Based on the case where the prediction type for the current block is the TIMD merge type, a TIMD merge list for the current block can be constructed. The TIMD merge list can be constructed based on neighboring blocks of the current block. The TIMD merge list can be constructed based on blocks among the neighboring blocks that are coded with the TIMD type (mode) or the TIMD merge type (mode). In this case, the TIMD merge list can be constructed sequentially based on the scan order of the neighboring blocks.

[0343] Candidates in the TIMD merge list can be sorted based on cost. The cost can be calculated based on SAD (sum of absolute difference), etc. Candidates in the TIMD merge list include n candidate intra prediction modes, and the cost for a candidate in the TIMD merge list can be calculated using the first candidate intra prediction mode among the n candidate intra prediction modes. In this case, if there is only one candidate having the minimum cost calculated using the first candidate intra prediction mode, the TIMD information (e.g., n intra prediction modes and weights) of the candidate can be used as the TIMD information of the current block. If there are multiple candidates having the minimum cost calculated using the first candidate intra prediction mode, a candidate having a higher priority in the scanning order can be selected. Alternatively, if there are multiple candidates having the minimum cost calculated using the first candidate intra prediction mode, the cost can be calculated using the second candidate intra prediction mode among the n candidate intra prediction modes for the multiple candidates, and one of the multiple candidates can be selected.

[0344] The encoding device encodes image information including the prediction-related information (S1750). For example, the prediction-related information may include CU syntax for the current block. The image information may be referred to as video information.

[0345] Additionally, the image information may include various information according to embodiments of the present disclosure. For example, the image information may include information disclosed in at least one of the tables described above.

[0346] Meanwhile, the image information may include residual information. The residual information is information about residual samples. The residual information may include information about quantized transform coefficients for the residual samples.

[0347] Encoded video information can be output in the form of a bitstream. The bitstream can be transmitted to a decoding device via a network or storage medium. For example, video data including the bitstream can be transmitted to a decoding device via a transmission device (or transmission unit). In this case, the video data including the bitstream can be transmitted to the decoding device via a streaming server.

[0348] In addition, as described above, the encoding device can generate a reconstructed picture (including reconstructed samples and a reconstructed block) based on the reference samples and the residual samples. This is to derive the same prediction result as that performed by the decoding device from the encoding device, thereby increasing coding efficiency. Accordingly, the encoding device can store the reconstructed picture (or reconstructed samples, reconstructed block) in memory and use it as a reference picture for inter prediction. As described above, an in-loop filtering procedure, etc. can be further applied to the reconstructed picture.

[0349] According to the above-described embodiment(s), intra prediction of a current block can be performed based on a template, thereby reducing side information for intra prediction mode signaling and improving intra prediction performance for the current block. In addition, intra prediction modes can be derived based on a template, and a (final) prediction block for the current block can be generated through fusion or weighted sum of prediction blocks using the derived intra prediction modes, thereby improving intra prediction efficiency. In addition, an intra prediction mode stored for a current block for which intra prediction is performed based on a template and / or an intra prediction mode used for selecting a transform kernel of the current block can be efficiently determined.

[0350] FIG. 18 schematically illustrates a video / image decoding method according to an embodiment(s) of the present disclosure. The method disclosed in FIG. 18 may be performed by the decoding device disclosed in FIG. 3. Specifically, for example, S1800 of FIG. 18 may be performed by the entropy decoding unit (310) of the decoding device (300), and S1810 to S1840 may be performed by the prediction unit (330) of the decoding device (300). The method disclosed in FIG. 18 may include the embodiments described above in the present disclosure.

[0351] Referring to FIG. 18, the decoding device obtains prediction-related information through the bitstream (S1800). The decoding device can obtain image information including the prediction-related information through the bitstream. The image information may further include residual information as described above.

[0352] The above prediction-related information may include the information disclosed in the above-described table. For example, the prediction-related information may include a TIMD (fusion of template-based intra-mode derivation) flag. The TIMD flag may indicate whether the TIMD type is applied as a prediction type for the current block. For example, the TIMD flag may have a signaling order that precedes the MPM (most probable mode) flag. For example, the TIMD flag may have a later signaling order than the DIMD (decoder side intra mode derivation) flag. For example, the TIMD flag may have a later signaling order than the DIMD (decoder side intra mode derivation) flag and the MPM (most probable mode) flag. The TIMD flag may be signaled based on a case where the value of the ISP (intra sub-partitions_mode) flag is 0. Based on a case where the value of the ISP flag is 1, the signaling of the TIMD flag may be omitted, and the value of the TIMD flag may be implicitly indicated as 0. The TIMD flag may be signaled based on a case where the value of the MRL index is 0, and when the value of the MRL index is not 0, the signaling of the TIMD flag may be omitted, and the value of the TIMD flag may be implicitly indicated as 0.

[0353] The above prediction-related information may include a TIMD merge flag. The TIMD merge flag may indicate whether the prediction type of the current block is a TIMD merge type.

[0354] The decoding device derives a prediction type for the current block based on the prediction-related information (S1810). The decoding device can determine whether the TIMD type applies to the current block based on the TIMD flag. The decoding device can also determine whether the prediction type is a TIMD merge type based on the TIMD merge flag.

[0355] Based on the case where the prediction type for the current block is the TIMD type, at least one of the intra prediction modes stored for the current block or the intra prediction modes for selecting a transformation set for the current block can be determined as a specific mode.

[0356] For example, the specific mode may be determined as the first DIMD (decoder side intra mode derivation) derivation mode among the DIMD derivation modes.

[0357] As another example, a candidate list may be formed with m intra prediction modes derived by TIMD, and any one of the candidate lists may be stored. Here, m may be equal to n, or m may be 2 or 3. In this case, the first predictor (first prediction samples) of the current block generated by TIMD or the reconstructed block may be compared with the predictors according to each of the m intra prediction modes, and the intra prediction mode having the lowest SAD / SATD among the m intra prediction modes may be determined as the specific mode.

[0358] As another example, for a block to which TIMD is applied, the first intra prediction mode derived based on the DIMD technique for the prediction samples of the current block (i.e., the intra prediction mode (the first intra prediction mode) with the highest HoG based on the Sobel filter-based HoG derived again for the prediction samples) may be determined as the specific mode. In this case, the first intra prediction mode derived based on the TIMD of the current block and the first intra prediction mode derived based on the predictor of the current block may be different.

[0359] Based on the case where the prediction type for the current block is the TIMD merge type, a TIMD merge list for the current block can be constructed. The TIMD merge list can be constructed based on neighboring blocks of the current block. The TIMD merge list can be constructed based on blocks among the neighboring blocks that are coded with the TIMD type (mode) or the TIMD merge type (mode). In this case, the TIMD merge list can be constructed sequentially based on the scan order of the neighboring blocks.

[0360] Candidates in the TIMD merge list can be sorted based on cost. The cost can be calculated based on SAD (sum of absolute difference), etc. Candidates in the TIMD merge list include n candidate intra prediction modes, and the cost for a candidate in the TIMD merge list can be calculated using the first candidate intra prediction mode among the n candidate intra prediction modes. In this case, if there is only one candidate having the minimum cost calculated using the first candidate intra prediction mode, the TIMD information (e.g., n intra prediction modes and weights) of the candidate can be used as the TIMD information of the current block. If there are multiple candidates having the minimum cost calculated using the first candidate intra prediction mode, a candidate having a higher priority in the scanning order can be selected. Alternatively, if there are multiple candidates having the minimum cost calculated using the first candidate intra prediction mode, the cost can be calculated using the second candidate intra prediction mode among the n candidate intra prediction modes for the multiple candidates, and one of the multiple candidates can be selected.

[0361] The encoding device derives a template for the current block (S1820). The template may include a left template and an upper template. The left template may be located at a left periphery of the current block, and the upper template may be located at an upper periphery of the current block. The n intra prediction modes may be derived based on the template and reference samples of the template. The reference samples of the template may include first reference samples of a left reference area and second reference samples of an upper reference area. The left reference area may be located at a left periphery of the left template, and the upper reference area may be located at an upper periphery of the upper template.

[0362] The width of the left template may be L1, and the height of the upper template may be L2. Based on the case where the current block is a non-square block, the L1 may be different from the L2. Based on the case where the current block is a non-square block, the height of the left reference area may be different from the width of the upper reference area. The height of the left reference area may be based on the height H of the current block and the height L2 of the upper template, and the width of the upper reference area may be based on the width W of the current block and the width L1 of the left template. For example, the height of the left reference area may be equal to 2(H+L2) or 2(H+L2)+1, and the width of the upper reference area may be equal to 2(W+L1) or 2(W+L1)+1.

[0363] The decoding device derives n intra prediction modes for the current block based on the template (S1830). The n intra prediction modes for the current block may be selected from candidate intra prediction modes related to the prediction type. In this case, a cost for each candidate intra prediction mode may be calculated based on the template, the reference samples of the template, and the candidate intra prediction modes, and the n intra prediction modes may be determined based on the cost for each candidate intra prediction mode. The cost for each candidate intra prediction mode may be calculated based on SAD, SATD, or MR SAD. If the n intra prediction modes having the lowest cost among the above candidate intra prediction modes are determined, and the first cost of the candidate intra prediction mode having the first lowest cost among the above candidate intra prediction modes is less than a predetermined threshold value, or the first cost of the candidate intra prediction mode having the first lowest cost is less than the second cost of the candidate intra prediction mode having the second lowest cost by a threshold value or more, then n may be determined to be 1.

[0364] The n intra prediction modes for the current block are selected from among candidate intra prediction modes related to the prediction type, and the candidate intra prediction modes may include at least one of candidate modes of a most probable mode (MPM) list or decoder side intra mode derivation (DIMD) derived modes.

[0365] The n intra prediction modes for the current block are selected from candidate intra prediction modes related to the prediction type, and the candidate intra prediction modes additionally or alternatively include a default mode, and the default mode may include at least one of a vertical mode, a horizontal mode, a left-down diagonal mode, a left-up diagonal mode, and a right-up diagonal mode.

[0366] When the TIMD merge type is applied to the current block, the above-described S1820 and / or S1830 may be omitted or replaced with another procedure. For example, n intra prediction modes and / or weights of the current block may be derived based on the TIMD merge list.

[0367] The decoding device generates a prediction block for the current block based on the n intra prediction modes (S1840). The decoding device can generate n prediction blocks based on the n intra prediction modes, and generate a (final) prediction block for the current block based on the n prediction blocks. The decoding device can generate the (final) prediction block for the current block based on a weighted sum or weight-based fusion of the n blocks.

[0368] As described above, in some cases, a prediction sample filtering procedure may be further performed on all or part of the prediction blocks (prediction samples) of the current block.

[0369] The decoding device can generate reconstructed samples based on prediction samples of the current block. For example, the decoding device can generate the reconstructed samples for the current block based on residual samples for the current block and the prediction samples. The residual samples for the current block can be generated based on received residual information. In addition, the decoding device can generate a reconstructed picture including the reconstructed samples, for example. As described above, an in-loop filtering procedure, etc. can be further applied to the reconstructed picture.

[0370] According to the above-described embodiment(s), intra prediction of a current block can be performed based on a template, thereby reducing side information for intra prediction mode signaling and improving intra prediction performance for the current block. In addition, intra prediction modes can be derived based on a template, and a (final) prediction block for the current block can be generated through fusion or weighted sum of prediction blocks using the derived intra prediction modes, thereby improving intra prediction efficiency. In addition, an intra prediction mode stored for a current block for which intra prediction is performed based on a template and / or an intra prediction mode used for selecting a transform kernel of the current block can be efficiently determined.

[0371] Although the methods described in the above-described embodiments are described based on a flowchart as a series of steps or blocks, the embodiments are not limited to the order of the steps, and some steps may occur in a different order or simultaneously with other steps described above. Furthermore, those skilled in the art will appreciate that the steps depicted in the flowchart are not exclusive, and that other steps may be included or one or more steps in the flowchart may be deleted without affecting the scope of the embodiments of the present disclosure.

[0372] The method according to the embodiments of the present disclosure described above can be implemented in the form of software, and the encoding device and / or decoding device according to the present disclosure can be included in a device that performs image processing, such as a TV, a computer, a smartphone, a set-top box, a display device, etc.

[0373] The embodiments of the present disclosure described above may also be implemented in the form of a recording medium containing computer-executable (program) instructions, such as program modules, executed by a computer. The modules may be stored in a memory and executed by a processor. The memory may be internal or external to the processor and may be connected to the processor by various well-known means. Computer-readable media may be any available media that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media. Furthermore, computer-readable media may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. Communication media typically includes computer-readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave, or other transport mechanism, and includes any information delivery media.

[0374] In addition, the embodiments of the present disclosure described above may be implemented as a computer program (or computer program product) including computer-executable instructions. The computer program includes programmable machine instructions processed by the processor, and may be implemented in a high-level programming language, an object-oriented programming language, assembly language, or machine language. In addition, the computer program may be recorded on a tangible computer-readable recording medium (e.g., memory, a hard disk, a magnetic / optical medium, or a solid-state drive (SSD), etc.).

[0375] Accordingly, the embodiments of the present disclosure described above can be implemented by executing the computer program described above on a computing device. The computing device may include a processor, memory, a storage device, a high-speed interface connecting the memory and a high-speed expansion port, and at least some of a low-speed interface connecting the low-speed bus and the storage device. Each of these components is connected to one another using various buses and may be mounted on a common motherboard or in another suitable manner.

[0376] Here, the processor can process instructions within the computing device, such as instructions stored in a memory or storage device to display graphical information for providing a graphical user interface (GUI) on an external input / output device, such as a display connected to a high-speed interface. In another embodiment, multiple processors and / or multiple buses may be utilized, as appropriate, together with multiple memories and memory types. The processor may also be implemented as a chipset comprising multiple independent analog and / or digital processors.

[0377] Memory also stores information within a computing device. For example, memory may consist of volatile memory units or a collection of volatile memory units. For another example, memory may consist of nonvolatile memory units or a collection of nonvolatile memory units. Memory may also be another form of computer-readable media, such as magnetic or optical disks.

[0378] A storage device can provide a large amount of storage space to a computing device. The storage device can be a computer-readable medium or a configuration including such a medium, and can include, for example, devices within a storage area network (SAN) or other configurations, and can be a floppy disk device, a hard disk device, an optical disk device, a tape device, flash memory, or other similar semiconductor memory device or device array.

[0379] Additionally, the network may be implemented as a wired network such as a Local Area Network (LAN), a Wide Area Network (WAN), or a Value Added Network (VAN), or as a wireless network of various types such as a mobile radio communication network or a satellite communication network.

[0380] Although the present disclosure has been described above with reference to the embodiments illustrated in the drawings, these are merely exemplary, and those skilled in the art will understand that various modifications and variations of the embodiments are possible from the above-described embodiments. In other words, the scope of the present disclosure is not limited to the above-described embodiments, and various modifications and improvements made by those skilled in the art using the basic concepts of the embodiments defined in the following claims also fall within the scope of the embodiments. Therefore, the true technical protection scope of the present disclosure should be determined by the technical spirit of the appended claims.

Claims

1. In a video decoding method performed by a decoding device, A step of obtaining prediction-related information through a bitstream; A step of deriving a prediction type for the current block based on the above prediction-related information; A step of deriving a template for the current block based on the above prediction type; A step of deriving n intra prediction modes for the current block based on a template for the current block; A step of generating a prediction block for the current block based on the n intra prediction modes is included. An image decoding method, characterized in that the size of the above template is determined based on the size of the current block.

2. In paragraph 1, The above template includes a left template and an upper template, The left template is located at the left periphery of the current block, and the upper template is located at the upper periphery of the current block. The above n intra prediction modes are derived based on the template and reference samples of the template, The reference samples of the above template include first reference samples of the left reference area and second reference samples of the upper reference area, The above left reference area is located on the left periphery of the above left template, An image decoding method, characterized in that the upper reference area is located at the upper periphery of the upper template.

3. In paragraph 2 The width of the left template above is L1, the height of the upper template above is L2, An image decoding method, wherein the L1 is different from the L2, based on the case where the current block is a non-square block.

4. In paragraph 3, An image decoding method, characterized in that the height of the left reference area is different from the width of the upper reference area, based on the case where the current block is a non-square block.

5. In paragraph 4 The height of the above left reference area is based on the height H of the current block and the height L2 of the upper template, An image decoding method, characterized in that the width of the upper reference area is based on the width W of the current block and the width L1 of the left template.

6. In Article 5 The height of the above left reference area is equal to 2(H+L2) or 2(H+L2)+1, An image decoding method, characterized in that the width of the upper reference area is equal to 2(W+L1) or 2(W+L1)+1.

7. In paragraph 2, The n intra prediction modes for the current block are selected from among candidate intra prediction modes related to the prediction type, A cost for each candidate intra prediction mode is calculated based on the above template, the above reference samples of the above template, and the above candidate intra prediction modes, A video decoding method, characterized in that the n intra prediction modes are determined based on the cost of each candidate intra prediction mode.

8. In paragraph 7, An image decoding method, characterized in that the cost for each of the above candidate intra prediction modes is calculated based on MR SAD (Mean-Removed Sum of Absolute Differences).

9. In Article 8 Among the above candidate intra prediction modes, the n intra prediction modes having the lowest cost are determined, A video decoding method, characterized in that n is determined to be 1 if a first cost of a candidate intra prediction mode having a first lowest cost among the candidate intra prediction modes is less than a predetermined threshold value, or if the first cost of the candidate intra prediction mode having the first lowest cost is less than a second cost of the candidate intra prediction mode having the second lowest cost by the threshold value or more.

10. In paragraph 1, The n intra prediction modes for the current block are selected from among candidate intra prediction modes related to the prediction type, A video decoding method, characterized in that the above candidate intra prediction modes include at least one of candidate modes of a most probable mode (MPM) list or decoder side intra mode derivation (DIMD) derived modes.

11. In paragraph 10, The n intra prediction modes for the current block are selected from among candidate intra prediction modes related to the prediction type, A method for decoding an image, wherein the candidate intra prediction modes further include a default mode, and the default mode includes at least one of a vertical mode, a horizontal mode, a left-down diagonal mode, a left-up diagonal mode, and a right-up diagonal mode.

12. In paragraph 1, The above prediction-related information includes a TIMD (fusion of template-based intra mode derivation) flag, The above TIMD flag indicates whether the TIMD type is applied as the prediction type for the current block. A video decoding method, characterized in that the TIMD flag has a signaling order that is earlier than the MPM (most probable mode) flag.

13. In paragraph 12, The above prediction-related information includes a TIMD (fusion of template-based intra mode derivation) flag, The above TIMD flag indicates whether the TIMD type is applied as the prediction type for the current block. A video decoding method, characterized in that the TIMD flag has a later signaling order than the DIMD (decoder side intra mode derivation) flag.

14. In paragraph 1 The above prediction-related information includes a TIMD (fusion of template-based intra mode derivation) flag, The above TIMD flag indicates whether the TIMD type is applied as the prediction type for the current block. A video decoding method, characterized in that the TIMD flag has a later signaling order than a DIMD (decoder side intra mode derivation) flag and an MPM (most probable mode) flag.

15. In paragraph 1 The above prediction-related information includes a TIMD (fusion of template-based intra mode derivation) flag, The above TIMD flag indicates whether the TIMD type is applied as the prediction type for the current block. The TIMD flag is signaled based on the case where the value of the ISP (intra sub-partitions_mode) flag is 0, A video decoding method, characterized in that when the value of the ISP flag is 1, signaling of the TIMD flag is omitted and the value of the TIMD flag is implicitly derived as 0.

16. In paragraph 1 The above prediction-related information includes the TIMD (fusion of template-based intra mode derivation) flag and the MRL (multi-reference line) index, The above TIMD flag indicates whether the TIMD type is applied as the prediction type for the current block. Based on the case where the value of the MRL index is 0, the TIMD flag is signaled, A video decoding method, characterized in that, based on a case where the value of the MRL index is not 0, signaling of the TIMD flag is omitted and the value of the TIMD flag is implicitly derived as 0.

17. In paragraph 1 The above prediction-related information includes a TIMD (fusion of template-based intra mode derivation) flag, The above TIMD flag indicates whether the TIMD type is applied as the prediction type for the current block. Based on the case where the prediction type for the current block above is the TIMD type, A video decoding method, characterized in that at least one of the intra prediction modes stored for the current block or the intra prediction modes for selecting a transformation set for the current block is determined as a first DIMD (decoder side intra mode derivation) mode among DIMD derivation modes.

18. In paragraph 1, Based on the case where the prediction type for the current block above is TIMD merge type, A TIMD merge list is constructed for the current block above, Candidates in the above TIMD merge list are sorted based on cost, The above cost is calculated based on SAD (sum of absolute difference), The candidates in the above TIMD merge list include n candidate intra prediction modes, Calculate the cost for a candidate in the TIMD merge list using the first candidate intra prediction mode among the n candidate intra prediction modes. An image decoding method, characterized in that when there are multiple candidates having a minimum cost calculated using the first candidate intra prediction mode, the cost is calculated using the second candidate intra prediction mode among the n candidate intra prediction modes for the multiple candidates, and one of the multiple candidates is selected.

19. In a video encoding method performed by an encoding device, A step for determining the prediction type for the current block; A step of deriving a template for the current block based on the above prediction type; A step of deriving n intra prediction modes for the current block based on a template for the current block; A step of generating a prediction block for the current block based on the n intra prediction modes; A step of generating prediction-related information based on the above-determined prediction type; and Comprising a step of encoding image information including the above prediction-related information, A video encoding method, characterized in that the size of the above template is determined based on the size of the current block.

20. In a transmission method for video data, A method for encoding an image, comprising: obtaining a bitstream generated by a video encoding method, the method comprising: a step of determining a prediction type for a current block; a step of deriving a template for the current block based on the prediction type; a step of deriving n intra prediction modes for the current block based on the template for the current block; a step of generating a prediction block for the current block based on the n intra prediction modes; a step of generating prediction-related information based on the determined prediction type; and a step of encoding video information including the prediction-related information; and Comprising a step of transmitting image data including the above bitstream, A transmission method, characterized in that the size of the above template is determined based on the size of the current block.

Images