Method for decoding image information, method for encoding image information and method for bitstream

Abstract

A method for decoding image information, according to the present disclosure, comprises: acquiring the image information comprising prediction information; deriving intra prediction for the current unit on the basis of the prediction information; deriving an intra mode on the basis of the intra prediction; and generating a prediction sample for the current unit on the basis of the intra mode. Deriving the intra mode comprises: deriving an intra mode being excluded; deriving intra mode candidates from which the intra mode being excluded has been excluded; and deriving the intra mode on the basis of the intra mode candidates from which the intra mode being excluded has been excluded.

Description

Methods for decoding video information, methods for encoding video information, and methods regarding bitstreams

[0001] The present disclosure relates to a method for decoding image information, a method for encoding image information, and a method for bitstreams.

[0002] Recently, the demand for high-resolution, high-quality video, such as HD (High Definition) and UHD (Ultra High Definition), has been increasing across various fields. As video data becomes higher in resolution and quality, the relative amount of information or bits transmitted increases compared to conventional video data. This increase in transmitted information or bits leads to higher transmission and storage costs.

[0003] Accordingly, high-efficiency video compression technology is required to effectively transmit, store, and play back high-resolution, high-quality video information.

[0004] The present disclosure aims to efficiently signal intra mode.

[0005] The present disclosure aims to improve data transmission efficiency.

[0006] The present disclosure aims to improve the prediction performance of intra-prediction.

[0007] The technical problems to be solved in this disclosure are not limited to those mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art to which this disclosure belongs from the description below.

[0008] A method for decoding image information according to one aspect of the present disclosure comprises acquiring the image information including prediction information, deriving a DIPM (derived intra prediction mode) for a current unit based on the prediction information, and generating a prediction sample for the current unit based on the DIPM. The DIPM is derived based on the size of the current unit.

[0009] According to one aspect of the present disclosure, an apparatus for decoding image information, wherein the above DIPM is derived based on the size of the current unit, comprises a memory and at least one processor connected to the memory, wherein the at least one processor acquires the image information including prediction information, derives a DIPM (derived intra prediction mode) for the current unit based on the prediction information, and generates a prediction sample for the current unit based on the DIPM. The DIPM is derived based on the size of the current unit.

[0010] In a method or device for decoding the above image information, the DIPM can be derived based on the fact that the product of the width and height of the current unit is less than or equal to a reference value.

[0011] In a method or device for decoding the above image information, the reference value may include at least one of 512, 1024, 2048, or 4096.

[0012] In the method or device for decoding the above image information, the DIPM can be derived based on the fact that the width of the current unit is less than or equal to a reference width and the height is less than or equal to a reference height.

[0013] In a method or device for decoding the above image information, the reference width may include at least one of 16, 32, or 64, and the reference height may include at least one of 16, 32, or 64.

[0014] In the method or device for decoding the above image information, the DIPM can be derived based on the fact that the current unit is not the upper-left unit of the current picture.

[0015] In a method or device for decoding the above image information, the DIPM can be derived based on the fact that MIP (matrix-based intra prediction) is not applied to the current unit.

[0016] In a method or device for decoding the above image information, the DIPM can be derived based on the fact that DIMD (Decoder-side intra mode derivation) is not applied to the current unit.

[0017] In a method or device for decoding the above image information, the DIPM can be derived based on the fact that TIMD (Template based intra mode derivation) is not applied to the current unit.

[0018] In a method or device for decoding the above image information, applying an MRL (multi-reference line) based intra prediction to the current unit based on the DIPM derived for the current unit may be restricted.

[0019] In a method or device for decoding the above image information, it may be restricted to apply an ISP (intra sub-partitions) based intra prediction to the current unit based on the DIPM derived for the current unit.

[0020] In a method or device for decoding the above image information, generating a prediction sample for the current unit may include deriving a DIPM candidate list based on the DIPM and generating a prediction sample for the current unit based on the DIPM candidate list.

[0021] In the method or device for decoding the above image information, the intra mode included in the DPIM candidate list may not be included in the MPM (most probable mode) list, the Secondary MPM list, and the remaining candidates.

[0022] According to one aspect of the present disclosure, a method for encoding image information comprises determining an intra prediction for a current unit, determining a DIPM (derived intra prediction mode) based on the intra prediction, generating a prediction sample for the current unit based on the DIPM, and encoding image information including prediction information related to the DIPM. The DIPM is determined based on the size of the current unit.

[0023] According to one aspect of the present disclosure, an apparatus for encoding image information comprises a memory and at least one processor connected to the memory, wherein the at least one processor determines an intra prediction for a current unit, determines a DIPM (derived intra prediction mode) based on the intra prediction, generates a prediction sample for the current unit based on the DIPM, and encodes image information including prediction information related to the DIPM. The DIPM is determined based on the size of the current unit.

[0024] A method for a bitstream according to one aspect of the present disclosure comprises generating a bitstream and transmitting data including said bitstream. Generating said bitstream comprises determining an intra prediction for a current unit, determining a derived intra prediction mode (DIPM) based on said intra prediction, generating a prediction sample for said current unit based on said DIPM, and encoding image information including prediction information related to said DIPM. said DIPM is determined based on the size of said current unit.

[0025] According to one aspect of the present disclosure, an apparatus for a bitstream comprises at least one processor for generating a bitstream and a transmission unit for transmitting data including said bitstream. The at least one processor determines an intra prediction for a current unit, determines a derived intra prediction mode (DIPM) based on said intra prediction, generates a prediction sample for said current unit based on said DIPM, and encodes image information including prediction information associated with said DIPM. The DIPM is determined based on the size of said current unit.

[0026] According to one aspect of the present disclosure, a computer-readable medium may store data comprising a bitstream generated based on determining an intra prediction for a current unit, determining a DIPM (derived intra prediction mode) based on the intra prediction, generating a prediction sample for the current unit based on the DIPM, and encoding image information including prediction information associated with the DIPM. The DIPM is determined based on the size of the current unit.

[0027] The features briefly summarized above regarding the present disclosure are merely exemplary aspects of the detailed description of the present disclosure that follows and do not limit the scope of the present disclosure.

[0028] According to the present disclosure, an intra mode can be efficiently signaled.

[0029] According to the present disclosure, data transmission efficiency can be improved.

[0030] According to the present disclosure, the prediction performance of intra-prediction can be improved.

[0031] The effects obtainable from the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which the present disclosure pertains from the description below.

[0032] FIG. 1 is a schematic diagram illustrating a video coding system to which an embodiment according to the present disclosure can be applied.

[0033] FIG. 2 is a schematic diagram showing an encoding device to which an embodiment according to the present disclosure can be applied.

[0034] FIG. 3 is a schematic diagram showing a decoding device to which an embodiment according to the present disclosure can be applied.

[0035] FIG. 4 is a drawing for illustrating a DIMD mode that can be applied to an embodiment according to the present disclosure.

[0036] FIG. 5 is a drawing for illustrating a DIMD mode that can be applied to an embodiment according to the present disclosure.

[0037] FIG. 6 is a drawing illustrating a method for decoding or encoding image information according to one example of the present disclosure.

[0038] FIG. 7 is a drawing showing a template area for decoding or encoding image information according to one example of the present disclosure.

[0039] FIG. 8 is a diagram illustrating a method for deriving an intra-mode candidate list to decode or encode image information according to one embodiment of the present disclosure.

[0040] FIG. 9 is a diagram illustrating a method for obtaining an error value to decode or encode image information according to one example of the present disclosure.

[0041] FIG. 10 is a diagram illustrating a method for decoding image information according to one embodiment of the present disclosure.

[0042] FIG. 11 is a drawing illustrating a method for encoding image information according to one embodiment of the present disclosure.

[0043] FIG. 12 illustrates an example of the size of a histogram corresponding to each of a plurality of intramodes according to one embodiment of the present disclosure.

[0044] FIG. 13 illustrates an example of the size of a histogram corresponding to each of a plurality of intramodes according to one embodiment of the present disclosure.

[0045] FIG. 14 illustrates an example of a method for deriving an intra mode according to one embodiment of the present disclosure.

[0046] FIG. 15 is a drawing illustrating an exemplary content streaming system to which an embodiment according to the present disclosure can be applied.

[0047] Hereinafter, embodiments of the present disclosure are described in detail with reference to the attached drawings so that those skilled in the art can easily implement them. However, the present disclosure may be embodied in various different forms and is not limited to the embodiments described herein.

[0048] In describing the embodiments of the present disclosure, detailed descriptions of known configurations or functions are omitted if it is determined that such descriptions could obscure the essence of the present disclosure. Additionally, parts of the drawings unrelated to the description of the present disclosure have been omitted, and similar parts are denoted by similar reference numerals.

[0049] In the present disclosure, when a component is described as being "connected," "combined," or "joined" with another component, this may include not only a direct connection but also an indirect connection in which another component exists in between. Furthermore, when a component is described as "comprising" or "having" another component, this means that, unless specifically stated otherwise, it does not exclude the other component but may include an additional component.

[0050] In the present disclosure, terms such as first, second, etc. are used solely for the purpose of distinguishing one component from another and do not limit the order or importance of the components unless specifically stated otherwise. Accordingly, within the scope of the present disclosure, a first component in one embodiment may be referred to as a second component in another embodiment, and likewise, a second component in one embodiment may be referred to as a first component in another embodiment.

[0051] In this disclosure, distinct components are intended to clearly describe their respective features and do not imply that the components are separate. That is, multiple components may be integrated to form a single hardware or software unit, or a single component may be distributed to form multiple hardware or software units. Accordingly, such integrated or distributed embodiments are included within the scope of this disclosure, unless otherwise noted.

[0052] In the present disclosure, the components described in various embodiments do not necessarily mean essential components, and some may be optional components. Accordingly, embodiments consisting of a subset of the components described in one embodiment are also included within the scope of the present disclosure. Furthermore, embodiments including additional components in addition to the components described in various embodiments are also included within the scope of the present disclosure.

[0053] The present disclosure relates to the encoding and decoding of images. For example, the methods and embodiments disclosed in this document may be applied to methods disclosed in the VVC (versatile video coding) standard, EVC (essential video coding) standard, AV1 (AOMedia Video 1) standard, AVS2 (2nd generation of audio video coding standard) or next-generation video / image coding standards (e.g., H.267 or H.268).

[0054] The present disclosure presents various embodiments relating to video / image coding, and unless otherwise stated, said embodiments may be performed in combination with one another.

[0055] Unless newly defined in this disclosure, the terms used herein may have the ordinary meanings commonly used in the technical field to which this disclosure belongs.

[0056] In this disclosure, "video" may refer to a set of images over time. In this disclosure, "picture" generally refers to a unit representing a single image at a specific time, and a slice / tile is a unit that constitutes a part of a picture in coding. A slice / tile may include one or more coding tree units (CTUs). A picture may be composed of one or more slices / tiles. A picture may be composed of one or more tile groups. A tile group may include one or more tiles. A brick may represent a rectangular area of ​​rows of CTUs within a tile in a picture. In this document, tile group and slice may be used interchangeably. For example, in this disclosure, a tile group / tile group header may be referred to as a slice / slice header.

[0057] In the present disclosure, "pixel" or "pel" may refer to the smallest unit constituting a picture (or image). Additionally, "sample" may be used as a term corresponding to pixel. A sample may generally represent a pixel or a pixel value, may represent only the pixel / pixel value of the luminance component, or may represent only the pixel / pixel value of the chroma component.

[0058] In this disclosure, "unit" may represent a basic unit of image processing. A unit may include at least one of a specific area of ​​a picture and information related to that area. A unit may include one luminance block and two chroma (e.g., cb, cr) blocks. Depending on the case, the term "unit" may be used interchangeably with terms such as "block" or "area." In general, an MxN block may include samples (or sample arrays) or a set (or array) of transform coefficients consisting of M columns and N rows.

[0059] In the present disclosure, "current block" may mean one of "current coding block," "current coding unit," "block to be encoded," "block to be decoded," or "block to be processed." When prediction is performed, "current block" may mean "current prediction block" or "block to be predicted." When transformation (inverse transformation) / quantization (inverse quantization) is performed, "current block" may mean "current transformation block" or "block to be transformed." When filtering is performed, "current block" may mean "block to be filtered."

[0060] In the present disclosure, "current block" may mean a block comprising both a luminous component block and a chroma component block, or "luma block of the current block," unless explicitly stated as a chroma block. The luminous component block of the current block may be expressed by including an explicit description of a luminous component block, such as "luma block" or "current luminous block." Additionally, the chroma component block of the current block may be expressed by including an explicit description of a chroma component block, such as "chroma block" or "current chroma block."

[0061] In the present disclosure, " / " and "," may be interpreted as "and / or." For example, "A / B" and "A, B" may be interpreted as "A and / or B." Additionally, "A / B / C" and "A, B, C" may mean "at least one of A, B and / or C."

[0062] In the present disclosure, "or" may be interpreted as "and / or". For example, "A or B" may mean 1) "A" only, 2) "B" only, or 3) "A and B". Alternatively, in the present disclosure, "or" may mean "additionally or alternatively".

[0063] FIG. 1 is a schematic diagram illustrating a video / image coding system to which an embodiment according to the present disclosure can be applied.

[0064] Referring to FIG. 1, a video / image coding system may include a first device (source device) and a second device (receiving device). The source device may transmit encoded video / image or data in the form of a file or streaming to the receiving device via a digital storage medium or a network.

[0065] The source device may include a video source, an encoding device, and a transmission unit. The receiving device may include a receiver, a decoding device, and a renderer. The encoding device may be called a video / image encoding device, and the decoding device may be called a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display unit, and the display unit may be composed of a separate device or an external component.

[0066] A video source may acquire video / images through processes such as video / image capture, synthesis, or generation. The video source may include a video / image capture device and / or a video / image generation device. The video / image capture device may include, for example, one or more cameras, a video / image archive containing previously captured video / images, etc. The video / image generation device may include, for example, a computer, a tablet, and a smartphone, etc., and may generate video / images (electronically). For example, virtual video / images may be generated through a computer, etc., in which case the video / image capture process may be replaced by a process in which related data is generated.

[0067] The encoding device can encode input video / images. The encoding device can perform a series of procedures, such as prediction, transformation, and quantization, for compression and coding efficiency. The encoded data (encoded video / image information) can be output in the form of a bitstream.

[0068] The transmission unit can transmit encoded video / image information or data output in the form of a bitstream to the receiving unit of a receiving device in the form of a file or streaming via a digital storage medium or a network. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmission unit may include elements for creating a media file through a predetermined file format and elements for transmission via a broadcasting / communication network. The receiving unit can receive / extract the bitstream and transmit it to a decoding device.

[0069] The decoding device can decode video / images by performing a series of procedures such as inverse quantization, inverse transform, and prediction corresponding to the operation of the encoding device.

[0070] The renderer can render the decoded video / image. The rendered video / image can be displayed through the display unit.

[0071] FIG. 2 is a schematic diagram illustrating an encoding device to which an embodiment according to the present disclosure can be applied.

[0072] Referring to FIG. 2, the encoding device (200) may be configured to include an image partitioner (210), a predictor (220), a residual processor (230), an entropy encoder (240), an adder (250), a filter (260), and a memory (270). The predictor (220) may include an inter-predictor (221) and an intra-predictor (222). The residual processor (230) may include a transformer (232), a quantizer (233), a dequantizer (234), and an inverse transformer (235). The residual processor (230) may further include a subtractor (231). The addition unit (250) may be referred to as a reconstructor or a reconstructed block generator. The above-described image segmentation unit (210), prediction unit (220), residual processing unit (230), entropy encoding unit (240), addition unit (250), and filtering unit (260) may be configured by one or more hardware components (e.g., an encoder chipset or processor) according to the embodiment. Additionally, the memory (270) may include a DPB (Decoded Picture Buffer) and may be configured by a digital storage medium. The hardware component may further include the memory (270) as an internal / external component.

[0073] The image segmentation unit (210) can divide an input image (or picture, frame) input to an encoding device (200) into one or more processing units. For example, the processing unit may be called a coding unit (CU). A coding unit may be recursively divided into a coding tree unit (CTU) or a largest coding unit (LCU) according to a QTBTTT (Quad-tree binary-tree ternary-tree) structure. For example, a single coding unit may be divided into multiple coding units of a deeper depth based on a quad-tree structure, a binary-tree structure, and / or a ternary-tree structure. For example, a quad-tree structure may be applied first, and a binary-tree structure and / or a ternary-tree structure may be applied later. Alternatively, a binary-tree structure may be applied first. A coding procedure according to the present disclosure may be performed based on the final coding unit that is no longer divided. In this case, based on coding efficiency according to image characteristics, the maximum coding unit may be used directly as the final coding unit, or, if necessary, the maximum coding unit may be recursively divided into lower-depth coding units so that a coding unit of the optimal size is used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and restoration described later. As another example, the processing unit may further include a prediction unit (PU) or a transformation unit (TU). The prediction unit and the transformation unit may each be divided or partitioned from the final coding unit.The above prediction unit may be a unit of sample prediction, and the above transformation unit may be a unit that derives transformation coefficients and / or a unit that derives a residual signal from transformation coefficients.

[0074] The term "unit" may be used interchangeably with terms such as "block" or "area" depending on the context. In general, an MxN block may represent a set of samples or transform coefficients consisting of M columns and N rows. A sample can generally represent a pixel or a pixel value, and may represent only the pixel / pixel value of the luminance component or only the pixel / pixel value of the chroma component. A sample may be used to refer to a single picture (or image) as a term corresponding to a pixel or pel.

[0075] The encoding device (200) can generate a residual signal (residual block, residual sample array) by subtracting a prediction signal (predicted block, prediction sample array) output from an inter prediction unit (221) or an intra prediction unit (222) from an input image signal (original block, original sample array), and the generated residual signal is transmitted to a conversion unit (232). In this case, as illustrated, the unit that subtracts the prediction signal (predicted block, prediction sample array) from the input image signal (original block, original sample array) within the encoding device (200) may be called a subtraction unit (231). The prediction unit (220) can perform a prediction for a block to be processed (hereinafter, current block) and generate a predicted block (predicted block) containing prediction samples for said current block. The prediction unit (220) can determine whether intra prediction is applied or inter prediction is applied in units of the current block or CU. The prediction unit (220) can generate various information regarding prediction, such as prediction mode information, as described below in the description of each prediction mode, and transmit it to the entropy encoding unit (240). The information regarding prediction can be encoded in the entropy encoding unit (240) and output in the form of a bitstream.

[0076] The intra prediction unit (222) can predict the current block by referring to samples within the current picture. The referenced samples may be located near the current block or away from it, depending on the prediction mode. In intra prediction, the prediction modes may include a plurality of non-directional modes and a plurality of directional modes. The non-directional modes may include, for example, a DC mode and a Planar mode. The directional modes may include, for example, 33 directional prediction modes or 65 directional prediction modes, depending on the degree of fineness of the prediction direction. However, this is merely an example, and depending on the settings, more or fewer directional prediction modes may be used. The intra prediction unit (222) may also determine the prediction mode applied to the current block by using the prediction mode applied to the surrounding blocks.

[0077] The inter prediction unit (221) can derive a predicted block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. At this time, to reduce the amount of motion information transmitted in the inter prediction mode, motion information can be predicted in blocks, sub-blocks, or samples based on the correlation of motion information between neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include information on the inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter prediction, neighboring blocks may include spatial neighboring blocks existing within the current picture and temporal neighboring blocks existing in the reference picture. The reference picture containing the reference blocks and the reference picture containing the temporal neighboring blocks may be the same or different from each other. The temporal neighboring blocks may be referred to by names such as collocated reference block, collocated CU (colCU), etc. A reference picture containing the aforementioned temporal surrounding blocks may be called a collocated picture (colPic). For example, the inter prediction unit (221) may construct a list of motion information candidates based on surrounding blocks and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Inter prediction may be performed based on various prediction modes, for example, in the case of skip mode and merge mode, the inter prediction unit (221) may use the motion information of surrounding blocks as motion information of the current block. In the case of skip mode, unlike merge mode, a residual signal may not be transmitted.In the motion vector prediction (MVP) mode, the motion vector of surrounding blocks is used as a motion vector predictor, and the motion vector of the current block can be indicated by signaling the motion vector difference.

[0078] The prediction unit (220) may generate a prediction signal based on various prediction methods and / or prediction techniques described below. For example, the prediction unit (220) may apply intra prediction or inter prediction for the prediction of the current block, as well as apply intra prediction and inter prediction simultaneously. A prediction method that applies intra prediction and inter prediction simultaneously for the prediction of the current block may be called combined inter and intra prediction (CIIP). Additionally, the prediction unit (220) may be based on an intra block copy (IBC) prediction mode or a palette mode for the prediction of the block. The IBC prediction mode or palette mode may be used for content video / video coding, such as in games, for example, screen content coding (SCC). IBC basically performs prediction within the current picture, but it may be performed similarly to inter prediction in that it derives a reference block within the current picture. That is, IBC may utilize at least one of the inter prediction techniques described in this disclosure. Palette mode can be viewed as an example of intra-coding or intra-prediction. When palette mode is applied, sample values ​​within a picture can be signaled based on information regarding palette tables and palette indices.

[0079] The prediction signal generated through the prediction unit (220) can be used to generate a restoration signal or to generate a residual signal. The subtraction unit (231) can generate a residual signal (residual signal, residual block, residual sample array) by subtracting the prediction signal (predicted block, prediction sample array) output from the prediction unit (220) from the input image signal (original block, original sample array). The generated residual signal can be transmitted to the conversion unit (232).

[0080] The transformation unit (232) can generate transform coefficients by applying a transformation technique to a residual signal. For example, the transformation technique may include at least one of a Discrete Cosine Transform (DCT), a Discrete Sine Transform (DST), a Karhunen-Loeve Transform (KLT), a Graph-Based Transform (GBT), or a Conditionally Non-linear Transform (CNT). Here, GBT refers to a transformation obtained from a graph when the relationship information between pixels is represented as a graph. CNT refers to a transformation obtained based on a prediction signal generated using all previously reconstructed pixels. The transformation process may be applied to a block of pixels of the same size in a square, or to a block of variable size that is not square.

[0081] The quantization unit (233) can quantize the transformation coefficients and transmit them to the entropy encoding unit (240). The entropy encoding unit (240) can encode the quantized signal (information regarding the quantized transformation coefficients) and output it as a bitstream. The information regarding the quantized transformation coefficients may be called residual information. The quantization unit (233) can rearrange the block-shaped quantized transformation coefficients into a one-dimensional vector form based on the coefficient scan order, and can also generate information regarding the quantized transformation coefficients based on the one-dimensional vector-shaped quantized transformation coefficients.

[0082] The entropy encoding unit (240) can perform various encoding methods such as, for example, exponential Golomb, CAVLC (context-adaptive variable length coding), CABAC (context-adaptive binary arithmetic coding), etc. The entropy encoding unit (190) may encode information required for video / image restoration (e.g., values ​​of syntax elements) together or separately, in addition to quantized transform coefficients. The encoded information (e.g., encoded video / image information) may be transmitted or stored in the form of a bitstream in units of NAL (network abstraction layer) units. The video / image information may further include information regarding various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). Additionally, the video / image information may further include general constraint information. The signaling information, transmitted information, and / or syntax elements mentioned in the present disclosure may be included in the video / image information. The video / image information may be encoded through the encoding procedure described above and included in the bitstream.

[0083] The above bitstream may be transmitted via a network or stored in a digital storage medium. Here, the network may include a broadcasting network and / or a communication network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmission unit (not shown) for transmitting a signal output from the entropy encoding unit (240) and / or a storage unit (not shown) for storing it may be provided as an internal / external element of the encoding device (200), or the transmission unit may be provided as a component of the entropy encoding unit (240).

[0084] The quantized transformation coefficients output from the quantization unit (233) can be used to generate a residual signal. For example, a residual signal (residual block or residual samples) can be restored by applying inverse quantization and inverse transformation to the quantized transformation coefficients through the inverse quantization unit (234) and the inverse transformation unit (235).

[0085] Meanwhile, LMCS (luma mapping with chroma scaling) may be applied during the picture encoding and / or restoration process.

[0086] The adder (250) can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the reconstructed residual signal to the prediction signal output from the inter prediction unit (221) or the intra prediction unit (222). In cases where there is no residual for the block to be processed, such as when a skip mode is applied, the predicted block can be used as the reconstructed block. The adder (250) may be called a reconstructed unit or a reconstructed block generation unit. The generated reconstructed signal can be used for intra prediction of the next block to be processed within the current picture, and can also be used for inter prediction of the next picture after undergoing filtering as described below.

[0087] The filtering unit (260) can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit (260) can generate a modified restored picture by applying various filtering methods to the restored picture, and can store the modified restored picture in memory (270), specifically in the DPB of memory (170). The various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc. The filtering unit (260) can generate various information regarding filtering and transmit it to the entropy encoding unit (240), as described below in the description of each filtering method. The information regarding filtering can be encoded in the entropy encoding unit (240) and output in the form of a bitstream.

[0088] The modified restored picture transmitted to the memory (270) can be used as a reference picture in the inter-prediction unit (221). Through this, the encoding device (200) can avoid prediction mismatches between the encoding device (200) and the decoding device when inter-prediction is applied, and can also improve encoding efficiency.

[0089] The DPB in memory (270) can store a modified restored picture to be used as a reference picture in the inter prediction unit (221). Memory (270) can store motion information of blocks from which motion information is derived (or encoded) in the current picture and / or motion information of blocks in the picture that have already been restored. The stored motion information can be transmitted to the inter prediction unit (221) to be used as motion information of spatially surrounding blocks or motion information of temporally surrounding blocks. Memory (270) can store restoration samples of restored blocks in the current picture and transmit them to the intra prediction unit (222).

[0090] FIG. 3 is a schematic diagram illustrating a decoding device to which an embodiment according to the present disclosure can be applied.

[0091] As illustrated in FIG. 3, the decoding device (300) may be configured to include an entropy decoder (310), a residual processor (320), a predictor (330), an adder (340), a filter (350), and a memory (360). The predictor (330) may include an inter-predictor (332) and an intra-predictor (331). The residual processor (320) may include a dequantizer (321) and an inverse transformer (321). The aforementioned entropy decoding unit (310), residual processing unit (320), prediction unit (330), addition unit (340), and filtering unit (350) may be configured by a single hardware component (e.g., a decoder chipset or a processor) according to an embodiment. Additionally, the memory (360) may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The hardware component may further include the memory (360) as an internal / external component.

[0092] When a bitstream containing video / image information is input, the decoding device (300) can restore the image by performing a process corresponding to the process performed by the encoding device (200) of FIG. 2. For example, the decoding device (300) can perform decoding using a processing unit applied in the encoding device (200). Thus, the processing unit for decoding may be, for example, a coding unit. The coding unit may be a coding tree unit, or a maximum coding unit may be obtained by dividing it according to a quad tree structure, a binary tree structure, and / or a binary tree structure. And, the restored image signal decoded and output through the decoding device (300) can be played back through a playback device (not shown).

[0093] The decoding device (300) can receive a signal output from the encoding device (200) of FIG. 2 in the form of a bitstream. The received signal can be decoded through an entropy decoding unit (310). For example, the entropy decoding unit (310) can parse the bitstream to derive information necessary for image restoration (or picture restoration) (e.g., video / image information). The video / image information may further include information regarding various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). Additionally, the video / image information may further include general constraint information. The decoding device (300) can decode the picture based further on the information regarding the parameter sets and / or the general constraint information. The signaling / received information and / or syntax elements described below can be obtained from the bitstream by decoding through the decoding procedure. For example, the entropy decoding unit (310) can decode information within the bitstream based on coding methods such as exponential chord coding, CAVLC, or CABAC, and output values ​​of syntax elements required for image restoration and quantized values ​​of transformation coefficients regarding residuals. More specifically, the CABAC entropy decoding method can receive bins corresponding to each syntax element in the bitstream, determine a context model using information on the syntax element to be decoded and decoding information of surrounding and decoding target blocks or information on symbols / bins decoded in the previous step, predict the probability of occurrence of the bin according to the determined context model, and perform arithmetic decoding of the bin to generate a symbol corresponding to the value of each syntax element.At this time, the CABAC entropy decoding method can update the context model using the decoded symbol / bin information for the context model of the next symbol / bin after determining the context model. Among the information decoded in the entropy decoding unit (310), information regarding prediction is provided to the prediction unit (330), and residual values ​​for which entropy decoding was performed in the entropy decoding unit (310), i.e., quantized transformation coefficients and related parameter information, can be input to the residual processing unit (320). The residual processing unit (320) can derive residual signals (residual blocks, residual samples, residual sample array). Additionally, among the information decoded in the entropy decoding unit (310), information regarding filtering can be provided to the filtering unit (350). Meanwhile, a receiving unit (not shown) that receives a signal output from an encoding device may be further configured as an internal / external element of the decoding device (300), or the receiving unit may be a component of the entropy decoding unit (310). Meanwhile, the decoding device according to the present disclosure may be called a video / image / picture decoding device, and the decoding device may be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include the entropy decoding unit (310), and the sample decoder may include at least one of the inverse quantization unit (321), inverse transform unit (322), adder (340), filtering unit (350), memory (360), inter prediction unit (332), and intra prediction unit (331).

[0094] In the inverse quantization unit (321), the quantized transformation coefficients can be inversely quantized to output transformation coefficients. The inverse quantization unit (321) can rearrange the quantized transformation coefficients into a two-dimensional block form. In this case, the rearrangement can be performed based on the coefficient scan order performed in the encoding device (200). The inverse quantization unit (321) can perform inverse quantization on the quantized transformation coefficients using quantization parameters (e.g., quantization step size information) and obtain transformation coefficients.

[0095] In the inverse conversion unit (322), the conversion coefficients can be inversely converted to obtain a residual signal (residual block, residual sample array).

[0096] The prediction unit (330) can generate a prediction signal based on various prediction methods described below. For example, the prediction unit may apply intra prediction or inter prediction for a single block, and may also apply intra prediction and inter prediction simultaneously. This may be called combined inter and intra prediction (CIIP). Additionally, the prediction unit may be based on an intra block copy (IBC) prediction mode or a palette mode for predicting a block. The IBC prediction mode or palette mode may be used for content video / video coding, such as in games, for example, screen content coding (SCC). IBC basically performs prediction within the current picture, but it may be performed similarly to inter prediction in that it derives a reference block within the current picture. That is, IBC may utilize at least one of the inter prediction techniques described in this disclosure. The palette mode can be viewed as an example of intra coding or intra prediction. When the palette mode is applied, information regarding the palette table and palette index can be included in the above video / image information and signaled.

[0097] The intra prediction unit (331) can predict the current block by referring to samples within the current picture. The description of the intra prediction unit (222) may be applied equally to the intra prediction unit (331). The referenced samples may be located in the neighborhood of the current block or located away from it, depending on the prediction mode. In intra prediction, the prediction modes may include a plurality of non-directional modes and a plurality of directional modes. The intra prediction unit (331) may determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring blocks.

[0098] The inter prediction unit (332) can derive a predicted block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. At this time, to reduce the amount of motion information transmitted in the inter prediction mode, motion information can be predicted in blocks, sub-blocks, or samples based on the correlation of motion information between neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include information on the inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter prediction, neighboring blocks may include spatial neighboring blocks existing within the current picture and temporal neighboring blocks existing in the reference picture. For example, the inter prediction unit (332) may construct a motion information candidate list based on the neighboring blocks and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. Inter-prediction can be performed based on various prediction modes (techniques), and information regarding the prediction may include information indicating the mode (technique) of inter-prediction for the current block.

[0099] The adder (340) can generate a restoration signal (restored picture, restored block, restored sample array) by adding the acquired residual signal to the prediction signal (predicted block, predicted sample array) output from the prediction unit (330) (including the inter prediction unit (332) and / or intra prediction unit (331)). In cases where there is no residual for the block to be processed, such as when a skip mode is applied, the predicted block can be used as the restoration block. The description of the adder (250) can be applied equally to the adder (340). The adder (340) may be called a restoration unit or a restoration block generation unit. The generated restoration signal can be used for intra prediction of the next block to be processed within the current picture, and can also be used for inter prediction of the next picture after undergoing filtering as described below.

[0100] Meanwhile, LMCS (luma mapping with chroma scaling) may be applied during the picture decoding process.

[0101] The filtering unit (350) can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit (350) can generate a modified restored picture by applying various filtering methods to the restored picture, and can store the modified restored picture in memory (360), specifically in the DPB of memory (360). The various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc.

[0102] The (modified) restored picture stored in the DPB of the memory (360) can be used as a reference picture in the inter-prediction unit (332). The memory (360) can store motion information of blocks from which motion information within the current picture has been derived (or decoded) and / or motion information of blocks within the picture that have already been restored. The stored motion information can be transmitted to the inter-prediction unit (332) to be used as motion information of spatially surrounding blocks or motion information of temporally surrounding blocks. The memory (360) can store restoration samples of blocks restored within the current picture and transmit them to the intra-prediction unit (331).

[0103] In this specification, the embodiments described in the filtering unit (260), inter prediction unit (221), and intra prediction unit (222) of the encoding device (200) may be applied to the filtering unit (350), inter prediction unit (332), and intra prediction unit (331) of the decoding device (300) in the same or corresponding manner.

[0104] Intra prediction may represent a prediction that generates prediction samples for the current block based on reference samples within the picture to which the current block belongs (hereinafter, the current picture). When intra prediction is applied to the current block, surrounding reference samples to be used for the intra prediction of the current block may be derived. The surrounding reference samples of the current block may include a sample adjacent to the left boundary of the current block of size nWxnH and a total of 2xnH samples adjacent to the bottom-left, a sample adjacent to the top boundary of the current block and a total of 2xnW samples adjacent to the top-right, and one sample adjacent to the top-left of the current block. Alternatively, the surrounding reference samples of the current block may include multiple columns of upper surrounding samples and multiple rows of left surrounding samples. Additionally, the surrounding reference samples of the current block may include a total of nH samples adjacent to the right boundary of the current block of size nWxnH, a total of nW samples adjacent to the bottom boundary of the current block, and one sample adjacent to the bottom-right of the current block.

[0105] However, some of the surrounding reference samples of the current block may not yet be decoded or may not be available. In this case, the decoder may construct the surrounding reference samples to be used for prediction by substituting the unavailable samples with available samples. Alternatively, the surrounding reference samples to be used for prediction may be constructed through the interpolation of available samples.

[0106] When neighboring reference samples are derived, a prediction sample can be derived based on the average or interpolation of the neighboring reference samples of the current block, and (ii) the prediction sample can also be derived based on a reference sample existing in a specific (prediction) direction with respect to the prediction sample among the neighboring reference samples of the current block. Case (i) may be called a non-directional mode or non-angular mode, and case (ii) may be called a directional mode or angular mode. Additionally, the prediction sample may be generated through interpolation between the first neighboring sample and the second neighboring sample located in the opposite direction of the prediction direction of the intra prediction mode of the current block relative to the prediction sample of the current block among the neighboring reference samples. The above case may be called Linear Interpolation Intra Prediction (LIP). Furthermore, chroma prediction samples may be generated based on luminance samples using a linear model. This case may be called LM mode. In addition, a provisional prediction sample of the current block may be derived based on filtered surrounding reference samples, and a prediction sample of the current block may be derived by performing a weighted sum of the provisional prediction sample and at least one reference sample derived according to the intra prediction mode among the existing surrounding reference samples, i.e., unfiltered surrounding reference samples. The above case may be called PDPC (Position dependent intra prediction).In addition, intra-prediction coding can be performed by selecting the reference sample line with the highest prediction accuracy among the surrounding multiple reference sample lines of the current block, deriving a prediction sample using a reference sample located in the prediction direction from that line, and signaling the used reference sample line to a decoding device. The above-described case may be referred to as multi-reference line (MRL) intra prediction or MRL-based intra prediction. Furthermore, the current block may be divided into vertical or horizontal subpartitions to perform intra prediction based on the same intra prediction mode, while deriving and utilizing surrounding reference samples at the subpartition level. That is, in this case, the intra prediction mode for the current block is applied identically to the subpartitions, but intra prediction performance can be improved depending on the circumstances by deriving and utilizing surrounding reference samples at the subpartition level. This prediction method may be referred to as intra sub-partitions (ISP) or ISP-based intra prediction. The above-described intra prediction methods may be referred to as intra prediction types to distinguish them from intra prediction modes. The above-mentioned intra-prediction type may be referred to by various terms, such as intra-prediction technique or additional intra-prediction mode. For example, the above-mentioned intra-prediction type (or additional intra-prediction mode, etc.) may include at least one of the aforementioned LIP, PDPC, MRL, and ISP. A general intra-prediction method excluding specific intra-prediction types such as LIP, PDPC, MRL, and ISP may be referred to as a normal intra-prediction type. The normal intra-prediction type may be generally applied when specific intra-prediction types such as the above are not applied, and prediction may be performed based on the aforementioned intra-prediction mode. Meanwhile, post-processing filtering may be performed on the derived prediction samples as necessary.

[0107] Specifically, the intra-prediction procedure may include an intra-prediction mode / type determination step, a peripheral reference sample derivation step, and an intra-prediction mode / type-based prediction sample derivation step. Additionally, a post-filtering step for the derived prediction samples may be performed as needed.

[0108] Meanwhile, in addition to the intra prediction types described above, affine linear weighted intra prediction (ALWIP) may be used. The ALWIP may also be referred to as LWIP (linear weighted intra prediction) or MIP (matrix weighted intra prediction or matrix-based intra prediction). When the MIP is applied to the current block, prediction samples for the current block may be derived by i) using surrounding reference samples for which an averaging procedure has been performed, ii) performing a matrix-vector-multiplication procedure, and iii) further performing horizontal / vertical interpolation procedures as necessary. The intra prediction modes used for the MIP may be configured differently from the intra prediction modes used in the LIP, PDPC, MRL, ISP intra prediction, or normal intra prediction described above. The intra prediction mode for the MIP may be referred to as the MIP intra prediction mode, MIP prediction mode, or MIP mode. For example, the metrics and offsets used in the matrix vector multiplication may be set differently depending on the intra prediction mode for the MIP. Here, the metrics may be called (MIP) weight metrics, and the offset may be called (MIP) offset vectors or (MIP) bias vectors.

[0109] When intra prediction is applied, the intra prediction mode applied to the current block can be determined using the intra prediction mode of the surrounding blocks. For example, the decoding device may select one of the mpm candidates within the mpm (most probable mode) list derived based on the intra prediction mode of the surrounding blocks of the current block (e.g., left and / or upper surrounding blocks) and additional candidate modes based on the received mpm index, or select one of the remaining intra prediction modes not included in the mpm candidates (and planner mode) based on remaining intra prediction mode information. The mpm list may be configured to include or not include the planner mode as a candidate. For example, if the mpm list includes the planner mode as a candidate, the mpm list may have 6 candidates, and if the mpm list does not include the planner mode as a candidate, the mpm list may have 3 candidates. If the above mpm list does not include planar mode as a candidate, a not planar flag (e.g., intra_luma_not_planar_flag) indicating whether the intra prediction mode of the current block is not planar mode may be signaled. For example, the mpm flag may be signaled first, and the mpm index and the not planar flag may be signaled when the value of the mpm flag is 1. Additionally, the mpm index may be signaled when the value of the not planar flag is 1. Here, the configuration in which the above mpm list does not include planar mode as a candidate is not because the planar mode is not mpm, but rather because planar mode is always considered as mpm, so the not planar flag is signaled first to check whether it is planar mode.

[0110] For example, whether the intra prediction mode applied to the current block is among the mpm candidates (and planner mode) or is in remaining mode can be indicated based on the mpm flag (e.g., intra_luma_mpm_flag). A value of 1 for the mpm flag may indicate that the intra prediction mode for the current block is among the mpm candidates (and planner mode), and a value of 0 for the mpm flag may indicate that the intra prediction mode for the current block is not among the mpm candidates (and planner mode). A value of 0 for the not planar flag (e.g., intra_luma_not_planar_flag) may indicate that the intra prediction mode for the current block is planner mode, and a value of 1 for the not planar flag may indicate that the intra prediction mode for the current block is not planner mode. The above mpm index may be signaled in the form of the mpm_idx or intra_luma_mpm_idx syntax element, and the above remaining intra prediction mode information may be signaled in the form of the rem_intra_luma_pred_mode or intra_luma_mpm_remainder syntax element. For example, the above remaining intra prediction mode information may point to one of the remaining intra prediction modes among all intra prediction modes that are not included in the mpm candidates (and planar modes) by indexing them in order of prediction mode number. The above intra prediction mode may be an intra prediction mode for the lumina component (sample). Hereinafter, the intra prediction mode information includes the above mpm flag (e.g., intra_luma_mpm_flag), the above not planar flag (e.g., intra_luma_not_planar_flag), and the above mpm index (e.g.It may include at least one of the remaining intra-pred mode information (rem_intra_luma_pred_mode or intra_luma_mpm_remainder) (mpm_idx or intra_luma_mpm_idx). In this disclosure, the MPM list may be referred to by various terms such as MPM candidate list, candModeList, etc. When a MIP is applied to the current block, a separate mpm flag (e.g., intra_mip_mpm_flag), mpm index (e.g., intra_mip_mpm_idx), and remaining intra-pred mode information (e.g., intra_mip_mpm_remainder) for the MIP may be signaled, and the not-planar flag is not signaled.

[0111] Generally, when an image is segmented into blocks, the current block to be coded and its neighboring blocks share similar image characteristics. Consequently, the current block and neighboring blocks are highly likely to have identical or similar intra prediction modes. Therefore, the encoder can utilize the intra prediction mode of the neighboring blocks to encode the intra prediction mode of the current block.

[0112] For example, an encoder / decoder can construct a list of most probable modes (MPM) for the current block. The above MPM list may also be referred to as an MPM candidate list. Here, MPM may refer to a mode used to improve coding efficiency by considering the similarity between the current block and surrounding blocks during intra-predictive mode coding. As described above, the MPM list may be constructed to include planner modes, or it may be constructed to exclude planner modes. For example, if the MPM list includes planner modes, the number of candidates in the MPM list may be 6. And, if the MPM list does not include planner modes, the number of candidates in the MPM list may be 5.

[0113] The encoder / decoder can configure an MPM list containing 6 MPMs.

[0114] To construct the MPM list, the types of modes described below may be considered.

[0115] - Default intra modes

[0116] - Neighbor intra modes

[0117] - Intra-modes derived from surrounding inter-modes (IPM modes)

[0118] - DIMD (Decoder-side intra mode derivation) modes

[0119] - Derived intra modes

[0120] For the aforementioned surrounding intra-modes, surrounding blocks, namely the left surrounding block, upper surrounding block, lower-left surrounding block, upper-right surrounding block, and upper-left surrounding block, may be considered. When utilizing surrounding intra-modes as an MPM, the input order may vary depending on the size information of the current block. For example, if the height of a block is greater than or equal to the width of a block, the intra-mode of the upper surrounding block may be considered first, followed by the intra-mode of the left surrounding block.

[0121] Intra-mode information can be obtained via the IPM buffer even when neighboring blocks are coded in Inter-mode rather than Intra-mode. If the location pointed to by the motion vector of an adjacent Inter-block is in Intra-mode, that Intra-mode can be stored in the IPM buffer. The Intra-mode stored in the IPM buffer can be utilized as the MPM mode of the neighboring block.

[0122] MPM candidate configurations can be made from the DIMD described below. If the current block is not in DIMD mode, an intra mode derived from DIMD can be used as an MPM candidate.

[0123] As described above, if the MPM list is configured not to include a planner mode, the planner mode is excluded from the list, and the number of MPM list candidates can be set to 5.

[0124] The encoder / decoder can construct a secondary most probable modes (secondary MPM) list for the current block. The secondary MPM list may also be referred to as a secondary MPM candidate list. Additionally, the secondary MPM list can be constructed by excluding duplicate intra modes from the MPM candidate list. For example, if the MPM list includes mode 50, the secondary MPM list may not include mode 50.

[0125] The encoder / decoder can form M secondary MPM lists, and M can be 16.

[0126] To configure the secondary MPM list, the types of modes described below may be considered.

[0127] - Default intra modes

[0128] - Neighbor intra modes

[0129] - Intra-modes derived from surrounding inter-modes (IPM modes)

[0130] - DIMD (Decoder-side intra mode derivation) modes

[0131] - Derived intra modes

[0132] For the aforementioned surrounding intra-modes, surrounding blocks, namely the left surrounding block, upper surrounding block, lower-left surrounding block, upper-right surrounding block, and upper-left surrounding block, may be considered. When utilizing surrounding intra-modes as a secondary MPM, the input order may vary depending on the size information of the current block. For example, if the height of a block is greater than or equal to the width of a block, the intra-mode of the upper surrounding block may be considered first, followed by the intra-mode of the left surrounding block.

[0133] Intra-mode information can be obtained via the IPM buffer even when neighboring blocks are coded in Inter-mode rather than Intra-mode. If the location pointed to by the motion vector of an adjacent Inter-block is in Intra-mode, that Intra-mode can be stored in the IPM buffer. The Intra-mode stored in the IPM buffer can be utilized as the secondary MPM mode of the neighboring block.

[0134] FIGS. 4 and FIGS. 5 are drawings for illustrating a DIMD mode that can be applied to an embodiment according to the present disclosure.

[0135] The Decoder-side intra mode derivation (DIMD) mode can be used by deriving it in the encoder and decoder without directly transmitting intra prediction mode information. First, the horizontal gradient and vertical gradient are obtained from the second neiboring sample column and row, and a Histogram of Gradients (HoG) can be constructed from them.

[0136] The HoG can be configured as shown in Fig. 4. The HoG can be obtained by applying a Sobel filter using an L-shaped row and column of 3 pixels around the current block. If the boundaries of the block exist in different CTUs, they are not used for texture analysis.

[0137] Subsequently, up to five intra modes with the largest histogram amplitude can be selected, and the final prediction block can be constructed by blending the prediction block predicted using these modes with the planar mode. Weights can be derived from the histogram amplitude. Additionally, a DIMD flag is transmitted on a block-by-block basis to check whether DIMD is used.

[0138] Figure 5 is an example of selecting two intra modes with the largest histogram amplitude and then blending the prediction block predicted using these modes with the planar mode to form a final prediction block.

[0139] For a W×H block, if the size of the upper or left histogram is twice the size of the other, the weights of each of the five derived modes are modified. In this case, the weights depend on the position and can be calculated as follows.

[0140] When the upper histogram is twice the size of the left histogram:

[0141] [Formula 1]

[0142]

[0143] When the left histogram is twice the size of the upper histogram:

[0144] [Equation 2]

[0145]

[0146] wDimd here iis the unmodified uniform weight of the selected DIMD, and Δ i can be predefined and set to 10.

[0147] In the MRL (Multiple-reference line) method, intra prediction can be performed using neighboring samples located on sample lines at a distance of one to three samples from the upper and / or left side of the current block as reference samples. The multiple reference line index (e.g., mrl_idx) indicates which line is used for intra prediction for the current block.

[0148] The Intra Sub-Paritions (ISP) prediction method performs intra-predictive coding by dividing the block currently to be encoded into horizontal or vertical segments. In this process, encoding and decoding are performed on a segmented block basis to generate a reconstructed block, which is then used as a reference block for the next segmented block.

[0149] Matrix-based Intra Prediction (MIP) can be referred to as Affine linear weighted intra prediction (ALWIP) or Matrix weighted intra prediction (MWIP). To predict rectangular block samples of width W and height H, MIP uses as input a line consisting of H reconstructed adjacent boundary samples to the left of the block and a line consisting of W reconstructed adjacent boundary samples above the block. If there are no reconstructed samples, they are generated in the same way as the conventional intra-prediction method.

[0150] Predictive signal generation can be based on the following three steps:

[0151] Among the boundary samples, 4 samples are averaged when W=H=4, and 8 samples are averaged in all other cases to extract samples. (Averaging process)

[0152] Matrix-vector multiplication is performed using the averaged samples as input, and an offset is added. The result is a reduced prediction signal for a set of partially sampled samples from the original block. (Matrix-vector multiplication process)

[0153] The predicted signals for the remaining locations are generated from the predicted signals of a partially sampled set through single-step linear interpolation for each direction. ((Linear) Interpolation Process)

[0154] In TIMD (Template-based intra-mode derivation), for the IPM intra-modes of adjacent intra-blocks and inter-blocks, the Sum of absolute transformed difference (SATD) between the predicted block predicted from the template region and the actual reconstructed sample is calculated, and the mode with the smallest SATD can be selected as the intra-mode of the current block.

[0155] Alternatively, two modes with the smallest SATD can be selected, and the prediction blocks for those two prediction modes can be blended using a weighted sum method to be used as the prediction block for the current block.

[0156] The method for blending two modes can be applied when the following conditions are met.

[0157] costMode2 < 2*costMode1

[0158] If the above conditions are satisfied, two modes are blended to create a prediction block, otherwise only one mode with the minimum SATD value can be selected.

[0159] The weight ratio when blending two prediction blocks is as follows.

[0160] weight1 = costMode2 / (costMode1+ costMode2)

[0161] weight2 = 1 - weight1

[0162] Similar to the geometric partitioning mode (GPM), the spatial geometric partitioning mode (SGPM) consists of one partition mode and two associated intra-prediction modes. If these modes were signaled directly into the bitstream, significant overhead bits would occur. To represent the necessary partition and prediction information more efficiently in the bitstream, a candidate list is used, and only candidate indices are signaled into the bitstream. Each candidate in the list can derive a combination of one partition mode and two intra-prediction modes.

[0163] The TMRL (Template-based multiple reference line intra prediction) mode combines a baseline and a prediction mode to construct a list of candidate combinations using a template matching method. The index for the list of candidate combinations is coded to indicate the baseline and prediction mode used in the current block coding. The general multiple baseline (MRL) in the parts where TIMD is not applied is replaced with the TMRL mode.

[0164] In EIP (Extrapolation filter-based intra prediction) mode, samples within the CU are predicted from the top-left position to the bottom-right position; this is accomplished by applying an extrapolation filter to surrounding reconstructed samples or predicted samples. EIP mode uses a 15-tab filter for prediction as follows:

[0165] [Equation 3]

[0166]

[0167] Here, pred (x,y)is the predicted value at location (x, y) within the CU, c_i is the filter coefficient, and t(x-offsetXi,y-offsetYi) is the reconstructed sample or predicted sample. The predicted sample value is clipped to the range of reference samples rather than the entire range of sample values. The reference sample region used to determine the range is the same as that used when generating the filter coefficients.

[0168] Each embodiment or combination of embodiments of the present disclosure relates to a method for inducing and utilizing an intra mode in an encoder and a decoder.

[0169] In the present disclosure, the intra mode derived from the encoder and decoder may be called VIPM (Virtual intra prediction mode) or DIPM (Dervied intra prediction mode).

[0170] One embodiment relates to a method for inducing and utilizing an intra mode in an encoder and a decoder. Inducing and utilizing an intra mode may be more efficient than utilizing an intra mode in a surrounding pre-recovered block. An intra mode in a surrounding pre-recovered block may not be an optimal intra mode.

[0171] In one embodiment, VIPM (Virtual intra prediction mode) or DIPM (Derived intra prediction mode) can be utilized as a new mode that induces an intra mode in the encoder and decoder.

[0172] VIPM defines separate syntax information in high-level parameters such as VPS, SPS, APS, PPS, Picture Header, or Slice Header, and its usage can be determined based on whether that information is a specific value. For example, if the value is 1, VIPM may be determined to be used.

[0173] FIG. 6 is a diagram illustrating a method for decoding or encoding image information according to an embodiment of the present disclosure. FIG. 7 is a diagram illustrating a template area for decoding or encoding image information according to an embodiment of the present disclosure. FIG. 8 is a diagram illustrating a method for deriving an intra-mode candidate list for decoding or encoding image information according to an embodiment of the present disclosure. FIG. 9 is a diagram illustrating a method for calculating an error value for decoding or encoding image information according to an embodiment of the present disclosure.

[0174] A flowchart of one embodiment may be as shown in FIG. 6. Each step may be changed or omitted.

[0175] A decoding method or an encoding method may include the operations described below. In other words, the operations described below can be applied to both the decoding method and the encoding method.

[0176] The terms or names described below (e.g., names of syntax elements or variables, etc.) are merely examples, and the technical features of the present disclosure are not limited to the terms or names described below. For example, the image information described below may include various information according to the embodiments described in the present disclosure and may include information described in at least one of the tables described above.

[0177] The operations described below do not constitute an essential component of the encoding method according to one embodiment, and at least some of the operations described below may be omitted. Furthermore, the operations described below do not constitute a sufficient component of the encoding method according to one embodiment, and the previously described operations may be added. Moreover, unless they contradict the previously described operations, the operations described below form one embodiment integrally with the previously described operations and do not form a separate embodiment distinct from the previously described operations.

[0178] The decoding method or the encoding method may be executed by a decoding device or an encoding device comprising a memory and a processor electrically connected to the memory, for example, by a processor.

[0179] S610) VIPM flag: This is flag information regarding whether to perform VIPM. Whether to perform VIPM mode can be determined through VIPM flag signaling. Additionally, the VIPM flag can be induced by pre-defining it between the encoder and decoder without signaling. For example, whether to use VIPM mode can be determined based on the block size or shape. For instance, if one of the block's width or height is 8 times greater than the other, VIPM mode can be induced to false. If the current block size (=width * height) is greater than 1024, the VIPM flag can be induced to false. If the current block size (=width * height) is less than or equal to 16, the VIPM flag can be induced to false. If VIPM is induced to false, the VIPM process is not performed. The cases mentioned above are examples where the use of VIPM is determined in relation to block size and shape, and may be determined and used based on other values ​​and / or other conditions.

[0180] Additionally, the use of VIPM mode can be signaled at the CTU / CU / TU / Slice / Tile level. If the VIPM usage flag is signaled as true in a corresponding CTU / Slice / Tile, all blocks within that CTU / Slice / Tile can be encoded and decoded in VIPM mode.

[0181] In addition, as a method that considers the above methods together, if the VIPM usage flag in a CTU / Slice / Tile is signaled as true, the VIPM mode may be determined and used or restricted to the size and shape of predefined blocks within the corresponding CTU / Slice / Tile.

[0182] As another example, after signaling whether to use VIPM mode in the High Level parameter, the application of VIPM can be determined by signaling again in the basic encoding unit such as CTU or CU. In this case, the application of VIPM again in the basic encoding unit such as CTU or CU may be signaled only when the use of VIPM mode in the High Level parameter is enabled. In this case, it can also be used in combination with the conditions regarding the block size and shape.

[0183] S620) Template area determination: This is the step of determining the template area to perform VIPM. The template area can be used to determine the intra mode.

[0184] As shown in Fig. 7,

[0185] The adjacent upper area A of the current block can be used as a template area.

[0186] The adjacent left L area of ​​the current block can be used as a template area.

[0187] The adjacent upper-left AL area of ​​the current block can be used as a template area.

[0188] The upper-right AR area of ​​the current block can be used as a template area.

[0189] The lower-left LB area of ​​the current block can be used as a template area.

[0190] In addition, a partial or whole combination of the above areas can be used as a template area.

[0191] - W, H: Width and height of the current block

[0192] - W A : A is the width of the template area, and can be equal to or smaller than W.

[0193] - W AR : The width of the AR area template area, which can be equal to or smaller than W.

[0194] - H A : This is the height of the AL, A, and AR area template area, and can be, for example, 3.

[0195] - W L : AL, L, LB are the widths of the template area, and can be, for example, 3.

[0196] - H L : L is the height of the template area, and can be equal to or less than H.

[0197] - H LB : The height of the LB area template area, which may be equal to or smaller than H.

[0198] The template area can be defined in the encoder / decoder dictionary without signaling, or a specific template area can be specified with signaling.

[0199] In one embodiment, the size of the template area may vary depending on the size of the current block. A small block may use a small template area, or a large block may use a large template area. For example, in the case of a 4x4 block, only the AL, A, and L areas may be used. For other blocks, all AL, A, AR, L, and LB areas may be used. Since the determination of the template area based on the block size is directly linked to the computational complexity required during the application of VIPM, a relatively small template area may be defined for small blocks to reduce the computational complexity required per pixel.

[0200] In one embodiment, a template area containing unavailable samples extending beyond boundaries such as picture, slice, tile, CTU, and virtual boundary may not be used in its entirety. Alternatively, even if unavailable samples are included within the template area, VIPM may be performed using only available samples as templates.

[0201] In one embodiment, H determines the sizes of regions A, AR, L, LB, and AL by considering the shape of the block (i.e., the ratio of width to height). A and W L The size of can be determined differently. For example, in the case of a 32x4 block, H is used to minimize the computational complexity and memory increase required for the VIPM process. A Use a relatively small value and W L Relatively large values ​​can be used. For example, H A is 3, W LA value of 4 can be used. The above-mentioned case is an example where the size of the area is determined in relation to the block shape, and it may be determined and used with other values ​​and / or other conditions.

[0202] In one embodiment, H determines the sizes of regions A, AR, L, LB, and AL according to the size of the block (the product of the block's width and height). A and W L The size of can be determined differently. For example, if the block size is 256 or greater, H A is 4, W L can use a value of 4. Or, if the block size is 64 or less, H A is 3, W L A value of 3 can be used. The above-mentioned case is an example where the size of the area is determined in relation to the block size, and it may be determined and used with other values ​​and / or other conditions.

[0203] Furthermore, it is obvious that the embodiments can be combined. For example, in the case where the block size is 16, only the AL, A, and L regions are used, and H A is 3, W L The value 3 can be used. Alternatively, if the block size is 256, the AL, A, AR, L, LB areas are used, and H A is 4, W L The value 4 can be used. Alternatively, for a 4x16 block, the AL, A, AR, L, LB areas are used, and H A is 4, W L You can use a value of 3.

[0204] In addition, in one embodiment and FIG. 7, the height of AL, A, and AR is H A It is identical to, and the width of AL, L, and LB is W LAlthough only cases where they are identical are shown, the scope of the present invention is not limited thereto, and AL, A, and AR may have different heights, and AL, L, and LB may also have different widths.

[0205] S630) Intramode Derivation: This is a step for deriving intramodes. Similarity with each intramode directionality can be determined by utilizing the distribution of restored pixel values ​​of the template area determined in the template area determination step, and through this, intramodes can be determined. Differential filtering applied to the pixel values ​​can be performed to determine the similarity between the distribution of pixel values ​​and the intramode directionality.

[0206] For example, the most similar intra-mode is determined based on the gradient of pixel values ​​obtained from the ratio of horizontal to vertical changes, and an amplitude value corresponding to the gradient indicating that intra-mode is assigned. This process can be applied to pixel values ​​within the template area by moving horizontally and vertically, and finally, the accumulated amplitude value for each intra-mode can be obtained.

[0207] - For example, the amplitude value can represent the sum of the magnitude of the horizontal change and the magnitude of the vertical change.

[0208] - The process of mapping the gradients of pixel values ​​to the most similar intra mode can be similar to / identical to DIMD.

[0209] Generally, when applying a differential filter in the process of calculating the accumulated amplitude for each of the above intra-modes, the amplitude value is calculated by moving the pixel values ​​within the template area vertically or horizontally in units of one pixel. However, within a given template area, the directionality is more likely to exist as a group of pixels rather than being determined finely at the pixel level. In the present invention, to reduce unnecessary computational complexity, a differential filter can be applied in units of N pixels in the process of obtaining the amplitude in the horizontal or vertical direction. N can be, for example, 2. Additionally, the pixel unit to which the differential filter is applied can be determined differently depending on the template area (A, AR, L, LB, AL). For example, the differential filter can be applied in units of two pixels in the A, AR, L, and LB regions, and in units of one pixel in the AL region to obtain the amplitude value.

[0210] For example, a Sobel filter can be used as a differential filter to calculate the changes in the horizontal and vertical directions, and other filters may also be applied to calculate the changes in pixel values. Horizontal differential filter M x and vertical differential filter M y For example, it can be as follows.

[0211] [Equation 4]

[0212] ,

[0213] [Formula 5]

[0214] ,

[0215] [Equation 6]

[0216] ,

[0217] In addition to Equations 4, 5, and 6, the size and coefficients of the derivative filter may vary. The derivative filter may be signaled or predefined between the encoder and decoder without signaling. One of the derivative filters, Equations 4, 5, or 6 may be used with specified signaling information. Alternatively, only derivative filter Equation 4, only Equation 5, or only Equation 6 may be used without signaling. Alternatively, different derivative filters may be used for blocks of a specific size. For example, if the block size is greater than or equal to 64, Equation 4 may be used, and for other blocks, Equation 5 or Equation 6 may be used. The cases mentioned above are examples where the derivative filter is determined in relation to the block size and template area, and may be determined and used with other values ​​and / or other conditions.

[0218] As another example, in the case of edges where brightness changes abruptly in the pixel distribution within the template area, it can help to accurately predict the direction. Therefore, before calculating the amplitude for each intra mode, a sharpening filter can be applied to the pixels within the template area to emphasize the edges. A 3x3 sharpening filter F can be, for example, as follows.

[0219] [Equation 7]

[0220]

[0221] As mentioned above, accumulated amplitude values ​​for each intra mode can be obtained by applying a differential filter, and intra modes can be utilized in order of increasing accumulated amplitude values.

[0222] In this case, the induced intramodes can vary depending on which template area is used for prediction. For example, the following intramodes can be induced.

[0223] Mode(1) All , Mode(2) All , Mode(3) All , Mode(4) All , ...

[0224] Mode(1) L , Mode(2) L , Mode(3) L , Mode(4) L , ...

[0225] Mode(1) L+LB , Mode(2) L+LB , Mode(3) L+LB , Mode(4) L+LB , ...

[0226] Mode(1) L+LB+AL , Mode(2) L+LB+AL , Mode(3) L+LB+AL , Mode(4) L+LB+AL , ...

[0227] Mode(1) A , Mode(2) A , Mode(3) A , Mode(4) A , ...

[0228] Mode(1) A+AR , Mode(2) A+AR , Mode(3) A+AR , Mode(4) A+AR , ...

[0229] Mode(1) A+AR+AL , Mode(2) A+AR+AL , Mode(3) A+AR+AL , Mode(4) A+AR+AL , ...

[0230] Mode(1) AL , Mode(2) AL , Mode(3) AL , Mode(4) AL , ...

[0231] - All means all templates, L means L template, L+LB means L and LB templates, L+LB+AL means L, LB, and AL templates, and the rest also mean each template area.

[0232] - It is obvious that the template area determined in the above Template area determination step must not be exceeded. For example, if the determined template area is AL, A, L, only combinations using only three template areas such as AL+A, AL+L, and AL+A+L may be allowed. Additionally, if some samples are not available in a specific template area, it is obvious that those samples cannot be used either.

[0233] - In Mode(k), k represents the intra mode with the k-th largest amplitude.

[0234] The intra-modes induced for a sample to which a sharpening filter has been applied may be as follows.

[0235] F_Mode(1) All , F_Mode(2) All , F_Mode(3) All , F_Mode(4) All , ...

[0236] F_Mode(1) L , F_Mode(2) L , F_Mode(3) L , F_Mode(4) L , ...

[0237] F_Mode(1) L+LB , F_Mode(2) L+LB , F_Mode(3) L+LB , F_Mode(4) L+LB , ...

[0238] F_Mode(1) L+LB+AL , F_Mode(2) L+LB+AL , F_Mode(3) L+LB+AL , F_Mode(4)L+LB+AL , ...

[0239] F_Mode(1) A , F_Mode(2) A , F_Mode(3) A , F_Mode(4) A , ...

[0240] F_Mode(1) A+AR , F_Mode(2) A+AR , F_Mode(3) A+AR , F_Mode(4) A+AR , ...

[0241] F_Mode(1) A+AR+AL , F_Mode(2) A+AR+AL , F_Mode(3) A+AR+AL , F_Mode(4) A+AR+AL , ...

[0242] F_Mode(1) AL , F_Mode(2) AL , F_Mode(3) AL , F_Mode(4) AL , ...

[0243] - All means all templates, L means L template, L+LB means L and LB templates, L+LB+AL means L, LB, and AL templates, and the rest also mean each template area.

[0244] - It is obvious that the template area determined in the above Template area determination step must not be exceeded. For example, if the determined template area is AL, A, L, only combinations using only three template areas such as AL+A, AL+L, and AL+A+L may be allowed. Additionally, if some samples are not available in a specific template area, it is obvious that those samples cannot be used either.

[0245] - F_Mode(k) represents the intra mode derived by applying a differential filter to a sample to which a sharpening filter has been applied, where k represents the k-th largest intra mode amplitude.

[0246] S640) Candidate List Configuration: This is a step of configuring a candidate list using the intramodes derived in the above intramode induction step.

[0247] For example, the intramode candidate list can be constructed as follows. The size of the list can be M (where M is a natural number). The value of M can be predefined between the encoder and decoder.

[0248] Intramode candidates derived using all templates: Mode(k) All

[0249] - For example, if M is 4, Mode(1) All , Mode(2) All , Mode(3) All , Mode(4) All

[0250] - For example, if M is 2, Mode(1) All , Mode(2) All

[0251] or,

[0252] Intramode candidates derived using AL+L+A: Mode(k) A+L+AL

[0253] - For example, if M is 4, Mode(1) A+L+AL , Mode(2) A+L+AL , Mode(3) A+L+AL , Mode(4) A+L+AL

[0254] - For example, if M is 2, Mode(1) A+L+AL , Mode(2) A+L+AL

[0255] or,

[0256] Combinations of intramode candidates derived from different templates

[0257] - For example, if M is 4, Mode(1) All , Mode(2) All , Mode(1) A+L+AL , Mode(2) A+L+AL Or, Mode(1) All , Mode(2) All , Mode(1) A+AR , Mode(1) L+LB Or, or Mode(1) All , Mode(1) A+AR , Mode(1) L+LB , F_Mode(1) All , or F_Mode(1) All , F_Mode(2) All , F_Mode(3) All , F_Mode(4) All

[0258] - For example, if M is 2, Mode(1) All , F_Mode(1) All

[0259] - When inducing an intra mode with different templates, the same intra mode may be induced. To increase encoding efficiency, the intra mode candidate list can be configured so that candidates are non-duplicate, and the candidates can be configured as follows to fill all M non-duplicate candidates. For example, Mode(1) All , Mode(2) All , Mode(1) A+AR+AL , Mode(1) L+LB+AL , Mode(3) All , Mode(4) All

[0260] The examples of constructing the above candidate list are merely a few. There can be a great many derived intra modes, such as the intra modes listed above. Accordingly, there can also be a great many ways to construct the candidate list. This can be defined in advance between the encoder and decoder.

[0261] For example, in the process of constructing a candidate list, if M candidates are not fully filled, the remaining candidates can be constructed using a default intra mode. The default intra mode may consist of multiple modes, and when the default intra mode is used, the missing number of modes can be constructed in a predefined order. It is obvious that the M candidates, including the default intra mode, can be constructed without duplication. For example, the default intra mode can be constructed in order of highest selection rate. Assuming 67 modes are used, the default intra mode may be as follows. This is just one example of the default intra mode, and the size, order of construction, and the constructed intra modes may vary.

[0262] default mode ={ DC mode, Vertical mode, Horizontal mode, Vertical mode - 4, Vertical mode + 4, Horizontal mode - 4, Horizontal mode + 4, Vertical mode - 8, Vertical mode + 8, Horizontal mode - 8, Horizontal mode + 8, Vertical mode - 12, Vertical mode + 12, Horizontal mode - 12, Horizontal mode + 12, 34, 66, 2, Vertical mode - 2, Vertical mode + 2, Horizontal mode -2, Horizontal mode +2}

[0263] In another embodiment, P (integers ≥ M) intramode candidates greater than or equal to M can be rearranged to form a candidate list of size M. This can be divided into three steps as shown in FIG. 8.

[0264] In the first step of configuring P intra-mode candidates, various intra-modes derived according to the template defined in one embodiment may be included. For example, the following intra-modes may be included.

[0265] - Mode(1) All , Mode(2) All , Mode(3) All , Mode(4) All ...

[0266] - Mode(1) L , Mode(2) L , Mode(3) L , Mode(4) L ...

[0267] - Mode(1) L+LB , Mode(2) L+LB , Mode(3) L+LB , Mode(4) L+LB ...

[0268] - Mode(1) L+LB+AL , Mode(2) L+LB+AL , Mode(3) L+LB+AL , Mode(4) L+LB+AL ...

[0269] - Mode(1) A , Mode(2) A , Mode(3) A , Mode(4) A ...

[0270] - Mode(1) A+AR , Mode(2) A+AR , Mode(3) A+AR , Mode(4) A+AR ...

[0271] - Mode(1)A+AR+AL , Mode(2) A+AR+AL , Mode(3) A+AR+AL , Mode(4) A+AR+AL ...

[0272] - Mode(1) AL , Mode(2) AL , Mode(3) AL , Mode(4) AL ...

[0273] - F_Mode(1) All , F_Mode(2) All , F_Mode(3) All , F_Mode(4) All ...

[0274] - F_Mode(1) L , F_Mode(2) L , F_Mode(3) L ,F_Mode(4) L ...

[0275] - F_Mode(1) L+LB , F_Mode(2) L+LB , F_Mode(3) L+LB , F_Mode(4) L+LB ...

[0276] - F_Mode(1) L+LB+AL , F_Mode(2) L+LB+AL , F_Mode(3) L+LB+AL , F_Mode(4) L+LB+AL ...

[0277] - F_Mode(1) A , F_Mode(2) A , F_Mode(3) A , F_Mode(4) A ...

[0278] - F_Mode(1) A+AR , F_Mode(2) A+AR , F_Mode(3) A+AR , F_Mode(4) A+AR ...

[0279] - F_Mode(1) A+AR+AL, F_Mode(2) A+AR+AL , F_Mode(3) A+AR+AL , F_Mode(4) A+AR+AL ...

[0280] - F_Mode(1) AL , F_Mode(2) AL , F_Mode(3) AL , F_Mode(4) AL ...

[0281] For example, P intramode candidates can be composed of the following candidates. This is just one example, and P intramode candidates can be defined by an agreement between the encoder and decoder.

[0282] - Mode(1) All , Mode(2) All , Mode(3) All , Mode(4) All , Mode(1) L+LB+AL , Mode(2) L+LB+AL , Mode(1) A+AR+AL , Mode(2) A+AR+AL , F_Mode(1) All , F_Mode(2) All , F_Mode(3) All , F_Mode(4) All

[0283] P is a value predefined between the encoder and decoder. For example, it can have a size of M*4, or a size of M, or 20. This is an example where it is determined as a multiple of M or a fixed value, and it can be fixed and used as a different value.

[0284] For example, in the process of constructing P intramode candidates, if not all P candidates are constructed, the remaining candidates can be constructed using a default intramode. It is obvious that the P candidates can be constructed without duplication.

[0285] In the P-intra-mode alignment step, P-intra-modes are aligned based on error values. As shown in Fig. 9, the error value between the predicted sample and the original sample predicted in the template area can be calculated, and the error value can be calculated for each intra-mode. Fig. 9 is an example of calculating the error value by generating a predicted sample in the template area in a vertical direction mode.

[0286] To calculate the error value, error calculation methods such as SAD (Sum of difference), SATD (Sum of transformed difference), SSE (Sum of squared error), MR-SAD (Mean-removed sum of difference), MR-SSE (Mean-removed sum of squared error), and MR-SATD (Mean-removed sum of transformed difference) may be used.

[0287] The template area for calculating the error value can be defined differently from that in FIG. 7 above. For example, H A is 1, W L It can be 1.

[0288] The template area for calculating the error value may use all of AL, A, and L, or partially. For example, the error value can be calculated using only A and L. Additionally, a template area containing unavailable samples that extend beyond boundaries such as picture, slice, tile, CTU, and virtual boundary may be omitted.

[0289] In the step of constructing the top M candidate list, the top M intra modes with the smallest error values ​​among the P intra modes sorted in ascending order based on error values ​​can be configured as a candidate list.

[0290] S650) Intra Mode Selection and Prediction Block Generation: This is the step of selecting an intra mode and generating a prediction block using the selected intra mode.

[0291] In one embodiment, one of the intra-mode candidate lists configured in the candidate list configuration step can be signaled. The intra-mode can be specified through the signaling information.

[0292] As another example, the intra mode in the candidate list can be identified without signaling by pre-defining it between the encoder and decoder. For instance, the first candidate in the list can be selected and used without signaling.

[0293] FIG. 10 is a diagram illustrating a method for decoding image information according to one embodiment of the present disclosure.

[0294] The decoding method (S1000) may include operations described below.

[0295] The terms or names described below (e.g., names of syntax elements or variables, etc.) are merely examples, and the technical features of the present disclosure are not limited to the terms or names described below. For example, the image information described below may include various information according to the embodiments described in the present disclosure and may include information described in at least one of the tables described above.

[0296] The operations described below do not constitute an essential component of the decoding method according to one embodiment, and at least some of the operations described below may be omitted. Furthermore, the operations described below do not constitute a sufficient component of the decoding method according to one embodiment, and the previously described operations may be added. Moreover, unless they contradict the previously described operations, the operations described below form an embodiment integrally with the previously described operations and do not form a separate embodiment distinct from the previously described operations.

[0297] The decoding method (S1000) can be executed by a decoding device including a memory and a processor electrically connected to the memory, for example, by a processor.

[0298] The decoding device can acquire image information (S1010).

[0299] For example, the processor of the decoding device can acquire image information including prediction information and residual information.

[0300] Image information may be in various forms. For example, image information may be a syntax element or a syntax structure containing one or more syntax elements. Additionally, image information may be a raw byte sequence payload (RBSP) containing one or more syntax elements or one or more syntax structures. Additionally, image information may be a Network Abstraction Layer (NAL) unit containing one or more RBSPs or a bitstream containing one or more NALs.

[0301] Prediction information may include information related to the prediction of the coding blocks included in each of the coded pictures. For example, prediction information may include information related to prediction modes, such as intra prediction mode, inter prediction mode, and intra block copy (IBC) prediction mode. For example, regarding intra prediction, prediction information may further include intra information related to non-directional mode, directional mode, matrix-weighted intra prediction (MIP), multi-reference line (MRL), or intra sub-partitions (ISP). For example, the prediction information may further include information related to inter modes related to inter prediction, such as skip mode, Regular MERGE mode, MMVD (Merge with Motion Vector Difference) mode, CIIP (Combined Inter and Intra Prediction) mode, TRIANGULAR mode, SbTMVP (Sbblock-based Temporal Motion Vector Prediction) mode, AFFINE MERGE mode, Regular AMVP (Regular Advanced Motion Vector Prediction) mode, SMVD (Symmetric MVD) mode, and AFFINE AMVP mode. For example, the prediction information may further include information related to screen content coding, such as Block-based Delta Pulse Code Modulation (BDPCM) and Palette.

[0302] Residual information may include information related to residual samples of coding blocks included in each of the coded pictures and information related to the processing of residual samples. For example, residual information may include information related to residual samples, information related to quantization parameters (QP), information related to multiple transform kernel selection (MTS), information related to sub-block transform (SBT), information related to low frequency non-separable transform (LFNST), etc.

[0303] The decoding device can derive DIPM (Dervied intra prediction mode) or VIPM (Virtual intra prediction mode) for the current unit within the current picture (S1020).

[0304] For example, the processor of the decoding device can derive an intra prediction for the current unit within the current picture based on prediction information included in the image information. For example, the prediction information may include prediction flag information or prediction index information indicating whether to apply an inter prediction or an intra prediction to the coding unit. Additionally, the processor of the decoding device can derive an intra mode based on the prediction information included in the image information.

[0305] When intra prediction is applied, the intra mode applied to the current block can be determined using the intra mode of the surrounding blocks. For example, the decoding device may select one of the mpm candidates within the mpm (most probable mode) list derived based on the intra mode of the surrounding blocks of the current block (e.g., left and / or upper surrounding blocks) and additional candidate modes based on the received mpm index, or select one of the remaining intra modes not included in the mpm candidates (and planner modes) based on the remaining intra prediction mode information.

[0306] In particular, when DIPM is applied, an intra mode applied to the current block can be derived using the reconstructed samples of the template region. For example, the decoding device can derive intra mode candidates based on the directionality of multiple reconstructed samples of the template region. Additionally, the decoding device can select an intra mode from among the intra mode candidates based on statistics of the directionality of the reconstructed samples or based on the error of the predicted sample of the template region predicted using the intra mode candidates.

[0307] Here, the DIPM can be derived based on the size of the current unit.

[0308] The above DIPM can be derived based on the fact that the product of the width and height of the current unit is less than or equal to a reference value. The reference value may include at least one of 512, 1024, 2048, or 4096.

[0309] The above DIPM can be derived based on the fact that the width of the current unit is less than or equal to the reference width and the height is less than or equal to the reference height. The reference width may include at least one of 16, 32, or 64, and the reference height may include at least one of 16, 32, or 64.

[0310] The above DIPM can be derived based on the fact that the current unit is not the top-left unit of the current picture.

[0311] The above DIPM can be derived based on the fact that MIP (matrix-based intra prediction) is not applied to the above current unit.

[0312] The above DIPM can be derived based on the fact that DIMD (Decoder-side intra mode derivation) is not applied to the above current unit.

[0313] The above DIPM can be derived based on the fact that TIMD (Template based intra mode derivation) is not applied to the above current unit.

[0314] Based on the DIPM derived for the current unit, it may be limited to apply MRL (multi-reference line) based intra-prediction to the current unit.

[0315] Based on the DIPM derived for the current unit, it may be restricted to apply an ISP (intra sub-partitions)-based intra prediction to the current unit.

[0316] Generating a prediction sample for the current unit may include deriving a DIPM candidate list based on the DIPM and generating a prediction sample for the current unit based on the DIPM candidate list. Intra modes included in the DPIM candidate list may not be included in the MPM (most probable mode) list, the Secondary MPM list, and the remaining candidates.

[0317] The method for a decoding device to obtain information regarding DIPM or VIPM in order to derive DIPM or VIPM is as follows.

[0318] The following example shows how to signal VIPM flag information and VIPM index information in VVC intra prediction syntax.

[0319] [Table 1]

[0320]

[0321] [Table 2]

[0322]

[0323] - sps_vipm_enabled_flag: Flag information indicating whether VIPM mode is enabled in HLS

[0324] - intra_vipm_flag: Flag information indicating whether VIPM mode is used

[0325] - intra_vipm_idx: This is the intra-mode index information of the VIPM mode, and through this information, candidate intra-modes within the intra-mode candidate list can be specified.

[0326] - intra_luma_mpm_remainder: Index information pointing to one of the remaining intra prediction modes, excluding MPM and VIPM candidates.

[0327] The index information of VIPM can be omitted from signaling. It can be omitted if the size of the candidate list is 1, or if the intra mode in the candidate list is known by being predefined between the encoder and decoder. For example, the first candidate in the candidate list can be selected and used without signaling.

[0328] [Table 3]

[0329]

[0330] As another example, to apply MRL mode and / or ISP mode information to VIPM, VIPM flag information and VIPM index information can be signaled after signaling MRL and ISP first.

[0331] [Table 4]

[0332]

[0333] As another example, to apply MRL mode and / or ISP mode information to VIPM, the MRL and ISP can be signaled first, followed by the VIPM flag information, and the VIPM index information can be omitted.

[0334] [Table 5]

[0335]

[0336] As another example, the selectivity of MRL and ISP may be lower than that of VIPM. Taking these selectivity rates into account, VIPM flag information and VIPM index information can be signaled prior to signaling MRL mode and ISP mode information.

[0337] [Table 6]

[0338]

[0339] As another example, the selectivity of MRL and ISP may be lower than that of VIPM. Considering these selectivity rates, VIPM flag information can be signaled before signaling MRL mode and ISP mode information, and VIPM index information can be omitted.

[0340] [Table 7]

[0341]

[0342] As another example, VIPM can be included in MPM. In this case, since the Planar mode of MPM is a commonly selected mode, VIPM mode signaling can be performed after the Planar flag as follows.

[0343] [Table 8]

[0344]

[0345] - intra_luma_mpm_idx: When using MPM, this is index information pointing to intra mode information within the mpm list. If intra_vipm_flag is false, the mpm list can be configured with the remaining intra modes, excluding the intra modes from the VIPM candidate list.

[0346] Alternatively, VIPM may be included in MPM, and the index information of VIPM may be omitted.

[0347] [Table 9]

[0348]

[0349] Alternatively, in mpm mode, the intra mode derived from surrounding samples may be more accurate than the planar mode. Considering this, the vipm mode can be signaled prior to the planar mode information signaling.

[0350] [Table 10]

[0351]

[0352] Alternatively, in mpm mode, VIPM mode can be signaled before Planar mode information signaling, and VIPM index information can be omitted.

[0353] [Table 11]

[0354]

[0355] The above VIPM index vipmIdx can be truncated binary coding, unary coding, golomb coding, exponential-golomb coding, truncated rice coding, or truncated unary coding. For example, when the size of the VIPM intramode candidate list is 4, the bins corresponding to each index can be generated as follows using truncated unary coding. The binary-coded bins can be input into a regular coding engine or a bypass coding engine. Here, being input into a regular coding engine may mean that they are encoded into context-coded bins.

[0356] [Table 12]

[0357]

[0358] After VIPM signaling, index information to specify intra modes in MPM, Secondary MPM, and non-MPM (intra prediction modes excluding MPM candidates) can be signaled. In this case, if VIPM is false, it may mean that the intra modes in the VIPM candidate list are not selected. Therefore, when VIPM is false, it is necessary to ensure that the list candidates for MPM, secondary MPM, and non-MPM do not overlap with the VIPM candidate intra modes.

[0359] In one embodiment, the intra-modes of the VIPM candidate list may not be included in the MPM, secondary MPM, and non-MPM lists. That is, the MPM list may be configured so that the intra-modes of the VIPM candidate list do not overlap. Additionally, the secondary list may be configured so that the intra-modes of the VIPM candidate list do not overlap. Additionally, the non-MPM list may be configured so that the intra-modes of the VIPM candidate list do not overlap. Here, in the case of the non-MPM list, the remaining mode information (e.g., intra_luma_mpm_remainder) pointing to one of the remaining intra-prediction modes excluding the VIPM and MPM candidates may be reduced. For example, if there are 67 intra-modes, the mpm size is 6, and the VIPM list size is 4, intra_luma_mpm_remainder may be changed to point to one of 67 - 6 - 4 = 57 intra-prediction modes.

[0360] In one embodiment, the intra mode of the VIPM candidate list may not be included in the MPM, secondary MPM, and non-MPM lists. That is, the MPM list may be configured not to include the intra mode of the VIPM candidate list. Additionally, the secondary list may be configured not to include the intra mode of the VIPM candidate list. Additionally, the non-MPM list may be configured not to include the intra mode of the VIPM candidate list. In the case of the non-MPM list, since it consists of the remaining intra prediction modes excluding the VIPM and MPM candidates, the amount of information (e.g., number of bits) for signaling remaining mode information (e.g., intra_luma_mpm_remainder) pointing to one of these may be reduced. For example, for 67 intra modes, if the mpm size is 6, the secondary mpm size is 16, and the VIPM list size is 4, intra_luma_mpm_remainder may be changed to point to one of 67 - 6 - 16 - 4 = 41 intra prediction modes. Additionally, considering the excluded intra-mode in one embodiment, the excluded intra-mode may not be included in the MPM, secondary MPM, and non-MPM lists. That is, the MPM list may be configured not to include the excluded intra-mode. Additionally, the secondary list may be configured not to include the excluded intra-mode. Additionally, the non-MPM list may be configured not to include the excluded intra-mode. Furthermore, the amount of information (e.g., number of bits) for signaling remaining mode information pointing to one of the remaining intra-prediction modes, excluding the VIPM, MPM candidate, secondary MPM candidate, and excluded intra-mode, may be reduced.For example, for 67 intra modes, with an mpm size of 6, a secondary mpm size of 16, a VIPM list size of 1, and 18 intra modes to be excluded, intra_luma_mpm_remainder can be modified to point to one of 67 - 6 - 16 - 1 - 18 = 26 intra prediction modes. Remaining mode information can be signaled by truncated binary (TB) binarization, fixed-length (FL) binarization, k-th order Exp-Golomb binarization, or truncated rice binarization.

[0361] In one embodiment, Vipm mode may not perform MRL. A condition may be added to apply the MRL execution condition when intra_vipm_flag is false.

[0362] [Table 13]

[0363]

[0364] - CtbSizeY: coding tree block size

[0365] - intra_vipm_flag: Flag information indicating whether VIPM mode is used

[0366] In one embodiment, Vipm mode may not perform ISP. A condition may be added so that the ISP execution condition is applied when intra_vipm_flag is false.

[0367] [Table 14]

[0368]

[0369] - MaxTbSizeY: maximum transform block size

[0370] - intra_vipm_flag: Flag information indicating whether VIPM mode is used

[0371] In one embodiment, the Vipm mode may perform signaling based on the block size. For example, the block size may be derived from the product of the width and height of the block. For example, the Vipm mode may be signaled based on a comparison between the block size and an arbitrary threshold K. Here, the threshold K may be 1024, but this is just one example, and it may be fixed to another fixed value predefined between the encoder and decoder (e.g., K is 512, 2048, or 4096).

[0372] As with the previous examples, signaling for intra_vipm_idx (vipm index information) can be omitted. It can be omitted if the size of the candidate list is 1, or if the intra mode in the candidate list can be specified by pre-defining it between the encoder and decoder. For example, the first candidate in the candidate list can be selected and used without signaling.

[0373] [Table 15]

[0374]

[0375] In one embodiment, the Vipm mode may perform signaling based on block location. For example, signaling may not be performed for a block located at the top-left corner of the picture (x=0, y=0). Signaling of intra_vipm_idx (vipm index information) may be omitted.

[0376] [Table 16]

[0377]

[0378] In one embodiment, when an intra mode that is signaled preferentially over the Vipm mode is used, the vipm flag may not be signaled. For example, if MIP mode, DIMD mode, or TIMD mode is signaled before VIPM, the signaling of VIPM may be omitted as follows.

[0379] [Table 17]

[0380]

[0381] - mipFlag: A flag indicating whether MIP is applied; if true, MIP mode is applied, and if false, it means MIP mode is not applied.

[0382] - dimdFlag: A flag indicating whether DIMD is applied; if true, DIMD mode is applied, and if false, DIMD mode is not applied.

[0383] - timdFlag: A flag indicating whether TIMD is applied; if true, TIMD mode is applied, and if false, TIMD mode is not applied.

[0384] Additionally, signaling may be applied in combination of two or more of the embodiments. For example, signaling may be performed as follows, taking into account the block location and block size. As with the examples above, signaling of intra_vipm_idx (vipm index information) may be omitted.

[0385] [Table 18]

[0386]

[0387] As another example, signaling can be performed as follows, considering the block position, block size, and the intra mode that is signaled preferentially. As with the previous examples, the signaling of intra_vipm_idx (vipm index information) can be omitted.

[0388] [Table 19]

[0389]

[0390] For example, the signaling location of VIPM may be as follows.

[0391] - VIPM can be signaled immediately before the MIP mode flag signaling.

[0392] - VIPM can be signaled immediately after MIP mode flag signaling.

[0393] - VIPM can be signaled immediately before the SGPM mode flag signaling.

[0394] - VIPM can be signaled immediately after SGPM mode flag signaling.

[0395] - VIPM can be signaled immediately before the TIMD mode flag signaling.

[0396] - VIPM can be signaled immediately after TIMD mode flag signaling.

[0397] - VIPM can be signaled immediately before the TMRL mode flag signaling.

[0398] - VIPM can be signaled immediately after TMRL mode flag signaling.

[0399] - VIPM can be signaled immediately before EIP mode flag signaling.

[0400] - VIPM can be signaled immediately after EIP mode flag signaling.

[0401] - VIPM can be signaled immediately before the DIMD mode flag signaling.

[0402] - VIPM can be signaled immediately after the DIMD mode flag signaling.

[0403] For example, if it is decided to apply VIPM before SGPM mode signaling (when the VIPM flag is true), SGPM mode may not be signaled.

[0404] For example, if it is decided to apply VIPM before TMRL mode signaling (when the VIPM flag is true), TMRL mode may not be signaled.

[0405] For example, if it is decided to apply VIPM before MIP mode signaling (when the VIPM flag is true), MIP mode may not be signaled.

[0406] For example, if it is decided to apply VIPM before TIMD mode signaling (when the VIPM flag is true), MIP mode may not be signaled.

[0407] For example, if it is decided to apply VIPM before signaling DIMD mode (when the VIPM flag is true), DIMD mode may not be signaled.

[0408] For example, if it is decided to apply VIPM before EIP mode signaling (when the VIPM flag is true), EIP mode may not be signaled.

[0409] In one embodiment, the VIPM flag may be input into a regular coding engine or a bypass coding engine. When using a regular coding engine, only one context index (ctxidx) may be used. Alternatively, multiple context indices may be used. For example, ctxidx may be 1 if the left or top side of the current coding block is coded as vipm, ctxidx may be 2 if both the left and top sides of the current coding block are coded as vipm, and ctxidx may be 0 if neither the left nor the top side of the current coding block is coded as vipm.

[0410] According to one embodiment, an effective signaling method applicable to various embodiments of the VIPM mode proposed in the present disclosure may be provided.

[0411] The decoding device can generate prediction samples based on DIPM or VIPM (S1030).

[0412] For example, the processor of the decoding device can generate a predicted sample array for the current block. The processor can determine an intra mode based on the DIPM. For example, the processor can derive an intra mode by performing operations S610, S620, S630, S640, and S650 described together with FIG. 6. Specifically, the processor can determine a template region, derive intra mode candidates, construct a candidate list, and select an intra mode.

[0413] The decoding device can generate a predicted sample array based on the selected intra mode.

[0414] The decoding device can derive a reference sample according to the intra prediction mode of the current block among the surrounding reference samples of the current block, and can generate a prediction sample of the current block based on the reference sample.

[0415] Neighboring reference samples to be used for intra prediction of the current block can be derived. The neighboring reference samples of the current block may include a sample adjacent to the left boundary of the current block of size nWxnH and a total of 2xnH samples adjacent to the bottom-left, a sample adjacent to the top boundary of the current block of size nWxnH and a total of 2xnW samples adjacent to the top-right, and 1 sample adjacent to the top-left of the current block. Alternatively, the neighboring reference samples of the current block may include multiple columns of upper neighboring samples and multiple rows of left neighboring samples. Additionally, the neighboring reference samples of the current block may include a total of nH samples adjacent to the right boundary of the current block of size nWxnH, a total of nW samples adjacent to the bottom boundary of the current block, and 1 sample adjacent to the bottom-right of the current block.

[0416] When neighboring reference samples are derived, in non-directional or non-angular mode, a prediction sample may be derived based on the average or interpolation of neighboring reference samples of the current block, and in directional or angular mode, a prediction sample may be derived based on reference samples existing in a specific (prediction) direction for the prediction sample among the neighboring reference samples of the current block.

[0417] The decoding device can derive a recovery sample for the current block (S1040).

[0418] For example, the processor of the decoding device can derive a restored sample array based on a predicted sample array and a residual sample array for the current block.

[0419] The processor can generate a prediction sample array using the method described above.

[0420] The processor can derive a residual sample array based on residual information included in the image information. For example, the processor can derive quantized transform coefficients based on information related to residual samples. The processor can derive transform coefficients by performing inverse quantization on the quantized transform coefficients based on information related to quantization parameters. The processor can derive modified transform coefficients by performing a second transform on the transform coefficients based on information related to low-frequency non-separable transform. The processor can derive a residual sample array by performing a first transform on the modified transform coefficients based on information related to multiple transform kernel selection.

[0421] The processor can derive a restored sample array based on the sum between the predicted sample array and the residual sample array.

[0422] According to one embodiment, the application of VIPM can be effectively signaled. Furthermore, data transmission efficiency can be improved through the application of VIPM.

[0423] FIG. 11 is a drawing illustrating a method for encoding image information according to one embodiment of the present disclosure.

[0424] The encoding method (S1100) may include operations described below.

[0425] The terms or names described below (e.g., names of syntax elements or variables, etc.) are merely examples, and the technical features of the present disclosure are not limited to the terms or names described below. For example, the image information described below may include various information according to the embodiments described in the present disclosure and may include information described in at least one of the tables described above.

[0426] The operations described below do not constitute an essential component of the encoding method according to one embodiment, and at least some of the operations described below may be omitted. Furthermore, the operations described below do not constitute a sufficient component of the encoding method according to one embodiment, and the previously described operations may be added. Moreover, unless they contradict the previously described operations, the operations described below form one embodiment integrally with the previously described operations and do not form a separate embodiment distinct from the previously described operations.

[0427] The encoding method (S1100) can be executed by an encoding device including a memory and a processor electrically connected to the memory, for example, by a processor.

[0428] The encoding device can determine DIPM (Dervied intra prediction mode) or VIPM (Virtual intra prediction mode) for the current unit within the current picture (S1110).

[0429] For example, the processor of the encoding device can compare the Rate Distortion (RD) costs for inter prediction, intra prediction, and intra block copy (IBC) prediction, and determine the intra prediction for the current unit in the current picture based on the RD costs. Additionally, the processor of the encoding device can compare the Rate Distortion (RD) costs for various intra modes in the intra prediction, and determine the intra mode for the current unit in the current picture based on the RD costs.

[0430] Regarding intra prediction, there are various intra modes / types, including non-directional mode, directional mode, matrix-weighted intra prediction (MIP), multi-reference line (MRL), or intra sub-partitions (ISP).

[0431] When intra prediction is applied, the intra mode applied to the current block can be determined using the intra prediction mode of the surrounding blocks. For example, the encoding device can determine the intra mode for the current block based on the RD cost. The encoding device can derive a list of mpm (most probable mode) based on the intra modes of the surrounding blocks of the current block (e.g., left and / or upper surrounding blocks) and additional candidate modes, and generate an mpm index based on one of the mpm candidates, or generate remaining intra prediction mode information based on one of the remaining intra modes not included in the said mpm candidates (and planner modes).

[0432] In particular, when DIPM is applied, the intra mode applied to the current block can be determined using the reconstructed samples of the template region. For example, the encoding device can determine intra mode candidates based on the directionality of multiple reconstructed samples of the template region. Additionally, the encoding device can select an intra mode from among the intra mode candidates based on statistics of the directionality of the reconstructed samples or based on the error of the predicted sample of the template region predicted using the intra mode candidates.

[0433] Here, the DIPM can be determined based on the size of the current unit.

[0434] The above DIPM may be determined based on the fact that the product of the width and height of the current unit is less than or equal to a reference value. The reference value may include at least one of 512, 1024, 2048, or 4096.

[0435] The above DIPM can be determined based on the fact that the width of the current unit is less than or equal to a reference width and the height is less than or equal to a reference height. The reference width may include at least one of 16, 32, or 64, and the reference height may include at least one of 16, 32, or 64.

[0436] The above DIPM can be determined based on the fact that the current unit is not the top-left unit of the current picture.

[0437] The above DIPM can be determined based on the fact that MIP (matrix-based intra prediction) is not applied to the above current unit.

[0438] The above DIPM can be determined based on the fact that DIMD (Decoder-side intra mode derivation) is not applied to the above current unit.

[0439] The above DIPM can be determined based on the fact that TIMD (Template based intra mode derivation) is not applied to the above current unit.

[0440] It may be restricted to apply MRL (multi-reference line) based intra prediction to the current unit based on the determination of the DIPM for the current unit.

[0441] Based on the determination of the DIPM for the current unit, it may be restricted to apply an ISP (intra sub-partitions) based intra prediction to the current unit.

[0442] Generating a prediction sample for the current unit may include deriving a DIPM candidate list based on the DIPM and generating a prediction sample for the current unit based on the DIPM candidate list. Intra modes included in the DPIM candidate list may not be included in the MPM (most probable mode) list, the Secondary MPM list, and the remaining candidates.

[0443] The encoding device can generate prediction samples based on DIPM or VIPM (S1120).

[0444] For example, the processor of the encoding device can generate a predicted sample array for the current block. The processor can determine an intra mode based on DIPM. For example, the processor can derive an intra mode by performing operations S610, S620, S630, S640, and S650 described together with FIG. 6. Specifically, the processor can determine a template region, derive intra mode candidates, construct a candidate list, and select an intra mode.

[0445] The encoding device can generate a predicted sample array based on the selected intra mode.

[0446] The method by which the encoding device generates a prediction sample may be the same as the method by which the decoding device generates a prediction sample in operation S1030 of Fig. 10.

[0447] For this reason, the description of how the encoding device generates a prediction sample is replaced with the description of how the decoding device generates a prediction sample in operation S1030 of FIG. 10.

[0448] The encoding device can derive a residual sample for the current block (S1130).

[0449] For example, the processor of the encoding device can derive a residual sample array based on the original sample array and the predicted sample array for the current block.

[0450] The processor can generate a prediction sample array using the method described above.

[0451] The processor can derive a residual sample array based on the difference between the original sample array and the predicted sample array.

[0452] The processor can derive transformation coefficients by performing a first transformation on a residual sample array and can also generate information related to the selection of multiple transformation kernels. The processor can derive modified transformation coefficients by performing a second transformation on the transformation coefficients and can generate information related to low-frequency inseparable transformations. The processor can derive quantized transformation coefficients by performing quantization on the modified transformation coefficients and can generate information related to quantization parameters. The processor can derive information related to residual samples based on the quantized transformation coefficients. Additionally, the processor can generate residual information based on information related to the selection of multiple transformation kernels, information related to low-frequency inseparable transformations, information related to quantization parameters, and information related to residual samples.

[0453] The encoding device can encode image information including prediction information (S1140).

[0454] For example, the processor of an encoding device can encode image information including prediction information and residual information.

[0455] Image information may be in various forms. For example, image information may be a syntax element or a syntax structure containing one or more syntax elements. Additionally, image information may be a raw byte sequence payload (RBSP) containing one or more syntax elements or one or more syntax structures. Additionally, image information may be a Network Abstraction Layer (NAL) unit containing one or more RBSPs or a bitstream containing one or more NALs.

[0456] The encoding device can encode image information containing information about the DIPM or VIPM for the current block.

[0457] The method by which the encoding device encodes video information containing information regarding DIPM or VIPM may be the same as the method by which the decoding device obtains information regarding DIPM or VIPM in operation S1020 of FIG. 6.

[0458] For this reason, the description of the method for an encoding device to encode video information containing information regarding DIPM or VIPM is replaced with the description of the method for a decoding device to acquire information regarding DIPM or VIPM in operation S1020 of FIG. 6.

[0459] Video information can be encoded in the form of a bitstream by an encoding device.

[0460] The bitstream can be stored on a computer-readable storage medium or transmitted through a transmission unit.

[0461] According to one embodiment, the application of VIPM can be effectively signaled. Furthermore, data transmission efficiency can be improved through the application of VIPM.

[0462] FIG. 12 illustrates an example of the size of a histogram corresponding to each of a plurality of intramodes according to one embodiment of the present disclosure. FIG. 13 illustrates an example of the size of a histogram corresponding to each of a plurality of intramodes according to one embodiment of the present disclosure. FIG. 14 illustrates an example of a method for deriving intramodes according to one embodiment of the present disclosure.

[0463] One embodiment relates to a method for effectively using intra modes. In current video compression systems such as VVC, intra prediction mode methods are defined as modes that perform prediction for the current block undirectedly from surrounding pixels, such as PLANAR and DC, and modes that perform intra-frame prediction with a specific direction. Generally, for each given Prediction Unit (PU) block, all applicable intra modes allowed by the corresponding video compression system can be considered. Furthermore, when the number of applicable intra modes is large (e.g., more than 67 in the case of VVC), it is very important to efficiently encode a specific intra mode. In existing video compression systems, to encode the intra prediction mode for each PU, the intra prediction mode with a high probability of occurrence for the current PU block is predicted based on the intra prediction modes of surrounding PUs and designated as the Most Probable Mode (MPM), thereby encoding the intra mode for the current block with a relatively small number of bits. If the intra mode for the current PU block does not belong to the MPM, the optimal intra mode is included in the remaining modes excluding the MPM, so the remaining modes perform truncated binary coding for 67 MPM candidates (e.g., 6), resulting in a relatively large (e.g., 5 or 6 bits) signaling overhead. In one embodiment, the number of occurrence bits required to encode the intra prediction mode can be reduced by efficiently defining the number of possible intra modes in the current PU using the statistical characteristics of surrounding recovered pixels, rather than targeting all modes allowed by the video compression system for the intra mode candidates applicable to each PU block.

[0464] In one embodiment, if the distribution of restored pixel values ​​does not exhibit a specific directionality, an intra mode mapped to that directionality may be excluded from the candidates. Exclusion from the candidates may correspond to the following cases.

[0465] - May be excluded from MPM candidacy

[0466] - Can be excluded from Secondary MPM candidates. Here, Secondary MPM can be viewed as a separate set of intra-modes defined in the system after MPM. For example, 16 Secondary MPMs can be configured after 6 MPM modes, and the remaining modes that do not belong to MPM or Secondary MPM can be included in the remaining modes.

[0467] - Can be excluded from Remaining Mode candidates.

[0468] - This is just one example and may be excluded from other intra prediction technologies utilizing intra mode.

[0469] In one embodiment, differential filtering can be performed to determine whether the distribution of restored pixel values ​​is associated with a specific direction.

[0470] For example, the most similar intra-mode is determined based on the gradient of pixel values ​​obtained from the ratio of horizontal to vertical changes, and an amplitude value corresponding to the gradient indicating that intra-mode is assigned. This process can be applied to pixel values ​​within the template area by moving horizontally and vertically, and finally, the accumulated amplitude value for each intra-mode can be obtained.

[0471] - For example, the amplitude value can be derived by considering both the magnitude of the horizontal change and the magnitude of the vertical change. (e.g., the sum of the magnitudes of the horizontal and vertical changes)

[0472] - The process of mapping the gradients of pixel values ​​to the most similar intra mode may be similar to / identical to the DIMD described earlier.

[0473] - The Template area can be similar to or identical to the DIMD described earlier.

[0474] The accumulated amplitude value for each intra mode can be obtained in the manner described above. An example of this may be as shown in Fig. 12.

[0475] In Figure 12, it can be seen that there are no accumulated amplitude values ​​in modes 34 and 35. These intra-modes are determined to be modes that are less related to the current block, and the corresponding intra-modes can be excluded.

[0476] As another example, intra modes that do not exceed a threshold can be determined to be modes that are less relevant to the current block, and such intra modes can be excluded. For example, modes 2, 34, and 66 in Fig. 13 can be excluded.

[0477] In one embodiment, an intra mode with an amplitude of 0 may be excluded.

[0478] In one embodiment, intra mode that does not exceed a threshold may be excluded. The threshold may be a fixed value. For example, it may be 10. The threshold may be determined as a different value by pre-defining between the encoder and decoder, or it may be transmitted through explicit signaling in the form of the threshold itself or a flag or index indicating the threshold.

[0479] In excluding the above intra mode, the effect obtained when encoding the intra mode may be as follows. Here, the number of intra modes allowed by the video coding system is defined as T, and if VVC allows 67 modes, the value of T becomes 67. If more modes than VVC are used, T can be a number greater than 67.

[0480] 1) MPM: Intra-modes that are excluded can be configured not to be included in the MPM list. This allows the MPM list to be composed of more predictable intra-modes.

[0481] 2) Secondary MPM: You can configure the Secondary MPM so that the excluded intra modes are not included in the Secondary MPM list.

[0482] 3) Remaining Mode: Excluded intra-modes can be configured not to be included in the remaining mode. Therefore, the number of remaining modes can be reduced by (T - number of MPM candidates - number of excluded intra-modes). By reducing the number, encoder complexity and signaling overhead can be reduced. If a secondary MPM exists, the number of remaining modes can be reduced by (T - number of MPM candidates - number of secondary MPM candidates - number of excluded intra-modes). That is, it can be configured not to include excluded intra-modes for both the MPM and the remaining mode. Similarly, if a secondary MPM exists, it can be configured not to include excluded intra-modes for the MPM, the secondary MPM, and the remaining mode.

[0483] In one embodiment, there may be N intra modes to be excluded. N is a value predefined between the encoder and decoder. For example, N may be 4. If the number of intra modes having an Amplitude of 0 exceeds N, intra modes may be excluded in a predefined order between the encoder and decoder until N intra modes are filled. For example, the order of intra modes may be 2, 66, 8, 28, 40, 60, ...

[0484] If the number of intra modes with an amplitude of 0 is less than N, intra modes to be excluded for the insufficient number can be defined in a predefined order between the encoder and decoder. For example, the order of intra modes can be 2, 66, 8, 28, 40, 60, ... or the intra modes for the insufficient number can be determined based on the amplitude value in order of decreasing value.

[0485] In one embodiment, the number of intra-modes N excluded may be determined to a different value depending on the block size. For example, if the product of the width and height of the block is less than 64, N may be 4, and if it is greater than or equal to 64, N may be 8. This is one example of how the number of intra-modes excluded is determined based on the block size, and other conditions and values ​​of N can be determined by pre-defining between the encoder and decoder.

[0486] In one embodiment, the number of intra-modes excluded was described as intra-modes with an amplitude of 0, but it can be equally applied to intra-modes with an amplitude smaller than the threshold.

[0487] One embodiment may be performed in an encoder / decoder as a process as illustrated in FIG. 14. First, an intra-mode to be excluded may be determined. This step includes both the previously mentioned number of intra-modes to be excluded and the determination of the intra-mode to be excluded. Regarding the number of intra-modes to be excluded, both the number of intra-modes with an amplitude of 0 and the number of intra-modes with an amplitude smaller than a threshold may be applied. Similarly, regarding the intra-modes to be excluded, both the intra-modes with an amplitude of 0 and the intra-modes with an amplitude smaller than a threshold may be applied. This may be performed by pre-defining between the encoder / decoder.

[0488] Subsequently, a step for configuring the candidate list is performed. This step involves configuring the MPM and remaining mode candidates, and excluded intra modes can be configured so that they are not included in the MPM and remaining modes.

[0489] FIG. 15 is a drawing illustrating an exemplary content streaming system to which an embodiment according to the present disclosure can be applied.

[0490] As illustrated in FIG. 15, a content streaming system to which an embodiment of the present disclosure is applied may largely include an encoding server, a streaming server, a web server, a media storage, a user device, and a multimedia input device.

[0491] The above encoding server compresses content input from multimedia input devices, such as smartphones, cameras, and camcorders, into digital data to generate a bitstream and transmits it to the streaming server. As another example, if multimedia input devices, such as smartphones, cameras, and camcorders, generate the bitstream directly, the encoding server may be omitted.

[0492] The bitstream may be generated by a video encoding method and / or encoding device to which an embodiment of the present disclosure is applied, and the streaming server may temporarily store the bitstream during the process of transmitting or receiving the bitstream.

[0493] The streaming server transmits multimedia data to a user device based on a user request through a web server, and the web server can act as a medium to inform the user of available services. When a user requests a desired service from the web server, the web server transmits it to the streaming server, and the streaming server can transmit multimedia data to the user. At this time, the content streaming system may include a separate control server, and in this case, the control server can perform the role of controlling commands and responses between each device within the content streaming system.

[0494] The streaming server can receive content from a media storage and / or an encoding server. For example, when receiving content from the encoding server, the content can be received in real time. In this case, to provide a seamless streaming service, the streaming server can store the bitstream for a certain period of time.

[0495] Examples of the above user devices may include mobile phones, smartphones, laptop computers, digital broadcasting terminals, PDAs (personal digital assistants), PMPs (portable multimedia players), navigation systems, slate PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, HMDs (head-mounted displays)), digital TVs, desktop computers, digital signage, etc.

[0496] Each server within the above-mentioned content streaming system can be operated as a distributed server, and in this case, data received from each server can be processed in a distributed manner.

[0497] The scope of the present disclosure includes software or machine-executable instructions (e.g., operating system, application, firmware, program, etc.) that enable an operation according to a method of various embodiments to be executed on a device or computer, and a non-transitory computer-readable medium on which such software or instructions, etc. are stored and executable on a device or computer.

[0498] An embodiment according to the present disclosure can be used to encode / decode images.

Claims

1. Regarding the method of decoding video information, Acquire the above image information including prediction information; Based on the above prediction information, derive the DIPM (derived intra prediction mode) for the current unit; It includes generating a prediction sample for the current unit based on the DIPM, and The above DIPM is a method derived based on the size of the current unit.

2. In Paragraph 1, The above DIPM is a method derived based on the fact that the product of the width and height of the current unit is less than or equal to a reference value.

3. In Paragraph 2, A method in which the above reference value includes at least one of 512, 1024, 2048, or 4096.

4. In Paragraph 1, The above DIPM is a method derived based on the fact that the width of the current unit is less than or equal to the reference width and the height is less than or equal to the reference height.

5. In Paragraph 4, A method in which the reference width comprises at least one of 16, 32, or 64, and the reference height comprises at least one of 16, 32, or 64.

6. In Paragraph 1, The above DIPM is a method derived based on the fact that the current unit is not the top-left unit of the current picture.

7. In Paragraph 1, The above DIPM is a method derived based on the fact that MIP (matrix-based intra prediction) is not applied to the above current unit.

8. In Paragraph 1, The above DIPM is a method derived based on the fact that DIMD (Decoder-side intra mode derivation) is not applied to the above current unit.

9. In Paragraph 1, The above DIPM is a method derived based on the fact that TIMD (Template based intra mode derivation) is not applied to the above current unit.

10. In Paragraph 1, A method in which applying MRL (multi-reference line) based intra-prediction to the current unit is restricted based on the DIPM derived for the current unit.

11. In Paragraph 1, A method in which applying an ISP (intra sub-partitions) based intra prediction to the current unit is restricted based on the DIPM derived for the current unit.

12. In paragraph 1, generating a prediction sample for the current unit is, Based on the above DIPM, derive a list of DIPM candidates; A method comprising generating a prediction sample for the current unit based on the above DIPM candidate list.

13. In Paragraph 12, A method in which an intra mode included in the above DPIM candidate list is not included in the MPM (most probable mode) list, the Secondary MPM list, and the remaining candidates.

14. Regarding the method of encoding video information, Determining intra-prediction for the current unit; Determining the DIPM (derived intra prediction mode) based on the above intra prediction; Generate a prediction sample for the current unit based on the above DIPM; It includes encoding image information containing prediction information related to the above DIPM, and The above DIPM is a method determined based on the size of the current unit.

15. Regarding methods concerning bitstreams, Generate a bitstream; It includes transmitting data including the above bitstream, Generating the above bitstream is, Determining intra-prediction for the current unit; Determining the DIPM (derived intra prediction mode) based on the above intra prediction; Generate a prediction sample for the current unit based on the above DIPM; It includes encoding image information containing prediction information related to the above DIPM, and The above DIPM is a method determined based on the size of the current unit.

Images