Image encoding / decoding method and method for transmitting bit stream

Abstract

The invention relates to an image encoding / decoding method and a method for transmitting a bitstream. An image encoding / decoding method and apparatus are provided. An image decoding method performed by an image decoding apparatus may include: identifying a prediction mode of a current block; determining a candidate intra prediction mode of the current block based on prediction modes of neighboring blocks located around the current block, on the basis that the intra prediction mode of the current block is an intra prediction mode; generating a candidate intra prediction mode list of the current block based on the candidate intra prediction modes; and determining an intra prediction mode of the current block based on the candidate intra prediction mode list. In this case, the prediction mode based on the neighboring block is an MIP mode, the candidate intra prediction mode may be determined as a predetermined intra prediction mode.

Description

[0001] This application is a divisional application of the patent application with the invention title "Image Encoding / Decoding Method and Method of Transmitting a Bitstream", application number 202080050958.5 (International Application Number PCT / KR2020 / 007727), with an international filing date of June 15, 2020 and a filing date of January 23, 2022 in the present application. Technical Field

[0002] The present disclosure relates to an image encoding / decoding method and apparatus, and more particularly, to an image encoding / decoding method and apparatus for signaling an intra prediction mode and a method of transmitting a bitstream generated by the image encoding method / apparatus of the present disclosure. Background Art

[0003] Recently, the demand for high-resolution and high-quality images, such as high-definition (HD) images and ultra-high-definition (UHD) images, is increasing in various fields. As the resolution and quality of image data increase, the amount of information or bits to be transmitted relatively increases compared to existing image data. The increase in the amount of information or bits to be transmitted results in an increase in transmission cost and storage cost.

[0004] Therefore, an efficient image compression technique is needed to effectively transmit, store, and reproduce information on high-resolution and high-quality images. Summary of the Invention

[0005] Technical Problem

[0006] An object of the present disclosure is to provide an image encoding / decoding method and apparatus having improved encoding / decoding efficiency.

[0007] Another object of the present disclosure is to provide an image encoding / decoding method and apparatus capable of reducing prediction complexity by mapping an intra prediction mode of neighboring blocks to a predetermined prediction mode.

[0008] Another object of the present disclosure is to provide a method of transmitting a bitstream generated by an image encoding method or apparatus according to the present disclosure.

[0009] Another object of the present disclosure is to provide a recording medium storing a bitstream generated by an image encoding method or apparatus according to the present disclosure.

[0010] Another object of the present disclosure is to provide a recording medium storing a bitstream received, decoded, and used for reconstructing an image by an image decoding apparatus according to the present disclosure.

[0011] The technical problems solved by the present disclosure are not limited to the above technical problems, and other technical problems not described herein will become apparent to those skilled in the art according to the following description.

[0012] Technical solution

[0013] An image decoding method performed by an image decoding device according to one aspect of the present disclosure may include: identifying a prediction mode of a current block; based on the prediction mode of the current block being an intra prediction mode, identifying whether the intra prediction mode of the current block is a MIP (matrix-based intra prediction) mode; based on the intra prediction mode of the current block not being a MIP mode, determining a candidate intra prediction mode of the current block based on prediction modes of neighboring blocks located around the current block; generating a candidate intra prediction mode list for the current block based on the candidate intra prediction mode; and determining the intra prediction mode of the current block based on the candidate intra prediction mode list. Based on the prediction mode of the neighboring block being a MIP mode, the candidate intra prediction mode may be determined as a predetermined intra prediction mode. The predetermined intra prediction mode may be any one of a planar mode, a DC mode, a horizontal mode, and a vertical mode.

[0014] It may be determined whether the prediction mode of the neighboring block is a MIP mode based on a MIP mode indicator of the neighboring block, and the MIP mode indicator may be obtained from a bitstream.

[0015] The candidate intra prediction mode list may be generated based on a first candidate intra prediction mode and a second candidate intra prediction mode, the first candidate intra prediction mode may be determined based on the prediction mode of a first neighboring block located around the current block, and the second candidate intra prediction mode may be determined based on the prediction mode of a second neighboring block located around the current block.

[0016] Based on the first candidate intra prediction mode and the second candidate intra prediction mode being the same and the first candidate intra prediction mode being an intra prediction mode having a value greater than a prediction mode value specifying a DC mode, the candidate intra prediction mode list may be determined to include the value of the first candidate intra prediction mode.

[0017] Based on both the prediction mode of the first neighboring block and the prediction mode of the second neighboring block being MIP modes, the candidate intra prediction mode list may be determined to have a predetermined candidate intra prediction mode.

[0018] The predetermined candidate intra prediction mode may include at least one of a DC mode or a vertical mode.

[0019] Based on the prediction mode of the first neighboring block being a MIP mode, the first candidate intra prediction mode and the second candidate intra prediction mode being different from each other, and the second candidate intra prediction mode being an intra prediction mode having a value greater than a prediction mode value specifying a DC mode, the candidate intra prediction mode list may be determined to include the second candidate intra prediction mode.

[0020] The intra prediction mode of the current block can be determined based on a candidate intra prediction mode list by determining any one of the candidate intra prediction modes included in the candidate intra prediction mode list as the intra prediction mode of the current block based on an intra prediction mode indicator obtained from a bitstream.

[0021] An image decoding method may include determining a reference mode for determining an intra prediction mode of a chrominance block corresponding to a current block and determining the intra prediction mode of the chrominance block based on the reference mode. In this case, the current block may be a luminance block, and based on the intra prediction mode of the current block being the MIP mode, the reference mode may be determined as the planar mode.

[0022] The intra prediction mode of the chrominance block may be determined as the reference mode.

[0023] Based on the intra prediction mode of the current block not being the MIP mode, the reference mode may be determined based on the intra prediction mode of the current block.

[0024] An image decoding apparatus according to an aspect of the present disclosure may include a memory and at least one processor. The at least one processor may identify a prediction mode of a current block, based on the prediction mode of the current block being an intra prediction mode, determine candidate intra prediction modes of the current block based on prediction modes of neighboring blocks located around the current block, generate a candidate intra prediction mode list of the current block based on the candidate intra prediction modes, and determine the intra prediction mode of the current block based on the candidate intra prediction mode list. Based on the prediction mode of the neighboring block being the MIP mode, the candidate intra prediction mode may be determined as a predetermined intra prediction mode.

[0025] An image encoding method performed by an image encoding apparatus according to an aspect of the present disclosure may include: identifying a prediction mode of a current block; based on the prediction mode of the current block being an intra prediction mode, determining candidate intra prediction modes based on prediction modes of neighboring blocks located around the current block; generating a candidate intra prediction mode list of the current block based on the candidate intra prediction modes; and encoding an intra prediction mode indicator specifying the intra prediction mode of the current block based on the candidate intra prediction mode list. Based on the prediction mode of the neighboring block being the MIP mode, the candidate intra prediction mode may be determined as a predetermined intra prediction mode.

[0026] The predetermined intra prediction mode may be any one of a planar mode, a DC mode, a horizontal mode, and a vertical mode.

[0027] A candidate intra prediction mode list may be generated based on a first candidate intra prediction mode and a second candidate intra prediction mode. The first candidate intra prediction mode may be determined based on the prediction mode of a first neighboring block located around the current block, and the second candidate intra prediction mode may be determined based on the prediction mode of a second neighboring block located around the current block. Based on both the prediction mode of the first neighboring block and the prediction mode of the second neighboring block being MIP modes, the candidate intra prediction mode list may be determined to have a predetermined candidate intra prediction mode.

[0028] The predetermined candidate intra prediction mode may include at least one of a DC mode or a vertical mode.

[0029] In addition, a transmission method according to another aspect of the present disclosure may transmit a bitstream generated by the image encoding apparatus or the image encoding method of the present disclosure.

[0030] In addition, a computer-readable recording medium according to another aspect of the present disclosure may store a bitstream generated by the image encoding apparatus or the image encoding method of the present disclosure.

[0031] The features briefly summarized above regarding the present disclosure are merely exemplary aspects of the following detailed description of the present disclosure and do not limit the scope of the present disclosure.

[0032] Advantageous Effects

[0033] According to the present disclosure, an image encoding / decoding method and apparatus having improved encoding / decoding efficiency may be provided.

[0034] In addition, according to the present disclosure, an image encoding / decoding method and apparatus capable of reducing prediction complexity by mapping the intra prediction mode of neighboring blocks to a predetermined prediction mode may be provided.

[0035] In addition, according to the present disclosure, a method of transmitting a bitstream generated by the image encoding method or apparatus according to the present disclosure may be provided.

[0036] In addition, according to the present disclosure, a recording medium storing a bitstream generated by the image encoding method or apparatus according to the present disclosure may be provided.

[0037] In addition, according to the present disclosure, a recording medium storing a bitstream received, decoded, and used for reconstructing an image by the image decoding apparatus according to the present disclosure may be provided.

[0038] Those skilled in the art will appreciate that the effects achievable through the present disclosure are not limited to what has been specifically described above, and other advantages of the present disclosure will be more clearly understood from the detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1is a view schematically showing a video encoding / decoding system to which embodiments of the present disclosure are applicable.

[0040] Figure 2 is a view schematically showing an image encoding device to which embodiments of the present disclosure are applicable.

[0041] Figure 3 is a view schematically showing an image decoding device to which embodiments of the present disclosure are applicable.

[0042] Figure 4 is a view illustrating a slice and tile structure according to an embodiment.

[0043] Figures 5 to 6 is a view illustrating a directional intra prediction mode according to an embodiment.

[0044] Figure 7 is a view illustrating a mapping table for mapping an MIP mode according to an embodiment to a normal intra prediction mode.

[0045] Figure 8 and Figure 9 is a reference view illustrating an MIP mode according to an embodiment.

[0046] Figures 10 to 12 is a view illustrating the syntax of a coding unit according to an embodiment.

[0047] Figure 13 is a view illustrating a mapping table for mapping a normal intra prediction mode according to an embodiment to an MIP mode.

[0048] Figure 14 is a view illustrating an MPM list configured with a predetermined MIP intra prediction mode according to an embodiment.

[0049] Figure 15 is a flowchart illustrating a method for encoding an intra prediction mode using an MPM list according to an embodiment.

[0050] Figure 16 is a flowchart illustrating a method for performing decoding by a decoding device using an MPM list according to an embodiment.

[0051] Figure 17 is a flowchart illustrating a method for generating an MPM list using a mapping method according to an embodiment.

[0052] Figure 18 is a flowchart illustrating a method for generating an MPM list using a mapping method according to another embodiment.

[0053] Figure 19It is a flowchart showing a method for generating an MPM list using a simplified mapping method according to an embodiment.

[0054] Figure 20 It is a flowchart showing a method for generating an MPM list by an encoding device using a simplified mapping method according to an embodiment.

[0055] Figure 21 It is a flowchart showing a method for generating an MPM list by a decoding device using a simplified mapping method according to an embodiment.

[0056] Figure 22 It is a view showing compilation performance data of a simplified mapping method using Figure 19

[0057] Figure 23 It is a flowchart showing a method for generating an MPM list using a simplified mapping method according to another embodiment.

[0058] Figure 24 It is a view showing compilation performance data of a simplified mapping method using Figure 23

[0059] Figure 25 It is a flowchart showing a method for generating an MPM list using a mapping method according to another embodiment.

[0060] Figure 26 It is a flowchart showing compilation performance data of a simplified mapping method according to another embodiment.

[0061] Figure 27 It is a view showing a content stream transmission system to which an embodiment of the present disclosure is applicable. Detailed Description

[0062] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. However, the present disclosure can be implemented in various different forms and is not limited to the embodiments described here.

[0063] When describing the present disclosure, if it is determined that the detailed description of related known functions or configurations makes the scope of the present disclosure unnecessarily ambiguous, the detailed description thereof will be omitted. In the drawings, parts irrelevant to the description of the present disclosure are omitted, and similar reference numerals are assigned to similar parts.

[0064] In the present disclosure, when a component is "connected", "coupled", or "linked" to another component, it may include not only a direct connection relationship but also an indirect connection relationship with an intermediate component present. In addition, when a component "includes" or "has" other components, unless otherwise stated, it means that other components may also be included, rather than excluding other components.

[0065] In the present disclosure, terms such as first and second are only used for the purpose of distinguishing one component from other components, and do not limit the order or importance of the components, unless otherwise specified. Accordingly, within the scope of the present disclosure, the first component in one embodiment may be referred to as the second component in another embodiment, and similarly, the second component in one embodiment may be referred to as the first component in another embodiment.

[0066] In the present disclosure, the components that are mutually distinguished are intended to clearly describe each feature, and do not mean that the components must be separated. That is, multiple components can be implemented by integrating them into one hardware or software unit, or one component can be distributed and implemented in multiple hardware or software units. Therefore, even without special description, the embodiments in which these components are integrated or distributed are included within the scope of the present disclosure.

[0067] In the present disclosure, the components described in each embodiment are not necessarily essential components, and some components can be optional components. Therefore, the embodiments composed of a subset of the components described in the embodiments are also included within the scope of the present disclosure. In addition, the embodiments that include other components in addition to the components described in various embodiments are included within the scope of the present disclosure.

[0068] The present disclosure relates to the encoding and decoding of images. Unless redefined in the present disclosure, the terms used in the present disclosure may have the general meanings commonly used in the technical field to which the present disclosure belongs.

[0069] In the present disclosure, a "picture" generally refers to a unit representing an image within a specific time period, while a slice / tile is a coding unit that forms part of a picture, and a picture can be composed of one or more slices / tiles. In addition, a slice / tile can include one or more coding tree units (CTUs).

[0070] In the present disclosure, a "pixel" or "pel" can mean the smallest individual that constitutes a picture (or image). In addition, "sample" can be used as a term corresponding to a pixel. A sample generally can represent a pixel or the value of a pixel, or can only represent the pixel / pixel value of the luminance component or only represent the pixel / pixel value of the chrominance component.

[0071] In the present disclosure, a "unit" can represent a basic unit of image processing. The unit can include at least one of a specific area of a picture and information related to the area. In some cases, the unit can be used interchangeably with terms such as "sample array", "block", or "area". Generally, an M×N block can include a set (or array) of samples (or sample arrays) or transform coefficients with M columns and N rows.

[0072] In the present disclosure, "current block" may refer to one of "current compilation block", "current compilation unit", "compilation target block", "decoding target block", or "processing target block". When performing prediction, "current block" may refer to "current prediction block" or "prediction target block". When performing transformation (inverse transformation) / quantization (dequantization), "current block" may refer to "current transformation block" or "transformation target block". When performing filtering, "current block" may refer to "filtering target block".

[0073] In addition, in the present disclosure, unless explicitly stated as a chrominance block, "current block" may refer to the luminance block of the "current block". The chrominance block of the "current block" can be expressed by including an explicit description of a chrominance block such as "chrominance block" or "current chrominance block".

[0074] In the present disclosure, the slashes " / " or "," can be interpreted as indicating "and / or". For example, "A / B" and "A,B" can mean "A and / or B". In addition, "A / B / C" and "A,B,C" can mean "at least one of A, B, and / or C".

[0075] In the present disclosure, the term "or" should be interpreted to indicate "and / or". For example, the expression "A or B" can include 1) only "A", 2) only "B", or 3) both "A and B". In other words, in the present disclosure, "or" should be interpreted to indicate "additionally or alternatively".

[0076] Overview of Video Compilation System

[0077] Figure 1 is a view schematically showing a video compilation system according to the present disclosure.

[0078] The video compilation system according to an embodiment may include an encoding device 10 and a decoding device 20. The encoding device 10 may deliver the encoded video and / or image information or data in the form of a file or a stream to the decoding device 20 via a digital storage medium or a network.

[0079] The encoding device 10 according to an embodiment may include a video source generator 11, a compilation unit 12, and a transmitter 13. The decoding device 20 according to an embodiment may include a receiver 21, a decoding unit 22, and a renderer 23. The compilation unit 12 may be referred to as a video / image compilation unit, and the decoding unit 22 may be referred to as a video / image decoding unit. The transmitter 13 may be included in the compilation unit 12. The receiver 21 may be included in the decoding unit 22. The renderer 23 may include a display, and the display may be configured as a separate device or an external component.

[0080] The video source generator 11 can obtain video / images through processes such as capturing, synthesizing, or generating video / images. The video source generator 11 can include a video / image capturing device and / or a video / image generating device. The video / image capturing device can include, for example, one or more cameras, a video / image archive including previously captured video / images, etc. The video / image generating device can include, for example, a computer, a tablet computer, and a smart phone, and can (electronically) generate video / images. For example, virtual video / images can be generated by a computer or the like. In this case, the video / image capturing process can be replaced by a process of generating relevant data.

[0081] The compiling unit 12 can encode the input video / images. For compression and compiling efficiency, the compiling unit 12 can perform a series of processes such as prediction, transformation, and quantization. The compiling unit 12 can output the encoded data (encoded video / image information) in the form of a bit stream.

[0082] The transmitter 13 can transmit the encoded video / image information or the data output in the form of a bit stream to the receiver 21 of the decoding device 20 in the form of a file or a stream through a digital storage medium or a network. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitter 13 can include elements for generating a media file in a predetermined file format and can include elements for transmitting through a broadcast / communication network. The receiver 21 can extract / receive the bit stream from the storage medium or the network and transmit the bit stream to the decoding unit 22.

[0083] The decoding unit 22 can decode the video / images by performing a series of processes corresponding to the operations of the compiling unit 12, such as dequantization, inverse transformation, and prediction.

[0084] The renderer 23 can render the decoded video / images. The rendered video / images can be displayed through a display.

[0085] Overview of Image Encoding Device

[0086] Figure 2 is a view schematically showing an image encoding device to which an embodiment of the present disclosure can be applied.

[0087] As Figure 2As shown, the image encoding apparatus 100 may include an image partitioner 110, a subtractor 115, a transformer 120, a quantizer 130, a dequantizer 140, an inverse transformer 150, an adder 155, a filter 160, a memory 170, an inter prediction unit 180, an intra prediction unit 185, and an entropy encoder 190. The inter prediction unit 180 and the intra prediction unit 185 may be collectively referred to as a "prediction unit". The transformer 120, the quantizer 130, the dequantizer 140, and the inverse transformer 150 may be included in a residual processor. The residual processor may further include the subtractor 115.

[0088] In some embodiments, all or at least some of the plurality of components configuring the image encoding apparatus 100 may be configured by one hardware component (e.g., an encoder or a processor). In addition, the memory 170 may include a decoded picture buffer (DPB) and may be configured by a digital storage medium.

[0089] The image partitioner 110 may partition an input image (or picture or frame) input to the image encoding apparatus 100 into one or more processing units. For example, a processing unit may be referred to as a coding unit (CU). Coding units may be obtained by recursively partitioning a coding tree unit (CTU) or a largest coding unit (LCU) according to a quadtree / binary tree / trinary tree (QT / BT / TT) structure. For example, a coding unit may be partitioned into multiple coding units of a deeper depth based on a quadtree structure, a binary tree structure, and / or a trinary tree structure. For partitioning of a coding unit, a quadtree structure may be first applied, and then a binary tree structure and / or a trinary tree structure may be applied. The coding process according to the present disclosure may be performed based on the final coding units that are no longer partitioned. The largest coding unit may be used as the final coding unit, or the coding units of a deeper depth obtained by partitioning the largest coding unit may be used as the final coding units. Here, the coding process may include processes of prediction, transformation, and reconstruction, which will be described later. As another example, the processing unit of the coding process may be a prediction unit (PU) or a transform unit (TU). The prediction unit and the transform unit may be split or partitioned from the final coding unit. The prediction unit may be a sample prediction unit, and the transform unit may be a unit for deriving transform coefficients and / or a unit for deriving a residual signal from the transform coefficients.

[0090] The prediction unit (the inter prediction unit 180 or the intra prediction unit 185) may perform prediction on a block to be processed (a current block) and generate a prediction block including prediction samples of the current block. The prediction unit may determine whether to apply intra prediction or inter prediction based on the current block or CU. The prediction unit may generate various information related to the prediction of the current block and transmit the generated information to the entropy encoder 190. The information about the prediction may be encoded in the entropy encoder 190 and output in the form of a bitstream.

[0091] The intra prediction unit 185 can predict the current block by referring to samples in the current picture. Depending on the intra prediction mode and / or intra prediction technique, the reference samples can be located among the neighbors of the current block or can be placed separately. The intra prediction mode can include a plurality of non - directional modes and a plurality of directional modes. The non - directional modes can include, for example, the DC mode and the planar mode. Depending on the level of detail of the prediction direction, the directional modes can include, for example, 33 directional prediction modes or 65 directional prediction modes. However, these are merely examples, and more or fewer directional prediction modes can be used according to the settings. The intra prediction unit 185 can determine the prediction mode applied to the current block by using the prediction mode applied to neighboring blocks.

[0092] The inter prediction unit 180 can derive the prediction block of the current block based on the reference block (reference sample array) specified by the motion vectors on the reference pictures. In this case, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information can be predicted in units of blocks, sub - blocks, or samples based on the correlation of the motion information between neighboring blocks and the current block. The motion information can include motion vectors and reference picture indices. The motion information can also include inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter prediction, the neighboring blocks can include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference pictures. The reference picture including the reference block and the reference picture including the temporal neighboring blocks can be the same or different. The temporal neighboring blocks can be referred to as collocated reference blocks, collocated CUs (colCUs), etc. The reference picture including the temporal neighboring blocks can be referred to as a collocated picture (colPic). For example, the inter prediction unit 180 can configure a motion information candidate list based on neighboring blocks and generate information specifying which candidate to use to derive the motion vector and / or reference picture index of the current block. The inter prediction can be performed based on various prediction modes. For example, in the case of the skip mode and the merge mode, the inter prediction unit 180 can use the motion information of neighboring blocks as the motion information of the current block. In the case of the skip mode, different from the merge mode, the residual signal may not be transmitted. In the case of the motion vector prediction (MVP) mode, the motion vector of the neighboring block can be used as a motion vector predictor, and the motion vector of the current block can be signaled by encoding the motion vector difference and an indicator of the motion vector predictor. The motion vector difference can mean the difference between the motion vector of the current block and the motion vector predictor.

[0093] The prediction unit may generate a prediction signal based on various prediction methods and prediction techniques described below. For example, the prediction unit may not only apply intra prediction or inter prediction, but also apply both intra prediction and inter prediction simultaneously to predict the current block. The prediction method of applying both intra prediction and inter prediction simultaneously to predict the current block may be referred to as combined inter and intra prediction (CIIP). In addition, the prediction unit may perform intra block copy (IBC) to predict the current block. Intra block copy may be used for content image / video compilation such as games, for example, screen content compilation (SCC). IBC is a method of predicting the current picture using a previously reconstructed reference block in the current picture at a position separated from the current block by a predetermined distance. When IBC is applied, the position of the reference block in the current picture may be encoded as a vector (block vector) corresponding to the predetermined distance. IBC basically performs prediction in the current picture, but may be performed similar to inter prediction because the reference block is derived within the current picture. That is, IBC may use at least one of the inter prediction techniques described in the present disclosure. IBC basically performs prediction in the current picture, but may be performed similar to inter prediction because the reference block is derived within the current picture. That is, IBC may use at least one of the inter prediction techniques described in the present disclosure.

[0094] The prediction signal generated by the prediction unit may be used to generate a reconstruction signal or generate a residual signal. The subtractor 115 may generate a residual signal (residual block or residual sample array) by subtracting the prediction signal (prediction block or prediction sample array) output from the prediction unit from the input image signal (original block or original sample array). The generated residual signal may be transmitted to the transformer 120.

[0095] The transformer 120 may generate transform coefficients by applying a transform technique to the residual signal. For example, the transform technique may include at least one of discrete cosine transform (DCT), discrete sine transform (DST), karhunen - loève transform (KLT), graph - based transform (GBT), or conditional non - linear transform (CNT). Here, GBT refers to a transform obtained from a graph when the relationship information between pixels is represented by a graph. CNT refers to a transform obtained based on a prediction signal generated using all previously reconstructed pixels. In addition, the transform processing may be applied to square pixel blocks of the same size or may be applied to blocks having a variable size rather than square.

[0096] The quantizer 130 may quantize the transform coefficients and transmit them to the entropy encoder 190. The entropy encoder 190 may encode the quantized signal (information on the quantized transform coefficients) and output a bitstream. The information on the quantized transform coefficients may be referred to as residual information. The quantizer 130 may rearrange the quantized transform coefficients in block form into a one-dimensional vector form based on the coefficient scan order, and generate information on the quantized transform coefficients based on the quantized transform coefficients in the one-dimensional vector form.

[0097] The entropy encoder 190 may perform various coding methods, such as exponential Golomb, context-adaptive variable length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), etc. The entropy encoder 190 may encode information required for video / image reconstruction (e.g., values of syntax elements, etc.) in addition to the quantized transform coefficients, either together or separately. The encoded information (e.g., encoded video / image information) may be transmitted or stored in the form of a bitstream in units of network abstraction layer (NAL). The video / image information may also include information on various parameter sets, such as adaptive parameter set (APS), picture parameter set (PPS), sequence parameter set (SPS), or video parameter set (VPS). In addition, the video / image information may also include general constraint information. The information signaled, transmitted, and / or syntax elements described in this disclosure may be encoded through the above coding process and included in the bitstream.

[0098] The bitstream may be transmitted through a network or may be stored in a digital storage medium. The network may include a broadcast network and / or a communication network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmitter (not shown) for transmitting the signal output from the entropy encoder 190 and / or a storage unit (not shown) for storing the signal may be included as internal / external elements of the image coding device 100. Alternatively, a transmitter may be provided as a component of the entropy encoder 190.

[0099] The quantized transform coefficients output from the quantizer 130 may be used to generate a residual signal. For example, the quantized transform coefficients may be dequantized and inverse-transformed by a dequantizer 140 and an inverse-transformer 150 to reconstruct a residual signal (residual block or residual sample).

[0100] The adder 155 adds the reconstructed residual signal to the predicted signal output from the inter-frame prediction unit 180 or the intra-frame prediction unit 185 to generate a reconstructed signal (reconstructed image, reconstructed block, reconstructed sample array). If there is no residual for the block to be processed, for example, in the case of applying the skip mode, the predicted block can be used as the reconstructed block. The adder 155 can be referred to as a reconstructor or a reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed in the current picture and can be used for inter-frame prediction of the next picture through filtering as described below.

[0101] The filter 160 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filter 160 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture and store the modified reconstructed picture in the memory 170, specifically, in the DPB of the memory 170. The various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filtering, bilateral filtering, etc. The filter 160 can generate various information related to the filtering and transmit the generated information to the entropy encoder 190, as described later in the description of each filtering method. The information related to the filtering can be encoded by the entropy encoder 190 and output in the form of a bitstream.

[0102] The modified reconstructed picture transmitted to the memory 170 can be used as a reference picture in the inter-frame prediction unit 180. When inter-frame prediction is applied by the image coding device 100, prediction mismatch between the image coding device 100 and the image decoding device can be avoided and the coding efficiency can be improved.

[0103] The DPB of the memory 170 can store the modified reconstructed picture to be used as a reference picture in the inter-frame prediction unit 180. The memory 170 can store the motion information of the block from which the motion information in the current picture is derived (or encoded) and / or the motion information of the blocks that have been reconstructed in the picture. The stored motion information can be transmitted to the inter-frame prediction unit 180 and used as the motion information of spatially adjacent blocks or temporally adjacent blocks. The memory 170 can store the reconstructed samples of the reconstructed blocks in the current picture and can transmit the reconstructed samples to the intra-frame prediction unit 185.

[0104] Overview of Image Decoding Device

[0105] Figure 3 is a view schematically showing an image decoding device to which an embodiment of the present disclosure is applicable.

[0106] As Figure 3As shown, the image decoding apparatus 200 may include an entropy decoder 210, a dequantizer 220, an inverse transformer 230, an adder 235, a filter 240, a memory 250, an inter prediction unit 260, and an intra prediction unit 265. The inter prediction unit 260 and the intra prediction unit 265 may be collectively referred to as a "prediction unit". The dequantizer 220 and the inverse transformer 230 may be included in a residual processor.

[0107] According to an embodiment, all or at least some of the plurality of components configuring the image decoding apparatus 200 may be configured by hardware components (e.g., a decoder or a processor). Further, the memory 250 may include a decoded picture buffer (DPB) or may be configured by a digital storage medium.

[0108] The image decoding apparatus 200 that has received a bitstream including video / image information may reconstruct an image by performing a process corresponding to the process performed by Figure 2 the image encoding apparatus 100. For example, the image decoding apparatus 200 may perform decoding using the processing units applied in the image encoding apparatus. Thus, the decoding processing units may be, for example, compilation units. The compilation units may be obtained through a partitioned compilation tree unit or a maximum compilation unit. The reconstructed image signal decoded and output by the image decoding apparatus 200 may be reproduced by a reproduction apparatus (not shown).

[0109] The image decoding apparatus 200 may receive, in the form of a bitstream, from Figure 2The signal output by the image encoding device. The received signal can be decoded by the entropy decoder 210. For example, the entropy decoder 210 can parse the bitstream to derive the information (e.g., video / image information) required for image reconstruction (or picture reconstruction). The video / image information can also include information about various parameter sets, such as the Adaptive Parameter Set (APS), Picture Parameter Set (PPS), Sequence Parameter Set (SPS), or Video Parameter Set (VPS). In addition, the video / image information can also include general constraint information. The image decoding device can also decode the picture based on the parameter set information and / or the general constraint information. The information and / or syntax elements signaled / received described in this disclosure can be decoded through the decoding process and obtained from the bitstream. For example, the entropy decoder 210 decodes the information in the bitstream based on an encoding method such as Exponential Golomb coding, CAVLC, or CABAC, and outputs the values of the syntax elements required for image reconstruction and the quantization values of the transformed coefficients of the residuals. More specifically, the CABAC entropy decoding method can receive the bins corresponding to each syntax element in the bitstream, use the decoding target syntax element information, neighboring blocks, and the decoding information of the decoding target block or the information of the symbols / bins decoded in the previous stage to determine the context model, perform arithmetic decoding on the bins by predicting the occurrence probability of the bins according to the determined context model, and generate symbols corresponding to the values of each syntax element. In this case, after determining the context model, the CABAC entropy decoding method can update the context model by using the information of the decoded symbols / bins for the context model of the next symbol / bin. The information related to prediction in the information decoded by the entropy decoder 210 can be provided to the prediction units (inter-frame prediction unit 260 and intra-frame prediction unit 265), and the residual values for which entropy decoding is performed in the entropy decoder 210, that is, the quantized transform coefficients and the related parameter information, can be input to the dequantizer 220. In addition, the information about filtering among the information decoded by the entropy decoder 210 can be provided to the filter 240. Meanwhile, the receiver (not shown) for receiving the signal output by the image encoding device can be further configured as an internal / external component of the image decoding device 200, or the receiver can be a component of the entropy decoder 210.

[0110] Meanwhile, the image decoding device according to the present disclosure can be referred to as a video / image / picture decoding device. The image decoding device can be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder can include the entropy decoder 210. The sample decoder can include at least one of the dequantizer 220, inverse transform unit 230, adder 235, filter 240, memory 250, inter-frame prediction unit 260, or intra-frame prediction unit 265.

[0111] The dequantizer 220 may dequantize the quantized transform coefficients and output the transform coefficients. The dequantizer 220 may rearrange the quantized transform coefficients in the form of two-dimensional blocks. In this case, the rearrangement may be performed based on the coefficient scan order executed in the image coding device. The dequantizer 220 may dequantize the quantized transform coefficients by using quantization parameters (e.g., quantization step information) and obtain the transform coefficients.

[0112] The inverse transformer 230 may perform an inverse transform on the transform coefficients to obtain a residual signal (residual block, residual sample array).

[0113] The prediction unit may perform prediction on the current block and generate a prediction block including the prediction samples of the current block. The prediction unit may determine whether to apply intra prediction or inter prediction to the current block based on the information about the prediction output from the entropy decoder 210, and may determine a specific intra / inter prediction mode (prediction technique).

[0114] Similar to that described for the prediction unit in the image coding device 100, the prediction unit may generate a prediction signal based on various prediction methods (techniques) described later.

[0115] The intra prediction unit 265 may predict the current block by referring to the samples in the current picture. The description of the intra prediction unit 185 is equally applicable to the intra prediction unit 265.

[0116] The inter prediction unit 260 may derive a prediction block of the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include information about an inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter prediction, the neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. For example, the inter prediction unit 260 may configure a motion information candidate list based on neighboring blocks, and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. The inter prediction may be performed based on various prediction modes, and the information about the prediction may include information indicating the inter prediction mode of the current block.

[0117] The adder 235 can generate a reconstructed block by adding the obtained residual signal to the prediction signal (prediction block, prediction sample array) output from the prediction unit (including the inter-frame prediction unit 260 and / or the intra-frame prediction unit 265). If the block to be processed has no residual, such as when the skip mode is applied, the prediction block can be used as the reconstructed block. The description of the adder 155 equally applies to the adder 235. The adder 235 can be referred to as a reconstructor or a reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed in the current picture and can be used for inter-frame prediction of the next picture through filtering as described below.

[0118] The filter 240 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filter 240 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture and store the modified reconstructed picture in the memory 250, specifically, in the DPB of the memory 250. Various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filtering, bilateral filtering, etc.

[0119] The (modified) reconstructed picture stored in the DPB of the memory 250 can be used as a reference picture in the inter-frame prediction unit 260. The memory 250 can store the motion information of the block from which the motion information in the current picture is derived (or decoded) and / or the motion information of the blocks that have been reconstructed in the picture. The stored motion information can be transmitted to the inter-frame prediction unit 260 to be used as the motion information of spatially adjacent blocks or temporally adjacent blocks. The memory 250 can store the reconstructed samples of the reconstructed blocks in the current picture and transmit the reconstructed samples to the intra-frame prediction unit 265.

[0120] In the present disclosure, the embodiments described in the filter 160, the inter-frame prediction unit 180, and the intra-frame prediction unit 185 of the image encoding device 100 can be equally or correspondingly applied to the filter 240, the inter-frame prediction unit 260, and the intra-frame prediction unit 265 of the image decoding device 200.

[0121] Partition Structure

[0122] The image encoding / decoding method according to the present disclosure may be performed based on the partitioning structure according to the embodiments. For example, processes such as prediction, residual processing ((inverse) transformation, (de) quantization, etc.), syntax element compilation, and filtering may be performed based on CTUs, CUs (and / or TUs or PUs) derived based on the partitioning structure. The block partitioning process may be performed by the image partitioner 110 of the above encoding device, and the partitioning-related information may be encoded (processed) by the entropy encoder 190 and sent to the decoding device in the form of a bitstream. The entropy decoder 210 of the decoding device may derive the block partitioning structure of the current picture based on the partitioning-related information obtained from the bitstream, and based on this, a series of processes (e.g., prediction, residual processing, block / picture reconstruction, in-loop filtering, etc.) may be performed for image decoding. The CU size and the TU size may be the same or multiple TUs may exist in the CU region. At the same time, the CU size may generally represent the CB size of the luminance component (samples). The TU size may generally represent the TB size of the luminance component (samples). The CB or TB size of the chrominance component (samples) may be derived based on the CB or TB size of the luminance component (samples) according to the component ratio according to the chrominance format (color format, e.g., 4:4:4, 4:2:2, 4:2:0, etc.) of the picture / image. The TU size may be derived based on maxTbSize that specifies the available maximum TB size. For example, when the CU size is larger than maxTbSize, multiple TUs (TBs) of maxTbSize may be derived from the CU, and the transformation / inverse transformation may be performed in units of TUs (TBs). Additionally, for example, when intra prediction is applied, the intra prediction mode / type may be derived in units of CUs (or CBs), and the neighboring reference sample derivation and predicted sample generation processes may be performed in units of TUs (or TBs). In this case, one or more TUs (or TBs) may exist in one CU (or CB) region, and in this case, multiple TUs (or TBs) may share the same intra prediction mode / type.

[0123] In addition, in the image encoding and decoding according to the present disclosure, the image processing unit may have a hierarchical structure. For example, a picture may be partitioned into one or more tiles or tile groups. A tile group may include one or more tiles. A tile may include one or more CTUs. As described above, a CTU may be partitioned into one or more CUs. A tile may be constituted by a rectangular region of CTUs included in a picture in a specific row and specific column combination. A tile group may include an integer number of tiles according to the tile raster scan. The tile group header may signal information / parameters applicable to the corresponding tile group. When the encoding / decoding device has a multi-core processor, the encoding / decoding process for the tile or tile group may be performed in parallel. Here, the tile group may have one of tile group types including an intra (I) tile group, a predictive (P) tile group, and a bi-predictive (B) tile group. For the blocks in the I tile group, inter-frame prediction may not be used and only intra-frame prediction may be used for prediction. Of course, even in this case, the original sample values may be compiled and signaled without prediction. For the blocks in the P tile group, intra-frame prediction or inter-frame prediction may be used, and only unidirectional prediction may be used during inter-frame prediction. Meanwhile, for the blocks in the B tile group, intra-frame prediction or inter-frame prediction may be used, and up to bi-prediction may be used when using inter-frame prediction.

[0124] In addition, a picture may be partitioned into one or more slices. A slice may be composed of an integer number of tiles or a set of CTUs in rows continuously arranged within a tile. Two slice modes may be supported. One is the raster scan slice mode and the other is the rectangular slice mode. In the raster scan slice mode, a slice may be composed of consecutive tiles present in a picture in raster scan order, as Figure 4 shown. In the rectangular slice mode, a slice may be composed of tiles present in a picture in a rectangular shape. The tiles in the rectangular slice may be scanned within the slice according to the tile raster scan order.

[0125] In the encoding device, the tile / tile group, slice, and the maximum and minimum compilation unit sizes may be determined according to the characteristics of the image (e.g., resolution) and in consideration of compilation efficiency or parallel processing, and information about them or information capable of deriving such information may be included in the bitstream.

[0126] In the decoder, information specifying that the CTUs in the slice, tile / tile group, or tile of the current picture are partitioned into multiple compilation units may be obtained. When such information is obtained (sent) only under specific conditions, the efficiency may be improved.

[0127] The slice header or tile group header (tile group header syntax) may include information / parameters that are commonly applicable to a slice or tile group. The APS (APS syntax) or PPS (PPS syntax) may include information / parameters that are commonly applicable to one or more pictures. The SPS (SPS syntax) may include information / parameters that are commonly applicable to one or more sequences. The VPS (VPS syntax) may include information / parameters that are commonly applicable to the entire video. In the present disclosure, a higher-level syntax may include at least one of the APS syntax, PPS syntax, SPS syntax, or VPS syntax.

[0128] Additionally, for example, information regarding the partitioning and construction of tiles / tile groups may be constructed through a higher-level syntax during the encoding stage and sent to the decoding device in the form of a bitstream.

[0129] Furthermore, in the image encoding / decoding according to the present disclosure, the compile tree scheme may support the luminance and chrominance component blocks to have separate block tree structures. The case where the luminance and chrominance blocks in a CTU have the same block tree structure may be represented as SINGLE_TREE. The case where the luminance and chrominance blocks in a CTU have separate block tree structures may be represented as DUAL_TREE. In this case, the block tree type of the luminance component may be referred to as DUAL_TREE_LUMA, and the block tree type of the chrominance component may be referred to as DUAL_TREE_CHROMA. For P and B slices / tile groups, the luminance and chrominance CTBs in a CTU may be restricted to having the same compile tree structure. However, for I slices / tile groups, the luminance and chrominance blocks may have separate block tree structures. When the separate block tree mode is applied, the luminance CTB may be partitioned into CUs based on a specific compile tree structure, and the chrominance CTB may be partitioned into chrominance CUs based on another compile tree structure. For example, a CU in an I slice / tile group may be composed of a compiled block of the luminance component or compiled blocks of two chrominance components, and a CU in a P or B slice / tile group may be composed of blocks of three color components. Hereinafter, in the present disclosure, a slice may be referred to as a tile / tile group, and a tile / tile group may be referred to as a slice.

[0130] Overview of Intra Prediction

[0131] Hereinafter, an intra prediction method according to an embodiment will be described. Intra prediction may indicate a prediction for generating a prediction sample for a current block based on reference samples in a picture (hereinafter referred to as a current picture) to which the current block belongs. When intra prediction is applied to a current block, neighboring reference samples to be used for intra prediction of the current block may be derived. The neighboring reference samples of the current block may include samples adjacent to the left boundary of the current block having a size of nW x nH and a total of 2 x nH samples adjacent to the left bottom, samples adjacent to the top boundary of the current block, and a total of 2 x nW samples adjacent to the right top and one sample adjacent to the left top of the current block. Alternatively, the neighboring reference samples of the current block may include multiple columns of top neighboring samples and multiple rows of left neighboring samples. In addition, the neighboring reference samples of the current block may include a total of nH samples adjacent to the right boundary of the current block having a size of nW x nH, a total of nW samples adjacent to the bottom boundary of the current block, and one sample adjacent to the right bottom of the current block. Meanwhile, when the ISP to be described later is applied, neighboring reference samples may be derived in units of sub-partitions.

[0132] On the other hand, some of the neighboring reference samples of the current block have not been decoded or may be unavailable. In this case, the decoding device may construct neighboring reference samples to be used for prediction by replacing unavailable samples with available samples. Alternatively, interpolation of available samples may be used to construct neighboring reference samples to be used for prediction.

[0133] When neighboring reference samples are derived, (i) a predicted sample can be derived based on an average or interpolation of neighboring reference samples of a current block, and (ii) a predicted sample can be derived based on reference samples among neighboring reference samples of the current block that exist in a specific (prediction) direction with respect to the predicted sample. The case of (i) can be referred to as a non-directional mode or non-angle mode and the case of (ii) can be referred to as a directional mode or angle mode. Additionally, a predicted sample can be generated by interpolation using a second neighboring sample and a first neighboring sample among neighboring reference samples, where the predicted sample of the current block based on the neighboring reference samples is in a direction opposite to the prediction direction of the intra prediction mode within the frame of the current block. The above case can be referred to as linear interpolation intra prediction (LIP). Additionally, a chrominance prediction sample can be generated based on luminance samples using a linear model. This case can be referred to as the LM mode. Additionally, a temporal prediction sample of the current block can be derived based on filtered neighboring reference samples, and a predicted sample of the current block can be derived by weighted summation of the temporal prediction sample and at least one reference sample derived according to the intra prediction mode among existing neighboring reference samples, i.e., unfiltered neighboring reference samples. The above case can be referred to as position-dependent intra prediction (PDPC). Additionally, a reference sample line with the highest prediction accuracy can be selected from multiple neighboring reference sample lines of the current block to use the reference samples located in the prediction direction in the corresponding line to derive a predicted sample, and at this time, intra prediction coding can be performed by indicating (signaling) the used reference sample line to a decoding device. The above case can be referred to as multi-reference line (MRL) intra prediction or MRL-based intra prediction. Additionally, the current block can be divided into vertical or horizontal sub-partitions to perform intra prediction based on the same intra prediction mode, and neighboring reference samples can be derived and used in units of sub-partitions. That is, in this case, the intra prediction mode of the current block is equally applied to the sub-partitions and neighboring reference samples are derived and used in units of sub-partitions, thereby improving intra prediction performance. Such a prediction method can be referred to as intra sub-partition (ISP) or ISP-based intra prediction. Additionally, when the prediction direction based on the predicted sample indicates the space between neighboring reference samples, that is, when the prediction direction indicates a fractional sample position, the value of the predicted sample can be derived by interpolation of multiple reference samples located around the prediction direction (around the fractional sample position). The above intra prediction method can be referred to as an intra prediction type to distinguish it from the intra prediction mode. Additionally, after generating a prediction signal for a subsampled pixel set of the current block using reconstructed neighboring pixels located to the left and above the current block, the generated prediction signal and neighboring sample values can be interpolated in the vertical and horizontal directions to generate a prediction signal of the original size, thereby applying matrix weighted intra prediction (MIP) to perform intra prediction of the current block.

[0134] The intra prediction type may be referred to by various terms such as an intra prediction scheme or an additional intra prediction mode. For example, the intra prediction type (or additional intra prediction mode) may include at least one of LIP, PDPC, MRL, ISP, or MIP. Information about the intra prediction type may be encoded by an encoding device, included in a bitstream, and signaled to a decoding device. The information about the intra prediction type may be implemented in various forms, such as flag information indicating whether each intra prediction type is applied or index information indicating one of several intra prediction types.

[0135] Meanwhile, when necessary, post-filtering may be performed with respect to the derived prediction samples. Specifically, the intra prediction process may include an intra prediction mode / type determination step, a neighboring reference sample derivation step, and a prediction sample derivation step based on the intra prediction mode / type. Additionally, when necessary, post-filtering may be performed with respect to the derived prediction samples.

[0136] Hereinafter, a video / image encoding method based on intra prediction will be described. First, an encoding device performs intra prediction with respect to a current block. The encoding device may derive an intra prediction mode / type of the current block, derive neighboring reference samples of the current block, and generate prediction samples in the current block based on the intra prediction mode / type and the neighboring reference samples. Here, the intra prediction mode / type determination, neighboring reference sample derivation, and prediction sample generation processes may be performed simultaneously or any one of the processes may be performed before the other processes. Meanwhile, when performing the following prediction sample filtering process, the intra prediction unit 185 may further include a prediction sample filter. The encoding device may determine the mode / type applied to the current block among a plurality of intra prediction modes / types. The encoding device may compare the rate-distortion (RD) cost of the intra prediction mode / type and determine the best intra prediction mode / type of the current block.

[0137] Meanwhile, the encoding device may perform a prediction sample filtering process. The prediction sample filtering may be referred to as post-filtering. Through the prediction sample filtering process, some or all of the prediction samples may be filtered. In some cases, the prediction sample filtering process may be omitted.

[0138] Next, the encoding device may generate residual samples of the current block based on the prediction samples. The encoding device may compare the original samples of the current block with the prediction samples in terms of phase and derive the residual samples.

[0139] Next, the encoding device may encode image information including information about intra prediction (prediction information) and residual information about the residual samples. The prediction information may include intra prediction mode information and intra prediction type information. The encoding device may output the encoded image information in the form of a bitstream. The output bitstream may be sent to a decoding device via a storage medium or a network.

[0140] The residual information may include a residual compilation syntax, which will be described later. The encoding device may transform / quantize the residual samples and derive quantized transform coefficients. The residual information may include information about the quantized transform coefficients.

[0141] Meanwhile, as described above, the encoding device may generate a reconstructed picture (including reconstructed samples and reconstructed blocks). To this end, the encoding device may perform inverse quantization / inverse transformation with respect to the quantized transform coefficients and derive (modified) residual samples. The reason for transforming / quantizing the residual samples and then performing inverse quantization / inverse transformation is to derive the same residual samples as those derived by the decoding device as described above. The encoding device may generate a reconstructed block including the reconstructed samples of the current block based on the prediction samples and the (modified) residual samples. Based on the reconstructed blocks, a reconstructed picture of the current picture may be generated. As described above, the in-loop filtering process is applicable to the reconstructed picture.

[0142] Hereinafter, a video / image encoding method based on intra prediction will be described. The decoding device may perform operations corresponding to the operations performed by the encoding device.

[0143] First, the decoding device may derive the intra prediction mode / type of the current block based on the received prediction information (intra prediction mode / type information). The decoding device may derive the neighboring reference samples of the current block. The decoding device may generate prediction samples in the current block based on the intra prediction mode / type and the neighboring reference samples. In this case, the decoding device may perform a prediction sample filtering process. The prediction sample filtering may be referred to as post-filtering. Through the prediction sample filtering process, some or all of the prediction samples may be filtered. In some cases, the prediction sample filtering process may be omitted.

[0144] The decoding device may generate the residual samples of the current block based on the received residual information. The decoding device may generate the reconstructed samples of the current block based on the prediction samples and the residual samples and derive a reconstructed block including the reconstructed samples. Based on the reconstructed blocks, a reconstructed picture of the current picture may be generated. The in-loop filtering process is also applicable to the reconstructed picture.

[0145] The intra prediction mode information may include, for example, flag information (e.g., intra_luma_mpm_flag) indicating whether the most probable mode (MPM) or the residual mode is applied to the current block, and when the MPM is applied to the current block, the prediction mode information may further include index information (e.g., intra_luma_mpm_idx) indicating one of the intra prediction mode candidates (MPM candidates). The intra prediction mode candidates (MPM candidates) may configure an MPM candidate list or an MPM list. For example, the MPM candidate list may include the intra prediction modes of neighboring blocks or preset basic intra prediction modes. Additionally, when the MPM is not applied to the current block, the intra prediction mode information may further include residual mode information (e.g., intra_luma_mpm_remainder) indicating one of the residual intra prediction modes excluding the intra prediction mode candidates (MPM candidates). The decoding device may determine the intra prediction mode of the current block based on the intra prediction mode information.

[0146] Meanwhile, when the above MIP mode is applied, the MPM list for the MIP mode may be configured to determine the MIP mode of the current block. The MPM list for the MIP mode may be configured in the same way as the above MPM list configured for the intra mode. For example, when the MIP mode is applied, the MPM candidate list for the MIP mode may be configured to include the MIP modes of neighboring blocks or a predetermined default MIP mode. Additionally, when the MPM is not applied to the current block, the intra prediction mode information may further include residual mode information (e.g., intra_luma_mpm_remainder) specifying one of the residual MIP modes other than the MIP mode candidates (MPM candidates). The decoding device may determine the MIP mode of the current block based on the intra prediction mode information.

[0147] Intra Prediction Mode

[0148] Hereinafter, the intra prediction mode will be described in more detail. Figure 5 FIG. shows the intra prediction direction according to an embodiment. To capture any edge direction presented in natural video, as Figure 5 shown, the intra prediction mode may include two non-directional intra prediction modes and 65 directional intra prediction modes. The non-directional intra prediction modes may include a planar intra prediction mode and a DC intra prediction mode, while the directional intra prediction modes may include a second intra prediction mode to a 66th intra prediction mode.

[0149] Meanwhile, in addition to the above intra prediction modes, the intra prediction modes may further include a Cross-Component Linear Model (CCLM) mode for chrominance samples. The CCLM mode may be divided into L_CCLM, T_CCLM, and LT_CCLM according to whether left samples, top samples, or both are considered for LM parameter derivation and may be applied only to the chrominance component. For example, the intra prediction modes may be indexed according to the intra prediction mode values shown in the following table.

[0150] [Table 1]

[0151] Intra Prediction Mode Related Names 0 INTRA_PLANAR 1 INTRA_DC 2..66 INTRA_ANGULAR2..INTRA_ANGULAR66 81..83 INTRA_LT_CCLM, INTRA_L_CCLM, INTRA_T_CCLM

[0152] Figure 6 Shows an intra prediction direction according to another embodiment. Here, the dotted line direction shows a wide-angle mode applied only to non-square blocks. As Figure 6 shown, in order to capture any edge direction presented in natural videos, the intra prediction modes according to the embodiment may include two non-directional intra prediction modes and 93 directional intra prediction modes. The non-directional intra prediction modes may include a planar intra prediction mode and a DC intra prediction mode, while the directional intra prediction modes may include a second intra prediction mode to an 80th intra prediction mode and a -1st intra prediction mode to a -14th intra prediction mode, as indicated by the Figure 6 arrows. The planar prediction mode may be represented by INTRA_PLANAR, and the DC prediction mode may be represented by INTRA_DC. Additionally, the directional intra prediction modes may be represented by INTRA_ANGULAR-14 to INTRA_ANGULAR-1 and INTRA_ANGULAR2 to INTRA_ANGULAR80.

[0153] Meanwhile, the intra prediction type (or additional intra prediction mode) may include at least one of LIP, PDPC, MRL, ISP, or MIP. The intra prediction type may be indicated based on the intra prediction type information, and the intra prediction type information may be implemented in various forms. For example, the intra prediction type information may include intra prediction type index information indicating one of the intra prediction types. As another example, the intra prediction type information may include at least one of the following: reference sample line information (e.g., intra_luma_ref_idx) indicating whether MRL is applied to the current block and which reference sample line is used if it is applied, ISP flag information (e.g., intra_subpartitions_mode_flag) indicating whether ISP is applied to the current block, ISP type information (e.g., intra_subpartitions_split_flag) indicating the split type of sub - partitions when ISP is applied, flag information indicating whether PDPC is applied, flag information indicating whether LIP is applied, or MIP flag information indicating whether MIP is applied.

[0154] The intra prediction mode information and / or the intra prediction type information may be encoded / decoded using the compilation methods described in the present disclosure. For example, the intra prediction mode information and / or the intra prediction type information may be encoded / decoded based on truncated (Rice) binary codes by entropy compilation (e.g., CABAC, CAVLC).

[0155] Intra Prediction of Chrominance Blocks

[0156] When performing intra prediction with respect to the current block, prediction of the luminance component block (luminance block) and the chrominance component block (chrominance block) of the current block may be performed. In this case, the intra prediction mode of the chrominance block may be set separately from the intra prediction mode of the luminance block.

[0157] For example, the intra prediction mode for the chrominance block may be specified based on the intra chrominance prediction mode information, and the intra chrominance prediction mode information may be signaled in the form of the intra_chroma_pred_mode syntax element. For example, the intra chrominance prediction mode information may indicate one of a planar mode, a DC mode, a vertical mode, a horizontal mode, a derived mode (DM), and CCLM. Here, the planar mode may indicate intra prediction mode #0, the DC mode may indicate intra prediction mode #1, the vertical mode may indicate intra prediction mode #26, and the horizontal mode may indicate intra prediction mode #10. DM may also be referred to as the direct mode. CCLM may be referred to as LM.

[0158] Meanwhile, DM and CCLM are for intra prediction modes of chrominance blocks using information about luminance blocks. DM can indicate a mode in which the same intra prediction mode as that used for the intra prediction mode of the luminance component is applied to the intra prediction mode of the chrominance component. Additionally, CCLM can indicate an intra prediction mode that uses samples derived by subsampling the reconstructed samples of the luminance block during the process of generating a prediction block for the chrominance block and then applying CCLM parameters α and β to the subsampled samples as the prediction samples of the chrominance block.

[0159] MPM List in Intra Prediction

[0160] When intra prediction is applied, the intra prediction mode of neighboring blocks can be used to determine the intra prediction mode applied to the current block. For example, the decoding device can select one of the MPM candidates in the MPM list derived based on the intra prediction modes of neighboring blocks (e.g., left and / or top neighboring blocks) of the current block and additional candidate modes based on the MPM index received using the bitstream (e.g., intra_luma_mpm_idx). Alternatively, the decoding device can select one of the remaining intra prediction modes not included in the MPM candidates based on the remaining mode information (e.g., intra_luma_mpm_remainder). For example, it can be determined which intra prediction mode is applied to the current block based on the mpm flag (e.g., intra_luma_mpm_flag) indicating whether the intra prediction mode of the current block is in the MPM candidates or in the remaining modes. A value of 1 for the mpm flag can indicate that the intra prediction mode of the current block is in the MPM list (candidates), while a value of 0 for the mpm flag can indicate that the intra prediction mode of the current block is not in the MPM list (candidates).

[0161] The mpm flag can be signaled in the form of the intra_luma_mpm_flag syntax element, the mpm index can be signaled in the form of the mpm_idx or intra_luma_mpm_idx syntax element, and the remaining intra prediction mode information can be signaled in the form of the rem_intra_luma_pred_mode or intra_luma_mpm_remainder syntax element. In an embodiment, the remaining intra prediction mode information can specify one of the remaining intra prediction modes not included in the mpm list of all intra prediction modes and is indexed in the order of the prediction mode numbers. The intra prediction mode can be an intra prediction mode for a luminance component (sample). Hereinafter, the intra prediction mode information can include at least one of an mpm flag (e.g., intra_luma_mpm_flag), an mpm index (e.g., mpm_idx or intra_luma_mpm_idx), or remaining intra prediction mode information (e.g., rem_intra_luma_pred_mode or intra_luma_mpm_remainder). In the present disclosure, the MPM list can be referred to by various terms such as an MPM candidate list, a candModeList, etc.

[0162] The MPM list can include candidate intra prediction modes (MPM candidates) that are highly likely to be applied to the current block. The MPM list can be configured to include the intra prediction modes of neighboring blocks and can be further configured to include predetermined intra prediction modes according to a predetermined method.

[0163] In an embodiment, to keep the complexity of generating the MPM list low, an MPM list including three MPMs can be generated. For example, even when 67 intra prediction modes are used, the MPM list can include three MPM candidates. When the intra prediction mode of the current block is not included in the MPM list, the remaining modes can be used. In this case, the remaining modes can include 64 remaining candidates, and the remaining intra prediction mode information specifying one of the 64 remaining candidates can be signaled. For example, the remaining intra prediction mode information can include a 6-bit syntax element (e.g., the rem_intra_luma_pred_mode or intra_luma_mpm_remainder syntax element).

[0164] In an embodiment, the MPM list can be configured considering neighboring intra modes, derived intra modes, and default intra modes. For example, the encoding device can use the prediction mode of a neighboring block to encode the prediction mode of the current block.

[0165] For example, when encoding neighboring blocks in an intra prediction mode, the encoding device may confirm or derive the prediction mode of the neighboring blocks. For example, the encoding device may determine the prediction mode of the current block based on the prediction mode of the left neighboring block and the prediction mode of the top neighboring block, and in this case, the prediction mode of the corresponding neighboring block may be determined as the most probable mode (MPM). In this regard, determining the MPM may be expressed as enumerating MPM candidates or configuring an MPM list.

[0166] In an embodiment, the left neighboring block may specify the uppermost block among the neighboring blocks adjacent to the left boundary of the current block. Additionally, the top neighboring block may specify the leftmost block among the neighboring blocks adjacent to the top boundary of the current block. The encoding device may determine whether the prediction mode of the left neighboring block and the prediction mode of the top neighboring block are the same. An initial MPM list may be formed by performing a pruning process on the intra prediction modes of two adjacent blocks. The pruning process may be performed such that only different prediction modes are included in the MPM list.

[0167] If the prediction mode of the left neighboring block and the prediction mode of the top neighboring block are not the same, the first MPM may be set to the prediction mode of the left neighboring block, the second MPM may be set to the prediction mode of the top neighboring block, and the third MPM may be set to one of the intra plane mode, the intra DC mode, or the intra vertical mode (intra prediction mode #50). Specifically, when the intra prediction modes of two neighboring blocks are different from each other, two intra prediction modes may be set as the MPMs, and after checking through MPM pruning, one of the default intra modes may be added to the MPM list. Here, the default intra modes may include the intra plane mode, the intra DC mode, and / or the intra vertical mode (intra prediction mode #50).

[0168] For example, when the prediction mode of the left neighboring block and the prediction mode of the top neighboring block are not the same, the MPM list may be configured according to the following cases.

[0169] Case 1: If neither the intra prediction mode of the left neighboring block nor the intra prediction mode of the top neighboring block is the intra plane mode, the MPM list may be configured to include the intra prediction mode block of the left neighboring block, the intra prediction mode of the top neighboring block, and the intra plane mode.

[0170] Case 2: When the conditions of Case 1 are not satisfied, if neither the intra prediction mode of the left neighboring block nor the intra prediction mode of the top neighboring block is the intra DC mode, the MPM list may be configured to include the intra prediction mode of the left neighboring block, the intra prediction mode of the top neighboring block, and the intra DC mode.

[0171] Case 3: When the conditions of Case 2 are not met, the MPM list can be configured to include the intra prediction mode of the left neighboring block, the intra prediction mode of the top neighboring block, and the intra vertical mode.

[0172] Meanwhile, when the prediction modes of the left neighboring block and the top neighboring block are the same, the encoding device can determine whether the prediction mode of the left neighboring block is less than 2. For example, the encoding device can determine whether the prediction mode of the left neighboring block is the intra planar mode, the intra DC mode, or a prediction mode as Figure 6 shown with a prediction mode indicating the directivity of the block located at the bottom of the current block.

[0173] If the prediction mode of the left neighboring block is less than 2, the first MPM can be set to the intra planar mode, the second MPM can be set to the intra DC mode, and the third MPM can be set to the intra vertical mode (intra prediction mode #50).

[0174] Meanwhile, if the prediction mode of the left neighboring block is not less than 2, the first MPM can be set to the prediction mode of the left neighboring block, the second MPM can be set to (the prediction mode of the left neighboring block - 1), and the third MPM can be set to (the prediction mode of the left neighboring block + 1).

[0175] For example, when the prediction modes of the left neighboring block and the top neighboring block are the same, the MPM list can be configured as follows.

[0176] Case 1: When the value of the intra prediction mode of the left neighboring block is less than 2, the MPM list can be configured to include the intra planar mode, the intra DC mode, and the intra vertical mode.

[0177] Case 2: When the conditions of Case 1 are not met, the MPM list can be configured to include the intra prediction mode of the left neighboring block and the intra prediction mode corresponding to the value 2 + ((A + 61) % 64) and the intra prediction mode corresponding to the value 2 + ((A - 1) % 64) when the value of the intra prediction mode of the left neighboring block is A.

[0178] Meanwhile, an additional pruning process for removing duplicate modes can be performed so that only unique modes are included. In addition, for the entropy coding of 64 non - MPM modes other than the three MPMs, a 6 - bit fixed - length code can be used. That is, the index indicating the 64 non - MPM modes can be entropy - coded into a 6 - bit fixed - length code (6 - bit FLC).

[0179] In addition, the encoding device can determine whether the best intra prediction mode to be applied to the current block belongs to the MPM candidates configured above.

[0180] If the intra prediction mode of the current block belongs to the MPM candidates, the encoding device may encode the MPM flag and the MPM index. Here, the MPM flag may specify whether the intra prediction mode of the current block is derived from neighboring intra prediction blocks (that is, the intra prediction mode of the current block belongs to the MPM). In addition, the MPM index may specify which MPM mode among the MPM candidates is applied as the intra prediction mode of the current block.

[0181] In contrast, if the intra prediction mode of the current block does not belong to the MPM candidates, the encoding device may use the residual mode to encode the intra prediction mode of the current block.

[0182] Meanwhile, in an embodiment, the encoding device and the decoding device may be configured to include an MPM list of 6 MPMs. To generate an MPM list including 6 MPMs, the default MPM list may be considered. When the value of the intra prediction mode of the left neighboring block is A, the default MPM list may be configured as follows.

[0183] Default 6-MPM list = {A, Planar(0) or DC(1), Vertical(50), HOR(18), VER - 4(46), VER + 4(54)}

[0184] In addition, by performing a pruning process on the intra modes of two neighboring blocks, the default 6-MPM list may be updated to generate a 6-MPM list. For example, when the intra prediction modes of two neighboring blocks are the same and the value of the intra prediction mode of the two neighboring blocks is greater than 1 of the intra DC mode value, the 6-MPM list may include the intra prediction mode of the left neighboring block, the intra planar mode, and the intra DC mode as default modes, and may also include three derived modes derived by adding a predetermined offset value to the intra prediction mode of the neighboring block and performing a modulo operation with respect to the total number of intra prediction modes.

[0185] Meanwhile, when the intra prediction modes of neighboring blocks are different from each other, the 6-MPM list may be configured by including the intra prediction modes of the two neighboring blocks as the first two MPM modes. The remaining four MPM modes may be derived from the default modes and the intra prediction modes of the neighboring blocks.

[0186] When MIP is not applied to the current block, the above MPM list configuration method can be used. For example, the above MPM list configuration method can be used to derive the intra prediction modes used in LIP, PDPC, MRL, ISP intra prediction, or normal intra prediction (non-directional intra prediction and directional intra prediction). However, the left neighboring block or the top neighboring block can be compiled based on the above MIP. In this case, if the MIP mode number of the neighboring block (left neighboring block / top neighboring block) to which MIP is applied is applied to the MPM list of the current block to which MIP is not applied without change, this may be inappropriate because an unintended intra prediction mode is indicated. Therefore, in this case, the intra prediction mode of the neighboring block (left neighboring block / top neighboring block) to which MIP is applied can be regarded as the DC mode or the planar mode. Alternatively, as another example, the intra prediction mode of the neighboring block (left neighboring block / top neighboring block) to which MIP is applied can be mapped to a normal intra prediction mode based on a mapping table and used to configure the MPM list. In this case, the mapping can be performed based on the block size type of the current block. For example, Figure 7 the mapping table according to the embodiment shown can be used for mapping.

[0187] In Figure 7 the table, MIP IntraPredMode[ xNbX ][ yNbX ] specifies the MIP mode of the neighboring block (left neighboring block / top neighboring block), and the block size type MipSizeId specifies the block size type of the neighboring block or the current block. The numbers below the block size type values 0, 1, and 2 indicate the normal intra prediction modes to which the MIP mode is mapped in each case of the block size type. For example, the case where the height and width of the current block are 4 can be referred to as block size type 0, the case where the height and width of the current block are both equal to or less than 8 can be referred to as block size type 1, and another case can be referred to as block size type 2.

[0188] Here, the normal intra prediction mode is an intra prediction mode other than the MIP mode and can mean a non-directional intra prediction mode or a directional intra prediction mode. For example, when the block size type of the current block is 0 and the MIP mode number of the neighboring block is 10, the mapped normal intra prediction mode number can be 18. However, the mapping relationship can be an example and can change.

[0189] In addition, in an embodiment, the intra-plane mode may not be included in the MPM list. To this end, information specifying whether the intra prediction mode of the current block is the intra-plane mode may be signaled separately. When the prediction mode of the current block is not the intra-plane mode, an MPM list may be generated to signal the intra prediction mode. The encoding device may use the MPM list generated as follows when encoding the current block to signal the intra prediction mode of the current block to the decoding device, and the decoding device may use the generated MPM list as follows to determine the intra mode of the current block.

[0190] The MPM list may be determined based on the intra prediction modes of neighboring blocks of the current block. For example, the MPM list may be determined based on the intra prediction modes of the left neighboring block and the top neighboring block of the current block. For example, the encoding device and the decoding device may determine the MPM list based on a first intra prediction candidate determined based on the intra prediction mode of the left neighboring block and a second intra prediction candidate determined based on the intra prediction mode of the top neighboring block.

[0191] Here, the top neighboring block may be the rightmost block among the blocks adjacent to the top of the current block. The left neighboring block may be the bottommost block among the blocks adjacent to the left of the current block. For example, when the coordinates of the current block are (xCb, yCb), the width of the current block is cbWidth, and the height of the current block is cbHeight, the coordinates of the left neighboring block may be (xCb − 1, yCb + cbHeight − 1), and the coordinates of the top neighboring block may be (xCb + cbWidth − 1, yCb − 1).

[0192] When the left neighboring block is not available, when the prediction mode of the left neighboring block is not an intra prediction mode, or when the prediction mode of the left neighboring block is the MIP mode, the encoding device and the decoding device may determine the value of the first intra prediction candidate as the value specifying the intra-plane mode (e.g., 0). When the left neighboring block does not satisfy such conditions, the encoding device and the decoding device may determine the value of the first intra prediction candidate as the value of the intra prediction mode of the specified left neighboring block.

[0193] In addition, when the top neighboring block is not available, when the mode of the top neighboring block is not an intra prediction mode, or when the prediction mode of the top neighboring block is the MIP mode, the encoding device and the decoding device may determine the value of the second intra prediction candidate as the value specifying the intra-plane mode (e.g., 0). When the top neighboring block does not satisfy such conditions, the encoding device and the decoding device may determine the value of the second intra prediction candidate as the value of the intra prediction mode of the specified top neighboring block.

[0194] In an embodiment, the MPM list may be configured to include five candidate modes. In an embodiment, the MPM list may be configured according to the following situations. Hereinafter, the first intra prediction candidate is referred to as candIntraPredModeA, the second intra prediction candidate is referred to as candIntraPredModeB, and the MPM list is referred to as candModeList[x]. Here, x may be an integer from 0 to 4.

[0195] Situation 1: When the value of the first intra prediction candidate and the value of the second intra prediction candidate are the same and the value of the first intra prediction candidate is greater than 1 (e.g., when it is not the intra-plane mode or the intra-DC mode), the MPM list candModeList[x] may be configured as follows.

[0196] candModeList[ 0 ] = candIntraPredModeA

[0197] candModeList[ 1 ] = 2 + ( ( candIntraPredModeA + 61 ) % 64 )

[0198] candModeList[ 2 ] = 2 + ( ( candIntraPredModeA − 1 ) % 64 )

[0199] candModeList[ 3 ] = 2 + ( ( candIntraPredModeA + 60 ) % 64 )

[0200] candModeList[ 4 ] = 2 + ( candIntraPredModeA % 64 )

[0201] Situation 2: When the conditions of Situation 1 are not met, when the value of the first intra prediction candidate and the value of the second intra prediction candidate are not the same and the value of the first intra prediction candidate or the value of the second intra prediction candidate is greater than 1 (e.g., not the intra-plane mode or the intra-DC mode), the MPM list candModeList[x] may be configured as follows.

[0202] First, minAB and maxAB may be calculated as follows.

[0203] minAB = Min( candIntraPredModeA, candIntraPredModeB )

[0204] maxAB = Max( candIntraPredModeA, candIntraPredModeB )

[0205] When the values of the first intra prediction candidate and the second intra prediction candidate are both greater than 1, the MPM lists candModeList[0] and candModeList[1] can be configured as follows.

[0206] candModeList[ 0 ] = candIntraPredModeA

[0207] candModeList[ 1 ] = candIntraPredModeB

[0208] In this case, when the value of maxAB - minAB is 1, candModeList[2] to candModeList[4] can be configured as follows.

[0209] candModeList[ 2 ] = 2 + ( ( minAB + 61 ) % 64 )

[0210] candModeList[ 3 ] = 2 + ( ( maxAB − 1 ) % 64 )

[0211] candModeList[ 4 ] = 2 + ( ( minAB + 60 ) % 64 )

[0212] Meanwhile, when the value of maxAB - minAB is equal to or greater than 62, candModeList[2] to candModeList[4] can be configured as follows.

[0213] candModeList[ 2 ] = 2 + ( ( minAB − 1 ) % 64 )

[0214] candModeList[ 3 ] = 2 + ( ( maxAB + 61 ) % 64 )

[0215] candModeList[ 4 ] = 2 + ( minAB % 64 )

[0216] Meanwhile, when the value of maxAB - minAB is 2, candModeList[2] to candModeList[4] can be configured as follows.

[0217] candModeList[2] = 2 + ((minAB - 1) % 64)

[0218] candModeList[3] = 2 + ((minAB + 61) % 64)

[0219] candModeList[4] = 2 + ((maxAB - 1) % 64)

[0220] Meanwhile, when the value of maxAB - minAB does not meet the above conditions, candModeList[2] to candModeList[4] can be configured as follows.

[0221] candModeList[2] = 2 + ((minAB + 61) % 64)

[0222] candModeList[3] = 2 + ((minAB - 1) % 64)

[0223] candModeList[4] = 2 + ((maxAB + 61) % 64)

[0224] Meanwhile, when the predicted candidates in the first frame and the predicted candidates in the second frame are both not greater than 1 and only any one of the predicted candidates in the first frame and the predicted candidates in the second frame is greater than 1, the MPM list candModeList[x] can be configured as follows.

[0225] candModeList[0] = maxAB

[0226] candModeList[1] = 2 + ((maxAB + 61) % 64)

[0227] candModeList[2] = 2 + ((maxAB - 1) % 64)

[0228] candModeList[3] = 2 + ((maxAB + 60) % 64)

[0229] candModeList[4] = 2 + (maxAB % 64)

[0230] Case 3: When the conditions of Case 2 are not met, the MPM list candModeList[x] can be configured as follows.

[0231] candModeList[ 0 ] = INTRA_DC

[0232] candModeList[ 1 ] = INTRA_ANGULAR50

[0233] candModeList[ 2 ] = INTRA_ANGULAR18

[0234] candModeList[ 3 ] = INTRA_ANGULAR46

[0235] candModeList[ 4 ] = INTRA_ANGULAR54

[0236] Overview of MIP

[0237] The matrix-based intra prediction (MIP) mode can be referred to as the affine linear weighted intra prediction (ALWIP) mode, the linear weighted intra prediction (LWIP) mode, or the matrix weighted intra prediction (MWIP) mode.

[0238] When the MIP mode is applied to the current block, i) neighboring reference samples on which an averaging step is performed can be used, ii) a matrix-vector multiplication step can be performed, and iii) horizontal / vertical interpolation can be further performed if necessary to derive the predicted samples of the current block.

[0239] The averaging step can be performed by averaging the values of the neighboring samples. When the width and height of the current block are 4 in pixel units as shown in (a) of Figure 8 , the averaging process can be performed by taking the average of each boundary and generating a total of four samples including two top samples and two left samples, while when the width and height of the current block are not 4 in pixel units as shown in (b) of Figure 8 , the averaging step can be performed by taking the average of each boundary and generating a total of eight samples including four top samples and four left samples.

[0240] The matrix-vector multiplication step can be performed by multiplying the averaged samples by a matrix vector and then adding an offset vector to generate a prediction signal for the subsampled pixel set of the original block. The size of the matrix and the offset vector can be determined according to the width and height of the current block.

[0241] The horizontal / vertical interpolation step is a step of generating a prediction signal of the original block size from the subsampled prediction signal. As shown in Figure 9As shown, a prediction signal of the original block size can be generated by performing vertical and horizontal interpolation using a subsampled prediction signal and neighboring pixel values. Figure 9 An embodiment of performing MIP prediction with respect to an 8x8 block is shown. In the case of an 8x8 block, as Figure 8 shown in (b) of, a total of eight average samples can be generated. By multiplying the eight average samples by a matrix vector and adding an offset vector, as Figure 9 shown in (a) of, 16 sample values can be generated at even coordinate positions. Thereafter, as Figure 9 shown in (b) of, vertical interpolation can be performed using the average value of the top samples of the current block. Thereafter, as Figure 9 shown in (c) of, horizontal interpolation can be performed using the left sample of the current block.

[0242] The intra prediction mode for the MIP mode can be configured differently from the intra prediction modes for the above-mentioned LIP, PDPC, MRL, and ISP intra prediction or normal intra prediction. The intra prediction mode for the MIP mode can be referred to as the MIP intra prediction mode, the MIP prediction mode, or the MIP mode. For example, the matrix and offset for matrix vector multiplication can be set differently according to the intra prediction mode for MIP. Here, the matrix can be referred to as the (MIP) weight matrix, and the offset can be referred to as the (MIP) offset vector or the (MIP) bias vector.

[0243] The above intra prediction type information can include an MIP flag (e.g., intra_mip_flag) specifying whether the MIP mode is applied to the current block. When the MIP mode is applied to the current block (e.g., the value of intra_mip_flag is 1), the MPM list for the MIP mode can be configured separately. In addition, the intra prediction type information can include an MIP MPM flag (e.g., intra_mip_mpm_flag) specifying whether the MPM list is used for the MIP mode, an MPM index (e.g., intra_mip_mpm_idx) specifying the MIP mode of the current block for the MPM list, and remaining intra prediction mode information (e.g., intra_mip_mpm_remainder) for directly specifying the MIP mode when the MIP mode of the current block is not used in the MPM list.

[0244] When the MIP mode is executed, various MIP modes can be set according to the configured MIP matrix and offset. The number of intra prediction modes for MIP can be set differently based on the size of the current block. For example, i) when the height and width of the current block (e.g., CB or TB) are 4, 35 intra prediction modes (i.e., intra prediction modes 0 to 34) can be available, ii) when the height and width of the current block are both less than or equal to 8, 19 intra prediction modes (i.e., intra prediction modes 0 to 18) can be available, and iii) in other cases, 11 intra prediction modes (i.e., intra prediction modes 0 to 10) can be available.

[0245] For example, when the case where the height and width of the current block are 4 is referred to as block size type 0, the case where the height and width of the current block are both less than or equal to 8 can be referred to as block size type 1, and other cases can be referred to as block size type 2. The number of intra prediction modes for MIP can be summarized as shown in the following table. However, this is an example and the block size types and the number of available intra prediction modes can be changed.

[0246] [Table 2]

[0247] Block Size Type (MipSizeId) Number of MIP Intra Prediction Modes MIP Intra Prediction Mode 0 35 0…34 1 19 0…18 2 11 0…10

[0248] In an embodiment, information about the intra prediction mode / type of the current block can be compiled and signaled at a level such as CU (CU syntax) or can be implicitly determined according to conditions. In this case, this can be explicitly signaled for some modes / types and can be implicitly derived for the remaining modes. For example, the CU syntax can carry information about the (intra) prediction mode / type, as Figures 10 to 12 shown.

[0249] Here, pred_mode_flag can specify the prediction mode of the current CU. For example, a value of 0 for pred_mode_flag can specify that the current CU is encoded in an inter prediction mode. A value of 1 for pred_mode_flag can specify that the current CU is encoded in an intra prediction mode.

[0250] pcm_flag[ x0 ][ y0 ] can specify whether Pulse Code Modulation (PCM) mode is applied to the current block. When the PCM mode is applied to the current block, the values of the original samples in the current block can be coded and signaled without applying prediction / transformation / quantization. For example, for a luminance CU corresponding to the (x0, y0) position, pcm_flag[ x0 ][ y0 ] can specify the existence of the pcm_sample syntax and the non-existence of the transfrom_tree() syntax. For example, a value of 1 for pcm_flag[ x0 ][ y0 ] can specify the existence of the pcm_sample() syntax and the non-existence of the transform_tree() syntax. A value of 0 for pcm_flag[ x0 ][ y0 ] can specify the existence of the pcm_sample() syntax and the existence of the transform_tree() syntax.

[0251] intra_mip_flag[ x0 ][ y0 ] can specify whether the current block is predicted in MIP mode. For example, the first value of intra_mip_flag[ x0 ][ y0 ] (e.g., 0) can specify that the current block is not predicted in MIP mode. The second value of intra_mip_flag[ x0 ][ y0 ] (e.g., 1) can specify that the current block is predicted in MIP mode.

[0252] When intra_mip_flag[x0][y0] has a second value (e.g., 1), information about the MIP mode can be further obtained from the bitstream. For example, the syntax elements intra_mip_mpm_flag[x0][y0], intra_mip_mpm_idx[x0][y0], and intra_mip_mpm_remainder[x0][y0], which are information specifying the MIP mode of the current block, can be further obtained from the bitstream. When the MIP prediction mode is applied to the current block, the MPM list for MIP can be configured, and intra_mip_mpm_flag can specify whether the MIP mode of the current block exists in the MPM list of MIP (or MPM candidates). intra_mip_mpm_idx can specify the index of the candidate used as the MIP prediction mode of the current block among the candidates in the MPM list when the MIP prediction mode of the current block exists in the MPM list for MIP (i.e., the value of intra_mip_mpm_flag is 1). intra_mip_mpm_remainder can specify the MIP prediction mode of the current block when the MIP prediction mode of the current block does not exist in the MPM list for MIP (i.e., the value of intra_mip_mpm_flag is 0), and can specify any one of all MIP prediction modes or any one of the remaining modes among all MIP prediction modes except the candidate modes in the MPM list for MIP as the MIP prediction mode of the current block.

[0253] Meanwhile, when intra_mip_flag[x0][y0] has a first value (e.g., 0), information about MIP can be not obtained from the bitstream, and intra-frame prediction information other than MIP can be obtained from the bitstream. In an embodiment, intra_luma_mpm_flag[x0][y0], which specifies whether to generate the MPM list for normal intra-frame prediction, can be obtained from the bitstream.

[0254] When an intra prediction mode is applied to a current block, an MPM list can be configured for it, and intra_luma_mpm_flag can specify that there is an intra prediction mode for the current block in the MPM list (or MPM candidates). For example, a first value of intra_luma_mpm_flag (e.g., 0) can specify that there is no intra prediction mode for the current block in the MPM list. A second value of intra_luma_mpm_flag (e.g., 1) can specify that there is an intra prediction mode for the current block in the MPM list. When the value of intra_luma_mpm_flag is 1, intra_luma_not_planar_flag can be obtained from the bitstream.

[0255] intra_luma_not_planar_flag can specify whether the intra prediction mode of the current block is a planar mode. For example, a first value of intra_luma_not_planar_flag (e.g., 0) can specify that the intra prediction mode of the current block is a planar mode. A second value of intra_luma_not_planar_flag (e.g., 1) can specify that the intra prediction mode of the current block is not a planar mode.

[0256] When intra_luma_not_planar_flag is 'true' (i.e., value 1), intra_luma_mpm_idx can be parsed and compiled. In an embodiment, the planar mode can always be included as a candidate in the MPM list. However, as described above, the planar mode can be excluded from the MPM list by first signaling intra_luma_not_planar_flag, and in this case, a unified MPM list can be configured among the various intra prediction types (normal intra prediction, MRL, ISP, LIP, etc.) described above. In this case, the number of candidates in the MPM list can be reduced to 5. intra_luma_mpm_idx can specify a candidate to be used in the intra prediction mode of the current block among the candidates included in the MPM list from which the planar mode has been excluded.

[0257] Meanwhile, when the value of intra_luma_mpm_flag is 0, intra_luma_mpm_remainder can be parsed / compiled. intra_luma_mpm_remainder can specify one of all intra prediction modes as the intra prediction mode of the current block or can specify any one of the remaining modes other than the candidate modes in the MPM list as the intra prediction mode of the current block.

[0258] MPM List Configuration in MIP

[0259] When MIP is applied to the current block, the MPM list of the current block to which MIP is applied can be configured individually. The MPM list can be referred to by various names such as the MIP MPM list (or the MPM list for MIP or candMipModeList) to distinguish it from the MPM list when MIP is not applied to the current block. Hereinafter, for the sake of distinction, this is expressed as the MIP MPM list or can also be referred to as the MPM list.

[0260] The MIP MPM list can include n candidates, and for example, n can be 3. The MIP MPM list can be configured based on the left neighboring block and the top neighboring block of the current block. Here, the left neighboring block can be the uppermost block among the neighboring blocks adjacent to the left boundary of the current block. Additionally, the top neighboring block can indicate the leftmost block among the neighboring blocks adjacent to the top boundary of the current block. For example, when the coordinates of the current block are (xCb, yCb), the coordinates of the left neighboring block can be (xCb - 1, yCb) and the coordinates of the top neighboring block can be (xCb, yCb - 1). Alternatively, the left neighboring block can be the lowermost block among the neighboring blocks adjacent to the left boundary of the current block. Additionally, the top neighboring block can be the rightmost block among the neighboring blocks adjacent to the top boundary of the current block.

[0261] When MIP is applied to the left neighboring block, the first candidate intra prediction mode can be set to be the same as the MIP intra prediction mode of the left neighboring block. Here, the first candidate intra prediction mode can be expressed as candMipModeA. Additionally, for example, when MIP is applicable to the top neighboring block, the second candidate intra prediction mode can be set to be the same as the MIP intra prediction mode of the top neighboring block. Here, the second candidate intra prediction mode can be expressed as candMipModeB.

[0262] Meanwhile, the candidate intra prediction mode can be determined by comparing the sizes of the current block and the neighboring block. For example, when MIP is applied to the left neighboring block and the block size type of the left neighboring block is the same as that of the current block, the first candidate intra prediction mode (e.g., candMipModeA) can be set to be the same as the MIP intra prediction mode of the left neighboring block. Additionally, when MIP is applied to the top neighboring block and the block size type of the top neighboring block is the same as that of the current block, the second candidate intra prediction mode (e.g., candMipModeB) can be set to be the same as the MIP intra prediction mode of the top neighboring block.

[0263] Meanwhile, the left neighboring block or the top neighboring block can be encoded based on intra prediction other than MIP. For example, the left neighboring block or the top neighboring block can be encoded in another intra prediction mode other than MIP. In this case, it is not appropriate to use the normal intra prediction mode number of the neighboring block (e.g., the left neighboring block or the top neighboring block) where MIP is not applied as the candidate intra mode where MIP is applied without change. Therefore, in this case, for example, the process can be performed by treating a predetermined MIP intra prediction mode as being applied to the neighboring block where MIP is not applied. For example, when MIP is not applied to the neighboring block, the MIP intra prediction mode of the corresponding block can be determined as a specific MIP intra prediction mode value (e.g., 0, 1, or 2), thereby generating a MIP MPM list.

[0264] Alternatively, as another example, the normal intra prediction mode of the neighboring block where MIP is not applied can be mapped to the MIP intra prediction mode based on a mapping table to be used for configuring the MIP MPM list. In this case, the mapping can be performed based on the block size type of the current block. For example, as the mapping table, the mapping table according to an embodiment shown Figure 13 can be used.

[0265] Figure 13 An embodiment of a mapping table for mapping the normal intra prediction mode of a neighboring block to the MIP intra prediction mode is shown. As Figure 13 shown, IntraPredModeY[xNbX][yNbX] indicates the intra prediction mode of the neighboring block (left neighboring block / top neighboring block). Here, the intra prediction mode of the neighboring block can be the intra prediction mode of the luminance component (sample). The block size type MipSizeId indicates the block size type of the neighboring block or the current block. The numbers below the block size type values 0, 1, and 2 indicate the MIP intra prediction modes to which the normal intra prediction mode is mapped in each case of the block size type. The block size type 0 can indicate the case where the block has a size of 4x4 pixels. The block size type 1 can indicate the case where the block has a size of 4x8, 8x4, or 8x8 pixels. The block size type 2 can indicate the case where the block size is larger than 8x8 pixels.

[0266] In an embodiment, the neighboring block (e.g., the left neighboring block / top neighboring block) may be unavailable because it is outside the current picture or outside the current tile / slice, or even if MIP has been applied, the MIP intra prediction mode that is not available for the current block can be applied according to the block size type. Additionally, a predefined MIP intra prediction mode can be used as the first candidate intra prediction mode, the second candidate intra prediction mode, and the third candidate intra prediction mode. Figure 14A table showing an embodiment of a predetermined MIP intra prediction mode that can be used according to the size of the current block in this case is shown. For example, when all MIP intra prediction information of neighboring blocks is not available, a MIPMPM list can be generated based on the size of the current block according to Figure 14 The example of

[0267] In an embodiment, the MIP intra prediction mode of neighboring blocks can be obtained. In this case, when the MIP intra prediction mode of the left neighboring block is different from the MIP intra prediction mode of the top neighboring block, the MIP intra prediction mode of the left neighboring block can be set as the first candidate intra prediction mode. Additionally, the MIP intra prediction mode of the top neighboring block can be set as the second candidate intra prediction mode. Therefore, the first candidate (e.g., candMipModeList[0]) of the MIP MPM list can be set as the MIP intra prediction mode of the left neighboring block, and the second candidate (e.g., candMipModeList[1]) of the MIP MPM list can be set as the MIP intra prediction mode of the top neighboring block.

[0268] The order of intra prediction candidates in the MIP list can be changed. For example, the MIP intra prediction mode of the top neighboring block can be included as the first candidate (e.g., candMipModeList[0]) of the MIP MPM list, and the MIP intra prediction mode of the left neighboring block can be included as the second candidate (e.g., candMipModeList[1]) of the MIP MPM list.

[0269] As the third candidate intra prediction mode, a predetermined MIP intra prediction mode according to Figure 14 can be used. For example, the third candidate intra prediction mode of Figure 14 can be used as the second candidate (e.g., candMipModeList[2]) of the MIP MPM list.

[0270] In another embodiment, the third candidate intra prediction mode can be determined as a MIP intra prediction mode that does not overlap with the first candidate intra prediction mode and the second candidate intra prediction mode that can be determined according to the order of the MIP intra prediction modes shown in Figure 14 . For example, when the first candidate intra prediction mode of Figure 14 is not used in the first candidate and the second candidate of the MIP MPM list, the first candidate intra prediction mode of Figure 14 can be used as the third candidate (e.g., candMipModeList[2]) of the MIP MPM list. Otherwise, for example, when the second candidate intra prediction mode of Figure 15 is not used in the first candidate and the second candidate of the MIP MPM list,Figure 14 The second candidate intra prediction mode of Figure 13 is used as the third candidate in the MIP MPM list (e.g., candMipModeList[2]). Otherwise,

[0271] Alternatively, when the MIP intra prediction mode of the left neighboring block is the same as the MIP intra prediction mode of the top neighboring block, one of the MIP intra prediction modes of the left neighboring block and the top neighboring block can be included as the first candidate in the MIP MPM list (e.g., candMipModeList[0]), and the second candidate (e.g., candMipModeList[1]) and the third candidate (e.g., candMipModeList[2]) in the MIP MPM list can use the predetermined MIP intra prediction modes shown Figure 15 above.

[0272] As described above, the MIP intra prediction mode of the current block can be derived based on the MIP MPM list. In this case, as described above, the MPM flag that can be included in the intra prediction mode information of MIP can be referred to as intra_mip_mpm_flag, the MPM index can be referred to as intra_mip_mpm_idx, and the remaining intra prediction mode information can be referred to as intra_mip_mpm_remainder.

[0273] Determining Intra Prediction Mode Using MPM List

[0274] The intra prediction mode signaling process of the encoding device and the intra prediction mode determination process of the decoding device can be performed as follows, for example.

[0275] Figure 15 is a flowchart illustrating a method of encoding an intra prediction mode using an MPM list. The encoding device can configure the MPM list for the current block as described above (S1510).

[0276] Next, the encoding device may determine the intra prediction mode of the current block (S1520). The encoding device may perform prediction based on various intra prediction modes and determine the optimal intra prediction mode based on rate-distortion optimization (RDO). In an embodiment, the encoding device may use only the MPM candidates configured in the MPM list to determine the optimal intra prediction mode, or may determine the optimal intra prediction mode by further using the remaining intra prediction modes and the MPM candidates configured in the MPM list. For example, if the intra prediction type of the current block is a specific type other than the normal intra prediction type (e.g., LIP, MRL, or ISP), the encoding device may consider only the MPM candidates as the intra prediction mode candidates of the current block to determine the optimal intra prediction mode. In this case, the intra prediction mode of the current block may be determined only based on the MPM candidates, and in this case, the mpm flag may not be encoded / signaled. In this case, the decoding device may estimate that the mpm flag is 1 without receiving the mpm flag separately.

[0277] The encoding device may encode and output the intra prediction mode information in the form of a bitstream (S1530). In an embodiment, the encoding device may signal whether the intra prediction mode of the current block is an intra planar mode by encoding information (e.g., intra_luma_not_planar_flag) specifying whether the intra prediction mode of the current block is an intra planar mode. When the intra prediction mode of the current block is an intra planar mode, the encoding device may set the value of intra_luma_not_planar_flag to a first value (e.g., 0). Meanwhile, when the intra prediction mode of the current block is not an intra planar mode, the encoding device may set the value of intra_luma_not_planar_flag to a second value (e.g., 1).

[0278] Meanwhile, when the intra prediction mode of the current block is not the intra planar mode, the encoding device may determine and signal the intra prediction mode according to whether block-based delta pulse code modulation (BDPCM) is applied to the current block and the application direction. In an embodiment, when BDPCM is applied to the current block, the encoding device may determine the intra prediction mode according to the BDPCM application direction. For example, the encoding device may determine the intra prediction mode as a horizontal mode or a vertical mode in the same direction based on that the BDPCM application direction is any one of the horizontal direction or the vertical direction. Additionally, in this case, the encoding device may signal the intra prediction mode of the current block by encoding and signaling information (intra_bdpcm_flag) specifying whether BDPCM is applied to the current block and information (intra_bdpcm_dir_flag) specifying the BDPCM application direction. In this case, the signaling of the mpm flag may be skipped.

[0279] Meanwhile, when the prediction mode of the current block is not the intra planar mode and BDPCM is not applied, the encoding device may encode intra prediction mode information including the above mpm flag (e.g., intra_luma_mpm_flag), mpm index (e.g., intra_luma_mpm_idx), and / or remaining intra prediction mode information (e.g., intra_luma_mpm_remainder) to signal the intra prediction mode. Generally, the mpm index and the remaining intra prediction mode information are mutually alternative and may not be signaled simultaneously when specifying the intra prediction mode for a block. That is, the mpm flag value 1 and the mpm index may be signaled together, or the mpm flag value 0 and the remaining intra prediction mode information may be signaled together. However, as described above, when a specific intra prediction type is applied to the current block, the mpm flag may not be signaled and only the mpm index may be signaled. That is, in this case, the intra prediction mode information may include only the mpm index.

[0280] Meanwhile, generally, when the intra prediction mode of the current block is one of the MPM candidates in the MPM list, the encoding device may generate an mpm index (e.g., intra_luma_mpm_idx) that specifies one of the MPM candidates. If the intra prediction mode of the current block does not exist in the MPM list, the remaining intra prediction mode information (e.g., intra_luma_mpm_remainder) that specifies the mode identical to the intra prediction mode of the current block among the remaining intra prediction modes not included in the MPM list may be generated. For example, when the intra prediction mode of the current block (e.g., IntraPredModeY) is encoded as intra_luma_mpm_remainder, the encoding device may first subtract 1 from IntraPredModeY, arrange the intra prediction modes belonging to the MPM list in descending order according to the magnitude of the intra prediction mode values, and while comparing the values from candModeList[0] to candModeList[4] with the value of IntraPredModeY, determine the value of IntraPredModeY determined by reducing the value of IntraPredModeY by 1 when the value of IntraPredModeY - 1 is less than the value of candModeList[] as intra_luma_mpm_remainder.

[0281] Meanwhile, when the intra prediction mode of the current block is the MIP mode, the encoding device may generate an MPM list for the MIP mode and encode the current block as described above. In this case, the MPM coding information for the MIP mode may be signaled. In this case, the MPM flag may be signaled as intra_mip_mpm_flag, the MPM index may be signaled as intra_mip_mpm_idx, and the remaining intra prediction mode information may be signaled as intra_mip_mpm_remainder.

[0282] Figure 16 is a flowchart illustrating a method of performing decoding by a decoding device using an MPM list according to an embodiment. The decoding device may determine the intra prediction mode according to the intra prediction mode information determined and signaled by the encoding device.

[0283] Referring to Figure 16 , the decoding device may obtain the intra prediction mode information from the bitstream (S1610). The intra prediction mode information may include at least one of the mpm flag, the mpm index, or the remaining intra prediction mode as described above.

[0284] The decoding device may configure the MPM list (S1620). The MPM list may be configured to be the same as the MPM list configured by the encoding device. That is, the MPM list may include the intra prediction modes of neighboring blocks and also include specific intra prediction modes according to a predetermined method.

[0285] In an embodiment, the decoding device may determine whether the intra prediction mode of the current block is the intra planar mode based on information (e.g., intra_luma_not_planar_flag) specifying whether the intra prediction mode of the current block is not the intra planar mode. When the value of intra_luma_not_planar_flag is a first value (e.g., 0), the decoding device may determine that the intra prediction mode of the current block is the intra planar mode. Meanwhile, when the value of intra_luma_not_planar_flag is a second value (e.g., 1), the decoding device may determine that the intra prediction mode of the current block is not the intra planar mode.

[0286] Meanwhile, when the intra prediction mode of the current block is not the intra planar mode, the decoding device may determine the intra prediction mode according to whether block-based differential pulse code modulation (BDPCM) is applied to the current block and the application direction. In an embodiment, when the information (intra_bdpcm_flag) obtained from the bitstream specifying whether BDPCM is applied to the current block specifies the application of BDPCM, the decoding device may determine at least one BDPCM application direction in the horizontal direction or the vertical direction based on the information (intra_bdpcm_dir_flag) specifying the application direction of BDPCM obtained from the bitstream. Additionally, in the direction same as the determined BDPCM application direction, the intra prediction mode may be determined as the horizontal mode or the vertical mode.

[0287] Meanwhile, when the prediction mode of the current block is not the intra planar mode and BDPCM is not applied, the decoding device may use the above method to generate the MPM list to determine the intra prediction mode. For example, the MPM list may be determined based on the intra prediction modes of neighboring blocks of the current block. The decoding device may determine the MPM list based on the intra prediction modes of the top neighboring block and the left neighboring block of the current block. For example, in an embodiment, the decoding device may determine the MPM list based on a first intra prediction candidate determined based on the intra prediction mode of the left neighboring block and a second intra prediction candidate determined based on the intra prediction mode of the top neighboring block.

[0288] The decoding device can use the MPM list to determine whether to determine the intra prediction mode of the current block (S1630). For example, when the value of the mpm flag is 1, the decoding device can derive the candidate specified by the mpm index among the MPM candidates in the MPM list as the intra prediction mode of the current block. For example, the decoding device can determine the intra prediction mode of the current block according to the value of intra_luma_mpm_idx which is the mpm index. For example, the decoding device can determine candModeList[intra_luma_mpm_idx] as the intra prediction mode of the current block.

[0289] As another example, when the value of the mpm flag is 0, the decoding device can derive the intra prediction mode specified by the remaining intra prediction mode information that is not included in the MPM list as the intra prediction mode of the current block (S1640).

[0290] For example, the decoding device can determine the intra prediction mode (e.g., IntraPredModeY) of the current block based on the remaining intra prediction mode information (e.g., intra_luma_mpm_remainder) that specifies the intra prediction mode of the current block. For example, the decoding device can set the value of IntraPredModeY to intra_luma_mpm_remainder + 1. Thereafter, the decoding device can arrange the intra prediction modes belonging to the MPM list in ascending order according to the magnitude of the intra prediction mode values, and while comparing with the value of IntraPredModeY from candModeList[0] to candModeList[4], determine the value of IntraPredModeY that specifies the intra prediction mode of the current block by incrementing the value of IntraPredModeY by one when the value of IntraPredModeY is less than the value of candModeList[].

[0291] Meanwhile, as another example, when the intra prediction type of the current block is a specific type (e.g., LIP, MRL, or ISP), the decoding device can derive the candidate specified by the mpm index in the MPM list as the intra prediction mode of the current block without checking the mpm flag.

[0292] Meanwhile, when the intra prediction mode of the current block is the MIP mode, the decoding device may generate an MPM list for MIP to decode the current block as described above. In this case, the MPM coding information of the MIP mode can be obtained from the bitstream. In this case, the MPM flag can be obtained through intra_mip_mpm_flag, the MPM index can be obtained using intra_mip_mpm_idx, and the remaining intra prediction mode information can be obtained using intra_mip_mpm_remainder.

[0293] Mapping Problem between MIP Intra Prediction Mode and Normal Intra Prediction Mode

[0294] As described above, to determine the MIP mode or intra prediction mode of the current block, an MPM list for the normal intra prediction mode or an MPM list for MIP can be generated based on information about neighboring blocks. In this case, the neighboring blocks may include the left neighboring block and the top neighboring block of the current block. Here, the normal intra prediction mode means an intra prediction mode other than the MIP mode. For example, the normal intra prediction mode may mean the intra planar mode and the intra DC mode as non - directional intra prediction modes and directional intra prediction modes.

[0295] When the MIP mode is applied to the current block but an intra prediction mode other than the MIP mode (normal intra prediction mode) is applied to the neighboring blocks, it is necessary to map the intra prediction mode of the neighboring blocks to the MIP mode to generate the MPM list of the current block using the prediction information of the neighboring blocks. Additionally, when the normal intra prediction mode is applied to the current block but the MIP mode is applied to the neighboring blocks, it is necessary to map the MIP mode of the neighboring blocks to the normal intra prediction mode to generate the MPM list of the current block using the prediction information of the neighboring blocks.

[0296] However, the reason why the MIP mode has problems is that since the MIP mode can have various numbers of prediction modes according to the size of the luma block as follows, it is difficult to map the normal intra prediction mode and the MIP mode in a one - to - one correspondence.

[0297] [Table 3]

[0298] Luminance Block Size Number of MIP Modes 4x4 Luminance Block 35 MIP Modes 4x8, 8x4, 8x8 Luminance Blocks 19 MIP Modes Other Luminance Blocks 11 MIP Modes

[0299] Since the number of normal intra prediction modes and the number of MIP modes are different, in order to interpolate and map them, it can be done through Figure 7 and Figure 13Perform the mapping between the MIP mode and the normal intra prediction mode using the mapping table shown. For example, when generating the MPM list of the current block encoded in the normal intra mode by referring to neighboring blocks, if the intra prediction mode of the neighboring block is the MIP mode, in order to map the MIP mode of the neighboring block to the intra prediction mode, it should be as Figure 17 shown to generate the MPM list. More specifically, during the encoding and decoding processes, the encoding device and the decoding device can identify that the prediction mode of the current block is the normal intra prediction mode (S1710), and identify that the prediction mode of the neighboring block is the MIP mode (S1720). When the prediction mode of the neighboring block is the MIP mode, the encoding device and the decoding device can check whether the neighboring block is a 4x4 luma block (S1730). When the neighboring block is a 4x4 luma block, the encoding device and the decoding device can determine the normal intra prediction mode corresponding to the MIP mode of the neighboring block according to the method of mapping Figure 7 35 MIP modes to 67 intra modes (S1740). When the neighboring block is not a 4x4 luma block, the encoding device and the decoding device can check whether the neighboring block is a 4x8, 8x4, or 8x8 luma block (S1750). When the neighboring block is a 4x8, 8x4, or 8x8 luma block, the encoding device and the decoding device can determine the normal intra prediction mode corresponding to the MIP mode of the neighboring block according to the method of mapping Figure 7 19 MIP modes to 67 intra modes (S1760). Alternatively, when the neighboring block is not a 4x8, 8x4, or 8x8 luma block, the encoding device and the decoding device can determine the normal intra prediction mode corresponding to the MIP mode of the neighboring block according to the method of mapping Figure 7 11 MIP modes to 67 intra modes (S1770). Finally, the encoding device and the decoding device can generate the MPM list of the current block using the determined normal intra prediction mode according to the above method (S1780).

[0300] In a similar manner, when generating the MPM list of the current block encoded in the MIP mode by referring to neighboring blocks, if the intra prediction mode of the neighboring block is the normal intra prediction mode, steps S1810 to S1880 should be performed as Figure 18 shown to map the intra prediction mode of the neighboring block to the MIP mode.

[0301] However, when performing such mapping, due to the correlation between the MIP mode and the intra prediction mode, it is necessary to perform a comparison between the sizes of the current block and the neighboring block, and additional memory for storing such a mapping table is required.

[0302] Mapping of MIP Intra Prediction Mode to Normal Intra Prediction Mode

[0303] Hereinafter, a mapping method according to an embodiment for reducing the complexity of a mapping algorithm by removing the correlation between the block size and the MIP mode and the intra prediction mode and saving the memory for storing the mapping table will be described.

[0304] When the MIP mode is mapped to a normal intra prediction mode, an encoding device and a decoding device according to an embodiment may determine the MIP mode as a predetermined intra prediction mode without using the block size and the mapping table.

[0305] For example, when the MIP mode is converted to an intra prediction mode, an encoding device and a decoding device according to an embodiment may map all MIP modes to the intra planar mode.

[0306] Alternatively, when the MIP mode is converted to an intra prediction mode, an encoding device and a decoding device according to an embodiment may map all MIP modes to the intra DC mode.

[0307] Alternatively, when the MIP mode is converted to an intra prediction mode, an encoding device and a decoding device according to an embodiment may map all MIP modes to the intra vertical mode.

[0308] Alternatively, when the MIP mode is converted to an intra prediction mode, an encoding device and a decoding device according to an embodiment may map all MIP modes to the intra horizontal mode.

[0309] In an embodiment, in order to determine the intra prediction mode of a current block, when searching for the intra prediction mode of neighboring blocks to generate an MPM list, if MIP prediction is applied to a neighboring block, the intra prediction mode of the neighboring block may be derived as the intra planar mode, thereby generating the current block MPM list.

[0310] Meanwhile, in the case where the current block (or coding unit) includes a luma block and a chroma block, when configuring the intra prediction mode of the chroma block, if MIP prediction is applied to the luma block corresponding to the position of the chroma block, the intra prediction mode specified by the DM (direct mode, using the intra prediction mode of the luma block corresponding to the chroma block) of the chroma block may be derived as the intra planar mode.

[0311] By mapping the MIP mode to the intra prediction mode, an encoding device or a decoding device may simply determine that all MIP modes are predetermined normal intra prediction modes and, when generating an MPM list when encoding or decoding the current block in the normal intra mode, generate the MPM list based on the corresponding normal intra prediction mode. Therefore, the MPM list generation step can be simplified as Figure 19 shown in the reference Figure 17 described. The reference Figure 19 , in the reference Figure 17In the described MPM list generation steps, it can be seen that steps S1730 to S1780 are simplified to step S1791 of determining the normal intra prediction mode corresponding to the MIP mode when all MIP modes are mapped to a predetermined normal intra prediction mode and step S1792 of generating the MPM list according to the determined normal intra prediction mode. Here, the predetermined normal intra prediction mode can be any one of an intra plane mode, an intra DC mode, an intra vertical mode, and an intra horizontal mode.

[0312] Similarly, even when the above intra prediction mode of the chrominance block is determined, when the luminance block corresponding to the chrominance block is in the MIP mode, the intra prediction mode corresponding to the luminance block can be determined as a predetermined normal intra prediction mode without performing mapping according to the size.

[0313] Hereinafter, reference will be made to Figure 20 an image encoding method performed by an encoding device according to an embodiment. The encoding device according to the embodiment may include a memory and at least one processor, and the following encoding method may be executed by the at least one processor.

[0314] The encoding device according to the embodiment may identify the prediction mode of the current block (S2010). When the prediction mode of the current block is an intra prediction mode, the encoding device may determine candidate intra prediction modes based on the prediction modes of neighboring blocks located around the current block (S2020). The candidate intra prediction modes may include a first candidate intra prediction mode and a second candidate intra prediction mode. The first candidate intra prediction mode may be determined based on the prediction mode of the first neighboring block located around the current block, and the second candidate intra prediction mode may be determined based on the prediction mode of the second neighboring block located around the current block. Here, the first candidate intra prediction mode may be the above first intra prediction candidate, and the second candidate intra prediction mode may be the above second intra prediction candidate. For example, the encoding device may determine the first candidate intra prediction mode (e.g., candIntraPredModeA) based on the intra prediction mode of the left neighboring block and determine the second candidate intra prediction mode (e.g., candIntraPredModeB) based on the intra prediction mode of the top neighboring block.

[0315] In this case, when the prediction mode of an adjacent block is the MIP mode, the encoding device may determine the candidate intra prediction mode of the corresponding adjacent block as a predetermined intra prediction mode. Here, the predetermined intra prediction mode may be any one of an intra planar mode, an intra DC mode, an intra horizontal mode, and an intra vertical mode. For example, when the intra prediction mode of the left adjacent block is the MIP mode, the encoding device may determine the first candidate intra prediction mode (e.g., candIntraPredModeA) as any one of an intra planar mode, an intra DC mode, an intra horizontal mode, and an intra vertical mode. Alternatively, when the intra prediction mode of the top adjacent block is the MIP mode, the encoding device may determine the second candidate intra prediction mode (e.g., candIntraPredModeB) as any one of an intra planar mode, an intra DC mode, an intra horizontal mode, and an intra vertical mode.

[0316] Next, the encoding device may generate a candidate intra prediction mode list for the current block based on the candidate intra prediction modes (S2030). The candidate intra prediction mode list may be the above-mentioned MPM list. For example, the encoding device may generate the candidate intra prediction mode list based on the first candidate intra prediction mode and the second candidate intra prediction mode as described above. In this case, when the prediction modes of the first adjacent block and the second adjacent block are both the MIP mode, the encoding device may determine that the candidate intra prediction mode list includes predetermined candidate intra prediction modes. Here, the predetermined candidate intra prediction modes may be at least one of a DC mode or a vertical mode.

[0317] Next, the encoding device may encode an intra prediction mode indicator indicating the intra prediction mode of the current block based on the candidate intra prediction mode list (S2040). Here, the intra prediction mode indicator may include an mpm flag signaled in the form of an intra_luma_mpm_flag syntax element, an mpm index signaled in the form of an mpm_idx or intra_luma_mpm_idx syntax element, or remaining intra prediction mode information signaled in the form of a rem_intra_luma_pred_mode or intra_luma_mpm_remainder syntax element. The encoding device may generate a bitstream by encoding the intra prediction mode indicator and send it to the decoding device.

[0318] Hereinafter, reference will be made to Figure 21 Describe an image decoding method performed by a decoding device according to an embodiment. The decoding device according to an embodiment may include a memory and at least one processor, and the following decoding method may be performed by the at least one processor.

[0319] First, the decoding apparatus according to the embodiment can identify the prediction mode of the current block (S2110). When the prediction mode of the current block is an intra prediction mode, the decoding apparatus can determine candidate intra prediction modes of the current block based on the prediction modes of neighboring blocks located around the current block (S2120).

[0320] When the prediction mode of the neighboring block is the MIP mode, the decoding apparatus can determine the candidate intra prediction mode as a predetermined intra prediction mode. Here, the predetermined intra prediction mode can be any one of an intra planar mode, an intra DC mode, an intra horizontal mode, and an intra vertical mode.

[0321] The decoding apparatus can determine whether the prediction mode of the neighboring block is the MIP mode based on the MIP mode indicator of the neighboring block. The MIP mode indicator can be the above-mentioned MIP flag (e.g., intra_mip_flag), and the decoding apparatus can obtain the MIP mode indicator from the bitstream.

[0322] The candidate intra prediction modes can include a first candidate intra prediction mode and a second candidate intra prediction mode. In this case, the first candidate intra prediction mode can be determined based on the prediction mode of the first neighboring block located around the current block, and the second candidate intra prediction mode can be determined based on the prediction mode of the second neighboring block located around the current block.

[0323] Here, the first candidate intra prediction mode can be the above-mentioned first intra prediction candidate, and the second candidate intra prediction mode can be the above-mentioned second intra prediction candidate. For example, the decoding apparatus can determine the first candidate intra prediction mode (e.g., candIntraPredModeA) based on the intra prediction mode of the left neighboring block and determine the second candidate intra prediction mode (e.g., candIntraPredModeB) based on the intra prediction mode of the top neighboring block.

[0324] For example, when the intra prediction mode of the left neighboring block is the MIP mode, the decoding apparatus can determine the first candidate intra prediction mode (e.g., candIntraPredModeA) as any one of an intra planar mode, an intra DC mode, an intra horizontal mode, and an intra vertical mode. Alternatively, when the intra prediction mode of the top neighboring block is the MIP mode, the decoding apparatus can determine the second candidate intra prediction mode (e.g., candIntraPredModeB) as any one of an intra planar mode, an intra DC mode, an intra horizontal mode, and an intra vertical mode.

[0325] In addition, the decoding device may generate a candidate list of intra prediction modes for the current block based on candidate intra prediction modes (S2130). The candidate list of intra prediction modes may be the above-mentioned MPM list. For example, the decoding device may generate a candidate list of intra prediction modes based on a first candidate intra prediction mode and a second candidate intra prediction mode as described above. In this case, when the prediction modes of the first neighboring block and the second neighboring block are both MIP modes, the decoding device may determine that the candidate list of intra prediction modes includes predetermined candidate intra prediction modes. Here, the predetermined candidate intra prediction modes may be at least one of the DC mode or the vertical mode.

[0326] In addition, when the first candidate intra prediction mode and the second candidate intra prediction mode are the same and the first candidate intra prediction mode is an intra prediction mode having a prediction mode value greater than the specified DC mode value, the decoding device may generate a candidate list of intra prediction modes including the value of the first candidate intra prediction mode.

[0327] In addition, when the prediction mode of the first neighboring block is an MIP mode, the first candidate intra prediction mode and the second candidate intra prediction mode are different from each other, and the second candidate intra prediction mode is an intra prediction mode having a prediction mode value greater than the value indicating the DC mode, the decoding device may generate a candidate list of intra prediction modes including the second candidate intra prediction mode.

[0328] In addition, the decoding device may determine the intra prediction mode of the current block based on the candidate list of intra prediction modes (S2140). The decoding device may determine any one of the candidate intra prediction modes included in the candidate list of intra prediction modes as the intra prediction mode of the current block based on the intra prediction mode indicator obtained from the bitstream. For example, the intra prediction mode indicator may be the above-mentioned mpm index and may be signaled in the bitstream in the form of the mpm_idx or intra_luma_mpm_idx syntax element.

[0329] In addition, the encoding device according to an embodiment may encode the intra prediction mode of a chrominance block according to the mapping of the above-mentioned MIP mode. The encoding device according to an embodiment may use the DM mode to signal the intra prediction mode of the chrominance block. In this case, the encoding device may determine the intra prediction mode applied according to the DM mode as the intra prediction mode specified by the reference mode. Here, the reference mode may be determined based on the prediction mode of the luminance block corresponding to the chrominance block, and the reference mode may be identified by the parameter of lumaIntraPredMode or IntraPredModeY.

[0330] For example, the encoding device may determine the intra prediction mode of the luminance block corresponding to the chrominance block as a reference mode. Therefore, the encoding device may determine the intra prediction mode of the chrominance block determined in the DM mode as the intra prediction mode of the luminance block.

[0331] In this case, when the luminance block is a luminance block to which the MIP mode is applied, the encoding device may determine the reference mode as the planar mode instead of the MIP mode. Therefore, the encoding device may determine the intra prediction mode of the chrominance block determined to be in the DM mode as the intra planar mode.

[0332] Alternatively, when the MIP mode is not applied to the luminance block, the encoding device may determine the reference mode according to the prediction mode of the luminance block. For example, when the luminance block is predicted in a predetermined mode, the encoding device may determine the reference mode as the intra DC mode. Here, the predetermined mode may include the IBC mode or other modes. Therefore, the encoding device may determine the intra prediction mode of the chrominance block determined to be in the DM mode as the intra DC mode.

[0333] In addition, the encoding device may encode the intra prediction mode of the chrominance block based on the reference mode. For example, the encoding device may select the intra planar mode as the best prediction mode for encoding the chrominance block, and when the prediction mode of the luminance block corresponding to the chrominance block is the MIP mode, encode the information indicating that the intra prediction mode of the chrominance block is the intra prediction mode recognized according to the DM mode.

[0334] In addition, consistent with the encoding method, the decoding device according to an embodiment may determine the intra prediction mode of the chrominance block according to the mapping of the above MIP mode. The decoding device according to an embodiment may determine a reference mode for determining the intra prediction mode of the chrominance block based on the prediction mode of the luminance block corresponding to the chrominance block. Here, the reference mode may be identified by parameters of lumaIntraPredMode or IntraPredModeY.

[0335] In this case, when the luminance block corresponding to the chrominance block is a luminance block to which the MIP mode is applied, the decoding device may determine the reference mode as the planar mode. Therefore, the decoding device may determine the intra prediction mode of the chrominance block determined to be in the DM mode as the intra planar mode.

[0336] Alternatively, when the MIP mode is not applied to the luminance block, the decoding device may determine the reference mode according to the prediction mode of the luminance block. For example, when the luminance block is predicted in the IBC mode or other predetermined mode, the decoding device may determine the reference mode as the intra DC mode. Therefore, the decoding device may determine the intra prediction mode of the chrominance block determined to be in the DM mode as the intra DC mode.

[0337] Alternatively, when the MIP mode is not applied to the luminance block and the luminance block is not predicted in the IBC mode or other predetermined mode, the decoding device may determine the reference mode as the intra-frame prediction mode of the luminance block. Therefore, the decoding device may determine the intra-frame prediction mode of the chrominance block that has been determined as the DM mode as the intra-frame prediction mode of the luminance block.

[0338] In addition, the decoding device may determine the intra prediction mode of the chroma block based on the reference mode. For example, when the intra prediction mode of the chroma mode is the DM mode, the decoding device may determine the intra prediction mode of the chroma block to be the intra prediction mode corresponding to the reference mode.

[0339] Therefore, even when the prediction mode of the luminance block or the neighboring block referenced when encoding or decoding the current block in the normal intra-frame mode is the MIP mode, the encoding device and the decoding device do not need to compare the size of the current block or the neighboring block, thereby reducing the computational complexity. In addition, since there is no need to use a mapping table for mapping, the memory space efficiency can be improved.

[0340] Figure 22 Experimental data are shown, which show the use of Figure 17 The mapping table method shown is compared to the method when the MIP mode of the neighboring block is converted into the intra prediction mode according to Figure 19 The mapping method maps all MIP modes to the intra-frame intra-plane modes to generate the coding rate of the MPM list of the current block. Figure 22 As shown, it can be seen that there is no difference in the coding rate. That is, by applying the above method, the algorithm complexity can be reduced while minimizing the coding loss and reducing the use of the memory for the mapping table.

[0341] Mapping of Normal Intra Prediction Mode to MIP Intra Prediction Mode

[0342] Hereinafter, a mapping method for reducing the complexity of a mapping algorithm and saving a memory for storing a mapping table by removing the correlation between a block size and an MIP mode and an intra prediction mode according to another embodiment will be described.

[0343] When the normal intra prediction mode is mapped to the MIP mode, the encoding apparatus and the decoding apparatus according to an embodiment may determine all normal intra prediction modes as a predetermined MIP mode without using a block size and a mapping table.

[0344] For example, when the normal intra prediction mode is converted into the MIP mode, the encoding apparatus and the decoding apparatus according to an embodiment may map all normal intra prediction modes to MIP mode #0.

[0345] Alternatively, when the normal intra prediction mode is converted into the MIP mode, the encoding device and the decoding device according to the embodiment may map all normal intra prediction modes to MIP mode #1.

[0346] Alternatively, when the normal intra prediction mode is converted into the MIP mode, the encoding device and the decoding device according to the embodiment may map all normal intra prediction modes to MIP mode #3.

[0347] Alternatively, when the normal intra prediction mode is converted into the MIP mode, the encoding device and the decoding device according to the embodiment may map all normal intra prediction modes to the MIP mode having the most likely selection rate during the encoding or decoding process.

[0348] By mapping the MIP mode to the intra prediction mode, the encoding device or the decoding device may simply determine all normal intra prediction modes as a predetermined MIP mode when generating the MPM list when encoding or decoding the current block with the MIP, and generate the MPM list based on the corresponding MIP mode. Therefore, as Figure 23 shown, the reference Figure 18 described MPM list generation steps can be simplified. Referring to Figure 23 , in the MPM list generation steps described in the reference Figure 18 , it can be seen that steps S1830 to S1880 are simplified to step S1891 (S1891) of determining the MIP mode corresponding to the normal intra prediction mode as all normal intra prediction modes are mapped to the predetermined MIP mode, and step S1892 (S1892) of generating the MPM list with the determined MIP mode. Here, the predetermined MIP mode may be any one of #0, #1, #3 and the MIP mode having the most likely selection rate during the encoding or decoding process.

[0349] Figure 24 shows experimental data, which shows the encoding rate when generating the MPM list for the current block by mapping all normal intra prediction modes to MIP mode #0 according to the above mapping method when the normal intra prediction mode of the neighboring block is converted into the MIP mode compared to the encoding rate when generating the MPM list described in the reference Figure 18 . As Figure 24 shown, it can be seen that there is no difference in terms of the encoding rate. That is, by applying the above method, the algorithm complexity can be reduced while minimizing the encoding loss and reducing the use of memory for the mapping table.

[0350] Alternatively, the encoding device and the decoding device according to the embodiment may use the simplified mapping table shown in Table 4 below to convert the normal intra prediction mode into the MIP mode.

[0351] [Table 4]

[0352]

[0353] For example, the encoding device and the decoding device according to the embodiment can map all normal intra prediction modes to MIP mode #17, 0, or 1 according to the size of the current block (MipSizeId).

[0354] As described above, the size 0 of the current block may mean a 4x4 luma block, the size 1 of the current block may mean a 4x8, 8x4, or 8x8 luma block, and the size 2 of the current block may mean a luma block larger than 8x8.

[0355] Alternatively, the encoding device and the decoding device according to the embodiment can use the simplified mapping table shown in Table 5 below to convert the normal intra prediction mode into the MIP mode.

[0356] [Table 5]

[0357]

[0358] For example, the encoding device and the decoding device according to the embodiment can map all normal intra prediction modes to MIP mode #5, 0, or 6 according to the size of the current block (MipSizeId). Alternatively, the encoding device and the decoding device according to the embodiment can use the simplified mapping table shown in Table 6 below to convert the normal intra prediction mode into the MIP mode.

[0359] [Table 6]

[0360]

[0361] For example, the encoding device and the decoding device according to the embodiment can map all normal intra prediction modes to the MIP mode with the highest likelihood selection rate for each block size according to the size of the current block (MipSizeId). The encoding device and the decoding device according to the embodiment can reduce the algorithm complexity by using the simplified mapping table, but in terms of comparing the sizes of the blocks, a more complex mapping can be performed compared to the above mapping method that maps all normal intra prediction modes to the MIP mode without comparing the sizes of the blocks.

[0362] Method for Generating MPM List of MIP Modes

[0363] As described above, when the prediction mode of the current block is the MIP mode, it is necessary to check the MIP modes of adjacent blocks to generate the MPM list of the current block. Figure 25 is a flowchart illustrating a method for determining candidate MIP modes for configuring the MPM list of the current block according to an embodiment.

[0364] Reference Figure 25 , in an embodiment, even if the prediction mode of an adjacent block is the MIP mode (S2510), when the number of MIP modes that the current block and the adjacent block can have is the same, that is, when the sizes of the current block and the adjacent block are the same (S2520), the encoding apparatus and the decoding apparatus may also determine the MIP mode of the adjacent block as a candidate MIP mode for configuring the MPM list of the current block (S2530). For example, even if the prediction mode of the adjacent block is the MIP mode (S2510), when the number of MIP modes that the current block and the adjacent block can have is different, that is, when the sizes of the current block and the adjacent block are different (S2520), the encoding apparatus and the decoding apparatus may determine the value of the candidate MIP mode for configuring the MPM list of the current block as -1 (S2540). The value -1 of the candidate MIP mode may specify that the MIP mode value from the adjacent block cannot be used.

[0365] In addition, when the prediction mode of the adjacent block is not the MIP mode (S2510), the encoding apparatus and the decoding apparatus may convert the normal intra prediction mode into a candidate MIP mode as described in Reference Figure 18 as Figure 18 described (S2550).

[0366] As in the method of Figure 25 , the encoding apparatus and the decoding apparatus should always check the sizes of the current block and the adjacent block in the process of referring to the adjacent block to determine the candidate MIP mode of the current block, and should perform the mapping as described in Reference Figure 18 when the prediction mode of the adjacent block is not the MIP mode, thereby increasing the computational complexity.

[0367] To reduce the computational complexity, the encoding apparatus and the decoding apparatus according to the embodiment may check whether the adjacent block is in the MIP mode when generating the MPM list of the current block encoded or decoded in the MIP mode and accordingly determine the candidate MIP mode. For example, when the encoding or decoding mode of the adjacent block is the MIP mode, the encoding apparatus and the decoding apparatus may set the candidate MIP mode to mode #0. Alternatively, when the encoding or decoding mode of the adjacent block is not the MIP mode, the encoding apparatus and the decoding apparatus may set the value of the MIP mode to -1. Therefore, since the encoding apparatus and the decoding apparatus need to only check whether the MIP mode is applicable to the adjacent block, the algorithm for determining the candidate MIP mode can be more simplified, and when the adjacent block is in the normal intra prediction mode, the mapping process for converting it into the MIP mode can be skipped.

[0368] Meanwhile, the encoding device and the decoding device can determine candidate MIP modes based on the sizes of the current block and neighboring blocks to improve prediction accuracy. For example, when the current block is in the MIP mode, referring to neighboring blocks to generate the MPM list and the prediction mode of the neighboring block is the MIP mode, the encoding device and the decoding device can determine the candidate MIP mode as mipMpmCand[sizeId][0] by referring to Table 7 below. sizeId can mean the size of the neighboring block, sizeId 0 can mean a 4x4 luma block, sizeId 1 can mean a 4x8, 8x4, or 8x8 luma block, and sizeId 2 can mean a luma block larger than 8x8.

[0369] [Table 7]

[0370]

[0371] For example, the encoding device and the decoding device can set the candidate MIP mode to #17 when the size of the neighboring block is 4x4, set the candidate MIP mode to #0 when the size of the neighboring block is 4x8, 8x4, or 8x8, and set the candidate MIP mode to #1 in other blocks. The encoding device and the decoding device can improve the MPM mode accuracy by adaptively selecting the default candidate MIP mode according to the size of the neighboring block. Alternatively, to reduce the computational complexity, the encoding device and the decoding device according to an embodiment can select the candidate MIP mode without considering the coding mode of the neighboring block and generate the MPM list by using it without change.

[0372] For example, when generating the MPM list for the MIP mode, the encoding device and the decoding device can fixedly determine the MPM list (e.g., candMipModeList[]) for the MIP mode as follows without considering the MIP mode of the neighboring block. For example, when generating three MIP MPM lists, x can have values from 0 to 2, so candMipModeList[x] can be configured as follows with reference to Table 7. In this case, sizeId represents the size of the neighboring block, but the encoding device and the decoding device can determine sizeId according to the size of the current block so as to skip the process of referring to the information about the neighboring block.

[0373] candMipModeList[0] = mipMpmCand [sizeId][0]

[0374] candMipModeList[1] = mipMpmCand[sizeId][1]

[0375] candMipModeList[2] = mipMpmCand[sizeId][2]

[0376] Figure 26 Shows experimental data that shows the coding rate when encoding an image by generating an MPM list from candidate MIP modes determined by the method according to Figure 25 compared to the coding rate when encoding an image by fixedly determining an MPM list for MIP modes as described above without considering the coding modes of neighboring blocks when generating the MPM list from candidate MIP modes determined by the above mapping method. As Figure 26 shown, no difference in coding rate can be seen. That is, by applying the above method, the algorithm complexity can be reduced while minimizing the coding loss and reducing the use of memory for the mapping table.

[0377] In another embodiment, when generating an MPM list for MIP modes, the encoding device and the decoding device may fixedly determine the MPM list (e.g., candMipModeList[]) for MIP modes based on mode selection probabilities without considering the coding modes of neighboring blocks as follows. For example, when generating three MIP MPM lists, x may have values from 0 to 2, and candMipModeList[x] may be configured as follows with respect to Table 8. In sortedmipMpmCand[sizeId][x], candidate MIP modes may be stored for each block size based on the MIP mode selection probability. For example, the candidate MIP mode with the highest selection frequency in the corresponding sizeId may be stored in sortedmipMpmCand[sizeId][0], and the candidate MIP mode with the second highest selection frequency in the corresponding sizeId may be stored in sortedmipMpmCand[sizeId][1]. In this case, sizeId represents the size of neighboring blocks, but the encoding device and the decoding device may determine sizeId according to the size of the current block in order to skip the process of referring to information about neighboring blocks.

[0378] candMipModeList[0] = sortedmipMpmCand[sizeId][0]

[0379] candMipModeList[1] = sortedmipMpmCand[sizeId][1]

[0380] candMipModeList[2] = sortedmipMpmCand[sizeId][2]

[0381] [Table 8]

[0382]

[0383] Application Example

[0384] Although, for clarity of description, the exemplary methods of the present disclosure are shown as a series of operations, it is not intended to limit the order of execution of the steps, and these steps may be performed simultaneously or in a different order when necessary. To implement the method according to the present invention, the described steps may further include other steps, may include the remaining steps except for some steps, or may include other additional steps except for some steps.

[0385] In the present disclosure, an image encoding device or an image decoding device that performs a predetermined operation (step) may perform an operation (step) of confirming the execution conditions or circumstances of the corresponding operation (step). For example, if it is described that a predetermined operation is performed when a predetermined condition is satisfied, the image encoding device or the image decoding device may perform the predetermined operation after determining whether the predetermined condition is satisfied.

[0386] The various embodiments of the present disclosure are not a list of all possible combinations and are intended to describe representative aspects of the present disclosure, and the matters described in the various embodiments may be applied independently or in combinations of two or more.

[0387] The various embodiments of the present disclosure may be implemented in hardware, firmware, software, or a combination thereof. In the case where the present disclosure is implemented by hardware, the present disclosure may be implemented by an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a general purpose processor, a controller, a microcontroller, a microprocessor, etc.

[0388] In addition, the image decoding device and the image encoding device according to embodiments of the present disclosure may be included in a multimedia broadcast transmission and reception device, a mobile communication terminal, a home theater video device, a digital cinema video device, a surveillance camera, a video chat device, a real-time communication device such as video communication, a mobile streaming device, a storage medium, a camera, a video-on-demand (VoD) service providing device, an over-the-top (OTT) video device, an Internet streaming service providing device, a three-dimensional (3D) video device, a video phone video device, a medical video device, etc., and may be used to process video signals or data signals. For example, the OTT video device may include a game console, a Blu-ray player, an Internet-connected TV, a home theater system, a smart phone, a tablet PC, a digital video recorder (DVR), etc.

[0389] Figure 27 is a view showing a content streaming system to which embodiments of the present disclosure can be applied.

[0390] As Figure 27 shown, the content streaming system to which embodiments of the present disclosure are applied may mainly include an encoding server, a streaming server, a network server, a media storage, a user device, and a multimedia input device.

[0391] The encoding server compresses the content input from a multimedia input device such as a smart phone, a camera, a portable video camera, etc. into digital data to generate a bitstream and sends the bitstream to the streaming server. As another example, when a multimedia input device such as a smart phone, a camera, a video camera, etc. directly generates a code stream, the encoding server may be omitted.

[0392] The bitstream may be generated by the image encoding method or the image encoding device according to embodiments of the present disclosure, and the streaming server may temporarily store the bitstream during the process of sending or receiving the bitstream.

[0393] The streaming server sends multimedia data to the user device based on a request from the user through the network server, and the network server serves as a medium for notifying the user of the service. When the user requests a desired service from the network server, the network server may deliver it to the streaming server, and the streaming server may send the multimedia data to the user. In this case, the content streaming system may include a separate control server. In this case, the control server serves as a command / response for controlling between the devices in the content streaming system.

[0394] The streaming server may receive content from the media storage and / or the encoding server. For example, when receiving content from the encoding server, the content may be received in real time. In this case, in order to provide a smooth streaming service, the streaming server may store the bitstream for a predetermined time.

[0395] Examples of user devices may include mobile phones, smartphones, laptop computers, digital broadcast terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation devices, tablet PCs, tablet computers, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, head-mounted displays), digital TVs, desktop computers, digital signage, etc.

[0396] Each server in the content streaming system may operate as a distributed server, in which case the data received from each server may be distributed.

[0397] The scope of the present disclosure includes software or executable commands (e.g., operating systems, applications, firmware, programs, etc.) for enabling the operation of methods according to various embodiments to be performed on a device or computer, and a non-transitory computer-readable medium having such software or commands stored thereon and executable on the device or computer.

[0398] Industrial Applicability

[0399] Embodiments of the present disclosure may be used to encode or decode images.

Claims

1. A decoding apparatus for image decoding, the decoding apparatus comprising: a memory; and at least one processor connected to the memory, the at least one processor being configured to: identify a prediction mode of a current block; based on the prediction mode of the current block being an intra prediction mode, identify whether the intra prediction mode of the current block is a matrix-based intra prediction (MIP) mode; based on the intra prediction mode of the current block not being the MIP mode, determine a candidate intra prediction mode for the current block based on prediction modes of neighboring blocks located around the current block; generate a candidate intra prediction mode list for the current block based on the candidate intra prediction mode; and determine the intra prediction mode of the current block based on the candidate intra prediction mode list, wherein, based on the prediction mode of the neighboring block being the MIP mode, the candidate intra prediction mode is determined to be a predetermined intra prediction mode.

2. The decoding device according to claim 1, wherein, The predetermined intra prediction mode is a planar mode.

3. The decoding apparatus according to claim 1, Among them, wherein whether the prediction mode of the neighboring block is the MIP mode is determined based on a MIP mode indicator for the neighboring block, and wherein the MIP mode indicator is obtained from a bitstream.

4. The decoding apparatus according to claim 1, Among them, wherein the candidate intra prediction mode list is generated based on a first candidate intra prediction mode and a second candidate intra prediction mode, wherein the first candidate intra prediction mode is determined based on a prediction mode of a first neighboring block located around the current block, and wherein the second candidate intra prediction mode is determined based on a prediction mode of a second neighboring block located around the current block.

5. The decoding device according to claim 4, wherein, Based on the first candidate intra prediction mode and the second candidate intra prediction mode being the same and the first candidate intra prediction mode being an intra prediction mode having a value greater than a prediction mode value specifying a DC mode, the candidate intra prediction mode list is determined to include the value of the first candidate intra prediction mode.

6. The decoding device according to claim 4, wherein, Based on both the prediction mode of the first neighboring block and the prediction mode of the second neighboring block being the MIP mode, the candidate intra prediction mode list is determined to have a predetermined candidate intra prediction mode.

7. The decoding device according to claim 6, wherein, The predetermined candidate intra prediction mode includes at least one of a DC mode or a vertical mode.

8. The decoding apparatus according to claim 4, wherein, Based on the prediction mode of the first neighboring block being the MIP mode, the first candidate intra prediction mode and the second candidate intra prediction mode being different from each other, and the second candidate intra prediction mode being an intra prediction mode having a value greater than a prediction mode value specifying a DC mode, the candidate intra prediction mode list is determined to include the second candidate intra prediction mode.

9. The decoding apparatus according to claim 1, wherein, Determining the intra prediction mode of the current block based on the candidate intra prediction mode list is performed by determining any one of the candidate intra prediction modes included in the candidate intra prediction mode list as the intra prediction mode of the current block based on an intra prediction mode indicator obtained from a bitstream.

10. The decoding device according to claim 1, wherein the at least one processor is further configured to: Determine a reference mode, where the reference mode is used to determine the intra prediction mode of a chrominance block corresponding to the current block; and Determine the intra prediction mode of the chrominance block based on the reference mode, Among them, The current block is a luminance block, and wherein, based on the intra prediction mode of the current block being the MIP mode, the reference mode is determined to be the planar mode.

11. The decoding device according to claim 10, wherein, The intra prediction mode of the chrominance block is determined to be the reference mode.

12. The decoding device according to claim 11, wherein, Based on the intra prediction mode of the current block not being the MIP mode, the reference mode is determined based on the intra prediction mode of the current block.

13. An encoding device for image coding, the encoding device comprising: A memory; And At least one processor, the at least one processor being connected to the memory, the at least one processor being configured to: Identify the prediction mode of the current block; Based on the prediction mode of the current block being an intra prediction mode, determine candidate intra prediction modes based on the prediction modes of neighboring blocks located around the current block; Generate a candidate intra prediction mode list of the current block based on the candidate intra prediction modes; And Encode an intra prediction mode indicator specifying the intra prediction mode of the current block based on the candidate intra prediction mode list, wherein, based on the prediction mode of the neighboring blocks being the matrix-based intra prediction MIP mode, the candidate intra prediction modes are determined to be predetermined intra prediction modes.

14. A device for transmitting data for an image, the device comprising: At least one processor, the at least one processor being configured to obtain a bitstream generated by an encoding method performed by an encoding device for image coding; And A transmitter, the transmitter being configured to transmit the bitstream, wherein the encoding method includes: Identify the prediction mode of the current block; Based on the prediction mode of the current block being an intra prediction mode, determine candidate intra prediction modes based on the prediction modes of neighboring blocks located around the current block; Generate a candidate intra prediction mode list of the current block based on the candidate intra prediction modes; and Encode an intra prediction mode indicator specifying the intra prediction mode of the current block into the bitstream based on the candidate intra prediction mode list, wherein, based on the prediction mode of the neighboring blocks being the matrix-based intra prediction MIP mode, the candidate intra prediction modes are determined to be predetermined intra prediction modes.

Images