Intra-frame prediction method and device based on MPM list
Through the intra prediction method based on MPM list, the compression efficiency problem of high-resolution images and video data is solved, and more efficient image and video encoding is achieved, which is suitable for multiple prediction modes.
Patent Information
- Application Number
- CN202411261034.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2019-02-20
- Filing Date
- 2020-02-20
- Publication Date
- 2024-11-15
AI Technical Summary
The prior art is difficult to effectively compress and transmit high-resolution, high-quality image and video data, resulting in increased transmission and storage costs, especially in the applications of immersive media such as virtual reality, artificial reality and holograms.
Using an intra prediction method based on the most likely mode (MPM) list, by constructing a candidate list including an intra prediction mode, an intra prediction mode for the current block is derived, and prediction samples are generated to improve image encoding efficiency.
The image/video compression efficiency is improved, the implementation complexity is reduced, and the encoding efficiency is enhanced by simplifying the intra prediction structure, which is suitable for normal intra prediction, multi-reference line intra prediction and sub-partition intra prediction.
Smart Images

Figure CN118972557A_ABST
Abstract
Description
[0001] Divisional application statement
[0002] This application is a divisional application based on the Chinese invention patent application with application number 202080019653.8, application date February 20, 2020, and invention name “Intra-frame prediction method and device based on MPM list”. Technical Field
[0003] The present document relates to an image coding technology, and more particularly, to an image coding method and an apparatus thereof using intra prediction based on a Most Probable Mode (MPM) list. Background Art
[0004] Recently, there is an increasing demand for high-resolution and high-quality images and videos such as ultra-high-definition (HUD) images and 4K or 8K or larger videos in various fields. As image and video data become high-resolution and high-quality, the amount of information or the number of bits transmitted relative to existing image and video data increases. Therefore, if a medium such as an existing wired or wireless broadband line is used to transmit image data or an existing storage medium is used to store image and video data, the transmission cost and storage cost increase.
[0005] In addition, recently, interest and demand for immersive media such as virtual reality (VR), artificial reality (AR) content or holograms are increasing. Broadcasting of images and videos whose image characteristics are different from those of real images, such as game images, is increasing.
[0006] Therefore, in order to effectively compress and transmit or store and play back information of high-resolution and high-quality images and videos having such various characteristics, efficient image and video compression technology is required. Summary of the invention
[0007] Technical issues
[0008] The present disclosure provides a method and apparatus for improving image coding efficiency.
[0009] The present disclosure also provides an efficient intra-frame prediction method and device.
[0010] The present disclosure also provides an image compilation method and device for exporting an MPM list.
[0011] The present disclosure also provides an image coding method and apparatus for deriving a unified MPM list for normal intra prediction, multi-reference line intra prediction, and sub-partition intra prediction.
[0012] Technical Solution
[0013] According to an exemplary embodiment of the present document, an image decoding method performed by a decoding device is provided. The method includes: obtaining most probable mode (MPM) index information and reference line index information from a bitstream; constructing an MPM list including candidate intra prediction modes for intra prediction of a current block; deriving an intra prediction mode for the current block from the candidate intra prediction modes included in the MPM list based on the MPM index information; generating a prediction sample for the current block based on the intra prediction mode; and generating a reconstructed sample for the current block based on the prediction sample, wherein in the construction of the MPM list, based on the case where the value of the reference line index information indicating the reference line for intra prediction of the current block is not equal to 0, the DC mode is derived as one of the candidate intra prediction modes and is included in the MPM list.
[0014] According to another exemplary embodiment of the present document, there is provided an image encoding method performed by an encoding device. The method includes: generating reference line index information indicating a reference line for intra prediction of a current block; constructing a most probable mode (MPM) list including candidate intra prediction modes for intra prediction of the current block; deriving an intra prediction mode for the current block from among the candidate intra prediction modes included in the MPM list; generating MPM index information indicating an intra prediction mode for the current block; and encoding image information including the MPM index information and the reference line index information, wherein in the construction of the MPM list, based on the case where the value of the reference line index information is not equal to 0, the DC mode is derived as one of the candidate intra prediction modes and is included in the MPM list.
[0015] According to another exemplary embodiment of the present document, a computer-readable digital storage medium is provided, which stores encoded image information, and the encoded image information enables a decoding device to perform an image decoding method. The image decoding method includes: obtaining most probable mode (MPM) index information and reference line index information from a bitstream; constructing an MPM list including candidate intra-frame prediction modes for intra-frame prediction of a current block; based on the MPM index information, deriving an intra-frame prediction mode for the current block from the candidate intra-frame prediction modes included in the MPM list; generating a prediction sample for the current block based on the intra-frame prediction mode; and generating a reconstructed sample for the current block based on the prediction sample, wherein in the construction of the MPM list, based on the case where the value of the reference line index information indicating the reference line for intra-frame prediction of the current block is not equal to 0, the DC mode is derived as one of the candidate intra-frame prediction modes and is included in the MPM list.
[0016] Beneficial Effects
[0017] According to the present disclosure, the overall image / video compression efficiency can be improved.
[0018] According to the present disclosure, the implementation complexity can be reduced through efficient intra-frame prediction, and the overall coding efficiency can be enhanced by improving the prediction performance.
[0019] According to the present disclosure, the intra prediction structure can be simplified by constructing a unified MPM list for normal intra prediction, multi-reference line intra prediction, and sub-partition intra prediction, and coding efficiency can be enhanced by efficiently coding the intra prediction mode. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] Figure 1 An example of a video / image coding system to which embodiments of this document may be applied is schematically illustrated.
[0021] Figure 2 is a view schematically illustrating the construction of a video / image encoding device to which embodiments of this document can be applied.
[0022] Figure 3 is a view schematically illustrating the construction of a video / image decoding device to which embodiments of this document can be applied.
[0023] Figure 4 An example of an intra-frame prediction-based image encoding method to which the exemplary embodiments of this document are applicable is schematically illustrated, and Figure 5 An intra predictor in an encoding device is schematically illustrated.
[0024] Figure 6 An example of an intra-frame prediction-based image decoding method to which the exemplary embodiments of this document are applicable is schematically illustrated, and Figure 7 An intra predictor in a decoding device is schematically illustrated.
[0025] Figure 8 An example of an intra prediction method based on the MPM mode in an encoding device to which exemplary embodiments of this document are applicable is illustrated.
[0026] Fig. 9 An example of an intra prediction method based on the MPM mode in a decoding device to which exemplary embodiments of this document are applicable is illustrated.
[0027] Fig.10 An example of an intra prediction mode to which embodiments of this document can be applied is illustrated.
[0028] Fig.11 Examples of reference sample lines for intra prediction using multiple reference lines are illustrated.
[0029] Fig.12 An example of dividing sub-partitions according to intra sub-partitioning (ISP) is illustrated.
[0030] Fig.13 is a diagram illustrating an implementation of a method for generating a unified MPM list according to the present document.
[0031] Fig.14 is a diagram illustrating another embodiment of a method for generating a unified MPM list according to the present document.
[0032] Fig.15 is a flowchart schematically illustrating an encoding method that can be performed by an encoding device according to an embodiment of the present disclosure.
[0033] Fig.16 is a flowchart schematically illustrating a decoding method that can be performed by a decoding device according to an embodiment of the present disclosure.
[0034] Fig.17 An example of a content streaming system to which the embodiments disclosed in this document can be applied is illustrated. DETAILED DESCRIPTION
[0035] This document can be modified in various ways and can have various embodiments, and specific embodiments will be illustrated in the drawings and described in detail. However, this is not intended to limit this document to specific embodiments. The terms commonly used in this specification are used to describe specific embodiments rather than to limit the technical spirit of this document. Unless otherwise clearly indicated in the context, singular expressions include plural expressions. Terms such as "including" or "having" in this specification should be understood to indicate the presence of characteristics, numbers, steps, operations, elements, components, or combinations thereof described in this specification, without excluding the possibility of the presence or addition of one or more characteristics, numbers, steps, operations, elements, components, or combinations thereof.
[0036] In addition, in order to facilitate the description related to different feature functions, the elements in the drawings described in this document are illustrated independently. This does not mean that each element is implemented as separate hardware or separate software. For example, at least two elements can be combined to form a single element, or a single element can be divided into multiple elements. Embodiments in which elements are combined and / or separated are also included in the scope of the rights of this document, unless it deviates from the essence of this document.
[0037] In this document, the term "A or B" may mean "only A", "only B", or "both A and B". In other words, in this document, the term "A or B" may be interpreted as indicating "A and / or B". For example, in this document, the term "A, B, or C" may mean "only A", "only B", "only C", or "any combination of A, B, and C".
[0038] A slash “ / ” or a comma used in this document may mean “and / or”. For example, “A / B” may mean “A and / or B”. Thus, “A / B” may mean “only A”, “only B”, or “both A and B”. For example, “A, B, C” may mean “A, B, or C”.
[0039] In this document, "at least one of A and B" may mean "only A", "only B", or "both A and B". In addition, in this document, the expression "at least one of A or B" or "at least one of A and / or B" may be interpreted as "at least one of A and B".
[0040] Furthermore, in this document, “at least one of A, B, and C” may mean “only A,” “only B,” “only C,” or “any combination of A, B, and C.” Furthermore, “at least one of A, B, or C” or “at least one of A, B and / or C” may mean “at least one of A, B, and C.”
[0041] In addition, brackets used in this document may mean "for example". Specifically, in the case of expressing "prediction (intra-frame prediction)", it may indicate that "intra-frame prediction" is proposed as an example of "prediction". In other words, the term "prediction" in this document is not limited to "intra-frame prediction", and it may indicate that "intra-frame prediction" is proposed as an example of "prediction". In addition, even in the case of expressing "prediction (i.e., intra-frame prediction)", it may indicate that "intra-frame prediction" is proposed as an example of "prediction".
[0042] In this document, technical features individually described in one drawing may be implemented individually, or may be implemented simultaneously.
[0043] Hereinafter, preferred embodiments of the present document will be described in more detail with reference to the accompanying drawings. Hereinafter, in the accompanying drawings, the same reference numerals are used in the same elements, and redundant descriptions of the same elements may be omitted.
[0044] This document relates to video / image coding. For example, the methods / implementations disclosed in this document may be applied to methods disclosed in the Versatile Video Coding (VVC) standard, the Essential Video Coding (EVC) standard, the AOMedia Video 1 (AV1) standard, the second generation audio video coding standard (AVS2), or the next generation video / image coding standard (e.g., H.267 or H.268, etc.).
[0045] In this document, various embodiments related to video / image coding may be provided, and unless otherwise specified, the embodiments may be combined with each other and performed.
[0046] In this document, video may mean a collection of a series of images over time. Generally, a picture means a unit representing an image of a specific time region, and a slice / tile is a unit that constitutes a part of a picture in coding. A slice / tile may include one or more coding tree units (CTUs). A picture may be composed of one or more slices / tiles. A picture may be composed of one or more tile groups. A tile group may include one or more tiles. A block may represent a rectangular area of a CTU row within a tile in a picture. A tile may be divided into a plurality of blocks, each of which is composed of one or more CTU rows within a tile. A tile that is not divided into a plurality of blocks may also be referred to as a block. Block scanning may be a specific order sorting of CTUs of a partitioned picture as follows: CTUs are continuously sorted in a block by a CTU raster scan, blocks within a tile are continuously sorted by a raster scan of a block of a tile, and tiles in a picture are continuously sorted by a raster scan of a tile of a picture. A tile is a rectangular area of a CTU within a specific tile column and a specific tile row in a picture. A tile column is a rectangular area whose height of the CTU is equal to the height of the picture and whose width is specified by a syntax element in a picture parameter set. A tile row is a rectangular area whose height of the CTU is specified by a syntax element in a picture parameter set and whose width is equal to the width of the picture. Tile scanning can be a specific order sorting of CTUs of the following partitioned pictures: CTUs can be sorted continuously in tiles by CTU raster scanning, and tiles in pictures can be sorted continuously by raster scanning of tiles of pictures. A slice includes an integer number of blocks of a picture that can be exclusively included in a single NAL unit. A slice can consist of multiple complete tiles or only of a complete block portion of a continuous sequence of tiles. In this document, tile groups and slices can be used interchangeably with each other. For example, in this document, a tile group / tile group header may be referred to as a slice / slice header.
[0047] A pixel or a picture element (pel) may refer to the smallest unit constituting a picture (or image). In addition, a "sample" may be used as a term corresponding to a pixel. A sample may generally represent a pixel or a value of a pixel, and may represent only a pixel / pixel value of a luminance component, or only a pixel / pixel value of a chrominance component.
[0048] A unit may represent a basic unit of image processing. A unit may include at least one of a specific area of a picture and information related to the area. A unit may include a luminance block and two chrominance (e.g., cb, cr) blocks. Depending on the situation, terms such as unit, block, area, etc. may be used interchangeably. In general, an M×N block may include a set (or array) of samples (or sample arrays) or transform coefficients consisting of M columns and N rows.
[0049] Figure 1 An example of a video / image coding system to which embodiments of this document may be applied is schematically illustrated.
[0050] Reference Figure 1 The video / image coding system may include a first device (source device) and a second device (receiving device). The source device may transmit the encoded video / image information or data to the receiving device in the form of a file or stream transmission via a digital storage medium or a network.
[0051] The source device may include a video source, an encoding device, and a transmitter. The receiving device may include a receiver, a decoding device, and a renderer. The encoding device may be referred to as a video / image encoding device, and the decoding device may be referred to as a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display, and the display may be configured as a separate device or an external component.
[0052] The video source can obtain the video / image by capturing, synthesizing or generating the process of the video / image. The video source may include a video / image capturing device and / or a video / image generating device. The video / image capturing device may include, for example, one or more cameras, a video / image archive including previously captured videos / images, etc. The video / image generating device may include, for example, a computer, a tablet computer, and a smart phone, and may (electronically) generate the video / image. For example, a virtual video / image may be generated by a computer, etc. In this case, the video / image capturing process may be replaced by the process of generating the relevant data.
[0053] The encoding device can encode the input video / image. The encoding device can perform a series of processes such as prediction, transformation and quantization for compression and coding efficiency. The encoded data (encoded video / image information) can be output in the form of a bitstream.
[0054] The transmitter may transmit the encoded video / image information or data output in the form of a bit stream to a receiver of a receiving device in the form of a file or stream transmission through a digital storage medium or a network. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitter may include an element for generating a media file in a predetermined file format, and may include an element for transmitting through a broadcast / communication network. The receiver may receive / extract a bit stream and transmit the received / extracted bit stream to a decoding device.
[0055] The decoding device may decode a video / image by performing a series of processes such as dequantization, inverse transformation, prediction, etc. corresponding to the operation of the encoding device.
[0056] The renderer can render the decoded video / image, and the rendered video / image can be displayed through a display.
[0057] Figure 2 is a diagram schematically describing a configuration of a video / image encoding device to which the present document can be applied. Hereinafter, the so-called video encoding device may include an image encoding device.
[0058] Reference Figure 2 , the encoding device 200 may include an image segmenter 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 may include an inter-frame predictor 221 and an intra-frame predictor 222. The residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234, and an inverse transformer 235. The residual processor 230 may also include a subtractor 231. The adder 250 may be referred to as a reconstructor or a reconstructed block generator. According to an embodiment, the image segmenter 210, the predictor 220, the residual processor 230, the entropy encoder 240, the adder 250, and the filter 260 described above may be composed of one or more hardware components (e.g., an encoder chipset or processor). In addition, the memory 270 may include a decoded picture buffer (DPB) and may be composed of a digital storage medium. The hardware component may also include a memory 270 as an internal / external component.
[0059] The image divider 210 divides the input image (or picture or frame) input to the encoding device 200 into one or more processing units. As an example, the processing unit may be referred to as a coding unit (CU). In this case, starting from a coding tree unit (CTU) or a maximum coding unit (LCU), the coding unit may be recursively divided according to a quadtree binary tree ternary tree (QTBTTT) structure. For example, a coding unit may be divided into a plurality of coding units with a deeper depth based on a quadtree structure, a binary tree structure, and / or a ternary tree structure. In this case, for example, a quadtree structure may be applied first, and then a binary tree structure and / or a ternary tree structure may be applied. Alternatively, a binary tree structure may be applied first. The coding process according to this document may be performed based on a final coding unit that is not further divided. In this case, based on coding efficiency according to image characteristics, the maximum coding unit may be directly used as a final coding unit. Alternatively, the coding unit may be recursively divided into coding units with a further deeper depth as needed, so that a coding unit of an optimal size may be used as a final coding unit. Here, the coding process may include processes such as prediction, transformation, and reconstruction, which will be described later. As another example, the processing unit may also include a prediction unit (PU) or a transformation unit (TU). In this case, the prediction unit and the transformation unit may be divided or partitioned from the above-mentioned final coding unit. The prediction unit may be a unit for sample prediction, and the transformation unit may be a unit for deriving a transformation coefficient and / or a unit for deriving a residual signal based on the transformation coefficient.
[0060] Depending on the situation, the unit and terms such as block, region, etc. may be used interchangeably. In general, an M×N block may represent a set of samples or transform coefficients consisting of M columns and N rows. A sample may generally represent a pixel or a value of a pixel, and may represent only a pixel / pixel value of a luminance component, or only a pixel / pixel value of a chrominance component. A sample may be used as a term corresponding to a pixel or a pel of a picture (or image).
[0061] In the encoding device 200, a prediction signal (prediction block, prediction sample array) output from the inter predictor 221 or the intra predictor 222 is subtracted from the input image signal (original block, original sample array) to generate a residual signal (residual block, residual sample array), and the generated residual signal is sent to the transformer 232. In this case, as shown, the unit that subtracts the prediction signal (prediction block, prediction sample array) from the input image signal (original block, original sample array) in the encoder 200 may be referred to as a subtractor 231. The predictor may perform prediction on a processing target block (hereinafter, referred to as a "current block"), and may generate a prediction block including prediction samples for the current block. The predictor may determine whether intra prediction or inter prediction is applied on a current block or CU basis. As discussed later in the description of each prediction mode, the predictor may generate various information related to prediction such as prediction mode information, and send the generated information to the entropy encoder 240. Information about the prediction may be encoded in the entropy encoder 240 and output in the form of a bitstream.
[0062] The intra-frame predictor 222 can predict the current block by referring to the samples in the current picture. Depending on the prediction mode, the reference sample can be located near the current block or separated from the current block. In intra-frame prediction, the prediction mode may include a variety of non-directional modes and a variety of directional modes. The non-directional mode may include, for example, a DC mode and a plane mode. Depending on the level of detail of the prediction direction, the directional mode may include, for example, 33 directional prediction modes or 65 directional prediction modes. However, this is only an example, and more or fewer directional prediction modes may be used depending on the setting. The intra-frame predictor 222 may determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring blocks.
[0063] The inter-frame predictor 221 can derive a prediction block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. At this time, in order to reduce the amount of motion information sent in the inter-frame prediction mode, the motion information can be predicted based on a block, a sub-block or a sample based on the correlation of the motion information between the neighboring block and the current block. The motion information may include a motion vector and a reference picture index. The motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter-frame prediction, the neighboring blocks may include spatial neighboring blocks existing in the current picture and temporal neighboring blocks existing in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same as or different from each other. The temporal neighboring block may be referred to as a collocated reference block, a collocated CU (colCU), etc., and the reference picture including the temporal neighboring block may be referred to as a collocated picture (colPic). For example, the inter-frame predictor 221 may configure a motion information candidate list based on the neighboring blocks, and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Inter-frame prediction may be performed based on various prediction modes. For example, in the case of skip mode and merge mode, the inter-frame predictor 221 can use the motion information of the neighboring block as the motion information of the current block. In skip mode, unlike merge mode, a residual signal cannot be sent. In the case of motion information prediction (motion vector prediction, MVP) mode, the motion vector of the neighboring block can be used as a motion vector prediction item, and the motion vector of the current block can be indicated by signaling a motion vector difference.
[0064] The predictor 220 can generate a prediction signal based on various prediction methods. For example, the predictor can apply intra prediction or inter prediction to the prediction of a block, and can also apply intra prediction and inter prediction at the same time. This can be called combined inter and intra prediction (CIIP). In addition, the predictor can be based on an intra block copy (IBC) prediction mode or a palette mode to perform prediction on the block. The IBC prediction mode or the palette mode can be used for content image / video coding such as games such as screen content coding (SCC). Although IBC basically performs prediction in the current picture, its execution is similar to inter prediction in that it derives a reference block in the current picture. That is, IBC can use at least one of the inter prediction techniques described in this document. The palette mode can be regarded as an example of intra coding or intra prediction. When the palette mode is applied, the sample values in the picture can be signaled based on information about the palette index and the palette table.
[0065] The prediction signal generated by the predictor (including the inter-frame predictor 221 and / or the intra-frame predictor 222) can be used to generate a reconstruction signal or to generate a residual signal. The transformer 232 can generate a transform coefficient by applying a transform technique to the residual signal. For example, the transform technique may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-Loève transform (KLT), a graph-based transform (GBT), or a conditional nonlinear transform (CNT). Here, GBT means a transform obtained from a curve graph when the relationship information between pixels is represented by a curve graph. CNT refers to a transform obtained based on a prediction signal generated using all previously reconstructed pixels. In addition, the transform process can be applied to square pixel blocks of the same size, or can be applied to blocks of variable size rather than square blocks.
[0066] The quantizer 233 may quantize the transform coefficients and send them to the entropy encoder 240, and the entropy encoder 240 may encode the quantized signal (information about the quantized transform coefficients) and output the encoded signal in the bitstream. The information about the quantized transform coefficients may be referred to as residual information. The quantizer 233 may rearrange the quantized transform coefficients of the block type into a one-dimensional vector form based on the coefficient scanning order, and generate information about the quantized transform coefficients based on the quantized transform coefficients in the one-dimensional vector form. The entropy encoder 240 may perform various encoding methods such as, for example, exponential Golomb, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), etc. The entropy encoder 240 may encode the information required for video / image reconstruction other than the quantized transform coefficients (e.g., the value of the syntax element, etc.) together or separately. The encoded information (e.g., the encoded video / image information) may be transmitted or stored on a unit basis of a network abstraction layer (NAL) in the form of a bitstream. The video / image information may also include information about various parameter sets such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), a video parameter set (VPS), etc. In addition, the video / image information may also include conventional constraint information. In this document, information and / or syntax elements sent from an encoding device to / signaled to a decoding device may be included in the video / image information. The video / image information may be encoded by the above-mentioned encoding process and included in a bitstream. The bitstream may be transmitted over a network or stored in a digital storage medium. Here, the network may include a broadcast network, a communication network, and / or the like, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmitter (not shown) that transmits a signal output from the entropy encoder 240 or a memory (not shown) that stores it may be configured as an internal / external element of the encoding device 200, or the transmitter may be included in the entropy encoder 240.
[0067] The quantized transform coefficients output from the quantizer 233 can be used to generate a prediction signal. For example, by applying dequantization and inverse transformation using the transform coefficients quantized by the dequantizer 234 and the inverse transformer 235, the residual signal (residual block or residual sample) can be reconstructed. The adder 155 adds the reconstructed residual signal to the prediction signal output from the inter-frame predictor 221 or the intra-frame predictor 222, so that a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) can be generated. When there is no residual for the processing target block as in the case of applying the skip mode, the prediction block can be used as a reconstructed block. The adder 250 can be referred to as a reconstructor or a reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next processing target block in the current picture, and as described later, can be used for inter-frame prediction of the next picture performed by filtering.
[0068] Furthermore, during the picture encoding and / or reconstruction process, luminance mapping and chrominance scaling (LMCS) may be applied.
[0069] The filter 260 can improve the subjective / objective video quality by applying filtering to the reconstructed signal. For example, the filter 260 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and the modified reconstructed picture can be stored in the memory 270, especially in the DPB of the memory 270. Various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive ring filtering, bilateral filtering, etc. As discussed later in the description of each filtering method, the filter 260 can generate various information related to filtering and send the generated information to the entropy encoder 240. Information about filtering can be encoded in the entropy encoder 240 and output in the form of a bitstream.
[0070] The modified reconstructed picture transmitted to the memory 270 may be used as a reference picture in the inter-frame predictor 221. Accordingly, the encoding device can avoid prediction mismatch in the encoding device 100 and the decoding device when applying inter-frame prediction, and can also improve encoding efficiency.
[0071] The memory 270DPB can store the modified reconstructed picture so that it can be used as a reference picture in the inter-frame predictor 221. The memory 270 can store the motion information of the block in the current picture from which the motion information has been derived (or encoded) and / or the motion information of the block in the reconstructed picture. The stored motion information can be sent to the inter-frame predictor 221 to be used as the motion information of the neighboring block or the motion information of the temporal neighboring block. The memory 270 can store the reconstructed samples of the reconstructed block in the current picture and send them to the intra-frame predictor 222.
[0072] Figure 3is a diagram schematically describing a configuration of a video / image decoding device to which this document can be applied.
[0073] Reference Figure 3 , the video decoding device 300 may include an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 may include an inter-frame predictor 331 and an intra-frame predictor 332. The residual processor 320 may include a dequantizer 321 and an inverse transformer 321. According to an embodiment, the entropy decoder 310, the residual processor 320, the predictor 330, the adder 340, and the filter 350 described above may be composed of one or more hardware components (e.g., a decoder chipset or a processor). In addition, the memory 360 may include a decoded picture buffer (DPB) and may be composed of a digital storage medium. The hardware component may also include a memory 360 as an internal / external component.
[0074] When a bit stream including video / image information is input, the decoding device 300 can Figure 2 The image is reconstructed correspondingly to the processing of the video / image information in the encoding device. For example, the decoding device 300 can derive the unit / block based on the information related to the block segmentation obtained from the bit stream. The decoding device 300 can perform decoding by using the processing unit applied in the encoding device. Therefore, the decoding processing unit can be, for example, a coding unit, which can be divided from the coding tree unit or the maximum coding unit along the quadtree structure, the binary tree structure and / or the ternary tree structure. One or more transform units can be derived from the coding unit. And, the reconstructed image signal decoded and output by the decoding device 300 can be reproduced by a reproducer.
[0075] The decoding device 300 may receive the Figure 2The signal output by the encoding device of the encoding device, and the received signal can be decoded by the entropy decoder 310. For example, the entropy decoder 310 can parse the bitstream to derive information (e.g., video / image information) required for image reconstruction (or picture reconstruction). The video / image information may also include information about various parameter sets such as adaptive parameter set (APS), picture parameter set (PPS), sequence parameter set (SPS), video parameter set (VPS), etc. In addition, the video / image information may also include conventional constraint information. The decoding device may further decode the picture based on the information about the parameter set and / or the conventional constraint information. In this document, the signaled / received information and / or syntax elements described later may be decoded and obtained from the bitstream through a decoding process. For example, the entropy decoder 310 may decode the information in the bitstream based on a coding method such as exponential Golomb coding, CAVLC, CABAC, etc., and may output the values of syntax elements required for image reconstruction and the quantized values of the transform coefficients of the residual. More specifically, the CABAC entropy decoding method can receive the bin corresponding to each syntax element in the bitstream, use the decoding target syntax element information and the decoding information of the neighboring and decoding target blocks or the information of the symbol / bin decoded in the previous step to determine the context model, predict the bin generation probability according to the determined context model and perform arithmetic decoding on the bin to generate the symbol corresponding to each syntax element value. Here, the CABAC entropy decoding method can update the context model using the information of the symbol / bin decoded by the context model for the next symbol / bin after determining the context model. The information about prediction among the information decoded in the entropy decoder 310 can be provided to the predictor (inter-frame predictor 332 and intra-frame predictor 331), and the residual value (ie, quantized transform coefficient) and the associated parameter information that have been entropy decoded in the entropy decoder 310 can be input to the residual processor 320. The residual processor 320 can derive a residual signal (residual block, residual sample, residual sample array). In addition, the information about filtering among the information decoded in the entropy decoder 310 can be provided to the filter 350. In addition, a receiver (not shown) receiving a signal output from the encoding device may also constitute the decoding device 300 as an internal / external element, and the receiver may be a component of the entropy decoder 310. In addition, the decoding device according to the present document may be referred to as a video / image / picture decoding device, and the decoding device may be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include the entropy decoder 310, and the sample decoder may include at least one of the dequantizer 321, the inverse transformer 322, the adder 340, the filter 350, the memory 360, the inter-frame predictor 332, and the intra-frame predictor 331.
[0076] The dequantizer 321 may output the transform coefficient by dequantizing the quantized transform coefficient. The dequantizer 321 may rearrange the quantized transform coefficient into the form of a two-dimensional block. In this case, the rearrangement may be performed based on the order of coefficient scanning that has been performed in the encoding device. The dequantizer 321 may perform dequantization on the quantized transform coefficient using a quantization parameter (e.g., quantization step size information) and obtain the transform coefficient.
[0077] The inverse transformer 322 obtains a residual signal (residual block, residual sample array) by performing inverse transformation on the transformation coefficients.
[0078] The predictor may perform prediction on the current block and generate a prediction block including prediction samples for the current block. The predictor may determine whether to apply intra prediction or inter prediction to the current block based on information on prediction output from the entropy decoder 310, and may specifically determine an intra / inter prediction mode.
[0079] The predictor 320 can generate a prediction signal based on various prediction methods. For example, the predictor can apply intra prediction or inter prediction to the prediction of a block, and can also apply intra prediction and inter prediction at the same time. This can be called combined inter and intra prediction (CIIP). In addition, the predictor can be based on an intra block copy (IBC) prediction mode or a palette mode to perform prediction on the block. The IBC prediction mode or the palette mode can be used for content image / video coding such as games such as screen content coding (SCC). Although IBC basically performs prediction in the current picture, its execution mode is similar to inter prediction in that it derives a reference block in the current picture. That is, IBC can use at least one of the inter prediction techniques described in this document. The palette mode can be regarded as an example of intra coding or intra prediction. When the palette mode is applied, information about the palette table and the palette index can be included in the video / image information and signaled.
[0080] The intra-frame predictor 331 can predict the current block by referring to samples in the current picture. Depending on the prediction mode, the reference sample can be located near the current block or separated from the current block. In intra-frame prediction, the prediction mode can include multiple non-directional modes and multiple directional modes. The intra-frame predictor 331 can determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring block.
[0081] The inter-frame predictor 332 may derive a prediction block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. At this time, in order to reduce the amount of motion information sent in the inter-frame prediction mode, the motion information may be predicted based on a block, a sub-block, or a sample based on the correlation of the motion information between the neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter-frame prediction, the neighboring blocks may include spatial neighboring blocks existing in the current picture and temporal neighboring blocks existing in the reference picture. For example, the inter-frame predictor 332 may configure a motion information candidate list based on neighboring blocks, and derive a motion vector and / or a reference picture index of the current block based on the received candidate selection information. Inter-frame prediction may be performed based on various prediction modes, and information about the prediction may include information indicating a mode of inter-frame prediction for the current block.
[0082] The adder 340 adds the obtained residual signal to the prediction signal (prediction block, prediction sample array) output from the predictor (inter-frame predictor 332 or intra-frame predictor 331), so that a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) can be generated. When there is no residual for the processing target block as in the case of applying the skip mode, the prediction block can be used as the reconstructed block.
[0083] The adder 340 may be referred to as a reconstructor or a reconstructed block generator. The generated reconstructed signal may be used for intra prediction of the next block to be processed in the current picture, may be output through filtering as described below, or may be used for inter prediction of the next picture.
[0084] In addition, luma mapping and chroma scaling (LMCS) can be applied to the picture decoding process.
[0085] The filter 350 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filter 350 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and store the modified reconstructed picture in the memory 360, specifically, in the DPB of the memory 360. The various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc.
[0086] The (modified) reconstructed picture stored in the DPB of the memory 360 may be used as a reference picture in the inter-frame predictor 332. The memory 360 may store the motion information of the block from which the motion information in the current picture is derived (or decoded) and / or the motion information of the block in the reconstructed picture. The stored motion information may be sent to the inter-frame predictor 332 so as to be used as the motion information of the spatial neighboring block or the motion information of the temporal neighboring block. The memory 360 may store the reconstructed samples of the reconstructed blocks in the current picture and transmit the reconstructed samples to the intra-frame predictor 331.
[0087] In the present disclosure, the implementation described in the filter 260, the inter-frame predictor 221 and the intra-frame predictor 222 of the encoding device 200 may be the same as the filter 350, the inter-frame predictor 332 and the intra-frame predictor 331 of the decoding device 300 or may be applied corresponding to the filter 350, the inter-frame predictor 332 and the intra-frame predictor 331 of the decoding device 300, respectively.
[0088] As described above, when performing video coding, prediction is performed to improve compression efficiency. A prediction block including prediction samples of the current block, that is, a target coding block, can be generated by prediction. In this case, the prediction block includes prediction samples in the spatial domain (or pixel domain). The prediction block is derived similarly in the encoding device and the decoding device. The encoding device can improve image coding efficiency by signaling information (residual information) about the original block rather than the original sample value of the original block itself and the residual between the prediction block. The decoding device can derive a residual block including residual samples based on the residual information, can generate a reconstructed block including reconstructed samples by adding the residual block to the prediction block, and can generate a reconstructed picture including the reconstructed block.
[0089] Residual information can be generated by a transformation and quantization process. For example, the encoding device can derive a residual block between the original block and the prediction block, can derive a transform coefficient by performing a transformation process on the residual samples (residual sample array) included in the residual block, can derive a quantized transform coefficient by performing a quantization process on the transform coefficient, and can signal the relevant residual information to the decoding device (through a bitstream). In this case, the residual information may include information such as value information, position information, a transformation scheme, a transform kernel, and a quantization parameter of the quantized transform coefficient. The decoding device can perform a dequantization / inverse transformation process based on the residual information, and can derive residual samples (or residual blocks). The decoding device can generate a reconstructed picture based on the prediction block and the residual block. In addition, the encoding device can derive a residual block for inter-frame prediction reference of subsequent pictures by dequantizing / inverse transforming the quantized transform coefficients, and can generate a reconstructed picture.
[0090] In addition, if intra prediction is performed, the correlation between samples can be used, and the difference between the original block and the predicted block, i.e., the residual, can be obtained. The above-mentioned transformation and quantization can be applied to the residual. Therefore, spatial redundancy can be reduced. Hereinafter, the encoding method and decoding method using intra prediction are specifically described.
[0091] Intra prediction refers to the prediction of generating a prediction sample of a current block based on a reference sample located outside the current block within a picture including the current block (hereinafter, the current picture). In this case, the reference sample located outside the current block may refer to a sample adjacent to the current block. If intra prediction is applied to the current block, neighboring reference samples to be used for intra prediction of the current block may be derived.
[0092] For example, when the size (width×height) of the current block is nW×nH, the neighboring reference samples of the current block may include a sample adjacent to the left boundary and a total of 2×nH samples adjacent to the lower left of the current block, a sample adjacent to the upper boundary and a total of 2×nW samples adjacent to the upper right of the current block, and one sample adjacent to the upper left of the current block. Alternatively, the neighboring reference samples of the current block may also include multiple columns of upper neighboring samples and multiple rows of left neighboring samples. In addition, the neighboring reference samples of the current block may also include a total of nH samples adjacent to the right boundary of the current block of size nW×nH, a total of nW samples adjacent to the lower boundary of the current block, and one sample adjacent to the lower right of the current block.
[0093] In this case, some of the neighboring reference samples of the current block have not been decoded or may not be available. In this case, the decoding device can configure the neighboring reference samples to be used for prediction by replacing the unavailable samples with available samples. Alternatively, the neighboring reference samples to be used for prediction can be constructed by interpolation of available samples.
[0094] If the neighboring reference samples are derived, (i) the prediction samples may be derived based on the average or interpolation of the neighboring reference samples of the current block, and (ii) the prediction samples may be derived based on reference samples existing in a specific (prediction) direction for the prediction samples among the neighboring reference samples of the current block. (i) may be applied when the intra prediction mode is a non-directional mode or a non-angular mode. (ii) may be applied when the intra prediction mode is a directional mode or an angular mode.
[0095] In addition, the prediction sample can also be generated by interpolating between the first neighboring sample located in the prediction direction of the intra prediction mode of the current block based on the prediction sample of the current block among the neighboring reference samples and the second neighboring sample located in the opposite direction of the prediction direction. The foregoing case may be referred to as linear interpolation intra prediction (LIP). In addition, the chrominance prediction sample may also be generated based on the luma sample using a linear model. This case may be referred to as LM mode.
[0096] In addition, a temporary prediction sample of the current block may be derived based on the filtered neighboring reference sample, and the prediction sample of the current block may also be derived by weighted summing at least one reference sample derived from among the normal neighboring reference samples (i.e., the unfiltered neighboring reference samples) according to the intra prediction mode and the temporary prediction sample. The foregoing may be referred to as position-dependent intra prediction (PDPC).
[0097] In addition, a prediction sample can be derived using a reference sample in a prediction direction located in a corresponding line by selecting a reference sample line with the highest prediction accuracy among neighboring multiple reference sample lines of the current block, and intra-frame prediction encoding can be performed by a method for indicating (signaling) the reference sample line used at this time to a decoding device. The foregoing case may be referred to as multi-reference line (MRL) intra-frame prediction or MRL-based intra-frame prediction.
[0098] In addition, intra prediction can be performed based on the same intra prediction mode by dividing the current block into vertical or horizontal sub-partitions, and neighboring reference samples can be derived and used in units of sub-partitions. That is, in this case, the intra prediction mode of the current block is equally applied to the sub-partitions, and neighboring reference samples can be derived and used in units of sub-partitions, thereby enhancing intra prediction performance in some cases. This prediction method may be referred to as intra sub-partition (ISP) intra prediction or ISP-based intra prediction.
[0099] The aforementioned intra prediction method may be referred to as an intra prediction type separate from the intra prediction mode. The intra prediction type may be referred to as various terms such as an intra prediction technique or an additional intra prediction mode. For example, the intra prediction type (or additional intra prediction mode, etc.) may include at least one of the aforementioned LIP, PDPC, MRL, and ISP. A general intra prediction method different from a specific intra prediction type such as LIP, PDPC, MRL, and ISP may be referred to as a normal intra prediction type. If a specific intra prediction type is not applied, a normal intra prediction type may generally be applied, and prediction may be performed based on the aforementioned intra prediction mode. In addition, if necessary, post-processing filtering may also be performed on the derived prediction samples.
[0100] The following drawings have been prepared to illustrate specific examples of this document. Since the titles of specific devices or specific terms or titles (e.g., grammatical titles, etc.) described in the drawings are presented exemplarily, the technical features of this document are not limited to the specific titles used in the following drawings.
[0101] Figure 4 An example of an intra-frame prediction-based image encoding method to which the exemplary embodiments of this document are applicable is schematically illustrated, and Figure 5 An intra predictor in an encoding device is schematically illustrated. Figure 5 The intra-frame predictor in the encoding device shown in FIG. 1 may also be applied equally or correspondingly to Figure 2 The intra predictor 222 of the encoding device 200 shown in FIG.
[0102] Reference Figure 4 and Figure 5 , S400 may be performed by the intra predictor 222 of the encoding device, and S410 may be performed by the residual processor 230 of the encoding device. Specifically, S410 may be performed by the subtractor 231 of the encoding device. In S420, the prediction information may be derived by the intra predictor 222 and encoded by the entropy encoder 240. In S420, the residual information may be derived by the residual processor 230 and encoded by the entropy encoder 240. The residual information indicates information about the residual sample. The residual information may include information about the quantized transform coefficient of the residual sample. As described above, the residual sample may be derived by the transform coefficient via the transformer 232 of the encoding device, and the transform coefficient may be derived by the quantized transform coefficient via the quantizer 233. The information about the quantized transform coefficient may be encoded by the entropy encoder 240 through the residual coding process.
[0103] The encoding device performs intra prediction of the current block (S400). The encoding device may derive the intra prediction mode / type of the current block, derive neighboring reference samples of the current block, and generate prediction samples in the current block based on the intra prediction mode / type and the neighboring reference samples. Here, the processes of determining the intra prediction mode / type, deriving neighboring reference samples, and generating prediction samples may also be performed simultaneously, and any one process may also be performed earlier than the other processes.
[0104] For example, the intra-frame predictor 222 of the encoding device may include an intra-frame prediction mode / type determiner 222-1, a reference sample deriver 222-2, and a prediction sample deriver 222-3, wherein the intra-frame prediction mode / type determiner 222-1 may determine the intra-frame prediction mode / type of the current block, the reference sample deriver 222-2 may derive the neighboring reference samples of the current block, and the prediction sample deriver 222-3 may derive the prediction sample of the current block. In addition, although not shown, if a prediction sample filtering process is performed, the intra-frame predictor 222 may also further include a prediction sample filter (not shown). The encoding device may determine a mode / type applied to the current block among a plurality of intra-frame prediction modes / types. The encoding device may compare the RD costs of the intra-frame prediction modes / types and determine the optimal intra-frame prediction mode / type for the current block.
[0105] As described above, the encoding device may also perform a prediction sample filtering process. Prediction sample filtering may be referred to as post-filtering. Some or all of the prediction samples may be filtered by the prediction sample filtering process. In some cases, the prediction sample filtering process may be omitted.
[0106] The encoding apparatus generates residual samples of the current block based on the (filtered) prediction samples (S410). The encoding apparatus may compare the prediction samples based on a phase in the original samples of the current block and derive the residual samples.
[0107] The encoding device may encode image information including information about intra prediction (prediction information) and residual information about residual samples (S420). The prediction information may include intra prediction mode information and intra prediction type information. The residual information may include residual coding syntax. The encoding device may derive quantized transform coefficients by transforming / quantizing residual samples. The residual information may include information about the quantized transform coefficients.
[0108] The encoding device can output the encoded image information in the form of a bit stream. The output bit stream can be delivered to the decoding device via a storage medium or a network.
[0109] As described above, the encoding device may generate a reconstructed picture (including reconstructed samples and reconstructed blocks). To this end, the encoding device may derive (modified) residual samples by dequantizing / inverse transforming the quantized transform coefficients again. As described above, the reason for transforming / quantizing the residual samples and then dequantizing / inverse transforming them again is to derive residual samples that are the same as the residual samples derived by the decoding device as described above. The encoding device may generate a reconstructed block including reconstructed samples of the current block based on the predicted samples and the (modified) residual samples. A reconstructed picture of the current picture may be generated based on the reconstructed block. As described above, an in-loop filtering process or the like may be further applied to the reconstructed picture.
[0110] Figure 6 An example of an intra-frame prediction-based image decoding method to which the exemplary embodiments of this document are applicable is schematically illustrated, and Figure 7 An intra predictor in a decoding device is schematically illustrated. Figure 7 The intra-frame predictor in the decoding device shown in FIG. 1 may also be applied equally or correspondingly to Figure 3 The intra predictor 331 of the decoding device 300 shown in FIG.
[0111] Reference Figure 6 and Figure 7, the decoding device may perform operations corresponding to the above operations performed by the encoding device. S600 to S620 may be performed by the intra-frame predictor 331 of the decoding device, and the prediction information in S600 and the residual information in S630 may be obtained from the bit stream by the entropy decoder 310 of the decoding device. The residual processor 320 of the decoding device may derive residual samples of the current block based on the residual information. Specifically, the dequantizer 321 of the residual processor 320 may derive transform coefficients by performing dequantization based on the quantized transform coefficients derived based on the residual information, and the inverse transformer 322 of the residual processor derives the residual samples of the current block by inversely transforming the transform coefficients. S640 may be performed by the reconstructor or adder 340 of the decoding device.
[0112] The decoding device may derive the intra prediction mode / type of the current block based on the received prediction information (intra prediction mode / type information) (S600). The decoding device may derive the neighboring reference samples of the current block (S610). The decoding device generates the prediction samples in the current block based on the intra prediction mode / type and the neighboring reference samples (S620). In this case, the decoding device may perform a prediction sample filtering process. Prediction sample filtering may be referred to as post filtering. Some or all of the prediction samples may be filtered by the prediction sample filtering process. In some cases, the prediction sample filtering process may be omitted.
[0113] The decoding device generates a residual sample of the current block based on the received residual information (S630). The decoding device may generate a reconstructed sample of the current block based on the predicted sample and the residual sample, and derive a reconstructed block including the reconstructed sample (S640). A reconstructed picture of the current picture may be generated based on the reconstructed block. As described above, an in-loop filtering process or the like may be further applied to the reconstructed picture.
[0114] Here, the intra-frame predictor 331 of the decoding device may include an intra-frame prediction mode / type determiner 331-1, a reference sample deriver 331-2, and a prediction sample deriver 331-3, wherein the intra-frame prediction mode / type determiner 331-1 may determine the intra-frame prediction mode / type of the current block based on the intra-frame prediction mode / type information acquired by the entropy decoder 310, the reference sample deriver 331-2 may derive the neighboring reference samples of the current block, and the prediction sample deriver 331-3 may derive the prediction sample of the current block. In addition, although not shown, if the above-mentioned prediction sample filtering process is performed, the intra-frame predictor 331 may also further include a prediction sample filter (not shown).
[0115] The intra prediction mode information may include, for example, flag information (e.g., intra_luma_mpm_flag) indicating whether the most probable mode (MPM) is applied to the current block or whether the remaining mode is applied thereto. At this time, if the MPM is applied to the current block, the prediction mode information may further include index information (e.g., intra_luma_mpm_idx) indicating one of the intra prediction mode candidates (MPM candidates). The intra prediction mode candidates (MPM candidates) may be composed of an MPM candidate list or an MPM list. In addition, if the MPM is not applied to the current block, the intra prediction mode information may further include remaining mode information (e.g., intra_luma_mpm_remainder) indicating one of the remaining intra prediction modes other than the intra prediction mode candidates (MPM candidates). The decoding device may determine the intra prediction mode of the current block based on the intra prediction mode information.
[0116] In addition, the intra prediction type information may be implemented in various forms. As an example, the intra prediction type information may include intra prediction type index information indicating one of the intra prediction types. As another example, the intra prediction type information may include at least one of the following: reference sample line information (e.g., intra_luma_ref_idx) indicating whether MRL is applied to the current block, and if MRL is applied, which reference sample line is used; ISP flag information (e.g., intra_subpartitions_mode_flag) indicating whether ISP is applied to the current block; ISP type information (e.g., intra_subpartitions_split_flag) indicating the partition type of the sub-partition when ISP is applied; flag information indicating whether PDCP is applied; or flag information indicating whether LIP is applied. In addition, the intra prediction type information may include a MIP flag indicating whether MIP is applied to the current block.
[0117] The above-mentioned intra-frame prediction mode information and / or intra-frame prediction type information can be encoded / decoded by the coding method described in this document. For example, the above-mentioned intra-frame prediction mode information and / or intra-frame prediction type information can be encoded / decoded by entropy coding (e.g., CABAC, CAVLC) based on a segmented (Rice) binary code.
[0118] Meanwhile, in the case of applying intra prediction, the intra prediction mode of the neighboring block may be used to determine the intra prediction mode applied to the current block. For example, the decoding device may select one of the mpm candidates in the most probable mode (mpm) list derived based on the intra prediction mode of the neighboring blocks (e.g., the left neighboring block and / or the upper neighboring block) of the current block and the additional candidate mode based on the received mpm index, or may select one of the remaining intra prediction modes not included in the mpm candidates (and the plane mode) based on the remaining intra prediction mode information. The MPM list may be constructed to include or exclude the plane mode as a candidate. For example, if the MPM list includes the plane mode as a candidate, the MPM list may have 6 candidates, whereas if the MPM list does not include the plane mode as a candidate, the MPM list may have 5 candidates. If the MPM list does not include the plane mode as a candidate, a non-planar flag (e.g., intra_luma_not_planar_flag) indicating whether the intra prediction mode of the current block is not a plane mode may be signaled. For example, the mpm flag may be signaled first, and the mpm index and the non-planar flag may be signaled when the value of the mpm flag is equal to 1. In addition, the mpm index may be signaled when the value of the non-planar flag is equal to 1. Here, because the planar mode is always considered to be mpm, rather than saying that the planar mode is not mpm, the MPM list is constructed to not include the planar mode as a candidate in order to first identify whether the intra prediction mode is the planar mode by first signaling the flag (non-planar flag).
[0119] For example, it can be indicated based on an mpm flag (e.g., intra_luma_mpm_flag) whether the intra prediction mode applied to the current block is in the mpm candidates (and planar mode) or in the remaining mode. An mpm flag value of 1 can indicate that the intra prediction mode for the current block is in the mpm candidates (and planar mode), while an mpm flag value of 0 can indicate that the intra prediction mode for the current block is not in the mpm candidates (and planar mode). A non-planar flag (e.g., intra_luma_not_planar_flag) value of 0 can indicate that the intra prediction mode for the current block is a planar mode, and a non-planar flag value of 1 can indicate that the intra prediction mode for the current block is not a planar mode. The mpm index can be signaled in the form of an mpm_idx or intra_luma_mpm_idx syntax element, and the remaining intra prediction mode information can be signaled in the form of a rem_intra_luma_pred_mode or intra_luma_mpm_remainder syntax element. For example, the remaining intra prediction mode information may index the remaining intra prediction modes that are not included in the mpm candidates (and planar modes) among all intra prediction modes in the order of their prediction mode numbers, and may indicate one of them. The intra prediction mode may be an intra prediction mode for a luma component (sample). In the following, the intra prediction mode information may include at least one of the following items: an mpm flag (e.g., intra_luma_mpm_flag), a non-planar flag (e.g., intra_luma_not_planar_flag), an mpm index (e.g., mpm_idx or intra_luma_mpm_idx), and remaining intra prediction mode information (rem_intra_luma_pred_mode or intra_luma_mpm_remainder). In this document, the MPM list may be referred to by various terms, such as an mpm candidate list, a candidate mode list (candModelList), and a candidate intra prediction mode list.
[0120] Typically, when dividing blocks of an image, the current block and the neighboring blocks to be coded have similar image characteristics. Therefore, the current block and the neighboring blocks are more likely to have the same or similar intra-frame prediction modes. Therefore, the encoder can use the intra-frame prediction mode of the neighboring block to encode the intra-frame prediction mode of the current block. For example, the encoder / decoder can form a most probable mode (MPM) list of the current block. The MPM list can also be referred to as an MPM candidate list. Here, MPM can represent a mode for improving coding efficiency by taking into account the similarity between the current block and the neighboring blocks when coding the intra-frame prediction mode.
[0121] Figure 8An example of an intra prediction method based on the MPM mode in an encoding device to which exemplary embodiments of this document are applicable is illustrated.
[0122] Reference Figure 8 , the encoding device constructs an MPM list for the current block (S800). The MPM list may include candidate intra prediction modes (MPM candidates) that are more likely to be applied to the current block. The MPM list may also include intra prediction modes of neighboring blocks, and further include specific intra prediction modes according to a predetermined method. A specific method for constructing the MPM list will be described later.
[0123] The encoding device determines the intra prediction mode of the current block (S810). The encoding device may perform prediction based on various intra prediction modes, and determine the optimal intra prediction mode based on rate-distortion optimization (RDO) based on the above prediction. In this case, the encoding device may also determine the optimal intra prediction mode using only the MPM candidates and the plane mode configured in the MPM list, or further use the remaining intra prediction modes and the MPM candidates and the plane mode configured in the MPM list to determine the optimal intra prediction mode.
[0124] Specifically, for example, if the intra prediction type of the current block is a specific type (e.g., LIP, MRL, or ISP) other than the normal intra prediction type, the encoding device may determine the optimal intra prediction mode by considering only the MPM candidates and the plane mode as the intra prediction mode candidates of the current block. That is, in this case, the intra prediction mode of the current block may be determined only among the MPM candidates and the plane mode, and in this case, the mpm flag may not be encoded / signaled. In this case, even without separate signaling of the mpm flag, the decoding device may estimate that the MPM flag is 1.
[0125] In general, if the intra prediction mode of the current block is not the planar mode but one of the MPM candidates in the MPM list, the encoding device generates an mpm index (mpm idx) indicating one of the MPM candidates. If the intra prediction mode of the current block does not even exist in the MPM list, the encoding device generates remaining intra prediction mode information indicating a mode such as the intra prediction mode of the current block among the remaining intra prediction modes not included in the MPM list (and the planar mode).
[0126] The encoding device may encode the intra-frame prediction mode information and output it in the form of a bitstream (S820). The intra-frame prediction mode information may include the aforementioned mpm flag, non-plane flag, mpm index and / or remaining intra-frame prediction mode information. In general, the mpm index and the remaining intra-frame prediction mode information have an alternating relationship and are not signaled at the same time when indicating the intra-frame prediction mode of a block. That is, the value 1 of the mpm flag is signaled together with the non-plane flag or the mpm index, or the value 0 of the mpm flag is signaled together with the remaining intra-frame prediction mode information. However, as described above, if a specific intra-frame prediction type is applied to the current block, the mpm flag is not signaled and only the non-plane flag and / or the mpm index may also be signaled. That is, in this case, the intra-frame prediction mode information may also include only the non-plane flag and / or the mpm index.
[0127] Fig. 9 An example of an intra prediction method based on the MPM mode in a decoding device to which exemplary embodiments of this document are applicable is illustrated. Fig. 9 The decoding device shown in FIG. 1 can determine the Figure 8 The encoding apparatus shown in determines and signals the intra-prediction mode corresponding to the intra-prediction mode information.
[0128] Reference Fig. 9 , the decoding apparatus obtains intra prediction mode information from the bitstream (S900). As described above, the intra prediction mode information may include at least one of an mpm flag, a non-planar flag, an mpm index, and a remaining intra prediction mode.
[0129] The decoding device constructs an MPM list (S910). The MPM list is composed of the same MPM list configured in the encoding device. That is, the MPM list may also include intra-frame prediction modes of neighboring blocks, and further include specific intra-frame prediction modes according to a predetermined method. The specific method for constructing the MPM list will be described later.
[0130] Although it is illustrated that S910 is executed later than S900 , this is illustrative, and S910 may be executed earlier than S900 , and S900 and S910 may be executed at the same time.
[0131] The decoding apparatus determines an intra prediction mode of a current block based on the MPM list and the intra prediction mode information (S920).
[0132] As an example, if the value of the mpm flag is 1, the decoding device may derive the plane mode as the intra prediction mode of the current block or (based on the non-plane flag) derive the candidate indicated by the mpm index among the MPM candidates in the MPM list as the intra prediction mode of the current block. Here, the MPM candidate may also indicate only the candidates included in the MPM list, or also include the plane mode applicable to the case where the value of the mpm flag is 1 and the candidates included in the MPM list.
[0133] As another example, if the value of the mpm flag is 0, the decoding apparatus may derive, as the intra prediction mode of the current block, an intra prediction mode indicated by the remaining intra prediction mode information among the remaining intra prediction modes not included in the MPM list and the planar mode.
[0134] As another example, if the intra prediction type of the current block is a specific type (e.g., LIP, MRL, or ISP), the decoding device may derive the candidate or planar mode indicated by the mpm index in the MPM list as the intra prediction mode of the current block even without confirmation of the mpm flag.
[0135] In addition, the intra prediction mode may include a non-directional (or non-angle) intra prediction mode and a directional (or angle) intra prediction mode. For example, in the HEVC standard, intra prediction modes including 2 non-directional prediction modes and 33 directional prediction modes are used. The non-directional prediction mode may include a plane intra prediction mode (ie, No. 0) and a DC intra prediction mode (ie, No. 1). The directional prediction mode may include intra prediction modes No. 2 to No. 34. The plane intra prediction mode may be referred to as a plane mode, and the DC intra prediction mode may be referred to as a DC mode.
[0136] Alternatively, to obtain a given edge direction proposed in a natural video, such as Fig.10 As in, the directional intra prediction mode can be expanded from the existing 33 modes to 65 modes. In this case, the intra prediction mode may include 2 non-directional intra prediction modes and 65 directional intra prediction modes. The non-directional intra prediction mode may include a plane intra prediction mode (i.e., No. 0) and a DC intra prediction mode (i.e., No. 1). The directional intra prediction mode may include intra prediction modes No. 2 to No. 66. The extended directional intra prediction mode can be applied to blocks of all sizes and can be applied to both luminance components and chrominance components. However, this is an example, and the embodiments of this document can be applied to situations where the number of intra prediction modes is different. Intra prediction mode No. 67 according to the situation can also be used. Intra prediction mode No. 67 may indicate a linear model (LM) mode.
[0137] Fig.10An example of an intra prediction mode to which embodiments of this document can be applied is illustrated.
[0138] Reference Fig.10 , based on the intra prediction mode No. 34 with the upper left diagonal prediction direction, the mode can be divided into an intra prediction mode with horizontal directionality and an intra prediction mode with vertical directionality. Fig.10 , H and V mean horizontal directivity and vertical directivity, respectively. Each of numbers -32 to 32 indicates a displacement of 1 / 32 unit at a sample grid position. Intra-prediction modes No. 2 to 33 have horizontal directivity, and intra-prediction modes No. 34 to 66 have vertical directivity. Intra-prediction mode No. 18 and intra-prediction mode No. 50 indicate a horizontal intra-prediction mode and a vertical intra-prediction mode, respectively. Intra-prediction mode No. 2 may be referred to as a lower left diagonal intra-prediction mode, intra-prediction mode No. 34 may be referred to as an upper left diagonal intra-prediction mode, and intra-prediction mode No. 66 may be referred to as an upper right diagonal intra-prediction mode.
[0139] In addition, intra prediction may use MRL using multiple reference lines. The MRL method may perform intra prediction using neighboring samples on a sample line located one or three sample distances away from the upper side and / or left side of the current block as reference samples.
[0140] Fig.11 Examples of reference sample lines for intra prediction using multiple reference lines are illustrated. Fig.11 The illustrated block unit may indicate a current block.
[0141] According to an exemplary embodiment, intra prediction may use a reference sample (or the first closest reference sample for the current block, i.e., a reference sample positioned at a zero sample distance from the current block) as a reference sample for prediction. According to another exemplary embodiment, multi-reference line (MRL) intra prediction is a method of using reference samples positioned at K sample distances (K is an integer of 1 or greater) from the left and upper boundaries of the current block, and may have more options and more accurate prediction performance for reference samples than in intra prediction using reference samples closest to the current block (i.e., positioned at a zero sample distance from the current block). Reference samples of the current block may also be referred to as neighboring samples of the current block or reference line samples of the current block, and reference line samples may also be referred to as samples on a reference line.
[0142] Reference Fig.11, the positions of neighboring reference samples located at 0, 1, 2, and 3 sample distances from the current block may be referred to as reference lines 0, 1, 2, and 3, respectively. Reference lines may be referred to as reference sample lines, reference sample rows, or reference sample columns, or also simply referred to as lines, rows, or columns. Reference lines 0, 1, 2, and 3 may be located in an order close to the current block. As an example, multi-reference line intra prediction may be performed based on reference lines 1 and 2. As another example, multi-reference line intra prediction may be performed based on reference lines 1 and 3. However, multi-reference line intra prediction according to the present document is not necessarily limited to these examples.
[0143] In addition, intra prediction based on multiple reference lines (MRL) can signal reference line information indicating which reference line is used. For example, reference line information can be signaled in the form of an intra_luma_ref_idx syntax element. If the intra_luma_ref_idx value is equal to 0, it can indicate that the first closest (i.e., located at zero sample distance) reference sample for the current block is used to perform intra prediction. If the intra_luma_ref_idx value is equal to 1, it can indicate that the second closest reference sample (i.e., located at one sample distance) for the current block is used to perform intra prediction. If the intra_luma_ref_idx value is equal to 2, it can indicate that the third or fourth closest (i.e., located at 2 or 3 sample distances) reference sample for the current block is used to perform intra prediction.
[0144] At the same time, when the block currently intended to be coded (encoded / decoded) is regarded as a coding unit, intra prediction can perform encoding / decoding without segmentation. In addition, intra prediction can also be performed by dividing the block currently intended to be coded into sub-partitions. Such an intra prediction method may be referred to as intra sub-partition (ISP) intra prediction or ISP-based intra prediction. That is, the ISP method can perform intra prediction by segmenting the block currently intended to be coded horizontally or vertically. In this case, a reconstructed block can be generated by performing encoding / decoding in units of segmented blocks, and the reconstructed block can be used as a reference block for the next segmented block. The current intra sub-partition (ISP) can be segmented according to the size of the block as in Table 1. Table 1 below shows the number of sub-partitions according to the block size when the intra sub-partition (ISP) mode is applied to the current block.
[0145] [Table 1]
[0146] Block size (CU) Number of partitions 4x4 Unavailable 4x8, 8x4 2 All other cases 4
[0147] Fig.12 An example of dividing sub-partitions according to intra sub-partitioning (ISP) is illustrated.
[0148] exist Fig.12 , (a) represents an example of partitioning a current block (original HxW partition, ie, a CU of an HxW size) in a horizontal direction and in a vertical direction when the current block is a 4x8 block or an 8x4 block.
[0149] like Fig.12 As illustrated in (a), the 4x8 block or the 8x4 block may be partitioned in the horizontal direction or in the vertical direction. When the block is partitioned in the horizontal direction, the block may be partitioned into two sub-partition blocks each having a size of (H / 2)xW, and when the block is partitioned in the vertical direction, the block may be partitioned into two sub-partition blocks each having a size of Hx(W / 2).
[0150] exist Fig.12 , (b) represents an example of partitioning the current block (original HxW partition, ie, HxW-sized CU) in the horizontal direction and in the vertical direction when the current block is a residual block excluding 4x4, 4x8, and 8x4 blocks.
[0151] like Fig.12 As illustrated in (b), the HxW block excluding 4x4, 4x8 and 8x4 blocks can be partitioned in the horizontal direction or in the vertical direction. In the case where the block is partitioned in the horizontal direction, the block can be partitioned into four sub-partition blocks each having a size of (H / 4)xW, and in the case where the block is partitioned in the vertical direction, the block can be partitioned into four sub-partition blocks each having a size of Hx(W / 4).
[0152] In order to reduce the coding complexity, the intra-frame sub-partitioning method generates an MPM list according to the corresponding partitioning method (horizontal partitioning and vertical partitioning), and generates the best mode by comparing the appropriate prediction mode among the prediction modes in the generated MPM list from the perspective of rate-distortion optimization (RDO). In addition, when the above-mentioned multi-reference line (MRL) intra-frame prediction is used, the intra-frame sub-partitioning method cannot be used. That is, the intra-frame sub-partitioning method can only be applied when the zeroth reference line (ie, the intra_luma_ref_idx value is equal to 0) is used. In addition, when the above-mentioned intra-frame sub-partitioning method is used, the above-mentioned PDPC cannot be used.
[0153] According to the intra subpartitioning method, whether intra subpartitioning is applied is first transmitted in units of blocks, and if the current block uses intra subpartitioning (intra_subpartitions_mode_flag), information on whether the partitioning method is horizontal partitioning or vertical partitioning (intra_subpartitions_split_flag) is encoded / decoded.
[0154] In the case of applying the intra sub-partitioning method, the intra prediction mode for the current block is equally applied to the sub-partition, and the neighboring reference samples are derived and used in units of sub-partitions to enhance the intra prediction performance. That is, in the case of applying the intra sub-partitioning method, the residual sample processing process is performed in units of sub-partitions. In other words, intra prediction samples are derived with respect to each sub-partition, and the residual signal (residual sample) of the corresponding sub-partition is added to the derived intra prediction sample to obtain a reconstructed sample. The residual signal (residual sample) can be derived through a dequantization / inverse transform process based on the residual information (quantized transform coefficient information or residual coding syntax) in the above-mentioned bitstream. That is, the derivation of the prediction sample for the first sub-partition and the derivation of the residual sample can be performed, and based on this, the reconstructed sample of the first sub-partition can be derived. In this case, in the case of deriving the prediction sample for the second sub-partition, some of the reconstructed samples in the first sub-partition (e.g., the upper or left neighboring reference sample of the second sub-partition) can be used as the neighboring reference sample for the second sub-partition. In the same manner, the derivation of the prediction samples and the derivation of the residual samples for the second sub-partition may be performed, and based on this, the reconstructed samples of the second sub-partition may be derived. In this case, in the case of deriving the prediction samples of the third sub-partition, some of the reconstructed samples in the second sub-partition (e.g., the upper or left neighboring reference samples of the third sub-partition) may be used as neighboring reference samples for the third sub-partition. The intra-sub-partitioning method may be applied in the same manner even with respect to the remaining sub-partitions.
[0155] As described above, as intra prediction, a multi-reference line (MRL)-based intra prediction method and a sub-partition (ISP)-based intra prediction method may be applied, or a normal intra prediction method excluding a specific intra prediction method such as MRL, ISP, etc. may be applied. In this case, normal intra prediction that is not a specific intra prediction type (e.g., MRL or ISP) uses 67 intra prediction modes to perform intra prediction encoding / decoding, and multi-reference line intra prediction uses 65 intra prediction modes excluding plane mode and DC mode to perform intra prediction encoding / decoding. In addition, sub-partition intra prediction uses 66 intra prediction modes excluding DC mode to perform intra prediction encoding / decoding. Since the above three types of intra prediction (existing intra prediction, multi-reference line intra prediction, and sub-partition intra prediction) use different numbers of intra prediction modes to perform intra prediction encoding / decoding, they have different MPM list generation methods for their predictions.
[0156] More specifically, normal intra prediction uses all 67 intra prediction modes to construct an MPM list including 6 MPM candidates. Multi-reference line intra prediction does not use planar mode and DC mode, and uses 65 intra prediction modes excluding planar mode and DC mode to construct an MPM list including 6 MPM candidates. Sub-partition intra prediction does not use DC mode, but uses 66 intra prediction modes excluding DC mode to construct an MPM list including 6 MPM candidates. In this case, sub-partition intra prediction constructs the MPM list in different methods depending on the horizontal partition and the vertical partition. As described above, for one intra prediction, different methods are used to construct an MPM list including 6 MPM candidates.
[0157] Therefore, in order to improve the coding efficiency of intra prediction, a unified MPM list generation method can be used. Therefore, this document proposes a scheme that can construct the MPM list used in normal intra prediction, multi-reference line intra prediction and sub-partition intra prediction in a unified method. In an embodiment, it is possible to generate an MPM list for normal intra prediction, multi-reference line intra prediction and sub-partition intra prediction after generating a unified temporary MPM list. In another embodiment, after generating a unified temporary MPM list, a specific prediction mode (e.g., DC mode) can be added according to a specific intra prediction type (i.e., normal intra prediction, multi-reference line intra prediction and sub-partition intra prediction), and taking this into account, an MPM list for normal intra prediction, multi-reference line intra prediction and sub-partition intra prediction can be generated.
[0158] By using the unified MPM list generation method according to the embodiment of this document, the encoding / decoding structure of intra-frame prediction can be simplified, and the video encoding / decoding efficiency can be improved by improving the intra-frame mode encoding / decoding efficiency.
[0159] Fig.13 is a diagram illustrating an implementation of a method for generating a unified MPM list according to the present document.
[0160] In this embodiment, a method for constructing a unified MPM list taking into account unused specific prediction modes (e.g., planar mode or DC mode) according to specific intra prediction types (i.e., normal intra prediction, multi-reference line intra prediction, and sub-partition intra prediction) will be described.
[0161] As an embodiment, the method for generating an MPM list including 6 MPM candidates used in normal intra prediction may be equally applied to the MPM list generation method for multi-reference line intra prediction and sub-partition intra prediction. In this case, the MPM list generation method used in normal intra prediction may be an existing MPM list generation method, or a method by improving the existing MPM list generation method. For example, Figure 8 and Fig. 9 The above method is used to construct the MPM list used in normal intra prediction.
[0162] Here, since the MPM list is generated by normal intra prediction taking into account all 67 intra prediction modes, it includes planar mode and DC mode. However, since multi-reference line intra prediction does not use planar mode and DC mode, and sub-partition intra prediction does not use DC mode, they can consider this to generate their corresponding MPM lists.
[0163] Reference Fig.13 , an MPM list including 6 MPM candidates used in normal intra prediction can be temporarily generated. For convenience of explanation, the MPM list temporarily generated first is referred to as a temporary MPM list. In other words, the temporary MPM list is an MPM list including 6 MPM candidates used in normal intra prediction (or an MPM list improved by various improvement methods), and can be generated according to Figure 8 and Fig. 9 The above method is used to construct the temporary MPM list. Even when generating the MPM list for multi-reference line intra prediction and sub-partition intra prediction, such a temporary MPM list construction method can be equally applied. Therefore, a temporary MPM list including equal 6 MPM candidates can be generated for all normal intra prediction, multi-reference line intra prediction and sub-partition intra prediction.
[0164] In this case, in the case of multi-reference line intra prediction, the planar mode and the DC mode may not be used, and in the case of sub-partition intra prediction, the DC mode may not be used. In this case, a specific mode not used in the corresponding prediction method can be removed from the equally generated temporary MPM list, and a suitable MPM list can be reconstructed according to the corresponding prediction method.
[0165] As an example, Fig.13 As illustrated in (a), (b) and (c) of FIG. 1 , a temporary MPM list may be generated with respect to normal intra prediction, multi-reference line intra prediction and sub-partition intra prediction. In this case, each of the temporary MPM lists equally includes 6 MPM candidates. In addition, since multi-reference line intra prediction does not use planar mode and DC mode, as shown in FIG. Fig.13As illustrated in (b), the plane mode of MPM index 0 and the DC mode of MPM index 2 can be removed from the temporary MPM list, and the MPM candidates in the temporary MPM list can be rearranged. Therefore, an MPM list including 4 MPM candidates can be finally generated. In addition, since the DC mode is not used in the sub-partition intra prediction, as shown in FIG. Fig.13 As illustrated in (c), the DC mode of MPM index 2 can be removed from the temporary MPM list, and the MPM candidates in the temporary MPM list can be rearranged. Therefore, an MPM list including 5 MPM candidates can be finally generated.
[0166] Fig.14 is a diagram illustrating another embodiment of a method for generating a unified MPM list according to the present document.
[0167] In this embodiment, a method will be described in which a specific prediction mode (e.g., DC mode) is added according to a specific intra prediction type (i.e., normal intra prediction, multi-reference line intra prediction, and sub-partition intra prediction), and a unified MPM list is constructed in consideration of this. As an example, a method for constructing a unified MPM list in the case where the DC mode is additionally used in multi-reference line intra prediction will be described.
[0168] As an embodiment, the method for generating an MPM list including 6 MPM candidates used in normal intra prediction can be equally applied to the MPM list generation method for multiple reference line intra prediction and sub-partition intra prediction. In this case, the MPM list generation method used in normal intra prediction can be an existing MPM list generation method, or a method by improving the existing MPM list generation method. For example, the above-mentioned Figure 8 and Fig. 9 Method to construct the MPM list used in normal intra prediction.
[0169] In this case, since prediction is performed by adding the DC mode in the multi-reference line intra prediction, the DC mode is used and the planar mode is not used. In addition, the DC mode is not used in the sub-partition intra prediction. In this embodiment, this can be considered to generate the corresponding MPM list.
[0170] Reference Fig.14 , an MPM list including 6 MPM candidates used in normal intra prediction can be temporarily generated. For convenience of explanation, the MPM list temporarily generated first is referred to as a temporary MPM list. In other words, the temporary MPM list is an MPM list including 6 MPM candidates used in normal intra prediction (or an MPM list improved by various improvement methods), and can be generated according to Figure 8 and Fig. 9The above method is used to construct the temporary MPM list. Even when generating the MPM list for multi-reference line intra prediction and sub-partition intra prediction, such a temporary MPM list construction method can be equally applied. Therefore, a temporary MPM list including equal 6 MPM candidates can be generated for all normal intra prediction, multi-reference line intra prediction and sub-partition intra prediction.
[0171] In this case, in the case of multi-reference line intra prediction, the DC mode may be used and the planar mode may not be used, whereas in the case of sub-partition intra prediction, the DC mode may not be used. In this case, a specific mode not used in the corresponding prediction method may be removed from the equally generated temporary MPM list, and a suitable MPM list may be reconstructed according to the corresponding prediction method.
[0172] As an example, Fig.14 As illustrated in (a), (b) and (c) of FIG. 1 , a temporary MPM list may be generated with respect to normal intra prediction, multi-reference line intra prediction and sub-partition intra prediction. In this case, each of the temporary MPM lists equally includes 6 MPM candidates. In addition, since multi-reference line intra prediction additionally uses the DC mode and does not use the planar mode, as shown in FIG. Fig.14 As illustrated in (b), the plane mode of MPM index 0 can be removed from the temporary MPM list, and the MPM candidates in the temporary MPM list can be rearranged. Therefore, an MPM list including 5 MPM candidates can be finally generated. In addition, since the sub-partition intra prediction does not use the DC mode, as shown in FIG. Fig.14 As illustrated in (c), the DC mode of MPM index 2 can be removed from the temporary MPM list, and the MPM candidates in the temporary MPM list can be rearranged. Therefore, an MPM list including 5 MPM candidates can be finally generated.
[0173] Fig.13 and Fig.14An example of generating an MPM list will be described, and the basic concept of the proposed method is to generate an MPM list including multiple (e.g., 6) MPM candidates in normal intra prediction, and then use the generated MPM list equally even in multi-reference line intra prediction and sub-partition intra prediction. However, since a specific mode (e.g., plane mode or DC mode) is not used in multi-reference line intra prediction and sub-partition intra prediction, the MPM candidates can be configured in consideration of this. If there is a specific mode (e.g., plane mode or DC mode) that is not used in the corresponding prediction method in the MPM list, the specific mode can be removed, and the MPM candidates can be rearranged so that the MPM list for the corresponding prediction method can be constructed. In addition, in the above-mentioned embodiment, it has been described that an MPM list (i.e., a temporary MPM list) including multiple (e.g., 6) MPM candidates for the corresponding prediction method is generated, and then the MPM list is finally constructed by removing the specific mode (e.g., plane mode or DC mode) that is not used in the corresponding prediction method. However, this is merely exemplary, and the process of generating a temporary MPM list can be omitted. For example, after deriving 6 MPM candidates (without generating a temporary MPM list), an MPM list suitable for the corresponding prediction method can be constructed by excluding a specific mode (e.g., planar mode or DC mode) not used in the corresponding prediction method. In this case, the planar mode is included in the 6 MPM candidates, and when the planar mode is not used like multi-reference line intra prediction, 5 MPM candidates excluding this mode can be derived, and an MPM list including the 5 MPM candidates can be constructed.
[0174] In addition, the above-mentioned embodiment ( Fig.13 and Fig.14 The embodiments described above are merely exemplary to help understand the basic concept of the unified MPM list generation method proposed in this document. The basic concept of the method proposed in this document is to generate a unified MPM list without distinguishing between normal intra prediction, multi-reference line intra prediction, and sub-partition intra prediction when generating the MPM list. Therefore, even in multi-reference line intra prediction and sub-partition intra prediction, the MPM list generated in normal intra prediction can be used equally. In this case, the intra prediction modes that are not used in multi-reference line intra prediction and sub-partition intra prediction can be considered to finally construct an MPM list suitable for the corresponding intra prediction.
[0175] That is, according to the implementation proposed in this document, since a unified MPM list can be constructed, the encoding / decoding structure of intra-frame prediction can be simplified. In addition, the overall video encoding / decoding performance can be improved by improving the encoding / decoding efficiency of the intra-frame prediction mode.
[0176] Fig.15 is a flowchart schematically illustrating an encoding method that can be performed by an encoding device according to an embodiment of the present disclosure.
[0177] Fig.15 The method disclosed in can be Figure 2 Specifically, Fig.15 Steps S1500 to S1530 may be performed by Figure 2 The predictor 220 disclosed in (specifically, the intra-frame predictor 222) is executed, and Fig.15 Step S1540 may be performed by Figure 2 The entropy encoder 240 disclosed in is executed. In addition, Fig.15 The method disclosed in may include the previous embodiments of this document. Fig.15 Detailed descriptions that are redundant with the descriptions of the previous implementation methods are omitted or simplified.
[0178] Reference Fig.15 , the encoding apparatus may generate reference line index information related to indicating a reference line used for intra prediction of a current block (S1500).
[0179] As described above, the reference line index information represents a reference line used for multi-reference line (MRL) based intra prediction, which may be information indicating neighboring reference samples located at 0, 1, 2, and 3 sample distances from the current block.
[0180] For example, the reference line index information may be represented in the form of the above-mentioned intra_luma_ref_idx syntax element, and may be an index value related to indicating any one of reference lines 0, 1, 2, and 3 based on the value of intra_luma_ref_idx. For example, if the value of the reference line index information (e.g., intra_luma_ref_idx) is equal to 0, it may mean that the first closest reference line ( Fig.11 Intra prediction is performed using samples of the second, third, or fourth closest reference line (reference line 1, 2, or 3) for the current block. If the value of the reference line index information (e.g., intra_luma_ref_idx) is not equal to 0 (i.e., the value is equal to 1, 2, or 3), it may mean that intra prediction is performed using samples of the second, third, or fourth closest reference line (reference line 1, 2, or 3) for the current block. That is, if the value of the reference line index information (e.g., intra_luma_ref_idx) is not equal to 0 (i.e., the value is equal to 1, 2, or 3), it may mean that a multi-reference line (MRL) based intra prediction method is used.
[0181] In an embodiment, the encoding device may determine whether to perform intra prediction using multiple reference lines for the current block, generate reference line index information based on the determination, and signal it to the decoding device.
[0182] The encoding apparatus may construct a most probable mode (MPM) list including candidate intra prediction modes (most probable mode (MPM) candidate modes) for intra prediction of a current block ( S1510 ).
[0183] In an embodiment, the encoding device may construct an MPM list based on whether a specific intra prediction method (e.g., multi-reference line intra prediction, sub-partition intra prediction, etc.) is applied. In this case, the previous embodiment may be applied in the process of constructing the MPM list, and reference has been made to Fig.13 and Fig.14 This is described in detail.
[0184] For example, the encoding device may generate an MPM list based on whether multi-reference line intra prediction is applied. For example, based on using multiple reference lines to perform intra prediction for the current block (that is, the case where the value of the reference line index information is not equal to 0), the encoding device may derive the DC mode as one of the candidate intra prediction modes and include it in the MPM list.
[0185] In addition, if intra prediction is performed using multiple reference lines as described above, the plane mode may not be available. Therefore, based on the case where the value of the reference line index information is not equal to 0, the encoding device may not use the plane mode as the intra prediction mode of the current block. For example, the encoding device may construct a temporary MPM list for multi-reference line intra prediction. In this case, if the plane mode is included in the candidate intra prediction modes in the temporary MPM list, the encoding device may remove the plane mode from the temporary MPM list and reconstruct the temporary MPM list into an MPM list. Here, the process of constructing the temporary MPM list may be omitted depending on the implementation method of the MPM list. For example, since the plane mode is not used when the value of the reference line index information is not equal to 0, it is first determined based on the value of the reference line index information that the plane mode is not used as the intra prediction mode, and then the five candidate intra prediction modes (including the DC mode) excluding the plane mode from the six candidate intra prediction modes can be derived and constructed into the MPM list. Therefore, according to the algorithm for implementing the MPM list, the MPM list can be implemented without the intermediate process of constructing the temporary MPM list.
[0186] In addition, the encoding device may generate and signal information related to indicating whether the planar mode is used as the intra-frame prediction mode of the current block. For example, the planar flag information may be used as information indicating whether the planar mode is used as the intra-frame prediction mode of the current block. The planar flag information may be the above-mentioned non-planar flag (e.g., intra_luma_not_planar_flag). If the value of the planar flag information (i.e., the non-planar flag) is equal to 1, it may mean that the planar mode is not used as the intra-frame prediction mode of the current block, and if the value of the planar flag information (i.e., the non-planar flag) is equal to 0, it may mean that the planar mode is used as the intra-frame prediction mode of the current block.
[0187] In an embodiment, based on the case where the value of the reference line index information is not equal to 0, the encoding device may not signal the plane flag information (i.e., the non-plane flag). If the plane flag information (i.e., the non-plane flag) is not signaled, the value of the plane flag information (i.e., the non-plane flag) may be inferred to be 1, which indicates that the plane mode is not used as the intra-frame prediction mode of the current block. That is, since the plane mode is not used as the intra-frame prediction mode of the current block when the value of the reference line index information is not equal to 0 as described above, the plane flag information (i.e., the non-plane flag) may not be signaled but its value may be inferred to be 1, thereby saving the number of bits.
[0188] Alternatively, for another example, the encoding device may generate an MPM list based on subpartition mode information related to indicating whether subpartition intra prediction is used for the current block. For example, the subpartition mode information may use the above-mentioned intra_subpartitions_mode_flag syntax element, and if the value of intra_subpartitions_mode_flag is equal to 1, it may mean that subpartition intra prediction is used for the current block, and if the value of intra_subpartitions_mode_flag is equal to 0, it may mean that subpartition intra prediction is not used for the current block. If subpartition intra prediction is used as described above, DC mode may not be available. Therefore, if the subpartition mode information indicates that subpartition intra prediction is used for the current block (for example, the value of intra_subpartitions_mode_flag is equal to 1), the encoding device may not use DC mode as the intra prediction mode of the current block. For example, the encoding device may construct a temporary MPM list for subpartition intra prediction. In this case, if the DC mode is included in the candidate intra prediction modes in the temporary MPM list, the encoding device can remove the DC mode from the temporary MPM list and reconstruct the temporary MPM list into an MPM list. Here, the process of constructing the temporary MPM list can be omitted depending on the implementation method of the MPM list. For example, since the DC mode is not used when sub-partition intra prediction is used, the DC mode is not included in the candidate intra prediction modes, thereby finally constructing an MPM list including five candidate intra prediction modes. Therefore, the MPM list can be implemented without the intermediate process of constructing the temporary MPM list.
[0189] In some embodiments, if the value of the reference line index information is equal to 0, the encoding device may generate sub-partition mode information and signal it to the decoding device. In other words, if multiple reference lines are not used (i.e., intra-frame prediction is performed using samples of the first closest reference line for the current block), the encoding device may determine whether to perform sub-partition intra-frame prediction and generate sub-partition mode information based on the determination. In this case, if the value of the reference line index information is equal to 0 and the value of the sub-partition mode information is equal to 1, the encoding device may not use the DC mode as the intra-frame prediction mode of the current block. That is, in this case, the encoding device can generate the MPM list by not including the DC mode in the candidate intra-frame prediction mode.
[0190] The encoding apparatus may derive an intra prediction mode for the current block from candidate intra prediction modes included in the MPM list ( S1520 ).
[0191] In an embodiment, the encoding device may derive an intra prediction mode with an optimal rate-distortion (RD) cost by performing various intra prediction modes on the current block, and may determine it as the intra prediction mode of the current block. In this case, the encoding device may derive the optimal intra prediction mode of the current block based on the intra prediction modes including two non-directional intra prediction modes and 65 directional intra prediction modes. Alternatively, the encoding device may determine the optimal intra prediction mode by using only the MPM candidates constructed in the MPM list.
[0192] For example, if the value of the reference line index information is not equal to 0, the encoding device can derive the optimal intra prediction mode of the current block by using the MPM candidate intra prediction mode included in the MPM list. That is, in this case, the intra prediction mode of the current block can be determined only from the candidate intra prediction modes including the DC mode in the MPM list. In addition, if the value of the reference line index information is not equal to 0, the encoding device may not encode / signal the MPM flag information. If the MPM flag information is not encoded / signaled, the value of the MPM flag information can be inferred to be 1. As described above, the MPM flag information can be represented in the form of an intra_luma_mpm_flag syntax element. For example, if the value of intra_luma_mpm_flag is equal to 1, it may mean that the intra prediction mode of the current block is selected from the MPM candidate intra prediction mode, and if the value of intra_luma_mpm_flag is equal to 0, it may mean that the intra prediction mode of the current block is not selected from the MPM candidate intra prediction mode.
[0193] The encoding apparatus may generate MPM index information related to indicating an intra prediction mode of the current block ( S1530 ).
[0194] In an embodiment, if the value of the reference line index information is not equal to 0 and the value of the MPM flag information is inferred to be 1, the encoding device may generate an index value related to indicating one of the candidate intra prediction modes in the MPM list and encode it as the MPM index information. That is, if the value of the reference line index information is not equal to 0 and the value of the MPM flag information is inferred to be 1, the MPM index information may be encoded / signaled.
[0195] The encoding apparatus may encode the image information including the MPM index information and the reference line index information (S1540).
[0196] In an embodiment, as described above, the encoding device may encode image information including reference line index information determined based on whether to apply multi-reference line-based intra prediction and MPM index information indicating an intra prediction mode of a current block derived based on an MPM list, and output the image information in the form of a bitstream. In addition, as described above, the encoding device may include sub-partition mode information determined based on whether to apply sub-partition intra prediction to the current block in the image information and encode it.
[0197] In addition, the encoding device may generate a prediction sample of the current block based on the intra prediction mode of the current block. In an embodiment, the encoding device may derive at least one neighboring reference sample from the neighboring reference samples of the current block based on the intra prediction mode, and generate a prediction sample based on the neighboring reference sample. Here, the neighboring reference sample may be derived based on the reference line index information, and may include, for example, a neighboring reference sample included in a reference line indicated by the reference line index information.
[0198] In addition, the encoding device may derive a residual sample of the current block based on the predicted sample of the current block and the original sample of the current block. In addition, the encoding device may generate residual information of the current block based on the residual sample and encode the image information including the residual information. Here, the residual information may include information such as value information, position information, a transform scheme, a transform kernel, and a quantization parameter of a quantized transform coefficient derived by performing transform and quantization on the residual sample.
[0199] That is, the encoding device can encode the above-mentioned image information including intra-frame prediction mode information (MPM index information, reference line index information, etc.) and / or residual information of the current block and output it in a bitstream.
[0200] The bitstream can be sent to a decoding device via a network or a (digital) storage medium. Here, the network can include a broadcast network and / or a communication network, and the digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc.
[0201] The above process of generating the prediction sample of the current block can be represented by Figure 2 The intra-frame predictor 222 of the encoding device 200 disclosed in the embodiment of the present invention is executed, and the process of deriving the residual sample can be performed by Figure 2 The subtractor 231 of the encoding device 200 disclosed in the embodiment of the present invention is executed, and the process of generating residual information and encoding it can be performed by Figure 2 The residual processor 230 and the entropy encoder 240 of the encoding device 200 disclosed in the disclosure are performed.
[0202] Fig.16is a flowchart schematically illustrating a decoding method that can be performed by a decoding device according to an embodiment of the present disclosure.
[0203] Fig.16 The method disclosed in can be Figure 3 Specifically, Fig.16 Steps S1600 to S1630 may be performed by Figure 3 The entropy decoder 310 and / or the predictor 330 (specifically, the intra-frame predictor 331) disclosed in Fig.16 Step S1640 may be performed by Figure 3 In addition, Fig.16 The method disclosed in may include the previous embodiments of this document. Fig.16 Detailed descriptions that are redundant with the description of the previous implementation methods are omitted or simplified.
[0204] Reference Fig.16 , the decoding apparatus may obtain most probable mode (MPM) index information and reference line index information from the bitstream (S1600).
[0205] As described above, the reference line index information represents a reference line used for multi-reference line (MRL) based intra prediction, which may be information indicating neighboring reference samples located at 0, 1, 2, and 3 sample distances from the current block.
[0206] For example, the reference line index information may be represented in the form of an intra_luma_ref_idx syntax element, and may be an index value related to indicating any one of reference lines 0, 1, 2, and 3 based on the value of intra_luma_ref_idx. For example, if the value of the reference line index information (e.g., intra_luma_ref_idx) is equal to 0, it may mean that the first closest reference line ( Fig.11 Intra prediction is performed using samples of the second, third, or fourth closest reference line (reference line 1, 2, or 3) for the current block. If the value of the reference line index information (e.g., intra_luma_ref_idx) is not equal to 0 (i.e., the value is equal to 1, 2, or 3), it may mean that intra prediction is performed using samples of the second, third, or fourth closest reference line (reference line 1, 2, or 3) for the current block. That is, if the value of the reference line index information (e.g., intra_luma_ref_idx) is not equal to 0 (i.e., the value is equal to 1, 2, or 3), it may mean that a multi-reference line (MRL) based intra prediction method is used.
[0207] In an embodiment, the decoding device may obtain a reference line index information (e.g., intra_luma_ref_idx) syntax element from a bitstream and parse (decode) it. In addition, the decoding device may obtain a value of the reference line index information (e.g., intra_luma_ref_idx) based on the parsing result and determine whether to apply multi-reference line intra prediction based on the value.
[0208] In addition, the decoding device can obtain MPM index information related to the intra-frame prediction mode for the current block among the candidate intra-frame prediction modes in the MPM list from the bitstream, and parse (decode) it. That is, the decoding device can derive the intra-frame prediction mode of the current block from the MPM list based on the MPM index information.
[0209] The decoding apparatus may construct an MPM list including candidate intra prediction modes for intra prediction of a current block ( S1610 ).
[0210] In an embodiment, the decoding device may construct an MPM list based on whether a specific intra prediction method (e.g., multi-reference line intra prediction, sub-partition intra prediction, etc.) is applied. In this case, the previous embodiment may be applied in the process of constructing the MPM list, and reference has been made to Fig.13 and Fig.14 This is described in detail.
[0211] In an example, the decoding device may determine whether to apply multi-reference line intra prediction based on the reference line index information, and generate an MPM list accordingly. For example, based on using multiple reference lines to perform intra prediction for the current block (that is, the case where the value of the reference line index information is not equal to 0), the decoding device may derive the DC mode as one of the candidate intra prediction modes and include it in the MPM list.
[0212] In addition, if intra prediction is performed using multiple reference lines as described above, the plane mode may not be available. Therefore, based on the case where the value of the reference line index information is not equal to 0, the decoding device may not use the plane mode as the intra prediction mode of the current block. For example, the decoding device may construct a temporary MPM list for multi-reference line intra prediction. In this case, if the plane mode is included in the candidate intra prediction modes in the temporary MPM list, the decoding device may remove the plane mode from the temporary MPM list and reconstruct the temporary MPM list into an MPM list. Here, the process of constructing the temporary MPM list may be omitted depending on the implementation method of the MPM list. For example, since the plane mode is not used when the value of the reference line index information is not equal to 0, it is first determined based on the value of the reference line index information that the plane mode is not used as the intra prediction mode, and then the five candidate intra prediction modes (including the DC mode) excluding the plane mode from the six candidate intra prediction modes can be derived and constructed into the MPM list. Therefore, according to the algorithm for implementing the MPM list, the MPM list can be implemented without the intermediate process of constructing the temporary MPM list.
[0213] In addition, the decoding device can obtain information related to whether the plane mode is used as the intra prediction mode of the current block from the bitstream. In this case, the plane flag information can be used as information indicating whether the plane mode is used as the intra prediction mode of the current block. The plane flag information can be the above-mentioned non-planar flag (e.g., intra_luma_not_planar_flag). If the value of the plane flag information (i.e., the non-planar flag) is equal to 1, it may mean that the plane mode is not used as the intra prediction mode of the current block, and if the value of the plane flag information (i.e., the non-planar flag) is equal to 0, it may mean that the plane mode is used as the intra prediction mode of the current block.
[0214] In an embodiment, if the value of the reference line index information is not equal to 0, the plane flag information (i.e., non-plane flag) may not be signaled. In this case, the decoding device may not obtain the plane flag information (i.e., non-plane flag) from the bitstream. Therefore, based on the case where the value of the reference line index information is not equal to 0, the value of the plane flag information (i.e., non-plane flag) may be inferred to be 1. If the value of the plane flag information (i.e., non-plane flag) is inferred to be 1, it may mean that the plane mode is not used as the intra-frame prediction mode of the current block. That is, since the plane mode is not used as the intra-frame prediction mode of the current block when the value of the reference line index information is not equal to 0 as described above, the plane flag information (i.e., non-plane flag) may not be signaled, but its value may be inferred to be 1, thereby saving the number of bits.
[0215] Alternatively, for another example, the decoding device may generate an MPM list based on subpartition mode information related to indicating whether subpartition intra prediction is used for the current block. For example, the subpartition mode information may use the above-mentioned intra_subpartitions_mode_flag syntax element, and if the value of intra_subpartitions_mode_flag is equal to 1, it may mean that subpartition intra prediction is used for the current block, and if the value of intra_subpartitions_mode_flag is equal to 0, it may mean that subpartition intra prediction is not used for the current block. If subpartition intra prediction is used as described above, DC mode may not be available. Therefore, if the subpartition mode information indicates that subpartition intra prediction is used for the current block (for example, the value of intra_subpartitions_mode_flag is equal to 1), the decoding device may not use DC mode as the intra prediction mode of the current block. For example, the decoding device may construct a temporary MPM list for subpartition intra prediction. In this case, if the DC mode is included in the candidate intra prediction modes in the temporary MPM list, the decoding device can remove the DC mode from the temporary MPM list and reconstruct the temporary MPM list into an MPM list. Here, the process of constructing the temporary MPM list can be omitted depending on the implementation method of the MPM list. For example, since the DC mode is not used when sub-partition intra prediction is used, the DC mode is not included in the candidate intra prediction modes, thereby finally constructing an MPM list including five candidate intra prediction modes. Therefore, the MPM list can be implemented without the intermediate process of constructing the temporary MPM list.
[0216] In some embodiments, if the value of the reference line index information is equal to 0, the decoding device may obtain the sub-partition mode information from the bitstream. In other words, if multiple reference lines are not used (i.e., intra-frame prediction is performed using samples of the first closest reference line for the current block), the decoding device may obtain sub-partition information related to an indication of whether to perform sub-partition intra-frame prediction and decode it. In this case, if the value of the reference line index information is equal to 0 and the value of the sub-partition mode information is equal to 1, the decoding device may not use the DC mode as the intra-frame prediction mode of the current block. That is, in this case, the decoding device may generate an MPM list by not including the DC mode in the candidate intra-frame prediction modes.
[0217] The decoding apparatus may derive an intra prediction mode for the current block from candidate intra prediction modes included in the MPM list based on the MPM index information ( S1620 ).
[0218] In an embodiment, the decoding device may obtain intra prediction mode information for the current block from the bitstream. The intra prediction mode information is information for indicating the intra prediction mode of the current block, which may include MPM flag information, MPM index information, residual mode information, and the like.
[0219] In this case, if the value of the reference line index information is not equal to 0, the MPM flag information may not be signaled from the encoding device. If the MPM flag information is not signaled, the decoding device may infer the value of the MPM flag information to be 1. As described above, the MPM flag information may be represented in the form of an intra_luma_mpm_flag syntax element. For example, if the value of intra_luma_mpm_flag is equal to 1, it may mean that the intra-frame prediction mode of the current block is selected from the MPM candidate intra-frame prediction mode, and if the value of intra_luma_mpm_flag is equal to 0, it may mean that the intra-frame prediction mode of the current block is not selected from the MPM candidate intra-frame prediction mode.
[0220] In addition, if the value of the reference line index information is not equal to 0 and the value of the MPM flag information is derived as 1, the MPM index information may be signaled from the encoding device. That is, the decoding device may obtain the MPM index information from the bitstream and decode it. As described above, the MPM index information includes an index value related to the intra prediction mode of the current block among the candidate intra prediction modes included in the MPM list, and may be represented, for example, in the form of an intra_luma_mpm_idx syntax element.
[0221] That is, if the value of the reference line index information is not equal to 0 and the value of the MPM flag information is derived as 1, the decoding device can obtain and decode the MPM index information, and derive the intra-frame prediction mode of the current block from the MPM list based on the MPM index information.
[0222] The decoding apparatus may generate a prediction sample for the current block based on the intra prediction mode of the current block ( S1630 ).
[0223] In an embodiment, the decoding device may derive at least one neighboring reference sample from the neighboring reference samples of the current block based on the derived intra prediction mode, and generate a prediction sample based on the neighboring reference sample. Here, the neighboring reference sample may be derived based on the reference line index information, and may include, for example, the neighboring reference sample included in the reference line indicated by the reference line index information.
[0224] The decoding apparatus may generate a reconstructed sample for the current block based on the prediction sample ( S1640 ).
[0225] In an embodiment, the decoding device may immediately use the prediction samples as reconstructed samples according to the prediction mode, or may add residual samples to the prediction samples to generate reconstructed samples.
[0226] If there are residual samples for the current block, the decoding device may receive information about the residual for the current block. The information about the residual may include transform coefficients for the residual samples. The decoding device may derive residual samples (or residual sample arrays) for the current block based on the residual information. The decoding device may generate reconstructed samples based on the predicted samples and the residual samples, and may derive a reconstructed block or a reconstructed picture based on the reconstructed samples. Thereafter, as described above, the decoding device may apply an in-loop filtering process such as a deblocking filter and / or an SAO process to the reconstructed picture when necessary to improve subjective / objective image quality.
[0227] In the above-mentioned embodiments, although the methods have been described based on flowcharts in the form of a series of steps or units, the embodiments of this document are not limited to the order of these steps, and some of these steps may be performed in an order different from the order of other steps or may be performed simultaneously with other steps. In addition, those skilled in the art will understand that the steps shown in the flowcharts are not exclusive, and without affecting the scope of rights of this document, these steps may include additional steps or one or more steps in the flowcharts may be deleted.
[0228] The above-mentioned method according to the present document may be implemented in software form, and the encoding device and / or decoding device according to the present document may be included in an apparatus for performing image processing, such as a TV, a computer, a smart phone, a set-top box, or a display device.
[0229] In this document, when the implementation is implemented in software form, the above-mentioned method can be implemented as a module (program, function, etc.) for performing the above-mentioned functions. The module can be stored in a memory and executed by a processor. The memory can be arranged inside or outside the processor and connected to the processor by various well-known means. The processor may include an application-specific integrated circuit (ASIC), other chipsets, logic circuits and / or data processing devices. The memory may include a read-only memory (ROM), a random access memory (RAM), a flash memory, a memory card, a storage medium and / or other storage devices. That is, the implementation described in this document can be implemented and executed on a processor, a microprocessor, a controller or a chip. For example, the functional unit illustrated in the accompanying drawings can be implemented and executed on a computer, a processor, a microprocessor, a controller or a chip. In this case, information (e.g., information about instructions) or an algorithm for such implementation can be stored in a digital storage medium.
[0230] In addition, the decoding device and the encoding device to which this document is applied may be included in a multimedia broadcast transmission and reception device, a mobile communication terminal, a home theater video device, a digital theater video device, a surveillance camera, a video chat device, a real-time communication device such as video communication, a mobile streaming device, a storage medium, a camera, a video on demand (VoD) service provider, an over-the-top (OTT) video device, an Internet streaming service provider, a three-dimensional (3D) video device, a virtual reality (VR) device, an augmented reality (AR) device, a video phone video device, a transportation tool terminal (e.g., a vehicle (including an autonomous vehicle) terminal, an aircraft terminal, and a ship terminal) and a medical video device, and may be used to process a video signal or a data signal. For example, an over-the-top (OTT) video device may include a game console, a Blueray player, an Internet access TV, a home theater system, a smart phone, a tablet PC, and a digital video recorder (DVR).
[0231] In addition, the processing method of applying this document can be generated in the form of a program executed by a computer, and can be stored in a computer-readable recording medium. The multimedia data with a data structure according to this document can also be stored in a computer-readable recording medium. Computer-readable recording media include all kinds of storage devices storing computer-readable data. Computer-readable recording media may include, for example, Blueray discs (BD), universal serial buses (USB), ROMs, PROMs, EPROMs, EEPROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, and optical data storage devices. In addition, computer-readable recording media include media implemented in the form of carrier waves (e.g., transmitted via the Internet). In addition, the bit stream generated using the encoding method can be stored in a computer-readable recording medium, or can be transmitted via wired and wireless communication networks.
[0232] In addition, the embodiments of this document may be implemented as a computer program product using program code. The program code may be executed by a computer according to the embodiments of this document. The program code may be stored on a carrier that can be read by a computer.
[0233] Fig.17 An example of a content streaming system to which the embodiments disclosed in this document can be applied is illustrated.
[0234] Reference Fig.17 , a content streaming system to which embodiments of this document are applied may basically include an encoding server, a streaming server, a web server, a media storage device, a user device, and a multimedia input device.
[0235] The encoding server compresses the content input from the multimedia input device such as a smart phone, a camera, a camcorder, etc. into digital data to generate a bit stream, and transmits the bit stream to the streaming server. As another example, when the multimedia input device such as a smart phone, a camera, a camcorder, etc. directly generates a bit stream, the encoding server can be omitted.
[0236] A bitstream may be generated by applying the encoding method or the bitstream generating method of the embodiment of this document, and the streaming server may temporarily store the bitstream in the process of transmitting or receiving the bitstream.
[0237] The streaming server sends multimedia data to the user device through the web server based on the user's request, and the web server is used as a medium to inform the user of the service. When the user requests the desired service from the web server, the web server transmits it to the streaming server, and the streaming server sends the multimedia data to the user. In this case, the content streaming system may include a separate control server. In this case, the control server is used to control the command / response between the devices in the content streaming system.
[0238] The streaming server can receive content from a media storage device and / or an encoding server. For example, when receiving content from an encoding server, the content can be received in real time. In this case, in order to provide a stable streaming service, the streaming server can store the bit stream for a predetermined time.
[0239] Examples of user equipment may include mobile phones, smart phones, laptop computers, digital broadcast terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigators, touch-screen PCs, tablet PCs, ultrabooks, wearable devices (e.g., smart watches, smart glasses, head-mounted displays), digital TVs, desktop computers, digital signage, etc.
[0240] The various servers in the content streaming system may operate as distributed servers, in which case data received from the various servers may be distributed.
[0241] The claims in this specification can be combined in various ways. For example, the technical features in the method claims of this specification can be combined to be implemented or performed in a device, and the technical features in the device claims can be combined to be implemented or performed in a method. In addition, the technical features in the method claims and the device claims can be combined to be implemented or performed in a device. In addition, the technical features in the method claims and the device claims can be combined to be implemented or performed in a method.
[0242] (The claims in this specification can be combined in various ways. For example, the technical features in the method claims of this specification can be combined to be implemented or executed in a device, and the technical features in the device claims can be combined to be implemented or executed in a method. Furthermore, the technical features in the method claims and the device claims can be combined to be implemented or executed in a device. Furthermore, the technical features in the method claims and the device claims can be combined to be implemented or executed in a method).
Claims
1. An image decoding method performed by a decoding device, the method comprising: Obtaining most probable mode (MPM) index information and reference line index information from a bitstream; constructing an MPM list including candidate intra prediction modes for intra prediction of a current block; deriving, based on the MPM index information, an intra prediction mode for the current block among the candidate intra prediction modes included in the MPM list; generating prediction samples for the current block based on the intra prediction mode; as well as generating reconstructed samples for the current block based on the predicted samples, Wherein, in constructing the MPM list, based on the fact that the value of the reference line index information indicating the reference line used for intra-frame prediction of the current block is not equal to 0, the DC mode is derived as one of the candidate intra-frame prediction modes and is included in the MPM list.
2. The image decoding method according to claim 1, wherein: Based on the case where the value of the reference line index information is not equal to 0, the planar mode is not used as the intra prediction mode of the current block.
3. The image decoding method according to claim 2, wherein: If the value of the reference line index information is not equal to 0, the plane flag information indicating whether the plane mode is used as the intra-frame prediction mode of the current block is not signaled, and the value of the plane flag information is inferred to be 1, indicating that the plane mode is not used as the intra-frame prediction mode of the current block.
4. The image decoding method according to claim 2, further comprising constructing a temporary MPM list, in, In the construction of the MPM list, if the planar mode is included among the candidate intra prediction modes in the temporary MPM list, the MPM list is reconstructed by removing the planar mode.
5. The image decoding method according to claim 1, wherein: Based on sub-partition mode information indicating whether sub-partition intra prediction is used for the current block, the DC mode is not used as the intra prediction mode of the current block.
6. The image decoding method according to claim 5, comprising: If the value of the reference line index information is equal to 0, the sub-partition mode information is obtained, If the sub-partition mode information indicates that the sub-partition intra prediction is used for the current block, the DC mode is not used as the intra prediction mode of the current block.
7. The image decoding method according to claim 5, further comprising: Build a temporary MPM list, Wherein, in constructing the MPM list, if the DC mode is included in the candidate intra prediction modes in the temporary MPM list, the MPM list is reconstructed by removing the DC mode.
8. The image decoding method according to claim 1, wherein: If the value of the reference line index information is equal to 0, it means that intra-frame prediction is performed using samples of the first closest reference line for the current block, and if the value of the reference line index information is not equal to 0, it means that intra-frame prediction is performed using samples of the second, third or fourth closest reference line for the current block.
9. The image decoding method according to claim 1, wherein: The number of candidate intra prediction modes in the MPM list is five.
10. An image encoding method performed by an encoding device, the method comprising: generating reference line index information indicating a reference line used for intra prediction of a current block; constructing a most probable mode (MPM) list comprising candidate intra prediction modes for intra prediction of the current block; deriving an intra prediction mode for the current block among the candidate intra prediction modes included in the MPM list; generating MPM index information indicating an intra prediction mode for the current block; as well as encoding the image information including the MPM index information and the reference line index information, In the construction of the MPM list, based on the case where the value of the reference line index information is not equal to 0, the DC mode is derived as one of the candidate intra prediction modes and is included in the MPM list.
11. The image encoding method according to claim 10, wherein: Based on the case where the value of the reference line index information is not equal to 0, the planar mode is not used as the intra prediction mode of the current block.
12. The image encoding method according to claim 11, wherein: If the value of the reference line index information is not equal to 0, the plane flag information indicating whether the plane mode is used as the intra-frame prediction mode of the current block is not signaled, and the value of the plane flag information is inferred to be 1, indicating that the plane mode is not used as the intra-frame prediction mode of the current block.
13. The image encoding method according to claim 11, further comprising: Build a temporary MPM list, Wherein, in constructing the MPM list, if the planar mode is included in the candidate intra prediction modes in the temporary MPM list, the MPM list is reconstructed by removing the planar mode.
14. The image encoding method according to claim 10, wherein: Based on sub-partition mode information indicating whether sub-partition intra prediction is used for the current block, the DC mode is not used as the intra prediction mode of the current block.
15. The image encoding method according to claim 14, comprising: If the value of the reference line index information is equal to 0, the sub-partition mode information is generated, If the sub-partition mode information indicates that the sub-partition intra prediction is used for the current block, the DC mode is not used as the intra prediction mode of the current block.
16. The image encoding method according to claim 14, further comprising: Build a temporary MPM list, Wherein, in constructing the MPM list, if the DC mode is included in the candidate intra prediction modes in the temporary MPM list, the MPM list is reconstructed by removing the DC mode.
17. A computer-readable digital storage medium storing encoded image information, the encoded image information causing a decoding device to perform an image decoding method, the image decoding method comprising: Obtaining most probable mode (MPM) index information and reference line index information from a bitstream; constructing an MPM list including candidate intra prediction modes for intra prediction of a current block; deriving, based on the MPM index information, an intra prediction mode for the current block among the candidate intra prediction modes included in the MPM list; generating prediction samples for the current block based on the intra prediction mode; as well as generating reconstructed samples for the current block based on the predicted samples, Wherein, in constructing the MPM list, based on the fact that the value of the reference line index information indicating the reference line used for intra-frame prediction of the current block is not equal to 0, the DC mode is derived as one of the candidate intra-frame prediction modes and is included in the MPM list.
