Image Encoding / Decoding Method Based on Multiple Reference Lines, Method for Transmitting a Bitstream, and Recording Medium Storing the Bitstream

Abstract

An image encoding / decoding method, a bitstream transmission method, and a computer-readable recording medium for storing a bitstream are provided. An image decoding method according to the present disclosure is an image decoding method performed by an image decoding apparatus, the method including: generating a predicted value by predicting a first reference line for a current block based on each of a plurality of reference lines for the current block; calculating an error between each predicted value and a restored value of the first reference line; and inducing at least one second reference line used for intra prediction of the current block among the plurality of reference lines based on the error. The method may be an image decoding method.

Description

Technical Field

[0001] The present invention relates to an image encoding / decoding method, a method for transmitting a bitstream, and a recording medium storing a bitstream, and relates to prediction based on multiple reference lines.

Background Art

[0002] Recently, the demand for high-resolution and high-quality images, such as HD (High Definition) images and UHD (Ultra High Definition) images, has been increasing in various fields. As the image data becomes higher in resolution and quality, the amount of information or bits to be transmitted relatively increases compared with conventional image data. The increase in the amount of information or bits to be transmitted brings about an increase in transmission cost and storage cost.

[0003] Therefore, there is a need for a highly efficient image compression technique for effectively transmitting, storing, and reproducing information of high-resolution and high-quality images.

Summary of the Invention

Problems to be Solved by the Invention

[0004] An object of the present disclosure is to provide an image encoding / decoding method and apparatus with improved encoding / decoding efficiency.

[0005] Another object of the present disclosure is to propose a multiple reference line mode based on a template.

[0006] Another object of the present disclosure is to propose a method for deriving a multiple reference line index based on a template.

[0007] Another object of the present disclosure is to propose a method for reordering a multiple reference line index candidate list based on a template.

[0008] Furthermore, an object of the present disclosure is to propose a method of combining a vertical multiple reference line index and a horizontal multiple reference line index.

[0009] Furthermore, an object of the present disclosure is to provide a non-transitory computer-readable recording medium that stores a bitstream generated by an image encoding method according to the present disclosure.

[0010] Furthermore, an object of the present disclosure is to provide a non-transitory computer-readable recording medium that stores a bitstream received by an image decoding apparatus according to the present disclosure, decoded, and used for restoring an image.

[0011] Furthermore, an object of the present disclosure is to provide a method of transmitting a bitstream generated by an image encoding method according to the present disclosure.

[0012] The technical problems to be solved in the present disclosure are not limited to the above-described technical problems, and other technical problems not described above will be clearly understood by those having ordinary knowledge in the technical field to which the present disclosure pertains from the following description.

Means for Solving the Problems

[0013] An image decoding method according to an aspect of the present disclosure is an image decoding method performed by an image decoding apparatus, the method including: generating a predicted value by predicting a first reference line for a current block based on each of a plurality of reference lines for the current block; calculating an error between each predicted value and a restored value of the first reference line; and inducing at least one second reference line used for intra prediction of the current block among the plurality of reference lines based on the error.

[0014] An image encoding method according to another aspect of the present disclosure is an image encoding method performed by an image encoding apparatus, the method including: generating a predicted value by predicting a first reference line for a current block based on each of a plurality of reference lines for the current block; calculating an error between each predicted value and a restored value of the first reference line; and inducing at least one second reference line used for intra prediction of the current block from among the plurality of reference lines based on the error.

[0015] A computer-readable recording medium according to another aspect of the present disclosure can store a bitstream generated by the image encoding method or apparatus of the present disclosure.

[0016] A transmission method according to another aspect of the present disclosure can transmit a bitstream generated by the image encoding method or apparatus of the present disclosure.

[0017] The features briefly summarized and described above with respect to the present disclosure are merely exemplary aspects of the detailed description of the present disclosure to be described later, and do not limit the scope of the present disclosure.

Advantages of the Invention

[0018] According to the present disclosure, it is possible to provide an image encoding / decoding method and apparatus with improved encoding / decoding efficiency.

[0019] Also, according to the present disclosure, by inducing without signaling a multi-reference line index, prediction performance can be improved and coding efficiency can be improved.

[0020] Also, according to the present disclosure, by reordering multi-reference line index candidates, bit efficiency for index signaling can be improved.

[0021] Also, according to the present disclosure, a non-transitory computer-readable recording medium for storing a bitstream generated by the image encoding method according to the present disclosure can be provided.

[0022] Also, according to the present disclosure, a non-transitory computer-readable recording medium for storing a bitstream that is received by the image decoding apparatus according to the present disclosure, decoded, and used for restoring an image can be provided.

[0023] According to the present disclosure, a method for transmitting a bitstream generated by the image encoding method according to the present disclosure can be provided.

[0024] The effects obtained in the present disclosure are not limited to the above-described effects, and other effects not described above will be clearly understood by those of ordinary skill in the technical field to which the present disclosure pertains from the following description.

Brief Description of the Drawings

[0025]

Figure 1

[0026]

Figure 2

[0027]

Figure 3

[0028]

Figure 4

[0029]

Figure 5

[0030]

Figure 6

[0031]

Figure 7

[0032]

Figure 8

[0033]

Figures 9 - 10

[0034]

Figure 11

[0035]

Figures 12 - 13

[0036]

Figure 14

[0037]

Figure 15

[0038]

Figure 16

[0039]

Figure 17

[0040]

Figure 18

DETAILED DESCRIPTION OF THE INVENTION

[0041] Hereinafter, with reference to the accompanying drawings, embodiments of the present disclosure will be described in detail so that those skilled in the art to which the present disclosure pertains can easily implement them. However, the present disclosure can be realized in various different forms and is not limited to the embodiments described herein.

[0042] In describing the embodiments of the present disclosure, when it is determined that a detailed description of a known configuration or function may obscure the gist of the present disclosure, the detailed description thereof will be omitted. In the drawings, parts not related to the description of the present disclosure are omitted, and the same reference numerals are given to the same parts.

[0043] In the present disclosure, when a certain component is "connected", "coupled", or "connected" to another component, this can include not only a direct connection relationship but also an indirect connection relationship in which another component exists between them. Also, when a certain component "includes" or "has" another component, this means that, unless otherwise stated to the contrary, it does not exclude other components but can further include other components.

[0044] In the present disclosure, terms such as "first" and "second" are used only for the purpose of distinguishing one component from another and do not limit the order or importance between components, etc., unless otherwise specifically mentioned. Therefore, within the scope of the present disclosure, the first component of one embodiment may be referred to as the second component in another embodiment, and similarly, the second component of one embodiment may be referred to as the first component in another embodiment.

[0045] In the present disclosure, the components that are distinguished from each other are for clearly explaining their respective features, and do not necessarily mean that the components are separated. That is, a plurality of components may be integrated and configured as one hardware or software unit, or one component may be distributed and configured as a plurality of hardware or software units. Therefore, even without separate mention, such integrated or distributed embodiments are also included in the scope of the present disclosure.

[0046] In the present disclosure, the components described in various embodiments do not necessarily mean essential components, and some may be optional components. Therefore, embodiments constituted by a subset of the components described in one embodiment are also included in the scope of the present disclosure. Further, embodiments that include additional components in addition to the components described in various embodiments are also included in the scope of the present disclosure.

[0047] The present disclosure relates to image encoding and decoding, and the terms used in the present disclosure can have the ordinary meanings in the technical field to which the present disclosure belongs, unless newly defined in the present disclosure.

[0048] In the present disclosure, "picture" generally means a unit indicating any one image in a specific time period, and a slice / tile is an encoding unit constituting a part of a picture, and one picture can be constituted by one or more slices / tiles. Further, a slice / tile can include one or more CTUs (coding tree units).

[0049] In the present disclosure, "pixel" or "pel" can mean the smallest unit constituting one picture (or image). Further, the term "sample" can be used as a term corresponding to a pixel. A sample can generally indicate a pixel or a pixel value, and can also indicate only the pixel / pixel value of the luma component, or can also indicate only the pixel / pixel value of the chroma component.

[0050] In the present disclosure, "unit" can indicate the basic unit of image processing. A unit can include at least one of a specific region of a picture and information related to the region. A unit can be used interchangeably with terms such as "sample array", "block", or "area" as the case may be. In general, an M×N block can include a set (or array) of samples (or sample arrays) or transform coefficients consisting of M columns and N rows.

[0051] In the present disclosure, "current block" can mean any one of "current coding block", "current coding unit", "block to be coded", "block to be decoded", or "block to be processed". When prediction is performed, "current block" can mean "current prediction block" or "block to be predicted". When transformation (inverse transformation) / quantization (inverse quantization) is performed, "current block" can mean "current transformation block" or "block to be transformed". When filtering is performed, "current block" can mean "block to be filtered".

[0052] Also, in the present disclosure, unless explicitly stated as a chroma block, "current block" can mean a block that includes both a luma component block and a chroma component block or "the luma block of the current block". The luma component block of the current block can be explicitly expressed including an explicit description of the luma component block such as "luma block" or "current luma block". Also, the chroma component block of the current block can be explicitly expressed including an explicit description of the chroma component block such as "chroma block" or "current chroma block".

[0053] In the present disclosure, " / " and "," can be interpreted as "and / or". For example, "A / B" and "A, B" can be interpreted as "A and / or B". Also, "A / B / C" and "A, B, C" can mean "at least one of A, B, and / or C".

[0054] In the present disclosure, "or" can be interpreted as "and / or". For example, "A or B" can mean 1) only "A", 2) only "B", or 3) "A and B". Alternatively, in the present disclosure, "or" can mean "additionally or alternatively".

[0055] Overview of Video Coding System

[0056] FIG. 1 is a diagram schematically showing a video coding system to which an embodiment according to the present disclosure can be applied.

[0057] A video coding system according to an embodiment can include an encoding device 10 and a decoding device 20. The encoding device 10 can transmit encoded video and / or image information or data in a file or streaming format to the decoding device 20 via a digital storage medium or a network.

[0058] An encoding device 10 according to an embodiment can include a video source generation unit 11, an encoding unit 12, and a transmission unit 13. A decoding device 20 according to an embodiment can include a reception unit 21, a decoding unit 22, and a rendering unit 23. The encoding unit 12 can be called a video / image encoding unit, and the decoding unit 22 can be called a video / image decoding unit. The transmission unit 13 can be included in the encoding unit 12. The reception unit 21 can be included in the decoding unit 22. The rendering unit 23 can also include a display unit, and the display unit can be configured as a separate device or an external component.

[0059] The video source generation unit 11 can acquire video / images through processes such as video / image capture, synthesis, or generation. The video source generation unit 11 can include a video / image capture device and / or a video / image generation device. The video / image capture device can include, for example, one or more cameras, a video / image archive including previously captured video / images, and the like. The video / image generation device can include, for example, a computer, a tablet, and a smartphone, etc., and can (electronically) generate a video / image. For example, a virtual video / image can be generated via a computer or the like. In this case, the video / image capture process can be replaced by a process in which related data is generated.

[0060] The encoding unit 12 can encode the input video / image. The encoding unit 12 can perform a series of procedures such as prediction, transformation, quantization, etc. for compression and encoding efficiency. The encoding unit 12 can output the encoded data (encoded video / image information) in the form of a bitstream.

[0061] The transmission unit 13 can acquire the encoded video / image information or data output in bitstream format, and transmit this to the receiving unit 21 of the decoding device 20 or other external objects via a digital storage medium or network in file or streaming format. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmission unit 13 can include elements for generating media files via a predetermined file format, and can include elements for transmission via a broadcast / communication network. The transmission unit 13 can be provided as a transmission device separate from the encoding device 12. In this case, the transmission device can include at least one processor that acquires the encoded video / image information or data output in bitstream format, and a transmission unit that transmits this in file or streaming format. The receiving unit 21 can extract / receive the bitstream from the storage medium or network and transmit it to the decoding unit 22.

[0062] The decoding unit 22 can decode a video / image by performing a series of procedures such as inverse quantization, inverse transformation, and prediction corresponding to the operation of the encoding unit 12.

[0063] The rendering unit 23 can render the decoded video / image. The rendered video / image can be displayed via the display unit.

[0064] Overview of Image Encoding Device

[0065] FIG. 2 is a diagram schematically showing an image encoding device to which the embodiment according to the present disclosure can be applied.

[0066] As shown in FIG. 2, the image encoding apparatus 100 can include an image dividing unit 110, a subtraction unit 115, a conversion unit 120, a quantization unit 130, an inverse quantization unit 140, an inverse conversion unit 150, an addition unit 155, a filtering unit 160, a memory 170, an inter prediction unit 180, an intra prediction unit 185, and an entropy encoding unit 190. The inter prediction unit 180 and the intra prediction unit 185 can be collectively referred to as a “prediction unit”. The conversion unit 120, the quantization unit 130, the inverse quantization unit 140, and the inverse conversion unit 150 can be included in a residual processing unit. The residual processing unit can further include the subtraction unit 115.

[0067] All or at least a part of the plurality of components constituting the image encoding apparatus 100 can be realized by one hardware component (for example, an encoder or a processor) according to an embodiment. Further, the memory 170 can include a DPB (decoded picture buffer) and can be realized by a digital storage medium.

[0068] The image segmentation unit 110 can divide an input image (or picture, frame) input to the image encoding apparatus 100 into one or more processing units. As an example, the processing unit can be called a coding unit (CU). The coding unit can be obtained by recursively dividing a coding tree unit (CTU) or a largest coding unit (LCU) in a QT / BT / TT (Quad-tree / binary-tree / ternary-tree) structure. For example, one coding unit can be divided into a plurality of coding units with a deeper depth based on a quadtree structure, a binary tree structure, and / or a ternary tree structure. For the division of the coding unit, the quadtree structure can be applied first, and the binary tree structure and / or the ternary tree structure can be applied later. Based on the final coding unit that cannot be further divided, the coding procedure according to the present disclosure can be performed. The largest coding unit can be used as the final coding unit, and the coding units with a deeper depth obtained by dividing the largest coding unit can also be used as the final coding unit. Here, the coding procedure can include procedures such as prediction, transformation, and / or restoration described later. As another example, the processing unit of the coding procedure can be a prediction unit (PU: Prediction Unit) or a transformation unit (TU: Transform Unit). The prediction unit and the transformation unit can be divided or partitioned from the final coding unit, respectively. The prediction unit can be a unit of sample prediction, and the transformation unit can be a unit for deriving transformation coefficients and / or a unit for deriving a residual signal from the transformation coefficients.

[0069] The prediction unit (inter prediction unit 180 or intra prediction unit 185) can perform prediction on a processing target block (current block) and generate a predicted block that includes prediction samples for the current block. The prediction unit can determine whether intra prediction is applied in units of the current block or CU, or whether inter prediction is applied. The prediction unit can generate various information related to the prediction of the current block and transmit it to the entropy encoding unit 190. The information related to the prediction can be encoded by the entropy encoding unit 190 and output in the form of a bitstream.

[0070] The intra prediction unit 185 can predict the current block by referring to samples within the current picture. The samples to be referred to can be located in the neighborhood of the current block or at a distance according to the intra prediction mode and / or intra prediction technique. The intra prediction mode can include a plurality of non-directional modes and a plurality of directional modes. The non-directional modes can include, for example, the DC mode and the Planar mode. The directional modes can include, for example, 33 directional prediction modes or 65 directional prediction modes according to the degree of fineness of the prediction direction. However, this is only an example, and more or fewer directional prediction modes can be used based on the settings. The intra prediction unit 185 can also determine the prediction mode to be applied to the current block using the prediction mode applied to the neighboring blocks.

[0071] The inter prediction unit 180 can derive a predicted block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. At this time, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information can be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the neighboring blocks and the current block. The motion information can include a motion vector and a reference picture index. The motion information can further include inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter prediction, the neighboring blocks can include spatial neighboring blocks existing within the current picture and temporal neighboring blocks existing in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different from each other. The temporal neighboring block can be called by names such as a collocated reference block and a collocated CU (colCU). The reference picture including the temporal neighboring block can be called a collocated picture (colPic). For example, the inter prediction unit 180 can construct a motion information candidate list based on the neighboring blocks, and generate information indicating which candidate is used to derive the motion vector and / or the reference picture index of the current block. Inter prediction can be performed based on various prediction modes. For example, in the case of the skip mode and the merge mode, the inter prediction unit 180 can use the motion information of the neighboring blocks as the motion information of the current block. In the case of the skip mode, unlike the merge mode, the residual signal cannot be transmitted.In the case of the motion information prediction (MVP) mode, the motion vectors of neighboring blocks are used as motion vector predictors, and the motion vector difference and the indicator for the motion vector predictor are encoded to signal the motion vector of the current block. The motion vector difference can mean the difference between the motion vector of the current block and the motion vector predictor.

[0072] The prediction unit can generate a prediction signal based on various prediction methods and / or prediction techniques described below. For example, the prediction unit can apply not only intra prediction or inter prediction for predicting the current block, but also apply intra prediction and inter prediction simultaneously. A prediction method that applies intra prediction and inter prediction simultaneously for predicting the current block can be called CIIP (combined inter and intra prediction). In addition, the prediction unit can also perform intra block copy (IBC) for predicting the current block. Intra block copy can be used for content image / video coding such as games, for example, like SCC (screen content coding). IBC is a method of predicting the current block using a restored reference block within the current picture at a position a predetermined distance away from the current block. When IBC is applied, the position of the reference block within the current picture can be encoded as a vector (block vector) corresponding to the predetermined distance. IBC basically performs prediction within the current picture, but can be performed in the same way as inter prediction in terms of deriving a reference block within the current picture. That is, IBC can use at least one of the inter prediction techniques described in the present disclosure.

[0073] The prediction signal generated by the prediction unit can be used to generate a restored signal or can be used to generate a residual signal. The subtraction unit 115 can subtract the prediction signal (predicted block, predicted sample array) output from the prediction unit from the input image signal (original block, original sample array) to generate a residual signal (residual signal, residual block, residual sample array). The generated residual signal can be transmitted to the conversion unit 120.

[0074] The conversion unit 120 can apply a conversion technique to the residual signal to generate transform coefficients. For example, the conversion technique can include at least one of DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), KLT (Karhunen-Loeve Transform), GBT (Graph-Based Transform), or CNT (Conditionally Non-linear Transform). Here, GBT means the conversion obtained from this graph when the relationship information between pixels is represented by a graph. CNT means the conversion obtained based on generating a prediction signal using all previously reconstructed pixels. The conversion process can also be applied to a pixel block having the same size of a square, or can be applied to a block of a variable size that is not square.

[0075] The quantization unit 130 can quantize the transform coefficients and transmit them to the entropy encoding unit 190. The entropy encoding unit 190 can encode the quantized signal (information regarding the quantized transform coefficients) and output it in the form of a bitstream. The information regarding the quantized transform coefficients can be called residual information. The quantization unit 130 can reorder the block-form quantized transform coefficients into a one-dimensional vector form based on the coefficient scan order, and can also generate the information regarding the quantized transform coefficients based on the quantized transform coefficients in the one-dimensional vector form.

[0076] The entropy encoding unit 190 can perform various encoding methods such as, for example, exponential Golomb, CAVLC (context-adaptive variable length coding), CABAC (context-adaptive binary arithmetic coding). The entropy encoding unit 190 can also encode, together or separately, information necessary for video / image restoration (such as the values of syntax elements) in addition to the quantized transform coefficients. The encoded information (such as the encoded video / image information) can be transmitted or stored in the form of a bitstream in units of NAL (network abstraction layer) units. The video / image information can further include information regarding various parameter sets such as an adaptive parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). Further, the video / image information can further include general constraint information. The signaling information, the transmitted information, and / or the syntax elements referred to in the present disclosure can be encoded through the above-described encoding procedure and included in the bitstream.

[0077] The bitstream can be transmitted via a network or stored in a digital storage medium. Here, the network can include a broadcast network and / or a communication network, etc., and the digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmission unit (not shown) for transmitting and / or a storage unit (not shown) for storing the signal output from the entropy encoding unit 190 can be provided as internal / external elements of the image encoding device 100, or the transmission unit can also be provided as a component of the entropy encoding unit 190.

[0078] The quantized transform coefficients output from the quantization unit 130 can be used to generate a residual signal. For example, by applying inverse quantization and inverse transformation to the quantized transform coefficients via the inverse quantization unit 140 and the inverse transformation unit 150, a residual signal (residual block or residual sample) can be restored.

[0079] The addition unit 155 can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the restored residual signal to the prediction signal output from the inter prediction unit 180 or the intra prediction unit 185. When there is no residual for the block to be processed, as in the case where the skip mode is applied, the predicted block can be used as the reconstructed block. The addition unit 155 can be called a restoration unit or a reconstructed block generation unit. The generated reconstructed signal can be used for intra prediction of the next block to be processed within the current picture and can also be used for inter prediction of the next picture after passing through filtering as described later.

[0080] The filtering unit 160 can apply filtering to the restored signal to improve the subjective / objective image quality. For example, the filtering unit 160 can apply various filtering methods to the restored picture to generate a modified restored picture, and can store the modified restored picture in the memory 170, specifically in the DPB of the memory 170. The various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, and the like. The filtering unit 160 can generate various information related to filtering as described later in the description of each filtering method and transmit it to the entropy encoding unit 190. The information related to filtering can be encoded by the entropy encoding unit 190 and output in the form of a bitstream.

[0081] The modified restored picture transmitted to the memory 170 can be used as a reference picture by the inter prediction unit 180. When inter prediction is applied through this, the image encoding apparatus 100 can avoid prediction mismatches between the image encoding apparatus 100 and the image decoding apparatus, and can also improve the encoding efficiency.

[0082] The DPB in the memory 170 can store the modified restored picture for use as a reference picture by the inter prediction unit 180. The memory 170 can store the motion information of the blocks in which the motion information in the current picture has been derived (or encoded) and / or the motion information of the blocks in the already restored picture. The stored motion information can be transmitted to the inter prediction unit 180 for utilization as the motion information of the spatial neighboring blocks or the motion information of the temporal neighboring blocks. The memory 170 can store the restored samples of the restored blocks in the current picture and transmit them to the intra prediction unit 185.

[0083] Overview of Image Decoding Device

[0084] FIG. 3 is a diagram schematically showing an image decoding apparatus to which an embodiment according to the present disclosure can be applied.

[0085] As shown in FIG. 3, the image decoding apparatus 200 can be configured to include an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an addition unit 235, a filtering unit 240, a memory 250, an inter prediction unit 260, and an intra prediction unit 265. The inter prediction unit 260 and the intra prediction unit 265 can be collectively referred to as a "prediction unit". The inverse quantization unit 220 and the inverse transform unit 230 can be included in a residual processing unit.

[0086] All or at least a part of the plurality of components constituting the image decoding apparatus 200 can be realized by one hardware component (for example, a decoder or a processor) according to an embodiment. Further, the memory 170 can include a DPB and can be realized by a digital storage medium.

[0087] The image decoding apparatus 200 that has received a bitstream including video / image information can execute a process corresponding to the process performed by the image encoding apparatus 100 in FIG. 2 to restore an image. For example, the image decoding apparatus 200 can perform decoding using the processing unit applied in the image encoding apparatus. Therefore, the decoding processing unit can be, for example, a coding unit. The coding unit can be obtained by dividing a coding tree unit or a maximum coding unit. Then, the restored image signal decoded and output via the image decoding apparatus 200 can be reproduced via a reproducing apparatus (not shown).

[0088] The image decoding device 200 can receive the signal output from the image encoding device in FIG. 2 in the form of a bit stream. The received signal can be decoded via the entropy decoding unit 210. For example, the entropy decoding unit 210 can parse the bit stream to derive information (e.g., video / image information) necessary for image restoration (or picture restoration). The video / image information can further include information regarding various parameter sets such as an Adaptive Parameter Set (APS), a Picture Parameter Set (PPS), a Sequence Parameter Set (SPS), or a Video Parameter Set (VPS). Also, the video / image information can further include general constraint information. The image decoding device can further use the information regarding the parameter set and / or the general constraint information to decode the image. The signaling information, the received information, and / or the syntax elements referred to in the present disclosure can be obtained from the bit stream by being decoded via the decoding procedure. For example, the entropy decoding unit 210 can decode the information in the bit stream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and output the value of the syntax element necessary for image restoration and the quantized value of the transform coefficient regarding the residual. More specifically, the CABAC entropy decoding method receives the bin corresponding to each syntax element from the bit stream, determines a context model using the syntax element information to be decoded, the decoding information of the surrounding blocks and the block to be decoded, or the information of the symbol / bin decoded in the previous step, predicts the occurrence probability of the bin based on the determined context model, and performs arithmetic decoding of the bin to generate a symbol corresponding to the value of each syntax element. At this time, the CABAC entropy decoding method can update the context model using the information of the decoded symbol / bin for the context model of the next symbol / bin after determining the context model.Of the information decoded by the entropy decoding unit 210, the information related to prediction is provided to the prediction units (inter prediction unit 260 and intra prediction unit 265), and the residual values entropy decoded by the entropy decoding unit 210, that is, the quantized transform coefficients and related parameter information, can be input to the inverse quantization unit 220. Also, of the information decoded by the entropy decoding unit 210, the information related to filtering can be provided to the filtering unit 240. On the other hand, a receiving unit (not shown) that receives a signal output from the image encoding device can be further provided as an internal / external element of the image decoding device 200, or the receiving unit can be provided as a component of the entropy decoding unit 210.

[0089] On the other hand, the image decoding device according to the present disclosure can be called a video / image / picture decoding device. The image decoding device can also include an information decoder (video / image / picture information decoder) and / or a sample decoder (video / image / picture sample decoder). The information decoder can include the entropy decoding unit 210, and the sample decoder can include at least one of the inverse quantization unit 220, the inverse transform unit 230, the addition unit 235, the filtering unit 240, the memory 250, the inter prediction unit 260, and the intra prediction unit 265.

[0090] In the inverse quantization unit 220, the quantized transform coefficients can be inverse quantized to output transform coefficients. The inverse quantization unit can reorder the quantized transform coefficients in a two-dimensional block format. In this case, the reordering can be performed based on the coefficient scan order performed by the image encoding device. The inverse quantization unit can perform inverse quantization on the quantized transform coefficients using a quantization parameter (for example, quantization step size information) to obtain transform coefficients.

[0091] In the inverse conversion unit 230, the conversion coefficients can be inversely converted to obtain a residual signal (residual block, residual sample array).

[0092] The prediction unit can perform prediction on the current block and generate a predicted block including predicted samples for the current block. The prediction unit can determine whether intra prediction or inter prediction is applied to the current block based on the information regarding the prediction output from the entropy decoding unit 210, and can determine a specific intra / inter prediction mode (prediction technique).

[0093] The prediction unit can generate a prediction signal based on various prediction methods (techniques) described later, which is the same as that described in the explanation of the prediction unit of the image encoding device 100.

[0094] The intra prediction unit 265 can predict the current block by referring to samples within the current picture. The explanation of the intra prediction unit 185 can be similarly applied to the intra prediction unit 265.

[0095] The inter prediction unit 260 can derive a predicted block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. At this time, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information can be predicted in units of blocks, sub-blocks, or samples based on the correlation of motion information between neighboring blocks and the current block. The motion information can include a motion vector and a reference picture index. The motion information can further include inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter prediction, neighboring blocks can include spatial neighboring blocks existing within the current picture and temporal neighboring blocks existing in the reference picture. For example, the inter prediction unit 260 can configure a motion information candidate list based on neighboring blocks, and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. Inter prediction can be performed based on various prediction modes (techniques), and the information regarding the prediction can include information indicating the mode (technique) of inter prediction for the current block.

[0096] The addition unit 235 can generate a restored signal (restored picture, restored block, restored sample array) by adding the obtained residual signal to a prediction signal (predicted block, predicted sample array) output from a prediction unit (including the inter prediction unit 260 and / or the intra prediction unit 265). When there is no residual for the processing target block as in the case where the skip mode is applied, the predicted block can be used as the restored block. The description of the addition unit 155 can be similarly applied to the addition unit 235. The addition unit 235 may also be referred to as a restoration unit or a restored block generation unit. The generated restored signal can be used for intra prediction of the next processing target block within the current picture, and can also be used for inter prediction of the next picture through filtering as described later.

[0097] The filtering unit 240 can apply filtering to the restored signal to improve the subjective / objective image quality. For example, the filtering unit 240 can apply various filtering methods to the restored picture to generate a modified restored picture, and can store the modified restored picture in the memory 250, specifically, in the DPB of the memory 250. The various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, and the like.

[0098] The (modified) restored picture stored in the DPB of the memory 250 can be used as a reference picture in the inter prediction unit 260. The memory 250 can store the motion information of the block in which the motion information in the current picture has been derived (or decoded) and / or the motion information of the block in the already restored picture. The stored motion information can be transmitted to the inter prediction unit 260 for utilization as the motion information of the spatial neighboring blocks or the motion information of the temporal neighboring blocks. The memory 250 can store the restored samples of the restored blocks in the current picture and can transmit them to the intra prediction unit 265.

[0099] In this specification, the embodiments described in the filtering unit 160, the inter prediction unit 180, and the intra prediction unit 185 of the image encoding apparatus 100 can be similarly or correspondingly applied to the filtering unit 240, the inter prediction unit 260, and the intra prediction unit 265 of the image decoding apparatus 200, respectively.

[0100] Intra Prediction

[0101] Intra prediction can indicate a prediction that generates prediction samples for a current block based on reference samples within the picture to which the current block belongs (hereinafter, the current picture). When intra prediction is applied to the current block, peripheral reference samples to be used for intra prediction of the current block can be derived. The peripheral reference samples of the current block can include samples adjacent to the left boundary of the current block of size nW×nH and a total of 2×nH samples adjacent to the bottom-left, samples adjacent to the top boundary of the current block and a total of 2×nW samples adjacent to the top-right, and one sample adjacent to the top-left of the current block. Alternatively, the peripheral reference samples of the current block can include a plurality of rows of upper peripheral samples and a plurality of columns of left peripheral samples. Also, the peripheral reference samples of the current block can include a total of nH samples adjacent to the right boundary of the current block of size nW×nH, a total of nW samples adjacent to the bottom boundary of the current block, and one sample adjacent to the bottom-right of the current block.

[0102] However, some of the peripheral reference samples of the current block may not have been decoded yet or may not be available. In this case, the image decoding device 200 can configure the peripheral reference samples to be used for prediction by replacing the unavailable samples with available samples (substitution). Alternatively, the peripheral reference samples to be used for prediction can be configured through interpolation of available samples.

[0103] When a peripheral reference sample is derived, (i) a predicted sample can be derived based on the average or interpolation of neighboring reference samples of the current block, and (ii) the predicted sample can also be derived based on reference samples existing in a specific (predicted) direction with respect to the predicted sample among the peripheral reference samples of the current block. In the case of (i), it may be called a non-directional mode or a non-angle mode, and in the case of (ii), it may be called a directional mode or an angular mode. Also, among the peripheral reference samples, based on the predicted sample of the current block, through interpolation between the second peripheral sample and the first peripheral sample located in the direction opposite to the prediction direction of the intra prediction mode of the current block, the predicted sample can also be generated. The above-mentioned case may be called linear interpolation intra prediction (LIP). Also, a chroma predicted sample can be generated based on luma samples using a linear model. In this case, it may be called the LM mode. Also, a temporary predicted sample of the current block is derived based on filtered peripheral reference samples, and a predicted sample of the current block is derived by performing a weighted sum of at least one reference sample derived according to the intra prediction mode among the existing peripheral reference samples, that is, the non-filtered peripheral reference samples and the temporary predicted sample. The above-mentioned case may be called PDPC (Position dependent intra prediction). Also, from among the peripheral multiple reference sample lines of the current block, the reference sample line with the highest prediction accuracy is selected, and a predicted sample is derived using the reference sample located in the prediction direction from this line. At this time, intra prediction coding can be performed by indicating (signaling) the used reference sample line to the image decoding device 200.In the above case, it may be called multi-reference line (MRL) intra prediction or MRL-based intra prediction. Also, the current block is divided into vertical or horizontal sub-partitions, and intra prediction is performed based on the same intra prediction mode, but the peripheral reference samples can be derived and used in units of the sub-partitions. That is, in this case, the intra prediction mode for the current block is applied identically to the sub-partitions, but by deriving and using the peripheral reference samples in units of the sub-partitions, the intra prediction performance can be enhanced as appropriate. Such a prediction method may be called intra sub-partitions (ISP) or ISP-based intra prediction. The intra prediction method described above may be called an intra prediction type when classified from the intra prediction mode in Table of Contents 1.2. The intra prediction type may be called by various terms such as an intra prediction technique or an additional intra prediction mode. For example, the intra prediction type (or an additional intra prediction mode, etc.) may include at least one of LIP, PDPC, MRL, and ISP described above. A general intra prediction method excluding specific intra prediction types such as LIP, PDPC, MRL, and ISP may be called a normal intra prediction type. The normal intra prediction type can be generally applied when the specific intra prediction types as described above are not applicable, and prediction can be performed based on the intra prediction mode described above. On the other hand, post-processing filtering for the derived prediction samples may be performed as necessary.

[0104] Specifically, the intra prediction procedure may include an intra prediction mode / type determination step, a peripheral reference sample derivation step, and an intra prediction mode / type-based prediction sample derivation step. Further, a post-filtering step for the derived prediction samples may be performed as necessary.

[0105] On the other hand, in addition to the above-described intra prediction type, ALWIP (affine linear weighted intra prediction) can be used. The ALWIP is also called LWIP (linear weighted intra prediction) or MIP (matrix weighted intra prediction or matrix based intra prediction). When the MIP is applied to the current block, i) using the peripheral reference samples for which an averaging procedure has been performed, ii) performing a matrix-vector-multiplication procedure, and iii) further performing a horizontal / vertical interpolation procedure as necessary to derive the prediction samples for the current block. The intra prediction mode used for the MIP can be configured to be different from the intra prediction modes used in the above-described LIP, PDPC, MRL, ISP intra prediction, and normal intra prediction. The intra prediction mode for the MIP is also called the MIP intra prediction mode, the MIP prediction mode, or the MIP mode. For example, depending on the intra prediction mode for the MIP, the matrix and offset used in the matrix-vector-multiplication can be set to be different. Here, the matrix can also be called the (MIP) weight matrix, and the offset may also be called the (MIP) offset vector or the (MIP) bias vector. A specific MIP method will be described later.

[0106] The block restoration procedure based on intra prediction and the intra prediction unit 185 in the image encoding apparatus 100 can be schematically included with reference to FIGS. 4 and 5 by way of example.

[0107] S400 can be performed by the intra prediction unit 185 of the image encoding apparatus 100, and S410 can be performed by the residual processing unit of the image encoding apparatus 100. Specifically, S410 can be performed by the subtraction unit 115 of the image encoding apparatus 100. In S420, the prediction information can be derived by the intra prediction unit 185 and encoded by the entropy encoding unit 190. In S420, the residual information can be derived by the residual processing unit and encoded by the entropy encoding unit 190. The residual information is information regarding the residual sample. The residual information can include information regarding the quantized transform coefficients for the residual sample. As described above, the residual sample can be derived into transform coefficients via the conversion unit 120 of the image encoding apparatus 100, and the transform coefficients can be derived into quantized transform coefficients via the quantization unit 130. The information regarding the quantized transform coefficients can be encoded by the entropy encoding unit 190 via the residual coding procedure.

[0108] The image encoding device 100 performs intra prediction on the current block (S400). The image encoding device 100 can derive an intra prediction mode / type for the current block and derive peripheral reference samples of the current block, and generate prediction samples within the current block based on the intra prediction mode / type and the peripheral reference samples. Here, the determination of the intra prediction mode / type, the derivation of the peripheral reference samples, and the generation procedure of the prediction samples may be performed simultaneously, or any one of the procedures may be performed prior to the other procedures. For example, the intra prediction unit 185 of the image encoding device 100 can include an intra prediction mode / type determination unit 186, a reference sample derivation unit 187, and a prediction sample derivation unit 188. The intra prediction mode / type determination unit 186 determines the intra prediction mode / type for the current block, the reference sample derivation unit 187 derives the peripheral reference samples of the current block, and the prediction sample derivation unit 188 can derive the prediction samples of the current block. On the other hand, although not shown in the figure, when a prediction sample filtering procedure described later is performed, the intra prediction unit 185 can further include a prediction sample filter unit (not shown). The image encoding device 100 can determine the mode / type applied to the current block among a plurality of intra prediction modes / types. The image encoding device 100 can compare the RD cost for the intra prediction mode / type and determine the optimal intra prediction mode / type for the current block.

[0109] On the other hand, the image encoding device 100 can also perform a prediction sample filtering procedure. The prediction sample filtering can be called post filtering. By the prediction sample filtering procedure, some or all of the prediction samples can be filtered. In some cases, the prediction sample filtering procedure can be omitted.

[0110] The image encoding device 100 generates a residual sample for the current block based on the (filtered) prediction sample (S410). The image encoding device 100 can compare the prediction sample with the original sample of the current block based on phase and derive a residual sample value.

[0111] The image encoding device 100 can encode image information including the information related to the intra prediction (prediction information) and the residual information related to the residual sample (S420). The prediction information can include the intra prediction mode information and the intra prediction type information. The image encoding device 100 can output the encoded image information in the form of a bitstream. The output bitstream can be transmitted to the image decoding device 200 via a storage medium or a network.

[0112] The residual information can include a residual coding syntax described later. The image encoding device 100 can convert / quantize the residual sample to derive quantized transform coefficients. The residual information can include information about the quantized transform coefficients.

[0113] On the one hand, as described above, the image encoding device 100 can generate a restored picture (including restored samples and restored blocks). For this purpose, the image encoding device 100 can re-perform inverse quantization / inverse transformation processing on the quantized transform coefficients to derive (corrected) residual samples. The reason for performing inverse quantization / inverse transformation again after converting / quantizing the residual samples in this way is to derive the same residual samples as those derived by the image decoding device 200 as described above. The image encoding device 100 can generate a restored block including restored samples for the current block based on the prediction samples and the (corrected) residual samples. A restored picture for the current picture can be generated based on the restored block. As described above, in-loop filtering procedures and the like can be further applied to the restored picture.

[0114] The video / image decoding procedure based on intra prediction and the intra prediction unit in the image decoding device 200 can generally include the following by way of example.

[0115] The image decoding device can perform operations corresponding to the operations performed by the image encoding device 100.

[0116] S600 to S620 can be performed by the intra prediction unit 265 of the image decoding apparatus 200, and the prediction information of S600 and the residual information of S630 can be obtained from the bit stream by the entropy decoding unit 210 of the image decoding apparatus 200. The residual processing unit of the image decoding apparatus 200 can derive residual samples for the current block based on the residual information. Specifically, the inverse quantization unit 220 of the residual processing unit performs inverse quantization based on the quantization conversion coefficients derived based on the residual information to derive conversion coefficients, and the inverse conversion unit 230 of the residual processing unit can perform inverse conversion on the conversion coefficients to derive residual samples for the current block. S640 can be performed by the addition unit 235 or the restoration unit of the image decoding apparatus 200.

[0117] Specifically, the image decoding apparatus 200 can derive the intra prediction mode / type for the current block based on the received prediction information (intra prediction mode / type information) (S600). The image decoding apparatus 200 can derive the peripheral reference samples of the current block (S610). The image decoding apparatus 200 generates prediction samples within the current block based on the intra prediction mode / type and the peripheral reference samples (S620). In this case, the image decoding apparatus 200 can perform a prediction sample filtering procedure. Prediction sample filtering is sometimes called post filtering. By the prediction sample filtering procedure, some or all of the prediction samples can be filtered. In some cases, the prediction sample filtering procedure can be omitted.

[0118] The image decoding device 200 generates residual samples for the current block based on the received residual information. The image decoding device 200 can generate restored samples for the current block based on the prediction samples and the residual samples, and derive a restored block including the restored samples (S630). A restored picture for the current picture can be generated based on the restored block. As described above, an in-loop filtering procedure or the like can be further applied to the restored picture.

[0119] Here, the intra prediction unit 265 of the image decoding device 200 can include an intra prediction mode / type determination unit 266, a reference sample derivation unit 267, and a prediction sample derivation unit 268. The intra prediction mode / type determination unit 266 determines the intra prediction mode / type for the current block based on the intra prediction mode / type information generated and signaled by the intra prediction mode / type determination unit 186 of the image encoding device 100. The reference sample derivation unit 267 derives the peripheral reference samples of the current block, and the prediction sample derivation unit 268 can derive the prediction samples of the current block. On the other hand, although not shown in the figure, when the above-described prediction sample filtering procedure is performed, the intra prediction unit 265 can further include a prediction sample filter unit (not shown).

[0120] The intra prediction mode information can include, for example, flag information (e.g., intra_luma_mpm_flag) indicating whether MPM (most probable mode) is applied to the current block or the remaining mode is applied. When the MPM is applied to the current block, the prediction mode information can further include index information (e.g., intra_luma_mpm_idx) indicating any one of the intra prediction mode candidates (MPM candidates). The intra prediction mode candidates (MPM candidates) can be composed of an MPM candidate list or an MPM list. Also, when the MPM is not applied to the current block, the intra prediction mode information can further include remaining mode information (e.g., intra_luma_mpm_remainder) indicating any one of the remaining intra prediction modes excluding the intra prediction mode candidates (MPM candidates). The image decoding device 200 can determine the intra prediction mode of the current block based on the intra prediction mode information. A separate MPM list can be configured for the above-described MIP.

[0121] Furthermore, the intra prediction technique information can be realized in various forms. As an example, the intra prediction type information can include intra prediction type index information indicating any one of the intra prediction types. As another example, the intra prediction type information can include reference sample line information (e.g., intra_luma_ref_idx) indicating whether the MRL is applied to the current block and, if applied, which reference sample line is used, ISP flag information (e.g., intra_subpartitions_mode_flag) indicating whether the ISP is applied to the current block, ISP type information (e.g., intra_subpartitions_split_flag) indicating the split type of subpartitions when the ISP is applied, flag information indicating whether PDPC is applied, or flag information indicating whether LIP is applied, and can include at least one of them. Also, the intra prediction type information can include an MIP flag indicating whether MIP is applied to the current block.

[0122] The intra prediction mode information and / or the intra prediction type information can be encoded / decoded via the coding method described in this document. For example, the intra prediction mode information and / or the intra prediction type information can be encoded / decoded via entropy coding (e.g., CABAC, CAVLC) based on truncated (rice) binary code.

[0123] Template based intra mode derivation (TIMD)

[0124] FIG. 8 is a diagram for explaining a method for inducing an intra mode based on a template (TIMD).

[0125] In TIMD, for the IPM (intra prediction mode) intra modes of adjacent intra-blocks and inter-blocks in the neighborhood, after obtaining the SATD (Sum of absolute transformed difference) between the predicted block predicted from the template region and the actual restored samples, the mode with the least SATD can be selected as the intra mode of the current block.

[0126] Alternatively, in TIMD, after selecting the two modes with the least SATD, the predicted blocks for the two prediction modes can be blended by the weighted sum method and used as the predicted block of the current block.

[0127] The method of blending two modes can be applied when the following condition (Equation 1) is satisfied.

[0128]

Equation

[0129] When the above condition is satisfied, blend two modes to generate a predicted block, otherwise, only select one mode with the minimum SATD value.

[0130] The ratio of the weights when blending two predicted blocks is as shown in Equation 2 below.

[0131]

Equation

[0132]

Equation

[0133] Multi - reference line (MRL) intra prediction

[0134] FIG. 9 and FIG. 10 are diagrams for explaining the multiple-reference line (MRL) (intra prediction) mode.

[0135] Conventional intra prediction uses only the peripheral samples of the topmost line and the leftmost line of the current block as reference samples for intra prediction. However, in the Multiple-reference line (MRL) method, intra prediction can be performed using peripheral samples located on sample lines that are one to three sample distances away from the top side and / or the left side of the current block as reference samples. FIG. 9 shows an example of the multiple-reference line, where the multiple-reference line index (e.g., mrl_idx) indicates which line is used for intra prediction for the current block.

[0136] For example, the multiple-reference line index can be signaled via the coding unit syntax in Table 1. The multiple-reference line index can be configured in the form of the intra_luma_ref_idx syntax element.

[0137] [Table 1]

[0138] intra_luma_ref_idx[x0][y0] can specify the intra reference line index IntraLumaRefLineIdx[x0][y0] shown in Table 2. If intra_luma_ref_idx does not exist, the value of intra_luma_ref_idx can be inferred to be 0. Intra_luma_ref_idx can be called the (intra) reference sample line index, mrl_idx, or index information. Also, intra_luma_ref_idx may sometimes be called intra_luma_ref_line_idx.

[0139]

Table 2

[0140] The MRL can be disabled for the block on the first line (row) within the CTU. This is to prevent the use of extended reference lines outside the current CTU line. Also, when the above-mentioned additional reference lines are used, the above-mentioned PDPC can be disabled.

[0141] In the MRL method, intra prediction was performed using peripheral samples located on sample lines one to three sample distances away from the upper side or the left side of the current block as reference samples. However, in the extended MRL (extended MRL) method, it is extended so that intra prediction can be performed using peripheral samples located on sample lines 12 sample distances away from the upper side and / or the left side of the current block as reference samples. The reference sample line indexes of the extended MRL list are composed of {1, 3, 5, 7, 12}. FIG. 10 shows an example of an extended MRL candidate list. Also, Table 3 shows the relationship between intra_luma_ref_idx and IntraLumaRefLineIdx[x0][y0] in the extended MRL.

[0142]

Table 3

[0143] [Embodiment]

[0144] The present disclosure proposes an embodiment of performing MRL based on a template. FIG. 11 is a flowchart showing an image encoding method and an image decoding method according to an embodiment of the present invention.

[0145] Referring to FIG. 11, the image encoding device 100 and the image decoding device 200 can generate a predicted value for the first reference line (S1110).

[0146] The first reference line can correspond to any one of a plurality of reference lines available for the MRL. Here, the plurality of reference lines can correspond to the reference line for the current block. Also, the first reference line can correspond to the template of the TIMD described above. The first reference line can be the reference line closest to the current block among the plurality of reference lines (for example, the reference line with an index value of 0).

[0147] The predicted value for the first reference line can be generated based on each of the plurality of reference lines. For example, among the plurality of reference lines, a predicted value for the first reference line can be generated based on any one of the reference lines, and another predicted value for the first reference line can be generated based on another one of the plurality of reference lines.

[0148] The image encoding device 100 and the image decoding device 200 can calculate the error between each predicted value and the restored value of the first reference line (the restored value of the first reference line) (S1120). Since the predicted value is generated based on each of the plurality of reference lines, the error can also be calculated for each of the plurality of reference lines.

[0149] Here, the error can be calculated based on the "difference between the predicted block predicted from the template area and the actual restored sample" of the TIMD described above. The error can be referred to as "cost", "error", "difference", etc. That is, in the present disclosure, "error", "cost", "error", "difference", etc. can have the same meaning.

[0150] The image encoding device 100 and the image decoding device 200 can derive a second reference line based on the calculated error (S1130). The second reference line can correspond to the reference line used for the intra prediction of the current block among the plurality of reference lines.

[0151] Hereinafter, the embodiments proposed through the present disclosure will be specifically described. The embodiments described below can be performed as at least one of the processes represented in FIG. 11.

[0152] Example 1

[0153] Embodiment 1 relates to a method of guiding an image encoding device 100 and an image decoding device 200 without signaling a multiple reference line index (ref_idx) during the execution of MRL. FIGS. 12 and 13 show diagrams for explaining Embodiment 1.

[0154] In FIG. 12, k is an integer greater than or equal to 1, and i is an integer greater than or equal to 0 and less than k. Referring to FIG. 12, samples of the i-th (ref_idx = i) reference line can be predicted using samples of a plurality of reference lines (ref_idx = 0... k) for the current block (CU). The i-th reference line can correspond to the first reference line. Among the predicted values of the i-th reference line, the predicted value most similar to the restored value (restored signal) of the i-th reference line, that is, the most similar reference line (second reference line), can be determined as the reference line (or the reference line index of the corresponding reference line) for predicting the current block. In the above description, the predicted value of the reference line can be the reference line index of the reference line.

[0155] In FIG. 13, Pred A(k) is the prediction signal of the region "A" predicted as the reference sample of ref_idx = k, and Pred L(k) is the prediction signal of the region "L" predicted as the reference sample of ref_idx = k, and Pred AL(k) can be the prediction signal of the region "AL" predicted as the reference sample of ref_idx = k. Also, Rec A is the already restored signal of the region "A", and Rec L is the already restored signal of the region "L", and Rec ALcan also be the already restored signal in the area "AL". Also, the width q A ×q A ) of the area "A"(p A may be different from the width W of the current block, and the height p A may be equal to 1 or greater than it, and may also be smaller than k. Also, the height p L ×q L ) of the area "L"(p L may be different from the height H of the current block, and the width q L may be equal to 1 or greater than it, and may also be smaller than k. Also, the p A ×q L ) of the area "AL"(p A , q L may be equal to 1 or greater than it, and may also be smaller than k.

[0156] Rec L 、Rec A and Rec AL each can correspond to the already restored signals of "L", "A" and "AL", which are areas adjacent to the current block CU, and the values (signals) Pred L 、Rec A and Rec AL predicted as reference samples with ref_idx = k at the same positions of each can be generated. L(k) 、Pred A(k) and Pred AL(k)

[0157] On the other hand, the error between the predicted value and the restored value can be calculated using SAD (sum of absolute differences), SATD or SSD (sum of squared differences), etc. For example, the error can be calculated by Equation 3 using the SAD between (Pred L(k) 、Rec L ), (Pred A(k) 、Rec A ), (Pred AL(k) 、Rec AL ). N indicates the number of predicted values.

[0158] [Number]

[0159] As another example, when only the left and above reference samples of the current block are available, the error can be calculated by Equation 4.

[0160] [Number]

[0161] As another example, when only the left reference sample of the current block is available, the error can be calculated by Equation 5.

[0162] [Number]

[0163] As another example, when only the above reference sample of the current block is available, the error can be calculated by Equation 6.

[0164] [Number]

[0165] Differing from the embodiments, when the current block is located at a slice boundary, a CTU boundary, a picture boundary, etc., the embodiments of the present disclosure can be disabled.

[0166] On the other hand, through a 1-bit first flag (derive_mrl_flag), the method of Embodiment 1 for deriving ref_idx and the existing MRL technology can be selectively used.

[0167] [Table 4]

[0168] As shown in Table 4, the value of mrl_flag indicating whether the MRL is applied is determined, and when the MRL is applied (mrl_flag = 0), derive_mrl_flag can be signaled. When derive_mrl_flag = 1, the reference line with the minimum error calculated as proposed in Example 1 can be determined as the reference line (second reference line) for the prediction of the current block. In contrast, when derive_mrl_flag = 0, the multiple reference line index (intra_luma_ref_idx) is signaled in the same way as the existing method, and the MRL mode can be executed using this.

[0169] Differing from the embodiment, as shown in Table 5, when mrl_flag is not 0, the second reference line can be derived without separately signaling a flag and the multiple reference line index.

[0170] [Table 5]

[0171] FIG. 14 is a flowchart showing the image encoding method according to Example 1, and FIG. 15 is a flowchart showing the image decoding method according to Example 1.

[0172] Referring to FIG. 14, the image encoding apparatus 100 determines whether to apply the MRL mode (S1410), and when it is determined not to apply the MRL mode, mrl_flag = 0 can be encoded (S1420).

[0173] When it is determined to apply the MRL mode, the image encoding device 100 can determine whether to signal the reference line index (ref_idx) (S1430). When it is determined to signal the reference line index (that is, when applying the conventional MRL method), the image encoding device 100 can encode mrl_flag = 1, derive_mrl_flag = 0, and ref_idx (S1440). In contrast, when it is determined not to signal the reference line index (that is, when applying the method proposed in the first embodiment), the image encoding device 100 can encode mrl_flag = 1 and derive_mrl_flag = 1 (S1450).

[0174] Referring to FIG. 15, the image decoding device 200 acquires mrl_flag from the bit stream (S1510), and based on the value of mrl_flag, can determine whether to apply the MRL mode (S1520).

[0175] When the MRL mode is applied (mrl_flag = 1), the image decoding device 200 acquires derive_mrl_flag from the bit stream (S1530) and can determine the value of derive_mrl_flag (S1540).

[0176] When derive_mrl_flag = 0, the existing MRL method is applied. Therefore, the image decoding device 200 acquires ref_idx from the bit stream (S1550), and among the plurality of reference lines, can perform intra prediction for the current block based on the reference line indicated by ref_idx.

[0177] In contrast, when derive_mrl_flag = 1, the method proposed by the first embodiment is applied. Therefore, the image decoding device 200 does not acquire ref_idx, induces the second reference line among the plurality of reference lines based on the aforementioned error, and can perform intra prediction for the current block based on the second reference line.

[0178] According to the embodiments described above, by inducing a multiple reference line index without signaling, prediction performance can be improved and coding efficiency can be improved.

[0179] Example 2

[0180] Embodiment 2 corresponds to a method of reordering multiple reference line candidates (reference line candidate lists or lists) based on errors.

[0181] FIG. 16 is a flowchart showing an image encoding method and an image decoding method according to Embodiment 2.

[0182] Referring to FIG. 16, an image encoding apparatus 100 and an image decoding apparatus 200 can determine an intra prediction mode (ipm) of a current block (S1610).

[0183] The image encoding apparatus 100 can encode and signal information about the determined intra prediction mode. The image decoding apparatus 200 can obtain information about the intra prediction mode from a bitstream and determine the intra prediction mode based on this.

[0184] A DIMD mode (decoder-side intra mode derivation), MIP (matrix-based intra prediction) mode, MPM (most probable modes), SMPM (secondary most probable mode) mode, ISP (intra sub-partitions) mode, etc. for obtaining an intra prediction mode can be included. Also, mrl_idx can be obtained in the process of S1610.

[0185] The image encoding apparatus 100 and the image decoding apparatus 200 can reorder the order of a plurality of reference lines (S1620).

[0186] The image encoding device 100 and the image decoding device 200 can calculate the error between the predicted value of the first reference line and the restored value of the first reference line, and reorder the reference line candidates based on the calculated error. The error calculation can be performed by the method described above. Specifically, the error can be calculated using any one of the methods of Formulas 3 to 6. For example, when the reference line candidate list is mrl_candidate_list = {0, 1, 3, 5, 7, 12} and it is possible to approach up to the 12th reference line from the current block, the errors are calculated for 1, 3, 5, 7, 12 which are the candidates in each list except the 0th reference line, and the reference line candidates can be reordered in ascending order based on the calculated error.

[0187] The image encoding device 100 and the image decoding device 200 can perform prediction and restoration for the current block based on the reordered list (S1630). Among the reordered reference line candidates, the reference line indicated by mrl_idx is determined as the second reference line, and intra prediction and restoration based on the second reference line can be performed.

[0188] According to the embodiment, the image encoding device 100 and the image decoding device 200 can perform the process of updating mrl_idx of the reference line candidates (reference line candidate list) in the S1620 process.

[0189] For example, if mrl_candidate_list = {0, 1, 3, 5, 7, 12}, mrl_idx = 7 is obtained in the S1610 process, and the reference line candidate list is reordered to mrl_candidate_list = {0, 1, 7, 5, 3, 12} in the S1620 process, since the 4th position in the reference line candidate list was 7, the value of mrl_idx can be updated to 3 based on the reordered reference line candidate list.

[0190] Example 3

[0191] Embodiment 3 is a method of configuring a reference line candidate as a combination of an upper reference line and a left reference line.

[0192] FIG. 17 shows a diagram for explaining the upper reference line and the left reference line. In FIG. 17, k A and k L can have the same or different values from each other. The upper reference line is a reference line located above (A) the current block (CU), and the left reference line can be a reference line located on the left side (L) of the current block.

[0193] The image encoding device 100 and the image decoding device 200 can configure a reference line candidate list. The reference line candidate list can be composed of a reference line candidate list (mrl_idx_candi_above) for the upper reference line and a reference line candidate list (mrl_idx_candi_left) for the left reference line.

[0194] The image encoding device 100 and the image decoding device 200 can generate a predicted value for the first reference line based on each of the combinations of the upper reference line and the left reference line, and derive the second reference line based on any one of the generated combinations.

[0195] Assuming mrl_idx_candi_above = {0, 1, 3, 5, 7, 12} and mrl_idx_candi_left = {0, 1, 3, 5, 7, 12}, errors can be calculated by various combinations of (k L , k A ). For example, (k L , k A ) can have combinations of (0, 3), (1, 3), (3, 3), (5, 3), (7, 3), (12, 3). Various combinations of (k L , k A ) can be defined as an agreement between the image encoding device 100 and the image decoding device 200.

[0196] According to the embodiment, when a reference line (ref_idx = k) having the minimum error is determined, combinations of (k L , k A ) can be additionally generated using the reference line. For example, (k L , k A ) can be composed of combinations of (k, k - 1), (k, k + 1), (k - 1, k), (k + 1, k) close to the determined reference line (ref_idx = k).

[0197] According to the embodiment, when the x value of the upper left corner position of the current block (CU) is 0 or the y value is 0, or when the current block is located at a slice boundary, CTU boundary, picture boundary, etc., (k L , k A ) can also be composed of the following combinations.

[0198] - When the x value of the upper left corner position of the CU is 0 or it is located at a slice boundary, CTU boundary, picture boundary, etc.: (k L , k A ) = (0, k)

[0199] - When the y value of the upper left corner position of the CU is 0 or it is located at a slice boundary, CTU boundary, picture boundary, etc.: (k L , k A ) = (k, 0)

[0200] - When both the x value and the y value are 0, or it is located at a slice boundary, CTU boundary, picture boundary, etc.: Do not perform the proposed combination

[0201] FIG. 18 is a diagram illustrating a content streaming system to which an embodiment of the present disclosure can be applied.

[0202] As shown in FIG. 18, a content streaming system to which an embodiment of the present disclosure is applied can generally include an encoding server, a streaming server, a web server, a media storage, a user device, and a multimedia input device.

[0203] The encoding server compresses the content input from multimedia input devices such as smartphones, cameras, and camcorders into digital data to generate a bitstream, and plays a role of transmitting this to the streaming server. As another example, when multimedia input devices such as smartphones, cameras, and camcorders directly generate a bitstream, the encoding server can be omitted.

[0204] The bitstream can be generated by an image encoding method and / or an image encoding device to which the embodiments of the present disclosure are applied, and the streaming server can temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[0205] The streaming server transmits multimedia data to a user device based on a user's request via a web server, and the web server can play a role of a medium for informing the user of what services are available. When the user requests a desired service from the web server, the web server transmits this to the streaming server, and the streaming server can transmit multimedia data to the user. At this time, the content streaming system can include a separate control server, and in this case, the control server can play a role of controlling commands / responses between each device in the content streaming system.

[0206] The streaming server can receive content from a media storage and / or an encoding server. For example, when receiving content from the encoding server, the content can be received in real time. In this case, in order to provide a smooth streaming service, the streaming server can store the bitstream for a certain period of time.

[0207] Examples of the user device may include a mobile phone, a smart phone, a laptop computer, a digital broadcast terminal, a PDA (personal digital assistants), a PMP (portable multimedia player), a navigation device, a slate PC, a tablet PC, an ultrabook, a wearable device, for example, a smartwatch, smart glass, an HMD (head mounted display), a digital TV, a desktop computer, a digital signage, and the like.

[0208] Each server in the content streaming system can be operated as a distributed server. In this case, the data received from each server can be distributedly processed.

[0209] The scope of the present disclosure includes software or machine-executable commands (e.g., an operating system, an application, firmware, a program, etc.) that enable the operations according to the methods of various embodiments to be executed on a device or a computer, and a non-transitory computer-readable medium on which such software or commands are stored and can be executed on the device or the computer.

Industrial Applicability

[0210] Examples according to the present disclosure can be used for encoding / decoding an image.

Claims

1. An image decoding method performed by an image decoding apparatus, comprising: generating a predicted value by predicting a first reference line for the current block based on each of a plurality of reference lines for the current block; calculating an error between each predicted value and a restored value of the first reference line; inducing at least one second reference line used for intra prediction of the current block from among the plurality of reference lines based on the error.

2. The image decoding method according to claim 1, wherein the first reference line is a reference line closest to the current block among the plurality of reference lines.

3. The image decoding method according to claim 1, wherein the second reference line is induced to a reference line having a minimum error among the plurality of reference lines.

4. The image decoding method according to claim 3, wherein the second reference line is induced to the reference line having the minimum error based on the first flag obtained from the bit stream indicating a first value.

5. The step of inducing the second reference line includes: rearranging an order of the plurality of reference lines based on the error; inducing a reference line indicated by index information among the rearranged plurality of reference lines to the second reference line.

6. The image decoding method according to claim 5, wherein the plurality of reference lines are rearranged in ascending order based on the error.

7. The image decoding method according to claim 5, wherein the index information is obtained from the bit stream based on the first flag obtained from the bit stream indicating a second value.

8. The plurality of reference lines include an upper reference line and a left reference line, the predicted value is generated based on each combination of the upper reference line and the left reference line, the second reference line is induced based on any one of the combinations.

9. An image encoding method performed by an image encoding apparatus, comprising: generating a predicted value by predicting a first reference line for the current block based on each of a plurality of reference lines for the current block; calculating an error between each predicted value and a restored value of the first reference line; Based on the error, inducing at least one second reference line used for intra prediction of the current block among the plurality of reference lines, in an image encoding method.

10. A method for transmitting a bitstream generated by an image encoding method, wherein the image encoding method includes generating a predicted value by predicting a first reference line for the current block based on each of a plurality of reference lines for the current block; calculating an error between each predicted value and a restored value of the first reference line; and based on the error, inducing at least one second reference line used for intra prediction of the current block among the plurality of reference lines, in the method.

Images