Method and apparatus for video coding and decoding using adaptive reference line candidate list

Abstract

The embodiment discloses a video coding and decoding method and device using an adaptive reference line candidate list. In this embodiment, an image decoding device decodes an intra prediction mode and a multiple reference line (MRL) index of a current block from a bitstream. An image decoding apparatus acquires a length of an MRL candidate list indicating a number of reference lines included in the MRL candidate list, and acquires a method for filling at least one MRL candidate list. The image decoding apparatus generates an MRL candidate list by adding a reference line corresponding to a length of the MRL candidate list to the MRL candidate list using a method for filling at least one MRL candidate list. An image decoding device derives a reference line from an MRL candidate list by using an MRL index, and then generates a prediction block of a current block according to an intra prediction mode by using the derived reference line.

Description

Technical Field

[0001] The present invention relates to a video encoding and decoding method and apparatus using an adaptive reference line candidate list. Background Art

[0002] The statements in this section merely provide background technical information related to the present invention and do not necessarily constitute prior art.

[0003] Since video data has a large amount of data compared to audio data or still image data, video data requires a large amount of hardware resources (including memory) to store or transmit uncompressed video data.

[0004] Accordingly, an encoder is generally used to compress and store or transmit video data. A decoder receives the compressed video data, decompresses the received compressed video data, and plays the decompressed video data. Video compression technologies include H.264 / Advanced Video Coding (AVC), High Efficiency Video Coding (HEVC), and Versatile Video Coding (VVC), and the Versatile Video Coding (VVC) has a coding and decoding efficiency that is about 30% or more higher than that of HEVC.

[0005] However, as the image size, resolution, and frame rate gradually increase, the amount of data to be encoded also increases. Accordingly, there is a need to provide a new compression technology with higher coding and decoding efficiency and improved image enhancement effects compared to existing compression technologies.

[0006] Intra prediction uses information about pixels within a common picture to predict the pixel values of a current block to be encoded. Intra prediction can select one of multiple intra prediction modes that best suit the characteristics of the picture and use the selected intra prediction mode to predict the current block. The encoder selects one of the multiple intra prediction modes and encodes the current block by using the selected mode. Then, the encoder can convey information about the mode to the decoder.

[0007] The HEVC technology uses a total of 35 intra prediction modes for intra prediction, including 33 angular modes with directionality and 2 non-angular modes without directionality. However, as the spatial resolution of the video increases from 720×480 to 2048×1024 or 8192×4096, the unit size of the prediction block increases accordingly, which requires adding more diverse intra prediction modes. As Figure 3a shown, the VVC technology uses 65 further subdivided prediction modes for intra prediction, which allows for a much greater diversity of prediction directions compared to prior art.

[0008] On the other hand, when performing intra prediction, since the prediction block is generated by using the pixels around the current block, the performance of intra prediction depends on the selection of appropriate reference pixels. As a method for selecting reference pixels, a method of obtaining reference pixels from a more accurate direction by ensuring the diversity of prediction modes or a method of increasing the number of available reference pixel candidates can be used. The prior art corresponding to the latter is called Multiple Reference Line (MRL) or Multiple Reference Line Prediction (MRLP). For example, when MRL is adopted for intra prediction of the current block, in addition to the reference line adjacent to the current block at a one-pixel interval, pixels at a farther distance can also be used as reference pixels for prediction.

[0009] In the conventional MRL technique, the MRL candidate list listing the reference lines that can be referred to is uniformly applied to all blocks without discrimination. Therefore, a method for improving the MRL technique is needed to improve video coding and decoding efficiency and enhance video quality. Summary of the Invention

[0010] Technical Problem

[0011] The present invention is dedicated to providing a video coding and decoding method and apparatus for the MRL technique for intra prediction, which adaptively determines the MRL candidate list filling method and the number of reference lines included in the MRL candidate list.

[0012] Technical Solution

[0013] At least one aspect of the present invention provides a method for reconstructing a current block by a video decoding apparatus. The method includes decoding an intra prediction mode and a multiple reference line (MRL) index of the current block from a bitstream, where the MRL index indicates a reference line to be used for intra prediction of the current block within an MRL candidate list. The method further includes obtaining a length of the MRL candidate list indicating the number of reference lines included in the MRL candidate list. The method further includes obtaining at least one or more filling methods. The method further includes generating an MRL candidate list by adding reference lines corresponding to the length of the MRL candidate list to the MRL candidate list by using at least one or more filling methods. The method further includes deriving a reference line from the MRL candidate list by using the MRL index. The method further includes generating a prediction block of the current block according to the intra prediction mode by using the reference line.

[0014] Another aspect of the present invention provides a method for encoding a current block by a video encoding device. The method includes determining an intra prediction mode of the current block and a multi-reference line (MRL) index, where the multi-reference line index indicates, within an MRL candidate list, a reference line to be used for intra prediction of the current block. The method further includes obtaining a length of the MRL candidate list indicating the number of reference lines included in the MRL candidate list. The method further includes obtaining at least one or more padding methods. The method further includes generating the MRL candidate list by adding, using at least one or more padding methods, reference lines corresponding to the length of the MRL candidate list to the MRL candidate list. The method further includes deriving a reference line from the MRL candidate list by using the MRL index. The method further includes generating a prediction block of the current block according to the intra prediction mode by using the reference line.

[0015] Still another aspect of the present invention provides a computer-readable recording medium storing a bitstream generated by a video encoding method. The video encoding method includes determining an intra prediction mode of the current block and a multi-reference line (MRL) index, where the multi-reference line index indicates, within an MRL candidate list, a reference line to be used for intra prediction of the current block. The video encoding method further includes obtaining a length of the MRL candidate list indicating the number of reference lines included in the MRL candidate list. The video encoding method further includes obtaining at least one or more padding methods. The video encoding method further includes generating the MRL candidate list by adding, using at least one or more padding methods, reference lines corresponding to the length of the MRL candidate list to the MRL candidate list. The video encoding method further includes deriving a reference line from the MRL candidate list by using the MRL index. The video encoding method further includes generating a prediction block of the current block according to the intra prediction mode by using the reference line.

[0016] Beneficial Effects

[0017] As described above, the present invention provides a video encoding / decoding method and apparatus for refining a motion vector at a decoder side by using an intra prediction value generated by intra predicting a current block or by using an amplitude ratio of two motion vectors in one-way prediction using one reference image or two-way prediction where a current picture temporally deviates from the center of two reference images. Therefore, the video encoding / decoding method and apparatus improve video encoding / decoding efficiency and enhance video quality. Description of the Drawings

[0018] Figure 1 is a block diagram of a video encoding device that can implement the technology of the present invention.

[0019] Figure 2 illustrates a method of partitioning a block by using a quadtree plus binary tree plus ternary tree (QTBTTT) structure.

[0020] Figure 3a and Figure 3b shows multiple intra prediction modes including a wide angle intra prediction mode.

[0021] Figure 4 shows adjacent blocks of a current block.

[0022] Figure 5 is a block diagram of a video decoding apparatus that can implement the technology of the present invention.

[0023] Figure 6 is a schematic diagram showing reference lines utilized by multiple reference lines (MRL).

[0024] Figure 7 is another illustrative schematic diagram of reference lines utilized by MRL.

[0025] Figure 8 is a schematic diagram showing a method for filling an MRL candidate list according to at least one embodiment of the present invention.

[0026] Figure 9 is a schematic diagram showing a method for filling an MRL candidate list according to another embodiment of the present invention.

[0027] Figure 10 is a schematic diagram showing a method for filling an MRL candidate list according to another embodiment of the present invention.

[0028] Figure 11 is a schematic diagram showing reference lines of adjacent blocks of a current block according to at least one embodiment of the present invention.

[0029] Figure 12 is a schematic diagram showing the position of a current block in a coding tree unit (CTU) according to at least one embodiment of the present invention.

[0030] Figure 13 is a schematic diagram showing reference lines of a previously reconstructed block according to at least one embodiment.

[0031] Figure 14a and Figure 14b is a schematic diagram showing co-located blocks in a reference image according to at least one embodiment.

[0032] Figure 15 is a schematic diagram showing pixels for representing adjacent blocks of a current block according to at least one embodiment.

[0033] Figure 16 is a flowchart of a method for encoding a current block by a video encoding apparatus according to at least one embodiment of the present invention.

[0034] Figure 17It is a flowchart of a method for reconstructing a current block by a video decoding device according to at least one embodiment of the present invention. Detailed implementation

[0035] Hereinafter, some embodiments of the present invention will be described in detail with reference to the accompanying illustrative drawings. In the following description, the same reference numerals denote the same elements, although the elements are shown in different drawings. In addition, in the following description of some embodiments, when the detailed description of related known components and functions is considered to obscure the subject matter of the present invention, the detailed description of related known components and functions may be omitted for the sake of clarity and conciseness.

[0036] Figure 1 It is a block diagram of a video encoding device that can implement the technology of the present invention. Hereinafter, with reference to Figure 1 the illustration of, the video encoding device and the components of the device will be described.

[0037] The encoding device may include: an image splitter 110, a predictor 120, a subtractor 130, a transformer 140, a quantizer 145, a rearrangement unit 150, an entropy encoder 155, an inverse quantizer 160, an inverse transformer 165, an adder 170, a loop filter unit 180, and a memory 190.

[0038] Each component of the encoding device may be implemented as hardware or software, or implemented as a combination of hardware and software. Additionally, the functions of each component may be implemented as software, and the microprocessor may also be implemented to execute the functions of the software corresponding to each component.

[0039] A video consists of one or more sequences including a plurality of images. Each image is segmented into a plurality of regions, and encoding is performed on each region. For example, an image is segmented into one or more tiles and / or slices. Here, one or more tiles can be defined as a tile group. Each tile and / or slice is segmented into one or more coding tree units (CTUs). In addition, each CTU is segmented into one or more coding units (CUs) through a tree structure. Information applied to each coding unit (CU) is encoded as the syntax of the CU, and information applied to the CUs included in one CTU is encoded as the syntax of the CTU. In addition, information applied to all blocks in a slice is encoded as the syntax of the slice header, and information applied to all blocks constituting one or more images is encoded as a Picture Parameter Set (PPS) or an image header. Furthermore, information commonly referred to by a plurality of images is encoded as a Sequence Parameter Set (SPS). In addition, information commonly referred to by one or more SPSs is encoded as a Video Parameter Set (VPS). Moreover, information applied to a tile or a tile group can also be encoded as the syntax of the tile or tile group header. The syntax included in the SPS, PPS, slice header, tile or tile group header can be referred to as high-level syntax.

[0040] The image splitter 110 determines the size of the coding tree unit (CTU). Information about the size of the CTU (CTU size) is encoded as the syntax of the SPS or PPS and is transmitted to the video decoding device.

[0041] The image splitter 110 segments each image constituting the video into a plurality of coding tree units (CTUs) having a predetermined size, and then recursively segments the CTUs by using a tree structure. The leaf nodes in the tree structure become coding units (CUs), and the CUs are the basic units for encoding.

[0042] The tree structure can be a quadtree (QT), where a higher node (or parent node) is divided into four lower nodes (or child nodes) of the same size. The tree structure can also be a binary tree (BT), where a higher node is divided into two lower nodes. The tree structure can also be a ternary tree (TT), where a higher node is divided into three lower nodes at a ratio of 1:2:1. The tree structure can also be a structure that mixes two or more of the QT structure, BT structure, and TT structure. For example, a quadtree plus binarytree (QTBT) structure can be used, or a quadtree plus binarytreeternarytree (QTBTTT) structure can be used. Here, the binarytreeternarytree (BTTT) is added to the tree structure to form a multiple-type tree (MTT).

[0043] Figure 2 is a schematic diagram for describing a method of dividing a block by using the QTBTTT structure.

[0044] As Figure 2 shown, the CTU can first be divided into a QT structure. The quadtree division can be recursive until the size of the divided block reaches the minimum block size (MinQTSize) of the leaf nodes allowed in the QT. The entropy encoder 155 encodes a first flag (QT_split_flag) indicating whether each node of the QT structure is divided into four lower nodes and signals it to the video decoding device. When the leaf node of the QT is not larger than the maximum block size (MaxBTSize) of the root node allowed in the BT, the leaf node can be further divided into at least one of the BT structure or the TT structure. There can be multiple division directions in the BT structure and / or TT structure. For example, there can be two directions, namely, the direction of horizontally dividing the block of the corresponding node and the direction of vertically dividing the block of the corresponding node. As Figure 2 shown, when the MTT division starts, the entropy encoder 155 encodes a second flag (mtt_split_flag) indicating whether the node is divided, and a flag and / or a flag indicating the division type (binary or ternary) additionally indicating the division direction (vertical or horizontal) in the case where the node is divided, and signals it to the video decoding device.

[0045] Alternatively, before encoding a first flag (QT_split_flag) indicating whether each node is split into four lower-layer nodes, a CU split flag (split_cu_flag) indicating whether a node is split may also be encoded. When the value of the CU split flag (split_cu_flag) indicates that each node is not split, the block of the corresponding node becomes a leaf node in the split tree structure and becomes a CU, which is a basic unit of encoding. When the value of the CU split flag (split_cu_flag) indicates that each node is split, the video encoding device starts encoding the first flag first with the above scheme.

[0046] When QTBT is used as another example of a tree structure, there may be two types, that is, a type in which the block of the corresponding node is horizontally split into two blocks of the same size (i.e., symmetric horizontal split) and a type in which the block of the corresponding node is vertically split into two blocks of the same size (i.e., symmetric vertical split). The entropy encoder 155 encodes a split flag (split_flag) indicating whether each node of the BT structure is split into lower-layer blocks and split type information indicating the split type, and transmits them to the video decoding device. On the other hand, there may additionally be a type in which the block of the corresponding node is split into two asymmetric blocks. The asymmetric form may include a form in which the block of the corresponding node is split into two rectangular blocks with a size ratio of 1:3, or may also include a form in which the block of the corresponding node is split in the diagonal direction.

[0047] A CU may have various sizes according to the QTBT or QTBTTT split from the CTU. Hereinafter, the block corresponding to the CU to be encoded or decoded (i.e., the leaf node of the QTBTTT) is referred to as the "current block". When QTBTTT split is adopted, in addition to the square shape, the shape of the current block may also be a rectangular shape.

[0048] The predictor 120 predicts the current block to generate a prediction block. The predictor 120 includes an intra predictor 122 and an inter predictor 124.

[0049] Generally, each of the current blocks in the image can be predictively encoded. Generally, the prediction of the current block can be performed by using an intra prediction technique (which uses data from the image including the current block) or an inter prediction technique (which uses data from an image encoded before the image including the current block). Inter prediction includes both unidirectional prediction and bidirectional prediction.

[0050] The intra predictor 122 predicts the pixels in the current block by using the pixels (reference pixels) adjacent to the current block in the current image including the current block. According to the prediction direction, there are multiple intra prediction modes. For example, as Figure 3aAs shown, multiple intra prediction modes may include two non-directional modes including Planar mode and DC mode, and may include 65 directional modes. Adjacent pixels to be used and algorithm equations are defined differently according to each prediction mode.

[0051] To perform efficient directional prediction on a current block having a rectangular shape, the directional modes shown by the dashed arrows in Figure 3b (modes #67 to #80, intra prediction modes #-1 to #-14) may additionally be used. The directional modes may be referred to as "wide angle intra-prediction modes". In Figure 3b the arrows indicate the corresponding reference samples for prediction, rather than representing the prediction direction. The prediction direction is opposite to the direction indicated by the arrows. When the current block has a rectangular shape, the wide angle intra-prediction mode is a mode that performs prediction in the direction opposite to a specific directional mode without additional bit transmission. In this case, in the wide angle intra-prediction mode, some wide angle intra-prediction modes available for the current block may be determined by the ratio of the width to the height of the current block having a rectangular shape. For example, when the current block has a rectangular shape with a height less than the width, wide angle intra-prediction modes having an angle less than 45 degrees (intra prediction modes #67 to #80) are available. When the current block has a rectangular shape with a width greater than the height, wide angle intra-prediction modes having an angle greater than -135 degrees are available.

[0052] The intra predictor 122 may determine the intra prediction to be used for encoding the current block. In some examples, the intra predictor 122 may encode the current block by using multiple intra prediction modes, and may also select an appropriate intra prediction mode to be used from test modes. For example, the intra predictor 122 may calculate rate-distortion values by using rate-distortion analysis of multiple tested intra prediction modes, and may also select an intra prediction mode having the best rate-distortion characteristics from test modes.

[0053] The intra predictor 122 selects one intra prediction mode from multiple intra prediction modes, and predicts the current block by using adjacent pixels (reference pixels) and algorithm equations determined according to the selected intra prediction mode. Information about the selected intra prediction mode is encoded by the entropy encoder 155 and transmitted to the video decoding device.

[0054] The inter - frame predictor 124 generates a predicted block of the current block by using motion - compensation processing. The inter - frame predictor 124 searches for the block most similar to the current block in a reference image that has been encoded and decoded earlier than the current image, and generates a predicted block of the current block by using the searched - for block. Additionally, a motion vector (MV) is generated, which corresponds to the displacement between the current block in the current image and the predicted block in the reference image. Generally, motion estimation is performed on the luma component, and the motion vector calculated based on the luma component is used for both the luma component and the chroma component. The entropy encoder 155 encodes the motion information including the information of the reference image and the information about the motion vector used for predicting the current block, and transmits it to the video decoding device.

[0055] The inter - frame predictor 124 may also perform interpolation of the reference image or reference block to increase the prediction accuracy. In other words, sub - samples are interpolated between two consecutive integer samples by applying filter coefficients to a plurality of consecutive integer samples including two integer samples. When performing the process of searching for the block most similar to the current block on the interpolated reference image, the motion vector can represent fractional - unit precision rather than integer - sample - unit precision. For each target region to be encoded, such as units like slices, tiles, CTUs, CUs, etc., the precision or resolution of the motion vector can be set differently. When applying such an adaptive motion vector resolution (AMVR), information about the motion vector resolution to be applied to each target region should be signaled. For example, when the target region is a CU, information about the motion vector resolution applied to each CU is signaled. The information about the motion vector resolution can be information representing the precision of the motion - vector difference described below.

[0056] On the other hand, the inter-frame predictor 124 can perform inter-frame prediction by using bidirectional prediction. In the case of bidirectional prediction, two reference images and two motion vectors representing the positions of the blocks most similar to the current block in each reference image are used. The inter-frame predictor 124 selects a first reference image and a second reference image from reference picture list 0 (RefPicList0) and reference picture list 1 (RefPicList1), respectively. The inter-frame predictor 124 also searches for the block most similar to the current block in the corresponding reference image to generate a first reference block and a second reference block. In addition, a predicted block of the current block is generated by averaging or weighted averaging the first reference block and the second reference block. Further, motion information including information about the two reference images used for predicting the current block and information about the two motion vectors is transmitted to the entropy encoder 155. Here, reference picture list 0 may be composed of images in the pre-reconstructed images that are before the current image in the display order, and reference picture list 1 may be composed of images in the pre-reconstructed images that are after the current image in the display order. However, although not particularly limited thereto, pre-reconstructed images after the current image in the display order may be additionally included in reference picture list 0. Conversely, pre-reconstructed images before the current image may also be additionally included in reference picture list 1.

[0057] To minimize the amount of bits consumed for encoding motion information, various methods can be used.

[0058] For example, when the reference image and motion vector of the current block are the same as those of an adjacent block, information of the adjacent block that can be identified is encoded to transmit the motion information of the current block to the video decoding device. This method is called the merge mode.

[0059] In the merge mode, the inter-frame predictor 124 selects a predetermined number of merge candidates (hereinafter referred to as "merge candidates") from the adjacent blocks of the current block.

[0060] As the adjacent blocks for deriving the merge candidates, all or some of the left block A0, lower left block A1, upper block B0, upper right block B1, and upper left block B2 adjacent to the current block in the current image can be used, as Figure 4 shown. In addition, in addition to the current image where the current block is located, blocks within the reference image (which may be the same as or different from the reference image used for predicting the current block) can also be used as merge candidates. For example, a co-located block of the current block within the reference image or a block adjacent to the co-located block can be additionally used as a merge candidate. If the number of merge candidates selected by the above method is less than the preset number, zero vectors are added to the merge candidates.

[0061] The inter-frame predictor 124 configures a merge list including a predetermined number of merge candidates by using adjacent blocks. A merge candidate to be used as the motion information of the current block is selected from among the merge candidates included in the merge list, and merge index information for identifying the selected candidate is generated. The generated merge index information is encoded by the entropy encoder 155 and transmitted to the video decoding device.

[0062] The merge skip mode is a special case of the merge mode. After quantization, when all the transform coefficients for entropy coding are close to zero, only the adjacent block selection information is transmitted without transmitting the residual signal. By using the merge skip mode, relatively high coding efficiency can be achieved for images with slight motion, still images, screen content images, etc.

[0063] Thereafter, the merge mode and the merge skip mode are collectively referred to as the merge / skip mode.

[0064] Another method for encoding motion information is the advanced motion vector prediction (AMVP) mode.

[0065] In the AMVP mode, the inter-frame predictor 124 derives motion vector prediction candidates for the motion vector of the current block by using adjacent blocks of the current block. As the adjacent blocks for deriving the motion vector prediction candidates, all or some of the left block A0, the lower left block A1, the upper block B0, the upper right block B1, and the upper left block B2 adjacent to the current block in the current image shown in Figure 4 can be used. In addition, in addition to the current image where the current block is located, blocks in a reference image (which may be the same as or different from the reference image used for predicting the current block) can also be used as adjacent blocks for deriving the motion vector prediction candidates. For example, the co-located block of the current block in the reference image or a block adjacent to the co-located block can be used. If the number of motion vector candidates selected by the above method is less than the preset number, a zero vector is added to the motion vector candidates.

[0066] The inter-frame predictor 124 derives motion vector prediction candidates by using the motion vectors of adjacent blocks, and determines the motion vector prediction of the motion vector of the current block by using the motion vector prediction candidates. In addition, the motion vector difference is calculated by subtracting the motion vector prediction from the motion vector of the current block.

[0067] Motion vector prediction can be obtained by applying a predefined function (e.g., median and mean calculations, etc.) to motion vector prediction candidates. In this case, the video decoding device also knows the predefined function. Additionally, since the neighboring blocks used to derive the motion vector prediction candidates are blocks that have already been encoded and decoded, the video decoding device may also already know the motion vectors of the neighboring blocks. Therefore, the video encoding device does not need to encode the information used to identify the motion vector prediction candidates. Accordingly, in this case, the information about the motion vector difference and the information about the reference image used to predict the current block are encoded.

[0068] On the other hand, motion vector prediction can also be determined by a scheme of selecting any one of the motion vector prediction candidates. In this case, the information used to identify the selected motion vector prediction candidate is additionally encoded together with the information about the motion vector difference and the information about the reference image used to predict the current block.

[0069] The subtractor 130 generates a residual block by subtracting the prediction block generated by the intra predictor 122 or the inter predictor 124 from the current block.

[0070] The transformer 140 transforms the residual signal in the residual block having pixel values in the spatial domain into transform coefficients in the frequency domain. The transformer 140 can transform the residual signal in the residual block by using the entire size of the residual block as the transform unit, or the residual block can also be divided into multiple sub-blocks, and the transform can be performed by using the sub-blocks as the transform unit. Alternatively, the residual block is divided into two sub-blocks, namely a transform region and a non-transform region, to transform the residual signal by using only the transform region sub-block as the transform unit. Here, the transform region sub-block can be one of two rectangular blocks having a size ratio of 1:1 based on the horizontal axis (or the vertical axis). In this case, the entropy encoder 155 encodes a flag (cu_sbt_flag) indicating only the transform sub-block, and the direction (vertical / horizontal) information (cu_sbt_horizontal_flag) and / or the position information (cu_sbt_pos_flag), and signals it to the video decoding device. Additionally, the size of the transform region sub-block can have a size ratio of 1:3 based on the horizontal axis (or the vertical axis). In this case, the entropy encoder 155 additionally encodes a flag (cu_sbt_quad_flag) for dividing the corresponding segmentation, and signals it to the video decoding device.

[0071] On the other hand, the transformer 140 can perform the transformation of the residual block separately in the horizontal and vertical directions. For this transformation, various types of transformation functions or transformation matrices can be used. For example, a pair of transformation functions for horizontal transformation and vertical transformation can be defined as a multiple transform set (MTS). The transformer 140 can select a pair of transformation functions with the highest transformation efficiency in the MTS, and can transform the residual block on each of the horizontal and vertical directions. The entropy encoder 155 encodes the information (mts_idx) regarding the pair of transformation functions in the MTS and signals it to the video decoding device.

[0072] The quantizer 145 quantizes the transform coefficients output from the transformer 140 using quantization parameters and outputs the quantized transform coefficients to the entropy encoder 155. The quantizer 145 can also quantize the relevant residual block immediately without transforming any block or frame. The quantizer 145 can also apply different quantization coefficients (scaling values) according to the positions of the transform coefficients in the transform block. The quantization matrix applied to the quantized transform coefficients arranged in two dimensions can be encoded and signaled to the video decoding device.

[0073] The rearrangement unit 150 can perform rearrangement of the coefficient values on the quantized residual values.

[0074] The rearrangement unit 150 can change the 2D coefficient array to a 1D coefficient sequence by using coefficient scanning. For example, the rearrangement unit 150 can scan the coefficients from the DC coefficient to the high-frequency region using a zig-zag scan or a diagonal scan to output a 1D coefficient sequence. According to the size of the transform unit and the intra prediction mode, a vertical scan that scans the 2D coefficient array in the column direction and a horizontal scan that scans the 2D block type coefficients in the row direction can also be used instead of the zig-zag scan. In other words, according to the size of the transform unit and the intra prediction mode, the scan method to be used can be determined among the zig-zag scan, diagonal scan, vertical scan, and horizontal scan.

[0075] The entropy encoder 155 encodes the sequence of 1D quantized transform coefficients output from the rearrangement unit 150 by using various coding schemes including Context-based Adaptive Binary Arithmetic Code (CABAC), Exponential Golomb, etc. to generate a bitstream.

[0076] In addition, the entropy encoder 155 encodes information related to block partitioning (e.g., CTU size, CTU partitioning flag, QT partitioning flag, MTT partitioning type, and MTT partitioning direction, etc.) so that the video decoding device can partition blocks in the same way as the video encoding device. In addition, the entropy encoder 155 encodes information regarding the prediction type indicating whether the current block is encoded by intra prediction or inter prediction. The entropy encoder 155 encodes intra prediction information (i.e., information regarding the intra prediction mode) or inter prediction information (merge index in the case of the merge mode, and information regarding the reference image index and motion vector difference in the case of the AMVP mode) according to the prediction type. In addition, the entropy encoder 155 encodes information related to quantization (i.e., information regarding the quantization parameter and information regarding the quantization matrix).

[0077] The inverse quantizer 160 inverse-quantizes the quantized transform coefficients output from the quantizer 145 to generate transform coefficients. The inverse transformator 165 transforms the transform coefficients output from the inverse quantizer 160 from the frequency domain to the spatial domain to reconstruct the residual block.

[0078] The adder 170 adds the reconstructed residual block and the prediction block generated by the predictor 120 to reconstruct the current block. When performing intra prediction on the next block, the pixels in the reconstructed current block are used as reference pixels.

[0079] The loop filter unit 180 performs filtering on the reconstructed pixels to reduce block artifacts, ringing artifacts, blurring artifacts, etc. that occur due to block-based prediction and transform / quantization. The loop filter unit 180, as an in-loop filter, may include all or some of a deblocking filter 182, a sample adaptive offset (SAO) filter 184, and an adaptive loop filter (ALF) 186.

[0080] The deblocking filter 182 filters the boundaries between the reconstructed blocks to remove blocking artifacts that occur due to block-based coding / decoding, and the SAO filter 184 and the ALF 186 perform additional filtering on the deblocked video. The SAO filter 184 and the ALF 186 are filters for compensating for the difference between the reconstructed pixels and the original pixels that occur due to lossy coding. The SAO filter 184 applies an offset in units of CTUs to enhance the subjective image quality and coding efficiency. On the other hand, the ALF 186 performs block-based filtering and applies different filters by dividing the degree of the boundary and the amount of change of the corresponding block to compensate for distortion. Information about the filter coefficients to be used for the ALF can be encoded and signaled to the video decoding apparatus.

[0081] The reconstructed blocks filtered by the deblocking filter 182, the SAO filter 184, and the ALF 186 are stored in the memory 190. When all the blocks in an image are reconstructed, the reconstructed image can be used as a reference image for inter prediction of blocks within a subsequently encoded image.

[0082] The video encoding apparatus can store the bitstream of the encoded video data in a non-volatile storage medium or transmit the bitstream to the video decoding apparatus through a communication network.

[0083] Figure 5 is a functional block diagram of a video decoding apparatus that can implement the technology of the present invention. Hereinafter, with reference to Figure 5 , the video decoding apparatus and the components of the apparatus are described.

[0084] The video decoding apparatus may include an entropy decoder 510, a rearrangement unit 515, an inverse quantizer 520, an inverse transform unit 530, a predictor 540, an adder 550, a loop filter unit 560, and a memory 570.

[0085] Similar to Figure 1 the video encoding apparatus, each component of the video decoding apparatus may be implemented as hardware or software, or implemented as a combination of hardware and software. In addition, the functions of each component may be implemented as software, and the microprocessor may also be implemented to execute the functions of the software corresponding to each component.

[0086] The entropy decoder 510 extracts information related to block partitioning by decoding the bitstream generated by the video encoding apparatus to determine the current block to be decoded, and extracts prediction information and information about the residual signal required to reconstruct the current block.

[0087] The entropy decoder 510 determines the size of a coding tree unit (CTU) by extracting information about the CTU size from a sequence parameter set (SPS) or a picture parameter set (PPS), and divides an image into CTUs of a determined size. In addition, the CTU is determined as the top layer (i.e., the root node) of a tree structure, and the segmentation information of the CTU is extracted to divide the CTU by using the tree structure.

[0088] For example, when dividing a CTU by using a QTBTTT structure, first, a first flag (QT_split_flag) related to the segmentation of a quantization tree (QT) is extracted to divide each node into four lower-layer nodes. In addition, a second flag (mtt_split_flag) related to the segmentation of a multi-tree transform (MTT), a segmentation direction (vertical / horizontal), and / or a segmentation type (binary / trinary) are extracted for a node corresponding to a leaf node of the QT to divide the corresponding leaf node into an MTT structure. As a result, each node below the leaf node of the QT is recursively divided into a binary tree (BT) or a ternary tree (TT) structure.

[0089] As another example, when dividing a CTU by using a QTBTTT structure, a CU segmentation flag (split_cu_flag) indicating whether to divide a coding unit (CU) is extracted. When dividing the corresponding block, the first flag (QT_split_flag) may also be extracted. During the division process, for each node, zero or more recursive MTT divisions may occur after zero or more recursive QT divisions. For example, for a CTU, the MTT division may occur immediately, or conversely, only multiple QT divisions may occur.

[0090] As another example, when dividing a CTU by using a QTBT structure, a first flag (QT_split_flag) related to the segmentation of a QT is extracted to divide each node into four lower-layer nodes. In addition, a segmentation flag (split_flag) indicating whether to further divide the node corresponding to the leaf node of the QT into a BT and segmentation direction information are extracted.

[0091] On the other hand, when the entropy decoder 510 determines a current block to be decoded by using the segmentation of a tree structure, the entropy decoder 510 extracts information about a prediction type indicating whether the current block is intra-frame predicted or inter-frame predicted. When the prediction type information indicates intra-frame prediction, the entropy decoder 510 extracts a syntax element for intra-frame prediction information (intra-frame prediction mode) of the current block. When the prediction type information indicates inter-frame prediction, the entropy decoder 510 extracts information about syntax elements representing inter-frame prediction information, that is, a motion vector and a reference image to which the motion vector refers.

[0092] In addition, the entropy decoder 510 extracts quantization-related information and extracts information on the quantized transform coefficients of the current block as information on the residual signal.

[0093] The rearrangement unit 515 can change the sequence of 1D quantized transform coefficients entropy decoded by the entropy decoder 510 back into a 2D coefficient array (i.e., a block) in the reverse order of the coefficient scan order performed by the video coding device.

[0094] The inverse quantizer 520 inverse quantizes the quantized transform coefficients and inverse quantizes the quantized transform coefficients by using the quantization parameter. The inverse quantizer 520 can also apply different quantization coefficients (scaling values) to the quantized transform coefficients arranged in 2D. The inverse quantizer 520 can perform inverse quantization by applying a matrix of quantization coefficients (scaling values) from the video coding device to the 2D array of quantized transform coefficients.

[0095] The inverse transformer 530 reconstructs the residual signal by inverse-transforming the inverse quantized transform coefficients from the frequency domain to the spatial domain to generate a residual block of the current block.

[0096] In addition, when the inverse transformer 530 inverse-transforms a partial region (sub-block) of the transform block, the inverse transformer 530 extracts a flag (cu_sbt_flag) for inverse-transforming only the sub-block of the transform block, direction (vertical / horizontal) information (cu_sbt_horizontal_flag) of the sub-block, and / or position information (cu_sbt_pos_flag) of the sub-block. The inverse transformer 530 also inverse-transforms the transform coefficients of the corresponding sub-block from the frequency domain to the spatial domain to reconstruct the residual signal, and fills the un-inverse-transformed region with the value "0" as the residual signal to generate the final residual block of the current block.

[0097] In addition, when applying MTS, the inverse transformer 530 determines the transform function or transform matrix to be applied in each of the horizontal and vertical directions by using the MTS information (mts_idx) signaled from the video coding device. The inverse transformer 530 also performs inverse transformation on the transform coefficients in the transform block in the horizontal and vertical directions by using the determined transform function.

[0098] The predictor 540 can include an intra predictor 542 and an inter predictor 544. When the prediction type of the current block is intra prediction, the intra predictor 542 is activated, and when the prediction type of the current block is inter prediction, the inter predictor 544 is activated.

[0099] The intra predictor 542 determines the intra prediction mode of the current block among multiple intra prediction modes according to the syntax element of the intra prediction mode extracted from the entropy decoder 510. The intra predictor 542 also predicts the current block by using the adjacent reference pixels of the current block according to the intra prediction mode.

[0100] The inter - frame predictor 544 determines the motion vector of the current block and the reference image for the motion vector reference by using the syntax element of the inter - frame prediction mode extracted from the entropy decoder 510.

[0101] The adder 550 reconstructs the current block by adding the residual block output from the inverse transformer 530 to the prediction block output from the inter - frame predictor 544 or the intra - frame predictor 542. When performing intra - frame prediction on the blocks to be decoded subsequently, the pixels within the reconstructed current block are used as reference pixels.

[0102] The loop filter unit 560 as an in - loop filter may include a de - blocking filter 562, a SAO filter 564, and an ALF 566. The de - blocking filter 562 performs de - blocking filtering on the boundaries between the reconstructed blocks to remove block artifacts that occur due to block - based unit decoding. The SAO filter 564 and the ALF 566 perform additional filtering on the reconstructed blocks after de - blocking filtering to compensate for the difference between the reconstructed pixels and the original pixels that occurs due to lossy coding. The filter coefficients of the ALF are determined by using the information about the filter coefficients decoded from the bitstream.

[0103] The reconstructed blocks filtered by the de - blocking filter 562, the SAO filter 564, and the ALF 566 are stored in the memory 570. When all the blocks in an image are reconstructed, the reconstructed image can be used as a reference image for inter - frame prediction of the blocks within the image to be encoded subsequently.

[0104] In some embodiments, the present invention relates to encoding and decoding video images as described above. More specifically, the present invention provides a video encoding and decoding method and apparatus that apply the multiple reference line (MRL) technique to intra - frame prediction, adaptively determine the MRL candidate list filling method, and adaptively determine the number of reference lines included in the MRL candidate list.

[0105] The following embodiments may be performed by the intra - frame predictor 122 in a video encoding device. The following embodiments may also be performed by the intra - frame predictor 542 in a video decoding device.

[0106] When encoding the current block, the video encoding device may generate signaling information associated with the present embodiment from the perspective of optimizing rate - distortion. The video encoding device may encode the signaling information using the entropy encoder 155 and send the encoded signaling information to the video decoding device. The video decoding device may decode the signaling information associated with the decoding of the current block from the bitstream using the entropy decoder 510.

[0107] In the following description, the term "target block" may be used interchangeably with the current block or coding unit (CU), or may refer to some regions of the coding unit.

[0108] In addition, a value of true for a flag indicates the case where the flag is set to 1. Further, a value of false for a flag indicates the case where the flag is set to 0.

[0109] I. Multiple Reference Lines (MRL)

[0110] Several techniques have been introduced to improve coding efficiency based on intra prediction. When predicting the current block according to intra prediction, the MRL technique can use neighboring pixels and pixels at a greater distance separated from the current block by one pixel as reference pixels for prediction. At this time, pixels at the same distance from the current block are grouped and named reference lines. The MRL technique performs intra prediction of the current block by utilizing the pixels on the selected reference line.

[0111] To indicate the reference line to be used when performing intra prediction, the video coding device signals the reference line index intra_luma_ref_idx to the video decoding device. In existing Versatile Video Coding (VVC), the reference line represented by each intra_luma_ref_idx is shown in Figure 6 Existing VVC uses intra_luma_ref_idx to indicate one of the three reference lines closest to the current block. The bit assignment for the corresponding reference line index values is shown in Table 1.

[0112] [Table 1]

[0113]

[0114] In the Enhanced Compression Model (ECM) (a technology beyond VVC), the number of reference lines that can be referred to in MRL is extended to six, thereby allowing the use of reference lines with intra_luma_ref_idx values of {0, 1, 3, 5, 7, 12}. In ECM, the reference line represented by the corresponding index intra_luma_ref_idx is shown in Figure 7 In addition, the bit assignment for the corresponding reference line index values is shown in Table 2.

[0115] [Table 2]

[0116]

[0117] In VVC, since the MRL cannot be applied to the blocks at the first line in a CTU, the blocks at that position are always predicted by using intra_luma_ref_idx 0 without parsing the information about the reference lines. Similarly, in ECM, since the MRL cannot be applied to the blocks at the first line in a CTU, the blocks at that position are always predicted by using intra_luma_ref_idx 0 without parsing the information about the reference lines. Additionally, in ECM, for predicting the blocks outside the first line within the current CTU, the video coding device does not test whether to use the reference lines included in the top CTU among the reference lines available for MRL. The video coding device may signal one of the tested reference lines to be used to the video decoding device based on Table 2.

[0118] The reference line index intra_luma_ref_idx for intra prediction used by VVC and the syntax for signaling the prediction mode of the current block are shown in Table 3.

[0119] [Table 3]

[0120]

[0121] The video decoding device parses intra_luma_ref_idx to determine the reference line index for prediction. The Intra Sub-Partition (ISP) technique is applied when the reference line index is 0, so if the reference line index is non-zero, the information related to ISP is not parsed. Additionally, when the prediction mode determined by MPM is not the planar mode, MRL is applied. Therefore, since a non-zero reference line index indicates the application of MRL, both intra_luma_mpm_flag and intra_luma_not_planar_flag are inferred to be 1.

[0122] However, the problem suffered by the existing MRL techniques is that all blocks receive the same application of the MRL candidate list, which lists the reference lines available for MRL. In other words, the existing MRL techniques fixedly use the same reference lines.

[0123] Since different blocks may have different reference lines that can generate better prediction values, constructing the MRL candidate list block by block by using the reference lines that can be selected with a higher probability would be feasible. Therefore, applying the same MRL candidate list to all blocks will not only degrade the prediction performance but also lead to low efficiency in MRL information transmission. The existing MRL techniques are inefficient in using a fixed-form MRL candidate list without adaptively generating the MRL candidate list by considering information about the current block, information about adjacent blocks, etc.

[0124] Hereinafter, the predictor and the prediction block may be used interchangeably.

[0125] The following embodiments are described with respect to a video decoding apparatus, but they may also be implemented by a video encoding apparatus in the same or similar manner.

[0126] II. Embodiments According to the Present Invention

[0127] By adaptively constructing a multi-reference line (MRL) candidate list on a per-block basis, the above problems of the prior art can be solved. Accordingly, embodiments according to the present invention can be applied to improve video codec efficiency and / or enhance video quality and picture sharpness. The video decoding apparatus uses information about the block and signaled information to determine the length of the MRL candidate list and the MRL candidate list filling method (or the filling method for the MRL candidate list). For example, with respect to N (N ≥ 1) reference lines that can be referred to for the MRL, the video decoding apparatus uses all or some K (K ≤ N) reference lines to adaptively construct the MRL candidate list for the current block. The video decoding apparatus signals an MRL index mrl_idx to indicate a reference line to be used for prediction among the reference lines included in the adaptively determined MRL candidate list for each block. mrl_idx indicates the position of the reference line in the MRL candidate list, i.e., the n th th point of the reference line in the list. In this case, the index values of the N reference lines that can be referred to are in the range of 0 to N - 1.

[0128] The definitions of intra_luma_ref_idx and mrl_idx used in the present invention are as follows.

[0129] The reference line index intra_luma_ref_idx is a value indicating the distance from the current block to the reference line to be indicated. intra_luma_ref_idx indicates the position of the reference line and can have a value greater than or equal to zero. For example, intra_luma_ref_idx can be expressed as the number of pixels, the number of blocks, etc. Hereinafter, intra_luma_ref_idx indicates the number of pixels.

[0130] The MRL index (mrl_idx) indicates the position of the reference line to be used for prediction within the MRL candidate list. mrl_idx can have a value greater than or equal to 0.

[0131] The MRL candidate list and the list may be used interchangeably. Additionally, the terms MRL candidate list filling method and filling method may be used interchangeably.

[0132] In order to adaptively construct the MRL candidate list, the video decoding device determines the length of the MRL candidate list and the MRL candidate list filling method. The video decoding device can adaptively construct the MRL candidate list by utilizing the MRL candidate list filling method (Implementation 1) without predetermining the length of the list, such as Figure 8 Alternatively, it may be possible to adaptively construct an MRL candidate list by utilizing (Implementation 2) an MRL candidate list filling method based on a predetermined length of the list, as shown in Figure 9 In this case, the length of the list indicates the number of reference lines included in the MRL candidate list. A preferred embodiment for solving the above problem is described below.

[0133] To indicate whether each of the embodiments described below is applied, the video encoding device may signal sps_adaptive_mrl_candidate_list_enabled_flag and pps_adaptive_mrl_candidate_list_enabled_flag to the video decoding device at a higher level such as a sequence parameter set (SPS) or a picture parameter set (PPS). Although the prior art MRL technology involves three reference lines in VVC and six reference lines in ECM, the present invention may be configured to involve more than three reference lines (e.g., N lines).

[0134] <Implementation 1> A method for filling an MRL candidate list without pre-determining the length of the list.

[0135] Figure 10 is a schematic diagram illustrating a method for populating an MRL candidate list according to another embodiment of the present invention.

[0136] In this embodiment, the video decoding device determines the MRL candidate list filling method without predetermining the length of the list, and then constructs the MRL candidate list of the current block according to the determined filling method. Figure 10As shown in the example of , several (i.e., a, b, …) non - repeating reference lines for each selection according to methods A, B, etc. can be added to the list. By adding or inserting the relevant reference lines into the list in a predetermined order, the video decoding device can construct an MRL candidate list. The length of the MRL candidate list, i.e., the number of reference lines in the list, can be determined by adding all possible additional reference lines to the list by using the MRL candidate list filling method determined in this embodiment. For example, if the list filling methods are determined to be A and B, the length of the list can be determined as "a + b" by adding all the reference lines determined by these two methods to the list, i.e., adding the "a" reference line and the "b" reference line.

[0137] The video decoding device parses mrl_idx into information about the reference lines of the current block. The video decoding device determines the MRL candidate list filling method according to the method of this embodiment and constructs the MRL candidate list by using the determined filling method. The video decoding device can then use the reference line indicated by mrl_idx in the MRL candidate list to perform intra - prediction on the current block.

[0138] In this embodiment, the MRL candidate list filling method can be determined by considering one or more information items on the block. The information items on the block are as follows. What can be used as the distance between the block and the reference line can be the index value of the reference line, the number of pixels between the block and the reference line, the number of blocks between the block and the reference line, etc.

[0139] What can be used as information about the block can be the characteristics of the current block, such as its position, prediction mode, reference pixels, any prediction factors that can be generated, the distance between the available reference lines and the current block, the pixel values of the available reference lines, width (W), height (H), area, aspect ratio (W, H, log2W, log2H, log2WH, WH, log2(W / H), W / H, log2(H / W), H / W), etc.

[0140] What can be used as information about the block can be the characteristics of the blocks adjacent to the current block in the current frame, such as the positions of the adjacent blocks, the pixel values generated by reconstructing the blocks, prediction modes, the reference lines used, whether MRL is enabled, the MRL candidate list, reference pixels, any prediction factors that can be generated, the distance between the available reference lines and the current block, the pixel values of the available reference lines, width (W), height (H), area, aspect ratio (W, H, log2W, log2H, log2WH, WH, log2(W / H), W / H, log2(H / W), H / W), etc.

[0141] The information used as information about a block may be features of the block, where the block includes co-located blocks of the current block and adjacent blocks of the co-located blocks in other pictures that can be referenced. Here, the features include the positions of the co-located blocks and adjacent blocks, pixel values generated by reconstructing the block, prediction modes, reference lines used, whether MRL is enabled, the MRL candidate list, reference pixels, any predictors that can be generated, the distances between available reference lines and the current block, pixel values of the available reference lines, width (W), height (H), area, aspect ratio (W, H, log2W, log2H, log2WH, WH, log2(W / H), W / H, log2(H / W), H / W), etc.

[0142] The information used as information about a block may be features of a block reconstructed earlier than the current block, such as the position of the reconstructed block, pixel values generated by reconstructing the block, prediction modes, reference lines used, whether MRL is enabled, the MRL candidate list, reference pixels, any predictors that can be generated, the distances between available reference lines and the current block, pixel values of the available reference lines, width (W), height (H), area, aspect ratio (W, H, log2W, log2H, log2WH, WH, log2(W / H), W / H, log2(H / W), H / W), etc.

[0143] Examples of reference lines used to populate the MRL candidate list by considering one or more of the above information about the block and examples of the use of the above reference lines are as described in Methods A to D below. Additionally, any reference lines available for populating the MRL candidate list in the current block can be used to construct the list. In addition to the factors considered during the filling process, each MRL candidate list filling method also includes considering the order of multiple reference lines and the number of reference lines to be added to the list. If each method cannot fill the list with the number of reference lines it has determined, such as if Method A is supposed to fill the list with three reference lines but fewer reference lines are to be added, the video decoding device can stop adding reference lines by using the relevant method, or it can fill the list with a predefined value until the determined number of reference lines is reached.

[0144] Method A: Use the reference lines of adjacent blocks of the current block within the current frame

[0145] In this method, the video decoding device fills the MRL candidate list of the current block with the reference lines of adjacent blocks of the current block. The reference lines of the adjacent blocks can be added to the MRL candidate list in a predetermined order. When a predetermined number of different reference lines have been added to the MRL candidate list, the video decoding device stops adding reference lines according to this method. In this case, the order of adding reference lines to the list and the number of reference lines to be added can be determined as preset values according to the relevant MRL candidate list filling method, or they can be determined by referring to the information about the block.

[0146] Figure 11 It is a schematic diagram showing reference lines of adjacent blocks of the current block according to at least one embodiment of the present invention.

[0147] An example case uses up to two reference lines to fill the MRL candidate list. First, the reference lines of adjacent blocks are added to the MRL candidate list with the reference line having a larger size, and the reference lines are added to the list by utilizing a preset order based on the block positions of blocks having the same size. In Figure 11 the example of, the video decoding device can first add intra_luma_ref_idx 1 of block 1 to the list, and then add intra_luma_ref_idx 3 of block 4 to the list. Since the number of reference lines for filling the list is reached according to this method, the video decoding device does not add the reference lines of other adjacent blocks to the MRL candidate list.

[0148] Another example case uses up to two reference lines to fill the MRL candidate list, and first adds the reference lines of adjacent blocks to the MRL candidate list by using the more frequently used reference lines. In Figure 11 the example of, the video decoding device first adds intraluma_rf_idx 0, which is the only reference line used twice as an adjacent block, to the list. Since all other reference lines are used once, the video decoding device can additionally fill the list with intra_luma_ref_idx 1, which is the reference line of block 1, considering the block number according to the preset position. Since the number of reference lines for filling the list is reached according to this method, the video decoding device does not add the reference lines of other adjacent blocks to the MRL candidate list.

[0149] Method B: Using reference lines according to a predetermined rule based on information about the current block

[0150] In this method, the video decoding device uses reference lines to fill the MRL candidate list of the current block according to a predetermined rule based on information about the current block. The information about the current block includes the characteristics of the current block among the above-mentioned information about the block. When a predetermined number of different reference lines have been added to the MRL candidate list, the video decoding device stops adding reference lines according to this method. In this case, the order of adding reference lines to the list and the number of reference lines to be added can be determined as preset values according to the relevant MRL candidate list filling method, or they can be determined by referring to the information about the block.

[0151] In one example, the video decoding device may refer to the width (W) of the current block as information about the block to determine log2W as the number of reference lines to be added to the list, and first add the reference lines with reference line indices of W-1 or less to the list having reference lines farther from the current block. For example, when the width of the current block is 8, the video decoding device may add three reference lines to the list in the following order: intra_luma_ref_idx 7, intra_luma_ref_idx 6, and intra_luma_ref_idx 5, thereby populating the MRL candidate list.

[0152] As another example, the video decoding device may populate the MRL candidate list by referring to the distance between the current block and the available reference lines as information about the block. When the number N of reference lines available for the current block is 8, the video decoding device may first add four reference lines to the MRL candidate list having reference lines closer to the current block. Accordingly, the video decoding device may populate the MRL candidate list with reference lines in the following order: intra_luma_ref_idx 0, intra_luma_ref_idx 1, intra_luma_ref_idx 2, and intra_luma_ref_idx 3.

[0153] As yet another example, the video decoding device may populate the MRL candidate list by referring to the prediction factors based on the reference lines available for the current block as information about the block. The video decoding device may add reference lines to the list in the order of generation of prediction factors different from those generated by intra_luma_ref_idx0. In this case, the preset value 2 may be determined as the number of reference lines to be added. The number N of reference lines available for the current block in the example case is 6, and the difference between the prediction factors is calculated based on the sum of absolute differences (SAD). The video decoding device may compare the SAD between the prediction factor generated by intra_luma_ref_idx 0 and the corresponding one of the prediction factors generated by intra_luma_ref_idx1-5, and first add the reference lines to the MRL candidate list having the reference lines causing the larger difference. In this case, metrics such as SAD, sum of absolute transformed differences (SATD), mean squared error (MSE), and mean absolute error (MAE) may be used as the difference between the prediction factors.

[0154] In addition, the video decoding device may refer to the pixel values of the reference lines instead of the prediction values as information about the block, in order to fill the MRL candidate list using a method based on the differences between all or some of the pixel values on each reference line.

[0155] Figure 12 is a schematic diagram showing the position of a current block in a coding tree unit (CTU) according to at least one embodiment of the present invention.

[0156] As another example, the video decoding device may refer to the position of the current block as information about the block. The video decoding device may add the reference lines of the current block within the current CTU to the MRL candidate list. The following describes the case where all reference lines are first added to the list with the reference lines closer to the current block. In Figure 12 the example of, since there are three reference lines of block 1 currently within the CTU, the video decoding device may construct the MRL candidate list for block 1 as {intra_luma_ref_idx 0, intra_luma_ref_idx 1, intra_luma_ref_idx 2}. In addition, since there are eight reference lines of block 2 currently within the CTU, the video decoding device may construct the MRL candidate list for block 2 as {intra_luma_ref_idx 0,

[0157]

[0158] intra_luma_ref_idx 7}.

[0159] Method C: Using the reference lines of the blocks reconstructed earlier than the current block

[0160] In this method, the video decoding device fills the MRL candidate list of the current block with the reference lines of the blocks reconstructed earlier than the current block. The reference lines of the reconstructed blocks may be added to the MRL candidate list in a predetermined order. When a predetermined number of different reference lines have been added to the MRL candidate list, the video decoding device stops adding reference lines according to this method. In this case, the order of adding the reference lines to the list and the number of reference lines to be added may be determined as preset values according to the relevant MRL candidate list filling method, or they may be determined by referring to the information about the block.

[0161] Figure 13 is a schematic diagram showing the reference lines of the previously reconstructed blocks according to at least one embodiment of the present invention.

[0162] In one example, the reference lines of the more recently reconstructed blocks may be first added to the MRL candidate list, and the number of further different reference lines to be added may be determined as 3. InFigure 13 In the example of Figure 13 , the video decoding device can add the reference lines of more recently reconstructed blocks to the list in the following non - repeating order: intra_luma_ref_idx 1, intra_luma_ref_idx 0, and intra_luma_ref_idx 3, to fill the MRL candidate list. Since the number of reference lines for filling the list is reached according to this method, the video decoding device does not add reference lines from other reconstructed blocks to the MRL candidate list.

[0163] Method D: Using the reference lines of a block, the block includes the co - located block of the current block and the adjacent blocks of the co - located block in other pictures that can be referenced

[0164] In this method, the video decoding device fills the MRL candidate list of the current block with the reference lines of a block, the block includes the co - located block of the current block and the adjacent blocks of the co - located block in other pictures that can be referenced. The reference lines of the co - located block and the adjacent blocks can be added to the MRL candidate list in a predetermined order. When a predetermined number of different reference lines have been added to the MRL candidate list, the video decoding device stops adding reference lines according to this method. Additionally, if there are multiple other pictures that can be referenced, the video decoding device can use all or some of the multiple pictures, and can determine which picture to use according to a predetermined method. In this case, the order of adding reference lines to the list, the number of reference lines to be added, and the reference images to be used can be determined as preset values according to the relevant MRL candidate list filling method, or they can be determined by referring to information about the block and the distance between pictures.

[0165] In one example, the video decoding device can add up to three reference lines from the reference image that is furthest in time from the current picture to the list. When filling the list, the video decoding device can first add the reference lines of the co - located block of the current block, and then consider the adjacent blocks of the co - located block that have the same aspect ratio as the current block. According to the position of the "adjacent blocks that have the same aspect ratio as the current block", the video decoding device can add the reference lines in a preset order. Then, according to the position of the "adjacent blocks that have a different aspect ratio from the current block", the video decoding device can add the reference lines in a preset order.

[0166] Figure 14a and Figure 14b are schematic diagrams showing the co - located blocks in the reference image according to at least one embodiment of the present invention.

[0167] In Figure 14a and Figure 14bIn the example, the video decoding device first adds intra_luma_ref_idx 5 to the list, where intra_luma_ref_idx 5 is the reference line of co-located block 1 in a reference picture that is temporally further from the current picture. The video decoding device then adds the reference lines of adjacent blocks 2 and 3 of block 1 that have the same aspect ratio as the current block. In this case, since these two blocks have the same reference line, intra_luma_ref_idx 0 is added to the list. According to the order of adjacent blocks of block 1 that have a different aspect ratio from the current block, the video decoding device adds the reference line intra_luma_ref_idx 1 of block 1 to the list. The video decoding device does not add reference lines from other blocks to the MRL candidate list because the number of reference lines for filling the list is reached according to this method.

[0168] In this embodiment, it is possible (Embodiment 1-1) to signal the video decoding device the MRL candidate list filling method, or it is possible (Embodiment 1-2) to infer the MRL candidate list filling method based on information about the blocks.

[0169] <Embodiment 1-1> Signaling the MRL candidate list filling method

[0170] In this embodiment, the video decoding device parses the MRL candidate list filling method and constructs the MRL candidate list according to the parsed filling method. At this time, one or more of the methods A to D and other methods as described above can be signaled.

[0171] When filling the MRL candidate list according to a filling method, a method lookup table including available filling methods is formed, and one filling method from the method lookup table is signaled as mrl_candidate_list_lines_select_method. Both the video decoding device and the video encoding device can classify the MRL candidate list filling method according to mrl_candidate_list_lines_select_method and can operate according to the classified method. For example, available MRL candidate list filling methods can be defined as shown in Table 4.

[0172] [Table 4]

[0173]

[0174]

[0175] When signaling mrl_candidate_list_lines_select_method 0 according to Table 4, the video decoding device may operate according to Method A to construct an MRL candidate list by using reference lines of neighboring blocks of a current block within a current frame.

[0176] When filling an MRL candidate list according to multiple filling methods, a method lookup table including available filling methods is constructed, and the multiple filling methods based on the method lookup table are signaled as mrl_candidate_list_lines_select_method. Both the video decoding device and the video encoding device may classify MRL candidate list filling methods according to mrl_candidate_list_lines_select_method and may operate according to the classified methods. When classifying available MRL candidate list filling methods by index, mrl_candidate_list_lines_select_method may indicate a list of multiple indexes, or it may indicate one of multiple index groups. For example, MRL candidate list filling methods may be classified by index as shown in Table 5.

[0177] [Table 5]

[0178] If two filling methods for constructing the list according to Table 5 are those indicated by index 0 and index 1, mrl_candidate_list_lines_select_method may be signaled in the form of the two listed indexes, such as "0 1" or "1 0". Alternatively, if multiple index groups exist as {0, 1}, {0, 2}, {1, 2} and the corresponding groups are indicated by mrl_candidate_list_lines_select_method 0, 1, 2, then mrl_candidate_list_lines_select_method may be signaled as 0.

[0179] The video decoding device may determine the order of use of multiple filling methods for filling an MRL candidate list based on the value of mrl_candidate_list_lines_select_method or based on information about a block, or may set the order of use of multiple filling methods to a preset order.

[0180] The first case is to determine the order of use based on the value of mrl_candidate_list_lines_select_method. If the value of mrl_candidate_list_lines_select_method is signaled in the form of multiple listed indexes, the video decoding device may consider the MRL candidate list filling methods indicated by the relevant indexes in the order of the listed indexes. If mrl_candidate_list_lines_select_method indicating one of multiple index groups is signaled, the video decoding device may consider the MRL candidate list filling method based on the order of the indexes within the relevant group. For example, if the listed indexes or index group is signaled as "0 1", the video decoding device may first consider the method indicated by index 0, and then consider the method indicated by index 1.

[0181] The second case is to determine the order of use based on the information about the block. The video decoding device determines the order of use of multiple filling methods by referring to the information about the block and the information related to the MRL candidate list filling method determined according to the information about the block. For example, when using methods A and B indicated by index 0 and index 1 when receiving the signal "0 1", the video decoding device may first use the filling method with a smaller change in the reference lines used by the reference block to fill the MRL candidate list. The following assumes that there is an adjacent block of the current block in the current frame, as Figure 11 shown, and it is assumed that the reference lines of the current block falling within the current CTU are added to the MRL candidate list, as Figure 12 shown. In Figure 11 the example of, according to method A, there are four types of reference lines used by the reference block for list construction, which are intra_luma_ref_idx 0, intra_luma_ref_idx 1, intra_luma_ref_idx 2, and intra_luma_ref_idx3. Additionally, in Figure 12 the example of, when the current block is block 1, according to method B, there are three types of reference lines used by the reference block for list construction, which are intra_luma_ref_idx 0, intra_luma_ref_idx 1, and intra_luma_ref_idx 2. Therefore, when filling the MRL candidate list, the method B indicated by index 1 may be considered first.

[0182] The third case is that the usage order is set to a preset order. The usage order of all available MRL candidate list filling methods can be set at a higher level such as SPS, PPS, etc. Alternatively, the usage order can always be a fixed order without separate setting. In the preset order or the fixed order, the video decoding device can consider the MRL candidate list filling method. The preset order or the fixed order can be equally applied to all or some CUs. In Table 5, three methods indicated by indices 0, 1, and 2 can be available, and in this case, the preset order can be set to order 2, 1, and 0, for example.

[0183] The syntax elements required according to this embodiment are as follows.

[0184] mrl_candidate_list_lines_select_method is one or more values indicating available MRL candidate list filling methods. mrl_candidate_list_lines_select_method can have a single value of 0 or greater, or multiple values of 0 or greater.

[0185] The MRL index (mrl_idx) is a value indicating the position of the reference line to be used for prediction within the MRL candidate list. mrl_idx can have a value of 0 or greater.

[0186] The specific pseudocode according to this embodiment can be implemented as follows. In this case, the video decoding device can first parse any one of the intra prediction mode, the MRL candidate list filling method, and the MRL index.

[0187]

[0188] On the other hand, the video encoding device can obtain the intra prediction mode, the MRL candidate list filling method, and the MRL index from a higher level such as SPS, PPS, etc. From the perspective of rate-distortion optimization, the higher level of the video encoding device can determine the intra prediction mode, the MRL candidate list filling method, and the MRL index.

[0189] According to the above pseudocode, the syntax required for transmission is shown in Table 6.

[0190] [Table 6]

[0191]

[0192] In Table 6, the video decoding device parses the syntax elements in the order of the MRL candidate list filling method, the reference line for prediction, and the intra prediction mode.

[0193] On the other hand, to allow the mrl_candidate_list_lines_select_method to indicate previously unused and new MRL candidate list filling methods, new methods can be added as available MRL candidate list filling methods. The new methods can be added at both the block level and a higher level (such as SPS and PPS). When appropriate syntax (e.g., an index) can identify one or more new MRL candidate list filling methods previously unused by the video decoding device and the video encoding device, one or more new MRL candidate list filling methods can be further determined by the signal of mrl_candidate_list_lines_select_method_register. Each new MRL candidate list filling method can be added to a predefined position in the method lookup table, i.e., at one of the first, second, …, and last points in the method lookup table. Alternatively, the new MRL candidate list filling method can be added to a position in the method lookup table signaled by mrl_candidate_list_lines_select_method_register_pos.

[0194] <Embodiment 1-2> Inferring MRL Candidate List Filling Method

[0195] In this embodiment, the video decoding device infers an MRL candidate list filling method and then constructs an MRL candidate list according to the inferred filling method. At this time, one or more of the methods A to D and other methods as described above can be inferred. To infer the MRL candidate list filling method, the video decoding device can (Embodiment 1-2-1) determine the MRL candidate list filling method based on information about the block, or (Embodiment 1-2-2) set the MRL candidate list filling method to a preset filling method.

[0196] <Embodiment 1-2-1> Determining MRL Candidate List Filling Method Based on Information about Block

[0197] In this embodiment, the video decoding device determines the MRL candidate list filling method based on information about the block and then constructs an MRL candidate list according to the determined filling method. At this time, one or more of the methods A to D and other methods as described above can be inferred.

[0198] In this embodiment, when inferring the MRL candidate list filling method, the video decoding device may consider one or more of the information about the block. As the available information about the block, the information items about the block described in Embodiment 1 may be used. In this case, the distance used as the distance between the block and the reference line may be the index value of the reference line, the number of pixels between the block and the reference line, the number of blocks between the block and the reference line, etc. In addition, the information about the MRL candidate list may be used, which includes any information about the MRL candidate list construction, such as the list filling method used, the length of the list, etc.

[0199] Examples of inferring the MRL candidate list filling method by considering one or more of the information about the block are as follows. By considering the reference lines of the adjacent blocks of the current block within the current frame, two or more adjacent blocks may use the same reference line, and in this case, the video decoding device may fill the list according to Method A described in Embodiment 1. If two blocks use intra_luma_ref_idx 0, then as Figure 11 shown in the example of, the video decoding device may construct the MRL candidate list according to Method A. In addition, the video decoding device may use the relevant MRL candidate list filling method to add up to two additional reference lines to the MRL candidate list. In this case, the video decoding device may fill the list with the reference lines of the adjacent blocks in order starting from the most frequently used reference line. If the reference lines have the same usage frequency, the video decoding device may add the reference lines to the list in order based on the position of the block. For Figure 11 the example in, the video decoding device first adds intra_luma_ref_idx 0 to the list because it is the only reference line used twice by the adjacent blocks. Since all other reference lines are used once, the video decoding device may consider the block number and then add the reference line intra_luma_ref_idx 1 of block 1 to the list. Since the number of reference lines for filling the list is reached according to this method, the video decoding device does not add the reference lines of other adjacent blocks to the MRL candidate list.

[0200] If multiple filling methods are inferred for constructing the MRL candidate list, the video decoding device may fill the list by using the multiple inferred filling methods. When doing so, it is necessary to further determine the order of considering the multiple filling methods. The video decoding device may determine the usage order of the multiple filling methods based on the information about the block. Alternatively, if there is a method lookup table that classifies the multiple filling methods by index, the video decoding device may determine the usage order in ascending / descending / random order based on the value of the index.

[0201] The first case is to determine the usage order based on information about blocks. The video decoding device determines the usage order of multiple filling methods by referring to information about blocks and information related to the inferred MRL candidate list filling method. An example assumes that in order to construct the MRL candidate list, method A is used to add two reference lines to the list, where the more frequently used reference line is added first, or if the usage frequencies are equal, they are added in order based on the position of the block, or method C is used to add three reference lines to the list, where the reference line of the more recently reconstructed block is added first. The usage order of the two methods can be determined by the smaller number of reference lines added by each method. That is, two reference lines are added by method A and three reference lines are added by method C, so the video decoding device can first use method A to fill the list. Alternatively, the greater the similarity between the reference lines of the blocks considered in each method, the higher the priority can be given to that method. An example assumes that the adjacent blocks of the current block used in method A all have different reference lines, and the reference lines of the previously reconstructed blocks used in method C are {1, 1, 1, 2, 1, 0, 3, 3,...} when listed in the order of the most recently reconstructed. Since the reference lines of the blocks identified in method C have greater similarity, the video decoding device can fill the list by first using method C.

[0202] The second case is to determine the usage order through an index that classifies multiple filling methods. When multiple MRL candidate list filling methods are inferred and each filling method is classified by an index, the video decoding device can consider using each MRL candidate list filling method according to the index. In this case, the order (ascending / descending / random order of the index) considering all available MRL candidate list filling methods classified by the index can be set at a higher level such as in the SPS, PPS, etc. Alternatively, the usage order can always be a fixed order without separate setting. In the preset order and the fixed order, the video decoding device can consider the MRL candidate list filling method. The preset order or the fixed order can be equally applied to all or some CUs. In Table 5, three methods indicated by indices 0, 1, and 2 can be available, and in this case, the preset order can be set to order 2, 1, and 0, for example.

[0203] <Embodiment 1-2-2> Set the MRL candidate list filling method to the preset filling method

[0204] In this embodiment, the video decoding device sets an MRL candidate list filling method to construct an MRL candidate list according to a preset filling method. At this time, one or more of the above-described Method A to Method D and other methods can be set. The MRL candidate list filling method can be set at a higher level such as SPS, PPS, etc. Alternatively, the MRL candidate list filling method can be a constant fixed method without separate setting. The preset or fixed method can be equally applied to all or some CUs.

[0205] When multiple filling methods are preset for constructing the MRL candidate list, it is necessary to further determine the order of considering the preset multiple filling methods. The video decoding device can determine the order of use of the multiple filling methods based on information about the block. Alternatively, if there is a method lookup table that classifies the multiple filling methods by index, the video decoding device can determine the order of use in ascending / descending / random order according to the value of the index. The order of considering the multiple filling methods according to this embodiment depends on Embodiment 1-2-1, and thus further description is omitted.

[0206] <Embodiment 2> MRL Candidate List Filling Method Based on a Predetermined Length of the List

[0207] In this embodiment, the video decoding device constructs an MRL candidate list for the current block according to the predetermined length of the MRL candidate list, that is, the number of reference lines included in the MRL candidate list. As Figure 9 shown, the video decoding device sequentially performs "determining the length of the MRL candidate list" and "determining the MRL candidate list filling method", and fills the list with the determined length with reference lines. Since the MRL candidate list filling method can be implemented in the same way as in Embodiment 1, the following embodiment describes the determination of the length of the MRL candidate list.

[0208] In this embodiment, the length of the MRL candidate list can be signaled to the video decoding device (Embodiment 2-1), or the length of the MRL candidate list can be inferred based on information about the block (Embodiment 2-2). If the method of Embodiment 1 is not sufficient to fill the list to the length determined in this embodiment, the video decoding device can use a predetermined method to add reference lines that are not duplicates of the already included reference lines to the list. Among other methods that can be implemented according to Embodiment 1, the predetermined method can include an unselected MRL candidate list filling method, a method of filling the list with predefined reference lines, etc.

[0209] The video decoding device resolves mrl_idx into information on a reference line for a current block. When performing intra prediction on the current block, the video decoding device determines a list length and a list filling method according to the present invention, and then constructs an MRL candidate list by using the determined list length and list filling method. The video decoding device may infer the reference line indicated by mrl_idx from the MRL candidate list, and use the derived reference line to perform intra prediction on the current block.

[0210] <Embodiment 2-1> Signaling the length of the MRL candidate list

[0211] In this embodiment, the video decoding device resolves the length of the MRL candidate list. The video encoding device signals mrl_candidate_list_len to the video decoding device, which indicates the length of the MRL candidate list. In some embodiments, mrl_candidate_list_len may directly represent a length value (x), or may represent a result value after applying a predetermined operation (f(x)) to the length value (x). Alternatively, when a length look-up table containing available values for the length of the MRL candidate list is provided, mrl_candidate_list_len may be an index indicating one of the values included in the length look-up table. Each definition for mrl_candidate_list_len is described below.

[0212] First, mrl_candidate_list_len only indicates the length value of the MRL candidate list. For example, if the length of the MRL candidate list is determined to be 6, that is, there are six reference lines in the MRL candidate list, then mrl_candidate_list_len may be signaled as 6.

[0213] Second, mrl_candidate_list_len indicates the result value after applying a predetermined operation (f(x)) to the length value (x) of the MRL candidate list. For example, if the length of the MRL candidate list is determined to be 6, that is, there are 6 reference lines in the MRL candidate list, then mrl_candidate_list_len may be signaled as 2 by applying the operation “f(x)=x / 3”.

[0214] Third, mrl_candidate_list_len may be an index indicating one of the values in the above-mentioned length look-up table. By using Table 7, an example describes the indication of one of the values included in the length look-up table.

[0215] [Table 7]

[0216] When signaling mrl_candidate_list_len as 2 according to Table 7, the video decoding device may determine the length of the MRL candidate list as 6.

[0217] The syntax elements required according to this embodiment are as follows.

[0218] mrl_candidate_list_len indicates the length of the MRL candidate list. In some embodiments, mrl_candidate_list_len may be a length value, a result value after applying a predetermined operation to the length value, an index indicating one of the available values in a length lookup table, etc.

[0219] The MRL index mrl_idx is a value indicating the position of the reference line to be used for prediction within the MRL candidate list. mrl_idx may have a value greater than or equal to 0.

[0220] The specific pseudocode according to this embodiment can be implemented as follows. The video decoding device may first parse any of the following: the intra prediction mode, the length of the MRL candidate list, the MRL candidate list filling method, and the MRL index.

[0221]

[0222] On the other hand, the video encoding device may obtain the intra prediction mode, the length of the MRL candidate list, the MRL candidate list filling method, and the MRL index from a higher level such as the SPS, PPS, etc. From the perspective of rate-distortion optimization, the higher level of the video encoding device may determine the intra prediction mode, the length of the MRL candidate list, the MRL candidate list filling method, and the MRL index.

[0223] According to the above pseudocode, the syntax required for transmission is shown in Table 8.

[0224] [Table 8]

[0225]

[0226]

[0227] In Table 8, the video decoding device parses the syntax elements in the order of the length of the MRL candidate list, the MRL candidate list filling method, the reference line for prediction, and the intra prediction mode.

[0228] When looking up a value in the length lookup table using an index to indicate the length, new lengths that do not exist in the existing length lookup table can be added to the length lookup table to allow mrl_candidate_list_len to indicate the new lengths. The new lengths can be added at both the block level and a higher level (such as SPS and PPS). The length value being added can be signaled using mrl_candidate_list_len_register. Each new length can be added to a predefined position in the length lookup table, i.e., at one of the first, second, …, or last points in the length lookup table. Alternatively, the new length can be added to the position signaled by mrl_candidate_list_len_register_pos.

[0229] <Embodiment 2-2> Inferring the Length of the MRL Candidate List

[0230] In this embodiment, the video decoding device infers the length of the MRL candidate list. To infer the length of the MRL candidate list, the video decoding device can (Embodiment 2-2-1) determine the length of the MRL candidate list based on information about the block, or (Embodiment 2-2-2) set the length of the MRL candidate list to a preset value.

[0231] <Embodiment 2-2-1> Determining the Length of the MRL Candidate List Based on Information about the Block

[0232] In this embodiment, the video decoding device determines the length of the MRL candidate list based on information about the block.

[0233] In this embodiment, when inferring the length of the MRL candidate list, the video decoding device can consider one or more of the information about the block. As the available information about the block, the information items about the block described in Embodiment 1 can be used. In this case, the distance between the block and the reference line can be the index value of the reference line, the number of pixels between the block and the reference line, the number of blocks between the block and the reference line, etc. Additionally, information about the MRL candidate list can be used, which includes the used MRL candidate list, the length of the used MRL candidate list, etc.

[0234] Examples of determining the length of the MRL candidate list by referring to the information about the block are described below.

[0235] In one example, when referring to the width of the current block (log2WH) as the information about the block, the length of the MRL candidate list can be determined based on the width of the current block, as shown in Table 9.

[0236] [Table 9]

[0237] <![CDATA[Block size (log2WH)]]> Length of the MRL candidate list Block size < 8 3 Block size ≥ 8 6

[0238] As another example, when using the reference lines used by adjacent blocks as information about a block, the length of the MRL candidate list can be determined by the difference in the reference lines of adjacent blocks (i.e., the different number of types of reference lines used by adjacent blocks). As shown in the example of Figure 15 it is assumed that in the adjacent blocks including pixels 1 to 5 adjacent to the current block, blocks 1, 2, and 3 use intra_luma_ref_idx 0 for prediction, and blocks 4 and 5 use intra_luma_ref_idx 2 for prediction. In the example of Figure 15 since the adjacent blocks use two types of reference lines for prediction, the length of the MRL candidate list of the current block can be determined to be 2.

[0239] Another example involves the prediction mode of the current block, the pixel values of the corresponding reference lines, and the prediction factors of the current block generated using each reference line instead of information about the block. The video decoding device uses the available reference lines to generate the prediction factors of the current block, and then calculates the SAD between the generated prediction factors. If at least one of the value of SAD, the mean of SAD, the median of SAD, or the maximum value of SAD is less than a preset threshold, the video decoding device can set the length of the MRL candidate list to a small value. On the other hand, if all of the value of SAD, the mean of SAD, the median of SAD, the maximum value of SAD, etc. are greater than or equal to the preset threshold, the video decoding device can set the length of the MRL candidate list to a large value. For example, the length of the MRL candidate list can be determined according to the maximum value of SAD between the above prediction factors, as shown in Table 10. According to Table 10, if the maximum value of SAD is less than 100, the video decoding device can determine the length of the MRL candidate list to be 3. On the other hand, if the maximum value of SAD is greater than or equal to 100, the video decoding device can determine the length of the MRL candidate list to be 6.

[0240] [Table 10]

[0241] Maximum SAD between predictors Length of the MRL candidate list Maximum SAD < 100 3 Maximum SAD ≥ 100 6

[0242] <Embodiment 2-2-2> Set the length of the MRL candidate list to a preset value

[0243] In this embodiment, the video decoding device sets the length of the MRL candidate list to a preset value. The length of the MRL candidate list can be set at a higher level such as SPS, PPS, etc. Alternatively, the length of the MRL candidate list can always be a fixed value without separate setting. The preset or fixed value can be equally applied to all or some CUs.

[0244] <Embodiment 3> Selectively Utilizing the Prior Art and Embodiments 1 and 2

[0245] In this embodiment, in order to selectively apply the prior art and the above-described Embodiments 1 and 2, the video decoding apparatus may parse an additional signal. The video encoding apparatus may send adaptive_mrl_candidate_list_flag to indicate information about the reference line to be used for predicting the current block. For example, as shown in Table 11, if adaptive_mrl_candidate_list_flag is 0, the video decoding apparatus uses the conventional technique of a fixed MRL candidate list. On the other hand, if adaptive_mrl_candidate_list_flag is 1, the video decoding apparatus may generate an MRL candidate list according to Embodiment 1.

[0246] [Table 11]

[0247]

[0248] As another example, if adaptive_mrl_candidate_list_flag is 1, as shown in Table 12, the video decoding apparatus may further parse adaptive_mrl_candidate_list_idx and select one of Embodiment 1 and Embodiment 2 based on the parsed index. Then, the video decoding apparatus may generate an MRL candidate list based on the selected technique.

[0249] [Table 12]

[0250]

[0251] Hereinafter, with reference to Figure 16 and Figure 17 , a method for adaptively determining the length of the MRL candidate list and an MRL candidate list filling method will be described.

[0252] Figure 16 is a flowchart of a method for encoding a current block by a video encoding apparatus according to at least one embodiment of the present invention.

[0253] The video encoding apparatus determines an MRL index and an intra prediction mode of the current block (S1600). Here, the MRL index indicates the reference line for intra prediction of the current block within the MRL candidate list. From the perspective of rate distortion optimization, the video encoding apparatus may determine the intra prediction mode and the MRL index.

[0254] The video encoding device obtains the length of the MRL candidate list (S1602). Here, the length of the MRL candidate list indicates the number of reference lines included in the MRL candidate list.

[0255] In one example, the video encoding device may determine the length of the MRL candidate list from the perspective of rate-distortion optimization. The video encoding device encodes the determined length of the MRL candidate list.

[0256] As another example, the video encoding device may determine the length of the MRL candidate list based on information about the block, or may set the length of the MRL candidate list to a preset value.

[0257] The video encoding device obtains at least one or more filling methods for filling the MRL candidate list (S1604).

[0258] In one example, the video encoding device may determine at least one or more filling methods from the perspective of rate-distortion optimization. The video encoding device encodes an index indicating at least one or more filling methods determined among the filling methods included in a preset method lookup table.

[0259] As another example, the video encoding device may determine at least one or more filling methods based on information about the block, or may set at least one or more filling methods to preset filling methods.

[0260] The video encoding device generates the MRL candidate list by adding reference lines corresponding to the length of the MRL candidate list to the MRL candidate list by using at least one or more filling methods (S1606).

[0261] The video encoding device derives a reference line from the MRL candidate list by using the MRL index (S1608).

[0262] The video encoding device uses the reference line according to the intra prediction mode to generate a predicted block of the current block (S1610).

[0263] Then, the video encoding device may subtract the predicted block from the original block of the current block to generate a residual block, and encode the residual block.

[0264] Figure 17 is a flowchart of a method for reconstructing a current block by a video decoding device according to at least one embodiment of the present invention.

[0265] The video decoding device decodes the MRL index and the intra prediction mode of the current block from the bitstream (S1700). Here, the MRL index indicates the reference line to be used for intra prediction of the current block within the MRL candidate list.

[0266] The video decoding device obtains the length of the MRL candidate list (S1702). Here, the length of the MRL candidate list indicates the number of reference lines included in the MRL candidate list.

[0267] In one example, the video decoding device decodes the length of the MRL candidate list from the bitstream.

[0268] As another example, the video decoding device may determine the length of the MRL candidate list based on information about the block, or may set the length of the MRL candidate list to a preset value.

[0269] The video decoding device obtains at least one or more filling methods for filling the MRL candidate list (S1704).

[0270] In one example, the video decoding device decodes from the bitstream an index indicating at least one or more filling methods among the filling methods included in a preset method lookup table.

[0271] As another example, the video decoding device may determine at least one or more filling methods based on information about the block, or may set at least one or more filling methods to preset filling methods.

[0272] By using at least one or more filling methods, the video decoding device generates the MRL candidate list by adding reference lines corresponding to the length of the MRL candidate list to the MRL candidate list (S1706).

[0273] The video decoding device derives a reference line from the MRL candidate list by using the MRL index (S1708).

[0274] The video decoding device uses the reference line according to the intra prediction mode to generate a predicted block of the current block (S1710).

[0275] Then, the video decoding device may decode the residual block from the bitstream and sum the residual block and the predicted block to generate a reconstructed block of the current block.

[0276] Although the steps in the respective flowcharts are described as being executed sequentially, these steps merely illustrate the technical ideas of some embodiments of the present invention. Therefore, those of ordinary skill in the art to which the present invention pertains can execute the steps by changing the order described in the respective drawings or by executing two or more steps in parallel. Therefore, the steps in the respective flowcharts are not limited to the order shown in the chronological order.

[0277] It should be understood that the above description presents illustrative embodiments that can be implemented in various other ways. The functions described in some embodiments can be implemented by hardware, software, firmware, and / or combinations thereof. It should also be understood that the functional components described in the present invention are marked as "…… unit" to emphasize the possibility of their independent implementation.

[0278] On the other hand, the various methods or functions described in some embodiments can be implemented as instructions stored in a non-volatile recording medium, which can be read and executed by one or more processors. The non-volatile recording medium can include various types of recording devices that store data in a form readable by a computer system. For example, the non-volatile recording medium can include storage media such as erasable programmable read-only memory (EPROM), flash drives, optical disk drives, magnetic hard disk drives, and solid-state drives (SSD), etc.

[0279] Although the exemplary embodiments of the present invention have been described for illustrative purposes, those of ordinary skill in the art to which the present invention pertains should understand that various modifications, additions, and substitutions can be made without departing from the spirit and scope of the present invention. Therefore, the embodiments of the present invention have been described for the sake of simplicity and clarity. The scope of the technical idea of the embodiments of the present invention is not limited by the examples. Accordingly, those of ordinary skill in the art to which the present invention pertains should understand that the scope of the present invention should not be limited by the embodiments clearly described above, but by the claims and their equivalents.

[0280] Reference Numerals

[0281] 122: Intra Predictor

[0282] 155: Entropy Encoder

[0283] 510: Entropy Decoder

[0284] 542: Intra Predictor.

[0285] Cross - Reference to Related Applications

[0286] This application claims the priority and benefits of Korean Patent Application No. 10 - 2022 - 0179526, filed on December 20, 2022, and Korean Patent Application No. 10 - 2023 - 0159458, filed on November 16, 2023, the entire contents of each of which are incorporated herein by reference.

Claims

1. A method for reconstructing a current block by a video decoding device, the method comprising: Decoding an intra prediction mode and a multi-reference line (MRL) index of a current block from a bitstream, the multi-reference line index indicating, within an MRL candidate list, a reference line to be used for intra prediction of the current block; Obtaining a length of the MRL candidate list, the length of the MRL candidate list indicating a number of reference lines included in the MRL candidate list; Obtaining at least one or more padding methods; Generating an MRL candidate list by adding, using at least one or more padding methods, reference lines corresponding to the length of the MRL candidate list to the MRL candidate list; Deriving a reference line from the MRL candidate list by using the MRL index; And Generating a prediction block of the current block according to the intra prediction mode by using the reference line.

2. The method according to claim 1, wherein Obtaining at least one or more padding methods includes: Decoding, from the bitstream, an index of at least one or more padding methods indicating padding methods included in a predefined method lookup table.

3. The method according to claim 2, wherein: When the at least one or more padding methods are multiple padding methods, generating the MRL candidate list includes: Determining an order of use of the multiple padding methods based on a value of the index, or determining the order of use based on information about the block, or setting the order of use to a preset order, Wherein the information about the block includes: Features of the current block, features of blocks adjacent to the current block in the current frame, features of blocks reconstructed earlier than the current block, or features of blocks including the current block and adjacent blocks of the current block in other pictures that can be referenced.

4. The method according to claim 1, wherein Obtaining at least one or more padding methods includes: Determining at least one or more padding methods based on information about the block, or setting the at least one or more padding methods to preset methods, Wherein the information about the block includes: Features of the current block, features of blocks adjacent to the current block in the current frame, features of blocks reconstructed earlier than the current block, or features of blocks including the current block and adjacent blocks of the current block in other pictures that can be referenced.

5. The method according to claim 4, wherein, When the at least one or more padding methods are multiple padding methods, generating the MRL candidate list includes: Determining an order of use of the multiple padding methods based on information about the block, or setting the order of use to a preset order.

6. The method according to claim 1, wherein, Each of the at least one or more padding methods includes: A method of using reference lines of adjacent blocks of the current block in the current frame, a method of using reference lines according to a predetermined rule based on features of the current block, a method of using reference lines of blocks reconstructed earlier than the current block, or a method of using reference lines of blocks including the current block and adjacent blocks of the current block in other pictures that can be referenced.

7. The method according to claim 1, wherein Obtaining the length of the MRL candidate list includes: Decoding the length of the MRL candidate list from the bitstream.

8. The method according to claim 7, wherein, Decoding the length of the MRL candidate list includes: Decoding an unchanged value of the length of the MRL candidate list, decoding a value obtained by applying a predetermined function to the length of the MRL candidate list, or decoding an index indicating one of values included in a predefined length lookup table for the length of the MRL candidate list.

9. The method according to claim 1, wherein Obtaining the length of the MRL candidate list includes: Determine the length of the MRL candidate list based on information about the block, or set the length of the MRL candidate list to a preset value, The information about the block includes: Features of the current block, features of blocks adjacent to the current block in the current frame, features of blocks reconstructed earlier than the current block, or features of blocks including the co-located block of the current block and blocks adjacent to the co-located block in other referenced pictures.

10. A method for encoding a current block by a video encoding apparatus, the method comprising: determining an intra prediction mode for a current block and a multiple reference line (MRL) index indicating a reference line to be used for intra prediction of the current block within an MRL candidate list; obtaining a length of the MRL candidate list, wherein the length of the MRL candidate list indicates the number of reference lines included in the MRL candidate list; obtaining at least one or more filling methods; generating the MRL candidate list by adding a reference line corresponding to a length of the MRL candidate list to the MRL candidate list using at least one or more padding methods; Derivation of reference lines from the MRL candidate list by utilizing the MRL index; as well as By using the reference line, a prediction block of the current block is generated according to the intra prediction mode.

11. The method according to claim 10, wherein: Obtaining at least one or more filling methods includes: An index indicating at least one or more fill methods included in a predefined method lookup table is obtained from a higher level.

12. The method according to claim 11, further comprising: Encode the index.

13. The method according to claim 10, wherein: Obtaining at least one or more filling methods includes: determining at least one or more filling methods based on information about the block, or setting at least one or more filling methods as preset methods, The information about the block includes: Features of the current block, features of blocks adjacent to the current block in the current frame, features of blocks reconstructed earlier than the current block, or features of blocks including the co-located block of the current block and blocks adjacent to the co-located block in other referenced pictures.

14. The method according to claim 10, wherein: Obtaining the length of the MRL candidate list includes: Get the length of the MRL candidate list from a higher level.

15. The method according to claim 14, further comprising: Encodes the length of the MRL candidate list.

16. The method according to claim 11, wherein, Obtaining the length of the MRL candidate list includes: Determine the length of the MRL candidate list based on information about the block, or set the length of the MRL candidate list to a preset value, The information about the block includes: Features of the current block, features of blocks adjacent to the current block in the current frame, features of blocks reconstructed earlier than the current block, or features of blocks including the co-located block of the current block and blocks adjacent to the co-located block in other referenced pictures.

17. A computer-readable recording medium storing a bitstream generated by a video encoding method, the video encoding method comprising: determining an intra prediction mode for a current block and a multiple reference line (MRL) index indicating a reference line to be used for intra prediction of the current block within an MRL candidate list; obtaining a length of the MRL candidate list, wherein the length of the MRL candidate list indicates the number of reference lines included in the MRL candidate list; obtaining at least one or more filling methods; generating the MRL candidate list by adding a reference line corresponding to a length of the MRL candidate list to the MRL candidate list using at least one or more padding methods; Derivation of reference lines from the MRL candidate list by utilizing the MRL index; as well as By using the reference line, a prediction block of the current block is generated according to the intra prediction mode.

Images