Video coding method and apparatus using selective multiple reference lines

By selectively determining and weighting combination of multiple reference lines, the problem of inefficient use of multiple reference lines in the prior art is solved, and the efficiency and quality of video encoding are improved.

CN118975231APending Publication Date: 2024-11-15HYUNDAI MOTOR CO LTD +2
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202380032569.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2023-03-09
Filing Date
2023-03-13
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

When existing video encoding technologies use multiple reference lines in intra prediction, a large number of candidate reference lines need to be considered, resulting in inefficiency and difficulty in effectively using reference lines to improve video encoding efficiency and image quality.

Method used

A method is proposed to improve intra prediction by selectively determining the reference line group in a multi-reference line group, selecting or combining the reference line in it to generate a predicted reference line for the current block, and generating a new reference line by weighted combination to improve intra prediction efficiency and quality.

Benefits of technology

The video decoding efficiency and video quality are improved, the complexity and computing burden of intra prediction mode selection are reduced, and the utilization efficiency of reference lines is enhanced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118975231A_ABST
    Figure CN118975231A_ABST
Patent Text Reader

Abstract

Disclosed are a video encoding method and apparatus using selective multiple rows. In this embodiment, an image decoding device decodes an intra prediction mode of a current block from a bitstream, derives a reference line group of the current block, and derives reference lines in the reference line group. Further, the image decoding device generates a prediction of the current block by using the reference line according to the intra prediction mode.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to a video coding method and apparatus using selective multiple reference lines. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.

[0003] Since video data has a large amount of data compared to audio or still image data, the video data requires a large amount of hardware resources (including memory) to store or transmit the video data without a process for compression.

[0004] Thus, an encoder is generally used to compress and store or transmit video data. A decoder receives compressed video data, decompresses the received compressed video data, and plays the decompressed video data. Video compression technologies include H.264 / Advanced Video Coding (AVC), High Efficiency Video Coding (HEVC), and Versatile Video Coding (VVC), with VVC having an improved coding efficiency of about 30% or more compared to HEVC.

[0005] However, as image size, resolution, and frame rate gradually increase, the amount of data to be encoded also increases. Thus, a new compression technology is needed that provides higher decoding efficiency and improved image enhancement effects than existing compression technologies.

[0006] Intra-frame prediction uses pixel information within the same picture to predict the pixel values ​​of the current block to be encoded. In intra-frame prediction, among multiple intra-frame prediction modes, a mode that best fits the characteristics of the picture and is used to predict the current block can be selected. The encoder selects and then uses one of the multiple intra-frame prediction modes to encode the current block. The encoder can then pass information about the mode to the decoder.

[0007] HEVC technology uses a total of 35 intra prediction modes for intra prediction, including 33 angular modes with directionality and two non-angular modes without directionality. However, as the spatial resolution of the video increases from 720×480 to 2048×1024 or 8192×4096, the unit size of the prediction block becomes larger and larger, and more diverse intra prediction modes need to be added. Figure 3a As shown, the VVC technology utilizes 67 more subdivided prediction modes for intra prediction, thereby allowing a wider variety of prediction directions than in the prior art.

[0008] Because in intra prediction, predictions are generated based on adjacent pixels of the current block, the performance of intra prediction techniques is related to the appropriate selection of reference pixels. In this regard, in addition to obtaining reference pixels from more precise angular directions by diversifying the prediction modes as described above, consideration may be given to increasing the number of available candidate reference pixels. Existing techniques for the latter include multiple reference lines (MRL) or multiple reference line prediction (MRLP). The MRL technique not only uses reference lines adjacent to the current block for prediction of the current block, but also can use farther pixels as reference pixels. However, MRL requires that multiple candidate reference lines be permanently considered, which is a problem to be solved. Therefore, there is a need to provide a method for effectively utilizing reference lines to improve video coding efficiency and improve video quality. Summary of the invention

[0009] Technical issues

[0010] The present disclosure attempts to provide a video coding method and apparatus for selectively determining available reference lines in intra prediction of a current block based on a reference line group including some of multiple reference lines.

[0011] In addition, the present disclosure attempts to provide a video coding method and apparatus using a new reference line generated by a weighted combination of multiple reference lines.

[0012] In addition, the present disclosure attempts to provide a video coding method and apparatus for controlling available reference lines among multiple reference lines.

[0013] Technical Solution

[0014] At least one aspect of the present disclosure provides a method for performing intra-frame prediction on a current block by a video decoding device. The method includes decoding an intra-frame prediction mode of the current block from a bitstream. The method also includes deriving a reference line group for the current block, and the reference line group includes at least one reference line. The method also includes deriving a reference line within the reference line group. Here, the reference line in the reference line group is indicated by a reference line candidate index. The method also includes: generating a prediction of the current block by using the reference line according to the intra-frame prediction mode.

[0015] Another aspect of the present disclosure provides a method for performing intra-frame prediction on a current block by a video encoding device. The method includes determining an intra-frame prediction mode of the current block. The method also includes deriving a reference line group for the current block, and the reference line group includes at least one reference line. The method also includes deriving a reference line within the reference line group. Here, the reference line in the reference line group is indicated by a reference line candidate index. The method also includes: generating a prediction of the current block by using the reference line according to the intra-frame prediction mode.

[0016] Another aspect of the present disclosure provides a computer-readable recording medium storing a bitstream generated by a video encoding method. The video encoding method includes determining an intra-frame prediction mode of a current block. The video encoding method also includes deriving a reference line group for the current block, and the reference line group includes at least one reference line. The video encoding method also includes deriving a reference line within the reference line group. Here, the reference line in the reference line group is indicated by a reference line candidate index. The video encoding method also includes: generating a prediction of the current block by using the reference line according to the intra-frame prediction mode.

[0017] Beneficial Effects

[0018] As described above, the present disclosure provides a video decoding method and apparatus for selectively determining available reference lines in intra prediction of a current block based on a reference line group including some of multiple reference lines, thereby improving video decoding efficiency and video quality.

[0019] In addition, the present disclosure provides a video decoding method and device using a new reference line generated by weighted combination of multiple reference lines, thereby improving video decoding efficiency and video quality.

[0020] In addition, the present disclosure provides a video decoding method and device for controlling available reference lines among multiple reference lines, thereby improving video decoding efficiency and video quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a block diagram of a video encoding device that may implement the techniques of this disclosure.

[0022] Figure 2 A method for partitioning blocks using a quadtree plus binary tree ternary tree (QTBTTT) structure is shown.

[0023] Figure 3a and Figure 3b Multiple intra prediction modes are shown including a wide-angle intra prediction mode.

[0024] Figure 4 Neighboring blocks of the current block are shown.

[0025] Figure 5 is a block diagram of a video decoding device that may implement the techniques of this disclosure.

[0026] Figure 6 is a diagram showing reference lines indicated by reference line indices.

[0027] Figure 7 is a diagram showing the search order of reference samples.

[0028] Figure 8a to Figure 8b is a diagram illustrating generation of reference samples.

[0029] Fig. 9 is a diagram showing a graph of the percentage of the reference line utilized by block size.

[0030] Fig.10a and Fig.10b is a diagram illustrating inference of a reference line group according to at least one embodiment of the present disclosure.

[0031] Fig.11 is a diagram illustrating a weighted combination of two reference lines according to at least one embodiment of the present disclosure.

[0032] Fig.12 is a diagram illustrating block types based on block positions in a video according to at least one embodiment of the present disclosure.

[0033] Fig.13 is a diagram illustrating positions of pixels for defining neighboring blocks according to at least one embodiment of the present disclosure.

[0034] Fig.14 is a diagram illustrating a plurality of adjacent blocks on the upper and left sides according to at least one embodiment of the present disclosure.

[0035] Fig.15 is a diagram illustrating pixels defining neighboring blocks according to at least one embodiment of the present disclosure.

[0036] Fig.16 is a diagram illustrating reference lines indicated by reference line indices based on block areas according to at least one embodiment of the present disclosure.

[0037] Fig.17 The present invention is a flowchart of a method for encoding a current block performed by a video encoding apparatus according to at least one embodiment of the present disclosure.

[0038] Fig.18 The present invention is a flowchart of a method for decoding a current block performed by a video decoding device according to at least one embodiment of the present disclosure. DETAILED DESCRIPTION

[0039] Hereinafter, some embodiments of the present disclosure are described in detail with reference to the accompanying drawings. In the following description, the same reference numerals represent the same elements, although the elements are shown in different drawings. In addition, in the following description of some embodiments, for the purpose of clarity and brevity, detailed descriptions of related known parts and functions may be omitted when it is considered to obscure the subject matter of the present disclosure.

[0040] Figure 1is a block diagram of a video encoding device that can implement the technology of the present disclosure. Figure 1 A diagram of a video encoding device and components of the device is described.

[0041] The encoding apparatus may include a picture divider 110 , a predictor 120 , a subtractor 130 , a transformer 140 , a quantizer 145 , a rearrangement unit 150 , an entropy encoder 155 , an inverse quantizer 160 , an inverse transformer 165 , an adder 170 , a loop filter unit 180 , and a memory 190 .

[0042] Each component of the coding device can be implemented as hardware or software or as a combination of hardware and software. In addition, the function of each component can be implemented as software, and a microprocessor can also be implemented to execute the function of the software corresponding to each component.

[0043] A video consists of one or more sequences including multiple pictures. Each picture is divided into multiple regions, and encoding is performed on each region. For example, a picture is divided into one or more tiles or / and slices. Here, one or more tiles can be defined as tile groups. Each tile or / and slice is divided into one or more coding tree units (CTUs). In addition, each CTU is divided into one or more coding units (CUs) through a tree structure. The information applied to each coding unit (CU) is encoded as the syntax of the CU, and the information commonly applied to the CU included in a CTU is encoded as the syntax of the CTU. In addition, the information commonly applied to all blocks in a slice is encoded as the syntax of the slice header, and the information applied to all blocks constituting one or more pictures is encoded as a picture parameter set (PPS) or a picture header. In addition, the information commonly referenced by multiple pictures is encoded into a sequence parameter set (SPS). In addition, the information commonly referenced by one or more SPSs is encoded into a video parameter set (VPS). In addition, the information commonly applied to a tile or tile group can also be encoded as the syntax of the tile or tile group header. The syntax included in the SPS, PPS, slice header, tile, or tile group header may be referred to as a high-level syntax.

[0044] The picture partitioner 110 determines the size of a coding tree unit (CTU). Information on the size of the CTU (CTU size) is encoded as a syntax of an SPS or a PPS and delivered to a video decoding apparatus.

[0045] The picture partitioner 110 partitions each picture constituting a video into a plurality of coding tree units (CTUs) having a predetermined size, and then recursively partitions the CTUs by using a tree structure. A leaf node in the tree structure becomes a coding unit (CU), which is a basic unit of encoding.

[0046] The tree structure can be a quadtree (QT), in which a higher node (or parent node) is split into four lower nodes (or child nodes) of the same size. The tree structure can also be a binary tree (BT), in which a higher node is split into two lower nodes. The tree structure can also be a ternary tree (TT), in which a higher node is split into three lower nodes at a ratio of 1:2:1. The tree structure can also be a structure in which two or more structures of the QT structure, the BT structure, and the TT structure are mixed. For example, a quadtree plus a binary tree (QTBT) structure can be used or a quadtree plus a binary tree ternary tree (QTBTTT) structure can be used. Here, a binary tree ternary tree (BTTT) is added to the tree structure to be referred to as a multi-type tree (MTT).

[0047] Figure 2 is a diagram for describing a method of partitioning a block by using a QTBTTT structure.

[0048] like Figure 2 As shown, the CTU can be first split into a QT structure. The quadtree splitting can be recursive until the size of the split block reaches the minimum block size (MinQTSize) of the leaf node allowed in QT. The first flag (QT_split_flag) indicating whether each node of the QT structure is split into four nodes of the lower layer is encoded by the entropy encoder 155 and sent to the video decoding device with a signal. When the leaf node of QT is not larger than the maximum block size (MaxBTSize) of the root node allowed in BT, the leaf node can be further split into at least one of the BT structure and the TT structure. There may be multiple splitting directions in the BT structure and / or the TT structure. For example, there may be two directions, namely, the direction in which the blocks of the corresponding nodes are split horizontally and the direction in which the blocks of the corresponding nodes are split vertically. As Figure 2 As shown, when MTT splitting begins, a second flag (mtt_split_flag) indicating whether a node is split and a flag further indicating the split direction (vertical or horizontal) and / or a flag indicating the split type (binary or ternary) if the node is split are encoded by the entropy encoder 155 and sent to the video decoding device using a signal.

[0049] Optionally, before encoding the first flag (QT_split_flag) indicating whether each node is split into four nodes of the lower layer, a CU split flag (split_cu_flag) indicating whether the node is split may also be encoded. When the value of the CU split flag (split_cu_flag) indicates that each node is not split, the block of the corresponding node becomes a leaf node in the split tree structure and becomes a CU as a basic unit of encoding. When the value of the CU split flag (split_cu_flag) indicates that each node is split, the video encoding device first starts encoding the first flag through the above scheme.

[0050] When QTBT is used as another example of a tree structure, there may be two types, namely, a type in which the block of the corresponding node is horizontally split into two blocks of the same size (i.e., symmetrical horizontal splitting) and a type in which the block of the corresponding node is vertically split into two blocks of the same size (i.e., symmetrical vertical splitting). A split flag (split_flag) indicating whether each node of the BT structure is split into blocks of the lower layer and split type information indicating the split type are encoded by the entropy encoder 155 and delivered to the video decoding device. At the same time, a type in which the block of the corresponding node is split into two blocks that are asymmetric to each other may be presented in addition. The asymmetric form may include a form in which the block of the corresponding node is split into two rectangular blocks having a size ratio of 1:3, or may also include a form in which the block of the corresponding node is split in a diagonal direction.

[0051] A CU may have different sizes depending on the QTBT or QTBTTT partition from a CTU. Hereinafter, a block corresponding to a CU to be encoded or decoded (i.e., a leaf node of a QTBTTT) is referred to as a "current block". Due to the adoption of QTBTTT partitioning, the shape of the current block may be a rectangular shape in addition to a square shape.

[0052] The predictor 120 predicts the current block to generate a predicted block. The predictor 120 includes an intra predictor 122 and an inter predictor 124.

[0053] Typically, each of the current blocks in a picture may be predictively encoded. Typically, the prediction of the current block may be performed using intra-prediction techniques (using data from the picture including the current block) or inter-prediction techniques (using data from a picture encoded before the picture including the current block). Inter-prediction includes both unidirectional prediction and bidirectional prediction.

[0054] The intra-frame predictor 122 predicts the pixels in the current block by using the pixels (reference pixels) located at the neighboring positions of the current block in the current picture including the current block. There are a plurality of intra-frame prediction modes according to the prediction direction. For example, Figure 3a As shown, the plurality of intra prediction modes may include 2 non-directional modes including a planar mode and a DC mode, and may include 65 directional modes. Neighboring pixels and an arithmetic equation to be used are defined differently according to each prediction mode.

[0055] In order to perform efficient directional prediction for a current block having a rectangular shape, it is possible to additionally use Figure 3b The directional modes (#67 to #80, intra prediction modes #-1 to #-14) are shown as dotted arrows in FIG. 1 . The directional modes may be referred to as "wide-angle intra prediction modes". Figure 3b, the arrows indicate the corresponding reference samples used for prediction and do not represent the prediction direction. The prediction direction is opposite to the direction indicated by the arrow. When the current block has a rectangular shape, the wide-angle intra prediction mode is a mode in which prediction is performed in a direction opposite to the specific direction mode without additional bit transmission. In this case, in the wide-angle intra prediction mode, some wide-angle intra prediction modes available for the current block can be determined by the ratio of the width and height of the current block having a rectangular shape. For example, when the current block has a rectangular shape with a height less than the width, a wide-angle intra prediction mode (intra prediction modes #67 to #80) with an angle less than 45 degrees is available. When the current block has a rectangular shape with a width greater than the height, a wide-angle intra prediction mode with an angle greater than -135 degrees can be used.

[0056] The intra-frame predictor 122 may determine an intra-frame prediction to be used to encode the current block. In some embodiments, the intra-frame predictor 122 may encode the current block by using a plurality of intra-frame prediction modes, and may further select an appropriate intra-frame prediction mode to be used from a test mode. For example, the intra-frame predictor 122 may calculate a rate-distortion value by using a rate-distortion analysis for a plurality of test intra-frame prediction modes, and may further select an intra-frame prediction mode having the best rate-distortion characteristics in the test mode.

[0057] The intra-frame predictor 122 selects an intra-frame prediction mode among a plurality of intra-frame prediction modes and predicts the current block by using adjacent pixels (reference pixels) and an arithmetic equation determined according to the selected intra-frame prediction mode. Information about the selected intra-frame prediction mode is encoded by the entropy encoder 155 and delivered to the video decoding device.

[0058] The inter-frame predictor 124 generates a prediction block for the current block by using motion compensation processing. The inter-frame predictor 124 searches for a block most similar to the current block in a reference picture that is encoded and decoded earlier than the current picture, and generates a prediction block for the current block by using the searched block. In addition, a motion vector (MV) is generated, which corresponds to the displacement between the current block in the current picture and the prediction block in the reference picture. Typically, motion estimation is performed on the luminance component, and the motion vector calculated based on the luminance component is used for both the luminance component and the chrominance component. Motion information including information about the reference picture and information about the motion vector for predicting the current block is encoded by the entropy encoder 155 and delivered to the video decoding device.

[0059] The inter-frame predictor 124 may also perform interpolation on a reference picture or a reference block in order to increase the accuracy of the prediction. In other words, the sub-sampling between two consecutive integer samples is interpolated by applying the filter coefficient to a plurality of consecutive integer samples including two integer samples. When the process of searching for the block most similar to the current block is performed for the interpolated reference picture, the decimal unit precision may be expressed instead of the integer sample unit precision for the motion vector. The precision or resolution of the motion vector may be set differently for each target region to be encoded (e.g., a unit such as a slice, a tile, a CTU, a CU, etc.). When such an adaptive motion vector resolution (AMVR) is applied, information about the motion vector resolution to be applied to each target region should be signaled for each target region. For example, when the target region is a CU, information about the motion vector resolution applied to each CU is signaled. The information about the motion vector resolution may be information representing the precision of the motion vector difference to be described below.

[0060] Meanwhile, the inter-frame predictor 124 may perform inter-frame prediction by using bidirectional prediction. In the case of bidirectional prediction, two reference pictures and two motion vectors representing the block position most similar to the current block in each reference picture are used. The inter-frame predictor 124 selects a first reference picture and a second reference picture from a reference picture list 0 (RefPicList0) and a reference picture list 1 (RefPicList1), respectively. The inter-frame predictor 124 also searches for a block most similar to the current block in the corresponding reference picture to generate a first reference block and a second reference block. In addition, a prediction block for the current block is generated by averaging or weighted averaging the first reference block and the second reference block. In addition, motion information including information about two reference pictures for predicting the current block and including information about two motion vectors is delivered to the entropy encoder 155. Here, the reference picture list 0 may be composed of pictures before the current picture in the pre-reconstructed picture in the display order, and the reference picture list 1 may be composed of pictures after the current picture in the pre-reconstructed picture in the display order. However, although not particularly limited thereto, a pre-reconstructed picture after the current picture in the display order may be additionally included in the reference picture list 0. Conversely, a pre-reconstructed picture before the current picture may also be additionally included in the reference picture list 1.

[0061] In order to minimize the number of bits consumed for encoding motion information, various methods may be used.

[0062] For example, when the reference picture and motion vector of the current block are the same as those of the adjacent block, information capable of identifying the adjacent block is encoded to deliver the motion information of the current block to the video decoding device. This method is called merge mode.

[0063] In the merge mode, the inter predictor 124 selects a predetermined number of merge candidate blocks (hereinafter, referred to as “merge candidates”) from the neighboring blocks of the current block.

[0064] As neighboring blocks for deriving merge candidates, such as Figure 4 As shown, all or some of the left block A0, the lower left block A1, the upper block B0, the upper right block B1, and the upper left block B2 adjacent to the current block in the current picture may be used. In addition, blocks other than the current picture where the current block is located, which are located in a reference picture (which may be the same as or different from the reference picture used to predict the current block), may also be used as merge candidates. For example, a block with a collocation of the current block or a block adjacent to the collocated block in the reference picture may be additionally used as a merge candidate. If the number of merge candidates selected by the method described above is less than the preset number, a zero vector is added to the merge candidate.

[0065] The inter-frame predictor 124 configures a merge list including a predetermined number of merge candidates by using adjacent blocks. A merge candidate to be used as motion information of the current block is selected from the merge candidates included in the merge list, and merge index information for identifying the selected candidate is generated. The generated merge index information is encoded by the entropy encoder 155 and delivered to the video decoding device.

[0066] Merge skip mode is a special case of merge mode. After quantization, when all transform coefficients for entropy coding are close to zero, only neighbor block selection information is sent without sending residual signals. By using merge skip mode, relatively high coding efficiency can be achieved for images with slight motion, still images, screen content images, etc.

[0067] Hereinafter, the merge mode and the merge skip mode are collectively referred to as the merge / skip mode.

[0068] Another method for encoding motion information is the Advanced Motion Vector Prediction (AMVP) mode.

[0069] In the AMVP mode, the inter-frame predictor 124 derives a motion vector prediction candidate for a motion vector of the current block by using the neighboring blocks of the current block. As the neighboring blocks for deriving the motion vector prediction candidate, Figure 4All or some of the left block A0, lower left block A1, upper block B0, upper right block B1 and upper left block B2 adjacent to the current block in the current picture shown. In addition, blocks other than the current picture where the current block is located, which are located in a reference picture (which may be the same as or different from the reference picture used to predict the current block), may also be used as adjacent blocks for deriving motion vector prediction candidates. For example, a block collocated with the current block or a block adjacent to the collocated block within the reference picture may be used. If the number of motion vector candidates selected by the above method is less than the preset number, a zero vector is added to the motion vector candidates.

[0070] The inter predictor 124 derives a motion vector prediction candidate by using the motion vectors of the neighboring blocks, and determines a motion vector prediction for the motion vector of the current block by using the motion vector prediction candidate. In addition, a motion vector difference is calculated by subtracting the motion vector prediction from the motion vector of the current block.

[0071] Motion vector prediction can be obtained by applying a predefined function (e.g., center value and average value calculation, etc.) to a motion vector prediction candidate. In this case, the video decoding device also knows the predefined function. In addition, because the neighboring blocks used to derive the motion vector prediction candidate are blocks for which encoding and decoding have been completed, the video decoding device may also already know the motion vectors of the neighboring blocks. Therefore, the video encoding device does not need to encode information for identifying the motion vector prediction candidate. Thus, in this case, information about the motion vector difference and information about the reference picture used to predict the current block are encoded.

[0072] Meanwhile, the motion vector prediction may also be determined by selecting any one of the motion vector prediction candidates. In this case, information for identifying the selected motion vector prediction candidate is additionally encoded jointly with information about the motion vector difference and information about the reference picture used to predict the current block.

[0073] The subtractor 130 generates a residual block by subtracting a prediction block generated by the intra predictor 122 or the inter predictor 124 from the current block.

[0074] The transformer 140 converts the residual signal in the residual block having the pixel value in the spatial domain into a transform coefficient in the frequency domain. The transformer 140 may transform the residual signal in the residual block by using the total size of the residual block as a transform unit, or may further divide the residual block into a plurality of sub-blocks, and may perform the transform by using the sub-block as a transform unit. Optionally, the residual block is divided into two sub-blocks, namely a transform region and a non-transform region, so as to transform the residual signal using only the transform region sub-block as a transform unit. Here, the transform region sub-block may be one of two rectangular blocks having a size ratio of 1:1 based on the horizontal axis (or vertical axis). In this case, the flag (cu_sbt_flag) indicates that only the sub-block is transformed, and the directional (vertical / horizontal) information (cu_sbt_horizontal_flag) and / or the position information (cu_sbt_pos_flag) are encoded by the entropy encoder 155 and sent to the video decoding device by a signal. In addition, the size of the transform region sub-block may have a size ratio of 1:3 based on the horizontal axis (or vertical axis). In this case, a flag (cu_sbt_quad_flag) of the division corresponding to the split is additionally encoded by the entropy encoder 155 and is signaled to the video decoding device.

[0075] Meanwhile, the transformer 140 may perform transformation on the residual block separately in the horizontal direction and the vertical direction. For the transformation, different types of transformation functions or transformation matrices may be used. For example, a pair of transformation functions for horizontal transformation and vertical transformation may be defined as a plurality of transformation sets (MTS). The transformer 140 may select a pair of transformation functions having the highest transformation efficiency in the MTS, and may transform the residual block in each of the horizontal and vertical directions. Information (mts_idx) of the transformation function pair in the MTS is encoded by the entropy encoder 155 and sent to the video decoding device with a signal.

[0076] The quantizer 145 quantizes the transform coefficients output from the transformer 140 using the quantization parameter and outputs the quantized transform coefficients to the entropy encoder 155. The quantizer 145 can also immediately quantize the relevant residual block without the need for transforming any block or frame. The quantizer 145 can also apply different quantization coefficients (scaling values) according to the position of the transform coefficient in the transform block. The quantization matrix applied to the quantized transform coefficients arranged in 2 dimensions can be encoded and sent to the video decoding device with a signal.

[0077] The rearrangement unit 150 may perform rearrangement of coefficient values ​​with respect to the quantized residual value.

[0078] The rearrangement unit 150 may change the 2D coefficient array into a 1D coefficient sequence by using coefficient scanning. For example, the rearrangement unit 150 may output a 1D coefficient sequence by scanning the DC coefficient into a high frequency domain coefficient using a zigzag scan or a diagonal scan. Depending on the size of the transform unit and the intra prediction mode, a vertical scan that scans the 2D coefficient array in the column direction and a horizontal scan that scans the 2D block type coefficients in the row direction may also be used instead of the zigzag scan. In other words, depending on the size of the transform unit and the intra prediction mode, the scanning method to be used may be determined among zigzag scanning, diagonal scanning, vertical scanning, and horizontal scanning.

[0079] The entropy encoder 155 generates a bitstream by encoding a sequence of 1D quantized transform coefficients output from the rearrangement unit 150 using various encoding schemes including context-based adaptive binary arithmetic coding (CABAC), exponential Golomb, and the like.

[0080] In addition, the entropy encoder 155 encodes information related to block segmentation (such as CTU size, CTU segmentation flag, QT segmentation flag, MTT segmentation type, MTT segmentation direction, etc.) to allow the video decoding device to equally segment the block to the video encoding device. In addition, the entropy encoder 155 encodes information about the prediction type indicating whether the current block is encoded by intra-frame prediction or inter-frame prediction. The entropy encoder 155 encodes intra-frame prediction information (i.e., information about the intra-frame prediction mode) or inter-frame prediction information (in the case of merge mode, merge index, and in the case of AMVP mode, information about reference picture index and motion vector difference) according to the prediction type. In addition, the entropy encoder 155 encodes information related to quantization (i.e., information about quantization parameters and information about quantization matrices).

[0081] The inverse quantizer 160 dequantizes the quantized transform coefficient output from the quantizer 145 to generate a transform coefficient. The inverse transformer 165 transforms the transform coefficient output from the inverse quantizer 160 from the frequency domain to the spatial domain to reconstruct a residual block.

[0082] The adder 170 reconstructs the current block by adding the reconstructed residual block and the prediction block generated by the predictor 120. When intra prediction is performed on the next sequential block, pixels in the reconstructed current block may be used as reference pixels.

[0083] The loop filter unit 180 performs filtering on the reconstructed pixels in order to reduce blocking effects, ringing effects, blurring effects, etc. that occur due to block-based prediction and transform / quantization. The loop filter unit 180 as a loop filter may include all or some of a deblocking filter 182, a sample adaptive offset (SAO) filter 184, and an adaptive loop filter (ALF) 186.

[0084] The deblocking filter 182 filters the boundaries between the reconstructed blocks to remove the block effect caused by block unit encoding / decoding, and the SAO filter 184 and the ALF 186 perform additional filtering for the deblocked filtered video. The SAO filter 184 and the ALF 186 are filters for compensating the difference between the reconstructed pixels and the original pixels, which occurs due to lossy decoding. The SAO filter 184 applies an offset as a CTU unit to enhance subjective image quality and coding efficiency. On the other hand, the ALF 186 performs block unit filtering and applies different filters to compensate for distortion by dividing the boundaries of the corresponding blocks and the degree of the change amount. Information about the filter coefficients to be used for the ALF can be encoded and sent to the video decoding device with a signal.

[0085] The reconstructed blocks filtered by the deblocking filter 182, the SAO filter 184, and the ALF 186 are stored in the memory 190. When all blocks in one picture are reconstructed, the reconstructed picture can be used as a reference picture for inter-frame prediction of blocks within a picture to be encoded later.

[0086] Figure 5 is a functional block diagram of a video decoding device that can implement the technology of the present disclosure. Figure 5 , describing a video decoding device and components of the device.

[0087] The video decoding apparatus may include an entropy decoder 510 , a rearrangement unit 515 , an inverse quantizer 520 , an inverse transformer 530 , a predictor 540 , an adder 550 , a loop filter unit 560 , and a memory 570 .

[0088] and Figure 1 Similar to the video encoding device of the present invention, each component of the video decoding device can be implemented as hardware or software or as a combination of hardware and software. In addition, the function of each component can be implemented as software, and a microprocessor can also be implemented to execute the function of the software corresponding to each component.

[0089] The entropy decoder 510 extracts information related to block partitioning by decoding a bitstream generated by a video encoding apparatus to determine a current block to be decoded, and extracts prediction information required for reconstructing the current block and information about a residual signal.

[0090] The entropy decoder 510 determines the size of the CTU by extracting information about the CTU size from a sequence parameter set (SPS) or a picture parameter set (PPS), and partitions the picture into CTUs of the determined size. In addition, the CTU is determined to be the highest level of the tree structure, i.e., the root node, and partition information of the CTU can be extracted to partition the CTU by using the tree structure.

[0091] For example, when CTU is segmented by using the QTBTTT structure, the first flag (QT_split_flag) related to the segmentation of QT is first extracted to segment each node into four nodes of the lower layer. In addition, for the node corresponding to the leaf node of QT, the second flag (mtt_split_flag) related to the segmentation of MTT, the segmentation direction (vertical / horizontal) and / or the segmentation type (binary / ternary) are extracted to segment the corresponding leaf node into the MTT structure. As a result, each node below the leaf node of QT is recursively segmented into a BT or TT structure.

[0092] As another example, when a CTU is split by using a QTBTTT structure, a CU split flag (split_cu_flag) indicating whether the CU is split is extracted. When the corresponding block is split, the first flag (QT_split_flag) may also be extracted. During the splitting process, for each node, 0 or more recursive MTT splits may occur after 0 or more recursive QT splits. For example, for a CTU, MTT splits may occur immediately, or conversely, only multiple QT splits may occur.

[0093] As another example, when CTU is segmented by using the QTBT structure, the first flag (QT_split_flag) related to the splitting of the QT is extracted to split each node into four nodes of the lower layer. In addition, a split flag (split_flag) indicating whether the node corresponding to the leaf node of the QT is further segmented into BTs and segmentation direction information are extracted.

[0094] Meanwhile, when the entropy decoder 510 determines the current block to be decoded by partitioning using a tree structure, the entropy decoder 510 extracts information about a prediction type indicating whether the current block is intra-predicted or inter-predicted. When the prediction type information indicates intra-prediction, the entropy decoder 510 extracts syntax elements of intra-prediction information (intra-prediction mode) for the current block. When the prediction type information indicates inter-prediction, the entropy decoder 510 extracts information representing syntax elements for inter-prediction information (i.e., motion vectors and reference pictures referenced by motion vectors).

[0095] Furthermore, the entropy decoder 510 extracts quantization-related information and extracts information on a quantized transform coefficient of the current block as information on a residual signal.

[0096] The rearrangement unit 515 may change the sequence of 1D quantized transform coefficients entropy-decoded by the entropy decoder 510 into a 2D coefficient array (ie, block) again in the reverse order of the coefficient scanning order performed by the video encoding apparatus.

[0097] The inverse quantizer 520 dequantizes the quantized transform coefficients and dequantizes the quantized transform coefficients by using quantization parameters. The inverse quantizer 520 may also apply different quantization coefficients (scaling values) to the quantized transform coefficients arranged in 2D. The inverse quantizer 520 may perform dequantization by applying a matrix (scaling value) of quantization coefficients from a video encoding device to a 2D array of quantized transform coefficients.

[0098] The inverse transformer 530 generates a residual block for the current block by reconstructing a residual signal by inversely transforming the dequantized transform coefficients from the frequency domain into the spatial domain.

[0099] In addition, when the inverse transformer 530 inversely transforms a partial region (sub-block) of the transform block, the inverse transformer 530 extracts a flag (cu_sbt_flag) that only the sub-block of the transform block is transformed, direction (vertical / horizontal) information (cu_sbt_horizontal_flag) of the sub-block, and / or position information (cu_sbt_pos_flag) of the sub-block. The inverse transformer 530 also inversely transforms the transform coefficient of the corresponding sub-block from the frequency domain to the spatial domain to reconstruct the residual signal and fills the region that is not inversely transformed with a value of "0" as the residual signal to generate a final residual block for the current block.

[0100] In addition, when MTS is applied, the inverse transformer 530 determines a transform index or a transform matrix applied in each of the horizontal and vertical directions by using MTS information (mts_idx) signaled from the video encoding device. The inverse transformer 530 also performs inverse transform on the transform coefficients in the transform block in the horizontal and vertical directions by using the determined transform function.

[0101] The predictor 540 may include an intra predictor 542 and an inter predictor 544. The intra predictor 542 is activated when the prediction type of the current block is intra prediction, and the inter predictor 544 is activated when the prediction type of the current block is inter prediction.

[0102] The intra predictor 542 determines an intra prediction mode of a current block among a plurality of intra prediction modes according to the syntax element of the intra prediction mode extracted from the entropy decoder 510. The intra predictor 542 also predicts the current block by using neighboring reference pixels of the current block according to the intra prediction mode.

[0103] The inter predictor 544 determines a motion vector of a current block and a reference picture to which the motion vector refers by using a syntax element for an inter prediction mode extracted from the entropy decoder 510 .

[0104] The adder 550 reconstructs the current block by adding the residual block output from the inverse transformer 530 and the prediction block output from the inter predictor 544 or the intra predictor 542. When intra prediction is performed on a block to be decoded later, pixels within the reconstructed current block are used as reference pixels.

[0105] The loop filter unit 560 as a loop filter may include a deblocking filter 562, an SAO filter 564, and an ALF 566. The deblocking filter 562 performs deblocking filtering on the boundary between the reconstructed blocks to remove the block effect that occurs due to block unit decoding. The SAO filter 564 and the ALF 566 perform additional filtering on the reconstructed blocks after deblocking filtering to compensate for the difference between the reconstructed pixels and the original pixels that occurs due to lossy decoding. The filter coefficients of the ALF are determined by using information about the filter coefficients decoded from the bitstream.

[0106] The reconstructed blocks filtered by the deblocking filter 562, the SAO filter 564, and the ALF 566 are stored in the memory 570. When all blocks in one picture are reconstructed, the reconstructed picture can be used as a reference picture for inter-frame prediction of blocks in a picture to be encoded later.

[0107] In some embodiments, the present disclosure relates to encoding and decoding video images as described above. More specifically, the present disclosure provides a video decoding method and device for selectively determining available reference lines in intra-frame prediction of a current block based on a reference line group including some of multiple reference lines. In addition, the present disclosure provides a video decoding method and device for using a new reference line generated by a weighted combination of multiple reference lines. In addition, the present disclosure provides a video decoding method and device for controlling available reference lines in multiple reference lines.

[0108] The following embodiments may be performed by the intra-frame predictor 122 in the video encoding apparatus. The following embodiments may also be performed by the intra-frame predictor 542 in the video decoding apparatus.

[0109] The video encoding device in the prediction of the current block may generate signaling information associated with the present embodiment in terms of optimizing rate distortion. The video encoding device may encode the signaling information using the entropy encoder 155 and send the encoded signaling information to the video decoding device. The video decoding device may decode the signaling information associated with the prediction of the current block from the bitstream using the entropy decoder 510.

[0110] In the following description, the term 'target block' may be used interchangeably with a current block or a coding unit (CU), or may refer to a certain region of a coding unit.

[0111] In addition, a flag value of true indicates when the flag is set to 1. In addition, a flag value of false indicates when the flag is set to 0.

[0112] Hereinafter, the embodiments are described centering on a video decoding apparatus, but the embodiments can be similarly applied to a video encoding apparatus.

[0113] I. MPM and MRL

[0114] Several techniques are introduced to improve coding efficiency by using intra prediction. The most probable mode (MPM) technique utilizes the intra prediction mode of a neighboring block for intra prediction of a current block. The video decoding device generates an MPM list to include intra prediction modes derived from predefined positions that are spatially adjacent to the current block. When the MPM mode is applied, the video encoding device may send a flag intra_luma_mpm_flag to the video decoding device indicating whether the MPM list is to be used. If intra_luma_mpm_flag does not exist, intra_luma_mpm_flag is inferred to be 1. In addition, the video encoding device may send an MPM index, intra_luma_mpm_idx, instead of an index of a prediction mode, thereby increasing the coding efficiency of the intra prediction mode.

[0115] When predicting the current block according to the intra prediction technology, the multi-reference line (MRL) technology can use not only the reference lines adjacent to the current block, but also the pixels existing farther away as reference pixels. In this case, the pixels with the same distance from the current block are grouped and called reference lines. The MRL technology uses the pixels located on the selected reference line to perform intra prediction of the current block.

[0116] The video encoding apparatus for indicating a reference line to be used when performing intra prediction signals a reference line index (hereinafter, used interchangeably with "intra_luma_ref_idx") to the video decoding apparatus. Bit allocation for each index may be shown in Table 1.

[0117]

Table 1

[0118] imra_luma_ref_idx Bit allocation 0 0 1 10 2 11

[0119] In addition to the plane, the video decoding device can consider whether to use additional reference lines by applying MRL to the intra prediction mode signaled according to the MPM. The reference line represented by each intra_luma_ref_idx is Figure 6In the example of . In the VVC (Universal Video Coding) technology, the video decoding device selects one of the three reference lines closer to the current block in distance for intra prediction of the current block. The syntax associated with signaling the reference line index of intra_luma_ref_idx for prediction and the prediction mode of the current block is shown in Table 2.

[0120]

Table 2

[0121]

[0122]

[0123] First, the video decoding apparatus parses intra_luma_ref_idx to determine a reference line index for prediction. Because an intra sub-partitioning (ISP) technique is applicable when the reference line index is zero, the video decoding apparatus does not parse information related to the ISP when the reference line index is non-zero.

[0124] The MRL technique and the MPM mode can be combined as follows.

[0125] First, if intra_luma_ref_idx is 0, a flag intra_luma_not_planar_flag indicating whether the planar mode is to be used may be signaled from the video encoding device to the video decoding device. If intra_luma_not_planar_flag is false, the intra prediction mode is set to planar mode, and if intra_luma_not_planar_flag is true, intra_luma_mpm_idx may be additionally signaled. If intra_luma_not_planar_flag is not present, intra_luma_not_planar_flag may be inferred to be 1.

[0126] Next, if intra_luma_ref_idx is non-zero, planar mode is not used. Therefore, intra_luma_not_planar_flag is not sent and is assumed to be true. Since intra_luma_not_planar_flag is true, intra_luma_mpm_idx may be signaled additionally.

[0127] As described above, intra prediction generates predictions by referring to the neighboring pixels of the current block. The neighboring pixels to be referenced are called reference samples. Before intra prediction, the video decoding device prepares reference samples. The video decoding device checks the availability of the reference sample of the position of the pixel to be referenced. If the reference sample does not exist, the pixel value is filled in the position of the pixel to be referenced according to the predetermined consistency between the video encoding device and the video decoding device. Then, the final reference sample can be generated by applying a filter to the prepared reference sample.

[0128] In this case, the reference sample refUnfilt[x][y] before the filter is applied may be generated as follows: In the following description, refIdx represents the index of a reference line, and refW and refH represent the width and height of a reference area, respectively.

[0129] If all samples in refUnfilt[x][y] are not available for intra prediction, all values ​​of refUnfilt[x][y] are set to 1<<(BitDepth-1). Here, x=-1-refIdx, y=-1-refIdx...refH-1, and x=-refIdx...refW-1, y=-1-refIdx are defined.

[0130] On the other hand, if some refUnfilt[x][y] values ​​are not available for intra prediction, the following method is applied.

[0131] If refUnfilt[-1-refIdx][refH-1] is not available, then an available refUnfilt[x][y] is searched from "x=-1-refIdx, y=refH-1" to "x=-1-refIdx, y=-1-refIdx", and then an available refUnfilt[x][y] is searched from "x=-refIdx, y=-1-refIdx" to "x=refW-1, y=-1-refIdx". When the search is terminated, refUnfilt[-1-refidx][refH-1] is set to refUnfilt[x][y].

[0132] Furthermore, if there are unavailable samples in the range of x=-1-refIdx, y=refH-2 . . . -1-refIdx, refUnfilt[x][y] is set to refUnfilt[x][y+1].

[0133] In addition, if there are unavailable samples in the range of x=-refIdx . . . refW-1, y=-1-refIdx, refUnfilt[x][y] is set to refUnfilt[x-1][y].

[0134] Figure 7 is a diagram showing the search order of reference samples.

[0135] To determine whether a reference sample is available, e.g. Figure 7 As shown in the example of , the video decoding device searches clockwise from the lower left pixel to the upper rightmost pixel.

[0136] Figure 8a and Figure 8b is a diagram illustrating generation of reference samples.

[0137] If all reference pixels are available, the video decoding device does not perform padding and uses each reference pixel value. On the other hand, as described above, if some of the available reference samples are not present, the pixel values ​​may be filled, such as Figure 8a and Figure 8b First, if the lower left reference sample is not available, the first available reference sample in the search order is copied and filled in the lower left reference sample, as shown in FIG. Figure 8a Then, if there is no reference sample other than the lower left reference sample, then Figure 8a and Figure 8b As shown in the example of , the pixel value of the immediately preceding position in the search order is copied and filled in the current position.

[0138] As described above, if there are no available reference samples at all positions, the video decoding apparatus uses 2 Bit -depth-1 (which is half of the maximum value a pixel can have). For example, if the bit depth is 8 bits, 128 can be used, and if the bit depth is 10 bits, 512 can be used.

[0139] After generating the reference sample according to the method described above, the video decoding device may apply the filter to generate the final reference sample p[x][y]. First, under the condition that the reference line index of refIdx is 0, the size of the current block is greater than 32, the current block is a luma component, the IntraSubPartitionsSplitType of the ISP mode is ISP_NO_SPLIT, and the flag refFilterFlag indicating the filtering of the reference sample is 1, the video decoding device may set the flag filterFlag indicating the application of the filter to 1. If any of the above conditions is not met, filterFlag may be set to 0.

[0140] Then, if filterFlag is true, the final reference sample p[x][y] can be calculated as shown in Equation 1.

[0141] [Equation 1]

[0142] p[-1][-1]=(refUnfilt[-1][0]+2·refUnfilt[-1][-1]+refUnfilt[0][-1]+2)>>2

[0143] p[-1][y]=(refUnfilt[-1][y+1]+2·refUnfilt[-1][y]+refUnfilt[-1][y-1]+2)>>2,y=0..refH-2

[0144] p[-1][refH-1]=refUnfilt[-1][refH-1]

[0145] p[x][-1]=(refUnfilt[x-1][-1]+2·refUnfilt[x][-1]+refUnfilt[x+1][-1]+2)>>2.x=0..refW-2

[0146] p[refW-1][-1]=refUnfilt[refW-1][-1]

[0147] On the other hand, if filterFlag is false, for x=-1-ref1dx, y=-1-ref1dx..refH-1, and x=-refIdx..refW-1, setting of y=-1-refIdx, p[x][y]=refUnfilt[x][y] is performed.

[0148] On the other hand, the existing MRL technique has the following disadvantages: the best reference line is always selected by comparing all candidate reference lines. The following is the inefficiency that may occur when using the MRL technique of VVC, which is a conventional technique considering 3 reference lines of intra_luma_ref_idx0, intra_luma_ref_idx1, intra_luma_ref_idx2. For example, by calculating the number of reference lines for each block size (log 2 The ratio of the reference line of WH) and normalizing the ratio relative to intra_luma_ref_idx1 and intra_luma_ref_idx2 can be as follows Fig. 9 The example in Draw the proportions of the block using two reference lines. Fig. 9In the example of FIG. 1 , it can be seen that as the block size increases, intra_luma_ref_idx2, which is two pixels away from the current block, uses up to 4 times as much as intra_luma_ref_idx1, which is one pixel away. Therefore, failure to consider factors that affect the selection of reference lines (such as block size) as in conventional techniques may lead to inefficiency. As the number of candidate reference lines that can be selected by the MRL technique increases, the inefficiency of the conventional technique of always considering all reference lines further increases, and the length of the bin assigned to each reference line may increase proportionally.

[0149] II. Selective Multiple Reference Lines (Selective MRL)

[0150] The present disclosure can solve these problems of the prior art by selectively determining which one of the multiple reference lines to use. To this end, the present disclosure redefines "reference line group". A reference line group is a grouping of some of the N available reference lines (where N is a natural number). There may be K reference line groups (where K is a natural number greater than or equal to 1), and each reference line group may contain the same number of candidate reference lines (m 1 =m 2 =m 3 , ...) or a different number of candidate reference lines. Here, m represents the size of the reference line group, that is, the number of reference lines included in the reference line group. If all N available reference lines are included in one reference line group, K=1 and m=N. In the following, for ease of explanation, the value of the index (intra_luma_ref_idx) of the N available reference lines may be represented as 0 to N-1.

[0151] As an example, Table 3 shows the structure of 3 reference line groups.

[0152]

Table 3

[0153] Guides included in guide set Group 0 {intra_luma_ref_idx 1, intra_luma_ref_idx 3} Group 1 {intra_luma_ref_idx 0, intra_luma_ref_idx 4, intra_luma_ref_idx 8} Group 2 {intra_luma_ref_idx 0}

[0154] Here, "Group 0" includes 2 reference lines, "Group 1" includes 3 reference lines, and "Group 2" includes 1 reference line.

[0155] The reference line index indicates a reference line that is a pixel distance away from the index value of the current block. Thus, intra_luma_ref_idx 0 indicates a reference line that is continuous (adjacent) to the current block, and intra_luma_ref_idx 1 indicates a reference line that is one pixel away from the current block. According to an embodiment, the reference line index may indicate a reference line that is a pixel distance or a block distance away from the index value, both of which may indicate the position of the reference line.

[0156] In order to solve the above problem, the present disclosure can determine the user by selecting one of the multiple reference line groups and selecting, combining or limiting one of the reference lines included in the selected reference line group.

[0157] A reference line for prediction of a current block. When predicting the current block by determining a reference line group used by the current block according to a predetermined method and selecting one of the reference lines included in the determined reference line group, a video decoding device that is notified that one of the reference lines included in the determined reference line group is used for prediction may parse a reference line candidate index (hereinafter, "ref_group_candidate_idx") instead of intra_luma_ref_idx indicating the reference line. Here, the first case may be used for ref_group_candidate_idx to indicate which reference line is to be used within the determined reference line group, and the second case may determine the reference line based on a predetermined mapping between ref_group_candidate_idx and intra_luma_ref_idx. In the second case, the video decoding device may perform a predetermined mapping for indicating the m reference lines included in the reference line group as ref_group_candidate_idx0 to ref_group_candidate_idxm-1 by using an order of increasing values ​​of reference line indices (intra_luma_ref_idx) in the reference line group, decreasing values ​​of intra_luma_ref_idx in the reference line group, or any other relationship between indices.

[0158] The first case is described below. For example, if the reference line group is "Group 1" in Table 3, where intra_luma_ref_idx 4 is used for prediction, the video decoding device determines which reference line to use by parsing ref_group_candidate_idx 1 of the current block. For convenience, the description assumes that the information about the reference line is encoded in a unary manner, and N, which represents the number of available reference lines, is greater than 8, that is, all 3 reference lines included in "Group 1" in Table 3 are available for use. First, if intra_luma_ref_idx is signaled, the codeword used is "11110" because "4" is encoded. On the other hand, if the information about the reference line group is determined by inference and ref_group_candidate_idx is signaled, the codeword used is "10" because "1" is encoded. Thus, if information about the reference line group is signaled and ref_group_candidate_idx is signaled, "Group 1 indication codeword + 10" can be used.

[0159] Using any m of the N reference lines, the reference line group can be organized in any way such as {intra_luma_ref_idx i, intra_luma_ref_idx j, intra_luma_ref_idx k...}, and an example method is as follows: In this case, intra_luma_ref_idx0 can be added regardless of the relationship with other index values ​​to establish geometric or arithmetic progression.

[0160] For example, the reference lines in the reference line group may be in arithmetic progression. The reference line group may be configured so that the index values ​​i, j, k, ... of the reference lines intra_luma_ref_idx i, intra_luma_ref_idx j, intra_luma_ref_idxk, ... included in the reference line group are in arithmetic progression. For example, when the common difference is 2, the reference line group may include intra_luma_ref_idx0, intra_luma_ref_idx2, intra_luma_ref_idx4. Optionally, when the common difference is 2, the reference line group may include intra_luma_ref_idx1, intra_luma_ref_idx3, intra_luma_ref_idx5.

[0161] As another example, the reference lines in the reference line group may be in geometric progression. The reference line group may be configured such that the index values ​​i, j, k, ... of intra_luma_ref_idx i, intra_luma_ref_idx j, intra_luma_ref_idx k, ... included in the reference line group are in geometric progression. For example, when the common ratio is 3, the reference line group may include intra_luma_ref_idx1, intra_luma_ref_idx 3, intra_luma_ref_idx 9.

[0162] Hereinafter, the above method for solving the problems of the conventional MRL technology is referred to as selective MRL. Some implementations of selective MRL are as follows.

[0163] To enable the application of each implementation (selective MRL), the video encoding device sends a sps_selective_mrl_enabled_flag or pps_selective_mrl_enabled_flag signal to the video decoding device at a higher level (such as a sequence parameter set (SPS) or a picture parameter set (PPS)). Although the MRL technology of the prior art may involve 3 reference lines, the present disclosure may enable the video decoding device to consider more than 3 reference lines, such as a natural number N reference lines. In the following, for convenience, the intra-frame prediction mode 18 in the horizontal direction is referred to as HOR_Idx, and the intra-frame prediction mode 50 in the vertical direction is referred to as VER_Idx.

[0164] <Implementation 1> Select a reference line group and use one of the reference lines

[0165] In this implementation, in order to selectively utilize multiple reference lines, the video decoding device selects one of the K reference line groups and uses one of the reference lines included in the selected reference line group for prediction. In this implementation, (implementation 1-1) the reference line group may be signaled by the video decoding device, or (implementation 1-2) inferred by the video decoding device.

[0166] <Implementation Method 1-1> Sending Reference Line Group Using Signal

[0167] In this implementation, in order to select one of the multiple reference line groups, the video encoding device sends a reference line group index (hereinafter, "ref_group_idx") indicating the reference line group to the video decoding device. For example, as shown in Table 3, if there are 3 reference line groups and "group 1" is selected, ref_group_idx 1 is sent by signal.

[0168] Subsequently, an index indicating which reference line is to be used for prediction among the reference lines included in the selected reference line group (i.e., a reference line candidate index) may be sent by signal (implementation 1-1-1) or inferred (implementation 1-1-2). The following describes a case where ref_group_candidate_idx indicates which reference line is to be used within the determined reference line group.

[0169] <Implementation Method 1-1-1> Signaling Reference Line Candidate Indexes

[0170] In this implementation, the video encoding device may signal a reference line candidate index (ref_group_candidate_idx) to indicate which reference line will be used within the selected reference line group. According to Table 3, if "group 1" is selected and ref_group_candidate_idx2 is signaled, intra_luma_ref_idx 8 is used for prediction of the current block. If there is only one reference line within the selected reference line group, the video decoding device may still parse ref_group_candidate_idx, or may omit parsing and may infer that ref_group_candidate_idx is 0.

[0171] <Implementation method 1-1-2> Inferring reference line candidate index

[0172] In this implementation, when ref_group_candidate_idx is inferred, the video decoding device may infer the ref_group_candidate_idx (i.e., reference line) used according to the block feature, or may use a reference line preset at a higher level (such as SPS, PPS, etc.). The block feature mentioned here may be at least one of block width, height, area, aspect ratio, and shape. These features are further described in detail below, and reference may be made to at least one of these features. In the following, the distance between a block and a reference line may be an index value of the reference line, the number of pixels between the two, the number of blocks between the two, etc.

[0173] First, the features of the current block may include position, prediction mode, reference pixels, any predictions that can be generated, the distance between the current block and the available reference lines included in the selected reference line group, and the pixel values ​​of the available reference lines. In addition, the features of the current block may include W, H, log related to width (W), height (H), area, and aspect ratio. 2 W, log 2 H.log 2 WH, WH, log 2 (W / H), W / H, log 2 (H / W), H / W, etc.

[0174] References to features of neighboring blocks of the current block may include features such as position, pixel values ​​from reconstructed blocks, prediction mode, reference lines used for prediction, reference line groups used, whether MRL is used, whether this implementation is used, information considered when using this implementation, reference pixels, any predictions that can be generated, distances between available reference lines and the current block, and pixel values ​​of available reference lines. Also referenced may be W, H, log related to width, height, area, and aspect ratio. 2 W, log2 H.log 2 WH, WH, log 2 (W / H), W / H, log 2 (H / W), H / W, etc.

[0175] References to features of a reconstructed block earlier than the current block may include position, pixel values ​​from the reconstructed block, prediction mode, reference lines used for prediction, reference line groups used, whether MRL is used, whether this implementation is used, information considered when using this implementation, reference pixels, any predictions that can be generated, distances between available reference lines and the current block, and pixel values ​​of available reference lines. References may also include W, H, log related to width, height, area, and aspect ratio. 2 W, log 2 H.log 2 WH, WH, log 2 (W / H), W / H, log 2 (H / W), H / W, etc.

[0176] The block collocated with the current block in another referenceable picture and the neighboring blocks of the collocated block have features that can be referenced, such as position, pixel values ​​from the reconstructed block, prediction mode, reference line used for prediction, reference line group used, whether MRL is used, whether this implementation is used, information considered when this implementation is used, reference pixels, any predictions that can be generated, the distance between the available reference line and the current block, and the pixel values ​​of the available reference line. In addition, references can be W, H, log related to width, height, area and aspect ratio. 2 W, log 2 H.log 2 WH, WH, log 2 (W / H), W / H, log 2 (H / W), H / W, etc.

[0177] When the current block uses the intra mode in the inter slice, the feature of referring to the collocated block using neighboring blocks of the current block and the collocated block in another referenceable picture may be applied more preferably or exclusively according to an embodiment.

[0178] An example of inferring ref_group_candidate_idx based on the characteristics of a block to be referenced is as follows.

[0179] For example, if an area is referenced as a block feature, the larger the area of ​​the block, the larger the reference line in its ref_group_candidate_idx (which is the value of the index in the selected reference line group) can be used for prediction. Alternatively, a reference line with a smaller index value can be used for prediction. Optionally, when using a reference line predetermined at a higher level (such as SPS, PPS, etc.), the video decoding device can use a reference line indicated by sps_ref_group_candidate_idx, pps_ref_group_candidate_idx, etc. from a reference line group selected by all CUs or some CUs, respectively, for prediction. For example, if sps_ref_group_candidate_idx is 1, a reference line with ref_group_candidate_idx1 in the reference line group can always be used.

[0180] The syntax elements according to this embodiment are as follows. At least one of these or multiple syntax elements may be used.

[0181] The selective MRL flag (hereinafter used interchangeably with "selective_mrl_flag") is a flag indicating whether selective MRL is applied, and can have values ​​of 0 and 1. If the flag is 0, intra_luma_ref_idx 0 is used for prediction, and if the flag is 1, the reference line group is signaled by the implementation to determine which reference line to use. If selective_mrl_flag is not present, it can be inferred that selective_mrl_flag is 0.

[0182] ref_group_idx indicates a determined reference line group, and may have a value of 0 or greater.

[0183] ref_group_candidate_idx is a reference line candidate index indicating a reference line to be used within the determined reference line group. ref_group_candidate_idx indicates a selected reference line within the reference line group, and may have a value of 0 or greater.

[0184] The preferred pseudo code according to this implementation can be implemented as follows.

[0185]

[0186] Here, the intra prediction mode or the selective MRL flag may be parsed first. If the reference line is determined by signaling the reference line, ref_group_candidate_idx is parsed, and if ref_group_candidate_idx is inferred, ref_group_candidate_idx is not parsed.

[0187] According to the above pseudo code, the syntax for transmission is shown in Table 4. In Table 4, the selective MRL flag is parsed first, and then the reference line candidate index is parsed from the determined reference line group.

[0188]

Table 4

[0189]

[0190] <Implementation Method 1-2> Infer Reference Line Group

[0191] In this implementation, the video decoding device infers one of the multiple reference line groups. According to the block characteristics, a reference line group can be selected (implementation 1-2-1), or a preset reference line group can be selected (implementation 1-2-2) for use.

[0192] <Implementation method 1-2-1> Select reference line group based on block features

[0193] In this implementation, the video decoding device selects one of the multiple reference line groups based on the block feature. The block features that can be referenced are W, H, log, and W related to width, height, area, and aspect ratio. 2 W, log 2 H.log 2 WH, WH, log 2 (W / H), W / H, log 2 (H / W), H / W. These features are further described below, and reference may be made to at least one of these features. In the following, the distance between a block and a reference line may be an index value of the reference line, the number of pixels between the two, the number of blocks between the two, etc.

[0194] First, the features of the current block may include position, prediction mode, reference pixels, any predictions that can be generated, the distance between the current block and the available reference lines included in the selected reference line group, and the pixel values ​​of the available reference lines. In addition, the features of the current block may include W, H, log related to width (W), height (H), area, and aspect ratio. 2 W, log 2 H.log 2 WH, WH, log 2 (W / H), W / H, log2 (H / W), H / W, etc.

[0195] Reference may be made to the features of the neighboring blocks of the current block described in implementation manner 1-1-2.

[0196] Reference may be made to features of a reconstructed block that is earlier in time than the current block described in implementation 1-1-2.

[0197] Reference may also be made to features of a block collocated with the current block and neighboring blocks of the collocated block in another reference picture, as described in implementation method 1-1-2.

[0198] When the current block uses the intra mode in the inter slice, the feature of referring to the collocated block using neighboring blocks of the current block and the collocated block in another referenceable picture may be applied more preferably or exclusively according to an embodiment.

[0199] An example of inferring a reference line group based on the features of a reference block is as follows. For example, if the area of ​​the block (log 2 WH) is referenced as a block feature, then the reference line group can be determined differently according to the area of ​​the current block, as shown in Table 5.

[0200]

Table 5

[0201]

[0202] Fig.10a and Fig.10b is a diagram illustrating inference of a reference line group according to at least one embodiment of the present disclosure.

[0203] exist Fig.10a In the example, because the current block has a width of 8 and a height of 16, and log 2 WH=7, so the reference line group is determined as {intra_luma_ref_idx 0, intra_luma_ref_idx1, intra_luma_ref_idx2} according to Table 5. In addition, a reference line is selected from the reference line group and used for prediction.

[0204] exist Fig.10b In the example, because the current block has a width of 16 and a height of 16, and log 2 WH=8, so the reference line group is determined as {intra_luma_ref_idx 0, intra_luma_ref_idx 1} according to Table 5. In addition, a reference line is selected from the reference line group and used for prediction.

[0205] Then, among the reference lines included in the selected one reference line group, the reference line to be used for prediction (i.e., the reference line candidate index) can be (implementation 1-2-1-1) signaled or (implementation 1-2-1-2) inferred. The following describes the case where ref_group_candidate_idx indicates which reference line to use within the determined reference line group.

[0206] <Implementation method 1-2-1-1> Signaling reference line candidate indexes

[0207] In this implementation, the video encoding device may signal a reference line candidate index ref_group_candidate_idx to indicate which reference line is to be used within the selected reference line group. According to Table 3, if "group 1" is selected and ref_group_candidate_idx 2 is signaled, intra_luma_ref_idx 8 is used for prediction of the current block. If there is only one reference line within the selected reference line group, the video decoding device may still parse ref_group_candidate_idx, or may omit parsing and may infer that ref_group_candidate_idx is 0.

[0208] <Implementation method 1-2-1-2> Inferring reference line candidate index

[0209] In this implementation, when ref_group_candidate_idx is inferred, the video decoding device may infer the ref_group_candidate_idx (i.e., reference line) used according to the block feature, or may use a reference line preset at a higher level (such as SPS, PPS, etc.). The block feature mentioned here may be at least one of block width, height, area, aspect ratio, and shape. These features are further described in detail below, and reference may be made to at least one of these features. In the following, the distance between a block and a reference line may be an index value of the reference line, the number of pixels between the two, the number of blocks between the two, etc.

[0210] First, the references as features of the current block may include position, prediction mode, reference pixels, any predictions that can be generated, the distance between the current block and the available reference lines included in the reference line group determined according to a predetermined method, and the pixel values ​​of the available reference lines. In addition, references may include W, H, log related to width (W), height (H), area, and aspect ratio. 2 W, log 2 H.log 2 WH, WH, log 2 (W / H), W / H, log 2(H / W), H / W, etc.

[0211] Reference may be made to the features of the neighboring blocks of the current block described in implementation manner 1-1-2.

[0212] Reference may be made to the features of the reconstructed block of the current block described earlier in Implementation 1-1-2.

[0213] You may also refer to features of blocks collocated with the current block and blocks adjacent to the collocated blocks in other reference pictures, as described in implementation method 1-1-2.

[0214] When the current block uses the intra mode in the inter slice, the feature of referring to the collocated block using neighboring blocks of the current block and the collocated block in another referenceable picture may be applied more preferably or exclusively according to an embodiment.

[0215] An example of inferring ref_group_candidate_idx based on the characteristics of a block to be referenced is as follows.

[0216] For example, if an area is referenced as a block feature, the larger the area of ​​the block, the larger the reference line in its ref_group_candidate_idx (which is the value of the index in the selected reference line group) can be used for prediction. Alternatively, a reference line with a smaller index value can be used for prediction. Optionally, when using a reference line determined at a higher level (such as SPS, PPS, etc.), the video decoding device can use a reference line indicated by sps_ref_group_candidate_idx, pps_ref_group_candidate_idx, etc. from a reference line group selected by all CUs or some CUs, respectively, for prediction. For example, if sps_ref_group_candidate_idx is 1, a reference line with ref_group_candidate_idx1 in the reference line group can always be used.

[0217] The syntax elements according to the implementation are as follows. At least one of these or more syntax elements may be used.

[0218] selective_mrl_flag is a flag indicating whether selective MRL is applied and can have values ​​of 0 and 1. If the flag is 0, intra_luma_ref_idx 0 is used for prediction, and if the flag is 1, the reference line group is inferred by the implementation to determine which reference line to use. If selective_mrl_flag is not present, selective_mrl_flag can be inferred to be 0.

[0219] ref_group_candidate_idx is a reference line candidate index indicating a reference line to be used within the determined reference line group. ref_group_candidate_idx indicates a selected reference line within the reference line group and may have a value of 0 or greater.

[0220] The preferred pseudo code according to this implementation can be implemented as follows.

[0221]

[0222]

[0223] Here, the intra prediction mode or the selective MRL flag may be parsed first. If the reference line is determined by signaling the reference line, ref_group_candidate_idx is parsed, and if the reference line is inferred, ref_group_candidate_idx is not parsed.

[0224] According to the above pseudo code, the syntax for transmission is shown in Table 6. In Table 6, the selective MRL flag is parsed first, and then the reference line candidate index is parsed from the determined reference line group.

[0225]

Table 6

[0226]

[0227] <Implementation method 1-2-2> Using preset reference line groups

[0228] In this implementation, the video decoding device uses a preset reference line group. One of the K reference line groups is set at a higher level (such as SPS, PPS, etc.), and the preset reference line group can be used in common for all CUs or some CUs. pps_ref_group_idx indicates a reference line group determined at the PPS, and sps_ref_group_idx indicates a reference line group determined at the SPS. As an example, if sps_ref_group_idx 1 is signaled, "group 1" may be selected.

[0229] Thereafter, among the reference lines included in a preset reference line group, the reference line to be used for prediction (i.e., the reference line candidate index) may be (implementation 1-2-2-1) signaled or (implementation 1-2-2-2) inferred. The following is a description of the case where ref_group_candidate_idx indicates which reference line to use within the determined reference line group.

[0230] <Implementation method 1-2-2-1> Signaling reference line candidate indexes

[0231] In this implementation, the video encoding device may signal a reference line candidate index ref_group_candidate_idx to indicate which reference line is to be used within the selected reference line group. In Table 3, if "group 1" is selected and ref_group_candidate_idx 2 is signaled, intra_luma_ref_idx 8 is used for prediction of the current block. If there is only one reference line within the selected reference line group, the video decoding device may still parse ref_group_candidate_idx, or may omit parsing and may infer that ref_group_candidate_idx is 0.

[0232] <Implementation method 1-2-2-2> Inferring reference line candidate index

[0233] In this implementation, when inferring ref_group_candidate_idx, the video decoding device may infer the ref_group_candidate_idx (i.e., reference line) used according to the block feature, or may use a reference line preset at a higher level (such as SPS, PPS, etc.). The block feature mentioned here may be at least one of block width, height, area, aspect ratio, and shape. These features are further described below, and reference may be made to at least one of these features. In the following, the distance between a block and a reference line may be an index value of the reference line, the number of pixels between the two, the number of blocks between the two, etc.

[0234] First, the features of the current block may include a position, a prediction mode, a reference pixel, any prediction that can be generated, a distance between the current block and an available reference line included in a selected reference line group determined by a predetermined method, and a pixel value of the available reference line. In addition, the features of the current block may include W, H, log related to the width (W), height (H), area, and aspect ratio. 2 W, log 2 H.log 2 WH, WH, log 2 (W / H), W / H, log 2 (H / W), H / W, etc.

[0235] Reference may be made to the features of the neighboring blocks of the current block described in implementation manner 1-1-2.

[0236] Reference may be made to the features of the reconstructed block of the current block described earlier in Implementation 1-1-2.

[0237] Reference may also be made to features of a block collocated with the current block and neighboring blocks of the collocated block in another reference picture, as described in implementation method 1-1-2.

[0238] When the current block uses the intra mode in the inter slice, the feature of referring to the collocated block using neighboring blocks of the current block and the collocated block in another referenceable picture may be applied more preferably or exclusively according to an embodiment.

[0239] An example of inferring a reference line group based on the characteristics of the block being referenced is as follows.

[0240] For example, if an area is referenced as a block feature, the larger the area of ​​the block, the larger the reference line in its ref_group_candidate_idx (which is the value of the index in the selected reference line group) can be used for prediction. Alternatively, a reference line with a smaller index value can be used for prediction. Optionally, when using a reference line determined at a higher level (such as SPS, PPS, etc.), the video decoding device can use a reference line indicated by spsref_group_candidate_idx, pps_ref_group_candidate_idx, etc. from a reference line group selected by all CUs or some CUs, respectively, for prediction. For example, if sps_ref_group_candidate_idx is 1, a reference line with ref_group_candidate_idx1 in the reference line group can always be used.

[0241] The syntax elements according to the implementation are as follows. At least one of these or more syntax elements may be used.

[0242] selective_mrl_flag is a flag indicating whether selective MRL is applied, and can have values ​​of 0 and 1. If the flag is 0, intra_luma_ref_idx0 is used for prediction, and if the flag is 1, the reference line group is inferred by the implementation to determine which reference line to use. If selective_mrl_flag is not present, selective_mrl_flag can be inferred to be 0.

[0243] ref_group_candidate_idx is a reference line candidate index indicating a reference line to be used within the determined reference line group. ref_group_candidate_idx indicates a selected reference line within the reference line group and may have a value greater than or equal to 0.

[0244] The preferred pseudo code according to this implementation may be the same as the pseudo code of implementation 1-2-1. Here, the intra prediction mode or the selective MRL flag may be parsed first. If the reference line is determined by signaling the reference line, ref_group_candidate_idx is parsed, and if the reference line is inferred, ref_group_candidate_idx is not parsed. In addition, the syntax for sending may be the same as Table 6, which shows the syntax of implementation 1-2-1.

[0245] <Implementation method 2> Generate a new reference line

[0246] In this implementation, in order to selectively use multiple reference lines, the video decoding apparatus generates a new reference line by weighted combination of the multiple reference lines. The video decoding apparatus may weighted combine the reference lines according to Equation 2.

[0247] [Equation 2]

[0248] luma_ref_line new =w i ×luma_ref_line i +w j ×luma_ref_line j

[0249] Here, luma_ref_line i represents the reference line indicated by intra_luma_ref_idx i, and luma_ref_line new represents the new reference line generated by the weighted combination. In addition, the sum of the weights satisfies 1 (W i +w j =1, where each weight value may be 0). In equation 2, two reference lines are weighted and combined, but this is not necessarily the case. n (n>2) reference lines may also be weighted and combined.

[0250] Fig.11 is a diagram illustrating a weighted combination of two reference lines according to at least one embodiment of the present disclosure.

[0251] When weighting and combining reference lines, the present disclosure may consider the directionality of the intra-frame prediction mode to select pixels of each reference line, which are used to calculate the value of each pixel in the new reference line. For example, describing the case where the value of the x coordinate increases from left to right for the x axis in the horizontal direction, and the value of the y coordinate increases from top to bottom for the y axis in the vertical direction, the prediction mode is the vertical prediction mode (prediction mode 50), and the position of the upper left pixel of the current block is (x0, y0). Fig.11As shown in the example of , in each reference line, a new pixel is generated by weighted combination of pixels at positions (x, y0-1) and (x, y0-3), where x0≤x≤x0+W-1. The pixels are indicated by a vertical pattern. The following describes a method for generating a new reference line, including (implementation method 2-1) a method for determining multiple reference lines and (implementation method 2-2) a method for determining weights.

[0252] <Implementation Method 2-1> Determine multiple reference lines

[0253] In this implementation, the video decoding device determines multiple reference lines for weighted combination. For this purpose, the video decoding device may (implementation 2-1-1) use a reference line group or may (implementation 2-1-2) not use a reference line group.

[0254] <Implementation method 2-1-1> Using reference line group

[0255] In this implementation, the video decoding device determines multiple reference lines by using a reference line group. The reference line group can be selected according to implementation 1. The example methods used may include (implementation 1-1) sending a reference line group with a signal, (implementation 1-2-1) inferring a reference line group based on block features, and (implementation 1-2-2) using a preset reference line group. Depending on implementation 1, the corresponding methods can be implemented with their required syntax.

[0256] As in implementation 1, whether selective MRL is applied may be indicated by a selective MRL flag, selective_mrl_flag. If the flag is 0, indicating that this implementation of weighted combination of multiple reference lines is not used for prediction in the current block, intra_luma_ref_idx 0 may be used as a single reference line. Optionally, intra_luma_ref_idx may be further parsed to signal a single reference line for prediction from multiple reference lines. After determining a reference line group according to implementation 1, the video decoding device may use all reference lines within the reference line group for weighted combination.

[0257] Example Consider the case where the reference line group is determined by signaling the reference line group according to (implementation 1-1). If there are 3 reference line groups as shown in Table 3, and ref_group_idx is signaled as 1, the reference lines indicated by the 3 reference line indices intra_luma_ref_idx 0, intra_luma_ref_idx4, and intra_luma_ref_idx 8 included in "group 1" can be used for weighted combination, as shown in Equation 3.

[0258] [Equation 3]

[0259] luma_ref_line new =w 0 ×luma_ref_line o +w 4 ×luma_ref_line 4 +w 8 ×luma_ref_line s

[0260] <Implementation method 2-1-2> Not using reference line group

[0261] In this implementation, the video decoding device determines multiple reference lines without using a reference line group. This can be done by (implementation 2-1-2-1) sending multiple reference lines with a signal, (implementation 2-1-2-3) using reference lines determined by block features, or (implementation 2-1-2-3) using predetermined multiple reference lines.

[0262] <Implementation method 2-1-2-1> Send multiple reference lines using signals

[0263] In this implementation, the video encoding device signals multiple reference lines to the video decoding device. The video decoding device can determine which reference lines to use to generate new reference lines by first parsing num_refLine (which represents the number of multiple reference lines to be used for prediction of each block at the CU level) and then parsing the reference line index by this value. In this case, num_refLine may not be signaled and a fixed value may be used regardless of the block. If num_refLine is 1, intra_luma_ref_idx 0 may be used as a single reference line because the prediction of the current block uses one reference line and excludes this implementation of a weighted combination of multiple reference lines. Optionally, intra_luma_ref_idx may be further parsed to signal a single reference line for prediction among the multiple reference lines.

[0264] The syntax elements according to the implementation are as follows. At least one of these or more syntax elements may be used.

[0265] num_refLine indicates the number of reference lines to be used in prediction and has a value of 1 or greater. When num_refLine is 1, intra_luma_ref_idx 0 can be stably used as a single reference line, or intra_luma_ref_idx can be further parsed to select a reference line from multiple reference lines for prediction. If num_refLine is not 1, the video decoding device parses the reference line index by the size of num_refLine. If num_refLine does not exist, it can be inferred that num_refLine is 1.

[0266] intra_luma_ref index indicates as many reference line indices as num_refLine to be used for weighted combination of reference lines. Each index is 0 or greater and can have different values.

[0267] intra_luma_ref_idx indicates one reference line index of multiple reference lines. In this implementation, intra_luma_ref_idx may be signaled when num_refLine is 1. intra_luma_ref_idx may have different values ​​of 0 or greater.

[0268] The preferred pseudo code according to this implementation can be implemented as follows.

[0269]

[0270]

[0271] Here, when num_refLine (the number of multiple reference lines) is used as a fixed value regardless of the block, the parsing of num_refLine is omitted and a predetermined value is used as num_refLine. The intra prediction mode or information about the reference line may be parsed first. The pseudo code described above is an example of parsing intra_luma_ref_idx to determine the reference line when num_refLine is 1. Thus, if num_refLine is 1 and intra_luma_ref_idx 0 is stably used, the parsing of the syntax may be omitted.

[0272] According to the above pseudo code, the syntax for transmission is shown in Table 7. In Table 7, the information of the reference line is parsed first, and if num_refLine is 1, the reference line is determined by parsing intra_luma_ref_idx.

[0273]

Table 7

[0274]

[0275] <Implementation method 2-1-2-2> Use the number of reference lines determined by the block characteristics

[0276] In this implementation, the video decoding device uses a predetermined number of reference lines based on block characteristics. For example, the implementation may consider the distance between the reference line used for prediction and the block side facing the reference line. To this end, the height of the block is considered for vertical mode 50 or higher prediction modes that use the upper reference line for prediction, and the width of the block is considered for horizontal mode 18 or lower prediction modes that use the left reference line for prediction. For prediction modes that use both the upper reference line and the left reference line for prediction that are larger than the horizontal mode 18 and smaller than the vertical mode 50, the larger of the width and height of the block may be considered. The video decoding device may determine multiple reference lines based on the width or height of the block under consideration, so that the larger the distance, the more reference lines can be used to improve the prediction accuracy, as shown in Table 8. Optionally, the smaller the distance, the fewer reference lines can be used.

[0277]

Table 8

[0278]

[0279] In addition, at least one of the area, prediction mode, and aspect ratio of the block may be referenced as a block feature to determine multiple reference lines to be used for prediction. These features are further described below, and at least one of these features may be referenced. In the following, the distance between a block and a reference line may be an index value of the reference line, the number of pixels between the two, the number of blocks between the two, etc.

[0280] First, the references to the features of the current block may include the position, prediction mode, reference pixels, any predictions that can be generated, the distance between the available reference line and the current block, and the pixel values ​​of the available reference lines. In addition, the references may be W, H, log related to the width (W), height (H), area, and aspect ratio. 2 W, log 2 H.log 2 WH, WH, log 2 (W / H), W / H, log 2 (H / W), H / W, etc.

[0281] References to the features of the neighboring blocks of the current block may include the position, pixel values ​​from the reconstructed block, prediction mode, reference lines for prediction, whether to use MRL, whether to use this implementation, information considered when using this implementation, reference pixels, any predictions that can be generated, the distance between the available reference lines and the current block, and features of the pixel values ​​of the available reference lines. In addition, references may include W, H, log related to width, height, area, and aspect ratio.2 W, log 2 H.log 2 WH, WH, log 2 (W / H), W / H, log 2 (H / W), H / W, etc.

[0282] References to features of a reconstructed block earlier than the current block may include position, pixel values ​​from the reconstructed block, prediction mode, reference lines used for prediction, whether to use MRL, whether to use this implementation, information considered when using this implementation, reference pixels, any predictions that can be generated, distances between available reference lines and the current block, and pixel values ​​of available reference lines. References may also include W, H, log related to width, height, area, and aspect ratio. 2 W, log 2 H.log 2 WH, WH, log 2 (W / H), W / H, log 2 (H / W), H / W, etc.

[0283] The block collocated with the current block in another referenceable picture and the neighboring blocks of the collocated block have features that can be referenced, such as position, pixel values ​​from the reconstructed block, prediction mode, reference line for prediction, whether to use MRL, whether to use this implementation, information considered when using this implementation, reference pixels, any predictions that can be generated, the distance between the available reference line and the current block, and the pixel value of the available reference line. In addition, references can be W, H, log related to width, height, area and aspect ratio. 2 W, log 2 H.log 2 WH, WH, log 2 (W / H), W / H, log 2 (H / W), H / W, etc.

[0284] When the current block uses the intra mode in the inter slice, the feature of referring to the collocated block using neighboring blocks of the current block and the collocated block in another referenceable picture may be applied more preferably or exclusively according to an embodiment.

[0285] As in implementation 1, whether selective MRL is applied may be indicated by a selective MRL flag, selective_mrl_flag. If the flag is 0, indicating that this embodiment of weighted combination of multiple reference lines is not used in the prediction of the current block, intra_luma_ref_idx0 may be used as a single reference line. Optionally, intra_luma_ref_idx may be further parsed to signal a single reference line for prediction from multiple reference lines.

[0286] The syntax elements according to the implementation are as follows. At least one of these or more syntax elements may be used.

[0287] selective_mrl_flag is a flag indicating whether selective MRL is applied, and can have values ​​of 0 and 1. If the flag is 0, intra_luma_ref_idx 0 can be stably used for prediction, or intra_luma_ref_idx can be further parsed to select one reference line to be used for prediction from multiple reference lines. If the flag is 1, the reference line group is signaled according to the implementation to determine which reference line to use. If selective_mrl_flag is not present, it can be inferred that selective_mrl_flag is 0.

[0288] intra_luma_ref_idx indicates one reference line index of multiple reference lines. In this implementation, if selectivemrl_flag is 0, intra_luma_ref_idx may be signaled. intra_luma_ref_idx may have different values ​​of 0 or greater.

[0289] The preferred pseudo code according to this implementation can be implemented as follows.

[0290]

[0291] Here, the intra prediction mode or the selective MRL flag may be parsed first. The above pseudo code is an example of parsing intra_luma_ref_idx to determine the reference line when selective_mrl_flag is 0. Thus, if selective_mrl_flag is 0 and intra_luma_ref_idx 0 is stably used, the parsing of the syntax may be omitted.

[0292] According to the above pseudo code, the syntax for transmission is shown in Table 9. In Table 9, the selective MRL flag is parsed first, and if selective_mrl_flag is 0, intra_luma_ref_idx is parsed to determine the reference line.

[0293]

Table 9

[0294]

[0295]

[0296] <Implementation method 2-1-2-3> Using predefined multiple reference lines

[0297] In this implementation, the video decoding apparatus uses multiple reference lines defined at a higher level such as SPS, PPS, etc. For example, a new reference line may be generated by weighted combination of two reference lines intra_luma_ref_idx 0 and intra_luma_ref_idx 2, as shown in Equation 4.

[0298] [Equation 4]

[0299] luma_ref_line new =w 0 ×luma_ref_line 0 +w 2 ×luma_ref_line 2

[0300] As in implementation 1, whether selective MRL is applied may be indicated by a selective MRL flag, selective_mrl_flag. If the flag is 0, indicating that this implementation of weighted combination of multiple reference lines is not used for prediction in the current block, intra_luma_ref_idx 0 may be used as a single reference line. Optionally, intra_luma_ref_idx may be further parsed to signal a single reference line among the multiple reference lines to be used for prediction.

[0301] The syntax elements according to the implementation are as follows. At least one of these or more syntax elements may be used.

[0302] selective_mrl_flag is a flag indicating whether selective MRL is applied, and can have values ​​of 0 and 1. If the flag is 0, intra_luma_ref_idx 0 can be stably used for prediction, or intra_luma_ref_idx can be further parsed to select one reference line to be used for prediction from multiple reference lines. If the flag is 1, the reference line group is signaled according to the implementation to determine which reference line to use. If selective_mrl_flag is not present, it can be inferred that selective_mrl_flag is 0.

[0303] intra_luma_ref_idx indicates one reference line index of multiple reference lines. In this implementation, if selective_mrl_flag is 0, intra_luma_ref_idx may be signaled. intra_luma_ref_idx may have different values ​​of 0 or greater.

[0304] The preferred pseudocode according to this implementation may be the same as the pseudocode of implementation 2-1-2-2. Here, the intra prediction mode or the selective MRL flag may be parsed first. The pseudocode mentioned above is an example of parsing intra_luma_ref_idx to determine the reference line when selective_mrl_flag is 0. Therefore, if selective_mrl_flag is 0 and intra_luma_ref_idx 0 is used stably, the parsing of the relevant syntax may be omitted. In addition, the syntax used for sending may be the same as Table 6, which shows the syntax of implementation 2-1-2-2.

[0305] <Implementation Method 2-2> Determine the weight

[0306] In this implementation, the video decoding device determines the weight of each reference line for weighted combination of different reference lines. The following describes a method for determining weights, which is (implementation 2-2-1) a method using predefined weights and a method for sending weights with a signal.

[0307] <Implementation 2-2-1> Using predefined weights

[0308] In this implementation, the video decoding device uses predefined weights. The predefined weights may include equal weights for each reference line, for example, 1:1 for two reference lines, 1:1:1 for three reference lines, etc., or the closer the distance between the reference line and the current block, the higher the weight, for example, 3:1 for two reference lines, 2:1:1 for three reference lines, etc.

[0309] For example, when a new reference line is generated based on two reference lines (intra_luma_ref_idx 0 and intra_luma_ref_idx 2), all reference lines may be equally weighted as shown in Equation 5.

[0310] [Equation 5]

[0311]

[0312] Alternatively, as shown in Equation 6, a higher weight may be set for reference lines that are closer to the current block.

[0313] [Equation 6]

[0314]

[0315] <Implementation method 2-2-1> Send weights using signals

[0316] In this implementation, the video encoding device signals the weights to the video decoding device.

[0317] When multiple reference lines are determined according to implementation 2-1-2-1, the video encoding device signals the multiple reference line indexes for each block at the CU level, and then signals the weights used to generate new reference lines based on those reference lines. The weights may be represented by generateref_weight, which may be used to signal the appropriate weight value for each reference line.

[0318] If multiple reference lines are determined without signaling, such as in implementation 2-1-2-2 or implementation 2-1-2-3, the video encoding device may signal the weights in order (i.e., in an order close to the current block) starting from the weight corresponding to the minimum value in the multiple reference line index. Alternatively, the weights may be signaled in reverse order.

[0319] If multiple reference lines are determined by using a reference line group, such as in implementation 2-1-1, the video encoding device signals the weight of each reference line in the selected reference line group in a random order. The random order may be a descending or ascending order of ref_group_candidate_idx, an order of reference lines arranged in a reference line group, or a descending or ascending order of intra_luma_ref_idx values ​​representing index values ​​of corresponding reference lines. In addition, if there is only one reference line in the selected reference line group, the weight may be regarded as 1 without being sent by signal, which is equivalent to performing prediction with a single reference line.

[0320] The syntax elements according to the implementation are as follows. At least one of these or more syntax elements may be used.

[0321] selective_mrl_flag is a flag indicating whether selective MRL is applied, and can have values ​​of 0 and 1. If the flag is 0, intra_luma_ref_idx 0 can be stably used for prediction, or intra_luma_ref_idx can be further parsed to select one reference line to be used for prediction from multiple reference lines. If the flag is 1, the implementation determines multiple reference lines. If selective_mrl_flag does not exist, it can be inferred that selective_mrl_flag is 0.

[0322] num_refLine indicates the number of reference lines to be used for prediction, and has a value of 1 or greater. When num_refLine is 1, intra_luma_ref_idx 0 can be stably used as a single reference line, or intra_luma_ref_idx can be further parsed to select a reference line from multiple reference lines for prediction. If num_refLine is not 1, the video decoding device parses the reference line index by the size of num_refLine. If num_refLine does not exist, it can be inferred that num_refLine is 1.

[0323] intra_luma_ref index represents as many reference line indices as num_refLine used for prediction. Each index is 0 or greater and can have different values.

[0324] intra_luma_ref_idx indicates a reference line index of multiple reference lines. In this implementation, when multiple reference lines are determined according to implementation 2-1-2-1 and num_refLine is 1, intra_luma_ref_idx may be signaled. Alternatively, when multiple reference lines are determined and do not need to be signaled as in implementation 2-1-2-2 or implementation 2-1-2-3 and selective_mrl_flag is 0, intra_luma_ref_idx may be signaled. intra_luma_ref_idx may have different values ​​of 0 or greater.

[0325] ref_group_idx indicates a determined reference line group, and may have a value of 0 or greater.

[0326] generateref_weight indicates the weight that each reference line uses for weighted combination to generate the new reference line.

[0327] When multiple reference lines are signaled and determined according to implementation 2-1-2-1, a preferred pseudo code according to this implementation may be implemented as follows.

[0328]

[0329] Here, if num_refLine (the number of reference lines) is used as a fixed value regardless of the block, the parsing of num_refLine is omitted and a predetermined value is used as num_refLine. The intra prediction mode or information about the reference line may be parsed first. The above pseudo code is an example of parsing intra_luma_ref_idx to determine the reference line when num_refLine is equal to 1. Thus, if num_refLine is 1 and intra_luma_ref_idx 0 is used stably, the parsing of the syntax may be omitted.

[0330] According to the above pseudo code, the syntax for transmission is shown in Table 10. In Table 10, information about the reference line is parsed first, and if num_refLine is 1, intra_luma_ref_idx is parsed to determine the reference line.

[0331]

Table 10

[0332]

[0333] When multiple reference lines are determined without signaling as in implementation 2-1-2-2 or implementation 2-1-2-3, a preferred pseudo code according to the implementation may be implemented as follows.

[0334]

[0335] Here, the intra prediction mode or the selective MRL flag may be parsed first. The above pseudo code is an example of parsing intra_luma_ref_idx to determine the reference line when selective_mrl_flag is 0. Thus, if selective_mrl_flag is 0 and intra_luma_ref_idx 0 is stably used, the parsing of the syntax may be omitted.

[0336] According to the above pseudo code, the syntax for transmission is shown in Table 11. In Table 11, the selective MRL flag is parsed first, and if selective_mrl_flag is 0, intra_luma_ref_idx is parsed to determine the reference line.

[0337]

Table 11

[0338]

[0339] When multiple reference lines are determined by using a reference line group as in implementation 2-1-1, a preferred pseudo code according to this implementation can be implemented as follows.

[0340]

[0341] Here, the intra prediction mode or selective MRL flag may be parsed first. If the reference line group is inferred, the parsing of the ref_group_idx syntax may be omitted. The above pseudo code is an example of parsing intra_luma_ref_idx to determine the reference line when selective_mrl_flag is 0. Thus, if selective_mrl_flag is 0 and intra_luma_ref_idx 0 is used stably, the parsing of the syntax may be omitted.

[0342] According to the above pseudo code, the syntax for transmission is shown in Table 12. In Table 12, the selective MRL flag is parsed first, and if selective_mrl_flag is 0, intra_luma_ref_idx is parsed to determine the reference line.

[0343]

Table 12

[0344]

[0345] <Implementation method 3> Control available reference lines

[0346] In this implementation, the video decoding device does not use all N available reference lines to predict the current block, but controls the use of some available reference lines based on specific conditions. For blocks that do not meet specific conditions, when using MRL, the video decoding device selects reference lines by applying the method of the prior art, the implementation of the present disclosure, or other methods using multiple reference lines. In addition, the video decoding device may limit the available reference lines for blocks that meet specific conditions. The following describes controlling the reference lines, which are (Implementation 3-1) a method using the prediction mode of the block and the position of the block in the video, and (Implementation 3-2) a method using the segmentation structure of the block.

[0347] <Implementation 3-1> Using the prediction mode of the block and the position of the block in the image

[0348] In this implementation, the video decoding device limits the reference line according to certain conditions based on the prediction mode of the block, the position of the block in the image, and all available information about them. If the current block is located at the boundary of the image, then according to Fig.12For the type of block shown, some or all of the reference pixels used for intra prediction can be filled to replace the reconstructed pixels. For example, for type 1 where the block is located in the upper left corner of the image and is tangent to the left and upper boundaries of the image, all reference pixels in the current block are generated based on padding and all have the same pixel value. For type 2 where the block is adjacent to the left boundary of the image, the reference pixels on the left reference line of the current block are generated based on padding and all have the same pixel value. In addition, for type 3 where the block is adjacent to the upper boundary of the image, the reference pixels on the upper reference line of the current block are generated based on padding and all have the same pixel value. In this case, padding is applied to reference lines further away from the block and to reference lines adjacent to the block in the same principle, so that if blocks of type 1 to type 3 use multiple reference lines in the left reference line direction or the upper reference line direction, all of the multiple reference lines in that direction may have the same pixel value depending on the type. Because it is quite inefficient to use multiple reference lines in this case, the video decoding device can control the available reference lines.

[0349] In this implementation, if the type of the current block is type 1 to type 3 located at the boundary of the image, the video decoding device restricts the MRL technology from utilizing multiple reference lines. That is, the video decoding device uses the reference line indicated by intra_luma_ref_idx 0, which is a single reference line adjacent to the block, specifically for prediction. This implementation can be configured to be applied to 7 combinations of type 1 to type 3: only applied to type 1, only applied to type 2, only applied to type 3, only applied to type 1 / type 2, only applied to type 1 / type 3, only applied to type 2 / type 3, or applied to both type 1 and type 3. The image can be a picture, a sub-picture, a slice, a tile, a CTU, etc.

[0350] In syntax transmission according to this implementation, if the current block is located at the boundary of the picture, the video decoding device does not parse the signal associated with the reference line. For example, if MRL is applied in implementation 1-1 (a reference line group is determined by sending a new number, and one reference line in the reference line group is used for prediction), and the reference line is controlled according to this implementation (when applied to both type 1 and type 3 according to this implementation), the syntax elements according to this implementation are as follows. At least one or more of these syntax elements may be used.

[0351] selective_mrl_flag is a flag indicating whether selective MRL is applied, and is parsed when the type of the current block is not one of type 1 to type 3, and can have values ​​of 0 and 1. If the flag is 0, intra_luma_ref_idx 0 is used for prediction, and if the flag is 1, a reference line group is signaled to determine which reference line to use according to the implementation. If selective_mrl_flag does not exist, it can be inferred that selective_mrl_flag is 0.

[0352] ref_group_idx indicates a determined reference line group, and may have a value of 0 or greater.

[0353] ref_group_candidate_idx is a reference line candidate index indicating a reference line to be used within the determined reference line group. ref_group_candidate_idx indicates a selected reference line within the reference line group, and may have a value of 0 or greater.

[0354] The preferred pseudo code according to this implementation can be implemented as follows.

[0355]

[0356] Here, the intra prediction mode or the selective MRL flag may be parsed first. If the reference line is determined by signaling the reference line, ref_group_candidate_idx is parsed, and if the reference line is inferred, ref_group_candidate_idx is not parsed.

[0357] According to the above pseudo code, the syntax for transmission is shown in Table 13. In Table 13, the selective MRL flag is parsed first, and then the reference line candidate index is parsed from the determined reference line group.

[0358]

Table 13

[0359]

[0360] As another example, in addition to the position of the block in the image, the video decoding device may also use the intra prediction mode of the current block as a condition for controlling the reference line. If the current block is located at the boundary of the image and uses reference pixels that are not reconstructed along the prediction direction, the video decoding device restricts the MRL technology from using multiple reference lines. That is, the video decoding device uses the reference line intra_luma_ref_idx 0 indicated by a single reference line adjacent to the block exclusively for prediction.

[0361] For example, if the block type is type 1, the reference line can be controlled in all intra-frame prediction modes because all reference pixels in the current block are generated based on padding and all have the same pixel value. If the block type is type 2, the reference line can be restricted in the prediction mode using the left reference line because the reference pixels on the left reference line of the current block are generated based on padding and all have the same pixel value. In this case, the prediction mode using the left reference line includes a prediction mode using only the left reference line (① HOR_Idx (mode 18) or less) or a mode using both the left reference line and the upper reference line (② less than VER_Idx mode (mode 50) and greater than HOR_Idx mode (mode 18). This implementation can be applied when the current block type is type 2 and has prediction mode ①, or when the block type is type 2 and has prediction modes ① and ②, this implementation can be applied.

[0362] In addition, if the block type is type 3, the reference pixels of the upper reference line of the current block are generated based on padding and all have the same pixel value, so the reference line can be controlled in the prediction mode using the upper reference line. In this case, the prediction mode using the upper reference line includes a mode using both the left reference line and the upper reference line (②) or a prediction mode using only the upper reference line (③VER_Idx mode (mode 50 or above)). This implementation can be applied when the current block type is type 3 and has prediction modes ② and ③, or when the current block type is type 3 and has prediction mode ③. In addition, the implementation can be configured to be applied to 7 combinations of type 1 to type 3: only applied to type 1, only applied to type 2, only applied to type 3, only applied to type 1 / type 2, only applied to type 1 / type 3, only applied to type 2 / type 3, or applied to both type 1 and type 3.

[0363] For example, if the current block has type 2 and is in prediction mode 2 using only the left reference line, the current block does not use multiple reference lines and always uses intra_luma_ref_idx 0.

[0364] Therefore, in syntax transmission, if the current block is located at the boundary of the image and is in the prediction mode subjected to the implementation, the video decoding device does not parse the signal associated with the reference line. For example, if MRL is applied according to implementation 1-1 (a reference line group is determined by signaling, and one of the reference line groups is used for prediction) and the reference line is controlled according to the implementation (it is applied to h types 1 to 3, but is type 2 and applied to prediction mode ①, or type 3 and applied to prediction mode ③), the syntax elements according to the implementation are as follows. At least one or more of these syntax elements may be used.

[0365] selective_mrl_flag is a flag indicating whether selective MRL is applied, and can have values ​​of 0 and 1. If the type of the current block is not one of type 1 to type 3, or if each type does not satisfy the prediction mode applied by the implementation, the flag is parsed. If the flag is 0, intra_luma_ref_idx 0 is used for prediction, and if the flag is 1, a reference line group is signaled according to the implementation to determine which reference line to use. If selective_mrl_flag is not present, it can be inferred to be 0.

[0366] ref_group_idx indicates a determined reference line group, and may have a value of 0 or greater.

[0367] ref_group_candidate_idx is a reference line candidate index indicating a reference line to be used within the determined reference line group. ref_group_candidate_idx indicates a selected reference line within the reference line group, and may have a value of 0 or greater.

[0368] The preferred pseudo code according to this implementation can be implemented as follows.

[0369]

[0370] Here, the intra prediction mode is parsed first because the reference line is parsed according to the intra prediction mode. If the reference line is determined by signaling the reference line, ref_group_candidate_idx is parsed, and if the reference line is inferred, ref_group_candidate_idx is not parsed.

[0371] According to the above pseudo code, the transmission syntax is as shown in Table 14. In Table 14, the selective MRL flag is parsed first, and then the reference line candidate index is parsed from the determined reference line group.

[0372]

Table 14

[0373]

[0374]

[0375] <Implementation 3-2> Using the block partition structure

[0376] In this implementation, the video decoding device controls the available reference lines based on the partition structure of the block. Among the N possible reference lines, the reference can be limited to the reference lines outside the s (s≥1) blocks adjacent to the upper and left boundaries of the current block. In this case, the s blocks adjacent to the left boundary of the current block, the s blocks adjacent to the upper boundary, or both can be considered. For the convenience of describing the implementation, s is 1, but s can be any value of 1 or greater.

[0377] In addition to including Fig.13 In addition to the pixels at the specific positions shown in , the blocks adjacent to the left side boundary or the upper side boundary of the current block may include one of the pixels adjacent to each block boundary. In this case, the positions of the pixels defining the left or upper side adjacent blocks may be different. When s is 2 or greater, the i-th block (2≤i≤s) in each of the upper direction and the left side direction is adjacent to the i-1th block in order of increasing distance from the current block, and the i-1th block is the block immediately before the i-th block. In this case, the i-th block in the upper direction is defined as a block containing pixels shifted in the -y direction by the sum of the heights of the previous blocks, starting from the pixel position used to define the first adjacent block. The i-th block in the left direction is defined as a block containing pixels shifted in the -x direction by the sum of the widths of the previous blocks, starting from the pixel position used to define the first adjacent block. For example, Fig.13 Pixel position 1 in the example of can be used to define the upper neighboring block, and Fig.13 Pixel position 4 in the example of can be used to define the left neighboring block. In this case, if Fig.14 As shown in the example, multiple adjacent blocks on the upper side and the left side can be defined.

[0378] When only the s blocks adjacent to the left boundary of the current block are considered, the video decoding device according to the implementation does not refer to any reference line beyond the left block in the N available reference lines. That is, the range of the reference line index (intra_luma_ref_idx) value is from 0 to the width of the left block (1eftBlockW) -1 or less. If only the s blocks adjacent to the upper boundary of the current block are considered, the video decoding device according to the implementation does not refer to any reference line outside the upper block in the N available reference lines. That is, the range of the reference line index (intra_luma_ref_idx) value is from 0 to the height of the upper block (aboveBlockH) -1 or less.

[0379] In addition, when both the left block and the upper block are considered, the maximum value of the reference line index (intra_luma_ref_idx) that can be used for prediction can be determined based on the height of the upper block (aboveBlockH) and the width of the left block (leftBlockW). The smaller or larger of the two values ​​can be selected, or the average of the two values ​​can be used. For example, if the smaller of the two values ​​is selected, the reference line index (intra_luma_ref_idx) value can be in the range from 0 to min (leftBlockW-1, aboveBlockH-1) or less.

[0380] The following describes a method of controlling available reference lines according to this implementation for the benefit of the prior art MRL technology and implementation 1. In addition, other methods of using multiple reference lines can be implemented similarly to the following.

[0381] In one example, when the implementation is applied to the conventional MRL technology, the number of reference lines used for prediction for each block is set differently depending on the partition structure of the block. For example, when considering a single block adjacent to the left, a block whose left block has a width of 4 may select one of a total of 4 reference lines (intra_luma_ref_idx 0 to intra_luma_ref_idx 3) to perform prediction. In addition, a block whose left block has a width of 32 may select one of a total of 32 reference lines (intra_luma_ref_idx 0 to intra_luma_ref_idx 31) to perform prediction.

[0382] As another example, when the implementation is applied to the conventional MRL technology, the reference line groups signaled or inferred for each block are organized differently depending on the partition structure of the block. For example, for a single left-neighboring block, it is assumed that the 3 groups in Table 3 are feasible as the reference line groups used by the block, and it is assumed that the width of the left block of block A is 4, the width of the left block of block B is 8, and the width of the left block of block C is 16. In addition, it is assumed that all three blocks A, B, and C determine their reference line groups by signaling the reference line groups, and the refgroup_idx signaled in all three blocks is 1. According to Table 3, "Group 1" is {intra_luma_ref_idx 0, intra_luma_ref_idx 4, intra_luma_ref_idx 8}, but according to the implementation, "Group 1" for each of blocks A, B, and C may have a different configuration as shown in Table 15.

[0383]

Table 15

[0384]

[0385] The syntax according to the implementation can be organized to be the same as the syntax used in the applied MRL method (such as the MRL technology of the prior art, the implementation of the present disclosure, or any other method using multiple reference lines). However, if the information about the neighboring blocks is used as the basis for controlling the reference lines that can be used for the current block, the range of the syntax elements (such as intra_luma_ref_idx and ref_group_candidate_idx) representing the reference lines can be limited. Therefore, when encoding the syntax elements representing the reference lines, bits can be allocated differently based on the reference line control method applied. For example, if the traditional MRL method is used and unary decoding is applied to a case with 5 referenceable reference lines (intra_luma_ref_idx 0 to 4), the corresponding reference line indexes are represented by 0, 10, 110, 1110 and 1111. If a single left neighboring block is considered for reference line control and the left block has a width of 4, only 4 reference lines (intra_luma_ref_idx 0 to 3) are available for the current block for use. If unary encoding is applied to this case, the corresponding reference line indices are represented by 0, 10, 110, and 111, and thus the controlled reference line results in encoding differences of the reference line indices compared to the uncontrolled case.

[0386] <Implementation 4> Adaptively adjust information based on reference line index

[0387] In this implementation, the video decoding device adaptively adjusts information according to the reference line index of the code table as the reference line index and the reference line indicated by the reference line index. Here, the code table represents a table of binary codewords (used interchangeably with "codewords") for each reference line index. In this implementation, the video decoding device can (implementation 4-1) adaptively adjust the code table according to the reference line index, or (implementation 4-2) adaptively adjust the reference line indicated by the reference line index.

[0388] <Implementation Method 4-1> Adaptively Adjust the Code Table Based on the Reference Line Index

[0389] In this implementation, in order to adaptively adjust the code table according to the reference line index, (implementation method 4-1-1) is used to change the mapping between the preset coding table and the reference line, or (implementation method 4-1-2) is used to adaptively change the definition of the code table.

[0390] <Implementation Method 4-1-1> Change the mapping between the preset coding table and the reference line

[0391] In this implementation, when a codeword for encoding a reference line index is set in the code table, the video decoding device changes the codeword mapped to each reference line. For example, intra_luma_ref_idx 0 to 2 are set as 3 reference line indexes for indicating when 3 reference lines are available. If the codewords to be encoded for the 3 reference line indexes are 0, 10, and 11, the codeword mapped to each reference line index may be changed, as shown in Table 16.

[0392]

Table 16

[0393] Code table Case 0 Case 1 Case 2 … 0 intra_luma_ref_idx_0 intra_luma_ref_idx 0 intra_luma_ref_idx 2 … 10 intra_luma_ref_idx 1 intra_luma_ref_idx 2 intra_luma_ref_idx 1 … 11 intra_luma_ref_idx 2 intra_luma_ref_idx 1 intra_luma_ref_idx 0 …

[0394] Here, "case" distinguishes which intra_luma_ref_idx maps to a given codeword, which is equivalent to distinguishing the code table between intra_luma_ref_idx and codewords. If this implementation is applied to N reference lines, there may be a total of N! (N factorial) ways to map between reference line indices and codewords.

[0395] Meanwhile, the code table corresponding to each case can be expressed as shown in Table 17.

[0396]

Table 17

[0397] Code table 0

[0398] Codeword Guides 0 intra_luma_ref_idx0 10 intra_luma_ref_idx 1 11 intra_luma_ref_idx 2

[0399] Table 1

[0400] Codeword Guides 0 intra_luma_ref_idx 0 10 intra_luma_ref_idx 2 11 intra_luma_ref_idx 1

[0401] Table 2

[0402] Codeword Guides 0 intra_luma_ref_idx 2 10 intra_luma_ref_idx 1 11 intra_luma_ref_idx 0

[0403] …

[0404] As described above, in order to change the mapping between the reference line index and the code table, the video decoding device can (implementation method 4-1-1-1) parse and apply the mapping or (implementation method 4-1-1-2) infer the mapping without sending a signal.

[0405] <Implementation method 4-1-1-1> Sending mapping using signal

[0406] In this implementation, regardless of the parsing of the information (i.e., syntax) on the reference line and its order, the video decoding device can determine how to interpret the corresponding bitstream by parsing the mapping code index ("mapping_code_idx"). In this case, mapping_code_idx indicates one of the "cases" in Table 16 (or "code table" in Table 17) that represents the mapping between the reference line index and the codeword. For example, with mapping_code_idx 1, the reference line index represented by the codeword is set according to "case 1" (or "code table 1"). This method changes the way the information sent by the signal is interpreted, so that for MRL and other intra-frame predictions, the corresponding syntax elements or mapping_code_idx can be parsed first. In addition, mapping_code_idx can be signaled at the CU level or at a higher level such as SPS, PPS, etc.

[0407] As another example, in addition to signaling the code table indicated by intra_luma_ref_idx, a code table determination method may also be signaled. In this case, the code table determination method may include any method that can be used to infer the mapping in implementation 4-1-1-2 without signaling. The video decoding device may determine the code table determination method by parsing the mapping method index (hereinafter, "mapping_method_idx"). The method changes the way the information sent by the signal is interpreted so that for MRL and other intra predictions, the corresponding syntax element or mapping_method_idx can be parsed first. In addition, mapping_method_idx can be signaled at the CU level or at a higher level such as SPS, PPS, etc.

[0408] <Implementation 4-1-1-2> Inferring mapping without signaling

[0409] In this implementation, the video decoding device infers the mapping between the reference line index and the code table by using a bitstream interpretation method for inference without signaling. In order to map a shorter and / or entropy decoding-favorable codeword to a reference line index expected to be used in each block, the video decoding device may infer the above-mentioned "situation" (or "code table") based on specific information. The reference may be the block width, height, area, and aspect ratio (W, H, log 1) of the current block. 2 W, log 2 H.log 2 WH, WH, log 2 (W / H), W / H, log 2 At least one or more of information related to characteristics of (H / W, H / W, etc.) and information about reference lines used by neighboring blocks of the current block.

[0410] These features are further described in detail below, and reference may be made to at least one of these features. In the following, the distance between a block and a reference line may be an index value of the reference line, the number of pixels between the two, the number of blocks between the two, etc. In addition, both a method of directly indicating mapping and a method of determining a code table for allowing inference of mapping may be used as code table information between intra_luma_ref_idx and a codeword.

[0411] First, the features of the current block may include position, prediction mode, reference pixels, any predictions that can be generated, available reference lines, the distance between the available reference lines and the current block, and the pixel value of the available reference lines. In addition, the references may be W, H, log related to width (W), height (H), area, and aspect ratio. 2 W, log 2 H.log 2 WH, WH, log 2 (W / H), W / H, log 2 (H / W), H / W, etc.

[0412] References to the features of the neighboring blocks of the current block may include the position, the pixel value from the reconstructed block, the prediction mode, the reference line used for prediction, whether to use MRL, the code table information between the intra_luma_ref_idx used and the codeword, the code table information between the best intra_luma_ref_idx and the codeword for the corresponding block, the reference pixel, any prediction that can be generated, the available reference line, the distance between the available reference line and the current block, and the pixel value of the available reference line. In addition, references may include W, H, log related to width, height, area, and aspect ratio. 2 W, log 2 H.log 2 WH, WH, log 2 (W / H), W / H, log 2 (H / W), H / W, etc.

[0413] The features referenced as features of a reconstructed block earlier than the current block may be features such as position, pixel values ​​from the reconstructed block, prediction mode, reference lines used for prediction, whether to use MRL, code table information between the intra_luma_ref_idx used and the codeword, code table information between the best intra_luma_ref_idx and the codeword for the corresponding block, reference pixels, any predictions that can be generated, the distance between the available reference line and the current block, and the pixel values ​​of the available reference lines. In addition, references may be W, H, log related to width, height, area, and aspect ratio. 2W, log 2 H.log 2 WH, WH, log 2 (W / H), W / H, log 2 (H / W), H / W, etc.

[0414] The collocated blocks and adjacent blocks of the collocated blocks in another referenceable picture with the current block have features that can be referenced, such as position, pixel values ​​from the reconstructed blocks, prediction mode, reference lines for prediction, whether to use MRL, code table information between the intra_luma_ref_idx used and the codeword, code table information between the intra_luma_ref_idx and the codeword that is optimal for the corresponding block, reference pixels, any predictions that can be generated, the distance between the available reference line and the current block, and the pixel values ​​of the available reference lines. In addition, references may be W, H, log related to width, height, area, and aspect ratio. 2 W, log 2 H.log 2 WH, WH, log 2 (W / H), W / H, log 2 (H / W), H / W, etc.

[0415] When the current block uses the intra mode in the inter slice, the feature of referring to the collocated block using neighboring blocks of the current block and the collocated block in another referenceable picture may be applied more preferably or exclusively according to an embodiment.

[0416] An example of determining a code table or an inferred code table for interpreting a bitstream of intra_luma_ref_idx based on the characteristics of a block to be referenced is as follows.

[0417] As an example, in order to infer a "case" (or "code table") by using information about the reference line used by the neighboring block, if the left neighboring block is predicted with intra_luma_ref_idx 2, "case 2" (or "code table 2") (where the least bit is allocated to the encoding of intra_luma_ref_idx 2) can be used to encode the reference line index of the current block. Alternatively, for the case where the "case" (or "code table") is inferred based on the size of the block, if the size of the current block (WH) is 256 or less, "case 0" (or "code table 0") can be used, and if the size of the current block is greater than 256, "case 2" (or "code table 2") can be used.

[0418] As another example, the prediction mode of the reference current block, the pixel values ​​of the reference line, and the prediction of the current block generated from the reference line are referenced. In this case, in order to predict the current block according to the predetermined reference line intra_luma_ref_idxA (pred,A ) and the prediction of the current block based on N or N-1 available reference lines (pred i ), the codeword that is shorter or more favorable for entropy decoding can be mapped to the corresponding reference line (intra_luma_ref_idx). In this case, if intra_luma_ref_idxA is one of the N available reference lines, N-1 reference lines less than the available reference lines are used, or N reference lines are used. In addition, the similarity can be calculated by referring to at least one of the following: the sum of absolute differences (SAD), the sum of absolute transformed differences (SATD), the mean square error (MSE), and the mean absolute error (MAE). In addition, if there are multiple reference lines with the same calculation results, the order of the reference lines can be determined by a predetermined method. The predetermined method can take into account, for example, the magnitude of the value of the corresponding reference line index, so that the larger (or smaller) the index value, the shorter or more favorable for entropy decoding The codeword can be mapped to the corresponding reference line.

[0419] Meanwhile, the SAD may be calculated according to Equation 7.

[0420] [Equation 7]

[0421] SAD i = SAD(pred A , pred i )

[0422] Here, i represents an index value of an available reference line. In addition, if intra_luma_ref_idx A is included in the N available reference lines, intra_luma_ref_idx A may be mapped with a codeword determined according to a predetermined method.

[0423] As a specific example, assume that 4 (N=4) reference lines {intra_luma_ref_idx 0, intra_luma_ref_idx 1, intra_luma_ref_idx 2, intra_luma_ref_idx 3} are used in the current block, and the 4 reference lines are represented by 4 code words "0, 10, 110, 111", and the predetermined reference line intra_luma_ref_idxA is intra_luma_ref_idx 0. By calculating pred 0 With pred 1 、pred 2 With pred 3 The mapping between intra_luma_ref_idx and codeword can be based on the result "SAD 3 >SAD 2 >SAD 1In this case, because intra_luma_ref_idxA is included in the four available reference lines for intra_luma_ref_idx 0, intra_luma_ref_idx 0 can be mapped to the shortest or entropy coding-favorable codeword. Therefore, intra_luma_ref_idx 0 can be mapped to codeword "0", intra_luma_ref_idx 3 to "10", intra_luma_ref_idx 2 to "110", and intra_luma_ref_idx 1 to "111". In this case, if a plurality of intra_luma_ref_idx are mapped to codewords of the same length, all possible cases can be used for mapping between reference line indexes and codewords. For example, intra_luma_ref_idx 1 can be mapped to codeword "110", and intra_luma_ref_idx 2 can be mapped with the codeword "111" because the same length codeword is used for both intra_luma_ref_idx 1 and intra_luma_ref_idx 2 in order of priority.

[0424] Depending on the implementation, the codewords that are shorter or have an advantage in entropy decoding can be mapped to the corresponding reference lines in the order of increasing values ​​(i.e., in the order of decreasing similarity) or in a predetermined order based on at least one of the above-mentioned methods for calculating similarity. The predetermined order may, for example, be an order that alternates between reference lines with smaller and larger similarity values. In addition, when comparing the values ​​of SAD, SATD, MSE, MAE, etc., instead of calculating the values ​​of SAD, SATD, MSE, MAE, etc. between predictions, the values ​​of SAD, SATD, MSE, MAE, etc. between all or some pixel values ​​of each reference line may be calculated. When all pixel values ​​of each reference line are selected, the number of pixels of each reference line may change, so calculations using average values ​​(such as MAE and MSE) may be preferred.

[0425] As another example, SAD, SATD, MSE, MAE, etc. may be compared between all pixel values ​​of each reference line and a predetermined value, between some pixel values ​​of each reference line and a predetermined value, or between a prediction based on each reference line and a predetermined value.

[0426] According to the above description, when parsing intra_luma_ref_idx, a method of interpreting a signaled bit stream may be determined by a predetermined method.

[0427] On the other hand, if the reference line group is used to determine the reference line, the predetermined method may be a method of interpreting the bitstream of ref_group_candidate_idx without changing the reference line (intra_luma_ref_idx) indicated by the reference line candidate index ref_group_candidate_idx. Alternatively, the reference line (intra_luma_ref_idx) indicated by ref_group_candidate_idx may be determined by a predetermined method without changing the method of interpreting the bitstream of ref_group_candidate_idx. This is described as follows.

[0428] First, when a reference line group determined according to a predetermined method is used and ref_group_candidate_idx is signaled to indicate one of the candidates, the video decoding device may change the method of interpreting ref_group_candidate_idx to the predetermined method and use the latter. By changing the codeword mapped to ref_group_candidate_idx, the mapping between the code table and the reference line is changed. In this case, ref_group_candidate_idx may indicate a reference line in a corresponding turn within the determined reference line group, or through a mapping between ref_group_candidate_idx and intra_luma_ref_idx in a predetermined relationship, the present disclosure may determine a reference line (intra_luma_ref_idx) indicated by ref_group_candidate_idx. Here, when there are m reference lines in a reference line group and for indicating the m reference lines as ref_group_candidate_idx 0 to ref_group_candidate_idx m-1, a mapping according to a predetermined relationship may be established by using an increasing value of a reference line index (intra_luma_ref_idx) in a reference line group, a decreasing value of a reference line index (intra_luma_ref_idx) in a reference line group, or an order of any other relationship between the indices.

[0429] For example, if ref_group_candidate_idx indicates a reference line in a corresponding turn within a reference line group, and the reference line group determined by a predetermined method is {intra_luma_ref_idx 0, intra_luma_ref_idx 1, intra_luma_ref_idx 5, intra_luma_ref_idx 3}, intra_luma_ref_idx indicated by ref_group_candidate_idx is as shown in Table 18.

[0430]

Table 18

[0431] ref_group_candidate_idx Guides 0 intra_luma_ref_idx 0 l intra_luma_ref_idx 1 2 intra_luma_ref_idx 5 3 intra_luma_ref_idx 3

[0432] In this case, when 4 of the ref_group_candidate_idx are decoded in a unary manner, 4 codewords 0, 10, 110, 111 are used, and various code tables may be used for mapping between ref_group_candidate_idx and the codewords, as shown in Table 19.

[0433]

Table 19

[0434] Code table 0

[0435] Codeword ref_group_candidate_idx 0 0 10 1 110 2 111 3

[0436] Table 1

[0437]

[0438]

[0439] Table 2

[0440] Codeword ref_group_candidate_idx 0 1 10 3 110 2 111 0

[0441] …

[0442] In this case, if ref_group_candidate_idx indicating the reference line is parsed as 10 and "code table 0" is selected according to a predetermined method, ref_group_candidate_idx may be interpreted as 1, and intra_luma_ref_idx 1 may be used for prediction of the current block. Alternatively, if "code table 1" is selected according to a predetermined method, ref_group_candidate_idx may be interpreted as 3, and intra_luma_ref_idx 3 may be used for prediction of the current block.

[0443] In another example, ref_group_candidate_idx indicates a reference line index (intra_luma_ref_idx) within a reference line group in order of increasing value, and the reference line group determined according to a predetermined method is {intra_luma_ref_idx 0, intra_luma_ref_idx 1, intra_luma_ref_idx 5, intra_luma_ref_idx 3}, and the intra_luma_ref_idx indicated by ref_group_candidate_idx is as shown in Table 20.

[0444]

Table 20

[0445] ref_group_candidate_idx Guides 0 intra_luma_ref_idx 0 1 intra_luma_ref_idx 1 2 intra_luma_ref_idx 3 3 intra_luma_ref_idx 5

[0446] In addition, the intra_luma_ref_idx indicated by ref_group_candidate_idx may be determined according to a predetermined method. In the case of decoding 4 of the ref_group_candidate_idx in a unary manner, 4 codewords 0, 10, 110, and 111 may be used, and various code tables may be used for mapping between ref_group_candidate_idx and the codeword, as shown in Table 19. In this case, if the ref_group_candidate_idx indicating the reference line is parsed as 10 and "code table 0" is selected according to a predetermined method, ref_group_candidate_idx may be interpreted as 1, and intra_luma_ref_idx 1 may be used for prediction of the current block. Alternatively, if "code table 1" is selected according to a predetermined method, ref_group_candidate_idx may be interpreted as 3, and intra_luma_ref_idx5 may be used for prediction of the current block.

[0447] As described above, in order to change the mapping between the reference line candidate index and the code table, the video decoding device can (implementation method 4-1-1-3) parse and apply the mapping or (implementation method 4-1-1-4) infer the mapping without sending a signal.

[0448] <Implementation method 4-1-1-3> Sending mapping using signal

[0449] In this implementation, regardless of the parsing of the information on the reference line (i.e., syntax) and its order, the video decoding device may parse mapping_code_idx to determine a method for interpreting the corresponding bitstream. In this case, mapping_code_idx indicates one of the "code tables" in Table 19 that represent the mapping between ref_group_candidate_idx and codewords. For example, if mapping_code_idx is 1, the ref_group_candidate_idx represented by the codeword is set according to "code table 1". This method changes the method of interpreting the information being signaled, so that for MRL and other intra-frame predictions, the corresponding syntax elements or mapping_code_idx can be parsed first. In addition, mapping_code_idx can be signaled at the CU level or at a higher level such as SPS, PPS, etc.

[0450] As another example, in addition to signaling the code table indicated by ref_group_candidate_idx, a code table determination method may also be signaled. In this case, the code table determination method may include any method that can be used to infer the mapping in implementation 4-1-1-4 without signaling. The video decoding device may parse mapping_method_idx to determine the code table determination method. Because this method changes the method of interpreting the information being signaled, for MRL and other intra-frame predictions, the corresponding syntax elements or mapping_method_idx may be parsed first. In addition, mapping_method_idx may be signaled at the CU level or at a higher level such as SPS, PPS, etc.

[0451] <Implementation 4-1-1-4> Inferring mapping without signaling

[0452] In this implementation, the video decoding device infers the mapping between ref_group_candidate_idx and the code table by using a bitstream interpretation method that is inferred without signaling. In order to map a shorter and / or entropy decoding-favorable codeword to the refgroup_candidate_idx expected to be used in each block, the video decoding device may infer the "code table" or code table determination method described above based on specific information. The reference may be the block width, height, area, and aspect ratio (W, H, log 2 W, log 2 H.log 2 WH, WH, log 2 (W / H), W / H, log 2At least one or more of information related to characteristics of (H / W, H / W, etc.) and information about reference lines used by neighboring blocks of the current block.

[0453] This can be further described as follows, wherein at least one of these can be referenced. In the following, the distance between the block and the reference line can be an index value of the reference line, the number of pixels between the two, the number of blocks between the two, etc. In addition, the code table information between ref_group_candidate_idx and the codeword can be used to directly indicate the mapping (the code table in Table 19) and determine the code table for allowing the inference of the mapping.

[0454] First, the references as features of the current block may include position, prediction mode, reference pixels, any prediction that can be generated, a reference line group determined by a predetermined method, a distance between the current block and an available reference line included in the reference line group determined by a predetermined method, and a pixel value of the available reference line. In addition, references may include W, H, log related to width, height, area, and aspect ratio. 2 W, log 2 H.log 2 WH, WH, log 2 (W / H), W / H, log 2 (H / W), H / W, etc.

[0455] The features of the neighboring blocks of the current block may be referenced to the position, the pixel value from the reconstructed block, the prediction mode, the reference line used for prediction, whether to use MRL, the reference line group used, the code table information between the ref_group_candidate_idx used and the codeword, the code table information between the ref_group_candidate_idx and the codeword that is best for the corresponding block, the reference pixel, any prediction that can be generated, the available reference line, the distance between the available reference line and the current block, and the pixel value of the available reference line. In addition, the reference may be W, H, log related to the width, height, area and aspect ratio. 2 W, log 2 H.log 2 WH, WH, log 2 (W / H), W / H, log 2 (H / W), H / W, etc.

[0456] References to features of a reconstructed block earlier than the current block may include the position, pixel values ​​from the reconstructed block, prediction mode, reference lines used for prediction, whether MRL is used, reference line groups used, code table information between the used ref_group_candidate_idx and the codeword, code table information between the best ref_group_candidate_idx and the codeword for the corresponding block, reference pixels, any predictions that can be generated, the distance between the available reference line and the current block, and the pixel values ​​of the available reference lines. In addition, references may include W, H, log related to width, height, area, and aspect ratio. 2 W, log 2 H.log 2 WH, WH, log 2 (W / H), W / H, log 2 (H / W), H / W, etc.

[0457] The block collocated with the current block in another referenceable picture and the neighboring blocks of the collocated block have features that can be referenced, such as position, pixel values ​​from the reconstructed block, prediction mode, reference line used for prediction, reference line group used for use, whether to use MRL, reference line group used, code table information between ref_group_candidate_idx used and codeword, code table information between ref_group_candidate_idx and codeword that is best for the corresponding block, reference pixels, any prediction that can be generated, the distance between the available reference line and the current block, and the pixel value of the available reference line. In addition, references can be W, H, log related to width, height, area and aspect ratio. 2 W, log 2 H.log 2 WH, WH, log 2 (W / H), W / H, log 2 (H / W), H / W, etc.

[0458] When the current block uses the intra mode in the inter slice, the feature of referring to the collocated block using neighboring blocks of the current block and the collocated block in another referenceable picture may be applied more preferably or exclusively according to an embodiment.

[0459] An example of determining a code table or an inferred code table for interpreting a bitstream of ref_group_candidate_idx based on the characteristics of a block to be referenced is as follows. The following example covers determining a reference line (intra_luma_ref_idx) to which ref_group_candidate_idx refers, wherein the value of ref_group_candidate_idx refers to a reference line of a corresponding order in a reference line group. However, depending on the application, the example may also include mapping ref_group_candidate_idx and intra_luma_ref_idx according to a predetermined relationship so that ref_group_candidate_idx indicates a reference line.

[0460] As a first example, when referring to the area of ​​the current block (log 2 WH), as shown in Table 21, the mapping method can be inferred by using different code tables depending on the area of ​​the current block.

[0461]

Table 21

[0462] Block area (log 2 WH)<8

[0463] Codeword ref_group_candidate_idx 0 0 10 1 11 2

[0464] Block area (log 2 WH)≥8

[0465] Codeword ref_group_candidate_idx 0 2 10 1 11 0

[0466] If the reference line group of the current block is determined as {intra_luma_ref_idx 0, intra_luma_ref_idx1, intra_luma_ref_idx 3}, and the area of ​​the current block (log 2 If WH) is 10 and the codeword sent by the signal is 0, then intra_luma_ref_idx can be determined to be 3 because ref_group_candidate_idx is 2.

[0467] As a second example, in order to refer to a reference line used by a neighboring block, if the reference line used by the neighboring block exists in the reference line group of the current block, a shorter or entropy decoding-favorable codeword may be mapped to ref_group_candidate_idx indicating the reference line. A specific example is a case where the reference line group of the current block is determined as {intra_luma_ref_idx 0, intra_luma_ref_idx 1, intra_luma_ref_idx2}, ref_group_candidate_idx is represented by 3 codewords "0, 10, 11", and the neighboring block is defined as a block including pixels 1 to 5 adjacent to the current block, such as Fig.15 As shown in the example of . In this case, when blocks 1, 2, and 4 use intra_luma_ref_idx 2 for prediction, block 3 uses intra_luma_ref_idx 1 for prediction, and block 5 uses intra_luma_ref_idx5 for prediction, intra_luma_ref_idx 2 and intra_luma_ref_idx 1 are included in the reference line group of the current block, so that the ref_group_candidate_idx indicating the reference line can be mapped with a shorter or entropy coding favorable codeword. In this case, the smaller the block number using the reference line (i.e., the minimum value in the block number when multiple blocks use the reference line), the shorter or more entropy coding favorable codewords can be mapped to the ref_group_candidate_idx indicating the reference line. In addition, if the reference line is not used by the adjacent block, but the reference line exists in the reference line group, the codeword mapped to the ref_group_candidate_idx indicating the reference line can be determined according to a predetermined method.

[0468] based on Fig.15According to the example illustrated in and the above description, it can be determined that codeword "0" is mapped to ref_group_candidate_idx 2 indicating intra_luma_ref_idx 2, codeword "10" is mapped to ref_group_candidate_idx 1 indicating intra_luma_ref_idx 1, and codeword "11" is mapped to ref_group_candidate_idx 0 indicating intra_luma_ref_idx 0. In this case, in the case where multiple ref_group_candidate_idx are mapped with codewords of the same length, all possible cases can be used for mapping between reference line candidate indices and codewords. For example, ref_group_candidate_idx 1 may be mapped with the codeword '11' and ref_group_candidate_idx0 may be mapped with the codeword '10' for ref_group_candidate_idx 0 because codewords of the same length are used for both ref_group_candidate_idx 1 and ref_group_candidate_idx 0 in priority order.

[0469] The third example describes the case where the prediction mode of the reference current block, the pixel values ​​of the reference line, and the prediction of the current block generated from the reference line. In these cases, a shorter or entropy-coding-favorable codeword may be used as the predictor (pred A ) and the predictor (pred i ) are mapped to the ref_group_candidate_idx indicating the reference line in an order of decreasing similarity between intra_luma_ref_idx A and intra_luma_ref_idx A. In this case, if intra_luma_ref_idx A exists in the reference line group, the reference line excluding intra_luma_ref_idxA is used, otherwise, all reference lines in the reference line group may be used. In addition, the similarity may be calculated by referring to at least one of SAD, SATD, MSE, and MAE. In addition, if there are reference lines with the same calculation results, the order used for the reference lines may be determined according to a predetermined method. Here, the predetermined method may, for example, take into account the magnitude of the value of the corresponding reference line index, so that the larger (or smaller) the index value, the shorter or more entropy decoding-favorable codeword may be mapped to the ref_group_candidate_idx indicating the corresponding reference line.

[0470] Meanwhile, the SAD may be calculated according to Equation 8.

[0471] [Equation 8]

[0472] SAD i = SAD(pred A , pred i )

[0473] Here, i indicates an index value of an available reference line in a reference line group. In addition, if intra_luma_ref_idxA is included in a reference line group, a codeword determined according to a predetermined method may be mapped to ref_group_candidate_idx indicating intra_luma_ref_idxA.

[0474] A specific example to be described is a case where the reference line group of the current block is determined as {intra_luma_ref_idx0, intra_luma_ref_idx1, intra_luma_ref_idx2, intra_luma_ref_idx4}, ref_group_candidate_idx is represented by 4 code words "0, 10, 110, 111", and the predetermined reference line intra_luma_ref_idx A is intra_luma_ref_idx0. 0 With pred 1 、pred 2 With pred 3 The SAD between ref_group_candidate_idx and codeword can be mapped according to the result "SAD 4 >SAD 2 >SAD 1" In this case, since intra_luma_ref_idx A which is intra_luma_ref_idx 0 is included in the reference line group, ref_group_candidate_idx indicating intra_luma_ref_idx 0 can be mapped to the shortest or entropy decoding-favorable codeword. Therefore, ref_group_candidate_idx 0 can be mapped to the codeword "0", ref_group_candidate_idx 3 indicating intra_luma_ref_idx 4 can be mapped to the codeword "10", ref_group_candidate_idx 2 indicating intra_luma_ref_idx 2 can be mapped to the codeword "110", and ref_group_candidate_idx 1 indicating intra_luma_ref_idx 2 can be mapped to the codeword "110". 1 can be mapped to the codeword "111". In this case, if multiple ref_group_candidate_idx are mapped with codewords of the same length, all possible cases can be used to map between reference line candidate indices and codewords. For example, ref_group_candidate_idx 1 and ref_group_candidate_idx 2 use the same length codeword in order of priority, so a codeword of "110" can be mapped to ref_group_candidate_idx 1 and "111" can be mapped to ref_group_candidate_idx 2.

[0475] Depending on the implementation, the codewords that are shorter or have an advantage in entropy decoding can be mapped to the corresponding reference lines in the order of increasing values ​​(i.e., in the order of decreasing similarity) or in a predetermined order based on at least one of the aforementioned methods for calculating similarity. The predetermined order may, for example, be an order that alternates between reference lines with smaller and larger similarity values. In addition, when comparing the values ​​of SAD, SATD, MSE, MAE, etc., instead of calculating the values ​​of SAD, SATD, MSE, MAE, etc. between predictions, the values ​​of SAD, SATD, MSE, MAE, etc. between all or some pixel values ​​of each reference line may be calculated. When all pixel values ​​of each reference line are selected, the number of pixels of each reference line may change, so calculations using average values ​​(such as MAE and MSE) may be preferred.

[0476] Depending on the implementation, SAD, SATD, MSE, MAE, etc. may be compared between all pixel values ​​of each reference line and predetermined values, between some pixel values ​​of each reference line and predetermined values, or between predictions based on each reference line and predetermined values.

[0477] The fourth example describes the code table information between the best ref_group_candidate_idx and the codeword with reference to the previously reconstructed block. In this case, the mapping between the ref_group_candidate_idx and the codeword in the current block can be determined according to the same code table as the reference information and according to the code table determination method. The optimal code table information between the ref_group_candidate_idx and the codeword of the previously reconstructed block can be determined as follows. Because there is a reconstruction result for the previously reconstructed block, a prediction can be generated by using the reconstructed block and the reference line in the reference line group of the reconstructed block. Then, the codeword that is shorter or entropy decoding is advantageous is mapped to the ref_group_candidate_idx, which indicates the reference line in the order of the reference line, and the reference line generates a prediction closest to the reconstruction value by comparing the reconstructed block with the prediction via SAD, SATD, MSE and MAE and similar methods. In addition, any code table information that can produce the same results as these mapping results can be used as the optimal code table information.

[0478] Next, when a reference line group determined according to a predetermined method is used to determine a reference line and when ref_group_candidate_idx is signaled to indicate one of the candidates, the video decoding device may determine the reference line (intra_luma_ref_idx) indicated by ref_group_candidate_idx according to a predetermined mapping method. When determining a codeword mapped to ref_group_candidate_idx, the video decoding device may change the codeword mapped to the reference line by using one type of code table and by changing the reference line (intra_luma_ref_idx) indicated by ref_group_candidate_idx. An example of using a code table that represents ref_group_candidate_idx as a codeword encoded in a unary manner is described below, as shown in Table 22. However, any other available code table may be used to represent the mapping between ref_group_candidate_idx and the codeword.

[0479]

Table 22

[0480] Codeword ref_group_candidate_idx 0 0 10 1 110 2 111 3

[0481] For example, if the reference line group determined according to a predetermined method is {intra_luma_ref_idx 0, intra_luma_ref_idx 1, intra_luma_ref_idx 5, intra_luma_ref_idx 3}, ref_group_candidate_idx may be used to indicate one of the reference line candidates, and ref_group_candidate_idx may be determined by interpreting the signaled bitstream as shown in Table 22. In this case, different cases may be described in Table 23 as a mapping method for determining a reference line (intra_luma_ref_idx) indicated by ref_group_candidate_idx.

[0482]

Table 23

[0483] Case 0

[0484] ref_group_candidate_idx Guides 0 intra_luma_ref_idx 0 10 intra_luma_rref_idx 1 110 intra_luma_ref_idx 3 111 intra_luma_ref_idx 5

[0485] Case 1

[0486] ref_group_candidate_idx Guides 0 intra_luma_ref_idx 1 10 intra_luma_ref_idx 0 110 intra_luma_ref_idx 5 111 intra_luma_ref_idx 3

[0487] Case 2

[0488] ref_group_candidate_idx Guides 0 intra_luma_ref_idx 0 10 intra_luma_ref_idx 5 110 intra_luma_ref_idx 3 111 intra_luma_ref_idx 1

[0489] …

[0490] In this case, if ref_group_candidate_idx indicating a reference line is parsed as 10 and intra_luma_ref_idx according to ref_group_candidate_idx is determined based on 'Case 2', ref_group_candidate_idx may be 1 and the reference line may be determined as intra_luma_ref_idx 5 according to 'Case 2'.

[0491] In addition, if the mapping between ref_group_candidate_idx and the reference line (intra_luma_ref_idx) is such that ref_group_candidate_idx indicates the reference lines in the corresponding order within the reference line group, the mapping can be achieved by changing the order of the reference lines within the reference line group. For example, if the reference line indicated by ref_group_candidate_idx is determined according to "Case 2", as in the above example, the reference line group can be expressed as {intra_luma_ref_idx 0, intra_luma_ref_idx 5, intra_luma_ref_idx 3, intra_luma_ref_idx1} by changing the order of the reference lines. Since ref_group_candidate_idx indicates the reference lines in the reference line group in this order, the mapping can be performed in the same manner as "Case 2".

[0492] On the other hand, the video decoding device may parse and apply the mapping between ref_group_candidate_idx and the reference line (intra_luma_ref_idx) as described above (implementation method 4-1-1-5), or infer the mapping without signaling (implementation method 4-1-1-6). In this case, as described above, if the mapping between ref_group_candidate_idx and the reference line (intra_luma_ref_idx) is such that ref_group_candidate_idx indicates the reference lines in the reference line group in that order, the mapping may be implemented by changing the order of the reference lines in the reference line group. Thus, all signaling and inference methods for determining the mapping between ref_group_candidate_idx and intra_luma_ref_idx below may also be implemented by changing the order of the reference lines in the reference line group.

[0493] <Implementation method 4-1-1-5> Sending mapping using signal

[0494] In this implementation, regardless of the parsing of the information about the reference line (i.e., syntax) and its order, the video decoding device can determine the reference line (intra_luma_ref_idx) indicated by ref_group_candidate_idx by parsing the mapping line index (hereinafter, "mapping_line_idx"). In this case, mapping_line_idx indicates one of the cases in Table 13 representing the mapping between ref_group_candidate_idx and intra_luma_ref_idx. For example, if mapping_line_idx is 1, the reference line represented by ref_group_candidate_idx is determined according to "Case 1". This method changes the method of interpreting the information sent by the signal so that for MRL and other intra-frame predictions, the corresponding syntax element or mapping_line_idx can be parsed first. In addition, mapping_line_idx can be sent by signal at the CU level or at a higher level such as SPS, PPS, etc.

[0495] As another example, in addition to signaling the intra_luma_ref_idx mapped to ref_group_candidate_idx in a tabular manner, a mapping method may also be signaled. In this case, the mapping method may include any method that can be used to infer the mapping in implementation 4-1-1-6 without signaling. The video decoding device can determine the mapping method by parsing mapping_method_idx. Because this method changes the method of interpreting the information sent by the signal, for MRL and other intra-frame predictions, any corresponding syntax elements and mapping_method_idx may be parsed first. In addition, mapping_method_idx may be signaled at the CU level or at a higher level such as SPS, PPS, etc.

[0496] <Implementation 4-1-1-6> Inferring mapping without signaling

[0497] In this implementation, the video decoding device infers the mapping between ref_group_candidate_idx and intra_luma_ref_idx by using an inference without the need for a signaled bitstream interpretation method. As mentioned above, based on the cases in Table 23, the expected reference line index to be used in each block can be mapped to a ref_group_candidate_idx with a shorter or entropy-coding-favorable codeword. In order to use these cases, the video decoding device can infer a mapping determination method or mapping based on specific information. The specific information can be described in more detail as follows, and at least one of them can be referenced. In the following, the distance between a block and a reference line can be an index value of a corresponding reference line, the number of pixels between the two, the number of blocks between the two, and the like. In addition, the mapping information between ref_group_candidate_idx and intra_luma_ref_idx can be both a direct indication method of the mapping between the two (as in the case of Table 23) and a determination method of the mapping.

[0498] First, the features of the current block may include a position, a prediction mode, a reference pixel, any prediction that can be generated, a reference line group determined by a predetermined method, a distance between the current block and an available reference line included in the reference line group determined by a predetermined method, and a pixel value of the available reference line. In addition, the features of the current block may include W, H, log (W), and W, H, and log (H) related to the width (W), height (H), area, and aspect ratio. 2 W, log 2 H.log 2 WH, WH, log 2 (W / H), W / H, log 2 (H / W), H / W, etc.

[0499] References to the features of the neighboring blocks of the current block may include the position, the pixel value from the reconstructed block, the prediction mode, the reference line used for prediction, whether to use MRL, the reference line group used, the mapping information between the used ref_group_candidate_idx and intra_luma_ref_idx, the mapping information between the best ref_group_candidate_idx and intra_luma_ref_idx for the corresponding block, the reference pixel, any prediction that can be generated, the distance between the available reference line and the current block, and the pixel value of the available reference line. In addition, references may include W, H, log related to width, height, area, and aspect ratio. 2 W, log 2 H.log 2 WH, WH, log 2 (W / H), W / H, log2 (H / W), H / W, etc.

[0500] References to features of a reconstructed block earlier than the current block may include the position, pixel values ​​from the reconstructed block, prediction mode, reference lines used for prediction, whether MRL is used, reference line groups used, mapping information between ref_group_candidate_idx and intra_luma_ref_idx used, mapping information between ref_group_candidate_idx and intra_luma_ref_idx that is optimal for the corresponding block, reference pixels, any predictions that can be generated, distances between available reference lines and the current block, and pixel values ​​of available reference lines. In addition, references may include W, H, log related to width, height, area, and aspect ratio. 2 W, log 2 H.log 2 WH, WH, log 2 (W / H), W / H, log 2 (H / W), H / W, etc.

[0501] The collocated blocks and neighboring blocks of the collocated blocks with the current block in another referenceable picture have features that can be referenced, such as position, pixel values ​​from the reconstructed block, prediction mode, reference lines used for prediction, whether to use MRL, reference line groups used, mapping information between ref_group_candidate_idx and intra_luma_ref_idx used, mapping information between ref_group_candidate_idx and intra_luma_ref_idx that are optimal for the corresponding block, reference pixels, any predictions that can be generated, mapping information between the distance between the available reference lines and the current block, and pixel values ​​of the available reference lines. In addition, references can be W, H, log related to width, height, area, and aspect ratio. 2 W, log 2 H.log 2 WH, WH, log 2 (W / H), W / H, log 2 (H / W), H / W, etc.

[0502] When the current block uses the intra mode in the inter slice, the feature of referring to the collocated block using neighboring blocks of the current block and the collocated block in another referenceable picture may be applied more preferably or exclusively according to an embodiment.

[0503] The following describes an example of determining a method for inferring a mapping between ref_group_candidate_idx and intra_luma_ref_idx based on the characteristics of the reference block or an example of inferring a mapping. The following example describes the use of a code table in which the value of ref_group_candidate_idx is encoded in a unary manner to represent the codeword corresponding to ref_group_candidate_idx. However, depending on the application, an example of using any code table that can represent a mapping between ref_group_candidate_idx and a codeword is also feasible.

[0504] As a first example, when referring to the area of ​​the current block (log 2 WH), the mapping can be determined differently depending on the area of ​​the current block, as shown in Table 24.

[0505]

Table 24

[0506] Block area (log 2 WH)<8

[0507] ref_group_candidate_idx Guides 0 intra_luma_ref_idx 0 1 intra_luma_ref_idx 1 2 intra_luma_ref_idx 2

[0508] Block area (log 2 WH)≥8

[0509] ref_group_candidate_idx Guides 0 intra_luma_ref_idx 2 1 intra_luma_ref_idx 1 2 intra_luma_ref_idx 0

[0510] If the reference line group of the current block is determined as {intra_luma_ref_idx 0, intra_luma_ref_idx1, intra_luma_ref_idx 2}, and the area of ​​the current block (log 2 If WH) is 10 and ref_group_candidate_idx is 2, intra_luma_ref_idx can be determined to be 0.

[0511] As a second example, in order to refer to a reference line used by a neighboring block, if the reference line used by the neighboring block exists in the reference line group of the current block, a shorter or entropy decoding-favorable codeword may be mapped to ref_group_candidate_idx indicating the reference line. A specific example is a case where the reference line group of the current block is determined as {intra_luma_ref_idx 0, intra_luma_ref_idx 1, intra_luma_ref_idx2} and the neighboring block is defined as a block containing pixels 1 to 5 adjacent to the current block, such as Fig.15As shown in the example of . In this case, when blocks 1, 2, and 4 use intra_luma_ref_idx2 for prediction, block 3 uses intra_luma_ref_idx 1 for prediction, and block 5 uses intra_luma_ref_idx 5 for prediction, intra_luma_ref_idx 2 and intra_luma_ref_idx1 are included in the reference line group of the current block, so that the ref_group_candidate_idx indicating the reference line having a shorter or entropy coding favorable codeword. In this case, as the block number using the reference line (i.e., the minimum value in the block number in the case where multiple blocks use the reference line) becomes smaller, the reference block can be mapped to the ref_group_candidate_idx represented by the shorter or entropy coding favorable codeword. In addition, if the reference line is not used by the adjacent block but the reference line exists in the reference line group, the ref_group_candidate_idx mapped to the reference line can be determined according to a predetermined method.

[0512] based on Fig.15 In the example shown and described above, the mapping may be determined such that ref_group_candidate_idx 0 indicates intra_luma_ref_idx 2, ref_group_candidate_idx 1 indicates intra_luma_ref_idx 1, and ref_group_candidate_idx 2 indicates intra_luma_ref_idx 0. In this case, if a plurality of ref_group_candidate_idx are mapped with codewords of the same length, all possible cases may be used to determine the intra_luma_ref_idx indicated by ref_group_candidate_idx. For example, ref_group_candidate_idx 1 and ref_group_candidate_idx 2 use codewords of the same length in a unary manner, such that ref_group_candidate_idx 1 may indicate intra_luma_ref_idx 0, and ref_group_candidate_idx 0 may indicate intra_luma_ref_idx 1.

[0513] The third example describes a prediction mode referring to a current block, a pixel value of a reference line, and a prediction of the current block generated from the reference line. In this case, according to the prediction of the current block according to the predetermined reference line intra_luma_ref_idxA (pred A ) and the prediction of the current block based on the available reference lines (predi ), the corresponding reference line may be mapped to a ref_group_candidate_idx represented by a shorter or entropy-coding-favorable codeword. In this case, if intra_luma_ref_idx A exists in the reference line group, the reference line excluding intra_luma_ref_idx A is used, or all reference lines in the reference line group may be used. In addition, the similarity may be calculated by referring to at least one of SAD, SATD, MSE, and MAE. In addition, if there are reference lines with the same calculation results, the order used for the reference lines may be determined by a predetermined method. The predetermined method may, for example, take into account the magnitude of the value of the corresponding reference line index, so that when the index value becomes larger (or smaller), the corresponding reference line may be mapped to a ref_group_candidate_idx represented by a shorter or coding-favorable codeword.

[0514] Meanwhile, the SAD may be calculated according to Equation 9.

[0515] [Equation 9]

[0516] SAD i = SAD(pred A , pred i )

[0517] Here, i represents an index value of an available reference line in a reference line group. In addition, if intra_luma_ref_idxA is included in a reference line group, intra_luma_ref_idxA may be mapped to a predetermined ref_group_candidate_idx.

[0518] The specific example describes a case where the reference line group of the current block is determined to be {intra_luma_ref_idx 0, intra_luma_ref_idx 1, intra_luma_ref_idx 2, intra_luma_ref_idx 4}, and the predetermined reference line of intra_luma_ref_idx A is intra_luma_ref_idx 0. By calculating pred 0 With pred 1 、pred 2 With pred 4 The mapping between ref_group_candidate_idx and intra_luma_ref_idx can be based on the result "SAD 4 >SAD 2 >SAD 1" is determined. In this case, since intra_luma_ref_idx A which is intra_luma_ref_idx 0 is included in the reference line group, intra_luma_ref_idx 0 can be mapped to ref_group_candidate_idx represented by the shortest or entropy decoding favorable codeword. Thus, mapping can be performed so that ref_group_candidate_idx 0 indicates intra_luma_ref_idx 0, ref_group_candidate_idx1 indicates intra_luma_ref_idx 4, ref_group_candidate_idx 2 indicates intra_luma_ref_idx 2, and ref_group_candidate_id x3 indicates intra_luma_ref_idx 1. In this case, if multiple ref_group_candidate_idx are mapped to the same length codeword, all possible cases can be used to determine which intra_luma_ref_idx is to be indicated by ref_group_candidate_idx. For example, ref_group_candidate_idx 3 and ref_group_candidate_idx 3 use the same length codeword in a unary manner, so ref_group_candidate_idx 2 can indicate intra_luma_ref_idx 1, and ref_group_candidate_idx 3 can indicate intra_luma_ref_idx2.

[0519] Depending on the implementation, by referring to at least one of the above-mentioned methods for calculating similarity, the corresponding reference lines may be mapped to the ref_group_candidate_idx represented by the codeword with the shortest or most entropy decoding advantage in order of increasing value (i.e., in order of decreasing similarity) or in a predetermined order. The predetermined order may, for example, be an order alternating between reference lines with smaller and larger similarity values. In addition, when comparing the values ​​of SAD, SATD, MSE, MAE, etc., instead of calculating the values ​​of SAD, SATD, MSE, MAE, etc. between predictions, the values ​​of SAD, SATD, MSE, MAE, etc. between all or some pixel values ​​of each reference line may be calculated. When all pixel values ​​of each reference line are selected, the number of pixels of each reference line may change, so calculations using average values ​​(such as MAE and MSE) may be preferred.

[0520] Depending on the implementation, SAD, SATD, MSE, MAE, etc. may be compared between all pixel values ​​of each reference line and predetermined values, between some pixel values ​​of each reference line and predetermined values, or between predictions based on each reference line and predetermined values.

[0521] The fourth example describes the case of referring to the information about the best mapping of the previously reconstructed block between refgroup_candidate_idx and intra_luma_ref_idx. The mapping between ref_group_candidate_idx and intra_luma_ref_idx in the current block can be determined according to the same mapping and mapping determination method as the reference information. The best mapping information of the previously reconstructed block between refgroup_candidate_idx and intra_luma_ref_idx can be determined as shown in the following example. Since there is a reconstruction result for the previously reconstructed block, a prediction can be generated by using the reconstructed block and the reference line in the reference line group of the reconstructed block. Then, the reconstructed block and the prediction are compared by SAD, SATD, MSE, MAE, etc. to ensure that the reference line that produces the most similar prediction to the reconstructed value is mapped earlier to the refgroup_candidate_idx represented by the shorter or entropy decoding-favorable codeword. In addition, any code table information that can produce the same result as the mapping can be used as the best code table information.

[0522] <Implementation Method 4-1-2> Adaptively Change the Definition of the Code Table

[0523] In this implementation, the video encoding device adaptively changes the definition of the code table depending on the situation to use multiple code tables for reference line indexes. An example case is described in which 3 reference lines are available and 3 reference line indexes are used to indicate 3 reference lines distinguished by intra_luma_ref_idx 0 to 2. In this case, if the codeword representing the 3 reference line indexes is "0, 10, 11", the video encoding device may select and use one of the code tables illustrated in Table 25. If this implementation is applicable to N reference lines, the number of available code tables is N! (N factorial) in total. Because this is a process for determining the codeword used to encode intra_luma_ref_idx, it is the inverse process of the process of interpreting the bitstream of intra_luma_ref_idx described in implementation 4-1-1. Therefore, the adaptive change of the code table described below can also be implemented by applying and adapting the method described in implementation 4-1-1.

[0524]

Table 25

[0525] Reference Line Index Code table 0 Table 1 Table 2 … intra_luma_re_idx 0 0 0 10 … intra_luma_ref_idx 1 10 11 0 … intra_luma_ref_idx 2 11 10 11 …

[0526] The video decoding device may (implementation method 4-1-2-1) parse and apply the adaptive changes in the definition of the code table as described above, or (implementation method 4-1-2-2) may infer the adaptive changes without sending them by signal.

[0527] <Implementation 4-1-2-1> Signaling and determining adaptive changes in the definition of the code table

[0528] In this implementation, regardless of the parsing of the information about the reference line (i.e., syntax) and its order, the video decoding device can determine the method of interpreting the corresponding bitstream by parsing the code table index (hereinafter, "code_table_idx"). In this case, code_table_idx indicates one of the "code tables" in Table 25 indicating the available code tables. For example, if code_table_idx is 1, the reference line index representing the codeword can be interpreted according to "code_table_1". This method changes the method of interpreting the information sent by the signal so that for MRL and other intra-frame predictions, the corresponding syntax element or code_table_idx can be parsed first. In addition, code_table_idx can be sent by signal at the CU level or at a higher level such as SPS, PPS, etc.

[0529] <Implementation 4-1-2-2> Adaptive Change of Definition of Inference Code Table

[0530] In this implementation, the video decoding device infers the adaptive change of the definition of the code table by using a bitstream interpretation method for inference without signaling. As described above, based on the code table in Table 25, the reference line index expected to be used in each block can be assigned a shorter or entropy decoding-favorable codeword. In order to use such a code table, the video decoding device can infer the above code table based on specific information. The reference can be the block width, height, area, and aspect ratio (W, H, log 2 W, log 2 H.log 2 WH, WH, log 2 (W / H), W / H, log 2 At least one or more of information about characteristics of (H / W, H / W, etc.) and information about reference lines used by neighboring blocks of the current block.

[0531] For example, for inferring a code table based on information of reference lines used by neighboring blocks, if the left neighboring block is predicted with intra_luma_ref_idx1, "code table 2" can be used to encode the reference line index of the current block. Alternatively, in the case of inferring a code table based on the size of the block, if the size (WH) of the current block is 256 or less, "code table 0" can be used, and if the size (WH) of the current block exceeds 256, "code table 2" can be used.

[0532] According to the above description, a codeword used to encode intra_luma_ref_idx may be determined as a predetermined codeword.

[0533] On the other hand, when a reference line group is used to determine a reference line, a codeword for encoding refgroup_candidate_idx may be determined as a predetermined codeword without changing the reference line (intra_luma_ref_idx) indicated by the reference line candidate index refgroup_candidate_idx. Alternatively, the reference line (intra_luma_ref_idx) indicated by refgroup_candidate_idx may be determined according to a predetermined method without changing the codeword for encoding refgroup_candidate_idx.

[0534] In this case, the process of determining the codeword for encoding refgroup_candidate_idx may be the inverse process of the process of interpreting the bitstream of refgroup_candidate_idx described in implementation 4-1-1. Thus, the adaptive change of the code table of refgroup_candidate_idx may also be implemented by applying and adapting the method described in implementation 4-1-1. The determination of the reference line (intra_luma_ref_idx) indicated by refgroup_candidate_idx may also be implemented by applying and adapting the method described in implementation 4-1-1.

[0535] <Implementation 4-2> Adaptively adjust the reference line indicated by the reference line index

[0536] In this implementation, the video decoding device adjusts the reference line indicated by the reference line index based on the block characteristics, so that different reference lines are used even for different blocks with the same reference line index. The reference may be the block width, height, area, and aspect ratio (W, H, log 2 W, log 2 H.log 2 WH, WH, log 2 (W / H), W / H, log 2At least one or more of the information related to the characteristics of (H / W, H / W, etc.). The specific information can also be described as follows, and reference can be made to at least one of the above. In the following, the distance between the block and the reference line can be the index value of the corresponding reference line, the number of pixels between the two, the number of blocks between the two, etc.

[0537] First, the features of the current block may include position, prediction mode, reference pixels, any predictions that can be generated, available reference lines, the distance between the available reference lines and the current block, and the pixel values ​​of the available reference lines. In addition, the references may be W, H, log related to width (W), height (H), area, and aspect ratio. 2 W, log 2 H.log 2 WH, WH, log 2 (W / H), W / H, log 2 (H / W), H / W, etc.

[0538] The features of the neighboring blocks of the current block may be referenced to the position, pixel values ​​from the reconstructed block, prediction mode, reference lines for prediction, whether to use MRL, reference pixels, any predictions that can be generated, available reference lines, the distance between the available reference lines and the current block, pixel values ​​of the available reference lines, etc. In addition, W, H, log related to width, height, area and aspect ratio may be referenced. 2 W, log 2 H.log 2 WH, WH, log 2 (W / H), W / H, log 2 (H / W), H / W, etc.

[0539] References to features of a reconstructed block earlier than the current block may include position, pixel values ​​from the reconstructed block, prediction mode, reference lines for prediction, whether to use MRL, reference pixels, any predictions that can be generated, available reference lines, distances between available reference lines and the current block, and pixel values ​​of available reference lines. References may also include W, H, log related to width, height, area, and aspect ratio. 2 W, log 2 H.log 2 WH, WH, log 2 (W / H), W / H, log 2 (H / W), H / W, etc.

[0540] The block collocated with the current block in another referenceable picture and the neighboring blocks of the collocated block have features that can be referenced, such as position, pixel values ​​from the reconstructed block, prediction mode, reference lines for prediction, whether to use MRL, reference pixels, any predictions that can be generated, available reference lines, the distance between the available reference lines and the current block, and the pixel values ​​of the available reference lines. In addition, references can be W, H, log related to width, height, area, and aspect ratio. 2 W, log 2 H.log 2 WH, WH, log 2 (W / H), W / H, log 2 (H / W), H / W, etc.

[0541] When the current block uses the intra mode in the inter slice, the feature of referring to the collocated block using neighboring blocks of the current block and the collocated block in another referenceable picture may be applied more preferably or exclusively according to an embodiment.

[0542] For example, when the area (WH) of a block is referred to as a block feature, a reference line indicated by a reference line index may be set according to the area of ​​the block, as shown in Table 26.

[0543]

Table 26

[0544]

[0545]

[0546] Here, intra_luma_ref_idx is a syntax for signaling an index indicating a reference line, and luma_ref_line i Indicates the reference line i pixels away from the current block. Table 26 can be illustrated as Fig.16 .

[0547] <Implementation 5> Using signals to send reference line information considering block features

[0548] In this implementation, the video encoding device reflects the block features in the context model for entropy decoding by considering the block features to signal information on the reference line. The reference may be the block width, height, area, and aspect ratio (W, H, log 2 W, log 2 H.log 2 WH, WH, log 2 (W / H), W / H, log 2At least one or more of the information related to the characteristics of (H / W, H / W, etc.). The specific information can also be described as follows, and at least one of the above can be referred to. In the following, the distance between the block and the reference line can be the index value of the corresponding reference line, the number of pixels between the two, the number of blocks between the two, etc.

[0549] First, you can refer to the characteristics of the current block described in Implementation 4-2.

[0550] Reference may be made to the features of the neighboring blocks of the current block described in Implementation 4-2.

[0551] Reference may be made to the features of the reconstructed block of the current block described earlier in Implementation 4-2.

[0552] Reference may also be made to features of a block collocated with the current block and blocks adjacent to the collocated block in another reference picture, which are described in implementation method 4-2.

[0553] When the current block uses the intra mode in the inter slice, the feature of referring to the collocated block using neighboring blocks of the current block and the collocated block in another referenceable picture may be applied more preferably or exclusively according to an embodiment.

[0554] Entropy decoding is a coding method that generates a bitstream based on probability, and its context model uses an initial probability value and a probability update rate. If a block feature can identify a possible reference line to be used in the current block, the efficiency of coding can be improved by incorporating the block feature into the context model. The implementation can select one of multiple context models with different initial probability values ​​or probability update rates based on the block feature, and then the selected context model can be used.

[0555] <Implementation 6> Combining this implementation with existing technology

[0556] In this implementation, additional syntax elements may be signaled to selectively apply the above methods of implementations 1 to 5. In this case, implementations 3 and 5 may be applied in combination with other implementations or prior art. To this end, the video encoding device may signal selective_mrl_flag to indicate information about how the current block uses multiple reference lines. For example, as shown in Table 27, if selective_mrl_flag is 0, conventional techniques may be used instead of following the present disclosure, and if selective_mrl_flag is 1, implementation 2 may be applied.

[0557]

Table 27

[0558]

[0559] Optionally, when selective_mrl_flag is 1, any method of implementation 1 to 5 or a combination thereof may be used by further signaling selective_mrl_idx, as shown in Table 28.

[0560]

Table 28

[0561]

[0562] References below Fig.17 and Fig.18 A method of intra-predicting and encoding / decoding a current block by a video encoding device or a video decoding device according to implementation 1 is described. For other implementations, similar descriptions about the method of intra-predicting and encoding / decoding a current block by a video encoding device or a video decoding device may be applied.

[0563] Fig.17 is a flowchart of a method for encoding a current block performed by a video encoding device according to at least one embodiment of the present disclosure.

[0564] The video encoding apparatus determines an intra prediction mode for a current block ( S1700 ).

[0565] The video encoding device determines a selective MRL flag (S1702). Here, the selective MRL flag selective_mrl_flag indicates whether multiple reference lines are selectively applied to the current block. The video encoding device may determine the selective MRL flag in terms of optimizing decoding efficiency. If the selective MRL flag is not determined, it may be inferred that the selective MRL flag is false.

[0566] The video encoding apparatus checks the selective MRL flag (S1704).

[0567] If the selective MRL flag is true (Yes in S1704), the video encoding apparatus performs the following steps.

[0568] The video encoding apparatus derives a reference line group of the current block (S1706). Here, the reference line group includes at least one or more reference lines. In addition, the reference line group may include reference lines adjacent to the current block.

[0569] In one example, according to implementation 1-1, the video encoding device may encode a reference line group index indicating a reference line group after determining one of the multiple reference line groups. In this case, the video encoding device may determine the reference line group in terms of optimizing rate distortion.

[0570] As another example, the video encoding device may select a reference line group from multiple reference line groups based on block features, as in implementation 1-2-1. Here, the block features may include features of the current block, features of neighboring blocks of the current block, features of reconstructed blocks earlier than the current block, and all or part of features of collocated blocks having the current block and neighboring blocks of the collocated blocks in another referenceable picture.

[0571] As yet another example, the video encoding device may use a preset reference line group, such as in implementation 1-2-2.

[0572] The video encoding apparatus derives reference lines within the reference line group (S1708). Here, the reference lines in the reference line group are indicated by a reference line candidate index ref_group_candidate_idx.

[0573] The reference line candidate index indicates which reference line to use in the reference line group. Optionally, the reference line candidate index is determined by mapping between the reference line candidate index and the reference line index, and the reference line index indicates a reference line of the current block.

[0574] In one example, the video encoding device may encode the reference line candidate index according to implementation 1-1-1, 1-2-1-1, or 1-2-2-1 after determining the reference line candidate index. In this case, the video encoding device may determine the reference line candidate index in terms of optimizing rate distortion.

[0575] As another example, the video encoding device may infer the reference line candidate index to be used based on the block features according to implementation 1-1-2, 1-2-1-2, or 1-2-2-2, or may use the reference line candidate index preset at a higher level such as SPS, PPS, etc. Here, the block features may include all or part of the features of the current block, the features of the neighboring blocks of the current block, the features of the reconstructed blocks earlier than the current block, and the features of the blocks collocated with the current block in another referenceable picture and the neighboring blocks of the collocated blocks.

[0576] On the other hand, when the reference line group includes a single reference line, the video encoding device may encode the reference line candidate index. In this case, the video decoding device may decode the reference line candidate index or infer the reference line candidate index to be zero.

[0577] The video encoding apparatus generates a prediction of a current block by using a reference line according to an intra prediction mode ( S1710 ).

[0578] The video encoding apparatus subtracts the prediction from the current block to generate a residual block (S1712).

[0579] The video encoding apparatus encodes the selective MRL flag, the intra prediction mode, and the residual block (S1714).

[0580] If the selective MRL flag is false (No in S1704 ), the video encoding apparatus determines a reference line adjacent to the current block as a reference line of the current block ( S1720 ).

[0581] The video encoding device may then perform steps S1710 to S1714.

[0582] Fig.18 is a flowchart of a method for decoding a current block performed by a video decoding device according to at least one embodiment of the present disclosure.

[0583] The video decoding apparatus decodes a selective MRL flag from a bitstream (S1800). Here, the selective MRL flag selective_mrl_flag indicates whether a multi-reference line is selectively applied to a current block.

[0584] The video decoding apparatus decodes an intra prediction mode and a residual block of a current block from a bitstream (S1802).

[0585] The video decoding device checks the selective MRL flag (S1804).

[0586] If the selective MRL flag is true (Yes in S1804), the video decoding apparatus performs the following steps.

[0587] The video decoding apparatus derives a reference line group of the current block (S1806). Here, the reference line group includes at least one or more reference lines. In addition, the reference line group may include reference lines adjacent to the current block.

[0588] In one example, the video decoding device may decode a reference line group index indicating a reference line group from a bitstream according to implementation 1-1, and may then determine a reference line group indicated by the reference line group index among multiple reference line groups.

[0589] As another example, the video decoding device may select one reference line group from multiple reference line groups based on block features, as in implementation 1-2-1. Here, the block features may include features of the current block, features of neighboring blocks of the current block, features of blocks reconstructed earlier than the current block, and all or part of features of blocks collocated with the current block and neighboring blocks of the collocated blocks in another referenceable picture.

[0590] As another example, the video decoding device may use a preset reference line group, such as in implementation 1-2-2.

[0591] The video decoding apparatus derives reference lines within the reference line group (S1808). Here, the reference lines in the reference line group are indicated by a reference line candidate index ref_group_candidate_idx.

[0592] The reference line candidate index indicates which reference line to use in the reference line group. Optionally, the reference line candidate index is determined by mapping between the reference line candidate index and a reference line index indicating a reference line of the current block.

[0593] In one example, the video decoding device may decode a reference line candidate index from a bitstream according to implementation 1-1-1, 1-2-1-1, or 1-2-2-1, and may then determine a reference line indicated by the reference line candidate index.

[0594] As another example, according to implementation 1-1-2, 1-2-1-2, or 1-2-2-2, the video decoding device may infer a reference line candidate index to be used based on block features or use a reference line preset at a higher level such as SPS, PPS, etc. Here, the block features may include features of the current block, features of neighboring blocks of the current block, features of a reconstructed block earlier than the current block, and all or part of features of a block collocated with the current block in another referenceable picture and neighboring blocks of the collocated block.

[0595] Meanwhile, when the reference line group includes a single reference line, the video decoding device may decode the reference line candidate index, or infer the reference line candidate index to be zero.

[0596] The video decoding apparatus generates a prediction of a current block by using a reference line according to an intra prediction mode ( S1810 ).

[0597] The video decoding apparatus generates a reconstructed block of the current block by summing the residual block and the prediction (S1812).

[0598] If the selective MRL flag is false (No in S1804 ), the video decoding apparatus determines a reference line adjacent to the current block as a reference line of the current block ( S1820 ).

[0599] The video decoding device may then perform steps S1810 and S1812.

[0600] Although the steps in the corresponding flow charts are described as being performed sequentially, these steps are merely illustrative of the technical concepts of some embodiments of the present disclosure. Therefore, a person of ordinary skill in the art to which the present disclosure belongs can perform the steps by changing the order described in the corresponding figures or by performing two or more steps in parallel. Therefore, the steps in the corresponding flow charts are not limited to the time series order shown.

[0601] It should be understood that the above description presents illustrative embodiments that can be implemented in different other ways. The functions described in some embodiments can be implemented by hardware, software, firmware and / or a combination thereof. It should also be understood that the functional components described in this disclosure are marked with "... units" to strongly emphasize the possibility of their independent implementation.

[0602] At the same time, the various methods or functions described in some embodiments may be implemented as instructions stored in a non-transitory recording medium that is read and executed by one or more processors. For example, the non-transitory recording medium may include various types of recording devices, in which data is stored in a form readable by a computer system. For example, the non-transitory recording medium may include a storage medium such as an erasable programmable read-only memory (EPROM), a flash drive, an optical drive, a magnetic hard drive, and a solid-state drive (SSD).

[0603] Although the embodiments of the present disclosure have been described for illustrative purposes, it should be appreciated by those skilled in the art that various modifications, additions, and substitutions are possible without departing from the concept and scope of the present disclosure. Therefore, for the sake of brevity and clarity, the embodiments of the present disclosure have been described. The scope of the technical concept of the embodiments of the present disclosure is not limited by the diagrams. Therefore, it should be understood by those skilled in the art that the scope of the present disclosure should not be limited by the embodiments explicitly described above, but by the claims and their equivalents.

[0604] (reference number)

[0605] 122: Intra-frame predictor

[0606] 155: Entropy Encoder

[0607] 510: Entropy Decoder

[0608] 542: Intra-frame predictor.

[0609] CROSS-REFERENCE TO RELATED APPLICATIONS

[0610] This application claims priority to and the benefit of Korean Patent Application No. 10-2022-0042016 filed on April 5, 2022, Korean Patent Application No. 10-2022-0111486 filed on September 2, 2022, and Korean Patent Application No. 10-2023-0031219 filed on March 9, 2023, the entire contents of each of which are incorporated herein by reference.

Claims

1. A method for performing intra-frame prediction on a current block performed by a video decoding device, the method include: decoding an intra prediction mode of the current block from a bitstream; deriving a reference line group of the current block, wherein the reference line group includes at least one reference line; deriving a reference line in the reference line group, wherein the reference line in the reference line group is indicated by a reference line candidate index; as well as A prediction of the current block is generated by using the reference line according to the intra prediction mode.

2. The method according to claim 1, further comprising: include: decoding, from the bitstream, a selective multiple reference line (MRL) flag indicating whether multiple reference lines are to be selectively applied to the current block; as well as checking the selective MRL flag; Wherein, when the selective MRL flag is true, the method further includes: The reference line set is exported.

3. The method of claim 2, further comprising, when the selective MRL flag is false: determining a reference line adjacent to the current block as a reference line of the current block; and A prediction of the current block is generated by using the reference line according to the intra prediction mode.

4. The method according to claim 1, in, The reference line group comprises: A reference line adjacent to the current block.

5. The method according to claim 1, in, Exporting the reference line group includes: decoding a reference line group index indicating the reference line group from the bitstream; and A reference line group indicated by the reference line group index is determined among the plurality of reference line groups.

6. The method according to claim 1, in, Exporting the reference line group includes: The reference line group is selected from the plurality of reference line groups based on block features, wherein the block features include all or part of features of the current block, features of neighboring blocks of the current block, features of blocks reconstructed earlier than the current block, and features of a block collocated with the current block and neighboring blocks of the collocated block in another referenceable picture.

7. The method according to claim 1, in, Exporting the reference line in the reference line group includes: decoding the reference line candidate index from the bitstream; and A reference line indicated by the reference line candidate index is determined.

8. The method according to claim 1, in, Exporting the reference line in the reference line group includes: The reference line candidate index is selected based on block features, wherein the block features include all or part of features of the current block, features of neighboring blocks of the current block, features of blocks reconstructed earlier than the current block, and features of a block collocated with the current block and neighboring blocks of the collocated block in another referenceable picture.

9. The method according to claim 1, in, The reference line candidate index indicates which reference line to use within the reference line group.

10. The method according to claim 1, in, The reference line candidate index is determined according to a mapping between the reference line candidate index and a reference line index indicating a reference line of the current block.

11. The method according to claim 1, further comprising, when the reference line group includes one reference line: The reference line candidate index is decoded or inferred to be zero.

12. A method for performing intra-frame prediction on a current block performed by a video encoding device, the method include: Determining an intra prediction mode of the current block; deriving a reference line group of the current block, wherein the reference line group includes at least one reference line; deriving a reference line within the reference line group, wherein the reference line in the reference line group is indicated by a reference line candidate index; and A prediction of the current block is generated by using the reference line according to the intra prediction mode.

13. The method according to claim 12, further comprising: include: determining a selective multiple reference line (MRL) flag indicating whether multiple reference lines are to be selectively applied to the current block; as well as checking the selective MRL flag; Wherein, when the selective MRL flag is true, the method further includes: The reference line set is exported.

14. The method according to claim 13, further comprising: include: The optional MRL flag is encoded.

15. The method of claim 14, further comprising, when the selective MRL flag is false: determining a reference line adjacent to the current block as a reference line of the current block; and A prediction of the current block is generated by using the reference line according to the intra prediction mode.

16. A computer-readable recording medium storing a bit stream generated by a video encoding method, the video encoding method include: Determining an intra prediction mode for a current block; deriving a reference line group of the current block, wherein the reference line group includes at least one reference line; deriving a reference line within the reference line group, wherein the reference line in the reference line group is indicated by a reference line candidate index; and A prediction of the current block is generated by using the reference line according to the intra prediction mode.

Citation Information

Cited By

  • Image processing method, processing device, and storage medium

    WO2025153116A3