Intra prediction mode derivation and merge prediction based on inter prediction signal

Abstract

Methods for deriving an intra prediction mode based on an inter prediction signal are disclosed. In the present embodiments, an image decoding device determines a motion vector for a current block from a motion vector candidate list for the current block and generates an inter prediction signal for the current block based on a reference block indicated by the motion vector for the current block. The image decoding device derives an intra prediction mode for the current block based on the reference block and a template for the reference block and generates an intra prediction signal for the current block based on the intra prediction mode. The image decoding device generates a final prediction signal for the current block based on the inter prediction signal and the intra prediction signal.

Description

Technical Field

[0001] This invention relates to a video encoding and decoding method and apparatus that derives and fuses predictions using intra-frame prediction modes based on inter-frame prediction signals. Background Technology

[0002] The statements in this section are merely background information relating to the present invention and do not necessarily constitute prior art.

[0003] Assuming that video data is much larger in volume than audio or still image data, unprocessed or uncompressed video data requires significant hardware resources (including memory) to store or transmit in its raw form.

[0004] Accordingly, when storing or transmitting video data, an encoder is typically used to compress the video data for storage or transmission, and a decoder receives the compressed video data and performs decompression and reconstruction. Such video compression technologies include H.264 / AVC, High Efficiency Video Coding (HEVC), and Versatile Video Coding (VVC), which offers approximately 30% or more improvement in coding efficiency compared to HEVC.

[0005] However, as video size, resolution, and frame rate continue to increase, and the amount of data that needs to be encoded and decoded increases accordingly, there is a need for new compression techniques that are more efficient in encoding and decoding and more effective in improving video quality than existing compression techniques.

[0006] The combined inter-frame and intra-frame prediction mode (CIIP mode) performs a weighted summation of the inter-frame prediction signals and the intra-frame prediction signals to generate the final prediction signal for the current block. When CIIP mode is used in VVC, the inter-frame prediction signal is generated based on the regular merging mode and signaling / parsing based on the merging index, while the intra-frame prediction signal is generated based on the plane mode. Therefore, to improve video encoding / decoding efficiency and enhance video quality, various methods for generating the inter-frame and intra-frame prediction signals need to be considered when using CIIP mode. Summary of the Invention

[0007] Technical issues This invention aims to provide a video encoding and decoding method and apparatus, which derives the intra-frame prediction mode based on the inter-frame prediction signal when generating the final prediction signal of the current block by weighted summation of inter-frame prediction signals and intra-frame prediction signals.

[0008] Technical solution At least one aspect of the present invention provides a method for reconstructing a current block using a video decoding apparatus. The method includes determining a motion vector of the current block from a motion vector candidate list based on information indicating a motion vector of the current block. The method further includes generating an inter-frame prediction signal for the current block based on a reference block present in a reference image and indicated by the motion vector of the current block. The method further includes deriving an intra-frame prediction mode for the current block based on the reference block and a template of the reference block. The method further includes generating an intra-frame prediction signal for the current block based on the intra-frame prediction mode. The method further includes generating a final prediction signal for the current block based on the inter-frame prediction signal and the intra-frame prediction signal.

[0009] Another aspect of the present invention provides a method for encoding a current block using a video encoding apparatus. The method includes obtaining information indicating a motion vector of the current block, and determining a motion vector of the current block from a motion vector candidate list based on the information indicating the motion vector. The method further includes generating an inter-frame prediction signal for the current block based on a reference block present in a reference picture and indicated by the motion vector of the current block. The method further includes deriving an intra-frame prediction mode for the current block based on the reference block and a template of the reference block. The method further includes generating an intra-frame prediction signal for the current block based on the intra-frame prediction mode. The method further includes generating a final prediction signal for the current block based on the inter-frame prediction signal and the intra-frame prediction signal.

[0010] Another aspect of the present invention provides a method for providing video data to a video decoding apparatus. The method includes encoding the video data into a bitstream and transmitting the bitstream to the video decoding apparatus. Encoding the video data includes obtaining information indicating a motion vector of a current block, and determining a motion vector of the current block from a motion vector candidate list based on the motion vector information. Encoding the video data also includes generating an inter-frame prediction signal for the current block based on a reference block present in a reference picture and indicated by the motion vector of the current block. Encoding the video data further includes deriving an intra-frame prediction mode for the current block based on the reference block and a template of the reference block. Encoding the video data also includes generating an intra-frame prediction signal for the current block based on the intra-frame prediction mode. Encoding the video data further includes generating a final prediction signal for the current block based on the inter-frame prediction signal and the intra-frame prediction signal.

[0011] Beneficial effects As described above, the present invention provides a video encoding / decoding method and apparatus that, when generating the final prediction signal for the current block by weighted summation of inter-frame prediction signals and intra-frame prediction signals, derives the intra-frame prediction mode based on the inter-frame prediction signals. Therefore, the video encoding / decoding method and apparatus improve objective video encoding / decoding efficiency and enhance subjective video quality. Attached Figure Description

[0012] Figure 1This is a block diagram of a video encoding device that can implement the technology of this invention.

[0013] Figure 2 This demonstrates a method for partitioning blocks using a quadtree plus binary tree ternary tree (QTBTTT) structure.

[0014] Figure 3a and Figure 3b Multiple intra-prediction modes, including the wide-angle intra-prediction mode, are shown.

[0015] Figure 4 Show the adjacent blocks of the current block.

[0016] Figure 5 This is a block diagram of a video decoding device that can implement the technology of this invention.

[0017] Figure 6 This is a schematic diagram showing adjacent blocks used to calculate weights.

[0018] Figure 7a and Figure 7b This is a schematic diagram showing the sub-blocks generated by the partitioning of the current block.

[0019] Figure 8 This is a detailed block diagram of a video decoding apparatus according to at least one embodiment of the present invention.

[0020] Figure 9a and Figure 9b This is a schematic diagram showing the adjacent positions of the current block according to at least one embodiment of the present invention.

[0021] Figure 10 This is a schematic diagram illustrating the region where template-based motion compensation is performed according to at least one embodiment of the present invention.

[0022] Figure 11 This is a schematic diagram illustrating a template of the current block according to at least one embodiment of the present invention.

[0023] Figure 12 This is a schematic diagram illustrating the determination of intra-frame prediction mode candidates based on the location of samples according to at least one embodiment of the present invention.

[0024] Figure 13 This is a schematic diagram showing reference lines of a template according to at least one embodiment of the present invention.

[0025] Figure 14 This is a schematic diagram illustrating a template for the adjacent reconstruction of the current block according to at least one embodiment of the present invention.

[0026] Figure 15This is a schematic diagram illustrating a gradient in a portion of a calculation template according to at least one embodiment of the present invention.

[0027] Figure 16 This is a schematic diagram illustrating a gradient in a portion of a calculation template according to another embodiment of the present invention.

[0028] Figure 17 This is a schematic diagram showing a reference block within a reference picture according to yet another embodiment of the present invention.

[0029] Figure 18 This is a schematic diagram illustrating a region for intra-frame prediction mode derivation according to at least one embodiment of the present invention.

[0030] Figure 19a and Figure 19b This is a schematic diagram illustrating a sub-block generated by partitioning the current block according to at least one embodiment of the present invention.

[0031] Figure 20 This is a schematic diagram illustrating a partition of a directional prediction pattern according to at least one embodiment of the present invention.

[0032] Figures 21a to 21c This is a schematic diagram illustrating a sub-block generated by partitioning the current block according to at least one embodiment of the present invention.

[0033] Figure 22 This is a flowchart of a method for encoding the current block by a video encoding device according to at least one embodiment of the present invention.

[0034] Figure 23 This is a flowchart of a method for reconstructing the current block by a video encoding device according to at least one embodiment of the present invention. Detailed Implementation

[0035] In the following description, some embodiments of the invention will be described in detail with reference to the accompanying illustrative drawings. In the description below, the same reference numerals denote the same elements, although the elements are shown in different drawings. Furthermore, in the following description of some embodiments, detailed descriptions of relevant known components and functions may be omitted for clarity and brevity when it is considered that such detailed descriptions obscure the subject matter of the invention.

[0036] Figure 1 This is a block diagram of a video encoding apparatus that can implement the technology of this invention. In the following text, reference is made to… Figure 1 The diagram illustrates the video encoding apparatus and its components.

[0037] The encoding device may include: an image segmenter 110, a predictor 120, a subtractor 130, a transformer 140, a quantizer 145, a rearrangement unit 150, an entropy encoder 155, an inverse quantizer 160, an inverse transformer 165, an adder 170, a loop filter unit 180, and a memory 190.

[0038] Each component of the encoding device can be implemented as hardware or software, or a combination of hardware and software. Furthermore, the function of each component can be implemented as software, and the microprocessor can be implemented to execute the software functions corresponding to each component.

[0039] A video consists of one or more sequences of images. Each image is segmented into multiple regions, and encoding is performed on each region. For example, an image is segmented into one or more tiles and / or slices. Here, one or more tiles can be defined as a tile group. Each tile and / or slice is segmented into one or more coding tree units (CTUs). Additionally, each CTU is segmented into one or more coding units (CUs) through a tree structure. Information applied to each coding unit (CU) is encoded as the syntax of the CU, and information commonly applied to CUs included in a CTU is encoded as the syntax of the CTU. Furthermore, information commonly applied to all blocks within a slice is encoded as the syntax of the slice header, while information applied to all blocks constituting one or more images is encoded as a Picture Parameter Set (PPS) or picture header. Moreover, information commonly referenced by multiple images is encoded as a Sequence Parameter Set (SPS). Additionally, information commonly referenced by one or more SPSs is encoded as a Video Parameter Set (VPS). Furthermore, information commonly applied to a tile or tile group can also be encoded as the syntax of the tile or tile group header. The syntax included in SPS, PPS, slice headers, and tile or tile group headers can be called high-level syntax.

[0040] Image splitter 110 determines the size of the coding tree unit (CTU). Information about the size of the CTU (CTU dimensions) is encoded into SPS or PPS syntax and transmitted to the video decoding device.

[0041] Image segmenter 110 segments each image constituting the video into multiple coding tree units (CTUs) of a predetermined size, and then recursively segments the CTUs using a tree structure. The leaf nodes in the tree structure become coding units (CUs), which are the basic units of coding.

[0042] A tree structure can be a quadtree (QT), where a higher node (or parent node) is split into four lower nodes (or child nodes) of the same size. A tree structure can also be a binary tree (BT), where a higher node is split into two lower nodes. A tree structure can also be a ternary tree (TT), where a higher node is split into three lower nodes in a 1:2:1 ratio. A tree structure can also be a combination of two or more of the following structures: QT, BT, and TT. For example, a quadtree plus binary tree (QTBT) structure can be used, or a quadtree plus binary tree ternary tree (QTBTTT) structure can be used. Here, a binary tree ternary tree (BTTT) is added to the tree structure to form a multiple-type tree (MTT).

[0043] Figure 2 This is a schematic diagram illustrating a method for segmenting blocks using a QTBTTT structure.

[0044] like Figure 2 As shown, the CTU can first be segmented into a QT structure. Quadtree segmentation can be recursive until the size of the segmented blocks reaches the minimum block size (MinQTSize) allowed for leaf nodes in the QT. The entropy encoder 155 encodes a first flag (QT_split_flag) indicating whether each node of the QT structure is segmented into the four nodes below it and signals this flag to the video decoding device. When the leaf node of the QT is not larger than the maximum block size (MaxBTSize) allowed for the root node in the BT, the leaf node can be further segmented using at least one of the BT or TT structures. Multiple segmentation directions can exist in the BT and / or TT structures. For example, two directions can exist: a horizontal direction for segmenting the blocks of the corresponding node and a vertical direction for segmenting the blocks of the corresponding node. Figure 2 As shown, when MTT splitting begins, the entropy encoder 155 encodes a second flag (mtt_split_flag) indicating whether a node has been split, as well as a flag indicating the splitting direction (vertical or horizontal) and / or the splitting type (binary or ternary) if the node has been split, and signals this to the video decoding device.

[0045] Alternatively, before encoding the first flag (QT_split_flag) indicating whether each node has been split into the four lower-level nodes, the CU split flag (split_cu_flag) indicating whether a node has been split can also be encoded. When the value of the CU split flag (split_cu_flag) indicates that each node has not been split, the block of the corresponding node becomes a leaf node in the split tree structure and becomes a CU, which is the basic unit of encoding. When the value of the CU split flag (split_cu_flag) indicates that each node has been split, the video encoding device begins encoding the first flag using the above scheme.

[0046] When QTBT is used as another example of a tree structure, two types can exist: one where the corresponding node's block is horizontally divided into two blocks of the same size (i.e., symmetrical horizontal division), and the other where the corresponding node's block is vertically divided into two blocks of the same size (i.e., symmetrical vertical division). The entropy encoder 155 encodes a split flag (split_flag) indicating whether each node of the BT structure is divided into blocks of the lower layer, and split type information indicating the split type, and transmits this information to the video decoding device. Alternatively, the type where the corresponding node's block is divided into two blocks that are asymmetrically divided can also exist. Asymmetrical forms can include the form where the corresponding node's block is divided into two rectangular blocks with a size ratio of 1:3, or it can also include the form where the corresponding node's block is divided diagonally.

[0047] The CU can have various sizes depending on the QTBT or QTBTTT segmentation from the CTU. In the following text, the block corresponding to the CU to be encoded or decoded (i.e., the leaf node of the QTBTTT) is called the "current block". When using QTBTTT segmentation, the current block can also be rectangular in shape in addition to a square shape.

[0048] Predictor 120 predicts the current block to generate a prediction block. Predictor 120 includes an intra-frame predictor 122 and an inter-frame predictor 124.

[0049] Typically, each block of the current image can be predicted and encoded. This prediction can usually be performed using either intra-frame prediction (which utilizes data from the image containing the current block) or inter-frame prediction (which utilizes data from an image encoded before the image containing the current block). Inter-frame prediction includes both one-way and two-way prediction.

[0050] Intra-predictor 122 predicts pixels in the current block by utilizing pixels (reference pixels) that are adjacent to the current block in the current image, including the current block. Depending on the prediction direction, multiple intra-prediction modes exist. For example, such as... Figure 3aAs shown, multiple intra-frame prediction modes can include two non-directional modes: Planar mode and DC mode, and can include 65 directional modes. The neighboring pixels and algorithm equations to be used are defined differently for each prediction mode.

[0051] To perform efficient orientation prediction for the current block with a rectangular shape, additional methods can be used. Figure 3b The directional modes indicated by the dashed arrows (intra-prediction modes #67 to #80, #-1 to #-14) can be referred to as "wide-angle intra-prediction modes." Figure 3b In the diagram, the arrow indicates the corresponding reference sample used for prediction, not the prediction direction. The prediction direction is opposite to the direction indicated by the arrow. When the current block has a rectangular shape, the wide-angle intra-frame prediction mode is a mode that performs prediction in the opposite direction to a specific directional mode without additional bit transmission. In this case, in the wide-angle intra-frame prediction mode, some wide-angle intra-frame prediction modes available for the current block can be determined by the ratio of the width to the height of the current block with a rectangular shape. For example, when the current block has a rectangular shape with a height less than its width, wide-angle intra-frame prediction modes with angles less than 45 degrees (intra-frame prediction modes #67 to #80) are available. When the current block has a rectangular shape with a width greater than its height, wide-angle intra-frame prediction modes with angles greater than -135 degrees (intra-frame prediction modes #-1 to #-14) are available.

[0052] Intra-predictor 122 can determine the intra-prediction to be used for encoding the current block. In some examples, intra-predictor 122 can encode the current block by utilizing multiple intra-prediction modes, and can also select the appropriate intra-prediction mode to use from test modes. For example, intra-predictor 122 can calculate rate-distortion values ​​by utilizing rate-distortion analysis of multiple test intra-prediction modes, and can also select the intra-prediction mode with the best rate-distortion characteristics from the test modes.

[0053] Intra-predictor 122 selects one intra-prediction mode from multiple intra-prediction modes and predicts the current block by utilizing neighboring pixels (reference pixels) determined according to the selected intra-prediction mode and an algorithmic equation. Entropy encoder 155 encodes the information about the selected intra-prediction mode and transmits it to the video decoding device.

[0054] Inter-frame predictor 124 generates a predicted block for the current block by utilizing motion compensation processing. Inter-frame predictor 124 searches for the most similar block to the current block in a reference image that was encoded and decoded earlier than the current image, and generates the predicted block for the current block using the searched block. Additionally, a motion vector (MV) is generated, corresponding to the displacement between the current block in the current image and the predicted block in the reference image. Typically, motion estimation is performed on the luma component, and the motion vector calculated based on the luma component is used for both the luma and chroma components. The motion information, including information from the reference image and information about the motion vector used to predict the current block, is encoded by entropy encoder 155 and transmitted to the video decoding device.

[0055] The inter-frame predictor 124 can also perform interpolation of a reference picture or reference block to increase prediction accuracy. In other words, subsamples are interpolated between two consecutive integer samples by applying filter coefficients to multiple consecutive integer samples comprising two integer samples. When performing the process of searching for the most similar block to the current block on the interpolated reference picture, the motion vector can be represented with fractional-unit precision instead of integer-sample-unit precision. For each target region to be encoded, such as a cell like a slice, tile, CTU, CU, etc., the precision or resolution of the motion vector can be set differently. When this adaptive motion vector resolution (AMVR) is applied, information about the motion vector resolution to be applied to each target region should be signaled for each target region. For example, when the target region is a CU, information about the motion vector resolution applied to each CU is signaled. The information about the motion vector resolution can be information representing the precision of the motion vector difference to be described below.

[0056] On the other hand, the inter-frame predictor 124 can perform inter-frame prediction by utilizing bidirectional prediction. In the case of bidirectional prediction, two reference images and two motion vectors representing the positions of the blocks most similar to the current block in each reference image are used. The inter-frame predictor 124 selects a first reference image and a second reference image from reference image list 0 (RefPicList0) and reference image list 1 (RefPicList1), respectively. The inter-frame predictor 124 also searches for the block most similar to the current block in the corresponding reference image to generate a first reference block and a second reference block. Furthermore, the predicted block for the current block is generated by averaging or weighted averaging the first reference block and the second reference block. In addition, motion information including information about the two reference images used to predict the current block and information about the two motion vectors is transmitted to the entropy encoder 155. Here, reference image list 0 may consist of images in the pre-reconstructed images that precede the current image in display order, and reference image list 1 may consist of images in the pre-reconstructed images that follow the current image in display order. However, although not particularly limited to this, pre-reconstructed images that follow the current image in display order may be additionally included in reference image list 0. Conversely, pre-reconstructed images preceding the current image can also be additionally included in the reference image list 1.

[0057] Various methods can be used to minimize the number of bits consumed in encoding motion information.

[0058] For example, when the reference image and motion vector of the current block are the same as those of the adjacent blocks, the information that can identify the adjacent blocks is encoded to transmit the motion information of the current block to the video decoding device. This method is called merge mode.

[0059] In merge mode, the inter-frame predictor 124 selects a predetermined number of merge candidate blocks (hereinafter referred to as "merge candidates") from the neighboring blocks of the current block.

[0060] As adjacent blocks used to derive merge candidates, all or some of the left block A0, lower left block A1, upper block B0, upper right block B1, and upper left block B2 adjacent to the current block in the current image can be used, such as Figure 4 As shown. In addition to the current image containing the current block, blocks located within a reference image (which may be the same as or different from the reference image used to predict the current block) can also be used as merge candidates. For example, a co-located block of the current block within the reference image, or a block adjacent to that co-located block, can be additionally used as a merge candidate. If the number of merge candidates selected using the above method is less than a preset number, a zero vector is added to the merge candidates.

[0061] The inter-frame predictor 124 configures a merge list including a predetermined number of merge candidates by utilizing neighboring blocks. It selects a merge candidate from the merge candidates included in the merge list to be used as motion information for the current block, and generates merge index information for identifying the selected candidate. The generated merge index information is encoded by the entropy encoder 155 and transmitted to the video decoding device.

[0062] The merge skip mode is a special case of the merge mode. After quantization, when all transform coefficients used for entropy coding are close to zero, only the adjacent block selection information is transmitted, and the residual signal is not transmitted. By utilizing the merge skip mode, relatively high coding efficiency can be achieved for images with slight motion, still images, and screen content images.

[0063] From then on, merge mode and merge skip mode were collectively referred to as merge / skip mode.

[0064] Another approach for encoding motion information is the Advanced Motion Vector Prediction (AMVP) model.

[0065] In AMVP mode, the inter-frame predictor 124 derives motion vector prediction candidates for the current block by utilizing neighboring blocks of the current block. As neighboring blocks used to derive motion vector prediction candidates, they can use... Figure 4 The current block in the current image shown includes all or some of the adjacent blocks: left block A0, lower left block A1, upper block B0, upper right block B1, and upper left block B2. Furthermore, blocks located in a reference image (which may be the same as or different from the reference image used to predict the current block) can also be used as neighboring blocks for deriving motion vector prediction candidates, in addition to the current image containing the current block. For example, a co-located block of the current block in the reference image or a block adjacent to that co-located block can be used. If the number of motion vector candidates selected by the above method is less than a preset number, a zero vector is added to the motion vector candidates.

[0066] The inter-frame predictor 124 derives motion vector prediction candidates by utilizing motion vectors from neighboring blocks, and determines the motion vector prediction for the current block by utilizing these candidates. Furthermore, the motion vector difference is calculated by subtracting the motion vector prediction from the motion vector prediction for the current block.

[0067] Motion vector predictions can be obtained by applying predefined functions (e.g., median and average calculations) to motion vector prediction candidates. In this case, the video decoding device is also aware of the predefined functions. Furthermore, since the neighboring blocks used to derive the motion vector prediction candidates are already encoded and decoded, the video decoding device may already know the motion vectors of the neighboring blocks. Therefore, the video encoding device does not need to encode the information used to identify the motion vector prediction candidates. Accordingly, in this case, information about the motion vector difference and information about the reference image used to predict the current block are encoded.

[0068] Alternatively, motion vector prediction can be determined by selecting any one of the motion vector prediction candidates. In this case, the information used to identify the selected motion vector prediction candidate is further encoded together with the information about the motion vector difference used to predict the current block and the information about the reference image.

[0069] Subtractor 130 generates a residual block by subtracting the predicted block generated by intra-predictor 122 or inter-predictor 124 from the current block.

[0070] Transformer 140 transforms the residual signal in the residual block, which has pixel values ​​in the spatial domain, into transform coefficients in the frequency domain. Transformer 140 can transform the residual signal in the residual block by using the entire size of the residual block as a transform unit, or it can divide the residual block into multiple sub-blocks and perform the transformation by using the sub-blocks as transform units. Alternatively, the residual block is divided into two sub-blocks, namely a transformed region and a non-transformed region, so that the residual signal is transformed by using only the transformed region sub-block as a transform unit. Here, the transformed region sub-block can be one of two rectangular blocks with a size ratio of 1:1 based on a horizontal axis (or a vertical axis). In this case, the entropy encoder 155 encodes a flag (cu_sbt_flag) indicating that only the transformed sub-block is used, as well as orientation (vertical / horizontal) information (cu_sbt_horizontal_flag) and / or position information (cu_sbt_pos_flag), and signals this information to the video decoding device. Additionally, the size of the transformed region sub-blocks can have a 1:3 size ratio based on the horizontal axis (or vertical axis). In this case, the entropy encoder 155 additionally encodes the flag (cu_sbt_quad_flag) for dividing the corresponding segment and signals it to the video decoding device.

[0071] On the other hand, the transformer 140 can perform the transformation of the residual block separately in the horizontal and vertical directions. Various types of transformation functions or transformation matrices can be used for this transformation. For example, the pair of transformation functions used for horizontal and vertical transformations can be defined as a multiple transform set (MTS). The transformer 140 can select the transform function pair with the highest transformation efficiency in the MTS and can transform the residual block in each of the horizontal and vertical directions. The entropy encoder 155 encodes the information (mts_idx) about the transform function pair in the MTS and signals it to the video decoding device.

[0072] Quantizer 145 quantizes the transform coefficients output from transformer 140 using quantization parameters and outputs the quantized transform coefficients to entropy encoder 155. Quantizer 145 can also quantize the relevant residual blocks immediately without transforming any block or frame. Quantizer 145 can also apply different quantization coefficients (scaling values) based on the position of the transform coefficients in the transform block. The quantization matrix applied to the quantized transform coefficients arranged in two dimensions can be encoded and signaled to the video decoding device.

[0073] The rearrangement unit 150 can rearrange the coefficient values ​​of the quantized residual values.

[0074] The rearrangement unit 150 can transform a 2D coefficient array into a 1D coefficient sequence by utilizing coefficient scanning. For example, the rearrangement unit 150 can use a zig-zag scan or a diagonal scan to scan the DC coefficients to the high-frequency region to output a 1D coefficient sequence. Depending on the size of the transform unit and the intra-frame prediction mode, a vertical scan scanning the 2D coefficient array in the column direction and a horizontal scan scanning the 2D block-type coefficients in the row direction can also be used instead of a zig-zag scan. In other words, the scanning method to be used can be determined from zig-zag scan, diagonal scan, vertical scan, and horizontal scan, depending on the size of the transform unit and the intra-frame prediction mode.

[0075] The entropy encoder 155 encodes the sequence of 1D quantized transform coefficients output from the rearrangement unit 150 by utilizing various encoding schemes, including Context-based Adaptive Binary Arithmetic Code (CABAC) and Exponential Golomb, to generate a bitstream.

[0076] Furthermore, the entropy encoder 155 encodes block-segmentation related information (e.g., CTU size, CTU segmentation flag, QT segmentation flag, MTT segmentation type, and MTT segmentation direction) so that the video decoding device can segment blocks equivalently to the video encoding device. Additionally, the entropy encoder 155 encodes information indicating whether the current block is intra-frame predictive coding or inter-frame predictive coding. The entropy encoder 155 encodes intra-frame prediction information (i.e., information about the intra-frame prediction mode) or inter-frame prediction information (the merge index in the case of merge mode, and information about the reference picture index and motion vector difference in the case of AMVP mode) according to the prediction type. Furthermore, the entropy encoder 155 encodes quantization-related information (i.e., information about quantization parameters and information about the quantization matrix).

[0077] Inverse quantizer 160 inverse quantizes the quantized transform coefficients output from quantizer 145 to generate transform coefficients. Inverse transformer 165 transforms the transform coefficients output from inverse quantizer 160 from the frequency domain to the spatial domain to reconstruct the residual block.

[0078] Adder 170 adds the reconstructed residual block to the prediction block generated by predictor 120 to reconstruct the current block. When performing intra-frame prediction for the next block, the pixels in the reconstructed current block are used as reference pixels.

[0079] The loop filtering unit 180 performs filtering on the reconstructed pixels to reduce blocking artifacts, ringing artifacts, blurring artifacts, etc., that occur due to block-based prediction and transform / quantization. The loop filtering unit 180, as an in-loop filter, may include all or some of the following: a deblocking filter 182, a sample adaptive offset (SAO) filter 184, and an adaptive loop filter (ALF) 186.

[0080] Deblocking filter 182 filters the boundaries between reconstructed blocks to remove blocking artifacts caused by block unit encoding / decoding, and SAO filter 184 and ALF 186 perform additional filtering on the deblocked video. SAO filter 184 and ALF 186 are filters used to compensate for the difference between reconstructed pixels and original pixels caused by lossy coding. SAO filter 184 applies an offset in CTU units to enhance subjective image quality and coding efficiency. On the other hand, ALF 186 performs block unit filtering and applies different filters to compensate for distortion by dividing the boundaries of the corresponding blocks and the degree of variation. Information about the filter coefficients to be used for ALF can be encoded and signaled to the video decoding device.

[0081] The reconstructed blocks filtered by deblocking filter 182, SAO filter 184, and ALF 186 are stored in memory 190. When all blocks in an image are reconstructed, the reconstructed image can be used as a reference image for inter-frame prediction of blocks in subsequent images to be encoded.

[0082] Video encoding devices can store the bitstream of encoded video data in a non-volatile storage medium or send the bitstream to a video decoding device via a communication network.

[0083] Figure 5 This is a functional block diagram of a video decoding device that can implement the technology of this invention. In the following text, reference is made to... Figure 5 It describes the video decoding device and its components.

[0084] The video decoding device may include an entropy decoder 510, a rearrangement unit 515, an inverse quantizer 520, an inverse transformer 530, a predictor 540, an adder 550, a loop filter unit 560, and a memory 570.

[0085] Similar to Figure 1 The video encoding device and the video decoding device each component can be implemented as hardware or software, or a combination of hardware and software. Furthermore, the function of each component can be implemented as software, and the microprocessor can also be implemented to execute the software functions corresponding to each component.

[0086] The entropy decoder 510 extracts information related to block segmentation by decoding the bitstream generated by the video encoding device to determine the current block to be decoded, and extracts the prediction information and information about the residual signal required to reconstruct the current block.

[0087] The entropy decoder 510 determines the size of the CTU by extracting information about the CTU size from the sequence parameter set (SPS) or the picture parameter set (PPS) and segments the image into CTUs of a defined size. Furthermore, the CTU is identified as the highest level (i.e., the root node) of the tree structure, and segmentation information of the CTU can be extracted to segment the CTU by utilizing the tree structure.

[0088] For example, when segmenting a CTU using a QTBTT structure, the first flag (QT_split_flag) related to the QT segmentation is first extracted to split each node into four lower-level nodes. Additionally, a second flag (mtt_split_flag), segmentation direction (vertical / horizontal), and / or segmentation type (binary / ternary) related to the MTT segmentation are extracted relative to the nodes corresponding to the leaf nodes of the QT to segment the corresponding leaf nodes using the MTT structure. As a result, each node below the leaf node of the QT is recursively segmented using either a BT or TT structure.

[0089] As another example, when splitting a CTU using a QTBTTT structure, a CU split flag (split_cu_flag) indicating whether a CU has been split is extracted. A first flag (QT_split_flag) can also be extracted when the corresponding block is split. During the splitting process, for each node, zero or more recursive MTT splits may occur after zero or more recursive QT splits. For example, for a CTU, MTT splits may occur immediately, or conversely, only multiple QT splits may occur.

[0090] As another example, when segmenting a CTU using a QTBT structure, the first flag (QT_split_flag) associated with the QT segmentation is extracted to split each node into four nodes at the lower level. Additionally, a split flag (split_flag) indicating whether the node corresponding to a leaf node of the QT will be further segmented using BT, as well as segmentation direction information, is extracted.

[0091] On the other hand, when the entropy decoder 510 determines the current block to be decoded by utilizing the segmentation of the tree structure, the entropy decoder 510 extracts information about the prediction type indicating whether the current block is predicted intra-frame or inter-frame. When the prediction type information indicates intra-frame prediction, the entropy decoder 510 extracts the syntax elements for the intra-frame prediction information (intra-frame prediction mode) for the current block. When the prediction type information indicates inter-frame prediction, the entropy decoder 510 extracts information representing the syntax elements of the inter-frame prediction information, namely, the motion vector and the reference picture of the motion vector reference.

[0092] In addition, the entropy decoder 510 extracts quantization-related information and extracts information about the quantization transform coefficients of the current block as information about the residual signal.

[0093] The rearrangement unit 515 can re-transform the sequence of 1D quantized transform coefficients entropy-decoded by the entropy decoder 510 into a 2D coefficient array (i.e., a block) in the reverse order of the coefficient scan order performed by the video encoding device.

[0094] The inverse quantizer 520 performs inverse quantization on the quantized transform coefficients, and performs inverse quantization on the quantized transform coefficients using quantization parameters. The inverse quantizer 520 can also apply different quantization coefficients (scaling values) to the quantized transform coefficients arranged in 2D. The inverse quantizer 520 can perform inverse quantization by applying a matrix of quantization coefficients (scaling values) from the video encoding device to a 2D array of quantized transform coefficients.

[0095] The inverse transformer 530 reconstructs the residual signal by inversely transforming the inverse-quantized transform coefficients from the frequency domain to the spatial domain to generate the residual block of the current block.

[0096] Furthermore, when the inverse transformer 530 performs an inverse transform on a portion (sub-block) of the transform block, it extracts a flag (cu_sbt_flag) indicating that only the sub-blocks of the transform block are transformed, the sub-block's orientation (vertical / horizontal) information (cu_sbt_horizontal_flag), and / or the sub-block's position information (cu_sbt_pos_flag). The inverse transformer 530 also inversely transforms the transform coefficients of the corresponding sub-blocks from the frequency domain to the spatial domain to reconstruct the residual signal, and fills the untransformed regions with values ​​"0" as the residual signal to generate the final residual block for the current block.

[0097] Furthermore, when applying MTS, the inverse transformer 530 determines the transform function or transform matrix to be applied in each of the horizontal and vertical directions by utilizing the MTS information (mts_idx) signaled from the video encoding device. The inverse transformer 530 also performs inverse transform on the transform coefficients in the transform block in both the horizontal and vertical directions using the determined transform function.

[0098] Predictor 540 may include intra-predictor 542 and inter-predictor 544. Intra-predictor 542 is activated when the prediction type of the current block is intra-prediction, and inter-predictor 544 is activated when the prediction type of the current block is inter-prediction.

[0099] Intra-predictor 542 determines the intra-prediction mode for the current block from among multiple intra-prediction modes based on the grammatical elements of the intra-prediction mode extracted from entropy decoder 510. Intra-predictor 542 also predicts the current block by utilizing neighboring reference pixels of the current block based on the intra-prediction mode.

[0100] The inter-frame predictor 544 determines the motion vector and the reference picture of the motion vector reference for the current block by utilizing the grammatical elements of the inter-frame prediction mode extracted from the entropy decoder 510, and predicts the current block by utilizing the motion vector and the reference picture.

[0101] Adder 550 reconstructs the current block by adding the residual block output from inverse transform 530 to the predicted block output from inter-frame predictor 544 or intra-frame predictor 542. Pixels within the reconstructed current block are used as reference pixels when performing intra-frame prediction for subsequent blocks to be decoded.

[0102] The loop filtering unit 560, acting as an in-loop filter, may include a deblocking filter 562, a SAO filter 564, and an ALF 566. The deblocking filter 562 performs deblocking filtering on the boundaries between reconstructed blocks to remove block artifacts caused by block unit decoding. The SAO filter 564 and ALF 566 perform additional filtering on the reconstructed blocks after deblocking to compensate for differences between reconstructed pixels and original pixels caused by lossy encoding. The filter coefficients of the ALF are determined by utilizing information about the filter coefficients decoded from the bitstream.

[0103] The reconstructed blocks filtered by deblocking filter 562, SAO filter 564, and ALF 566 are stored in memory 570. When all blocks in an image are reconstructed, the reconstructed image can be used as a reference image for inter-frame prediction of blocks in subsequent images to be encoded.

[0104] In some embodiments, the present invention relates to encoding and decoding video images as described above. More specifically, the present invention provides a video encoding / decoding method and apparatus for deriving an intra-frame prediction mode based on the inter-frame prediction signal when generating the final prediction signal of the current block by weighted summation of inter-frame prediction signals and intra-frame prediction signals.

[0105] The following implementation scheme can be performed by the predictor 120 in the video encoding device. The following implementation scheme can also be performed by the predictor 540 in the video decoding device.

[0106] When encoding the current block, the video encoding apparatus can generate signaling information associated with this embodiment from the perspective of optimizing rate distortion. The video encoding apparatus can encode the signaling information using the entropy encoder 155 and can send the encoded signaling information to the video decoding apparatus. The video decoding apparatus can decode the signaling information associated with the decoding of the current block from the bitstream using the entropy decoder 510.

[0107] In the following description, the term "target block" may be used interchangeably with the current block or coding unit (CU), or may refer to some region of a coding unit.

[0108] In addition, a flag value of true indicates that the flag is set to 1. Conversely, a flag value of false indicates that the flag is set to 0.

[0109] I-1. Merge / Skip Mode for Inter-Frame Prediction and MMVD The following describes a method for constructing a merge candidate list for motion information in the merge / skip mode of inter-frame prediction. To support the merge / skip mode, the video coding apparatus can select a preset number (e.g., 6) of merge candidates to form the merge candidate list.

[0110] The video coding unit searches the space to merge candidates. The video coding unit merges candidates from adjacent blocks in the search space, such as... Figure 4 As shown, up to four space merging candidates can be selected.

[0111] The video encoding device searches for time-merging candidates. The video encoding device can add blocks within a reference image that are in the same position as the current block as time-merging candidates. The reference image can be the same as or different from the reference image used to predict the current block. One time-merging candidate can be selected.

[0112] The video coding unit searches for History-based Motion Vector Predictor (HMVP) candidates. The unit stores the motion vectors of the previous h CUs (where h is a natural number) in a table, and these motion vectors can be used as merging candidates. The table is 6 in size and stores the motion vectors of previous CUs using a first-in-first-out (FIFO) method. This indicates that up to 6 HMVP candidates can be stored in the table. The video coding unit can then set the latest motion vector as a merging candidate from the HMVP candidates stored in the table.

[0113] The video encoding device searches for pairwise average MVP (PAMVP) candidates. The video encoding device can set the average of the motion vectors of the first and second candidates in the merge candidate list as the merge candidate.

[0114] If the merge candidate list cannot be filled even after performing all the above search procedures (i.e., if the preset count cannot be filled), the video encoding device adds a zero motion vector as a merge candidate.

[0115] From the perspective of optimizing encoding and decoding efficiency, the video encoding device can determine the merge index of a candidate within the merge candidate list. The video encoding device can use the merge index to derive the motion vector predictor (MVP) from the merge candidate list, and then determine the MVP as the motion vector of the current block. Additionally, the video encoding device can signal the merge index to the video decoding device.

[0116] In skip mode, the video encoder uses the same motion vector transfer method as in merge mode, but does not transfer residual blocks that are equal to the difference between the current block and the predicted block.

[0117] The method for constructing the above-described merge candidate list can be performed equivalently by a video decoding device. The video decoding device can decode the merge index. The video decoding device can use the merge index to derive the MVP from the merge candidate list, and then the MVP can be determined as the motion vector of the current block.

[0118] On the other hand, when using Merge mode with Motion Vector Difference (MMVD) technology, the video encoding device can use a merge index to derive the MVP from the merge candidate list. For example, the first or second candidate in the merge candidate list can be used as the MVP. Furthermore, from the perspective of encoding / decoding efficiency optimization, the video encoding device determines the distance index and the orientation index. The video encoding device can use the distance index and the orientation index to derive the Motion Vector Difference (MVD), and then sum the MVD and the MVP to reconstruct the motion vector of the current block. Additionally, the video encoding device can signal the merge index, distance index, and orientation index to the video decoding device.

[0119] The aforementioned MMVD technique can be equivalently performed by the video decoding device at inter-frame prediction unit 544. The video decoding device can decode the merge index, distance index, and orientation index. The video decoding device can construct a merge candidate list, and then use the merge index to derive the MVP from the merge candidate list. The video decoding device can use the distance index and orientation index to derive the MVD, and then sum the MVD and MVP to reconstruct the motion vector of the current block.

[0120] I-2. Combined Inter and Intra Prediction (CIIP) CIIP mode generates the final prediction signal of the current block with a size of W×H by weighted summation of the inter-frame prediction signal and the intra-frame prediction signal.

[0121] When CIIP mode is used in VVC, inter-frame prediction signals are generated based on signaling / parsing according to the merge index of the regular merging mode, while intra-frame prediction signals are generated based on the plane mode. For example... Figure 6 As shown, based on the prediction technique for the reconstructed blocks to the left (L) and top (T) of the current block, weights are determined for the weighted summation of intra-frame and inter-frame prediction signals. If intra-frame prediction is applied to both the left and top blocks, the weight ratio between the intra-frame and inter-frame prediction signals is determined to be 3:1. If intra-frame prediction is applied to one of the two blocks, the weight ratio is determined to be 1:1. If inter-frame prediction is applied to both blocks, the weight ratio is determined to be 1:3. The CIIP mode generates the final prediction signal based on these determined weights.

[0122] When using CIIP mode in the Enhanced Compression Model (ECM) corresponding to Beyond VVC, inter-frame prediction signals are generated in the following order: CIIP mode constructs a regular merge candidate list for the current block, where the candidate list size is 2. For candidates in the constructed list, CIIP mode corrects motion based on template matching. CIIP mode calculates the cost between the template of the corrected candidate and the template of the current block, thereby reordering the two candidates. CIIP mode uses the candidate with the lower cost to generate the inter-frame prediction signal. Intra-frame prediction signals are generated in the following order: CIIP mode uses the template of the current block to apply the Template-based IntraMode Derivation (TIMD) method to the most probable mode (MPM) candidate, thereby deriving the intra-frame prediction mode. CIIP mode uses the derived intra-frame prediction mode to generate the intra-frame prediction signal for the current block.

[0123] When generating inter-frame prediction and intra-frame prediction signals according to the above process, the CIIP mode calculates weights based on the directionality of the derived intra-frame prediction mode. For example, if the derived intra-frame prediction mode is a horizontal mode, the current block can be vertically divided into sub-blocks, such as... Figure 7a As shown. If the derived intra-frame prediction mode is vertical, the current block can be horizontally divided into sub-blocks, such as... Figure 7b As shown. In Figure 7a and Figure 7b In the table, the numbers within a sub-block indicate the sub-block index. The CIIP mode determines the weight ratio of the intra-predictive signal and the inter-predictive signal for each sub-block, as shown in Table 1. The CIIP mode generates the final predictive signal based on the determined weights.

[0124] [Table 1] In Table 1, w 帧内 and w 帧间 These represent the intra-frame prediction signal weights and inter-frame prediction signal weights, respectively.

[0125] The following implementation schemes are described with video decoding devices as the central focus, but they can be implemented in the same or similar ways in video encoding devices.

[0126] II. Embodiments according to the present invention Figure 8 This is a detailed block diagram of a video decoding apparatus according to at least one embodiment of the present invention.

[0127] According to this embodiment, the video decoding apparatus can determine prediction and transform units, and for the current block corresponding to the determined units, use determined prediction techniques and prediction modes to perform prediction and inverse transform, thereby ultimately generating a reconstructed block of the current block. This can be performed by the inverse transformer 530, predictor 540, and adder 550 of the video decoding apparatus. Figure 8 The operation is shown. Alternatively, it can be performed in a video encoding apparatus via inverse transformer 165, image segmenter 110, predictor 120, and adder 170. Figure 8 The operation shown is the same. In this case, the video decoding device utilizes the encoded information parsed from the bitstream, while the video encoding device can utilize the encoded information from a higher-level setting in terms of minimizing rate distortion. For convenience, this embodiment will be described below focusing on the video decoding device.

[0128] like Figure 5 As shown in the example, according to the prediction technique, predictor 540 includes an intra-frame predictor 542 and an inter-frame predictor 544. However, as... Figure 8As shown, the predictor 540 may include all or part of the prediction unit determiner 802, the prediction technique determiner 804, the prediction mode determiner 806, and the prediction executor 808.

[0129] When the input video's color format is YUV (YUV420, YUV411, YUV422, YUV444, etc.), the video decoding device can perform prediction and reconstruction of the luminance component, and then perform prediction and reconstruction of the chrominance component. In other words, the luminance and chrominance components can be determined by... Figure 8 The components shown are reconstructed sequentially. On the other hand, when the input video's color format is RGB, the video encoding device can perform a color format conversion from RGB to YUV, and then encode the converted video. Here, in the case of YUV format, the color format represents the correspondence between pixels in the luminance component and pixels in the chrominance component.

[0130] Prediction unit determiner 802 determines the prediction unit (PU). Prediction technique determiner 804 determines the prediction technique relative to the prediction unit, such as intra-frame prediction, inter-frame prediction, intra-block copy (IBC) mode, palette mode, etc. Prediction mode determiner 806 determines the detailed prediction mode used for the prediction technique. Prediction executor 808 generates the prediction block for the current block based on the determined prediction mode.

[0131] The inverse transformer 530 includes a transformer unit determiner 810 and an inverse transformer actuator 812. The transformer unit determiner 810 determines the transformer unit (TU) relative to the inverse quantization signal of the current block, and the inverse transformer actuator 812 performs an inverse transformer on the transformer unit represented by the inverse quantization signal to generate a residual signal.

[0132] Adder 550 sums the predicted block and the residual signal to generate a reconstructed block. The reconstructed block is stored in memory and can subsequently be used to predict other blocks.

[0133] The prediction unit determined by prediction unit determiner 802 can be a sub-block of the current block or a sub-block segmented from the current block. In this case, depending on the color format, the prediction unit for the chroma component can correspond in size to the prediction unit for the luminance component. Alternatively, prediction units for the luminance and chroma components can be determined separately, and prediction can be performed relative to the prediction unit for the chroma component.

[0134] Prediction technique determiner 804 determines the prediction technique used by the prediction unit. As described above, the prediction technique can be one of inter-frame prediction, intra-frame prediction, IBC mode, and palette mode. In this case, the prediction technique for the chroma component can be determined to be the same as the prediction technique for the corresponding luma component without signaling and parsing separate information.

[0135] For example, if the prediction technique for the current block is not intra-frame prediction, the video decoder parses a 1-bit flag. For example, if the parsed flag indicates a skip mode, the video decoder determines that the prediction mode for the current block is either a merging mode for inter-frame prediction or an IBC merging mode. In skip mode, the video decoder can use the prediction signal for reconstructing the signal while skipping the inverse transform process; that is, there is no need to parse the residual signal.

[0136] Conversely, if the parsed flags do not indicate the skip mode used for the current block, the prediction technique determiner 804 can parse a series of 1-bit flags to determine whether the prediction technique for the current block is one such technique as inter-frame prediction, intra-frame prediction, IBC mode, palette mode, etc.

[0137] For example, if no skip is applied to the current block and the prediction technique is determined to be inter-frame prediction or IBC mode, the video decoding device resolves a 1-bit flag. Based on the resolved flag, the prediction mode for the current block can be determined to be either regular merging mode or advanced motion vector prediction (AMVP) mode.

[0138] The prediction pattern determiner 806 determines the detailed prediction pattern of the prediction technique.

[0139] For example, if the prediction technique for the current block is inter-frame prediction, the prediction mode determiner 806 can determine either a regular merging mode or an AMVP mode for the prediction mode of the current block. In the regular merging mode or AMVP mode, the video decoding device generates prediction blocks based on one or more motion compensations based on parsed motion information, and performs a weighted summation of the generated prediction blocks to generate the final prediction signal for the current block.

[0140] As another example, when the prediction technique for the current block is inter-frame prediction, the prediction mode determiner 806 can determine that geometric partitioning mode (GPM) is the prediction mode for the current block. Under GPM, the video decoding device divides the current block into two or more partitioned blocks according to geometric partitioning, generates prediction blocks based on one or more motion compensations using the motion information resolved from the current block, and performs a weighted sum of the generated prediction blocks to generate the final prediction signal for the current block. For example, as described above, the current block can be divided into two partitioned blocks.

[0141] The prediction executor 808 generates a prediction block for the current block based on the determined prediction technique and prediction pattern.

[0142] As an example, the predictive executor 808 generates a predicted block for the current block based on the prediction mode, and the adder 550 sums the predicted block and the residual signal for the current block to generate a reconstructed block.

[0143] On the other hand, when a quadratic transform is applied, the entropy decoder 510 can reconstruct the quantized quadratic transform coefficients. Conversely, when no quadratic transform is applied, the entropy decoder 510 can reconstruct the quantized master transform coefficients. The inverse quantizer 520 can apply inverse quantization to the reconstructed transform coefficients based on the quantization parameters to generate inverse-quantized transform coefficients.

[0144] The transform unit determiner 810 of the inverse transformer 530 determines the transform units (TUs) for the transform coefficients used for inverse quantization. At this time, when a single TU is divided into multiple sub-blocks, a single sub-block can be used as a TU.

[0145] When a quadratic transform is applied, the inverse transformer 530 can determine the inseparable quadratic inverse transform kernel and the separable master inverse transform kernel. Conversely, when no quadratic transform is applied, the inverse transformer 530 can determine either the separable master inverse transform kernel or the inseparable master inverse transform kernel. The inverse transformer 530 can use the inverse transform kernel to perform an inverse transform on the inverse-quantized transform coefficients. The inverse transformer 530 can determine whether to perform an inseparable master inverse transform and whether to perform an inseparable quadratic inverse transform based on signaling / parsing, or the inverse transformer 530 can determine this implicitly based on information such as the size and aspect ratio of the current TU.

[0146] The following describes in detail the operation of the prediction mode determiner 806 in deriving the intra-frame prediction mode.

[0147] As an example, when the prediction technique for the current block is inter-frame prediction and the intra-frame prediction signal is used to generate the final prediction signal for the current block, the video decoding device can define the adjacent reconstructed regions of the current block as templates and can use these templates to derive the intra-frame prediction pattern. The templates can include regions not adjacent to the current block. The video decoding device can generate the final prediction signal for the current block based on the derived intra-frame prediction pattern. For example, the video decoding device can generate the final prediction signal by performing a weighted summation of the intra-frame prediction signal determined based on derivation and / or resolution and the inter-frame prediction signal determined based on derivation and / or resolution. The weights used in the weighted summation can vary depending on the location of each sample. For example, the weights can be adaptively determined based on the prediction information of the current block, information about the adjacent reconstructed regions, aspect ratio, etc.

[0148] For example, when the prediction technique for the current block is intra-frame prediction, the prediction mode for the current block can be a directional prediction mode, a planar mode (horizontal plane, vertical plane, or regular plane), or a DC mode.

[0149] As another example, when the prediction technique for the current block is intra-frame prediction, the video decoding device can determine the prediction mode for the current block as a prediction mode based on intra-frame geometric partitioning. The current block can be divided into one or more sub-regions according to the geometric partitioning, and prediction modes within the intra-frame prediction technique (including different directional prediction modes, planar modes, or DC modes) can be applied to each sub-region. The video decoding device can then perform a weighted summation of the prediction signals from each region to generate the final prediction signal for the current block.

[0150] As another example, when the prediction technique for the current block is intra-frame prediction, the video decoding device can determine that the prediction mode for the current block is a matrix-based intra-frame prediction mode. Based on the protocol between the video encoding and decoding devices, either the matrix index is signaled / resolved based on a predefined matrix, or the matrix itself is signaled / resolved. The video decoding device can generate the prediction signal for the current block based on either the matrix index or the matrix itself.

[0151] As another example, when the prediction technique for the current block is intra-frame prediction, the video decoding device can determine that the prediction mode for the current block is intra-frame template matching prediction mode. The video decoding device can define the adjacent reconstructed regions of the current block as templates and perform template matching within the defined regions to generate the prediction signal for the current block.

[0152] In yet another example, when the prediction technique for the current block is intra-frame prediction, the video decoding device can define the adjacent reconstructed regions of the current block as templates, and can use the same templates to derive the intra-frame prediction mode. By utilizing the derived mode, the final predicted signal for the current block can be generated. The template can include regions that are not adjacent to the current block.

[0153] The method for deriving the intra-prediction mode for the current block can vary. To indicate one or more methods, signaling / resolving flags and / or indexes can be used. Alternatively, a specific prediction mode can be used in a stable manner.

[0154] The following describes in detail the operation of the predictive actuator 808 in generating the final predictive signal.

[0155] This describes an example of determining the prediction mode of the current block as a mode for generating the final prediction signal based on a weighted sum of intra-prediction and inter-prediction signals. This mode can be determined based on signaling / parsing with a 1-bit flag. The intra-prediction and inter-prediction signals of the current block can be generated from derived information or based on signaling / parsing with some information items. The weights used in the weighted sum can be fixed values. The weights can be implicitly determined by utilizing information such as the prediction mode, the prediction signals, information about adjacent reconstructed regions of the current block, the size and / or aspect ratio of the current block, etc.

[0156] For example, inter-frame prediction signals can be generated as follows. For instance, a video decoding device can construct a list of motion vector candidates from the vicinity of the current block and from predefined locations that are not adjacent to it. The video decoding device can utilize neighborhood spatial motion vector candidates (… Figure 9a Candidates for non-neighboring spatial motion vectors (A0, A1, B0, B1, B2). Figure 9b Candidates for motion vectors in neighboring time (1 to 18) Figure 9a The candidate list is constructed using C0 and C1 (from the image), pairwise motion vector predictors (PAMVP), history-based motion vector predictors (HMVP), and zero motion vectors. When constructing the candidate list, a predetermined order can be utilized based on a protocol between the video encoding and decoding units. Here, based on the protocol between the video encoding and decoding units, the maximum number of candidates included in the candidate list can be n, where n is an integer of 1 or greater. Figure 9b In the diagram, w and h represent the width and height of the current block, respectively.

[0157] Figure 10This is a schematic diagram illustrating the region where template-based motion compensation is performed according to at least one embodiment of the present invention.

[0158] For example, a 1-bit flag indicating the use of template matching can be used for signal notification / parsing. If the above flag is set to 1, the video decoding device can perform template-based motion compensation for each candidate in the motion vector candidate list, such as... Figure 10 As shown. In Figure 10 In this context, the search range for template-based motion compensation is -b to +b on the x-axis and -a to +a on the y-axis. a and b can be fixed values ​​and / or values ​​determined based on the aspect ratio of the current block. For template-based motion compensation, a cost function can be used, such as the mean squared error (MSE) or sum of absolute errors (SAE) between the templates of the current block and the reference block. The video decoding device can perform motion compensation towards reducing the cost function. Motion compensation can be performed at the location of the sampled pixels within the search range.

[0159] For example, a video decoding device can calculate the cost between the template of the predicted block and the template of the current block for each candidate that has undergone motion compensation and is relative to the predicted block generated by motion compensation. The video decoding device can use the calculated cost as a basis for rearranging the candidates in the candidate list in ascending or descending order.

[0160] For example, a video decoding apparatus can generate an inter-frame prediction signal by utilizing the 0th motion vector candidate from a rearranged candidate list. Alternatively, the video decoding apparatus can generate an inter-frame prediction signal from a rearranged candidate list by utilizing motion vector candidates determined by signaling / parsing of indices or flags. In this case, as described above, template-based motion compensation is pre-applied to the motion vector candidates within the candidate list.

[0161] Alternatively, if the 1-bit flag indicating the use of template matching is set to 1, the video decoding apparatus can use motion vector candidates from a motion vector candidate list constructed based on the adjacency information of the current block, determined by signaling / resolving indices or flags, to generate an inter-frame prediction signal. In this case, as described above, template-based motion compensation is pre-applied to the motion vector candidates in the candidate list.

[0162] For example, an intra-frame prediction signal can be generated as follows. For instance, a video decoding apparatus can generate an intra-frame prediction signal by utilizing an intra-frame prediction mode determined based on signaling / parsing of prediction mode information. The video decoding apparatus can construct a most probable mode (MPM) list based on information about adjacent reconstructed regions of the current block. According to the implementation, the MPM list may include a primary MPM (PMPM) list and a secondary MPM (SMPM) list. The video decoding apparatus can construct the MPM list based on the prediction modes of adjacent reconstructed regions of the current block. A 1-bit flag can be used to indicate whether a prediction mode within the MPM list should be used. If the MPM list consists of a PMPM list and an SMPM list and the MPM flag is 1, a 1-bit flag can be used to indicate whether the index of a mode in the PMPM list or the index of a mode in the SMPM list should be used for signaling / parsing. If the MPM flag is 0, the intra prediction mode of the current block can be determined by signaling / parsing an index of one of the intra prediction modes other than those in the MPM list.

[0163] For example, a video decoding device can deduce the intra-prediction mode of the current block. In this case, one or more of various methods can be used as the derivation method, or a fixed method can be used. Alternatively, the derivation method can be determined by utilizing signaling / parsing.

[0164] The Template-based Intra Mode Derivation (TIMD) method is described below.

[0165] Video decoding devices can use the templates reconstructed from adjacent blocks and the template reference lines to deduce the intra-prediction mode of the current block. For example... Figure 11 As shown, the template for the current block is the reconstructed area on the top and left sides, and the template reference lines can be defined by one or more lines. Figure 11 In this context, b, c, d, e, and f can be integers greater than or equal to 1, can be fixed values, and can change according to the size and aspect ratio of the current block.

[0166] Figure 12 This is a schematic diagram illustrating the determination of intra-frame prediction mode candidates based on the location of samples according to at least one embodiment of the present invention.

[0167] As an example, candidates for prediction modes to be used in TIMD could include all available intra-directed prediction modes, DC modes, planar modes, etc., for the current block. Alternatively, some intra-prediction modes can be candidates based on the prediction modes of adjacent reconstructed regions of the current block (e.g., the left and top regions). For example, intra-prediction mode candidates could be determined based on the location of available samples within the reconstructed regions of the current block. Figure 12 As shown, this invention can check the availability of samples at positions A, B, C, and D relative to the current block, and can add candidate patterns based on the aspect ratio of the current block. Here, the values ​​of "a" and "b" can be determined based on the size of the current block.

[0168] As an example, for a current block of size W×H, when W / H=1 and sample A is available, candidates can add some modes with smaller angles (e.g., mode 67 or larger) than the 45° intra-directional prediction mode (e.g., mode 66). When sample B is available, the available mode with the smallest angle (e.g., mode 80) can be added to the candidates. The foregoing can also be applied to samples C and D. Furthermore, when the value of W / H is not 1, the availability of samples at positions A, B, C, and D can limit the candidateability of wide-angle modes, such as intra-directional prediction modes less than 45° or greater than 225°.

[0169] When determining intra-frame prediction mode candidates, the video decoding device can use template reference lines to predict the template for each candidate prediction mode, and can determine the order of prediction modes based on the cost between the prediction signal and the template. The reference line can be multiple lines within a reference line area. For example, a reference line and a group of prediction modes can be set as a candidate. Figure 13 As shown, if the available reference line is (1, 5) and the number of intra-prediction mode candidates is 3, six candidate groups can be formed. The video decoding device can predict templates for the six candidate groups and can rearrange the prediction modes in ascending or descending order based on the cost between the prediction signal and the template. The video decoding device can determine the intra-prediction mode based on the signaling / parsing of the index. Alternatively, the video decoding device can determine that the intra-prediction mode for the current block is the candidate at index 0 and the reference line in the rearranged candidates.

[0170] The following describes the decoder-side Intra Mode Derivation (DIMD) method.

[0171] Figure 14 This is a schematic diagram illustrating a template for the adjacent reconstruction of the current block according to at least one embodiment of the present invention.

[0172] Video decoding devices can use the templates reconstructed from adjacent blocks to deduce the intra-prediction mode of the current block. For example... Figure 14 As shown, a region of adjacent reconstructions of the current block with a size of W×H can be defined as a template. Figure 14 In this context, m and n can be integers of 1 or greater. If the upper right region of the current block is available ( Figure 14 (③) can then define the template to include the upper right region. If the lower left region of the current block is available ( Figure 14 (as shown in ②), then the template can be defined to include the lower left region. If both the upper right and lower left regions are available ( Figure 14 (④ in the original text) Then the template can be defined to include both the upper right region and the lower left region. If neither region is available ( Figure 14 If ① is defined, then a template can be defined to exclude these two regions.

[0173] exist Figure 14 In this context, a and b can be integers of 1 or greater. Here, a can be in the range of 0 to min (the length of the maximum available region, W or 2W), and b can be in the range of 0 to min (the height of the maximum available region, H or 2H). If part or all of the left-hand region of the current block is unavailable in the regions of the current block's adjacent reconstructions, then only the top-hand template can be used. Alternatively, if part or all of the top-hand region is unavailable in the regions of the current block's adjacent reconstructions, then only the left-hand template can be used. As another example, a template can be defined to exclude the top-left position of the current block.

[0174] When the template for the current block is defined as described above, the video decoding device can use the template's directionality to deduce the intra-frame prediction mode. For example, in Figure 14 In this context, when m and n are 3 or greater and p and q are 2 or greater integers, the video decoding device can calculate the gradients of p×q blocks in all regions of the template and, based on the calculated gradients, derive the intra-frame prediction mode corresponding to the most dominant gradient sequence. If in Figure 14 If m is less than p or n is less than q, the video decoding device can copy the outermost pixel value from the template so that m and n are equal to p and q respectively, and then the above process can be performed.

[0175] For example, for the template described above, the video decoding device can calculate its gradient. The video decoding device can calculate the gradient after filtering the template. Alternatively, the video decoding device can calculate the gradient of a portion of the template. For example, the template can be filtered using a smoothing filter with coefficients [1 / 4, 2 / 4, 1 / 4].

[0176] When calculating the gradient of a portion of the template, such as Figure 15As shown, the video decoding device can calculate a value containing u pixels for each u pixels ( Figure 15 p×p blocks (u=2) in ( Figure 15 The gradient of p=3 in the current video. u and / or p can be adaptively determined based on information such as the size of the current block, the resolution of the current video, and the aspect ratio of the current block.

[0177] For example, when calculating the directionality (gradient) block by block for the positions of pixels sampled from adjacent reconstructed templates, various methods can be used to determine the sampling positions. For instance... Figure 15 As shown, sampling positions can be set with equal pixel intervals, and gradients can be calculated at the sampling positions. Figure 16 As shown, the sampling position can be set by utilizing the partitioning and prediction information of the templates reconstructed from adjacent blocks. For example, relative to the boundaries of the experienced block partitions, for a certain number of pixels, the video decoding device can calculate the gradient of the region that is not adjacent to the block partition boundary without calculating the directionality.

[0178] For example, a video decoding apparatus can compute the gradients of adjacent reconstructed templates block by block, and then store the computed gradients in a table. The video decoding apparatus can map each computed gradient to a directional pattern that can be used to predict the current block, thus storing both the gradient strength and the corresponding prediction pattern in the table (hereinafter referred to as the "mapping table"). The gradient strength can be the absolute value of the gradient. The gradient strength can be the sum of the absolute values ​​of the gradient in the x-direction and the absolute values ​​of the gradient in the y-direction. The directional pattern that can be used for the current block can vary depending on the aspect ratio of the current block, the extent of the available area of ​​the reconstructed template, and other factors. For example, the video decoding apparatus first constructs a table (hereinafter referred to as the "directional table") to store the computed gradients. Subsequently, the video decoding apparatus can map the most dominant gradients in the directional table to the directional patterns that can be used to predict the current block, thereby determining the intra-frame prediction pattern for the current block.

[0179] For example, for adjacent refactored templates, the range of available templates can indicate the determination of the available patterns for the current block. Furthermore, the available patterns for the current block can be determined by considering both the aspect ratio of the current block and / or the available area within the template of the current block.

[0180] For adjacent reconstructed templates, as gradients are computed block by block to construct the table, the video decoding device defines the available directional prediction modes for the current block. Subsequently, relative to the current block, depending on the position of each block, the video decoding device can selectively add some directional prediction modes from the available directional prediction modes to the table.

[0181] For adjacent reconstructed templates, when deriving the directional prediction pattern of the current block on a block-by-block basis by acquiring information (e.g., gradients) and then accumulating the acquired information, the accumulative directionality (e.g., gradients) can vary depending on the location of each block. At this point, the definition of the location of each block and the different directionality by location can be determined as one of various examples.

[0182] For adjacent reconstructed templates, when calculating gradients block by block to construct a table, template regions are defined based on the positions of adjacent reconstructed templates in the current block. Subsequently, for each region, the directionality (gradient) or the corresponding directional intra-prediction mode that can be included in the table can be determined according to the protocol between the video encoding and decoding devices. The method of constructing the table can vary. For example, the video decoding device can construct a separate table for each template region, perform normalization for each template region, then aggregate the normalized tables, and use the aggregated tables to derive the intra-prediction mode. Alternatively, the video decoding device can construct a single table for all template regions, and use the constructed table to derive the intra-prediction mode.

[0183] For example, during the process of obtaining the template's gradient, if the gradient in the x-direction is zero or below a certain threshold, a vertical prediction pattern can be added to the table. Similarly, during the process of calculating the template's gradient, if the gradient in the y-direction is zero or below a certain threshold, a horizontal prediction pattern can be added to the table. Here, the threshold can be a fixed value or the median of the brightness values ​​of adjacent reconstructed templates.

[0184] In this invention, the video decoding device can derive the intra-frame prediction mode of the current block using 1) a reference block and a template of the inter-frame prediction signal reference block of the current block, 2) a reference block and a template of the reference block within a reference image of the current block, 3) a reference block, a template of the inter-frame prediction signal reference block, and a template of the current block, and 4) a reference block, a template of the reference block within a reference image, and a template of the current block. In this case, the reference block within the reference image can be obtained using a merge index.

[0185] For example, as shown in Figure 7, the video decoding device can use a reference block within a reference image to derive the intra-prediction mode of the current block. In this case, the reference motion vector (reference MV) indicating the reference block can be the final motion vector and / or the initial motion vector of the inter-prediction signal for the current block. The reference MV can be determined by constructing a list of motion vector candidates from adjacent reconstructed regions of the current block, followed by indexing signaling / parsing. For example, the final motion vector can be generated by applying template-based motion compensation to the initial motion vector, as described above.

[0186] For example, the intra-prediction mode of the current block can be derived by utilizing the reference block and the template of the reference block.

[0187] The video decoding apparatus uses reference lines within a template of a reference block to generate a prediction signal based on each intra-prediction mode candidate for the current block. The video decoding apparatus can calculate the cost between the reference block and each prediction signal, and can determine that the reference lines and intra-prediction modes for the current block correspond to the reference lines and intra-prediction modes with the minimum cost. Intra-prediction mode candidates for the current block can be generated based on information about adjacent reconstructed regions, the size of the current block, the aspect ratio of the current block, etc., as described in the TIMD method above. The information about the reference lines can be the same as the information about the reference lines that can be used for the current block. For example, the nearest neighbor line can be used as a reference line.

[0188] When generating a prediction signal based on each intra-frame prediction mode candidate of the current block by utilizing reference lines within the template of the reference block, prediction can be performed only on a portion of the reference block, rather than the entire reference block, such as... Figure 18 As shown, a portion of the reference block is the reference block region excluding the lower right (Wb) × (Ha) region. The video decoding device can calculate the cost between the reference block in the same region and each predicted signal, and can determine the reference line corresponding to the minimum cost and determine that the intra-prediction mode of the current block is the intra-prediction mode corresponding to the minimum cost. A portion of the region can be as follows: Figure 18 The values ​​of a and b are defined as shown. a and b can be integers greater than 1 and less than or equal to W and H, respectively. The values ​​of a and b can be predefined values ​​based on the protocol between the video encoding and decoding devices, or they can be values ​​determined based on the aspect ratio of the current block.

[0189] The video decoding apparatus can use a reference block and its template to calculate the directionality of a p×q block with integers p and q≥2. Based on the calculated directionality, the video decoding apparatus can construct a mapping table or a directionality table as described in the DIMD method above. The video decoding apparatus can derive the intra-frame prediction mode with the highest cumulative count value in the mapping table as the intra-frame prediction mode for the current block. Alternatively, the video decoding apparatus can search for the directionality with the highest cumulative count value in the directionality table, and then derive the intra-frame prediction mode corresponding to the searched directionality (mapped to the searched directionality) as the intra-frame prediction mode for the current block. In this case, when calculating a region-specific table for a table composed of reference block templates, a region corresponding to the same partition as the reference block can be set to construct a region-specific table.

[0190] When using a reference block and its template, in order to calculate the directionality of a p×q block of integers p and q greater than or equal to 2, the video decoding device does not calculate the directionality in all regions within the reference block. Instead, as... Figure 18 As shown, directionality can be calculated only in certain regions.

[0191] As another example, the intra-prediction mode of the current block can be derived by utilizing the template of the reference block, the reference block itself, and the template of the current block.

[0192] The video decoding apparatus can use the reference lines of the template and the intra-prediction mode candidates of the current block as a basis to form n groups of intra-prediction mode candidates (where n is an integer of 1 or greater), as described in the TIMD method above. Here, each group of intra-prediction mode candidates includes reference line candidates and intra-prediction mode candidates. The video decoding apparatus generates a prediction signal based on the intra-prediction mode of each candidate group from the template of the reference block. The video decoding apparatus can calculate the cost between the reference block and each prediction signal, and determine the intra-prediction mode of the current block based on each candidate group corresponding to the minimum cost. For example, the video decoding apparatus can determine that the reference lines and intra-prediction mode of the current block are reference line candidates and intra-prediction mode candidates from the candidate group corresponding to the minimum cost. The video decoding apparatus may not predict the entire reference block, but instead generate a prediction signal for a portion of the reference block, such as... Figure 18 As shown, the cost of each candidate is calculated from this.

[0193] The video decoding apparatus uses the directionality generated according to the DIMD method described above as a basis to construct a table for its template of the current block, and also constructs tables for a reference block and its template. The video decoding apparatus can combine the two tables into a single mapping table or directionality table to sum the same directionality values. When combined, the weights assigned to the cumulative values ​​of the directionality or prediction modes in the table constructed from the template of the current block can be different from the weights assigned to the cumulative values ​​of the same directionality or prediction modes in the tables constructed from the reference block and its template. The video decoding apparatus can deduce the intra-prediction mode of the current block from the intra-directional prediction mode with the highest cumulative count value in the newly constructed mapping table. Alternatively, the video decoding apparatus can search for the directionality with the highest cumulative count value in the directionality table, and then determine the intra-directional prediction mode corresponding to the searched directionality (mapped to the searched directionality) as the intra-prediction mode of the current block.

[0194] The process for determining the weights used to generate the final predicted signal is described below.

[0195] The video decoding device can generate inter-frame prediction signals and intra-frame prediction signals separately, and then generate the final prediction signal by weighted summation of the two signals.

[0196] As an example, when generating intra-frame prediction signals based on directional prediction modes, the weights of inter-frame and intra-frame prediction signals can be determined based on the directionality of the prediction mode. When intra-frame prediction signals are not generated based on directional prediction modes, the weights can be determined based on the prediction techniques and prediction modes of the adjacent reconstructed regions of the current block.

[0197] For example, relative to the intra-frame directional prediction mode at 135° ( Figure 3a In mode 34), the directional prediction mode can be divided into a first region and a second region. Here, the first region includes directional prediction modes of mode 34 or higher, and the second region includes directional prediction modes of mode 34 and lower. If the intra-frame prediction mode of the current block falls within the first region, the video decoding device can horizontally divide the current block into n parts (where n is an integer of 1 or greater), such as... Figure 19a As shown in the diagram, the weights of inter-frame prediction blocks and intra-frame prediction blocks can be determined for each region. If the intra-frame prediction mode of the current block falls within the second region, the video decoding device can vertically divide the current block into n parts (where n is an integer greater than or equal to 1), as shown in the diagram. Figure 19b As shown, the weights of inter-frame prediction blocks and intra-frame prediction blocks can be determined for each region. At this point, the first and second regions can be divided simultaneously, including a wide angle. Figure 3b Patterns 67 to 80, -14 to -1). For an integer k of 1 or greater, n can be 2. k And it can be determined differently based on the current block size and aspect ratio. Figure 19a and Figure 19b In the text, the numbers within a sub-block indicate the sub-block index.

[0198] For example, when n is 4, the current block is divided into 4 sub-blocks horizontally or vertically, and a weight can be determined for each sub-block as shown in Table 2.

[0199] [Table 2] Furthermore, when n is 8, the current block is divided into 8 sub-blocks horizontally or vertically, and a weight can be determined for each sub-block as shown in Table 3.

[0200] [Table 3] According to Table 2 or Table 3, the closer a sub-block is to the reference line, the greater the weight assigned to the intra-frame prediction signal.

[0201] As another example, such as Figure 20 As shown, the directional pattern can be divided into regions 0 to 4. In Figure 20 In this context, region 0 includes modes 56 and above, and the first region includes modes 45 to 55. The second region includes modes 24 to 44, the third region includes modes 13 to 23, and the fourth region includes modes 12 and below. The current block can be divided into sub-blocks based on each region, and the weights of the inter-frame prediction signal and intra-frame prediction signal for the current block can be determined. At this point, the current block can be divided into n parts (where n is an integer of 1 or greater). For an integer k of 1 or greater, n can be 2^k. k Furthermore, it can be determined differently based on the current block size and aspect ratio.

[0202] For example, if the intra-prediction mode of the current block falls within the first region, the video decoding device can, as follows: Figure 19a The current block is horizontally divided as shown, and the weight of each sub-block can be determined as shown in Table 2 or Table 3. If the intra-prediction mode of the current block falls within the third region, the video decoding device can proceed as follows: Figure 19b The current block is divided vertically as shown, and the weight of each sub-block can be determined as shown in Table 2 or Table 3.

[0203] If the intra-prediction mode of the current block falls within region 0, the video decoding device can, as follows: Figure 21a The current block is divided as shown, and the weight of each sub-block can be determined as shown in Table 2 or Table 3. If the intra-prediction mode of the current block falls within the second region, the video decoding device can proceed as follows: Figure 21b The current block is divided as shown, and the weight of each sub-block can be determined as shown in Table 2 or Table 3. If the intra-prediction mode of the current block falls within the fourth region, the video decoding device can proceed as follows: Figure 21c The current block is divided as shown, and the weight of each sub-block can be determined as shown in Table 2 or Table 3. Figures 21a to 21c In the diagram, the numbers within the sub-block indicate the sub-block index. As stated above, according to Table 2 or Table 3, the closer the sub-block is to the reference line, the greater the weight assigned to the intra-frame prediction signal.

[0204] The video decoding device can use the weights determined as described above to perform a weighted summation of the inter-frame prediction signal and the intra-frame prediction signal, thereby generating the final prediction signal for the current block.

[0205] In the following text, by utilizing Figure 22 and Figure 23 The illustration describes the method of deriving intra-frame prediction modes based on inter-frame prediction signals.

[0206] Figure 22 This is a flowchart of a method for encoding the current block by a video encoding device according to at least one embodiment of the present invention.

[0207] The video encoding apparatus obtains information indicating the motion vector of the current block (S2200). For example, the video encoding apparatus can obtain the information indicating the motion vector from a higher level. Alternatively, the video encoding apparatus can determine the information indicating the motion vector from the perspective of rate-distortion optimization. The information indicating the motion vector can be a flag or an index.

[0208] The video encoding device can construct a list of motion vector candidates from predefined positions that are adjacent to and not adjacent to the current block.

[0209] The video encoding device can obtain a flag from a higher level instructing it to use template matching.

[0210] If the flag indicating the use of template matching is true, the video coding apparatus generates a reference block candidate indicated by each candidate motion vector in the motion vector candidate list. The video coding apparatus can perform template-based motion compensation on each candidate motion vector based on the template of the current block and the template of the reference block candidate. The video coding apparatus can rearrange the candidate motion vectors in the motion vector candidate list by applying template matching to the template of the prediction block generated by motion compensation and the template of the current block.

[0211] Subsequently, the video encoding device can encode the flag indicating the use of template matching.

[0212] The video encoding device determines the motion vector of the current block from the motion vector candidate list based on the information indicating the motion vector (S2202).

[0213] The video encoding device generates the inter-frame prediction signal of the current block based on the reference block that exists within the reference picture and is indicated by the motion vector of the current block (S2204).

[0214] The video coding device derives the intra-prediction mode of the current block based on the reference block and the template of the reference block (S2206).

[0215] For example, the intra-prediction mode of the current block can be derived by using the templates of the reference block and the reference block of the current block.

[0216] The video coding apparatus can construct intra-prediction mode candidates for the current block based on information about adjacent reconstructed regions, the size of the current block, or the aspect ratio of the current block, as described in the TIMD method above. The video coding apparatus can use reference lines within a template of a reference block to generate a prediction signal based on each intra-prediction mode candidate for the current block. The video coding apparatus can calculate the cost between the reference block and the generated prediction signal and determine whether the intra-prediction mode for the current block is the intra-prediction mode candidate corresponding to the minimum cost.

[0217] By utilizing reference blocks and their templates, the video coding apparatus can calculate the directionality of blocks of a preset size, as in the DIMD method described above, and can construct a mapping table or a directionality table based on the calculated directionality. The video coding apparatus can derive the intra-prediction mode with the highest cumulative count value from the mapping table of intra-prediction modes for the current block. The video coding apparatus can search for the directionality with the highest cumulative count value in the directionality table, and then derive the intra-directional prediction mode corresponding to the searched directionality (mapped to the searched directionality) as the intra-prediction mode for the current block.

[0218] As another example, the intra-prediction mode of the current block can be derived by utilizing the reference block of the current block, the template of the reference block, and the template of the current block.

[0219] The video coding apparatus can generate candidate groups of intra-prediction modes for the current block based on the reference lines of the template and intra-prediction mode candidates, as in the TIMD method described above. Here, each intra-prediction mode candidate group can include reference line candidates and intra-prediction mode candidates. The video coding apparatus can use the reference lines within the template of the reference block to generate a prediction signal based on each intra-prediction mode candidate group of the current block. The video coding apparatus can calculate the cost between the reference block and the generated prediction signal, and can determine the intra-prediction mode and reference lines for the current block based on the intra-prediction mode candidate groups corresponding to the minimum cost.

[0220] The video coding apparatus can form a first table based on the directionality within the template of the current block, and can form a second table based on the directionality within the template of a reference block. The video coding apparatus can combine the first and second tables to form a mapping table or a directionality table. The video coding apparatus can derive the intra-prediction mode of the current block from the mapping table, which is the intra-directional prediction mode with the highest cumulative count value. Alternatively, the video coding apparatus can search for the directionality with the highest cumulative count value in the directionality table, and then derive the intra-directional prediction mode corresponding to the searched directionality (mapped to the searched directionality) as the intra-prediction mode of the current block.

[0221] The video encoding device generates the intra-prediction signal for the current block based on the intra-prediction mode (S2208).

[0222] The video coding device generates weights for inter-frame prediction signals and intra-frame prediction signals based on the directionality of the intra-frame prediction mode (S2210).

[0223] The video coding device divides the current block horizontally or vertically to generate sub-blocks based on the directionality of the intra-prediction mode. As shown in Table 2 or Table 3, the closer each sub-block is to the reference line, the greater the weight the video coding device can allocate to the intra-prediction signal.

[0224] The video coding device performs a weighted summation of the inter-frame prediction signal and the intra-frame prediction signal based on weights to generate the final prediction signal for the current block (S2212).

[0225] The video encoding device encodes the information indicating the motion vector (S2214).

[0226] The video coding apparatus generates a residual block by subtracting the final predicted signal from the current block. The video coding apparatus transforms / quantizes the residual block to generate quantized transform coefficients and encodes the quantized transform coefficients.

[0227] Figure 23 This is a flowchart of a method for reconstructing the current block by a video encoding device according to at least one embodiment of the present invention.

[0228] The video decoding device decodes the information indicating the motion vector of the current block (S2300). The information indicating the motion vector can be a flag or an index.

[0229] The video decoding device can construct a list of motion vector candidates from the predefined positions that are adjacent to and not adjacent to the current block.

[0230] The video decoding device can use a template matching flag from the bitstream decoding instruction.

[0231] If the flag indicating the use of template matching is true, the video decoding apparatus generates a candidate reference block indicated by each candidate motion vector in the motion vector candidate list. The video decoding apparatus can perform template-based motion compensation on each candidate motion vector based on the template of the current block and the template of the candidate reference block. The video decoding apparatus can rearrange the candidate motion vectors in the motion vector candidate list by applying template matching to the template of the prediction block generated by motion compensation and the template of the current block.

[0232] The video decoding device determines the motion vector of the current block from the motion vector candidate list based on the information indicating the motion vector of the current block (S2302).

[0233] The video decoding device generates an inter-frame prediction signal for the current block based on a reference block that exists within the reference image and is indicated by the motion vector of the current block (S2304).

[0234] The video decoding device derives the intra-prediction mode of the current block based on the reference block and the template of the reference block (S2306).

[0235] For example, the intra-prediction mode of the current block can be derived by utilizing the reference block and the template of the reference block.

[0236] The video decoding apparatus can construct intra-prediction mode candidates for the current block based on information about adjacent reconstructed regions, the size of the current block, or the aspect ratio of the current block, as in the TIMD method described above. The video decoding apparatus can use reference lines within a template of a reference block to generate a prediction signal based on each intra-prediction mode candidate for the current block. The video decoding apparatus can calculate the cost between the reference block and the generated prediction signal, and can determine the intra-prediction mode candidate corresponding to the minimum cost of the intra-prediction modes for the current block.

[0237] The video decoding device can use a reference block and a template of the reference block to calculate the directionality of a block of a preset size, as in the DIMD method described above, and can construct a mapping table or a directionality table based on the calculated directionality. The video decoding device can deduce the intra-prediction mode of the current block, which is the intra-directional prediction mode with the highest cumulative count value in the mapping table. The video decoding device can search for the directionality with the highest cumulative count value in the directionality table, and then deduce the intra-directional prediction mode corresponding to the searched directionality (mapped to the searched directionality) as the intra-prediction mode of the current block.

[0238] As another example, the intra-prediction mode of the current block can be derived by utilizing the reference block of the current block, the template of the reference block, and the template of the current block.

[0239] The video decoding apparatus can generate groups of intra-prediction mode candidates for the current block based on the reference lines of the template and intra-prediction mode candidates, as in the TIMD method. Here, each group of intra-prediction mode candidates can include reference line candidates and intra-prediction mode candidates. The video decoding apparatus can use the reference lines within the template of the reference block to generate a prediction signal based on each group of intra-prediction mode candidates for the current block. The video decoding apparatus can calculate the cost between the reference block and the generated prediction signal, and can determine the intra-prediction mode and reference lines for the current block based on the group of intra-prediction mode candidates corresponding to the minimum cost.

[0240] The video decoding apparatus can form a first table based on the directionality within the template of the current block, and a second table based on the directionality within the template of a reference block. The video decoding apparatus can combine the first and second tables to form a mapping table or a directionality table. The video decoding apparatus can derive the intra-prediction mode of the current block, which is the intra-directional prediction mode with the highest accumulated count value in the mapping table. Alternatively, the video decoding apparatus can search for the directionality with the highest accumulated count value in the directionality table, and then derive the intra-directional prediction mode corresponding to the searched directionality (mapped to the searched directionality) as the intra-prediction mode of the current block.

[0241] The video decoding device generates the intra-prediction signal for the current block based on the intra-prediction mode (S2308).

[0242] The video decoding device generates weights for inter-frame prediction signals and intra-frame prediction signals based on the directionality of the intra-frame prediction mode (S2310).

[0243] The video decoding device divides the current block horizontally or vertically to generate sub-blocks based on the directionality of the intra-frame prediction mode. As shown in Table 2 or Table 3, the closer each sub-block is to the reference line, the greater the weight the video decoding device can allocate to the intra-frame prediction signal.

[0244] The video decoding device performs a weighted summation of the inter-frame prediction signal and the intra-frame prediction signal based on the weights to generate the final prediction signal for the current block (S2312).

[0245] The video decoding device decodes the quantized transform coefficients from the bitstream and performs inverse quantization / inverse transform on the quantized transform coefficients to reconstruct the residual block. The video decoding device sums the final predicted signal of the current block and the residual block to generate the reconstructed block of the current block.

[0246] Although the steps in the various flowcharts are described in sequence, these steps merely exemplify the technical ideas of some embodiments of the invention. Therefore, those skilled in the art to which this invention pertains can perform the steps by changing the order described in the various figures or by performing two or more steps in parallel. Thus, the steps in the various flowcharts are not limited to the chronological order shown.

[0247] It should be understood that the above description presents illustrative embodiments that can be implemented in various other ways. The functionality described in some embodiments can be implemented by hardware, software, firmware, and / or combinations thereof. It should also be understood that the functional components described in this invention are designated as "...units" to emphasize their potential for independent implementation.

[0248] On the other hand, the various methods or functions described in some embodiments can be implemented as instructions stored in a non-volatile recording medium, which can be read and executed by one or more processors. The non-volatile recording medium can include various types of recording devices, such as those storing data in a computer system-readable form. For example, the non-volatile recording medium can include storage media such as erasable programmable read-only memory (EPROM), flash memory drives, optical disk drives, magnetic hard disk drives, and solid-state drives (SSDs), etc.

[0249] Although exemplary embodiments of the invention have been described for illustrative purposes, those skilled in the art will understand that various modifications, additions, and substitutions can be made without departing from the spirit and scope of the invention. Therefore, embodiments of the invention have been described for the sake of brevity and clarity. The scope of the technical concept of the embodiments of the invention is not limited to the examples. Accordingly, those skilled in the art will understand that the scope of the invention should not be limited by the embodiments explicitly described above, but rather by the claims and their equivalents.

[0250] Cross-reference to related applications This application claims priority and benefit to Korean Patent Application No. 10-2023-0170532, filed on November 30, 2023, and Korean Patent Application No. 10-2024-0146612, filed on October 24, 2024, the entire contents of which are incorporated herein by reference.

Claims

1. A method for reconstructing a current block using a video decoding device, the method comprising: Based on the information indicating the motion vector of the current block, determine the motion vector of the current block from the motion vector candidate list; The inter-frame prediction signal for the current block is generated based on a reference block that exists within the reference image and is indicated by the motion vector of the current block. The intra-prediction mode of the current block is derived based on the reference block and the template of the reference block. Intra-prediction signals for the current block are generated based on intra-prediction modes. as well as The final prediction signal for the current block is generated based on the inter-frame prediction signal and the intra-frame prediction signal.

2. The method according to claim 1, further comprising: Decode information indicating motion vectors from the bitstream; as well as Construct a list of motion vector candidates at adjacent positions of the current block and at predefined positions that are not adjacent to the current block.

3. The method according to claim 1, further comprising: Generate reference block candidates indicated by each candidate motion vector in the list of motion vector candidates; as well as Motion compensation is performed on each candidate motion vector based on the template of the current block and the templates of the candidate reference blocks.

4. The method of claim 3, further comprising: Based on motion compensation and the template of the current block, candidate motion vectors in the motion vector candidate list are rearranged by applying template matching to the template of the prediction block.

5. The method according to claim 1, wherein, The derivation of intra-frame prediction modes includes: The prediction signal is generated based on each intra-frame prediction mode candidate of the current block by utilizing reference lines within the template of the reference block. Calculate the cost between the reference block and the generated predicted signal; and The candidate intra-prediction mode corresponding to the minimum cost is determined as the intra-prediction mode for the current block.

6. The method of claim 5, further comprising: Based on information about the adjacent reconstructed regions of the current block, the size of the current block, or the aspect ratio of the current block, construct intra-prediction mode candidates for the current block.

7. The method according to claim 1, wherein, The derivation of intra-frame prediction modes includes: The orientation of a block of a preset size is calculated by using a reference block and a template of the reference block. The calculated directionality is used to form a table; and The directional prediction mode with the highest cumulative count value in the table is derived as the intra-prediction mode for the current block.

8. The method according to claim 1, wherein, The derivation of intra-frame prediction modes includes: The prediction signal is generated based on each intra-frame prediction mode candidate group of the current block by utilizing reference lines within the template of the reference block. Calculate the cost between the reference block and the generated predicted signal; and The intra-prediction mode and reference line for the current block are determined based on the intra-prediction mode candidate group corresponding to the minimum cost.

9. The method of claim 8, further comprising: The intra-prediction mode candidate group for the current block is generated based on the reference line of the template for the current block and the intra-prediction mode candidates. Each of the intra-prediction mode candidate groups includes a reference line candidate and an intra-prediction mode candidate.

10. The method according to claim 1, wherein, The derivation of intra-frame prediction modes includes: The first table is formed based on the directionality within the template of the current block; The second table is formed based on the directionality within the reference block and the template of the reference block; A combined table is generated by combining the first table and the second table; and The directional prediction mode with the highest cumulative count value in the combined table is derived as the intra-prediction mode for the current block.

11. The method according to claim 1, wherein, The generation of the final prediction signal includes: Weights for generating inter-frame and intra-frame prediction signals are generated based on the directionality of the intra-frame prediction mode; and The final prediction signal for the current block is generated by weighted summation of the inter-frame prediction signal and the intra-frame prediction signal based on weights.

12. The method according to claim 11, wherein, The generated weights include: Based on the directionality of intra-frame prediction modes, sub-blocks are generated by horizontally or vertically dividing the current block; and As each sub-block gets closer to the reference line of the current block, a larger weight is assigned to the intra-frame prediction signal.

13. A method for encoding a current block using a video encoding device, the method comprising: Obtain information indicating the motion vector of the current block; Based on the information indicating the motion vector, determine the motion vector of the current block from the motion vector candidate list; The inter-frame prediction signal for the current block is generated based on a reference block that exists within the reference image and is indicated by the motion vector of the current block. The intra-prediction mode of the current block is derived based on the reference block and the template of the reference block. Intra-prediction signals for the current block are generated based on intra-prediction modes. as well as The final prediction signal for the current block is generated based on the inter-frame prediction signal and the intra-frame prediction signal.

14. The method of claim 13, further comprising: Construct a list of motion vector candidates at adjacent positions of the current block and at predefined positions that are not adjacent to the current block; as well as Encodes information indicating motion vectors.

15. The method of claim 13, further comprising: Generate reference block candidates indicated by each candidate motion vector in the list of motion vector candidates; as well as Based on the template of the current block and the template of the reference block, perform template-based motion compensation for each candidate motion vector in the motion vector candidate list.

16. The method according to claim 13, wherein, The generation of the final prediction signal includes: Weights for generating inter-frame and intra-frame prediction signals are generated based on the directionality of the intra-frame prediction mode; and The final prediction signal for the current block is generated by weighted summation of the inter-frame prediction signal and the intra-frame prediction signal based on weights.

17. A method for providing video data to a video decoding device, the method comprising: Encode video data into a bitstream; as well as Send the bitstream to the video decoding device. The encoded video data includes: Obtain information indicating the motion vector of the current block; Based on the information indicating the motion vector, determine the motion vector of the current block from the motion vector candidate list; The inter-frame prediction signal for the current block is generated based on a reference block that exists within the reference image and is indicated by the motion vector of the current block. The intra-prediction mode of the current block is derived based on the reference block and the template of the reference block. Generate the intra-prediction signal for the current block based on the intra-prediction mode; and The final prediction signal for the current block is generated based on the inter-frame prediction signal and the intra-frame prediction signal.

Images