Video encoding method and apparatus using template-based prediction

Abstract

The embodiment discloses a video encoding method and apparatus using template-based prediction. In the present embodiment, a video decoding device decodes an index indicating a distance between a current block and a template region (adj). A video decoding apparatus identifies a template region based on a distance between a current block and the template region and a shape of the template region. The video decoding apparatus searches for an optimal template in the search region using the template region based on the template matching, and then generates a prediction block of the current block from a reference block corresponding to the optimal template.

Description

Technical Field

[0001] The present invention relates to a video encoding method and device using template-based prediction. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and may not constitute prior art.

[0003] Since video data has a large amount of data compared to audio or still image data, without a process for compression, the video data requires a large amount of hardware resources (including memory) to store or transmit the video data.

[0004] Thus, encoders are generally used to compress and store or transmit video data. Decoders receive compressed video data, decompress the received compressed video data, and play the decompressed video data. Video compression technologies include H.264 / Advanced Video Coding (AVC), High Efficiency Video Coding (HEVC), and Versatile Video Coding (VVC), which have improved coding efficiency of about 30% or more compared to HEVC.

[0005] However, as image size, resolution and frame rate gradually increase, the amount of data to be encoded also increases. Thus, a new compression technology that provides higher encoding efficiency and improved image enhancement effect than existing compression technologies is needed.

[0006] Template matching-based prediction selects multiple reconstructed samples adjacent to the current block as templates, and then uses the selected templates to detect or search for the most similar (i.e., the template with the best loss value within the reconstructed area of ​​the current block) template. Template matching-based prediction generates a prediction block for the current block based on a reference block corresponding to the best template detected. Template matching-based prediction can use loss functions such as the sum of absolute differences (SAD), mean square error (MSE), sum of absolute transform differences (SATD), etc. to measure the similarity between the template of the current block and the candidate templates in the reconstructed area. For the current block, the encoder can signal a 1-bit flag indicating whether to use template matching-based prediction, and the decoder can parse the flag and follow the same method as the encoder to search for the template with the best loss value, and the best template can be used to generate the prediction block of the current block.

[0007] When prediction based on template matching is performed as described above, the neighboring area of ​​the current block is used as the template of the current block. Therefore, in order to increase video encoding efficiency and enhance video quality, measures for effectively utilizing neighboring samples of the current block are required. Summary of the invention

[0008]

Technical issues

[0009] The present disclosure attempts to provide a video encoding method and apparatus for setting an adjacent or non-adjacent area as a template according to information of a neighboring reconstruction area of ​​a current block. The video encoding method and apparatus reconstruct the current block by performing prediction based on template matching, which is based on a set template.

[0010]

Technical solution

[0011] At least one aspect of the present invention provides a method for reconstructing a current chrominance block by a video decoding device. The method includes decoding an index (adjacent_idx) indicating a distance between a current block and a template area from a bitstream. The method also includes determining a template area based on the distance between the current block and the template area and the shape of the template area. The method also includes searching for an optimal template in a search area by using the template area and based on template matching. Here, the search area is limited to a reconstruction area of ​​the current block. The method also includes generating a prediction block of the current block based on a reference block corresponding to the optimal template.

[0012] Another aspect of the present invention provides a method for encoding a current block by a video encoding device. The method includes determining a distance index (adjacent_idx) indicating a distance between the current block and a template area. The method also includes determining a template area based on the distance between the current block and the template area and the shape of the template area. The method also includes searching for an optimal template in a search area by using the template area and based on template matching. Here, the search area is limited to a reconstruction area of ​​the current block. The method also includes generating a prediction block of the current block based on a reference block corresponding to the optimal template.

[0013] Another aspect of the present invention provides a computer-readable recording medium storing a bitstream generated by a video encoding method. The video encoding method includes determining a distance index (adjacent_idx) indicating a distance between a current block and a template area. The video encoding method also includes determining a template area based on the distance between the current block and the template area and the shape of the template area. The video encoding method also includes searching for an optimal template in a search area by using the template area and based on template matching. Here, the search area is limited to a reconstruction area of ​​the current block. The video encoding method also includes generating a prediction block of the current block based on a reference block corresponding to the optimal template.

[0014]

Beneficial Effects

[0015] As described above, the present disclosure provides a video encoding method and device for setting an adjacent or non-adjacent area as a template according to information of a neighboring reconstruction area of ​​a current block. The video encoding method and device reconstruct the current block by performing a prediction based on template matching (which is based on a set template). As a result, the video encoding method and device improve video encoding efficiency and improve video quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a block diagram of a video encoding device that may implement the techniques of this disclosure.

[0017] Figure 2 A method for partitioning blocks using a quadtree plus binary tree ternary tree (QTBTTT) structure is described.

[0018] Figure 3a and 3b Multiple intra prediction modes are shown including a wide-angle intra prediction mode.

[0019] Figure 4 Shows the neighboring blocks of the current block.

[0020] Figure 5 is a block diagram of a video decoding device that may implement the techniques of this disclosure.

[0021] Figure 6 is a block diagram detailing a portion of a video decoding device according to at least one embodiment of the present disclosure.

[0022] Figure 7 is a diagram illustrating a template region of a current block according to at least one embodiment of the present disclosure.

[0023] Figures 8a to 8c is a diagram illustrating the position and shape of a template area of ​​a current block according to some embodiments of the present disclosure.

[0024] Figure 9a and Figure 9b is a simplified diagram showing directional intra prediction modes.

[0025] Fig.10 is a flowchart of a prediction method of a current block performed by a video decoding apparatus according to at least one embodiment of the present invention.

[0026] Fig.11 is a flowchart of a prediction method of a current block performed by a video decoding apparatus according to another embodiment of the present invention.

[0027] Fig.12 is a diagram illustrating an initial template region according to at least one embodiment of the present disclosure.

[0028] Fig.13 is a diagram illustrating a final template region according to at least one embodiment of the present disclosure.

[0029] Fig.14a and Fig.14b is a diagram illustrating a final template region according to other embodiments of the present disclosure.

[0030] Fig.15 is a flowchart of a prediction method for a current block performed by a video encoding apparatus according to at least one embodiment of the present invention. DETAILED DESCRIPTION

[0031] Hereinafter, some embodiments of the present disclosure are described in detail with reference to the accompanying drawings. In the following description, the same reference numerals represent the same elements, although the elements are shown in different drawings. In addition, in the following description of some embodiments, for the purpose of clarity and brevity, detailed descriptions of related known components and functions may be omitted when it is considered to obscure the subject matter of the present disclosure.

[0032] Figure 1 is a block diagram of a video encoding device that can implement the technology of the present disclosure. Figure 1 A diagram of a video encoding apparatus and components of the apparatus is described.

[0033] The encoding device may include a picture separator 110, a predictor 120, a subtractor 130, a transformer 140, a quantizer 145, a rearrangement unit 150, an entropy encoder 155, an inverse quantizer 160, an inverse transformer 165, an adder 170, a loop filter unit 180 and a memory 190.

[0034] Each component of the encoding device can be implemented as hardware or software or as a combination of hardware and software. In addition, the function of each component can be implemented as software, and a microprocessor can also be implemented to execute the function of the software corresponding to each component.

[0035] A video is composed of one or more sequences including multiple pictures. Each picture is divided into multiple regions, and encoding is performed on each region. For example, a picture is split into one or more tiles or / and slices. Here, one or more tiles can be defined as tile groups. Each tile or / and slice is split into one or more coding tree units (CTUs). In addition, each CTU is split into one or more coding units (CUs) through a tree structure. The information applied to each coding unit (CU) is encoded as the syntax of the CU, and the information commonly applied to the CU contained in a CTU is encoded as the syntax of the CTU. In addition, the information commonly applied to all blocks in a slice is encoded as the syntax of the slice header, and the information applied to all blocks constituting one or more pictures is encoded as a picture parameter set (PPS) or a picture header. In addition, the information commonly referenced by multiple pictures is encoded into a sequence parameter set (SPS). In addition, the information commonly referenced by one or more SPSs is encoded into a video parameter set (VPS). Further, the information commonly applied to a tile or tile group can also be encoded as the syntax of the tile or tile group header. The syntax included in the SPS, PPS, slice header, tile, or tile group header may be referred to as a high-level syntax.

[0036] The picture partitioner 110 determines the size of a coding tree unit (CTU). Information on the size of the CTU (CTU size) is encoded as a syntax of an SPS or a PPS and delivered to a video decoding device.

[0037] The picture divider 110 divides each picture constituting a video into a plurality of coding tree units (CTUs) having a predetermined size, and then recursively divides the CTUs by using a tree structure. A leaf node in the tree structure becomes a coding unit (CU), which is a basic unit of encoding.

[0038] The tree structure may be a quadtree (QT), in which a higher node (or parent node) is divided into four lower nodes (or child nodes) of the same size. The tree structure may also be a binary tree (BT), in which a higher node is divided into two lower nodes. The tree structure may also be a ternary tree (TT), in which a higher node is divided into three lower nodes at a ratio of 1:2:1. The tree structure may also be a structure in which two or more of the QT structure, the BT structure, and the TT structure are mixed. For example, a quadtree plus binary tree (QTBT) structure may be used or a quadtree plus binary tree ternary tree (QTBTTT) structure may be used. Here, a binary tree ternary tree (BTTT) is added to the tree structure to be referred to as a multi-type tree (MTT).

[0039] Figure 2 is a diagram for describing a method of partitioning a block by using a QTBTTT structure.

[0040] like Figure 2As shown, the CTU can be first split into a QT structure. The quadtree partitioning can be recursive until the size of the partitioned block reaches the minimum block size (MinQTSize) of the leaf node allowed in QT. The first flag (QT_split_flag) indicating whether each node of the QT structure is divided into four nodes of the lower layer is encoded by the entropy encoder 155 and transmitted to the video decoding device. When the leaf node of QT is not larger than the maximum block size (MaxBTSize) of the root node allowed in BT, the leaf node can be further divided into at least one of the BT structure or the TT structure. There may be multiple partitioning directions in the BT structure and / or the TT structure. For example, there may be two directions, namely, the direction in which the blocks of the corresponding node are horizontally divided and the direction in which the blocks of the corresponding node are vertically divided. As Figure 2 As shown, when MTT splitting starts, a second flag (mtt_split_flag) indicating whether a node is split and a flag additionally indicating a split direction (vertical or horizontal) and / or a flag indicating a split type (binary or ternary) if the node is split are encoded by the entropy encoder 155 and signaled to the video decoding device.

[0041] Alternatively, before encoding the first flag (QT_split_flag) indicating whether each node is split into four nodes of the lower layer, a CU split flag (split_cu_flag) indicating whether the node is split may also be encoded. When the value of the CU split flag (split_cu_flag) indicates that each node is not split, the block of the corresponding node becomes a leaf node in the split tree structure and becomes a CU as a basic unit of encoding. When the value of the CU split flag (split_cu_flag) indicates that each node is split, the video encoding device first starts encoding the first flag through the above scheme.

[0042] When QTBT is used as another example of a tree structure, there may be two types, namely, a type in which the block of the corresponding node is horizontally divided into two blocks of the same size (i.e., symmetrical horizontal division) and a type in which the block of the corresponding node is vertically divided into two blocks of the same size (i.e., symmetrical vertical division). A split flag (split_flag) indicating whether each node of the BT structure is split into blocks of the lower layer and split type information indicating the split type are encoded by the entropy encoder 155 and delivered to the video decoding device. At the same time, a type in which the block of the corresponding node is split into two blocks that are asymmetric to each other may be presented in addition. The asymmetric form may include a form in which the block of the corresponding node is divided into two rectangular blocks having a size ratio of 1:3, or may also include a form in which the block of the corresponding node is divided in a diagonal direction.

[0043] A CU may have different sizes depending on the QTBT or QTBTTT split from a CTU. Hereinafter, a block corresponding to a CU to be encoded or decoded (i.e., a leaf node of a QTBTTT) is referred to as a "current block". Due to the QTBTTT split, the shape of the current block may be a rectangular shape in addition to a square shape.

[0044] The predictor 120 predicts the current block to generate a predicted block. The predictor 120 includes an intra predictor 122 and an inter predictor 124.

[0045] In general, each of the current blocks in a picture may be predictively encoded. Typically, the prediction of the current block may be performed using intra-prediction techniques (using data from the picture including the current block) or inter-prediction techniques (using data from a picture coded before the picture including the current block). Inter-prediction includes both unidirectional prediction and bidirectional prediction.

[0046] The intra-frame predictor 122 predicts the pixels in the current block by using the pixels (reference pixels) located on the neighbors of the current block in the current picture including the current block. There are multiple intra-frame prediction modes according to the prediction direction. For example, as shown in FIG. 3A, the multiple intra-frame prediction modes may include 2 non-directional modes including a plane mode and a DC mode, and may include 65 directional modes. Neighboring pixels and the arithmetic equation to be used are defined differently according to each prediction mode.

[0047] In order to perform effective directional prediction on the current block having a rectangular shape, a directional mode (#67 to #80, intra prediction mode #-1 to #-14) as illustrated by the dotted arrows in FIG. 3B may be additionally used. The directional mode may be referred to as a "wide-angle intra prediction mode". In FIG. 3B, the arrow indicates the corresponding reference sample used for prediction and does not indicate the prediction direction. The prediction direction is opposite to the direction indicated by the arrow. When the current block has a rectangular shape, the wide-angle intra prediction mode is a mode in which prediction is performed in a direction opposite to a specific directional mode without additional bit transmission. In this case, in the wide-angle intra prediction mode, some wide-angle intra prediction modes available for the current block may be determined by the ratio of the width and height of the current block having a rectangular shape. For example, when the current block has a rectangular shape with a height less than the width, a wide-angle intra prediction mode (intra prediction mode #67 to #80) with an angle less than 45 degrees is available. When the current block has a rectangular shape with a width greater than the height, a wide-angle intra prediction mode with an angle greater than -135 degrees may be used.

[0048] The intra-frame predictor 122 may determine an intra-frame prediction to be used for encoding the current block. In some embodiments, the intra-frame predictor 122 may encode the current block by using a plurality of intra-frame prediction modes, and may further select an appropriate intra-frame prediction mode to be used from a test mode. For example, the intra-frame predictor 122 may calculate a rate-distortion value by using a rate-distortion analysis for a plurality of tested intra-frame prediction modes, and may further select an intra-frame prediction mode having the best rate-distortion characteristics in the test mode.

[0049] The intra-frame predictor 122 selects an intra-frame prediction mode among a plurality of intra-frame prediction modes and predicts the current block by using adjacent pixels (reference pixels) and an arithmetic equation determined according to the selected intra-frame prediction mode. Information about the selected intra-frame prediction mode is encoded by the entropy encoder 155 and delivered to the video decoding device.

[0050] The inter-frame predictor 124 generates a prediction block for the current block by using a motion compensation process. The inter-frame predictor 124 searches for a block most similar to the current block in a reference picture that is encoded and decoded earlier than the current picture, and generates a prediction block for the current block by using the searched block. In addition, a motion vector (MV) is generated, which corresponds to the displacement between the current block in the current picture and the prediction block in the reference picture. Typically, motion estimation is performed on the luminance component, and the motion vector calculated based on the luminance component is used for both the luminance component and the chrominance component. Motion information including information about the reference picture and information about the motion vector used to predict the current block is encoded by the entropy encoder 155 and delivered to the video decoding device.

[0051] The inter-frame predictor 124 may also perform interpolation on a reference picture or a reference block in order to increase the accuracy of the prediction. In other words, the subsamples between two consecutive integer samples are interpolated by applying the filter coefficients to a plurality of consecutive integer samples including two integer samples. When the process of searching for the block most similar to the current block is performed for the interpolated reference picture, the decimal unit precision may be used instead of the integer sample unit precision for the motion vector representation. The precision or resolution of the motion vector may be set differently for each target region to be encoded (e.g., units such as slices, tiles, CTUs, CUs, etc.). When this adaptive motion vector resolution (AMVR) is applied, information about the motion vector resolution to be applied to each target region should be signaled for each target region. For example, when the target region is a CU, information about the motion vector resolution applied to each CU is signaled. The information about the motion vector resolution may be information representing the precision of the motion vector difference to be described below.

[0052] Meanwhile, the inter-frame predictor 124 may perform inter-frame prediction by using bidirectional prediction. In the case of bidirectional prediction, two reference pictures and two motion vectors representing the block position most similar to the current block in each reference picture are used. The inter-frame predictor 124 selects a first reference picture and a second reference picture from reference picture list 0 (RefPicList0) and reference picture list 1 (RefPicList1), respectively. The inter-frame predictor 124 also searches for a block most similar to the current block in the corresponding reference picture to generate a first reference block and a second reference block. In addition, a prediction block of the current block is generated by averaging or weighted averaging the first reference block and the second reference block. In addition, motion information including information about two reference pictures for predicting the current block and including information about two motion vectors is delivered to the entropy encoder 155. Here, the reference picture list 0 may be composed of pictures before the current picture in the display order among the pre-reconstructed pictures, and the reference picture list 1 may be composed of pictures after the current picture in the display order among the pre-reconstructed pictures. However, although not particularly limited thereto, a pre-reconstructed picture after the current picture in the display order may be additionally included in the reference picture list 0. Conversely, a pre-reconstructed picture before the current picture may also be additionally included in the reference picture list 1.

[0053] In order to minimize the number of bits consumed for encoding motion information, various methods may be used.

[0054] For example, when the reference picture and motion vector of the current block are the same as those of the neighboring block, information capable of identifying the neighboring block is encoded to deliver the motion information of the current block to the video decoding device. This method is called merge mode.

[0055] In the merge mode, the inter predictor 124 selects a predetermined number of merge candidate blocks (hereinafter, referred to as “merge candidates”) from neighboring blocks of the current block.

[0056] As a neighboring block for deriving merge candidates, such as Figure 4 As illustrated in the figure, all or some of the left block A0, the lower left block A1, the upper block B0, the upper right block B1, and the upper left block B2 adjacent to the current block in the current picture can be used. Further, blocks other than the current picture where the current block is located, which are located in a reference picture (which may be the same as or different from the reference picture used to predict the current block), can also be used as merge candidates. For example, a block located at the same location as the current block or a block adjacent to the block located at the same location in the reference picture can be used as a merge candidate in addition. If the number of merge candidates selected by the method described above is less than the preset number, a zero vector is added to the merge candidate.

[0057] The inter-frame predictor 124 configures a merge list including a predetermined number of merge candidates by using adjacent blocks. A merge candidate to be used as motion information of the current block is selected from the merge candidates included in the merge list, and merge index information for identifying the selected candidate is generated. The generated merge index information is encoded by the entropy encoder 155 and delivered to the video decoding device.

[0058] Merge skip mode is a special case of merge mode. After quantization, when all transform coefficients for entropy coding are close to zero, only the neighbor block selection information is transmitted without the residual signal. By using merge skip mode, relatively high coding efficiency can be achieved for pictures with slight motion, still pictures, screen content pictures, etc.

[0059] Hereinafter, the merge mode and the merge skip mode are collectively referred to as the merge / skip mode.

[0060] Another method for encoding motion information is the Advanced Motion Vector Prediction (AMVP) mode.

[0061] In the AMVP mode, the inter-frame predictor 124 derives a motion vector predictor candidate for a motion vector of a current block by using neighboring blocks of the current block. As neighboring blocks for deriving motion vector predictor candidates, the same as in Figure 4 All or some of the left block A0, lower left block A1, upper block B0, upper right block B1 and upper left block B2 adjacent to the current block in the current picture illustrated in the figure. In addition, blocks other than the current picture where the current block is located, which are located in a reference picture (which may be the same as or different from the reference picture used to predict the current block), may also be used as adjacent blocks for deriving motion vector predictor candidates. For example, blocks co-located with the current block or blocks adjacent to co-located blocks within the reference picture may be used. If the number of motion vector candidates selected by the above method is less than the preset number, a zero vector is added to the motion vector candidates.

[0062] The inter predictor 124 derives a motion vector predictor candidate by using the motion vectors of the neighboring blocks, and determines a motion vector predictor for the motion vector of the current block by using the motion vector predictor candidate. In addition, a motion vector difference is calculated by subtracting the motion vector predictor from the motion vector of the current block.

[0063] The motion vector predictor can be obtained by applying a predefined function (e.g., center value and average value calculation, etc.) to the motion vector predictor candidate. In this case, the video decoding device also knows the predefined function. In addition, since the neighboring blocks used to derive the motion vector predictor candidate are blocks for which encoding and decoding have been completed, the video decoding device may also already know the motion vectors of the neighboring blocks. Therefore, the video encoding device does not need to encode the information for identifying the motion vector predictor candidate. Thus, in this case, information about the motion vector difference and information about the reference picture used to predict the current block are encoded.

[0064] Meanwhile, the motion vector predictor may also be determined by selecting any one of the motion vector predictor candidates. In this case, information for identifying the selected motion vector predictor candidate is additionally encoded jointly with information about the motion vector difference and information about the reference picture used to predict the current block.

[0065] The subtractor 130 generates a residual block by subtracting a prediction block generated by the intra predictor 122 or the inter predictor 124 from the current block.

[0066] The transformer 140 converts the residual signal in the residual block having the pixel value in the spatial domain into a transform coefficient in the frequency domain. The transformer 140 may transform the residual signal in the residual block by using the total size of the residual block as a transform unit, or may further divide the residual block into a plurality of sub-blocks, and may perform the transform by using the sub-block as a transform unit. Optionally, the residual block is divided into two sub-blocks, namely a transform region and a non-transform region, so as to transform the residual signal using only the transform region sub-block as a transform unit. Here, the transform region sub-block may be one of two rectangular blocks having a size ratio of 1:1 based on the horizontal axis (or vertical axis). In this case, the flag (cu_sbt_flag) indicates that only the sub-block is transformed, and the directional (vertical / horizontal) information (cu_sbt_horizontal_flag) and / or the position information (cu_sbt_pos_flag) are encoded by the entropy encoder 155 and signaled to the video decoding device. In addition, the size of the transform region sub-block may have a size ratio of 1:3 based on the horizontal axis (or vertical axis). In this case, a flag (cu_sbt_quad_flag) for dividing the corresponding division is additionally encoded by the entropy encoder 155 and signaled to the video decoding apparatus.

[0067] Meanwhile, the transformer 140 may perform transformation on the residual block separately in the horizontal direction and the vertical direction. For the transformation, different types of transformation functions or transformation matrices may be used. For example, a pair of transformation functions for horizontal transformation and vertical transformation may be defined as a multiple transformation set (MTS). The transformer 140 may select a transformation function pair with the highest transformation efficiency in the MTS, and may transform the residual block in each of the horizontal and vertical directions. Information (mts_idx) of the transformation function pair in the MTS is encoded by the entropy encoder 155 and notified to the video decoding device with a signal.

[0068] The quantizer 145 quantizes the transform coefficients output from the transformer 140 using the quantization parameter and outputs the quantized transform coefficients to the entropy encoder 155. The quantizer 145 may also immediately quantize the relevant residual block without transforming any block or frame. The quantizer 145 may also apply different quantization coefficients (scaling values) according to the position of the transform coefficient in the transform block. The quantization matrix applied to the quantized transform coefficients arranged in 2 dimensions may be encoded and signaled to the video decoding device.

[0069] The rearrangement unit 150 may perform rearrangement of coefficient values ​​for the quantized residual value.

[0070] The rearrangement unit 150 may change the 2D coefficient array into a 1D coefficient sequence by using coefficient scanning. For example, the rearrangement unit 150 may output an ID coefficient sequence by scanning the DC coefficient into a high frequency domain coefficient using a zigzag scan or a diagonal scan. Depending on the size of the transform unit and the intra prediction mode, a vertical scan that scans the 2D coefficient array in the column direction and a horizontal scan that scans the 2D block type coefficients in the row direction may also be used instead of the zigzag scan. In other words, depending on the size of the transform unit and the intra prediction mode, the scanning method to be used may be determined among zigzag scanning, diagonal scanning, vertical scanning, and horizontal scanning.

[0071] The entropy encoder 155 generates a bitstream by encoding the sequence of ID-quantized transform coefficients output from the rearrangement unit 150 using various encoding schemes including context-based adaptive binary arithmetic coding (CABAC), exponential Golomb, and the like.

[0072] In addition, the entropy encoder 155 encodes information related to block partitioning (such as CTU size, CTU partition flag, QT partition flag, MTT partition type, MTT partition direction, etc.) to allow the video decoding device to equally partition the block to the video encoding device. In addition, the entropy encoder 155 encodes information about the prediction type indicating whether the current block is encoded by intra-frame prediction or inter-frame prediction. The entropy encoder 155 encodes intra-frame prediction information (i.e., information about the intra-frame prediction mode) or inter-frame prediction information (in the case of merge mode, merge index, and in the case of AMVP mode, information about reference picture index and motion vector difference) according to the prediction type. In addition, the entropy encoder 155 encodes information related to quantization (i.e., information about quantization parameters and information about quantization matrices).

[0073] The inverse quantizer 160 dequantizes the quantized transform coefficient output from the quantizer 145 to generate a transform coefficient. The inverse transformer 165 transforms the transform coefficient output from the inverse quantizer 160 from the frequency domain to the spatial domain to reconstruct a residual block.

[0074] The adder 170 reconstructs the current block by adding the reconstructed residual block and the prediction block generated by the predictor 120. When intra prediction is performed on a next sequential block, pixels in the reconstructed current block may be used as reference pixels.

[0075] The loop filter unit 180 performs filtering on the reconstructed pixels in order to reduce blocking artifacts, ringing artifacts, blurring artifacts, etc. occurring due to block-based prediction and transform / quantization. The loop filter unit 180 as a loop filter may include all or some of a deblocking filter 182, a sample adaptive offset (SAO) filter 184, and an adaptive loop filter (ALF) 186.

[0076] The deblocking filter 182 filters the boundaries between the reconstructed blocks to remove the block effect caused by block unit encoding / decoding, and the SAO filter 184 and the ALF 186 perform additional filtering for the deblocked filtered video. The SAO filter 184 and the ALF 186 are filters for compensating the difference between the reconstructed pixels and the original pixels, which is caused by lossy coding. The SAO filter 184 applies an offset as a CTU unit to enhance subjective picture quality and coding efficiency. On the other hand, the ALF 186 performs block unit filtering and applies different filters to compensate for distortion by dividing the boundaries of the corresponding blocks and the degree of variation. Information about the filter coefficients to be used for the ALF can be encoded and notified to the video decoding device with a signal.

[0077] The reconstructed blocks filtered by the deblocking filter 182, the SAO filter 184, and the ALF 186 are stored in the memory 190. When all blocks in one picture are reconstructed, the reconstructed picture can be used as a reference picture for inter-frame prediction of blocks in a picture to be encoded later.

[0078] The video encoding device may store a bit stream of encoded video data in a non-transitory storage medium, or transmit the bit stream to a video decoding device through a communication network.

[0079] Figure 5 is a functional block diagram of a video decoding device that can implement the techniques of the present invention. Figure 5 , describing a video decoding apparatus and components of the apparatus.

[0080] The video decoding device may include an entropy decoder 510 , a rearrangement unit 515 , an inverse quantizer 520 , an inverse transformer 530 , a predictor 540 , an adder 550 , a loop filter unit 560 , and a memory 570 .

[0081] and Figure 1 Similar to the video encoding device of the present invention, each component of the video decoding device can be implemented as hardware or software or as a combination of hardware and software. In addition, the function of each component can be implemented as software, and a microprocessor can also be implemented to execute the function of the software corresponding to each component.

[0082] The entropy decoder 510 extracts information related to block partitioning by decoding a bitstream generated by a video encoding apparatus to determine a current block to be decoded, and extracts prediction information required to reconstruct the current block and information about a residual signal.

[0083] The entropy decoder 510 determines the size of the CTU by extracting information about the CTU size from a sequence parameter set (SPS) or a picture parameter set (PPS), and divides the picture into CTUs of the determined size. In addition, the CTU is determined to be the highest level of the tree structure, that is, the root node, and the partition information of the CTU can be extracted to partition the CTU using the tree structure.

[0084] For example, when a CTU is divided using a QTBTTT structure, a first flag (QT_split_flag) related to the division of the QT is first extracted to divide each node into four nodes of the lower layer. In addition, for a node corresponding to a leaf node of the QT, a second flag (mtt_split_flag) related to the division of the MTT, a division direction (vertical / horizontal) and / or a division type (binary / ternary) are extracted to divide the corresponding leaf node into the MTT structure. As a result, each node below the leaf node of the QT is recursively split into a BT or TT structure.

[0085] As another embodiment, when a CTU is divided by using a QTBTTT structure, a CU split flag (split_cu_flag) indicating whether the CU is divided is extracted. When the corresponding block is divided, a first flag (QT_split_flag) may also be extracted. During the splitting process, for each node, 0 or more recursive MTT splits may occur after 0 or more recursive QT splits. For example, for a CTU, MTT splitting may occur immediately, or conversely, only multiple QT splits may occur.

[0086] For another example, when the CTU is split using the QTBT structure, the first flag (QT_split_flag) related to the splitting of the QT is extracted to split each node into four nodes of the lower layer. In addition, the split flag (split_flag) indicating whether the node corresponding to the leaf node of the QT is further split into BTs and the split direction information are extracted.

[0087] Meanwhile, when the entropy decoder 510 determines the current block to be decoded by using the partitioning of the tree structure, the entropy decoder 510 extracts information about the prediction type indicating whether the current block is intra-predicted or inter-predicted. When the prediction type information indicates intra-prediction, the entropy decoder 510 extracts syntax elements of intra-prediction information (intra-prediction mode) for the current block. When the prediction type information indicates inter-prediction, the entropy decoder 510 extracts information representing syntax elements for inter-prediction information (i.e., motion vectors and reference pictures referenced by motion vectors).

[0088] Furthermore, the entropy decoder 510 extracts quantization-related information and extracts information on a quantized transform coefficient of the current block as information on a residual signal.

[0089] The rearrangement unit 515 may change the sequence of 1D quantized transform coefficients entropy-decoded by the entropy decoder 510 into a 2D coefficient array (ie, block) again in the reverse order of the coefficient scanning order performed by the video encoding device.

[0090] The inverse quantizer 520 dequantizes the quantized transform coefficients and dequantizes the quantized transform coefficients by using quantization parameters. The inverse quantizer 520 may also apply different quantization coefficients (scaling values) to the quantized transform coefficients arranged in 2D. The inverse quantizer 520 may perform inverse quantization by applying a matrix (scaling value) of quantization coefficients from a video encoding device to a 2D array of quantized transform coefficients.

[0091] The inverse transformer 530 generates a residual block for the current block by reconstructing a residual signal by inversely transforming the dequantized transform coefficients from the frequency domain into the spatial domain.

[0092] In addition, when the inverse transformer 530 inversely transforms a partial region (sub-block) of the transform block, the inverse transformer 530 extracts a flag (cu_sbt_flag) that only the sub-block of the transform block is transformed, direction (vertical / horizontal) information (cu_sbt_horizontal_flag) of the sub-block, and / or position information (cu_sbt_pos_flag) of the sub-block. The inverse transformer 530 also inversely transforms the transform coefficient of the corresponding sub-block from the frequency domain to the spatial domain to reconstruct the residual signal and fills the region that is not inversely transformed with a value of "0" as the residual signal to generate a final residual block for the current block.

[0093] In addition, when MTS is applied, the inverse transformer 530 determines a transform index or a transform matrix applied in each of the horizontal and vertical directions by using MTS information (mts_idx) signaled from the video encoding device. The inverse transformer 530 also performs inverse transform on the transform coefficients in the transform block in the horizontal and vertical directions by using the determined transform function.

[0094] The predictor 540 may include an intra predictor 542 and an inter predictor 544. The intra predictor 542 is activated when the prediction type of the current block is intra prediction, and the inter predictor 544 is activated when the prediction type of the current block is inter prediction.

[0095] The intra predictor 542 determines an intra prediction mode of a current block among a plurality of intra prediction modes according to a syntax element of the intra prediction mode extracted from the entropy decoder 510. The intra predictor 542 also predicts the current block by using neighboring reference pixels of the current block according to the intra prediction mode.

[0096] The inter predictor 544 determines a motion vector of a current block and a reference picture to which the motion vector refers by using a syntax element for the inter prediction mode extracted from the entropy decoder 510 .

[0097] The adder 550 reconstructs the current block by adding the residual block output from the inverse transformer 530 and the prediction block output from the inter predictor 544 or the intra predictor 542. When intra prediction is performed on a block to be decoded later, pixels within the reconstructed current block are used as reference pixels.

[0098] The loop filter unit 560 as a loop filter may include a deblocking filter 562, an SAO filter 564, and an ALF 566. The deblocking filter 562 performs deblocking filtering on the boundary between the reconstructed blocks in order to remove the block effect that occurs due to block unit decoding. The SAO filter 564 and the ALF 566 perform additional filtering on the reconstructed blocks after deblocking filtering in order to compensate for the difference between the reconstructed pixels and the original pixels that occurs due to lossy encoding. The filter coefficients of the ALF are determined by using information about the filter coefficients decoded from the bitstream.

[0099] The reconstructed blocks filtered by the deblocking filter 562, the SAO filter 564, and the ALF 566 are stored in the memory 570. When all blocks in one picture are reconstructed, the reconstructed picture can be used as a reference picture for inter-frame prediction of blocks in a picture to be encoded later.

[0100] In some embodiments, the present invention relates to encoding and decoding a video image as described above. More specifically, the present disclosure provides a video encoding method and apparatus for setting an adjacent region or a non-adjacent region as a template according to information of a neighboring reconstruction region of a current block. The video encoding method and apparatus reconstruct the current block by performing a prediction based on template matching (which is based on a set template).

[0101] The following embodiments may be performed by the predictor 120 in the video encoding apparatus. The following embodiments may also be performed by the predictor 540 in the video decoding apparatus.

[0102] The video encoding device in the encoding of the current block may generate signaling information associated with the present embodiment in terms of optimizing rate distortion. The video encoding device may encode the signaling information using the entropy encoder 155 and transmit the encoded signaling information to the video decoding device. The video decoding device may decode the signaling information associated with the decoding of the current block from the bitstream using the entropy decoder 510.

[0103] In the following description, the term 'target block' may be used interchangeably with a current block or a coding unit (CU), or may refer to some regions of a coding unit.

[0104] In addition, a flag value of true indicates when the flag is set to 1. In addition, a flag value of false indicates when the flag is set to 0.

[0105] The following embodiments are described with respect to a video decoding device, but may be implemented in a video encoding device in the same or similar manner.

[0106] Figure 6 is a block diagram detailing a portion of a video decoding device according to at least one embodiment of the present disclosure.

[0107] According to some embodiments, the video decoding device can determine a prediction unit and a transform unit, and for a current block corresponding to the determined unit, perform prediction and inverse transform by using the determined prediction technology and prediction mode to finally generate a reconstructed block of the current block. Figure 6 The operations shown in FIG. 5 can be performed by the inverse transformer 530, the predictor 540, and the adder 550 of the video decoding device. Figure 6 The same operations shown in the figure can be performed by the inverse transformer 165, the picture divider 110, the predictor 120, and the adder 170 of the video encoding device. In this case, the video decoding device uses the encoding information parsed from the bit stream, but the video encoding device can use the encoding information set from a higher level in terms of minimizing the bit rate distortion. In the following, for convenience, the embodiment is described with the video decoding device as the center.

[0108] like Figure 5 As shown, the predictor 540 includes an intra-frame predictor 542 and an inter-frame predictor 544, depending on the prediction technique, but as shown in FIG. Figure 6 As shown, the predictor 540 may include all or part of a prediction unit determiner 602 , a prediction technique determiner 604 , a prediction mode determiner 606 , and a prediction executor 608 .

[0109] When the color format of the input video is a YUV format (YUV420, YUV411, YUV422, YUV444, etc.), the video decoding apparatus may perform prediction and reconstruction of the luminance component, and then may perform prediction and reconstruction of the chrominance component. In other words, the luminance component and the chrominance component may be obtained by Figure 6 On the other hand, when the color format of the input video is RGB, the video encoding device may perform a color format conversion from RGB to YUV, and then may encode the converted video. Here, in the case of the YUV format, the color format represents the correspondence between the pixels in the luminance component and the pixels in the chrominance component.

[0110] The prediction unit determiner 602 determines a prediction unit (PU). The prediction technique determiner 604 determines a prediction technique for the prediction unit, such as intra prediction, inter prediction or intra block copy (IBC) mode, palette mode, template-based prediction, etc. The prediction mode determiner 606 determines a detailed prediction mode for the prediction technique. The prediction executor 608 generates a prediction block of the current block according to the determined prediction mode.

[0111] The inverse transformer 530 includes a transform unit determiner 610 and an inverse transform performer 612. The transform unit determiner 610 determines a transform unit (TU) in response to an inverse quantization signal of a current block, and the inverse transform performer 612 inversely transforms the transform unit represented by the inverse quantization signal to generate a residual signal.

[0112] The adder 550 sums the predicted block and the residual signal to generate a reconstructed block. The reconstructed block is stored in a memory and can be used to predict other blocks in the future.

[0113] The prediction unit determined by the prediction unit determiner 602 may become a sub-block of the current block or a sub-block partitioned from the current block. In this case, the prediction unit of the chrominance component may correspond in size to the prediction unit of the luma component depending on the color format. Alternatively, the prediction units of the luma component and the chrominance component may be determined separately, and prediction may be performed for the prediction unit of the chrominance component.

[0114] The prediction technique determiner 604 determines the prediction technique for the prediction unit. As described above, the prediction technique may be one of inter prediction, intra prediction, IBC mode, palette mode, and template-based prediction. In this case, the prediction technique of the chroma component may be determined to be the same as the prediction technique of the corresponding luma component without signaling and parsing separate information.

[0115] When the prediction technique of the current block is intra prediction, the prediction mode determiner 606 may determine template based prediction as the prediction mode by signaling and parsing a 1-bit flag.

[0116] As another example, the prediction mode determiner 606 may use signaling and parsing of a 1-bit flag to determine the prediction technique and prediction mode for the current block as template-based prediction.

[0117] As another example, when the prediction technique of the current block is inter-frame prediction, the prediction mode determiner 606 can utilize template matching in response to the prediction signal obtained based on the initial motion information. The video decoding device can utilize template matching to correct the motion information of the initial prediction signal, and then can use the corrected motion information to generate a final prediction signal.

[0118] The following describes the operation of the prediction performer 608 for prediction based on template matching.

[0119] Figure 7 is a diagram illustrating a template region of a current block according to at least one embodiment of the present disclosure.

[0120] As an example, a case is described in which the prediction technique of the current block having a size of W×H (W is the block width and H is the block height) is intra prediction, and the prediction mode is replaced by template-based prediction determined by signaling and parsing a 1-bit flag (hereinafter referred to as a "template matching flag"). Figure 7 As shown, the video decoding device sets the neighboring reconstruction area of ​​the current block as a template, and uses the set template to perform template matching on the reconstruction area in the current frame to search for a template with the best loss value. The video decoding device can generate a prediction signal for the current block from a reference block corresponding to the detected best template. Template matching is a process of calculating the similarity between the template of the current block and the candidate template in the reconstruction area to detect a template with the best similarity between its pixels (i.e., with the best loss value). The similarity can be calculated based on a loss function. The video decoding device can use one of the methods for calculating the similarity between pixels (e.g., mean square error (MSE), sum of absolute differences (SAD), sum of absolute transform differences (SATD), etc.) as a loss function. At the same time, the relationship between the current block and the template corresponds to the relationship between the reference block and the detected best template.

[0121] As another example, in template matching based on a loss function, the video decoding apparatus may search for N (natural number) templates in order of increasing loss values, and may perform weighted summation on reference blocks corresponding to the detected N templates to generate a final prediction signal for the current block.

[0122] exist Figure 7 In the example, W and H represent the width and height of the current block respectively.

[0123] exist Figure 7 In the example of , the template area is L-shaped. Together with W and H, "a" and "b" define the size of the template area. "a" represents the bottom width of the template area on the left side of the current block, and "b" represents the right height of the template area on the upper side of the current block. In this case, "a" and "b" can be implicitly determined based on the size of the current block. Alternatively, "a" and "b" can be fixed values ​​regardless of the size of the current block.

[0124] exist Figure 7 In the example, SR W and SR H Defines the size of the search area. SR W Indicates the bottom width of the search area on the left side of the current block, SR H Indicates that the search area is located at the right height of the upper side of the current block. Figure 7 As shown, the search area is limited to the reconstruction area of ​​the current block and can be all or part of the reconstruction area. The SR can be implicitly determined based on the size of the current block. W and SRH Alternatively, SR W and SR H can be determined implicitly based on the resolution of the input video. Alternatively, SR W and SR H Can be a fixed value regardless of the current block size.

[0125] Figures 8a to 8c is a diagram illustrating the position and shape of a template area of ​​a current block according to some embodiments of the present disclosure.

[0126] In the following, "the position and shape of the template area" and "the shape of the template area" are used interchangeably, because the position of the template area may also determine the shape of the template.

[0127] As another example, Figures 8a to 8c As shown, the template area for prediction based on template matching can be determined as an adjacent or non-adjacent reconstruction area at the upper left of the current block, at the left of the current block, or at the top of the current block. Figure 8a In the example of , when the template area exists in the upper left of the current block, the template area is L-shaped. Figure 8a In the example of , the template area can be defined based on the width and height (W, H) of the current block, the bottom width "a" of the template area existing on the left side of the current block, the right height "b" of the template area existing on the top of the current block, and the distance "r" between the current block and the template area.

[0128] In addition, as in Figure 8b In the example of , when the template area exists on the left side of the current block, the template area has a rectangular shape. Figure 8b In the example of , the template area has a width "a" and a height H+p (where "p" is an integer). "p" represents the difference between the height of the template area and the height of the current block. Figure 8c As shown in FIG. 1 , when the template area exists in the upper part of the current block, the template area has a rectangular shape. Figure 8c In the example of , the template region has a width W+q (where “q” is an integer) and a height “b”. “q” represents the difference between the width of the template region and the width of the current block.

[0129] Here, W, H, a, b, p, q, and r are values ​​in pixels.

[0130] As mentioned above, in Figures 8a to 8cIn the example of , "r" represents the distance between the current block and the template area. "p" and "q" may be implicitly determined based on the size of the current block or the aspect ratio of the current block. Alternatively, "p" and "q" may be fixed values. In addition, "r" may be signaled and parsed in the form of an index (hereinafter, "distance index, adjacent_idx").

[0131] In one example, the video decoding device may parse an index indicating the shape of the template region. Based on the parsed index, the video decoding device may determine the shape of the template region to be present above the left of the current block, to the left of the current block, or above or around the current block, such as Figures 8a to 8c Finally, the video decoding apparatus may set a distance value "r" based on the parsed distance index to determine a template region of the current block.

[0132] As another example, the video decoding device may implicitly determine the template area based on the aspect ratio of the current block having a size of W×H. Figure 8a As shown in , if W=H, the upper left area of ​​the current block can be determined as the template area. Figure 8b As shown in FIG. 1 , if W < H, the left area of ​​the current block can be determined as the template area. If W > H, the upper area of ​​the current block can be determined as the template area, as shown in FIG. Figure 8c Finally, the video decoding device may set the distance value "r" according to the parsed distance index to determine the template area of ​​the current block.

[0133] On the other hand, for a chroma block, the video decoding apparatus may implicitly determine the position and shape of the template region according to a prediction mode of a luma block co-located with the chroma block (hereinafter, “corresponding luma block”).

[0134] In one example, if the prediction mode corresponding to the central area of ​​the luminance block is an intra prediction mode (which is a directional prediction mode between 45 degrees and 135 degrees), such as Figure 9a As shown in , the video decoding device can determine the reconstructed upper area of ​​the current chrominance block as the template area, such as Figure 8c As shown in .

[0135] If the prediction mode corresponding to the central area of ​​the luminance block is an intra prediction mode (which is a directional prediction mode between 135 degrees and 225 degrees), as in Figure 9b As shown in , the video decoding device may determine the reconstructed left area of ​​the current chrominance block as the template area, as shown in Figure 8b As shown in .

[0136] Alternatively, if the prediction mode of the central area of ​​the corresponding luminance block is an intra prediction mode as a planar mode or a DC mode, the video decoding apparatus may determine the reconstructed upper left area of ​​the current chrominance block as the template area, such as Figure 8a shown.

[0137] Finally, the video decoding device may set the distance value 'r' based on the parsed distance index or by directly parsing the distance value 'r' to determine the template region of the current chroma block.

[0138] As another example, with respect to a chroma block, the video decoding apparatus may implicitly determine the distance between the current chroma block and the template region based on a distance from a non-adjacent reference line at an upper left boundary region of a corresponding luma block.

[0139] For example, if the color format of the input video is YUV420, and the upper left boundary area of ​​the corresponding luminance block is predicted from non-adjacent reference lines at distances k and m, respectively, the video decoding device may determine the distance between the current chrominance block and the left template area as k>>1, and implicitly determine the distance between the current chrominance block and the upper template area as m>>1. Thus, the video decoding device may adjust the distance between the current chrominance block and the template area according to the color format of the input video.

[0140] As another example, a case is described in which, for a chrominance block, a single tree structure is used, and its corresponding luminance block is predicted according to a prediction mode based on template matching. Here, in the single tree structure, the luminance component and the chrominance component have the same block partition structure. If the sampling between the luminance component and the chrominance component is different according to the color format of the input video, the video decoding device can determine the template area of ​​the chrominance component by sampling the template area corresponding to the template area of ​​the luminance component.

[0141] At the same time, according to Fig.10 and Fig.11 According to the flowchart shown in , the video decoding device can determine the template area of ​​the current block and perform prediction based on the template matching based on the determined template area.

[0142] Fig.10 is a flowchart of a prediction method of a current block performed by a video decoding apparatus according to at least one embodiment of the present invention.

[0143] The video decoding apparatus decodes a template matching flag from a bitstream (S1000). Here, the template matching flag indicates whether a prediction mode based on template matching is used when a prediction technique of a current block is intra prediction or template-based prediction.

[0144] The video decoding device checks a template matching flag ( S1002 ).

[0145] If the template matching flag is true (“Yes” in S1002 ), the video decoding apparatus may perform steps related to template matching ( S1004 to S1008 ).

[0146] The video decoding apparatus decodes a distance index from a bitstream ( S1004 ).

[0147] Based on the distance index (adjacent_idx), the video decoding apparatus may determine a distance 'r' between the current block and the template region. The video decoding apparatus may also determine the size of the template region by using fixed or implicitly determined 'a' and 'b'.

[0148] Meanwhile, the distance index may be defined by a lookup table agreed upon between the video encoding device and the video decoding device. Alternatively, if the intra prediction mode based on non-adjacent regions is applicable to the video encoding device, the prediction based on template matching may share the distance index lookup table for intra prediction based on non-adjacent regions with intra prediction based on non-adjacent regions. Here, intra prediction based on non-adjacent regions utilizes non-adjacent reference lines based on, for example, multiple reference lines (MRLs).

[0149] The video decoding device determines an initial template area (S1006). Figures 8a to 8c As shown, the video decoding device may determine the initial template area based on the distance "r" and the size of the template area.

[0150] By using the initial template area, the video decoding apparatus may search for an optimal template based on template matching (S1008). The video decoding apparatus searches for an area having an optimal loss value in the search area of ​​the current block.

[0151] The video decoding apparatus generates a prediction block of a current block using a reference block corresponding to the detected optimal template ( S1010 ).

[0152] On the other hand, if the template matching flag is false (No in S1002), the video decoding device may skip steps related to template matching. For example, when the prediction technique is intra-frame prediction, the video decoding device may generate a prediction block for the current block based on the parsed intra-frame prediction mode. Alternatively, when the prediction technique is not prediction based on template matching, the video decoding device may generate a prediction block for the current block by using a different prediction technique and its accompanying prediction mode.

[0153] Fig.11 is a flowchart of a prediction method of a current block performed by a video decoding apparatus according to another embodiment of the present invention.

[0154] Fig.11 Examples of this are usually used as Fig.10The step of decoding the template matching flag ( S1100 ) by determining the initial template area ( S1106 ) is shown, so the subsequent steps thereof are described.

[0155] The video decoding device calculates a gradient ( S1108 ).

[0156] The video decoding apparatus determines a final template region by using the calculated gradient ( S1110 ).

[0157] The video decoding apparatus searches for an optimal template using the final template area based on template matching (S1112). The video decoding apparatus searches for an area having an optimal loss value in the search area of ​​the current block.

[0158] The video decoding apparatus generates a prediction block of the current block using a reference block corresponding to the detected optimal template ( S1114 ).

[0159] The steps of using gradient to determine the final template area are described in detail below.

[0160] As an example, the video decoding device may divide the initial template area into sub-blocks having a×b size, such as Fig.12 As described in , and then the directionality and magnitude of the gradient of each sub-block can be calculated. For example, each sub-block may have a size implicitly determined by the size of the current block, or may have a preset size. The video decoding device can estimate the directionality and magnitude of the gradient on a sub-block by sub-block basis by applying a differential filter to each sub-block. The differential filter can be one of filters such as a Perwitt filter, a Roberts filter, or a Sobel filter applied in the vertical direction and the horizontal direction.

[0161] The video decoding apparatus may apply a differential filter to the sub-block in the initial template area to calculate the gradients dx and dy in the horizontal and vertical directions. The video decoding apparatus may calculate Grad by using Equation 1 or Equation 2. mag (the magnitude of the gradient of each sub-block).

[0162] [Equation 1]

[0163]

[0164] [Equation 2]

[0165]

[0166] The video decoding device can also calculate Grad ang , which is the directionality of the gradient of each sub-block, as shown in Equation 3.

[0167] [Equation 3]

[0168]

[0169] Fig.13 is a diagram illustrating a final template region according to at least one embodiment of the present disclosure.

[0170] After calculating the magnitude and directionality of the gradient of each sub-block, the video decoding device may generate a histogram of the magnitude or directionality of the gradient. Then, based on the directional histogram of the gradient, the video decoding device may determine the region of the sub-block with the most frequent directionality as the final template region, such as Fig.13 As shown. In addition, the video decoding device may consider the magnitude histogram of the gradient. For example, after first selecting a sub-block having a gradient magnitude of a predetermined threshold or greater, the video decoding device may reselect a sub-block having the most frequent directionality from the selected sub-blocks.

[0171] Alternatively, based on the directional histogram of the gradient generated according to the above process, the video decoding device may determine the region of the sub-block having a directionality corresponding to a frequency of a predetermined threshold or higher as the final template region, such as Fig.13 In addition, the video decoding device may consider a gradient magnitude histogram. For example, after first selecting a sub-block having a gradient magnitude of a predetermined threshold or higher, the video decoding device may reselect a sub-block having a directionality corresponding to a frequency of a predetermined threshold or higher from the selected sub-blocks.

[0172] Fig.14a and Fig.14b is a diagram illustrating a final template region according to other embodiments of the present disclosure.

[0173] As another example, after determining the initial template region, such as Fig.12 As shown, the video decoding device may divide the initial template area into sub-blocks of size n×m, where the integer "n" is less than or equal to "a", and the integer "m" is less than or equal to "b", such as Fig.14a As shown. Thus, each sub-block may have a size implicitly determined based on the size of the current block, or may have a preset size. The video decoding device may calculate the directionality and magnitude of the gradient for each sub-block by using equations 1 to 3, and generate a directional histogram of the gradient. Then, based on the directional histogram of the gradient, the video decoding device may determine the region of the sub-block with the most frequent directionality as the final template region, such as Fig.14b shown.

[0174] Fig.15 is a flowchart of a prediction method for a current block performed by a video encoding apparatus according to at least one embodiment of the present invention.

[0175] The video encoding apparatus determines a distance index ( S1500 ).

[0176] Here, the distance index indicates the distance between the current block and the template region. The video encoding device may determine the distance index according to rate-distortion optimization. Alternatively, the video encoding device may obtain the distance index from a higher level.

[0177] The video encoding apparatus determines a template region based on a distance between a current block and the template region and a shape of the template region (S1502).

[0178] By using the template area, the video encoding apparatus searches for an optimal template in a search area based on template matching (S1504). Here, the search area may be limited within a reconstruction area of ​​the current block.

[0179] The video encoding apparatus generates a first prediction block of the current block from a reference block corresponding to the optimal template (S1506).

[0180] The video encoding apparatus generates a second prediction block of the current block by using a different prediction technique (S1508).

[0181] For example, when the prediction technique is intra-frame prediction, the video encoding device may generate the second prediction block based on the intra-frame prediction mode. Alternatively, when the prediction technique is not prediction based on template matching, the video encoding device may generate the second prediction block by using a different prediction technique and its accompanying prediction mode.

[0182] The video encoding device determines a template matching flag based on the first prediction block and the second prediction block (S1510). The video encoding device may determine the template matching flag in terms of rate-distortion optimization. For example, when the first prediction block is optimal, the video encoding device sets the template matching flag to true. Alternatively, when the second prediction block is optimal, the video encoding device may set the template matching flag to false.

[0183] The video encoding apparatus encodes the distance index and the template matching flag ( S1512 ).

[0184] Although the steps in each flowchart are described as being performed sequentially, these steps are merely illustrative of the technical concepts of some embodiments of the present disclosure. Therefore, a person of ordinary skill in the art to which the present disclosure belongs can perform the steps by changing the order described in each figure or by performing two or more steps in parallel. Therefore, the steps in each flowchart are not limited to the time series order shown.

[0185] It should be understood that the above description presents illustrative embodiments that can be implemented in different other ways. The functions described in some embodiments can be implemented by hardware, software, firmware and / or a combination thereof. It should also be understood that the functional components described in this disclosure are marked with "... units" to strongly emphasize the possibility of their independent implementation.

[0186] Meanwhile, various methods or functions described in some embodiments may be implemented as instructions stored in a non-transitory recording medium that can be read and executed by one or more processors. For example, the non-transitory recording medium may include various types of recording devices in which data is stored in a form readable by a computer system. For example, the non-transitory recording medium may include a storage medium such as an erasable programmable read-only memory (EPROM), a flash drive, an optical drive, a magnetic hard drive, and a solid-state drive (SSD).

[0187] Although the embodiments of the present disclosure have been described for illustrative purposes, however, those of ordinary skill in the art to which the present disclosure belongs should recognize that various modifications, additions, and substitutions are possible without departing from the concept and scope of the present disclosure. Therefore, for the sake of brevity and clarity, the embodiments of the present disclosure have been described. The scope of the technical concept of the embodiments of the present disclosure is not limited by the diagrams. Therefore, those of ordinary skill in the art to which the present disclosure belongs should understand that the scope of the present disclosure should not be limited by the embodiments explicitly described above, but by the claims and their equivalents.

[0188] (reference number)

[0189] 120 Predictor

[0190] 540 Predictor

[0191] 602 prediction unit determiner

[0192] 604 Prediction Technology Determiner

[0193] 606 Prediction Mode Determiner

[0194] 608 Predictive Executor

[0195] CROSS-REFERENCE TO RELATED APPLICATIONS

[0196] This application claims priority to and the benefit of Korean Patent Application No. 10-2022-0099346, filed on August 9, 2022, and Korean Patent Application No. 10-2023-0086439, filed on July 4, 2023, the entire contents of each of which are incorporated herein by reference.

Claims

1. A method for reconstructing a current block by a video decoding device, the method comprising: Decoding from a bitstream an index indicating a distance between the current block and the template region (adjacent_idx); determining the template region based on a distance between the current block and the template region and a shape of the template region; searching for an optimal template in a search area by using the template area and based on template matching, wherein the search area is limited within a reconstruction area of ​​the current block; as well as A prediction block of the current block is generated from a reference block corresponding to the best template.

2. The method according to claim 1, wherein: The shape of the template area includes: an L shape, the L shape existing at the upper left of the current block, and the L shape is defined based on the width of the current block, the height of the current block, the bottom width of the template area existing at the left side of the current block, the right height of the template area existing at the top of the current block, and the distance between the current block and the template area, Wherein, based on the size of the current block, the bottom width of the template area existing on the left side of the current block and the right height of the template area existing on the top of the current block are implicitly determined or fixed.

3. The method according to claim 1, wherein: The shape of the template area includes: a rectangular shape, the rectangular shape existing on the left side of the current block and being defined based on a width and a height of a template area existing on the left side of the current block, The difference between the height of the template region and the height of the current block is implicitly determined based on the size of the current block or is a fixed value, and the width of the template region is implicitly determined based on the size of the current block or is a fixed value.

4. The method according to claim 1, wherein: The shape of the template area includes: a rectangular shape, the rectangular shape existing at an upper portion of the current block and being defined based on a width and a height of a template area existing at an upper portion of the current block, The difference between the width of the template region and the width of the current block is implicitly determined based on the size of the current block or is a fixed value, and the height of the template region is implicitly determined based on the size of the current block or is a fixed value.

5. The method according to claim 1, wherein: Determining the template area includes: decoding an index indicating a shape of the template region from the bitstream; and A shape of the template region is determined based on the index.

6. The method according to claim 1, wherein: Determining the template area includes: The shape of the template region is implicitly determined based on the aspect ratio of the current block.

7. The method according to claim 1, wherein: Determining the template area includes: For the chroma block of the current block, the position and shape of the template area of ​​the chroma block are implicitly determined according to the prediction mode of the luminance block coexisting with the chroma block.

8. The method according to claim 1, wherein: Searching for the best templates includes: The template is searched in the order of increasing loss values ​​of the natural number N templates caused by the template matching.

9. The method according to claim 8, wherein: Generating the prediction block comprises: A weighted sum is performed on the reference blocks corresponding to the N templates to generate a prediction block of the current block.

10. The method according to claim 1, further comprising: Calculating the gradient of the template area; as well as The template region is reset by using the gradient.

11. The method according to claim 10, wherein: Calculating the gradient includes: Dividing the template area into sub-blocks; For each of the sub-blocks, calculating a magnitude and a directionality of the gradient; and A directional histogram of the gradients is generated.

12. The method according to claim 11, wherein: Each of the sub-blocks has a size implicitly determined by the size of the current block, or has a preset size.

13. The method according to claim 11, wherein: Resetting the template area includes: A region of the sub-block having the most frequent directionality is determined as the template region based on the directionality histogram.

14. The method according to claim 11, wherein: Resetting the template area includes: generating a magnitude histogram of the gradient; selecting a sub-block having a gradient magnitude equal to or greater than a preset threshold based on the magnitude histogram; and Based on the directional histogram, a region of the selected sub-block having the most frequent directionality is determined as the template region.

15. A method for encoding a current block by a video encoding device, the method comprising: Determine an index indicating a distance between the current block and a template region (adjacent_idx); determining the template region based on a distance between the current block and the template region and a shape of the template region; searching for an optimal template in a search area by using the template area and based on template matching, wherein the search area is limited within a reconstruction area of ​​the current block; as well as A prediction block of the current block is generated from a reference block corresponding to the best template.

16. The method according to claim 15, further comprising: The distance index is encoded.

17. A computer-readable recording medium storing a bit stream generated by a video encoding method, the video encoding method comprising: Determine an index indicating the distance between the current block and the template region (adjacent_idx); determining the template region based on a distance between the current block and the template region and a shape of the template region; searching for an optimal template in a search area by using the template area and based on template matching, wherein the search area is limited within a reconstruction area of ​​the current block; as well as A prediction block of the current block is generated from a reference block corresponding to the best template.

Images