Reference block setting method using template matching, and image encoding and decoding apparatus for performing same

Abstract

The present invention relates to a reference block setting method using template matching, and an image encoding and decoding apparatus for performing same, and the method comprises the steps of: configuring a first template including samples located above a target block; configuring a second template including samples located on the left side of the target block; determining one or more templates to be used for template matching from among the first and second templates; and acquiring a reference block for the target block by performing template matching by using the determined one or more templates.

Description

Method for setting reference blocks using template matching, and image encoding and decoding devices for performing the same

[0001] The present invention relates to a method for performing intra prediction or inter prediction on an image by setting a reference block using template matching.

[0002] Compression encoding of video signals is achieved by removing redundant information by considering spatial correlation, temporal correlation, and probabilistic correlation.

[0003] Among the prediction methods used in image encoding, intra prediction is a method of predicting the current block by referring to samples that have already been reconstructed around the block to be encoded.

[0004] Recently, due to the development of various media and data transmission media, user demand for high-resolution, high-quality images is increasing.

[0005] Since the encoded data of a high-resolution image contains more information than the encoded data of a low-resolution or medium-resolution image, the cost of transmitting or storing it increases.

[0006] To solve this problem, research is ongoing on encoding and decoding methods to effectively reduce the amount of encoding data for high-resolution images.

[0007] The purpose of the present invention is to provide a method for setting a reference block using template matching capable of improving image compression performance, and an image encoding and decoding device for performing the method.

[0008] A method for setting a reference block according to an embodiment of the present invention is a method for setting a reference block for generating a prediction block using template matching, the method comprising: configuring a first template composed of samples located above a target block; configuring a second template composed of samples located to the left of the target block; determining one or more templates to be used for template matching among the first and second templates; and performing template matching using the determined one or more templates to obtain a reference block for the target block.

[0009] The video encoding device according to one embodiment of the present invention performs a reference block setting method using template matching as described above, and inserts an index value for indicating one or more templates to be used for template matching among the first and second templates into at least one of a Sequence Parameter Set (SPS), a Picture Parameter Set (PPS), a Picture Header (PH), and a Slice Header (SH), and transmits the same as a bitstream.

[0010] The image decoding device according to one embodiment of the present invention performs a reference block setting method using template matching as described above, and determines a template to be used for the template matching based on at least one of the shape and position of the target block.

[0011] Meanwhile, at least some steps of the above methods may be implemented as a computer-readable recording medium recording a program for execution on a computer, or may be provided as a program itself.

[0012] According to one embodiment of the present invention, the encoder does not transmit specific compression information for a target block, but allows a decoder to derive the corresponding compression information for the target block using a template matching method, thereby reducing the number of transmitted bits and improving image compression performance accordingly.

[0013] According to another embodiment of the present invention, when at least a part of a region of a template used for template matching goes beyond a picture boundary, the location of a block can be limited, another region within the picture can be used to fill it, or the region going beyond the picture boundary can be excluded from the template, so that template matching can be performed even for a target block or reference block located adjacent to the picture boundary.

[0014] Figure 1 is a block diagram illustrating the configuration and operation of a video encoder for encoding an image.

[0015] Figure 2 is a drawing for explaining an embodiment of a method for dividing blocks of an image.

[0016] Figures 3 and 4 are drawings for explaining embodiments of an intra prediction method.

[0017] Figure 5 is a diagram for explaining embodiments of an inter prediction method.

[0018] Figure 6 is a block diagram illustrating the configuration and operation of a video decoder for decoding an image.

[0019] FIG. 7 is a diagram illustrating an example of an intra prediction method in IBC (Intra Block Copy) mode.

[0020] FIG. 8 is a diagram illustrating an example of a method for performing intra prediction in IBC mode using template matching.

[0021] FIGS. 9 to 14 are drawings for explaining embodiments of a method for performing inter prediction using template matching.

[0022] Figure 15 is a drawing for explaining examples of the configuration of a template used for template matching.

[0023] Fig. 16 is a flowchart illustrating a method for setting a reference block using template matching according to one embodiment of the present invention.

[0024] FIGS. 17 and 18 are drawings for explaining embodiments of a method for determining a template used for template matching.

[0025] FIGS. 19 to 26 are drawings for explaining embodiments of a method for processing a template area that is outside the picture boundary.

[0026] Hereinafter, with reference to the attached drawings, a method and device for encoding and decoding an image according to an embodiment of the present invention will be described in detail.

[0027] In the following description of the present invention, detailed descriptions of related known functions or configurations will be omitted if they are deemed to unnecessarily obscure the gist of the invention. Furthermore, the terms described below are defined in light of their functions within the present invention and may vary depending on the intent or custom of the user or operator. Therefore, their definitions should be based on the overall content of the present invention.

[0028] In addition, in order to efficiently explain the technical components that make up the present invention, the preferred embodiments of the present invention implemented below omit, as much as possible, the system functional components that are already provided in each system functional configuration or are commonly provided in the technical field to which the present invention belongs, and focus on explaining the functional components that must be additionally provided for the present invention.

[0029] Anyone having ordinary skill in the art to which the present invention pertains will be able to easily understand the functions of components that have been conventionally used among the functional configurations that are not illustrated below, and will also be able to clearly understand the relationship between the components omitted as described above and the components added for the present invention.

[0030] In this specification, a device that encodes an image to generate a video signal bitstream is referred to as an encoding device, an encoding device, or an encoder, and a device that decodes the video signal bitstream to restore an image is referred to as a decoding device, a decoding device, or a decoder.

[0031] A pixel or pel is the smallest unit that constitutes an image, and the terms pixel and sample can be used interchangeably. A sample can generally represent a pixel or a pixel value, or it can represent only a pixel or pixel value of the luminance component, or only a pixel or pixel value of the chroma component.

[0032] In addition, the unit is used to refer to a basic unit of image processing or a specific location of a picture, and represents an image area including at least one of a luminance component and a chrominance component. Specifically, the unit can be used as a concept including a coding tree unit (CTU), a coding unit (CU), a prediction unit (PU), and a transform unit (TU). In addition, a block represents an image area including a specific component among luminance components and chrominance components, and an MxN block can represent a set of samples or transform coefficients consisting of M columns and N rows. Here, the terms unit, block, partition, signal, and area may be used interchangeably.

[0033] Meanwhile, a picture refers to a field or a frame, and these can be used interchangeably. For example, if the image is an interlaced image, one frame is divided into an odd (or odd, top) field and an even (or even, bottom) field, and each field is composed of a single picture unit, which can be encoded or decoded. If the image is a progressive image, one frame can be composed as a picture, which can be encoded or decoded.

[0034]

[0035] FIG. 1 is a block diagram illustrating an encoding device according to an embodiment of the present invention, and is intended to explain the configuration and operation of a video encoder for encoding an image.

[0036] Referring to FIG. 1, a video encoder (100) may be configured to include a transformation unit (110), a quantization unit (120), an inverse quantization unit (130), an inverse transformation unit (140), a filtering unit (150), a prediction unit (160), a DPB (Decoded Picture Buffer, 170), and an entropy coding unit (180).

[0037] The conversion unit (110) converts the residual signal, which is the difference between the input video signal and the prediction signal generated by the prediction unit (160), to obtain a conversion coefficient value.

[0038] For example, the Discrete Cosine Transform (DCT), the Discrete Sine Transform (DST), or the Wavelet Transform can be used.

[0039] The transform kernel used for transforming the residual block may be a transform kernel having separable vertical and horizontal transform properties. In this case, the transform for the residual block may be performed separately as vertical and horizontal transforms. For example, the encoder may perform a vertical transform by applying the transform kernel in the vertical direction of the residual block. Alternatively, the encoder may perform a horizontal transform by applying the transform kernel in the horizontal direction of the residual block.

[0040] Meanwhile, the transform kernel may be used as a term referring to a set of parameters used for transforming the residual signal, such as a transform matrix, a transform array, a transform function, or a transform, and may be any one of a plurality of available kernels, and transform kernels based on different transform types may be used for each of the vertical transform and the horizontal transform.

[0041] The transformation coefficients may be distributed such that higher coefficients are distributed toward the upper left corner of the block, and coefficients closer to '0' are distributed toward the lower right corner of the block. Furthermore, as the current block size increases, there is a possibility that many '0' coefficients exist in the lower right area. To reduce the transformation complexity of large blocks, only the upper left area can be left, and the remaining areas can be reset to '0'.

[0042] Additionally, error signals may exist only in some regions of a coding block, in which case the conversion process may be performed only on some arbitrary regions. For example, in a block of size 2Nx2N, error signals may exist only in the first 2NxN block, in which case the conversion process may be performed only on the first 2NxN block, but the conversion process may not be performed on the second 2NxN block and may not be encoded or decoded.

[0043] The encoder may perform additional transformations before the transform coefficients are quantized. The transformation method described above may be referred to as a primary transform, and the additional transformation may be referred to as a secondary transform.

[0044] The secondary transform can be optional for each residual block, for example, the encoder can improve coding efficiency by performing the secondary transform for areas where it is difficult to concentrate energy in the low-frequency region using only the primary transform.

[0045] Specifically, a secondary transformation may be additionally performed on blocks in which residual values ​​appear significantly in directions other than the horizontal or vertical direction of the residual block, and the secondary transformation may not be performed separately into vertical transformation and horizontal transformation, unlike the primary transformation. The secondary transformation as described above may be referred to as a low frequency non-separable transform (LFNST).

[0046] The quantization unit (120) quantizes the transformation coefficient value output from the transformation unit (110).

[0047] In order to increase coding efficiency, a method is used in which, instead of coding the picture signal as is, a picture is predicted using an already coded area through a prediction unit (160), and a restored picture is obtained by adding a residual value between the original picture and the predicted picture to the predicted picture.

[0048] To avoid mismatches between the encoder and decoder, when performing prediction in the encoder, information available in the decoder must also be used, and for this purpose, the encoder can perform a process of reconstructing the current block that it has encoded.

[0049] The inverse quantization unit (130) inversely quantizes the transform coefficient values, and the inverse transformation unit (140) restores the residual values ​​using the inverse quantized transform coefficient values.

[0050] The filtering unit (150) performs filtering operations using a deblocking filter, a sample adaptive offset (SAO), an adaptive loop filter (ALF), etc. to improve the quality of the restored picture and enhance encoding efficiency.

[0051] A deblocking filter is a filter for removing distortion within a block generated at the boundary between blocks in a restored picture. The encoder can determine whether to apply a deblocking filter to a boundary based on the distribution of pixels included in several columns or rows based on an arbitrary boundary within a block.

[0052] When a deblocking filter is applied, the filtering unit (150) can apply a long filter, a strong filter, or a weak filter depending on the deblocking filtering strength, and can process horizontal filtering and vertical filtering in parallel.

[0053] Sample Adaptive Offset (SAO) can be used to correct the offset from the original image on a pixel-by-pixel basis for a residual block to which a deblocking filter is applied. In order to correct the offset for a specific picture, the filtering unit (150) can use a method (Band Offset) that divides the pixels included in the image into a certain number of regions, determines the regions to perform offset correction, and applies the offset to the regions. In addition, the filtering unit (150) can use a method (Edge Offset) that applies the offset by considering the edge information of each pixel.

[0054] Adaptive Loop Filtering (ALF) is a method that divides pixels in an image into predetermined groups, determines a filter to be applied to each group, and performs differential filtering for each group. Information regarding whether to apply an adaptive loop filter can be signaled on a coding unit basis, and the shape and filter coefficients of the ALF filter to be applied can vary depending on the block. Furthermore, the same adaptive loop filter can be applied regardless of the characteristics of the target block.

[0055] The filtered picture can be stored in DPB (170) to be used as a reference picture.

[0056] The prediction unit (160) includes an intra / IBC prediction unit (161) and an inter prediction unit (165). The intra / IBC prediction unit (161) performs intra prediction and IBC prediction within the current picture, and the inter prediction unit (165) performs inter prediction to predict the current picture using a reference picture stored in the DPB (170).

[0057] The intra / IBC prediction unit (161) performs intra prediction from restored areas within the current picture and transmits intra encoding information to the entropy coding unit (180). Here, the intra encoding information may include at least one of an intra prediction mode, an MPM (Most Probable Mode) flag, an MPM index, and information about a reference sample.

[0058] Additionally, the intra / IBC prediction unit (161) can perform IBC (Intra BLock Copy) prediction from restored samples within the current picture and transmit IBC encoding information to the entropy coding unit (180).

[0059] In this case, the intra / IBC prediction unit (161) obtains a block vector value indicating a reference area used for prediction of the current area by referring to a specific area within the current picture, and can perform IBC prediction using the obtained block vector value.

[0060] The intra / IBC prediction unit (161) can transmit IBC encoding information including at least one of the size information of the reference area and block vector information (index information for block vector prediction of the current block within the motion candidate list, block vector difference information) to the entropy coding unit (160).

[0061] The inter prediction unit (165) refers to a specific area of ​​the restored reference picture to find the part most similar to the current area, obtains a motion vector value which is the distance between the areas, and transmits motion information (reference direction indication information (L0 prediction, L1 prediction, bidirectional prediction), reference picture index, motion vector information, etc.) for the obtained reference area to the entropy coding unit (180).

[0062] Additionally, the inter prediction unit (165) performs motion compensation using motion information to generate a prediction block for the current block, and transmits inter encoding information including motion information for the reference area to the entropy coding unit (180).

[0063] Meanwhile, the quantized transform coefficients in the form of a two-dimensional array can be rearranged into a one-dimensional array for entropy coding.

[0064] The method of scanning the quantized transform coefficients can be determined according to the size of the transform block and the intra prediction mode, and diagonal, vertical, and horizontal scans can be applied, and the scan information can be signaled on a block-by-block basis or derived from the decoder according to a set rule.

[0065] The entropy coding unit (180) generates a bitstream by entropy coding information representing quantized transform coefficients, intra-coding information, and inter-coding information, and for this purpose, a variable length coding (VLC) method and an arithmetic coding method can be used.

[0066] Variable-length coding (VLC) converts input symbols into a series of codewords, each of which can be of variable length. For example, frequently occurring symbols can be represented by shorter codewords, while less frequently occurring symbols can be represented by longer codewords.

[0067] As a variable length coding method, a context-based adaptive variable length coding (CAVLC) method can be used.

[0068] Arithmetic coding converts consecutive data symbols into a single prime number using the probability distribution of each data symbol, thereby obtaining the optimal prime number bits required to express each symbol.

[0069] As an arithmetic coding method, the context-based adaptive binary arithmetic code (CABAC) method can be used.

[0070] CABAC is a binary arithmetic encoding method that utilizes multiple context models generated based on experimentally derived probabilities. The context models can also be referred to as context models. First, if the symbols are not in binary form, the encoder binarizes each symbol using methods such as exp-Golomb. The binarized 0s and 1s can be represented as bins.

[0071] The CABAC initialization process is divided into context initialization and arithmetic coding initialization. Context initialization initializes the occurrence probability of each symbol, determined by the symbol type, quantization parameter (QP), and slice type (I, P, B). A context model with this initialization information can use probability-based values ​​obtained through experiments.

[0072] The context model provides the probability of occurrence of the Least Probable Symbol (LPS) or Most Probable Symbol (MPS) for the symbol currently being coded, and information (valMPS) on which empty value between 0 and 1 corresponds to the MPS.

[0073] One of several context models is selected through a context index (ctxIdx), and the context index can be derived through information of the block currently to be encoded or information of surrounding blocks.

[0074] Initialization for binary arithmetic coding is performed based on the probability model selected from the context model. Binary arithmetic coding is performed by dividing the data into probability intervals based on the occurrence probabilities of 0 and 1, and then encoding the probability interval corresponding to the bin to be processed becomes the entire probability interval for the bin to be processed next.

[0075] The location information within the probability interval processed for the last bin is output. However, since the probability interval cannot be infinitely divided, if it shrinks to a certain size, a renormalization process is performed to expand the probability interval and output the corresponding location information. Furthermore, after each bin is processed, a probability update process can be performed, which sets a new probability for the next bin to be processed based on the information from the processed bin.

[0076] As described above, the generated bitstream is encapsulated into NAL (Network Abstraction Layer) units as basic units.

[0077] NAL units are divided into VCL (Video Coding Layer) NAL units that contain video data and non-VCL NAL units that contain parameter information for decoding video data, and there may be various types of VCL or non-VCL NAL units.

[0078] A NAL unit consists of NAL header information and data, a Raw Byte Sequence Payload (RBSP). The NAL header information includes summary information about the RBSP. The RBSP of a VCL NAL unit contains an integer number of encoded coding tree units.

[0079] In order to decode a bitstream in a decoder, the bitstream must first be divided into NAL units, and then each divided NAL unit must be decoded. Meanwhile, the information required for decoding the bitstream can be transmitted as part of a picture parameter set (PPS), a sequence parameter set (SPS), a video parameter set (VPS), etc.

[0080] Meanwhile, the configuration and operation of the encoder described with reference to FIG. 1 are according to one embodiment of the present invention, and some configurations may be omitted or added as needed.

[0081]

[0082] Additionally, a single picture may be encoded by dividing it into sub-pictures, slices, tiles, etc. A sub-picture may include one or more slices or tiles. When a single picture is encoded by dividing it into multiple slices or tiles, all slices or tiles within the picture must be decoded before it can be displayed on the screen.

[0083] When a single picture is encoded into multiple subpictures, only any subpicture can be decoded and displayed on the screen. A slice can contain multiple tiles or subpictures, and a tile can contain multiple subpictures or slices.

[0084] Subpictures, slices, and tiles can be encoded or decoded independently of each other, which is effective for parallel processing and processing speed improvement, but the amount of bits may increase because the encoded information of adjacent subpictures, slices, and tiles cannot be used.

[0085] And subpictures, slices, and tiles can be encoded by being divided into multiple coding tree units (CTUs).

[0086] A coding tree unit can be composed of a luminance coding tree block (CTB) of size 128x128 and two chrominance coding tree blocks of size 64x64.

[0087] A single coding tree unit may be undivided and constitute a single coding unit (CU) itself, or may be divided into multiple coding units as illustrated in Fig. 2. A coding unit may be composed of a luminance coding block (CB) and two chrominance coding blocks.

[0088] A coding unit may consist of one transform unit (TU), or may be split into multiple transform units. A transform unit may consist of a luminance transform block (TB) and two chrominance transform blocks.

[0089] Here, a coding unit represents a basic unit for processing a picture in the process of prediction, transformation, quantization, entropy coding, and decoding, and the size and shape of a coding unit within a picture may not be constant.

[0090] A coding unit may have a square or non-square shape, and a rectangular coding unit may include a vertical coding unit whose height is greater than its width and a horizontal coding unit whose width is greater than its height.

[0091] The coding tree unit is first partitioned into a Quad Tree (QT) structure, such that a single node of size 2NX2N can be partitioned into four nodes of size NXN. Furthermore, the Quad Tree partitioning can be performed recursively, and not all nodes need to be partitioned to the same depth.

[0092] Leaf nodes of a quadtree can be further partitioned into a multi-type tree (MTT) structure. For example, in a multi-type tree structure, a single node can be partitioned into a binary or ternary tree structure with horizontal or vertical partitioning. Accordingly, a multi-type tree structure can have four partitioning structures: vertical binary partitioning, horizontal binary partitioning, vertical ternary partitioning, and horizontal ternary partitioning.

[0093] In each tree structure, both the width and height of a node can have a power of 2. For example, in a binary tree (BT) structure, a node of size 2NX2N can be split into two NX2N nodes by vertical binary splitting, and into two 2NXN nodes by horizontal binary splitting.

[0094] Also, in the Ternary Tree (TT) structure, a node of size 2NX2N can be split into nodes of size (N / 2)X2N, NX2N, and (N / 2)X2N by vertical ternary splitting, and into nodes of size 2NX(N / 2), 2NXN, and 2NX(N / 2) by horizontal ternary splitting. This multi-type tree splitting can be performed recursively.

[0095] A leaf node of a multi-type tree can be a coding unit. If the coding unit is no larger than the maximum transformation length, the coding unit can be used as a unit for prediction and transformation without further splitting. On the other hand, if the width or height of the coding unit is larger than the maximum transformation length, the coding unit can be split into multiple transformation units without explicit signaling regarding the splitting.

[0096] The tree partitioning structure as described above may have the same shape (Single Tree) for the luminance block and the chrominance block, or different shapes (Dual Tree) for the luminance block and the chrominance block.

[0097] Meanwhile, the block division from the coding tree unit (CTU) to the coding unit (CU) as described above can be performed through a rate-distortion optimization (RDO) process, whereby the division structure with the smallest rate-distortion cost (RD cost) value within the allowable size and depth conditions is selected to determine the final coding unit.

[0098]

[0099] Hereinafter, embodiments of the intra prediction method will be described in more detail with reference to FIGS. 3 and 4.

[0100] For intra prediction, intra prediction mode information indicating an intra prediction direction may be signaled, and the intra prediction mode information indicates any one of a plurality of intra prediction modes constituting an intra prediction mode set.

[0101] As illustrated in FIG. 3, the intra prediction mode set may include a planar mode, a DC mode, and 65 directional modes, and each intra prediction mode may be indicated through an intra prediction mode index.

[0102] For example, intra prediction mode index "0" indicates planar mode, intra prediction mode index "1" indicates DC mode, and intra prediction mode indexes "2" to "66" can indicate different directional modes, respectively.

[0103] The directional modes each indicate different angles within a preset angular range, for example, a directional mode can indicate an angle within an angular range from 45 degrees to -135 degrees clockwise.

[0104] In this case, the intra prediction mode index "2" may indicate the Horizontal Diagonal (HDIA) mode, the intra prediction mode index "18" may indicate the Horizontal (HOR) mode, the intra prediction mode index "34" may indicate the Diagonal (DIA) mode, the intra prediction mode index "50" may indicate the Vertical (VER) mode, and the intra prediction mode index "66" may indicate the Vertical Diagonal (VDIA) mode.

[0105] If the current block is a non-square block, 20 additional wide angular modes can be used, indicating angles greater than 45 degrees clockwise or less than -135 degrees.

[0106] Based on the intra prediction mode information as described above, reference samples to be used for intra prediction for the current block are determined.

[0107] For example, if the intra prediction mode index indicates a specific directional mode, the reference sample corresponding to that angle from the current sample of the current block is used for prediction for the current sample.

[0108] As described above, for intra prediction, surrounding already restored samples are used as reference samples, and the reference samples may be restored samples located to the left or above the current block.

[0109] Referring to FIG. 4, the reference samples may be samples adjacent to the left boundary and upper boundary of the current block.

[0110] For example, if the size of the current block is NxN and samples of a single reference line adjacent to the current block are used for intra prediction, reference samples can be set using (2N*2+1) surrounding samples located on the left (L, Left), top (T, Top), and top-left (TL, Top-left) of the current block.

[0111] Meanwhile, samples of multiple reference lines (MRL) may be used for intra prediction of the current block, and the multiple reference lines may be composed of n reference lines located within a preset range from the current block. In this case, separate reference line index information indicating the reference lines to be set as reference pixels may be signaled.

[0112] Additionally, if at least some of the samples to be used as reference samples have not yet been restored, reference samples can be obtained through a reference sample padding process, and a reference sample filtering process can be performed to reduce errors in intra prediction.

[0113]

[0114] Hereinafter, with reference to FIG. 5, embodiments of the inter prediction method will be described in more detail.

[0115] Referring to FIG. 5, inter prediction is a prediction method that obtains prediction information for the current block, which is a block to be encoded, from a reference block located within a reference picture, and uses a motion vector (mv) indicating the location of the reference block.

[0116] Motion information used for inter prediction may include reference direction indication information (inter_pred_idc), reference picture indexes (ref_idx_l0, ref_idx_l1), and motion vectors (mvL0, mvL1).

[0117] Meanwhile, for unidirectional prediction using L0 reference pictures, predFlagL0=1, predFlagL1=0 are set, for unidirectional prediction using L1 reference pictures, predFlagL0=0, predFlagL1=1 are set, and for bidirectional prediction using both L0 and L1 reference pictures, predFlagL0=1, predFlagL1=1 can be set.

[0118] If the current block is a coding unit (CU), the coding unit (CU) can be divided into multiple sub-blocks, the motion information for each sub-block can be the same or different, and each sub-block can be encoded or decoded independently of each other.

[0119] And since the motion vector (mv) of the current block is likely to be similar to the motion vector of the surrounding blocks, the motion vector of the surrounding blocks can be used as a motion information prediction value (MVP, Motion Vector Predictor), and in this case, the motion vector of the current block can be derived using the motion vector of the surrounding blocks.

[0120] Additionally, to improve the accuracy of the motion vector, the difference (MVD, Motion Vector Difference) between the optimal motion vector for the current block found in the original image by the encoder and the motion vector prediction value can be signaled.

[0121] Motion vectors can have various resolutions, and the resolution of motion vectors can vary on a block-by-block basis. Motion vector resolution can be expressed in integer units, half-pixel units, quarter-pixel units, 1 / 16-pixel units, and integer-to-4 pixel units.

[0122] Here, whether to selectively apply motion vector resolution on a block-by-block basis is signaled with amvr_flag, and which motion vector resolution to apply to the current block can be signaled with amvr_precision_idx.

[0123] When bidirectional prediction is applied, the weights for the two prediction blocks can be the same or different when applying weighted averaging, and information about the weights can be signaled via bcw_idx.

[0124] And to increase the accuracy of the prediction value for motion information, Merge mode or AMVP (Advanced Motion Vector Prediction) mode can be selectively used on a block-by-block basis.

[0125] Merge mode is a method of configuring the motion information of the current block to be identical to the motion information of the surrounding blocks adjacent to the current block, and has the advantage of increasing the encoding efficiency of motion information by spatially propagating the motion information without change in a motion area having homogeneity.

[0126] Meanwhile, AMVP mode is a method of signaling the most optimal motion information by predicting motion information in the L0 and L1 prediction directions, respectively, to express more accurate motion information.

[0127] As a method for deriving motion information in merge mode or AMVP mode, a motion candidate list is constructed using predicted values ​​of motion information derived from surrounding blocks of the current block, and index information for the optimal motion candidate can be signaled.

[0128] For example, in merge mode, one merge index (merge_idx) may be signaled, and in AMVP mode, motion candidate indices (mvp_l0_flag, mvp_l1_flag) for L0 and L1, respectively, may be signaled.

[0129] Meanwhile, the motion candidate list derived for one coding unit may be diverse, and a merge index or motion candidate index may be signaled for each motion candidate list.

[0130] To construct a motion candidate list in inter prediction, the surrounding blocks spatially adjacent to the current block may include at least one of a Left block, a Left Below block, an Above block, an Above Right block, and an Above Left block.

[0131] Additionally, a temporally adjacent neighboring block to the current block may be a block that contains the upper left pixel position of the bottom right block of the current block in the corresponding picture (collocated picture).

[0132] Here, if a temporally adjacent neighboring block to the current block is encoded in intra mode, or if a temporally adjacent neighboring block to the current block exists in an unusable position, a block containing the horizontal and vertical center pixel position of the current block in a collocated picture corresponding to the current picture can be used as a temporal neighboring block.

[0133] The motion candidate information derived from the corresponding picture (collocated picture) as above can be referred to as TMVP (Temporal Motion Vector Predictor), and sbTMVP (sub-block Temporal Motion Vector Predictor), which is a sub-block unit TMVP derivation method in which TMVP candidates are derived for each of the sub-blocks into which the current block is divided, can also be used.

[0134] Meanwhile, in Merge Skip mode, information about residual blocks may not exist for blocks encoded in Merge mode.

[0135] Additionally, in AMVP-merge mode, the merge mode and AMVP mode can be used interchangeably for bidirectional motion information of the current block, for example, motion information in the L0 direction can be derived using the AMVP mode, and motion information in the L1 direction can be derived using the merge mode.

[0136] For bidirectional prediction, the SMVD (Symmetric MVD) mode can be used to reduce the amount of motion information transmitted by making the MVD values ​​for the L0 direction and L1 direction symmetrical to each other.

[0137] In this case, MVD information for the L1 direction, which is symmetrical to the L0 direction, is not transmitted, and reference picture information for the L0 and L1 directions is also not transmitted and can be derived during the decoding process.

[0138] OBMC (Overlapped Block Motion Compensation) is a method that generates prediction blocks for the current block using motion information from surrounding blocks when the motion information between blocks differs, and then weights and averages the prediction blocks to generate a final prediction block for the current block. In this case, blocking artifacts occurring at block boundaries in motion-compensated images can be reduced.

[0139] Meanwhile, MMVD (Merge mode with MVD) mode can be used to derive more accurate motion vectors by adding motion vector difference (MVD) to the motion vector derived in merge mode.

[0140] For example, the motion vector of one of the first two candidates included in the candidate list of the merge mode may be used as the initial motion vector, and the final motion vector may be obtained by adding the MVD derived from the MVD direction and motion distance information to the initial motion vector.

[0141] DMVR (Decoder-side Motion Vector Refinement) is a method of correcting motion information through the correlation of already restored reference images to find more accurate motion information. By using the bidirectional motion information of the current block, the best matching point between reference blocks within an arbitrary set area of ​​two reference pictures can be used as new bidirectional motion information.

[0142] Meanwhile, in the above DMVR, MP-DMVR (Multi-pass DMVR) may be used, in which the encoder performs DMVR on one block to correct motion information, then divides the block into sub-blocks and performs DMVR on each sub-block to correct motion information of the sub-block.

[0143] Additionally, LIC (Local Illumination Compensation) is a method for compensating for luminance changes between blocks. It derives a linear model using surrounding pixels adjacent to the current block, and then compensates for the luminance information of the current block through the linear model.

[0144] Additionally, to express movements such as zoom in, zoom out, and rotation, an Affine model-based motion prediction method using a four (rotation) or six (zoom in, zoom out, rotation) parameter model can be used.

[0145] For example, in the case of BDOF (Bi-Directional Optical Flow), the amount of pixel change is estimated based on optical flow from a reference block of a block composed of bidirectional motion, and can be used to correct the prediction block.

[0146] PROF (Prediction refinement with optical flow) is intended to improve the accuracy of sub-block unit affine motion prediction to be similar to the accuracy of pixel unit motion prediction. Similar to BDOF, it calculates a correction value on a pixel-by-pixel basis for pixel values ​​that have been affine motion compensated on a sub-block-by-subblock basis based on optical flow, thereby obtaining a final prediction signal.

[0147] Meanwhile, in the case of CIIP (Combined Inter- / Intra-picture Prediction), when generating a prediction block for the current block, the final prediction block can be generated by weighting and averaging the prediction blocks generated by the intra-picture prediction method and the inter-picture prediction method.

[0148] And BCW (Bi-prediction with CU-level Weights) is a method of performing weighted averaging by applying weights on a block-by-block basis for two motion-compensated prediction blocks from different reference pictures.

[0149] Additionally, multi-hypothesis prediction (MHP) can be used to perform weight prediction using various prediction signals by transmitting additional motion information to unidirectional and bidirectional motion information.

[0150]

[0151] FIG. 6 is a block diagram illustrating a decoding device according to an embodiment of the present invention, and is intended to explain the configuration and operation of a video decoder for encoding an image.

[0152] Referring to FIG. 6, the video decoder (200) may be configured to include an entropy decoding unit (210), an inverse quantization unit (220), an inverse transformation unit (230), a filtering unit (240), a prediction unit (260), and a DPB (Decoded Picture Buffer, 270).

[0153] The entropy decoding unit (210) entropy decodes the bitstream to extract transform coefficient information, intra-coding information, inter-coding information, etc. for each region.

[0154] For example, the entropy decoding unit (210) can obtain a binary code for transform coefficient information of a specific area from a bitstream, and can obtain a quantized transform coefficient by de-binarizing the binary code.

[0155] The inverse quantization unit (220) inversely quantizes the quantized transform coefficients, and the inverse transformation unit (230) restores the residual value using the inverse quantized transform coefficients.

[0156] Meanwhile, the residual value obtained from the inverse transformation unit (230) is combined with the predicted value obtained from the prediction unit (260) to restore the original pixel value.

[0157] The filtering unit (240) performs a filtering operation using a deblocking filter, a sample adaptive offset, an adaptive loop filter, etc. to improve the image quality of the restored picture, and the filtered picture can be output or stored in the DPB (270) to be used as a reference picture for the next picture.

[0158] The prediction unit (260) includes an intra / IBC prediction unit (261) and an inter prediction unit (265), and generates a prediction picture by utilizing the encoding type decoded through the entropy decoding unit (210), the transform coefficient for each region, and the intra / inter encoding information.

[0159] To restore the current block in which decoding is performed, the decoded region of the current picture or other pictures containing the current block can be used. A picture (or tile / slice) that performs intra prediction or intra BC prediction using only the current picture for restoration is called an intra picture or I picture (or tile / slice), and a picture (or tile / slice) that can perform all of intra prediction, inter prediction, and intra BC prediction is called an inter picture (or tile / slice).

[0160] Meanwhile, a picture (or tile / slice) that uses at most one motion vector and reference picture index to predict sample values ​​of each block among inter-pictures (or tiles / slices) is called a predictive picture or P-picture (or tile / slice), and a picture (or tile / slice) that uses at most two motion vectors and reference picture indices is called a bi-predictive picture or B-picture (or tile / slice).

[0161] That is, a P picture (or tile / slice) uses at most one motion information set to predict each block, and a B picture (or tile / slice) uses at most two motion information sets to predict each block. Here, a motion information set may include one or more motion vectors and one reference picture index.

[0162] The intra / IBC prediction unit (261) generates a prediction block using intra encoding information and restored samples within the current picture, and the intra encoding information may include at least one of an intra prediction mode, an MPM (Most Probable Mode) flag, and an MPM index.

[0163] The intra / IBC prediction unit (261) can predict sample values ​​of the current block using restored samples located on the left and / or upper side of the current block as reference samples.

[0164] For example, the reference samples may be samples adjacent to the left boundary of the current block and / or samples adjacent to the upper boundary, and may be samples located on a line within a preset distance from the left boundary of the current block among samples of neighboring blocks of the current block and / or samples located on a line within a preset distance from the upper boundary of the current block. In this case, the neighboring blocks of the current block may include at least one of a left (L) block, an upper (A) block, a below left (BL) block, an above right (AR) block, or an above left (AL) block adjacent to the current block.

[0165] Additionally, the intra / IBC prediction unit (261) can restore the current region by referring to a specific region including restored samples within the current picture. To this end, the intra / IBC prediction unit (261) can perform IBC prediction using IBC encoding information obtained from the entropy decoding unit (210), and the IBC encoding information can include block vector information.

[0166] The inter prediction unit (265) generates a prediction block using the reference picture and inter encoding information stored in the DPB (270), and the inter encoding information may include a set of motion information (reference picture index, motion vector information, etc.) of the current block for the reference block.

[0167] Meanwhile, inter prediction can include L0 prediction, L1 prediction, and bidirectional prediction (Bi-prediction).

[0168] L0 prediction refers to prediction using a single reference picture included in the L0 picture list, and L1 prediction refers to prediction using a single reference picture included in the L1 picture list. For this, a set of motion information (e.g., motion vector and reference picture index) may be required.

[0169] Bidirectional prediction can utilize up to two reference regions, which may reside in the same reference picture or in different pictures. Accordingly, among the two sets of motion information used in bidirectional prediction, two motion vectors may correspond to the same reference picture index or to different reference picture indices.

[0170] At this time, the reference pictures are pictures that are located temporally before or after the current picture, and may be completed pictures that have already been restored, and the two reference areas used in the bidirectional prediction method may be areas selected from the L0 picture list and the L1 picture list, respectively.

[0171] The inter prediction unit (265) can obtain a reference block of the current block using a motion vector and a reference picture index, and the reference block exists in a reference picture corresponding to the reference picture index.

[0172] Additionally, the sample values ​​of a block specified by a motion vector or their interpolated values ​​can be used as a predictor of the current block. For motion prediction with subpel-level pixel accuracy, an 8-tap interpolation filter can be used for the luminance signal, and a 4-tap interpolation filter can be used for the chrominance signal.

[0173] Meanwhile, the configuration and operation of the decoder described with reference to FIG. 6 are according to one embodiment of the present invention, and some configurations may be omitted or added as needed, and the decoder may decode an image by performing the reverse process of the encoding method of the encoder described above.

[0174]

[0175] According to an embodiment of the present invention, intra or inter prediction as described above can be performed in an encoder or decoder using template matching.

[0176] Template matching is a method of constructing a template using restored pixels surrounding a block, finding the block with the highest similarity between the current block and the template, and using it for intra or inter prediction.

[0177] Hereinafter, with reference to FIGS. 7 to 14, embodiments of a method for performing intra or inter prediction using template matching will be described.

[0178]

[0179] According to one embodiment of the present invention, template matching can be used for intra prediction in IBC mode.

[0180] Referring to FIG. 7, in the IBC mode, prediction information for the current block, which is the encoding target block, can be obtained from a reference block that has already been decoded and is located within the same screen.

[0181] At this time, the vector information (including direction and size) from the current block to the reference block is referred to as a block vector (BV).

[0182] Although the IBC prediction method described above is an intra prediction method, it is similar to an inter prediction method in that it derives and transmits motion information of the block to be encoded (current block).

[0183] Accordingly, the IBC prediction method can be divided into ibc_skip / merge mode and ibc_amvp mode, and ibc_skip / merge and ibc_amvp can use skip, merge, AMVP (Advanced Motion Vector Prediction) and similar encoding, information transmission and decoding methods in inter-screen prediction.

[0184] The syntax structure for IBC prediction can be as shown in Table 1 below, and the sps_ibc_enabled_flag of the SPS (Sequence Parameter Set) and the syntax (sps_six_minus_max_num_ibc_merge_cand) indicating the maximum size of the candidate list of the ibc_skip / merge mode when the flag is enabled can be transmitted.

[0185]

[0186] Referring to Table 2 below, in a coding unit (CU) that is a block to be encoded, a flag (pred_mode_ibc_flag) indicating whether the block is in IBC mode and a flag (general_merge_flag) indicating whether the block is in ibc_merge mode can be transmitted.

[0187] And in the case of ibc_amvp mode, the block vector (BV) value of the corresponding block can be derived using the mvd, mvp, and amvr values.

[0188]

[0189] Meanwhile, referring to Table 3 below, in the case of ibc_skip / merge mode, the block vector (BV) value of the corresponding block can be derived using merge_idx information.

[0190]

[0191] FIG. 8 illustrates an example of a method for performing intra prediction in IBC mode using template matching.

[0192] As described above, in the IBC mode, in order to use a reference block within the same screen as prediction information for the target block to be encoded, a block vector (BV), which is motion information for the target block, is derived and transmitted to the decoding device.

[0193] In the IBC mode using template matching, the use of a reference block within the same screen as the prediction information for the target block is the same as the IBC mode above, but the block vector (BV), which is motion information for the target block, is not transmitted to the decoding device, and the decoding device can find the prediction information by searching for the reference block on its own through the template matching process.

[0194] Referring to Fig. 8, a restored template area is defined around the current block (Cur. block), and a template area (Best matching Template) that most closely matches the template area of ​​the current block in the area of ​​restored blocks within the same screen (Reconstructed area) is searched, thereby obtaining a predicted block (ref. block).

[0195] And, based on the position of the predicted block (ref. block) obtained as described above, a block vector (BV) value for the current block (Cur. block) can be derived.

[0196] As described above, in the IBC mode using template matching, a block vector (BV) for a target block is derived in a decoding device, and the block vector (BV) value can be used for intra prediction in the same way as the block vector (BV) value of the IBC mode.

[0197] As another embodiment of using template matching for intra prediction in IBC mode, after a merge candidate list or an AMVP candidate list for IBC is constructed, an error value for each candidate is calculated through template matching, and the candidate list can be reordered according to the calculated template error value.

[0198] For example, the block vector candidate list (BV candidate list) may include BV(s) of one or more surrounding blocks adjacent to the target block, BV(s) stored in a block vector buffer (BV buffer), and a default BV, but the present invention is not limited thereto.

[0199] Meanwhile, the error value represents the difference between the template of the target block and the template of the reference block pointed to by the candidate, and can be calculated as SAD (Sum of Absolute Differences), but the present invention is not limited thereto, and various values ​​such as MRSAD (Mean Removal SAD), SATD (Sum of Absolute Transformed Differences), weighted SAD, or MSE (Mean Squared Error) can be adaptively used depending on the intra prediction mode of the target block, etc.

[0200] The encoder can rearrange the candidates included in the candidate list in order of the smallest template error value, select the final candidate through the Rate-Distortion Optimization (RDO) process, and transmit the index value of the selected candidate to the decoder through the bitstream.

[0201] As described above, since the candidate list is rearranged in order of decreasing template error value, the number of bits required to transmit the candidate index value to the decoder can be saved, and also, if the candidate with the smallest template error value is selected as the final candidate, the index value may not be transmitted.

[0202] For example, if the maximum number of candidates included in the merge candidate list is 5, the binary representation for the 5 candidates can be as shown in Table 4 below.

[0203]

[0204] In the case of Table 4, if the BV (0,1) value with index 2 is selected as the final candidate through the RDO process among the block vector candidate list, the BV index value can be included in the bitstream as a 3-bit codeword, "110", and transmitted to the decoder.

[0205] Table 5 below shows an example of the results of rearranging the list of block vector candidates shown in Table 4 in order of decreasing template error values.

[0206]

[0207] In the case of Table 5, if the BV (0,1) value is selected as the final candidate through the RDO process among the block vector candidate list, the BV index value is expressed as a 1-bit codeword, “0,” so the number of bits for transmitting the BV index value to the decoder can be saved.

[0208] The maximum number of candidates, binarization method, BV value, etc. of the candidate list described with reference to Table 4 and FIG. 5 above are examples, and the present invention is not limited thereto, and image compression performance can be improved through candidate list reordering using template matching as described above.

[0209] As another embodiment of using template matching for intra prediction in IBC mode, instead of transmitting a value indicating the resolution or precision for a block vector (BV) to the decoder, the BV value can be transmitted to the decoder with a high resolution (or precision), and the decoder can compensate for the BV value with a low resolution (or precision) using template matching.

[0210] For example, the encoder may transmit BV values ​​to the decoder at 4-pixel resolution, and the decoder may compensate or refine the BV values ​​at 1-pixel resolution using template matching.

[0211] The encoder can specify the BV values ​​in units of 0, 4, 8, 12, and 16 pixels, which are multiples of 4, and transmit that information to the decoder. For example, the encoder can transmit (12,8) as the BV value of the target block to the decoder.

[0212] In this case, the decoder sets the search range for template matching to between 8 and 16 on the x-axis and between 4 and 12 on the y-axis, searches for the template with the minimum error value in units of 1 pixel within the search range, and sets the block with the template with the minimum error value as the final reference block.

[0213] In this way, the encoder does not transmit an index value (e.g., abvr index) to indicate the resolution of the BV value, but transmits the BV value at the default resolution of 4 pixels, and allows the decoder to compensate the BV value in units of 1 pixel, thereby saving the number of bits required to transmit the index (e.g., abvr index).

[0214] Alternatively, the resolution of the BV values ​​can be reduced from the existing resolutions (e.g., 4-pixel, 1-pixel, 1 / 4-pixel) to a smaller number of resolutions (e.g., 4-pixel, 1-pixel), and the encoder can transmit an index value for one of the smaller number of resolutions (e.g., 4-pixel, 1-pixel) to the decoder.

[0215] In this case, the decoder can compensate the BV value to a lower resolution than the transmitted resolution through a template matching process, thereby saving the number of bits for expressing and transmitting the index (e.g., abvr index) indicating the resolution of the BV value.

[0216] For example, as shown in Table 6 below, when the resolution of the BV value can be transmitted in three resolutions (4-pixel, 1-pixel, 1 / 4-pixel), if the resolution of the BV value for the target block is 1-pixel, the corresponding resolution value can be expressed and transmitted as a 2-bit codeword, “10.”

[0217]

[0218] Meanwhile, as shown in Table 7 below, when the resolution of the BV value is reduced to two resolutions (4-pixel, 1-pixel), the resolution value of the BV value for the target block, 1-pixel, can be expressed and transmitted as a 1-bit codeword, “1.”

[0219]

[0220] In this case, the decoder can compensate the BV values ​​transmitted at a resolution of 1 pixel in 1 / 4-pixel units through a template matching process.

[0221] The maximum number of indices and resolution values ​​indicating the resolution of the BV value described with reference to Tables 6 and 7 above are examples, and the present invention is not limited thereto, and image compression performance can be improved through compensation of the BV value using template matching as described above.

[0222] The intra prediction method described with reference to FIGS. 7 and 8 can be performed by a video encoding device and a video decoding device according to an embodiment of the present invention, and can be performed, for example, by an intra / IBC prediction unit (161) of a video encoder (100) illustrated in FIG. 1 or an intra / IBC prediction unit (261) of a video decoder (200) illustrated in FIG. 4.

[0223]

[0224] Meanwhile, the method of using template matching for intra prediction as described above can also be applied to inter prediction.

[0225] Hereinafter, embodiments of methods for using template matching for inter prediction will be described, and a description of methods that are identical to the methods for using template matching for intra prediction described with reference to FIGS. 7 and 8 will be omitted.

[0226]

[0227] According to one embodiment of the present invention, template matching can be used for inter prediction in Merge / Skip mode.

[0228] A method of utilizing template matching in merge and skip modes among inter prediction modes, wherein after a merge candidate list is constructed, an error value for each candidate is calculated through template matching, and the candidate list can be reordered according to the calculated template error value.

[0229] Here, the error value represents the difference between the template of the target block and the template of the reference block in the reference picture pointed to by the candidate, and can be calculated with various values ​​such as SAD, MRSAD, SATD, weighted SAD, or MSE.

[0230] The encoder can rearrange the candidates included in the candidate list in order of the smallest template error value, select the final candidate through the rate-distortion optimization (RDO) process, and transmit the index value of the selected candidate to the decoder through the bitstream.

[0231] As described above, since the merge candidate list is rearranged in order of decreasing template error value, the number of bits required to transmit the candidate index value to the decoder can be saved, and in addition, if the candidate with the smallest template error value is selected as the final candidate, the index value may not be transmitted.

[0232] For example, if the maximum number of candidates included in the merge candidate list is 5, the binary representation for the 5 candidates may be as shown in Table 8 below.

[0233]

[0234] In the case of Table 8, if the MV (0,1) value with index 2 is selected as the final candidate through the RDO process among the motion vector (MV) candidate list, the MV index value can be included in the bitstream as a 3-bit codeword, "110", and transmitted to the decoder.

[0235] Table 9 below shows an example of the results of rearranging the motion vector candidate list shown in Table 8 in order of decreasing template error value.

[0236]

[0237] In the case of Table 9, if the MV (0,1) value is selected as the final candidate through the RDO process among the motion vector candidate list, the MV index value is expressed as a 1-bit codeword, “0,” so that the number of bits for transmitting the MV index value to the decoder can be saved.

[0238] The maximum number of candidates, binarization method, MV value, etc. of the candidate list described with reference to Tables 8 and 9 above are examples, and the present invention is not limited thereto, and image compression performance can be improved through candidate list reordering using template matching as described above.

[0239]

[0240] According to another embodiment of the present invention, template matching can be used for inter prediction in Advanced Motion Vector Prediction (AMVP) mode.

[0241] As a first embodiment of a method of utilizing template matching in AMVP mode among inter prediction modes, after an AMVP candidate list is constructed, an error value for each candidate is calculated through template matching, and the candidate list can be reordered according to the calculated template error value.

[0242] The encoder can rearrange the candidates included in the candidate list in order of the smallest template error value, select the final candidate through the rate-distortion optimization (RDO) process, and transmit the index value of the selected candidate to the decoder through the bitstream.

[0243] Meanwhile, if the maximum number of candidates included in the AMVP candidate list is 2, the encoder can construct an AMVP candidate list including 2 candidates and then select a candidate with a smaller error value through a template matching process for each candidate.

[0244] In this case, the encoder does not transmit the AMVP index value to the decoder, but the decoder can derive and use the AMVP index value on its own through the template matching process.

[0245] As a second embodiment of a method of using template matching in AMVP mode among inter prediction modes, a reference picture used in AMVP mode can be selected using template matching.

[0246] When motion vector (MV) related information for the target block, such as the Motion Vector Difference (MVD) and MVP index values, is passed to the decoder, the decoder can derive the MV values ​​even if it does not know the location of the reference picture.

[0247] Meanwhile, since the slice header information of the current picture includes information about the reference picture list used in the slice, the decoder can know the list of reference pictures available for the current block.

[0248] For example, as illustrated in FIG. 9, if a (t-1) reference picture, a (t-2) reference picture, and a (t-3) reference picture can be used for the current block (Cur. block) according to slice header information, reference block (Ref. block) candidates at the same position indicated by the MV values ​​derived from the MVD and MVP index values ​​in each of the three reference pictures can be obtained.

[0249] In this case, for each of the three reference block candidates, an error value between the template (Template) around the current block (Cur. block) and the template (Template 1, 2, 3) around the reference block (Ref. block) is calculated, and the reference block candidate with the smallest calculated error value can be selected as the final reference block.

[0250] As shown in Fig. 10, when the position of an object is moved in pictures, among the templates of three reference block candidates (Template 1, 2, 3), the template with the smallest error value with respect to the template of the current block (Cur. block) may be the template (Template 2) in the (t-2) reference picture.

[0251] Accordingly, the reference picture for the current block (Cur. block) can be determined as the (t-2) reference picture, and the decoder can select the reference picture for the current block (Cur. block) without receiving a reference picture index value from the encoder.

[0252] Alternatively, after a reference picture list for the current block (Cur. block) is constructed, a template error value for a reference block pointed to by an MV value within each reference picture may be calculated through template matching, and the reference picture list may be rearranged in order of the smallest calculated template error value.

[0253] In this case, the encoder can rearrange the reference pictures included in the reference picture list in order of the smallest template error value, select a final reference picture through a rate-distortion optimization (RDO) process, and transmit the reference picture index value of the selected candidate to the decoder through the bitstream.

[0254] As described above, since the reference picture list is rearranged in order of decreasing template error value, a reference picture in which a reference block similar to the target block is located can be placed higher in the reference picture list through the template matching process, thereby saving the number of bits for transmitting the reference picture index value to the decoder.

[0255] Fig. 11 is a flowchart illustrating an embodiment of a method for selecting a reference picture from a reference picture list using template matching. A description of the methods illustrated that are identical to those described above will be omitted.

[0256] Referring to FIG. 11, information about a reference picture list (ref.pic.list) is parsed from a slice header (SH, Slice Header) for a target block (step S1100).

[0257] MVD and MVP indexes are parsed (step S1110), and MV values ​​are derived using the parsed MVD and MVP index values ​​(step S1120).

[0258] i is set to “0”, and the minimum error value (min_error) is set to a preset value (MAX) (step S1130).

[0259] After that, for the reference block pointed to by the MV value derived in step S1120 within the i-th reference picture among the reference picture list, a template error value (TM_error) with respect to the target block is calculated (step S1140).

[0260] If the template error value (TM_error) calculated in step S1140 is smaller than the minimum error value (min_error) (step S1150), the minimum error value (min_error) is set to the template error value (TM_error) calculated in step S1140 (step S1160), and k is set to i (step S1170).

[0261] Next, if the i-th reference picture is the final reference picture (final ref.pic) (step S1180), the k-th reference picture is selected as the reference picture for the target block, and the block pointed to by the MV value within the k-th reference picture is selected as the reference block for the target block (step S1190).

[0262] Meanwhile, if the judgment result at step S1150 is that the template error value (TM_error) calculated at step S1140 is not smaller than the minimum error value (min_error), steps S1160 and S1170 are skipped, and step S1180 is performed.

[0263] And as a result of the judgment at step S1180, if the i-th reference picture is not the last reference picture (final ref.pic), i is increased by "1" (step S1185), and then the process returns to step S1140, and the above-described steps are repeated for the next reference picture in the reference picture list.

[0264] As a third embodiment of a method of using template matching in AMVP mode among inter prediction modes, instead of transmitting a value indicating the resolution or precision of a motion vector (MV) to a decoder, an MV value can be transmitted to the decoder with a high resolution (or precision), and the decoder can compensate for the MV value with a low resolution (or precision) using template matching.

[0265] For example, the encoder may transmit MV values ​​to the decoder at 1 / 4-pixel resolution, and the decoder may compensate or refine the MV values ​​at 1 / 16-pixel resolution using template matching.

[0266] In this way, the number of bits required to transmit an index (e.g., amvr index) can be saved by having the encoder not transmit an index value (e.g., amvr index) to indicate the resolution of the MV value, but instead having the decoder compensate for the MV value in pixel units lower than the resolution of the transmitted MV value.

[0267] Alternatively, the resolution of the MV value can be reduced from the existing four resolutions (1 / 4-pixel, 1 / 2-pixel, 1-pixel, 4-pixel) to a smaller number, for example, two resolutions (1 / 4-pixel, 1-pixel), so that the encoder transmits the index value for one of the two resolutions to the decoder, thereby saving the number of bits for transmitting the index (e.g., amvr index).

[0268] Here, the method by which the decoder compensates the MV value using template matching may be the same as the method by which the decoder compensates the BV value using template matching in intra prediction in IBC mode, so a detailed description thereof will be omitted.

[0269] As a fourth embodiment of a method of using template matching in AMVP mode among inter prediction modes, in the case of bidirectional prediction using BCW (Bi-prediction with CU-level Weight), the encoder does not transmit BCW index values ​​for indicating weights applied to bidirectional reference blocks to the decoder, but rather allows the decoder to derive weight values ​​using template matching.

[0270] In the case of BCW, to indicate the weight (w1) of one of the weights (w1, w2) for each of the bidirectional reference blocks, the encoder can specify a BCW index value as shown in Table 10 below and transmit it to the decoder.

[0271]

[0272] Meanwhile, the decoder can generate one reference block by applying a corresponding weight value (w1) to the reference block in the list_0 direction according to the BCW index value transmitted from the encoder, and applying a weight value (w2) obtained by subtracting the corresponding weight value (w1) from "1" to the reference block in the list_1 direction.

[0273] In contrast, according to an embodiment of the present invention, the decoder may calculate an error value between a template of a target block and a template of the corresponding reference block for each of the bidirectional reference blocks, and determine a weight for each of the bidirectional reference blocks based on the calculated template error value.

[0274] Specifically, the decoder can generate a single reference block through a weighted average of bidirectional reference blocks by assigning a high weight to a reference block with a small template error value and a lower weight to a reference block with a large template error value.

[0275] Referring to FIG. 12, for the current block (Cur. block), a reference block (Ref. block) in the (t-1) reference picture pointed to by the motion vector (mvLO) in the list_0 direction and a reference block (Ref. block) in the (t+1) reference picture pointed to by the motion vector (mvL1) in the list_1 direction can be obtained.

[0276] And for each of the bidirectional reference blocks, an error value can be calculated between the template (Template) around the current block (Cur. block) and the template (Template 1, 2) around the corresponding reference block (Ref. block).

[0277] In this case, a weight of 5 / 8 may be assigned to a reference block with a small calculated error value among bidirectional reference blocks, and a weight of 3 / 8 may be assigned to a reference block with a large error value.

[0278] For example, when the position of an object is moved in pictures as illustrated in Fig. 13, among the templates (Template 1, 2) of the bidirectional reference blocks, the template with a smaller error value with respect to the template of the current block (Cur. block) may be the template (Template 1) in the (t-1) reference picture.

[0279] Accordingly, a weight of 5 / 8 may be assigned to a reference block within the (t-1) reference picture, and a weight of 3 / 8 may be assigned to a reference block within the (t+1) reference picture.

[0280] In this embodiment, bidirectional prediction may use reference pictures in different directions relative to the current picture, such as the (t-1) reference picture and the (t+1) reference picture, or may use different reference pictures in the same direction relative to the current picture, such as the (t-1) reference picture and the (t-2) reference picture. That is, bidirectional prediction in the present invention is not limited to a specific direction, and may include all prediction methods that use two reference pictures.

[0281] Additionally, in the present embodiment, bidirectional prediction may utilize reference blocks pointed to by two different motion vectors within the (t-1) reference picture. That is, bidirectional prediction in the present invention is not limited to two different reference pictures, and may include all prediction methods that utilize two different reference blocks within a single reference picture.

[0282] As described above, by not transmitting an index value (e.g., BCW index) for indicating weights for bidirectional reference blocks, and by having the decoder derive weights for bidirectional reference blocks using template matching, the number of bits for transmitting the index (e.g., BCW index) can be saved.

[0283] Fig. 14 is a flowchart illustrating an embodiment of a method for deriving weights for bidirectional reference blocks using template matching. A description of the same method as described above among the illustrated methods will be omitted.

[0284] Referring to Fig. 14, bidirectional reference blocks are derived using the motion vectors (mvL0, mvL1) of the target block (step S1400).

[0285] The first weight (w1) assigned to the bidirectional reference blocks is set to 5 / 8, and the second weight (w2) is set to 3 / 8 (step S1410).

[0286] Here, the values ​​(5 / 8, 3 / 8) set as the first and second weights (w1, w2) are an example, and the present invention is not limited thereto, and the first weight (w1) may be set to other values ​​so that the second weight (w2) is greater.

[0287] Among the bidirectional reference blocks, a template error value (TM1_error) with respect to the target block is calculated for the first reference block (step S1420), and a template error value (TM2_error) with respect to the target block is calculated for the second reference block (step S1430).

[0288] Next, the template error value (TM1_error) for the first reference block and the template error value (TM2_error) for the second reference block are compared (step S1440).

[0289] As a result of the comparison, if the template error value (TM1_error) for the first reference block is smaller than the template error value (TM2_error) for the second reference block, a first weight (w1) is assigned to the first reference block, and a second weight (w2) is assigned to the second reference block (step S1450).

[0290] Conversely, if the template error value (TM1_error) for the first reference block is not less than the template error value (TM2_error) for the second reference block, a second weight (w2) is assigned to the first reference block, and a first weight (w1) is assigned to the second reference block (step S1460).

[0291] Meanwhile, if the template error value (TM1_error) for the first reference block and the template error value (TM2_error) for the second reference block have similar values ​​within a preset error range, the same weight, i.e., 1 / 2, may be assigned to the first and second reference blocks.

[0292] After that, the weights assigned as above are applied to the bidirectional reference blocks to calculate the average value, thereby generating the final reference block for the target block (step S1470).

[0293] In the above, an embodiment of the present invention was described by taking as an example a case where the encoder does not transmit BCW index values ​​to the decoder, but the number of bits can be saved by reducing the number of BCW indices to two and transmitting the selected index values ​​to the decoder.

[0294]

[0295] In the above, the embodiments of the present invention have been described by taking as an example the template being composed of samples positioned around the target block in an "L" shape, as shown in (a) of FIG. 15, but the present invention is not limited thereto and may be composed of samples of various shapes positioned around the target block.

[0296] For example, referring to (b) of FIG. 15, the template may be composed of an upper template (T_Template) composed of samples located on the upper side of the target block (Block) and a left template (L_Template) composed of samples located on the left side of the target block (Block), excluding the upper left region (LT) in the "L"-shaped template.

[0297] FIG. 16 is a flowchart illustrating an embodiment of a method for setting a reference block using template matching according to the present invention. Among the methods illustrated, descriptions of those identical to those described with reference to FIGS. 1 to 15 will be omitted.

[0298] Referring to FIG. 16, a first template is formed of samples located on the upper side of the target block (step S1600), and a second template is formed of samples located on the left side of the target block (step S1610).

[0299] For example, the first template configured in step S1600 may be the same as the upper template (T_Template) illustrated in (b) of FIG. 15, and the second template configured in step S1610 may be the same as the left template (L_Template) illustrated in (b) of FIG. 15, but the present invention is not limited thereto.

[0300] Next, among the first and second templates configured in steps S1600 and S1610, one or more templates to be used for template matching are determined (step S1620).

[0301] For example, in step S1620, any one of three templates consisting of a first template, a second template, and a combination of the first and second templates may be determined as the template to be used for template matching.

[0302] Meanwhile, at step S1620, a flag indicating whether to use three templates can be checked, and the flag can be included in the bitstream as HLS (High Level Syntax) and located in at least one of SPS (Sequence Parameter Set), PPS (Picture Parameter Set), PH (Picture Peader), and SH (Slice Header).

[0303] Additionally, if three templates are available, an index value indicating which template among the three templates is to be used for template matching can be identified, and the index value can be included in the bitstream in units of blocks at the CU level.

[0304] In this case, in the process of setting a reference block used to generate a prediction block for a target block through template matching as described above, information about templates around the target block and reference block used for template matching can be explicitly signaled from the encoder to the decoder.

[0305] Referring to FIG. 17 and Table 11 below, an index value of "0" may be assigned to a template (Top & Left) that includes both a top template (T_Template) and a left template (L_Template), an index value of "1" may be assigned to a template (Only Top) that includes only a top template (T_Template), and an index value of "2" may be assigned to a template (Only Left) that includes only a left template (L_Template).

[0306]

[0307] For example, if the flag included in the HLS (High Level Syntax) for the target block is "on", the three templates above (Top & Left, Only Top, Only Left) can be used for the target block.

[0308] And the encoder can include an index value indicating the template used for the target block among the three templates (Top & Left, Only Top, Only Left) in the bitstream and transmit it to the decoder.

[0309] That is, only when the flag included in HLS (High Level Syntax) is "on", the index value indicating the template used in the target block can be included in the bitstream.

[0310] As another example, in the process of setting a reference block used to generate a prediction block for a target block through template matching as described above, information about templates around the target block and reference block used for template matching may be implicitly derived from the decoder without being transmitted to the decoder.

[0311] More specifically, the template to be used for template matching can be determined based on at least one of the shape and position of the target block.

[0312] For example, if the shape of the target block is a square, a combination of the first and second templates can be determined as the template to be used for template matching.

[0313] Referring to (a) of Fig. 18, when the width (W) and height (H) of the target block are the same, a template (Top & Left) including both the upper template (T_Template) and the left template (L_Template) can be used.

[0314] Additionally, if the width of the target block is greater than the height, a first template composed of samples located on the upper side of the target block may be determined as the template to be used for template matching.

[0315] Referring to (b) of Fig. 18, when the width (W) of the target block is greater than the height (H) and has a long rectangular shape in the horizontal direction, a template (Only Top) including only the upper template (T_Template) can be used.

[0316] Meanwhile, if the height of the target block is greater than the width, a second template consisting of samples located on the left side of the target block may be determined as the template to be used for template matching.

[0317] Referring to (c) of Fig. 18, when the height (H) of the target block is greater than the width (W) and has a shape of a long rectangle in the vertical direction, a template (Only Left) including only the left template (L_Template) can be used.

[0318] As another example, if the width (W) of the target block is greater than the height (H), a left template (L_Template) composed of samples located on the left side of the target block may be determined as the template to be used for template matching, and if the height (H) of the target block is greater than the width (W), an upper template (T_Template) composed of samples located on the upper side of the target block may be determined as the template to be used for template matching.

[0319] As described with reference to (b) and (c) of FIG. 18, when the shape of the target block is rectangular, using only one of the upper template (T_Template) and the left template (L_Template) can be applied only when the width (W) and height (H) of the target block differ by more than twice, but the present invention is not limited thereto.

[0320]

[0321] Hereinafter, embodiments of a method for configuring a template when a reference block (or target block) is located at the boundary of a corresponding picture (picture boundary) and at least a part of a template of the reference block (or target block) extends beyond the picture boundary, will be described in detail.

[0322] Hereinafter, embodiments of the present invention will be described using the case of a reference block as an example, but the present invention is not limited thereto, and embodiments of the present invention can also be applied to a current block within a current picture.

[0323] Referring to FIG. 19, when a reference block (Ref. block) is located adjacent to the boundary of a reference picture, some areas of a template for the reference block may fall outside the reference picture.

[0324] As a first embodiment for configuring a template when some areas of the template extend beyond the picture boundary, the location of the reference block may be restricted so that the template of the reference block does not extend beyond the picture boundary.

[0325] Specifically, when the template has a width corresponding to n pixels, the location of the reference block (Ref. block) can be restricted so that the reference block (Ref. block) is located n pixels inside from the boundary of the reference picture, as illustrated in FIG. 20.

[0326] For example, if the template width is 4 pixels and the reference picture size is 1920x1080, the upper leftmost sample of the reference block can be positioned from 4 to 1919 in the horizontal direction (from 0 to 1919) and from 4 to 1079 in the vertical direction (from 0 to 1079).

[0327] Since the template is located to the left and above the reference block, the positional constraints for the reference block as described above are applicable not only to the left boundary of the picture but also to the upper boundary.

[0328] As a second embodiment for configuring a template when some areas of the template extend beyond the picture boundary, a template area extending beyond the boundary of the reference picture can be generated by copying pixel values ​​closest to the boundary of the reference picture.

[0329] Referring to (a) of Fig. 21, when the left area of ​​the template of the reference block (Ref. block) goes beyond the boundary of the reference picture, the pixel values ​​of the edge closest to the left boundary of the reference picture can be copied to fill in the pixel values ​​of the left area of ​​the template that goes beyond the picture boundary.

[0330] Also, referring to (b) of FIG. 21, when the upper area of ​​the template of the reference block (Ref. block) goes beyond the boundary of the reference picture, the pixel values ​​of the edge closest to the upper boundary of the reference picture can be copied to fill in the pixel values ​​of the upper area of ​​the template that goes beyond the picture boundary.

[0331] For example, using the mathematical expression 1 below, pixel values ​​at the very edge of the reference picture can be copied as described above to fill in pixel values ​​of the template that are outside the picture boundary.

[0332]

[0333] In mathematical expression 1, x is the location information of a pixel included in the template, and xA is the updated location information for the pixel.

[0334] Here, if x is less than 0, it may mean that the corresponding pixel in the template is outside the picture boundary.

[0335] In this case, if x is less than 0, the position of the corresponding pixel can be updated to the position of 0, so that the 0th pixel value can be copied and used.

[0336] As a third embodiment for configuring a template, if some areas of the template fall outside the picture boundary, the areas of the template that fall outside the picture boundary can be excluded from the template matching process.

[0337] Referring to (a) of Fig. 22, when the left area of ​​the template of the reference block (Ref. block) goes beyond the boundary of the reference picture, template matching can be performed using only the upper template (T_Template).

[0338] Also, referring to (b) of FIG. 22, when the upper area of ​​the template of the reference block is outside the boundary of the reference picture, template matching can be performed using only the left template (L_Template).

[0339] In the above, embodiments of the present invention have been described by taking as an example a case where the template of a reference block goes beyond the boundary of a reference picture, but the present invention is not limited thereto and can also be applied to a case where the template of a block to be encoded goes beyond the boundary of a picture.

[0340] In the present embodiment, in a process of setting a reference block used to generate a prediction block for a target block through template matching, information about templates surrounding the target block and reference block used for template matching may be one of the methods in which the information is implicitly derived from the decoder without being transmitted to the decoder.

[0341] More specifically, a template to be used for template matching can be determined based on at least one of the shape and position of the target block, and the present embodiment is an embodiment corresponding to the position of the target block.

[0342] As shown in (a) of Fig. 22, when the location of the reference block is adjacent to the left border of the picture, only the upper template (T_Template) is used, and as shown in (b) of Fig. 22, when the location of the reference block is adjacent to the upper border of the picture, only the left template (L_Template) is used, and in other locations of the reference block, a template (Top & Left) including both the upper template (T_Template) and the left template (L_Template) can be used.

[0343] As a fourth embodiment for configuring a template when some areas of the template go beyond the picture boundary, the pixel values ​​of the areas of the template that go beyond the picture boundary can be filled by copying them with the pixel values ​​of the corresponding areas located on the opposite boundary of the picture.

[0344] Referring to (a) of Fig. 23, when the left area of ​​the template of the reference block (Ref. block) goes beyond the boundary of the reference picture, the pixel values ​​of the corresponding area (Wr) located at the right boundary of the reference picture can be copied to fill in the pixel values ​​of the left area of ​​the template that goes beyond the picture boundary.

[0345] For example, the left area of ​​the template marked “1234” can be matched with the area (Wr) marked “1234” located at the right boundary of the reference picture, so that pixel values ​​for the left area outside the picture boundary among the templates can be filled in.

[0346] As described above, the method of filling the template area outside the picture boundary using the corresponding area on the opposite boundary can be performed when a specific flag (e.g., ref_wraparound_enabled_flag) is “on”.

[0347] ref_wraparound_enabled_flag is a flag value indicating related processing for ERP (equirectangular projection) among 360 video formats. When the flag value is “on”, the left boundary area of ​​the picture and the right boundary area of ​​the picture may belong to adjacent areas in the 3D real world as a characteristic of 360 video according to the ERP video format. Accordingly, information about the left boundary area of ​​the picture and the right boundary area of ​​the picture may be shared with each other.

[0348] For example, using the mathematical expression 2 below, pixel values ​​of a corresponding area located on the opposite boundary of a reference picture as described above can be copied to fill in pixel values ​​of a template that are outside the picture boundary.

[0349]

[0350] In mathematical expression 2, x is the location information of a pixel included in the template, and xA is the updated location information for the pixel.

[0351] Here, if x is less than 0 or greater than (picWidth-1), it may mean that the corresponding pixel in the template is out of the picture boundary.

[0352] In this case, if x is less than 0, you can update the position of the pixel to the right boundary of the picture by adding picWidth to x, and if x is greater than (picWidth-1), you can update the position of the pixel to the left boundary of the picture by subtracting picWidth from x.

[0353] Meanwhile, as described above, pixel values ​​of the template that are outside the picture boundary may be filled in depending on the location of the updated pixel.

[0354] As illustrated in (b) of Fig. 23, when the left template (L_Template) among the templates (Top & Left template) of the reference block (Ref. block) is outside the boundary of the reference picture, the left template (L_Template) that is outside the picture boundary can be filled using the corresponding area (Wr) located at the right boundary of the reference picture.

[0355] And depending on the ref_wraparound_enabled_flag value, either the method of the second embodiment or the method of the fourth embodiment described above can be used to fill the template area outside the picture boundary.

[0356] Referring to FIG. 24, the ref_wraparound_enabled_flag value is checked (step S2300), and if it is "1", a template is generated by filling a template area outside the picture boundary using a corresponding area located on the opposite side of the picture according to the method of the fourth embodiment described above using mathematical expression 2 (step S2310).

[0357] Meanwhile, if the ref_wraparound_enabled_flag value is not "1", a template is generated by filling the template area outside the picture boundary using pixel values ​​located at the very edge of the picture according to the method of the second embodiment described above using mathematical expression 1 (step S2320).

[0358] As another example, if the ref_wraparound_enabled_flag value is not "1", the method of the second embodiment using mathematical expression 1 may be applied instead of the second embodiment, without being limited to the method of the second embodiment or the method of the third embodiment.

[0359] As a fifth embodiment for configuring a template when some areas of the template go beyond the picture boundary, pixel values ​​of areas of the template that go beyond the sub picture boundary can be copied and filled with pixel values ​​located at the very edge of the corresponding sub picture.

[0360] As described above, the method of filling the template area outside the subpicture boundary using pixel values ​​located at the very edge of the subpicture can be performed when a specific flag (e.g., subpic_treated_as_pic_flag) is “on” while using the subpicture.

[0361] subpic_treated_as_pic_flag can indicate whether the boundary of each subpicture can be processed independently when a picture is divided into multiple subpictures, and if the flag value is “on”, the subpicture can be encoded and decoded independently from other subpictures.

[0362] For example, using the mathematical expression 3 below, pixel values ​​located at the very edge of the subpicture can be copied as described above to fill in pixel values ​​of the template that fall outside the subpicture boundary.

[0363]

[0364] In mathematical expression 3, x is the location information of a pixel included in the template, and xA is the updated location information for the pixel.

[0365] Here, if x is less than subPicLeftBoundaryPos, it may mean that the corresponding pixel in the template is outside the subpicture boundary.

[0366] In this case, if x is less than subPicLeftBoundaryPos, the position of the corresponding pixel can be updated to the position of subPicLeftBoundaryPos, and the value of the subPicLeftBoundaryPos-th pixel can be copied and used.

[0367] As another example, when the subpic_treated_as_pic_flag value is "1", the method of the second embodiment is not limited to filling pixel values ​​of a template outside the subpicture boundary by copying pixel values ​​located at the very edge of the subpicture, and the first embodiment or the third embodiment can also be applied instead of the second embodiment.

[0368] And, depending on the value of at least one of subpic_treated_as_pic_flag and Ref_wraparound_enabled_flag, any one of the methods of the first to fifth embodiments described above may be used to fill the template area outside the picture boundary.

[0369] Referring to FIG. 25, the subpic_treated_as_pic_flag value is checked (step S2400), and if it is "1", the ref_wraparound_enabled_flag value is checked (step S2410).

[0370] As a result of the verification at step S2410, if the ref_wraparound_enabled_flag value is "1", a template is generated by filling the template area outside the picture boundary using the corresponding area located on the opposite side of the picture according to the method of the fourth embodiment described above using mathematical expression 2 (step S2420).

[0371] As a result of the verification at step S2410, if the ref_wraparound_enabled_flag value is not "1", a template is generated by filling the template area outside the sub-picture boundary using pixel values ​​located at the very edge of the sub-picture according to the method of the fifth embodiment described above using mathematical expression 3 (step S2430). Here, the fifth embodiment can also be applied to the first embodiment or the third embodiment instead of the second embodiment.

[0372] Meanwhile, if the subpic_treated_as_pic_flag value is not "1", the ref_wraparound_enabled_flag value is checked (step S2440), and if the ref_wraparound_enabled_flag value is "1", a template is generated by filling in a template area outside the picture boundary using a corresponding area located on the opposite side of the picture according to the method of the fourth embodiment described above using mathematical expression 2 (step S2420).

[0373] As a result of the verification at step S2440, if the ref_wraparound_enabled_flag value is not "1", a template is generated by filling the template area outside the picture boundary using pixel values ​​located at the very edge of the picture according to the method of the second embodiment described above using mathematical expression 1 (step S2450). Here, the first embodiment or the third embodiment may also be applied instead of the second embodiment.

[0374] Referring to FIG. 26, the ref_wraparound_enabled_flag value is first checked (step S2500), and if it is "1", a template is generated by filling a template area outside the picture boundary using a corresponding area located on the opposite side of the picture according to the method of the fourth embodiment described above using mathematical expression 2 (step S2510).

[0375] As a result of the verification at step S2500, if the ref_wraparound_enabled_flag value is not "1", the subpic_treated_as_pic_flag value is checked (step S2520), and if it is "1", the template area outside the subpicture boundary is filled using pixel values ​​located at the very edge of the subpicture according to the method of the fifth embodiment described above using mathematical expression 3 to generate a template (step S2530). Here, the fifth embodiment can be applied to the first embodiment or the third embodiment instead of the second embodiment.

[0376] Meanwhile, if neither the ref_wraparound_enabled_flag value nor the subpic_treated_as_pic_flag value is "1", a template is generated by filling the template area outside the picture boundary using pixel values ​​located at the very edge of the picture according to the method of the second embodiment described above using mathematical expression 1 (step S2540). Here, the first embodiment or the third embodiment may also be applied instead of the second embodiment.

[0377]

[0378] The methods described herein may be performed by a processor of a video encoder or video decoder. Furthermore, the encoder may generate a bitstream that is decoded by a video signal processing method, and the bitstream generated by the encoder may be stored in a computer-readable, non-transitory storage medium (recording medium).

[0379] The embodiments of the present invention described above may be implemented through various means. For example, the embodiments of the present invention may be implemented using hardware, firmware, software, or a combination thereof.

[0380] Some embodiments may also be implemented in the form of a computer-executable storage medium containing computer-executable instructions, such as program modules executed by a computer. Computer-readable media may be any available media that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media.

[0381] Additionally, computer-readable media may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. Communication media typically includes other data, such as computer-readable instructions, data structures, or program modules, in a modulated data signal, or other transport mechanism, and includes any information delivery media.

[0382] In addition, although the preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and various modifications can be made by a person having ordinary skill in the art to which the invention pertains without departing from the gist of the present invention as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present invention.

Claims

1. A method for setting a reference block for generating a prediction block using template matching, A step of forming a first template consisting of samples positioned on the upper side of the target block; A step of forming a second template consisting of samples located on the left side of the target block; A step of determining one or more templates to be used for template matching among the first and second templates; and A method for setting a reference block using template matching, comprising: a step of performing template matching using one or more of the determined templates to obtain a reference block for the target block; 2. In paragraph 1, the step of determining the template is A method for setting a reference block using template matching, wherein one of three templates consisting of the first template, the second template, and a combination of the first and second templates is determined as a template to be used for the template matching.

3. In the second paragraph, the step of determining the template is A method for setting a reference block using template matching, comprising: a step of checking a flag indicating whether to use the above three templates; 4. In the third paragraph, the flag A method for setting a reference block using template matching included in at least one of SPS (Sequence Parameter Set), PPS (Picture Parameter Set), PH (Picture Peader), and SH (Slice Header).

5. In paragraph 2, A method for setting a reference block using template matching, further comprising: a step of checking an index value indicating a template to be used for template matching among the three templates mentioned above.

6. In the first paragraph, the step of determining the template is A method for setting a reference block using template matching, wherein a template to be used for template matching is determined based on at least one of the shape and position of the target block.

7. In paragraph 6, A method for setting a reference block using template matching, wherein a combination of the first and second templates is determined as a template to be used for template matching when the shape of the target block is a square.

8. In paragraph 6, If the width of the target block is greater than the height, the first template is determined as the template to be used for template matching, A method for setting a reference block using template matching, wherein the second template is determined as a template to be used for template matching when the height of the target block is greater than the width.

9. An image encoding device that performs a reference block setting method using template matching of Article 1.

10. In paragraph 9, A video encoding device that inserts an index value for indicating one or more templates to be used for template matching among the first and second templates into at least one of a Sequence Parameter Set (SPS), a Picture Parameter Set (PPS), a Picture Header (PH), and a Slice Header (SH), and transmits the same as a bitstream.

11. A bitstream generated by the video encoding device of Article 9.

12. An image decoding device that performs a reference block setting method using template matching of Article 1.

13. In paragraph 12, An image decoding device that determines a template to be used for template matching based on at least one of the shape and position of the target block.

Images