Method for processing picture boundary region, and video encoding and decoding apparatus performing same

Abstract

The present invention relates to a method for processing a picture boundary region and a video encoding and decoding apparatus for performing same. The method comprises the steps of: when at least a part of a peripheral region of a first reference block used to perform prediction for a target block is located outside a picture boundary, obtaining first vector information for the first reference block; identifying a second reference block referenced by the first reference block according to the first vector information; and configuring the peripheral region of the first reference block located outside the picture boundary by using peripheral samples of the second reference block.

Description

Method for processing a picture boundary region, and an image encoding and decoding device for performing the same

[0001] The present invention relates to a method for processing a picture boundary area when an area used for prediction exceeds the picture boundary when applying template matching or motion compensation, etc.

[0002] Compression encoding of video signals is achieved by removing redundant information by considering spatial correlation, temporal correlation, and probabilistic correlation.

[0003] Among the prediction methods used in image encoding, intra prediction is a method of predicting the current block by referring to samples that have already been reconstructed around the block to be encoded.

[0004] Recently, due to the development of various media and data transmission media, user demand for high-resolution, high-quality images is increasing.

[0005] Since the encoded data of a high-resolution image contains more information than the encoded data of a low-resolution or medium-resolution image, the cost of transmitting or storing it increases.

[0006] To solve this problem, research is ongoing on encoding and decoding methods to effectively reduce the amount of encoding data for high-resolution images.

[0007] The purpose of the present invention is to provide a picture boundary area processing method capable of improving image compression performance, and an image encoding and decoding device for performing the same.

[0008] A method for processing a picture boundary area according to an embodiment of the present invention is a method for processing an area outside the picture boundary, the method comprising: when at least a part of a surrounding area of ​​a first reference block used for performing prediction on a target block is located outside the picture boundary, obtaining first vector information for the first reference block; identifying a second reference block referenced by the first reference block according to the first vector information; and constructing a surrounding area of ​​the first reference block located outside the picture boundary using surrounding samples of the second reference block.

[0009] The image encoding device according to one embodiment of the present invention performs the picture boundary area processing method, and inserts a flag (falg) for indicating the picture boundary area processing method into at least one of SPS (Sequence Parameter Set), PPS (Picture Parameter Set), PH (Picture Header), and SH (Slice Header) and transmits it as a bitstream.

[0010] The image decoding device according to one embodiment of the present invention performs the picture boundary area processing method.

[0011] Meanwhile, at least some steps of the above methods may be implemented as a computer-readable recording medium recording a program for execution on a computer, or may be provided as a program itself.

[0012] According to one embodiment of the present invention, by constructing an area located outside the picture boundary using surrounding samples of a reference block during a template matching or motion compensation process, prediction for a target block can be performed regardless of the location of the target block or the reference block.

[0013] Figure 1 is a block diagram illustrating the configuration and operation of a video encoder for encoding an image.

[0014] Figure 2 is a drawing for explaining an embodiment of a method for dividing blocks of an image.

[0015] Figures 3 and 4 are drawings for explaining embodiments of an intra prediction method.

[0016] Figure 5 is a diagram for explaining embodiments of an inter prediction method.

[0017] Figure 6 is a block diagram illustrating the configuration and operation of a video decoder for decoding an image.

[0018] FIG. 7 is a diagram illustrating an example of an intra prediction method in IBC (Intra Block Copy) mode.

[0019] FIG. 8 is a diagram illustrating an example of a method for performing intra prediction in IBC mode using template matching.

[0020] FIGS. 9 to 14 are drawings for explaining embodiments of a method for performing inter prediction using template matching.

[0021] Figure 15 is a drawing for explaining examples of the configuration of a template used for template matching.

[0022] Fig. 16 is a flowchart illustrating a method for setting a reference block using template matching according to one embodiment of the present invention.

[0023] FIGS. 17 and 18 are drawings for explaining embodiments of a method for determining a template used for template matching.

[0024] Figures 19 and 20 are drawings for explaining embodiments for cases where the area used for prediction goes beyond the picture boundary.

[0025] FIG. 21 is a drawing for explaining a first embodiment of a method for processing a picture boundary area.

[0026] Figures 22 to 25 are drawings for explaining a second embodiment of a method for processing a picture boundary area.

[0027] Figure 26 is a drawing for explaining a third embodiment of a method for processing a picture boundary area.

[0028] FIGS. 27 to 32 are flowcharts illustrating embodiments of a method of mixing and using two or more embodiments of a method of processing a picture boundary area.

[0029] Hereinafter, with reference to the attached drawings, a method and device for encoding and decoding an image according to an embodiment of the present invention will be described in detail.

[0030] In the following description of the present invention, detailed descriptions of related known functions or configurations will be omitted if they are deemed to unnecessarily obscure the gist of the invention. Furthermore, the terms described below are defined in light of their functions within the present invention and may vary depending on the intent or custom of the user or operator. Therefore, their definitions should be based on the overall content of the present invention.

[0031] In addition, in order to efficiently explain the technical components that make up the present invention, the preferred embodiments of the present invention implemented below omit, as much as possible, the system functional components that are already provided in each system functional configuration or are commonly provided in the technical field to which the present invention belongs, and focus on explaining the functional components that must be additionally provided for the present invention.

[0032] Anyone having ordinary skill in the art to which the present invention pertains will be able to easily understand the functions of components that have been conventionally used among the functional configurations that are not illustrated below, and will also be able to clearly understand the relationship between the components omitted as described above and the components added for the present invention.

[0033] In this specification, a device that encodes an image to generate a video signal bitstream is referred to as an encoding device, an encoding device, or an encoder, and a device that decodes the video signal bitstream to restore an image is referred to as a decoding device, a decoding device, or a decoder.

[0034] A pixel or pel is the smallest unit that constitutes an image, and the terms pixel and sample can be used interchangeably. A sample can generally represent a pixel or a pixel value, or it can represent only a pixel or pixel value of the luminance component, or only a pixel or pixel value of the chroma component.

[0035] In addition, the unit is used to refer to a basic unit of image processing or a specific location of a picture, and represents an image area including at least one of a luminance component and a chrominance component. Specifically, the unit can be used as a concept including a coding tree unit (CTU), a coding unit (CU), a prediction unit (PU), and a transform unit (TU). In addition, a block represents an image area including a specific component among luminance components and chrominance components, and an MxN block can represent a set of samples or transform coefficients consisting of M columns and N rows. Here, the terms unit, block, partition, signal, and area may be used interchangeably.

[0036] Meanwhile, a picture refers to a field or a frame, and these can be used interchangeably. For example, if the image is an interlaced image, one frame is divided into an odd (or odd, top) field and an even (or even, bottom) field, and each field is composed of a single picture unit, which can be encoded or decoded. If the image is a progressive image, one frame can be composed as a picture, which can be encoded or decoded.

[0037]

[0038] FIG. 1 is a block diagram illustrating an encoding device according to an embodiment of the present invention, and is intended to explain the configuration and operation of a video encoder for encoding an image.

[0039] Referring to FIG. 1, a video encoder (100) may be configured to include a transformation unit (110), a quantization unit (120), an inverse quantization unit (130), an inverse transformation unit (140), a filtering unit (150), a prediction unit (160), a DPB (Decoded Picture Buffer, 170), and an entropy coding unit (180).

[0040] The conversion unit (110) converts the residual signal, which is the difference between the input video signal and the prediction signal generated by the prediction unit (160), to obtain a conversion coefficient value.

[0041] For example, the Discrete Cosine Transform (DCT), the Discrete Sine Transform (DST), or the Wavelet Transform can be used.

[0042] The transform kernel used for transforming the residual block may be a transform kernel having separable vertical and horizontal transform properties. In this case, the transform for the residual block may be performed separately as vertical and horizontal transforms. For example, the encoder may perform a vertical transform by applying the transform kernel in the vertical direction of the residual block. Alternatively, the encoder may perform a horizontal transform by applying the transform kernel in the horizontal direction of the residual block.

[0043] Meanwhile, the transform kernel may be used as a term referring to a set of parameters used for transforming the residual signal, such as a transform matrix, a transform array, a transform function, or a transform, and may be any one of a plurality of available kernels, and transform kernels based on different transform types may be used for each of the vertical transform and the horizontal transform.

[0044] The transformation coefficients may be distributed such that higher coefficients are distributed toward the upper left corner of the block, and coefficients closer to '0' are distributed toward the lower right corner of the block. Furthermore, as the current block size increases, there is a possibility that many '0' coefficients exist in the lower right area. To reduce the transformation complexity of large blocks, only the upper left area can be left, and the remaining areas can be reset to '0'.

[0045] Additionally, error signals may exist only in some regions of a coding block, in which case the conversion process may be performed only on some arbitrary regions. For example, in a block of size 2Nx2N, error signals may exist only in the first 2NxN block, in which case the conversion process may be performed only on the first 2NxN block, but the conversion process may not be performed on the second 2NxN block and may not be encoded or decoded.

[0046] The encoder may perform additional transformations before the transform coefficients are quantized. The transformation method described above may be referred to as a primary transform, and the additional transformation may be referred to as a secondary transform.

[0047] The secondary transform can be optional for each residual block, for example, the encoder can improve coding efficiency by performing the secondary transform for areas where it is difficult to concentrate energy in the low-frequency region using only the primary transform.

[0048] Specifically, a secondary transformation may be additionally performed on blocks in which residual values ​​appear significantly in directions other than the horizontal or vertical direction of the residual block, and the secondary transformation may not be performed separately into vertical transformation and horizontal transformation, unlike the primary transformation. The secondary transformation as described above may be referred to as a low frequency non-separable transform (LFNST).

[0049] The quantization unit (120) quantizes the transformation coefficient value output from the transformation unit (110).

[0050] In order to increase coding efficiency, a method is used in which, instead of coding the picture signal as is, a picture is predicted using an already coded area through a prediction unit (160), and a restored picture is obtained by adding a residual value between the original picture and the predicted picture to the predicted picture.

[0051] To avoid mismatches between the encoder and decoder, when performing prediction in the encoder, information available in the decoder must also be used, and for this purpose, the encoder can perform a process of reconstructing the current block that it has encoded.

[0052] The inverse quantization unit (130) inversely quantizes the transform coefficient values, and the inverse transformation unit (140) restores the residual values ​​using the inverse quantized transform coefficient values.

[0053] The filtering unit (150) performs filtering operations using a deblocking filter, a sample adaptive offset (SAO), an adaptive loop filter (ALF), etc. to improve the quality of the restored picture and enhance encoding efficiency.

[0054] A deblocking filter is a filter for removing distortion within a block generated at the boundary between blocks in a restored picture. The encoder can determine whether to apply a deblocking filter to a boundary based on the distribution of pixels included in several columns or rows based on an arbitrary boundary within a block.

[0055] When a deblocking filter is applied, the filtering unit (150) can apply a long filter, a strong filter, or a weak filter depending on the deblocking filtering strength, and can process horizontal filtering and vertical filtering in parallel.

[0056] Sample Adaptive Offset (SAO) can be used to correct the offset from the original image on a pixel-by-pixel basis for a residual block to which a deblocking filter is applied. In order to correct the offset for a specific picture, the filtering unit (150) can use a method (Band Offset) that divides the pixels included in the image into a certain number of regions, determines the regions to perform offset correction, and applies the offset to the regions. In addition, the filtering unit (150) can use a method (Edge Offset) that applies the offset by considering the edge information of each pixel.

[0057] Adaptive Loop Filtering (ALF) is a method that divides pixels in an image into predetermined groups, determines a filter to be applied to each group, and performs differential filtering for each group. Information regarding whether to apply an adaptive loop filter can be signaled on a coding unit basis, and the shape and filter coefficients of the ALF filter to be applied can vary depending on the block. Furthermore, the same adaptive loop filter can be applied regardless of the characteristics of the target block.

[0058] The filtered picture can be stored in DPB (170) to be used as a reference picture.

[0059] The prediction unit (160) includes an intra / IBC prediction unit (161) and an inter prediction unit (165). The intra / IBC prediction unit (161) performs intra prediction and IBC prediction within the current picture, and the inter prediction unit (165) performs inter prediction to predict the current picture using a reference picture stored in the DPB (170).

[0060] The intra / IBC prediction unit (161) performs intra prediction from restored areas within the current picture and transmits intra encoding information to the entropy coding unit (180). Here, the intra encoding information may include at least one of an intra prediction mode, an MPM (Most Probable Mode) flag, an MPM index, and information about a reference sample.

[0061] Additionally, the intra / IBC prediction unit (161) can perform IBC (Intra BLock Copy) prediction from restored samples within the current picture and transmit IBC encoding information to the entropy coding unit (180).

[0062] In this case, the intra / IBC prediction unit (161) obtains a block vector value indicating a reference area used for prediction of the current area by referring to a specific area within the current picture, and can perform IBC prediction using the obtained block vector value.

[0063] The intra / IBC prediction unit (161) can transmit IBC encoding information including at least one of the size information of the reference area and block vector information (index information for block vector prediction of the current block within the motion candidate list, block vector difference information) to the entropy coding unit (160).

[0064] The inter prediction unit (165) refers to a specific area of ​​the restored reference picture to find the part most similar to the current area, obtains a motion vector value which is the distance between the areas, and transmits motion information (reference direction indication information (L0 prediction, L1 prediction, bidirectional prediction), reference picture index, motion vector information, etc.) for the obtained reference area to the entropy coding unit (180).

[0065] Additionally, the inter prediction unit (165) performs motion compensation using motion information to generate a prediction block for the current block, and transmits inter encoding information including motion information for the reference area to the entropy coding unit (180).

[0066] Meanwhile, the quantized transform coefficients in the form of a two-dimensional array can be rearranged into a one-dimensional array for entropy coding.

[0067] The method of scanning the quantized transform coefficients can be determined according to the size of the transform block and the intra prediction mode, and diagonal, vertical, and horizontal scans can be applied, and the scan information can be signaled on a block-by-block basis or derived from the decoder according to a set rule.

[0068] The entropy coding unit (180) generates a bitstream by entropy coding information representing quantized transform coefficients, intra-coding information, and inter-coding information, and for this purpose, a variable length coding (VLC) method and an arithmetic coding method can be used.

[0069] Variable-length coding (VLC) converts input symbols into a series of codewords, each of which can be of variable length. For example, frequently occurring symbols can be represented by shorter codewords, while less frequently occurring symbols can be represented by longer codewords.

[0070] As a variable length coding method, a context-based adaptive variable length coding (CAVLC) method can be used.

[0071] Arithmetic coding converts consecutive data symbols into a single prime number using the probability distribution of each data symbol, thereby obtaining the optimal prime number bits required to express each symbol.

[0072] As an arithmetic coding method, the context-based adaptive binary arithmetic code (CABAC) method can be used.

[0073] CABAC is a binary arithmetic encoding method that utilizes multiple context models generated based on experimentally derived probabilities. The context models can also be referred to as context models. First, if the symbols are not in binary form, the encoder binarizes each symbol using methods such as exp-Golomb. The binarized 0s and 1s can be represented as bins.

[0074] The CABAC initialization process is divided into context initialization and arithmetic coding initialization. Context initialization initializes the occurrence probability of each symbol, determined by the symbol type, quantization parameter (QP), and slice type (I, P, B). A context model with this initialization information can use probability-based values ​​obtained through experiments.

[0075] The context model provides the probability of occurrence of the Least Probable Symbol (LPS) or Most Probable Symbol (MPS) for the symbol currently being coded, and information (valMPS) on which empty value between 0 and 1 corresponds to the MPS.

[0076] One of several context models is selected through a context index (ctxIdx), and the context index can be derived through information of the block currently to be encoded or information of surrounding blocks.

[0077] Initialization for binary arithmetic coding is performed based on the probability model selected from the context model. Binary arithmetic coding is performed by dividing the data into probability intervals based on the occurrence probabilities of 0 and 1, and then encoding the probability interval corresponding to the bin to be processed becomes the entire probability interval for the bin to be processed next.

[0078] The location information within the probability interval processed for the last bin is output. However, since the probability interval cannot be infinitely divided, if it shrinks to a certain size, a renormalization process is performed to expand the probability interval and output the corresponding location information. Furthermore, after each bin is processed, a probability update process can be performed, which sets a new probability for the next bin to be processed based on the information from the processed bin.

[0079] As described above, the generated bitstream is encapsulated into NAL (Network Abstraction Layer) units as basic units.

[0080] NAL units are divided into VCL (Video Coding Layer) NAL units that contain video data and non-VCL NAL units that contain parameter information for decoding video data, and there may be various types of VCL or non-VCL NAL units.

[0081] A NAL unit consists of NAL header information and data, a Raw Byte Sequence Payload (RBSP). The NAL header information includes summary information about the RBSP. The RBSP of a VCL NAL unit contains an integer number of encoded coding tree units.

[0082] In order to decode a bitstream in a decoder, the bitstream must first be divided into NAL units, and then each divided NAL unit must be decoded. Meanwhile, the information required for decoding the bitstream can be transmitted as part of a picture parameter set (PPS), a sequence parameter set (SPS), a video parameter set (VPS), etc.

[0083] Meanwhile, the configuration and operation of the encoder described with reference to FIG. 1 are according to one embodiment of the present invention, and some configurations may be omitted or added as needed.

[0084]

[0085] Additionally, a single picture may be encoded by dividing it into sub-pictures, slices, tiles, etc. A sub-picture may include one or more slices or tiles. When a single picture is encoded by dividing it into multiple slices or tiles, all slices or tiles within the picture must be decoded before it can be displayed on the screen.

[0086] When a single picture is encoded into multiple subpictures, only any subpicture can be decoded and displayed on the screen. A slice can contain multiple tiles or subpictures, and a tile can contain multiple subpictures or slices.

[0087] Subpictures, slices, and tiles can be encoded or decoded independently of each other, which is effective for parallel processing and processing speed improvement, but the amount of bits may increase because the encoded information of adjacent subpictures, slices, and tiles cannot be used.

[0088] And subpictures, slices, and tiles can be encoded by being divided into multiple coding tree units (CTUs).

[0089] A coding tree unit can be composed of a luminance coding tree block (CTB) of size 128x128 and two chrominance coding tree blocks of size 64x64.

[0090] A single coding tree unit may be undivided and constitute a single coding unit (CU) itself, or may be divided into multiple coding units as illustrated in Fig. 2. A coding unit may be composed of a luminance coding block (CB) and two chrominance coding blocks.

[0091] A coding unit may consist of one transform unit (TU), or may be split into multiple transform units. A transform unit may consist of a luminance transform block (TB) and two chrominance transform blocks.

[0092] Here, a coding unit represents a basic unit for processing a picture in the process of prediction, transformation, quantization, entropy coding, and decoding, and the size and shape of a coding unit within a picture may not be constant.

[0093] A coding unit may have a square or non-square shape, and a rectangular coding unit may include a vertical coding unit whose height is greater than its width and a horizontal coding unit whose width is greater than its height.

[0094] The coding tree unit is first partitioned into a Quad Tree (QT) structure, such that a single node of size 2NX2N can be partitioned into four nodes of size NXN. Furthermore, the Quad Tree partitioning can be performed recursively, and not all nodes need to be partitioned to the same depth.

[0095] Leaf nodes of a quadtree can be further partitioned into a multi-type tree (MTT) structure. For example, in a multi-type tree structure, a single node can be partitioned into a binary or ternary tree structure with horizontal or vertical partitioning. Accordingly, a multi-type tree structure can have four partitioning structures: vertical binary partitioning, horizontal binary partitioning, vertical ternary partitioning, and horizontal ternary partitioning.

[0096] In each tree structure, both the width and height of a node can have a power of 2. For example, in a binary tree (BT) structure, a node of size 2NX2N can be split into two NX2N nodes by vertical binary splitting, and into two 2NXN nodes by horizontal binary splitting.

[0097] Also, in the Ternary Tree (TT) structure, a node of size 2NX2N can be split into nodes of size (N / 2)X2N, NX2N, and (N / 2)X2N by vertical ternary splitting, and into nodes of size 2NX(N / 2), 2NXN, and 2NX(N / 2) by horizontal ternary splitting. This multi-type tree splitting can be performed recursively.

[0098] A leaf node of a multi-type tree can be a coding unit. If the coding unit is no larger than the maximum transformation length, the coding unit can be used as a unit for prediction and transformation without further splitting. On the other hand, if the width or height of the coding unit is larger than the maximum transformation length, the coding unit can be split into multiple transformation units without explicit signaling regarding the splitting.

[0099] The tree partitioning structure as described above may have the same shape (Single Tree) for the luminance block and the chrominance block, or different shapes (Dual Tree) for the luminance block and the chrominance block.

[0100] Meanwhile, the block division from the coding tree unit (CTU) to the coding unit (CU) as described above can be performed through a rate-distortion optimization (RDO) process, whereby the division structure with the smallest rate-distortion cost (RD cost) value within the allowable size and depth conditions is selected to determine the final coding unit.

[0101]

[0102] Hereinafter, embodiments of the intra prediction method will be described in more detail with reference to FIGS. 3 and 4.

[0103] For intra prediction, intra prediction mode information indicating an intra prediction direction may be signaled, and the intra prediction mode information indicates any one of a plurality of intra prediction modes constituting an intra prediction mode set.

[0104] As illustrated in FIG. 3, the intra prediction mode set may include a planar mode, a DC mode, and 65 directional modes, and each intra prediction mode may be indicated through an intra prediction mode index.

[0105] For example, intra prediction mode index "0" indicates planar mode, intra prediction mode index "1" indicates DC mode, and intra prediction mode indexes "2" to "66" can indicate different directional modes, respectively.

[0106] The directional modes each indicate different angles within a preset angular range, for example, a directional mode can indicate an angle within an angular range from 45 degrees to -135 degrees clockwise.

[0107] In this case, the intra prediction mode index "2" may indicate the Horizontal Diagonal (HDIA) mode, the intra prediction mode index "18" may indicate the Horizontal (HOR) mode, the intra prediction mode index "34" may indicate the Diagonal (DIA) mode, the intra prediction mode index "50" may indicate the Vertical (VER) mode, and the intra prediction mode index "66" may indicate the Vertical Diagonal (VDIA) mode.

[0108] If the current block is a non-square block, 20 additional wide angular modes can be used, indicating angles greater than 45 degrees clockwise or less than -135 degrees.

[0109] Based on the intra prediction mode information as described above, reference samples to be used for intra prediction for the current block are determined.

[0110] For example, if the intra prediction mode index indicates a specific directional mode, the reference sample corresponding to that angle from the current sample of the current block is used for prediction for the current sample.

[0111] As described above, for intra prediction, surrounding already restored samples are used as reference samples, and the reference samples may be restored samples located to the left or above the current block.

[0112] Referring to FIG. 4, the reference samples may be samples adjacent to the left boundary and upper boundary of the current block.

[0113] For example, if the size of the current block is NxN and samples of a single reference line adjacent to the current block are used for intra prediction, reference samples can be set using (2N*2+1) surrounding samples located on the left (L, Left), top (T, Top), and top-left (TL, Top-left) of the current block.

[0114] Meanwhile, samples of multiple reference lines (MRL) may be used for intra prediction of the current block, and the multiple reference lines may be composed of n reference lines located within a preset range from the current block. In this case, separate reference line index information indicating the reference lines to be set as reference pixels may be signaled.

[0115] Additionally, if at least some of the samples to be used as reference samples have not yet been restored, reference samples can be obtained through a reference sample padding process, and a reference sample filtering process can be performed to reduce errors in intra prediction.

[0116]

[0117] Hereinafter, with reference to FIG. 5, embodiments of the inter prediction method will be described in more detail.

[0118] Referring to FIG. 5, inter prediction is a prediction method that obtains prediction information for the current block, which is a block to be encoded, from a reference block located within a reference picture, and uses a motion vector (mv) indicating the location of the reference block.

[0119] Motion information used for inter prediction may include reference direction indication information (inter_pred_idc), reference picture indexes (ref_idx_l0, ref_idx_l1), and motion vectors (mvL0, mvL1).

[0120] Meanwhile, for unidirectional prediction using L0 reference pictures, predFlagL0=1, predFlagL1=0 are set, for unidirectional prediction using L1 reference pictures, predFlagL0=0, predFlagL1=1 are set, and for bidirectional prediction using both L0 and L1 reference pictures, predFlagL0=1, predFlagL1=1 can be set.

[0121] If the current block is a coding unit (CU), the coding unit (CU) can be divided into multiple sub-blocks, the motion information for each sub-block can be the same or different, and each sub-block can be encoded or decoded independently of each other.

[0122] And since the motion vector (mv) of the current block is likely to be similar to the motion vector of the surrounding blocks, the motion vector of the surrounding blocks can be used as a motion information prediction value (MVP, Motion Vector Predictor), and in this case, the motion vector of the current block can be derived using the motion vector of the surrounding blocks.

[0123] Additionally, to improve the accuracy of the motion vector, the difference (MVD, Motion Vector Difference) between the optimal motion vector for the current block found in the original image by the encoder and the motion vector prediction value can be signaled.

[0124] Motion vectors can have various resolutions, and the resolution of motion vectors can vary on a block-by-block basis. Motion vector resolution can be expressed in integer units, half-pixel units, quarter-pixel units, 1 / 16-pixel units, and integer-to-4 pixel units.

[0125] Here, whether to selectively apply motion vector resolution on a block-by-block basis is signaled with amvr_flag, and which motion vector resolution to apply to the current block can be signaled with amvr_precision_idx.

[0126] When bidirectional prediction is applied, the weights for the two prediction blocks can be the same or different when applying weighted averaging, and information about the weights can be signaled via bcw_idx.

[0127] And to increase the accuracy of the prediction value for motion information, Merge mode or AMVP (Advanced Motion Vector Prediction) mode can be selectively used on a block-by-block basis.

[0128] Merge mode is a method of configuring the motion information of the current block to be identical to the motion information of the surrounding blocks adjacent to the current block, and has the advantage of increasing the encoding efficiency of motion information by spatially propagating the motion information without change in a motion area having homogeneity.

[0129] Meanwhile, AMVP mode is a method of signaling the most optimal motion information by predicting motion information in the L0 and L1 prediction directions, respectively, to express more accurate motion information.

[0130] As a method for deriving motion information in merge mode or AMVP mode, a motion candidate list is constructed using predicted values ​​of motion information derived from surrounding blocks of the current block, and index information for the optimal motion candidate can be signaled.

[0131] For example, in merge mode, one merge index (merge_idx) may be signaled, and in AMVP mode, motion candidate indices (mvp_l0_flag, mvp_l1_flag) for L0 and L1, respectively, may be signaled.

[0132] Meanwhile, the motion candidate list derived for one coding unit may be diverse, and a merge index or motion candidate index may be signaled for each motion candidate list.

[0133] To construct a motion candidate list in inter prediction, the surrounding blocks spatially adjacent to the current block may include at least one of a Left block, a Left Below block, an Above block, an Above Right block, and an Above Left block.

[0134] Additionally, a temporally adjacent neighboring block to the current block may be a block that contains the upper left pixel position of the bottom right block of the current block in the corresponding picture (collocated picture).

[0135] Here, if a temporally adjacent neighboring block to the current block is encoded in intra mode, or if a temporally adjacent neighboring block to the current block exists in an unusable position, a block containing the horizontal and vertical center pixel position of the current block in a collocated picture corresponding to the current picture can be used as a temporal neighboring block.

[0136] The motion candidate information derived from the corresponding picture (collocated picture) as above can be referred to as TMVP (Temporal Motion Vector Predictor), and sbTMVP (sub-block Temporal Motion Vector Predictor), which is a sub-block unit TMVP derivation method in which TMVP candidates are derived for each of the sub-blocks into which the current block is divided, can also be used.

[0137] Meanwhile, in Merge Skip mode, information about residual blocks may not exist for blocks encoded in Merge mode.

[0138] Additionally, in AMVP-merge mode, the merge mode and AMVP mode can be used interchangeably for bidirectional motion information of the current block, for example, motion information in the L0 direction can be derived using the AMVP mode, and motion information in the L1 direction can be derived using the merge mode.

[0139] For bidirectional prediction, the SMVD (Symmetric MVD) mode can be used to reduce the amount of motion information transmitted by making the MVD values ​​for the L0 direction and L1 direction symmetrical to each other.

[0140] In this case, MVD information for the L1 direction, which is symmetrical to the L0 direction, is not transmitted, and reference picture information for the L0 and L1 directions is also not transmitted and can be derived during the decoding process.

[0141] OBMC (Overlapped Block Motion Compensation) is a method that generates prediction blocks for the current block using motion information from surrounding blocks when the motion information between blocks differs, and then weights and averages the prediction blocks to generate a final prediction block for the current block. In this case, blocking artifacts occurring at block boundaries in motion-compensated images can be reduced.

[0142] Meanwhile, MMVD (Merge mode with MVD) mode can be used to derive more accurate motion vectors by adding motion vector difference (MVD) to the motion vector derived in merge mode.

[0143] For example, the motion vector of one of the first two candidates included in the candidate list of the merge mode may be used as the initial motion vector, and the final motion vector may be obtained by adding the MVD derived from the MVD direction and motion distance information to the initial motion vector.

[0144] DMVR (Decoder-side Motion Vector Refinement) is a method of correcting motion information through the correlation of already restored reference images to find more accurate motion information. By using the bidirectional motion information of the current block, the best matching point between reference blocks within an arbitrary set area of ​​two reference pictures can be used as new bidirectional motion information.

[0145] Meanwhile, in the above DMVR, MP-DMVR (Multi-pass DMVR) may be used, in which the encoder performs DMVR on one block to correct motion information, then divides the block into sub-blocks and performs DMVR on each sub-block to correct motion information of the sub-block.

[0146] Additionally, LIC (Local Illumination Compensation) is a method for compensating for luminance changes between blocks. It derives a linear model using surrounding pixels adjacent to the current block, and then compensates for the luminance information of the current block through the linear model.

[0147] Additionally, to express movements such as zoom in, zoom out, and rotation, an Affine model-based motion prediction method using a four (rotation) or six (zoom in, zoom out, rotation) parameter model can be used.

[0148] For example, in the case of BDOF (Bi-Directional Optical Flow), the amount of pixel change is estimated based on optical flow from a reference block of a block composed of bidirectional motion, and can be used to correct the prediction block.

[0149] PROF (Prediction refinement with optical flow) is intended to improve the accuracy of sub-block unit affine motion prediction to be similar to the accuracy of pixel unit motion prediction. Similar to BDOF, it calculates a correction value on a pixel-by-pixel basis for pixel values ​​that have been affine motion compensated on a sub-block-by-subblock basis based on optical flow, thereby obtaining a final prediction signal.

[0150] Meanwhile, in the case of CIIP (Combined Inter- / Intra-picture Prediction), when generating a prediction block for the current block, the final prediction block can be generated by weighting and averaging the prediction blocks generated by the intra-picture prediction method and the inter-picture prediction method.

[0151] And BCW (Bi-prediction with CU-level Weights) is a method of performing weighted averaging by applying weights on a block-by-block basis for two motion-compensated prediction blocks from different reference pictures.

[0152] Additionally, multi-hypothesis prediction (MHP) can be used to perform weight prediction using various prediction signals by transmitting additional motion information to unidirectional and bidirectional motion information.

[0153]

[0154] FIG. 6 is a block diagram illustrating a decoding device according to an embodiment of the present invention, and is intended to explain the configuration and operation of a video decoder for decoding an image.

[0155] Referring to FIG. 6, the video decoder (200) may be configured to include an entropy decoding unit (210), an inverse quantization unit (220), an inverse transformation unit (230), a filtering unit (240), a prediction unit (260), and a DPB (Decoded Picture Buffer, 270).

[0156] The entropy decoding unit (210) entropy decodes the bitstream to extract transform coefficient information, intra-coding information, inter-coding information, etc. for each region.

[0157] For example, the entropy decoding unit (210) can obtain a binary code for transform coefficient information of a specific area from a bitstream, and can obtain a quantized transform coefficient by de-binarizing the binary code.

[0158] The inverse quantization unit (220) inversely quantizes the quantized transform coefficients, and the inverse transformation unit (230) restores the residual value using the inverse quantized transform coefficients.

[0159] Meanwhile, the residual value obtained from the inverse transformation unit (230) is combined with the predicted value obtained from the prediction unit (260) to restore the original pixel value.

[0160] The filtering unit (240) performs a filtering operation using a deblocking filter, a sample adaptive offset, an adaptive loop filter, etc. to improve the image quality of the restored picture, and the filtered picture can be output or stored in the DPB (270) to be used as a reference picture for the next picture.

[0161] The prediction unit (260) includes an intra / IBC prediction unit (261) and an inter prediction unit (265), and generates a prediction picture by utilizing the encoding type decoded through the entropy decoding unit (210), the transform coefficient for each region, and the intra / inter encoding information.

[0162] To restore the current block in which decoding is performed, the decoded region of the current picture or other pictures containing the current block can be used. A picture (or tile / slice) that performs intra prediction or intra BC prediction using only the current picture for restoration is called an intra picture or I picture (or tile / slice), and a picture (or tile / slice) that can perform all of intra prediction, inter prediction, and intra BC prediction is called an inter picture (or tile / slice).

[0163] Meanwhile, a picture (or tile / slice) that uses at most one motion vector and reference picture index to predict sample values ​​of each block among inter-pictures (or tiles / slices) is called a predictive picture or P-picture (or tile / slice), and a picture (or tile / slice) that uses at most two motion vectors and reference picture indices is called a bi-predictive picture or B-picture (or tile / slice).

[0164] That is, a P picture (or tile / slice) uses at most one motion information set to predict each block, and a B picture (or tile / slice) uses at most two motion information sets to predict each block. Here, a motion information set may include one or more motion vectors and one reference picture index.

[0165] The intra / IBC prediction unit (261) generates a prediction block using intra encoding information and restored samples within the current picture, and the intra encoding information may include at least one of an intra prediction mode, an MPM (Most Probable Mode) flag, and an MPM index.

[0166] The intra / IBC prediction unit (261) can predict sample values ​​of the current block using restored samples located on the left and / or upper side of the current block as reference samples.

[0167] For example, the reference samples may be samples adjacent to the left boundary of the current block and / or samples adjacent to the upper boundary, and may be samples located on a line within a preset distance from the left boundary of the current block among samples of neighboring blocks of the current block and / or samples located on a line within a preset distance from the upper boundary of the current block. In this case, the neighboring blocks of the current block may include at least one of a left (L) block, an upper (A) block, a below left (BL) block, an above right (AR) block, or an above left (AL) block adjacent to the current block.

[0168] Additionally, the intra / IBC prediction unit (261) can restore the current region by referring to a specific region including restored samples within the current picture. To this end, the intra / IBC prediction unit (261) can perform IBC prediction using IBC encoding information obtained from the entropy decoding unit (210), and the IBC encoding information can include block vector information.

[0169] The inter prediction unit (265) generates a prediction block using the reference picture and inter encoding information stored in the DPB (270), and the inter encoding information may include a set of motion information (reference picture index, motion vector information, etc.) of the current block for the reference block.

[0170] Meanwhile, inter prediction can include L0 prediction, L1 prediction, and bidirectional prediction (Bi-prediction).

[0171] L0 prediction refers to prediction using a single reference picture included in the L0 picture list, and L1 prediction refers to prediction using a single reference picture included in the L1 picture list. For this, a set of motion information (e.g., motion vector and reference picture index) may be required.

[0172] Bidirectional prediction can utilize up to two reference regions, which may reside in the same reference picture or in different pictures. Accordingly, among the two sets of motion information used in bidirectional prediction, two motion vectors may correspond to the same reference picture index or to different reference picture indices.

[0173] At this time, the reference pictures are pictures that are located temporally before or after the current picture, and may be completed pictures that have already been restored, and the two reference areas used in the bidirectional prediction method may be areas selected from the L0 picture list and the L1 picture list, respectively.

[0174] The inter prediction unit (265) can obtain a reference block of the current block using a motion vector and a reference picture index, and the reference block exists in a reference picture corresponding to the reference picture index.

[0175] Additionally, the sample values ​​of a block specified by a motion vector or their interpolated values ​​can be used as a predictor of the current block. For motion prediction with subpel-level pixel accuracy, an 8-tap interpolation filter can be used for the luminance signal, and a 4-tap interpolation filter can be used for the chrominance signal.

[0176] Meanwhile, the configuration and operation of the decoder described with reference to FIG. 6 are according to one embodiment of the present invention, and some configurations may be omitted or added as needed, and the decoder may decode an image by performing the reverse process of the encoding method of the encoder described above.

[0177]

[0178] According to one embodiment of the present invention, intra or inter prediction as described above can be performed in an encoder or decoder using template matching.

[0179] Template matching is a method of constructing a template using restored pixels surrounding a block, finding the block with the highest similarity between the current block and the template, and using it for intra or inter prediction.

[0180] Hereinafter, with reference to FIGS. 7 to 14, embodiments of a method for performing intra or inter prediction using template matching will be described.

[0181] According to one embodiment of the present invention, template matching can be used for intra prediction in IBC mode.

[0182] Referring to FIG. 7, in the IBC mode, prediction information for the current block, which is the encoding target block, can be obtained from a reference block that has already been decoded and is located within the same screen.

[0183] At this time, the vector information (including direction and size) from the current block to the reference block is referred to as a block vector (BV).

[0184] Although the IBC prediction method described above is an intra prediction method, it is similar to an inter prediction method in that it derives and transmits motion information of the block to be encoded (current block).

[0185] Accordingly, the IBC prediction method can be divided into ibc_skip / merge mode and ibc_amvp mode, and ibc_skip / merge and ibc_amvp can use skip, merge, AMVP (Advanced Motion Vector Prediction) and similar encoding, information transmission and decoding methods in inter-screen prediction.

[0186] The syntax structure for IBC prediction can be as shown in Table 1 below, and the sps_ibc_enabled_flag of the SPS (Sequence Parameter Set) and the syntax (sps_six_minus_max_num_ibc_merge_cand) indicating the maximum size of the candidate list of the ibc_skip / merge mode when the flag is enabled can be transmitted.

[0187]

[0188] Referring to Table 2 below, in a coding unit (CU) that is a block to be encoded, a flag (pred_mode_ibc_flag) indicating whether the block is in IBC mode and a flag (general_merge_flag) indicating whether the block is in ibc_merge mode can be transmitted.

[0189] And in the case of ibc_amvp mode, the block vector (BV) value of the corresponding block can be derived using the mvd, mvp, and amvr values.

[0190]

[0191] Meanwhile, referring to Table 3 below, in the case of ibc_skip / merge mode, the block vector (BV) value of the corresponding block can be derived using merge_idx information.

[0192]

[0193] FIG. 8 illustrates an example of a method for performing intra prediction in IBC mode using template matching.

[0194] As described above, in the IBC mode, in order to use a reference block within the same screen as prediction information for the target block to be encoded, a block vector (BV), which is motion information for the target block, is derived and transmitted to the decoding device.

[0195] In the IBC mode using template matching, the use of a reference block within the same screen as the prediction information for the target block is the same as the IBC mode above, but the block vector (BV), which is motion information for the target block, is not transmitted to the decoding device, and the decoding device can find the prediction information by searching for the reference block on its own through the template matching process.

[0196] Referring to Fig. 8, a restored template area is defined around the current block (Cur. block), and a template area (Best matching Template) that most closely matches the template area of ​​the current block in the area of ​​restored blocks within the same screen (Reconstructed area) is searched, thereby obtaining a predicted block (ref. block).

[0197] And, based on the position of the predicted block (ref. block) obtained as described above, a block vector (BV) value for the current block (Cur. block) can be derived.

[0198] As described above, in the IBC mode using template matching, a block vector (BV) for a target block is derived in a decoding device, and the block vector (BV) value can be used for intra prediction in the same way as the block vector (BV) value of the IBC mode.

[0199] As another embodiment of using template matching for intra prediction in IBC mode, after a merge candidate list or an AMVP candidate list for IBC is constructed, an error value for each candidate is calculated through template matching, and the candidate list can be reordered according to the calculated template error value.

[0200] For example, the block vector candidate list (BV candidate list) may include BV(s) of one or more surrounding blocks adjacent to the target block, BV(s) stored in a block vector buffer (BV buffer), and a default BV, but the present invention is not limited thereto.

[0201] Meanwhile, the error value represents the difference between the template of the target block and the template of the reference block pointed to by the candidate, and can be calculated as SAD (Sum of Absolute Differences), but the present invention is not limited thereto, and various values ​​such as MRSAD (Mean Removal SAD), SATD (Sum of Absolute Transformed Differences), weighted SAD, or MSE (Mean Squared Error) can be adaptively used depending on the intra prediction mode of the target block, etc.

[0202] The encoder can rearrange the candidates included in the candidate list in order of the smallest template error value, select the final candidate through the Rate-Distortion Optimization (RDO) process, and transmit the index value of the selected candidate to the decoder through the bitstream.

[0203] As described above, since the candidate list is rearranged in order of decreasing template error value, the number of bits required to transmit the candidate index value to the decoder can be saved, and also, if the candidate with the smallest template error value is selected as the final candidate, the index value may not be transmitted.

[0204] For example, if the maximum number of candidates included in the merge candidate list is 5, the binary representation for the 5 candidates can be as shown in Table 4 below.

[0205]

[0206] In the case of Table 4, if the BV (0,1) value with index 2 is selected as the final candidate through the RDO process among the block vector candidate list, the BV index value can be included in the bitstream as a 3-bit codeword, "110", and transmitted to the decoder.

[0207] Table 5 below shows an example of the results of rearranging the list of block vector candidates shown in Table 4 in order of decreasing template error values.

[0208]

[0209] In the case of Table 5, if the BV (0,1) value is selected as the final candidate through the RDO process among the block vector candidate list, the BV index value is expressed as a 1-bit codeword, “0,” so the number of bits for transmitting the BV index value to the decoder can be saved.

[0210] The maximum number of candidates, binarization method, BV value, etc. of the candidate list described with reference to Table 4 and FIG. 5 above are examples, and the present invention is not limited thereto, and image compression performance can be improved through candidate list reordering using template matching as described above.

[0211] As another embodiment of using template matching for intra prediction in IBC mode, instead of transmitting a value indicating the resolution or precision for a block vector (BV) to the decoder, the BV value can be transmitted to the decoder with a high resolution (or precision), and the decoder can compensate for the BV value with a low resolution (or precision) using template matching.

[0212] For example, the encoder may transmit BV values ​​to the decoder at 4-pixel resolution, and the decoder may compensate or refine the BV values ​​at 1-pixel resolution using template matching.

[0213] The encoder can specify the BV values ​​in units of 0, 4, 8, 12, and 16 pixels, which are multiples of 4, and transmit that information to the decoder. For example, the encoder can transmit (12,8) as the BV value of the target block to the decoder.

[0214] In this case, the decoder sets the search range for template matching to between 8 and 16 on the x-axis and between 4 and 12 on the y-axis, searches for the template with the minimum error value in units of 1 pixel within the search range, and sets the block with the template with the minimum error value as the final reference block.

[0215] In this way, the encoder does not transmit an index value (e.g., abvr index) to indicate the resolution of the BV value, but transmits the BV value at the default resolution of 4 pixels, and allows the decoder to compensate the BV value in units of 1 pixel, thereby saving the number of bits required to transmit the index (e.g., abvr index).

[0216] Alternatively, the resolution of the BV values ​​can be reduced from the existing resolutions (e.g., 4-pixel, 1-pixel, 1 / 4-pixel) to a smaller number of resolutions (e.g., 4-pixel, 1-pixel), and the encoder can transmit an index value for one of the smaller number of resolutions (e.g., 4-pixel, 1-pixel) to the decoder.

[0217] In this case, the decoder can compensate the BV value to a lower resolution than the transmitted resolution through a template matching process, thereby saving the number of bits for expressing and transmitting the index (e.g., abvr index) indicating the resolution of the BV value.

[0218] For example, as shown in Table 6 below, when the resolution of the BV value can be transmitted in three resolutions (4-pixel, 1-pixel, 1 / 4-pixel), if the resolution of the BV value for the target block is 1-pixel, the corresponding resolution value can be expressed and transmitted as a 2-bit codeword, “10.”

[0219]

[0220] Meanwhile, as shown in Table 7 below, when the resolution of the BV value is reduced to two resolutions (4-pixel, 1-pixel), the resolution value of the BV value for the target block, 1-pixel, can be expressed and transmitted as a 1-bit codeword, “1.”

[0221]

[0222] In this case, the decoder can compensate the BV values ​​transmitted at a resolution of 1 pixel in 1 / 4-pixel units through a template matching process.

[0223] The maximum number of indices and resolution values ​​indicating the resolution of the BV value described with reference to Tables 6 and 7 above are examples, and the present invention is not limited thereto, and image compression performance can be improved through compensation of the BV value using template matching as described above.

[0224] The intra prediction method described with reference to FIGS. 7 and 8 can be performed by a video encoding device and a video decoding device according to an embodiment of the present invention, and can be performed, for example, by an intra / IBC prediction unit (161) of a video encoder (100) illustrated in FIG. 1 or an intra / IBC prediction unit (261) of a video decoder (200) illustrated in FIG. 4.

[0225] Meanwhile, the method of using template matching for intra prediction as described above can also be applied to inter prediction.

[0226] Hereinafter, embodiments of methods for using template matching for inter prediction will be described, and a description of methods that are identical to the methods for using template matching for intra prediction described with reference to FIGS. 7 and 8 will be omitted.

[0227] According to one embodiment of the present invention, template matching can be used for inter prediction in Merge / Skip mode.

[0228] A method of utilizing template matching in merge and skip modes among inter prediction modes, wherein after a merge candidate list is constructed, an error value for each candidate is calculated through template matching, and the candidate list can be reordered according to the calculated template error value.

[0229] Here, the error value represents the difference between the template of the target block and the template of the reference block in the reference picture pointed to by the candidate, and can be calculated with various values ​​such as SAD, MRSAD, SATD, weighted SAD, or MSE.

[0230] The encoder can rearrange the candidates included in the candidate list in order of the smallest template error value, select the final candidate through the rate-distortion optimization (RDO) process, and transmit the index value of the selected candidate to the decoder through the bitstream.

[0231] As described above, since the merge candidate list is rearranged in order of decreasing template error value, the number of bits required to transmit the candidate index value to the decoder can be saved, and in addition, if the candidate with the smallest template error value is selected as the final candidate, the index value may not be transmitted.

[0232] For example, if the maximum number of candidates included in the merge candidate list is 5, the binary representation for the 5 candidates may be as shown in Table 8 below.

[0233]

[0234] In the case of Table 8, if the MV (0,1) value with index 2 is selected as the final candidate through the RDO process among the motion vector (MV) candidate list, the MV index value can be included in the bitstream as a 3-bit codeword, "110", and transmitted to the decoder.

[0235] Table 9 below shows an example of the results of rearranging the motion vector candidate list shown in Table 8 in order of decreasing template error value.

[0236]

[0237] In the case of Table 9, if the MV (0,1) value is selected as the final candidate through the RDO process among the motion vector candidate list, the MV index value is expressed as a 1-bit codeword, “0,” so that the number of bits for transmitting the MV index value to the decoder can be saved.

[0238] The maximum number of candidates, binarization method, MV value, etc. of the candidate list described with reference to Tables 8 and 9 above are examples, and the present invention is not limited thereto, and image compression performance can be improved through candidate list reordering using template matching as described above.

[0239] According to another embodiment of the present invention, template matching can be used for inter prediction in Advanced Motion Vector Prediction (AMVP) mode.

[0240] As a first embodiment of a method of utilizing template matching in AMVP mode among inter prediction modes, after an AMVP candidate list is constructed, an error value for each candidate is calculated through template matching, and the candidate list can be reordered according to the calculated template error value.

[0241] The encoder can rearrange the candidates included in the candidate list in order of the smallest template error value, select the final candidate through the rate-distortion optimization (RDO) process, and transmit the index value of the selected candidate to the decoder through the bitstream.

[0242] Meanwhile, if the maximum number of candidates included in the AMVP candidate list is 2, the encoder can construct an AMVP candidate list including 2 candidates and then select a candidate with a smaller error value through a template matching process for each candidate.

[0243] In this case, the encoder does not transmit the AMVP index value to the decoder, but the decoder can derive and use the AMVP index value on its own through the template matching process.

[0244] As a second embodiment of a method of using template matching in AMVP mode among inter prediction modes, a reference picture used in AMVP mode can be selected using template matching.

[0245] When motion vector (MV) related information for the target block, such as the Motion Vector Difference (MVD) and MVP index values, is passed to the decoder, the decoder can derive the MV values ​​even if it does not know the location of the reference picture.

[0246] Meanwhile, since the slice header information of the current picture includes information about the reference picture list used in the slice, the decoder can know the list of reference pictures available for the current block.

[0247] For example, as illustrated in FIG. 9, if a (t-1) reference picture, a (t-2) reference picture, and a (t-3) reference picture can be used for the current block (Cur. block) according to slice header information, reference block (Ref. block) candidates at the same position indicated by the MV values ​​derived from the MVD and MVP index values ​​in each of the three reference pictures can be obtained.

[0248] In this case, for each of the three reference block candidates, an error value between the template (Template) around the current block (Cur. block) and the template (Template 1, 2, 3) around the reference block (Ref. block) is calculated, and the reference block candidate with the smallest calculated error value can be selected as the final reference block.

[0249] As shown in Fig. 10, when the position of an object is moved in pictures, among the templates of three reference block candidates (Template 1, 2, 3), the template with the smallest error value with respect to the template of the current block (Cur. block) may be the template (Template 2) in the (t-2) reference picture.

[0250] Accordingly, the reference picture for the current block (Cur. block) can be determined as the (t-2) reference picture, and the decoder can select the reference picture for the current block (Cur. block) without receiving a reference picture index value from the encoder.

[0251] Alternatively, after a reference picture list for the current block (Cur. block) is constructed, a template error value for a reference block pointed to by an MV value within each reference picture may be calculated through template matching, and the reference picture list may be rearranged in order of the smallest calculated template error value.

[0252] In this case, the encoder can rearrange the reference pictures included in the reference picture list in order of the smallest template error value, select a final reference picture through a rate-distortion optimization (RDO) process, and transmit the reference picture index value of the selected candidate to the decoder through the bitstream.

[0253] As described above, since the reference picture list is rearranged in order of decreasing template error value, a reference picture in which a reference block similar to the target block is located can be placed higher in the reference picture list through the template matching process, thereby saving the number of bits for transmitting the reference picture index value to the decoder.

[0254] Fig. 11 is a flowchart illustrating an embodiment of a method for selecting a reference picture from a reference picture list using template matching. A description of the methods illustrated that are identical to those described above will be omitted.

[0255] Referring to FIG. 11, information about a reference picture list (ref.pic.list) is parsed from a slice header (SH, Slice Header) for a target block (step S1100).

[0256] MVD and MVP indexes are parsed (step S1110), and MV values ​​are derived using the parsed MVD and MVP index values ​​(step S1120).

[0257] i is set to “0”, and the minimum error value (min_error) is set to a preset value (MAX) (step S1130).

[0258] After that, for the reference block pointed to by the MV value derived in step S1120 within the i-th reference picture among the reference picture list, a template error value (TM_error) with respect to the target block is calculated (step S1140).

[0259] If the template error value (TM_error) calculated in step S1140 is smaller than the minimum error value (min_error) (step S1150), the minimum error value (min_error) is set to the template error value (TM_error) calculated in step S1140 (step S1160), and k is set to i (step S1170).

[0260] Next, if the i-th reference picture is the final reference picture (final ref.pic) (step S1180), the k-th reference picture is selected as the reference picture for the target block, and the block pointed to by the MV value within the k-th reference picture is selected as the reference block for the target block (step S1190).

[0261] Meanwhile, if the judgment result at step S1150 is that the template error value (TM_error) calculated at step S1140 is not smaller than the minimum error value (min_error), steps S1160 and S1170 are skipped, and step S1180 is performed.

[0262] And as a result of the judgment at step S1180, if the i-th reference picture is not the last reference picture (final ref.pic), i is increased by "1" (step S1185), and then the process returns to step S1140, and the above-described steps are repeated for the next reference picture in the reference picture list.

[0263] As a third embodiment of a method of using template matching in AMVP mode among inter prediction modes, instead of transmitting a value indicating the resolution or precision of a motion vector (MV) to a decoder, an MV value can be transmitted to the decoder with a high resolution (or precision), and the decoder can compensate for the MV value with a low resolution (or precision) using template matching.

[0264] For example, the encoder may transmit MV values ​​to the decoder at 1 / 4-pixel resolution, and the decoder may compensate or refine the MV values ​​at 1 / 16-pixel resolution using template matching.

[0265] In this way, the number of bits required to transmit an index (e.g., amvr index) can be saved by having the encoder not transmit an index value (e.g., amvr index) to indicate the resolution of the MV value, but instead having the decoder compensate for the MV value in pixel units lower than the resolution of the transmitted MV value.

[0266] Alternatively, the resolution of the MV value can be reduced from the existing four resolutions (1 / 4-pixel, 1 / 2-pixel, 1-pixel, 4-pixel) to a smaller number, for example, two resolutions (1 / 4-pixel, 1-pixel), so that the encoder transmits the index value for one of the two resolutions to the decoder, thereby saving the number of bits for transmitting the index (e.g., amvr index).

[0267] Here, the method by which the decoder compensates the MV value using template matching may be the same as the method by which the decoder compensates the BV value using template matching in intra prediction in IBC mode, so a detailed description thereof will be omitted.

[0268] As a fourth embodiment of a method of using template matching in AMVP mode among inter prediction modes, in the case of bidirectional prediction using BCW (Bi-prediction with CU-level Weight), the encoder does not transmit BCW index values ​​for indicating weights applied to bidirectional reference blocks to the decoder, but rather allows the decoder to derive weight values ​​using template matching.

[0269] In the case of BCW, to indicate the weight (w1) of one of the weights (w1, w2) for each of the bidirectional reference blocks, the encoder can specify a BCW index value as shown in Table 10 below and transmit it to the decoder.

[0270]

[0271] Meanwhile, the decoder can generate one reference block by applying a corresponding weight value (w1) to the reference block in the list_0 direction according to the BCW index value transmitted from the encoder, and applying a weight value (w2) obtained by subtracting the corresponding weight value (w1) from "1" to the reference block in the list_1 direction.

[0272] In contrast, according to an embodiment of the present invention, the decoder may calculate an error value between a template of a target block and a template of the corresponding reference block for each of the bidirectional reference blocks, and determine a weight for each of the bidirectional reference blocks based on the calculated template error value.

[0273] Specifically, the decoder can generate a single reference block through a weighted average of bidirectional reference blocks by assigning a high weight to a reference block with a small template error value and a lower weight to a reference block with a large template error value.

[0274] Referring to FIG. 12, for the current block (Cur. block), a reference block (Ref. block) in the (t-1) reference picture pointed to by the motion vector (mvLO) in the list_0 direction and a reference block (Ref. block) in the (t+1) reference picture pointed to by the motion vector (mvL1) in the list_1 direction can be obtained.

[0275] And for each of the bidirectional reference blocks, an error value can be calculated between the template (Template) around the current block (Cur. block) and the template (Template 1, 2) around the corresponding reference block (Ref. block).

[0276] In this case, a weight of 5 / 8 may be assigned to a reference block with a small calculated error value among bidirectional reference blocks, and a weight of 3 / 8 may be assigned to a reference block with a large error value.

[0277] For example, when the position of an object is moved in pictures as illustrated in Fig. 13, among the templates (Template 1, 2) of the bidirectional reference blocks, the template with a smaller error value with respect to the template of the current block (Cur. block) may be the template (Template 1) in the (t-1) reference picture.

[0278] Accordingly, a weight of 5 / 8 may be assigned to a reference block within the (t-1) reference picture, and a weight of 3 / 8 may be assigned to a reference block within the (t+1) reference picture.

[0279] In this embodiment, bidirectional prediction may use reference pictures in different directions relative to the current picture, such as the (t-1) reference picture and the (t+1) reference picture, or may use different reference pictures in the same direction relative to the current picture, such as the (t-1) reference picture and the (t-2) reference picture. That is, bidirectional prediction in the present invention is not limited to a specific direction, and may include all prediction methods that use two reference pictures.

[0280] Additionally, in the present embodiment, bidirectional prediction may utilize reference blocks pointed to by two different motion vectors within the (t-1) reference picture. That is, bidirectional prediction in the present invention is not limited to two different reference pictures, and may include all prediction methods that utilize two different reference blocks within a single reference picture.

[0281] As described above, by not transmitting an index value (e.g., BCW index) for indicating weights for bidirectional reference blocks, and by having the decoder derive weights for bidirectional reference blocks using template matching, the number of bits for transmitting the index (e.g., BCW index) can be saved.

[0282] Fig. 14 is a flowchart illustrating an embodiment of a method for deriving weights for bidirectional reference blocks using template matching. A description of the same method as described above among the illustrated methods will be omitted.

[0283] Referring to Fig. 14, bidirectional reference blocks are derived using the motion vectors (mvL0, mvL1) of the target block (step S1400).

[0284] The first weight (w1) assigned to the bidirectional reference blocks is set to 5 / 8, and the second weight (w2) is set to 3 / 8 (step S1410).

[0285] Here, the values ​​(5 / 8, 3 / 8) set as the first and second weights (w1, w2) are an example, and the present invention is not limited thereto, and the first weight (w1) may be set to other values ​​so that the second weight (w2) is greater.

[0286] Among the bidirectional reference blocks, a template error value (TM1_error) with respect to the target block is calculated for the first reference block (step S1420), and a template error value (TM2_error) with respect to the target block is calculated for the second reference block (step S1430).

[0287] Next, the template error value (TM1_error) for the first reference block and the template error value (TM2_error) for the second reference block are compared (step S1440).

[0288] As a result of the comparison, if the template error value (TM1_error) for the first reference block is smaller than the template error value (TM2_error) for the second reference block, a first weight (w1) is assigned to the first reference block, and a second weight (w2) is assigned to the second reference block (step S1450).

[0289] Conversely, if the template error value (TM1_error) for the first reference block is not less than the template error value (TM2_error) for the second reference block, a second weight (w2) is assigned to the first reference block, and a first weight (w1) is assigned to the second reference block (step S1460).

[0290] Meanwhile, if the template error value (TM1_error) for the first reference block and the template error value (TM2_error) for the second reference block have similar values ​​within a preset error range, the same weight, i.e., 1 / 2, may be assigned to the first and second reference blocks.

[0291] After that, the weights assigned as above are applied to the bidirectional reference blocks to calculate the average value, thereby generating the final reference block for the target block (step S1470).

[0292] In the above, an embodiment of the present invention was described by taking as an example a case where the encoder does not transmit BCW index values ​​to the decoder, but the number of bits can be saved by reducing the number of BCW indices to two and transmitting the selected index values ​​to the decoder.

[0293] In the above, the embodiments of the present invention have been described by taking as an example the template being composed of samples positioned around the target block in an "L" shape, as shown in (a) of FIG. 15, but the present invention is not limited thereto and may be composed of samples of various shapes positioned around the target block.

[0294] For example, referring to (b) of FIG. 15, the template may be composed of an upper template (T_Template) composed of samples located on the upper side of the target block (Block) and a left template (L_Template) composed of samples located on the left side of the target block (Block), excluding the upper left region (LT) in the "L"-shaped template.

[0295] FIG. 16 is a flowchart illustrating an embodiment of a method for setting a reference block using template matching according to the present invention. Among the methods illustrated, descriptions of those identical to those described with reference to FIGS. 1 to 15 will be omitted.

[0296] Referring to FIG. 16, a first template is formed of samples located on the upper side of the target block (step S1600), and a second template is formed of samples located on the left side of the target block (step S1610).

[0297] For example, the first template configured in step S1600 may be the same as the upper template (T_Template) illustrated in (b) of FIG. 15, and the second template configured in step S1610 may be the same as the left template (L_Template) illustrated in (b) of FIG. 15, but the present invention is not limited thereto.

[0298] Next, among the first and second templates configured in steps S1600 and S1610, one or more templates to be used for template matching are determined (step S1620).

[0299] For example, in step S1620, any one of three templates consisting of a first template, a second template, and a combination of the first and second templates may be determined as the template to be used for template matching.

[0300] Meanwhile, at step S1620, a flag indicating whether to use three templates can be checked, and the flag can be included in the bitstream as HLS (High Level Syntax) and located in at least one of SPS (Sequence Parameter Set), PPS (Picture Parameter Set), PH (Picture Peader), and SH (Slice Header).

[0301] Additionally, if three templates are available, an index value indicating which template among the three templates is to be used for template matching can be identified, and the index value can be included in the bitstream in units of blocks at the CU level.

[0302] In this case, in the process of setting a reference block used to generate a prediction block for a target block through template matching as described above, information about templates surrounding the target block and reference block used for template matching can be explicitly signaled from the encoder to the decoder.

[0303] Referring to FIG. 17 and Table 11 below, an index value of "0" may be assigned to a template (Top & Left) that includes both a top template (T_Template) and a left template (L_Template), an index value of "1" may be assigned to a template (Only Top) that includes only a top template (T_Template), and an index value of "2" may be assigned to a template (Only Left) that includes only a left template (L_Template).

[0304]

[0305] For example, if the flag included in the HLS (High Level Syntax) for the target block is "on", the three templates above (Top & Left, Only Top, Only Left) can be used for the target block.

[0306] And the encoder can include an index value indicating the template used for the target block among the three templates (Top & Left, Only Top, Only Left) in the bitstream and transmit it to the decoder.

[0307] That is, only when the flag included in HLS (High Level Syntax) is "on", the index value indicating the template used in the target block can be included in the bitstream.

[0308] As another example, in the process of setting a reference block used to generate a prediction block for a target block through template matching as described above, information about templates around the target block and reference block used for template matching may be implicitly derived from the decoder without being transmitted to the decoder.

[0309] More specifically, the template to be used for template matching can be determined based on at least one of the shape and position of the target block.

[0310] For example, if the shape of the target block is a square, a combination of the first and second templates can be determined as the template to be used for template matching.

[0311] Referring to (a) of Fig. 18, when the width (W) and height (H) of the target block are the same, a template (Top & Left) including both the upper template (T_Template) and the left template (L_Template) can be used.

[0312] Additionally, if the width of the target block is greater than the height, a first template composed of samples located on the upper side of the target block may be determined as the template to be used for template matching.

[0313] Referring to (b) of Fig. 18, when the width (W) of the target block is greater than the height (H) and has a long rectangular shape in the horizontal direction, a template (Only Top) including only the upper template (T_Template) can be used.

[0314] Meanwhile, if the height of the target block is greater than the width, a second template consisting of samples located on the left side of the target block may be determined as the template to be used for template matching.

[0315] Referring to (c) of Fig. 18, when the height (H) of the target block is greater than the width (W) and has a shape of a long rectangle in the vertical direction, a template (Only Left) including only the left template (L_Template) can be used.

[0316] As another example, if the width (W) of the target block is greater than the height (H), a left template (L_Template) composed of samples located on the left side of the target block may be determined as the template to be used for template matching, and if the height (H) of the target block is greater than the width (W), an upper template (T_Template) composed of samples located on the upper side of the target block may be determined as the template to be used for template matching.

[0317] As described with reference to (b) and (c) of FIG. 18, when the shape of the target block is rectangular, using only one of the upper template (T_Template) and the left template (L_Template) can be applied only when the width (W) and height (H) of the target block differ by more than twice, but the present invention is not limited thereto.

[0318]

[0319] Hereinafter, embodiments of a method for processing a picture boundary area when a reference block (or target block) is located at a picture boundary and at least some area used for prediction (or matching) is outside the picture boundary, when performing a prediction method as described with reference to FIGS. 1 to 18, will be described.

[0320] Hereinafter, embodiments of the present invention will be described using a reference block as an example, but the present invention is not limited thereto, and embodiments of the present invention can also be applied to a current block for which matching is to be performed within the current picture.

[0321] According to one embodiment of the present invention, when an area used to perform a template matching or motion compensation process is located outside a picture boundary, various embodiments may be used to pad the area located outside the picture boundary.

[0322] During the template matching process, if the current block (Cur. block) is located adjacent to the boundary of the current picture, some areas of the template for the current block may extend beyond the current picture.

[0323] Referring to FIG. 19, in the template matching process, if a reference block (Ref. block) is located adjacent to the boundary of a reference picture, some areas of the template for the reference block may go outside the reference picture.

[0324] And even during the motion compensation process, at least part of the MC (Motion Compensation) area of ​​the reference block used for motion compensation may go beyond the picture boundary.

[0325] For example, referring to (a) of FIG. 20, in the motion compensation process, the motion vector (MV) for the current block (Cur, Block) may point outside the reference picture, and accordingly, some areas of the reference block (Ref. block) may be located outside the reference picture.

[0326] Also, referring to (b) of FIG. 20, in the motion compensation process, if the motion vector (MV) for the current block (Cur, Block) points to the boundary of the reference picture, some of the surrounding samples of the reference block (Ref. block) to which the interpolation filter is applied may go beyond the reference picture.

[0327]

[0328] Hereinafter, with reference to FIGS. 21 to 26, embodiments of a method for processing a picture boundary area in a template matching process or a motion compensation process will be described.

[0329] Hereinafter, embodiments of the present invention will be described by taking as an example a case where a template of a reference block (or target block) goes beyond a picture boundary during a template matching process, but the present invention is not limited thereto, and embodiments of the present invention can also be applied to a case where a reference block used in a motion compensation process or surrounding samples of the reference block go beyond a picture boundary.

[0330] Meanwhile, in the template matching process, the template of the reference block (or target block) and the reference block or surrounding samples of the reference block used in the motion compensation process may be reconstructed pixel values ​​or values ​​derived using the reconstructed pixel values.

[0331] Additionally, in the following, the reconstructed pixel value closest to the boundary of the picture may be conveniently expressed as a pixel value.

[0332]

[0333] As a first embodiment of a method for handling a picture boundary area, boundary pixel padding may be used to construct an area outside the picture boundary by copying the reconstructed pixel values ​​closest to the picture boundary.

[0334] Specifically, if some area of ​​the template extends beyond the picture boundary, the pixel values ​​closest to the boundary of the picture can be copied to form the template area extending beyond the picture boundary.

[0335] Additionally, if some of the MC areas of the reference block used for motion compensation extend beyond the picture boundary, the MC areas extending beyond the picture boundary may be constructed by copying the reconstructed pixel values ​​closest to the boundary of the picture.

[0336] For example, referring to (a) of Fig. 21, when the left area of ​​the template of the reference block (Ref. block) goes beyond the boundary of the reference picture, the pixel values ​​of the edge closest to the left boundary of the reference picture can be copied to fill in the pixel values ​​of the left area of ​​the template that goes beyond the picture boundary.

[0337] Also, referring to (b) of FIG. 21, when the upper area of ​​the template of the reference block (Ref. block) goes beyond the boundary of the reference picture, the pixel values ​​of the edge closest to the upper boundary of the reference picture can be copied to fill in the pixel values ​​of the upper area of ​​the template that goes beyond the picture boundary.

[0338] For example, using the mathematical expression 1 below, pixel values ​​at the very edge of the reference picture can be copied as described above to fill in pixel values ​​of the template that are outside the picture boundary.

[0339]

[0340] In mathematical expression 1, x is the location information of a pixel included in the template, and xA is the updated location information for the pixel.

[0341] Here, if x is less than 0, it may mean that the corresponding pixel in the template is outside the picture boundary.

[0342] In this case, if x is less than 0, the position of the corresponding pixel can be updated to the position of 0, so that the 0th pixel value can be copied and used.

[0343]

[0344] As a second embodiment of a method for processing a picture boundary area, motion compensated padding may be used to construct an area outside the boundary of a picture using motion information of a reference block.

[0345] Specifically, if some areas of the template of the reference block extend beyond the picture boundary, the motion information of the corresponding reference block area can be used to construct a template area extending beyond the picture boundary. Here, the number of motion information pieces for the corresponding reference block area may be 1 or 2 or more.

[0346] Additionally, if a portion of the MC region of a reference block used for motion compensation extends beyond the picture boundary, the motion information of the corresponding reference block region may be used to construct the MC region extending beyond the picture boundary. Here, the number of motion information pieces for the corresponding reference block region may be one or two or more.

[0347] Figure 22 is a flowchart illustrating a second embodiment of a method for processing a picture boundary area. Figure 22 illustrates an example where there is only one piece of motion information for a reference block area, but the present invention is not limited thereto, and the reference block area may include two or more pieces of motion information.

[0348] Referring to FIG. 22, if at least a part of the surrounding area of ​​the first reference block used to perform prediction for the target block (or, current block) is located outside the picture boundary (step S2200), first vector information for the first reference block area is obtained (step S2210).

[0349] For example, the surrounding area of ​​the first reference block may include surrounding samples of the first reference block used for template matching with the target block, or may include surrounding samples of the first reference block used for performing motion compensation for the target block.

[0350] Meanwhile, the first vector information for the first reference block area may be a motion vector (MV) or a block vector (BV) of the first reference block area, but the present invention is not limited thereto, and may be various pieces of information indicating a block referenced by the first reference block area.

[0351] Thereafter, a second reference block area referenced by the first reference block area is identified based on the first vector information obtained in step S2210 (step S2220), and a surrounding area of ​​the first reference block located outside the picture boundary is configured using surrounding samples of the second reference block area (step S2230).

[0352] Referring to FIG. 23, the current block (Cur. block), which is a target block for which prediction is to be performed, refers to the reference block (Ref. block) of the (t-1) reference picture, and some areas of the template of the reference block (Ref. block) may go beyond the boundary of the (t-1) reference picture.

[0353] In this case, since there are no reconstructed pixel values ​​in the template area outside the boundary of the (t-1) reference picture, padding may be required for the template area outside the picture boundary for template matching.

[0354] Here, information about the (t-1) reference picture may include a reconstructed picture and MV storage.

[0355] For example, the reconstructed picture includes pixel values ​​restored through the encoding or decoding process of the (t-1) reference picture, and the MV storage may represent a buffer in which motion vector (MV) values ​​for the (t-1) reference picture are stored in units of NxN (e.g., 4x4).

[0356] Referring to (a) of Fig. 24, in order to pad areas (P1, P2) outside the boundary of the (t-1) reference picture, 4x4 unit-wise motion information for the (t-1) reference picture, for example, reference picture index and motion vector values, can be obtained from MV storage.

[0357] Here, the MV storage may store either the original motion vector (original MV) value or the scaled motion vector (scaled MV) value.

[0358] When motion vector (original MV) values ​​are stored in MV storage, motion information (e.g., reference picture index and motion vector value) for each block can be stored in MV storage in 4x4 unit units.

[0359] Meanwhile, when the scaled motion vector (scaled MV) value is stored in the MV storage, since the reference pictures referenced by each block may be different, the original motion vector (original MV) value may be scaled based on a specific reference picture and stored in the MV storage.

[0360] In this case, since the scaled motion vector (scaled MV) values ​​for all blocks refer to the same specific reference picture, the motion information stored in the MV storage may not include a reference picture index.

[0361] Embodiments according to the present invention can be applied regardless of the form of motion vectors stored in MV storage.

[0362] For example, as illustrated in (a) of FIG. 24, a first area (B1) of 4x4 size can refer to a first area (B1') in a (t-2) reference picture according to a first motion vector value (MV1), and a second area (B2) of another 4x4 size can refer to a second area (B2') in a (t-3) reference picture according to a second motion vector value (MV2).

[0363] In this case, the first padding area (P1) located outside the boundary of the (t-1) reference picture adjacent to the first area (B1) can be filled with samples of the first padding area (P1') adjacent to the first area (B1') within the (t-2) reference picture.

[0364] Here, the size of the first padding areas (P1, P1') may correspond to the size of the target area corresponding to the boundary of the (t-2) reference picture from the position indicated by the first motion vector value (MV1).

[0365] Additionally, the second padding area (P2) located outside the boundary of the (t-1) reference picture adjacent to the second area (B2) can be filled with samples of the second padding area (P2') adjacent to the second area (B2') within the (t-3) reference picture.

[0366] Here, the size of the second padding areas (P2, P2') may correspond to the size of the target area corresponding to the boundary of the (t-3) reference picture from the position indicated by the second motion vector value (MV2).

[0367] Meanwhile, the target region used to pad the region beyond the reference picture boundary as described above may have a height equal to the height (N) of the unit in which motion information is stored in the MV storage, and a width set to the distance to the picture boundary or limited to a predefined first value (k1).

[0368] For example, the first value (k1), which is a limit value for the width of the target area, may be 16, but the present invention is not limited thereto.

[0369] In addition, when the width of the target area is set to the first value (k1), the target area may go beyond the boundary of the corresponding reference picture. For example, when the first padding area (P1) located outside the boundary of the (t-1) reference picture adjacent to the first area (B1) is filled with samples of the first padding area (P1') adjacent to the first area (B1') in the (t-2) reference picture, part or all of the first padding area (P1') may be located outside the boundary of the (t-2) reference picture. In this case, the area outside the boundary of the (t-2) reference picture may also have a padding value.

[0370] For example, if the first motion vector value (MV1) of the first region (B1) is (0,0), the first region (B1') is located at the (t-2) reference picture boundary, and the first padding region (P1) located outside the (t-1) reference picture boundary adjacent to the first region (B1) may be filled with samples of the first padding region (P1') outside the reference picture boundary adjacent to the first region (B1') located at the (t-2) reference picture boundary. Here, the entire first padding region (P1') may be padded sample values.

[0371] And, unlike the case illustrated in (a) of FIG. 24, when the padding area is located outside the upper or lower boundary of the reference picture, the width of the target area may be equal to the width (N) of the unit in which motion information is stored in the MV storage, and the height of the target area may be set to the distance to the picture boundary or may be limited to a predefined second value (k2).

[0372] Additionally, when the height of the target area is set to the second value (k2), the target area may extend beyond the boundary of the corresponding reference picture. In this case, the area extending beyond the boundary of the corresponding reference picture may also have a padding value.

[0373] For example, the second value (k2), which is a limit value for the height of the target area, may be 16, but the present invention is not limited thereto.

[0374] (t-1) The first and second padding areas (P1, P2) outside the reference picture boundary are filled as described above, and the remaining padding areas that are not yet filled can be filled by copying the pixel values ​​closest to the boundary of the (t-2) reference picture or the (t-3) reference picture using boundary pixel padding according to the first embodiment.

[0375] In addition, the first and second padding areas (P1, P2) outside the (t-1) reference picture boundary are filled as described above, and the remaining padding areas that are not yet filled can be filled by copying the edge pixel values ​​of the first and second padding areas (P1, P2) using boundary pixel padding according to the first embodiment.

[0376] For example, if the width (or height) of the target area used for padding is less than the preset value of 16, the remaining padding area that is not filled with pixel values ​​of the target area can be filled by copying the pixel value closest to the picture boundary among the target areas.

[0377] As another example, if the width (or height) of the target area used for padding is a predefined first value (or second value), and the target area can exceed the boundary of the corresponding reference picture, there is no remaining padding area in the target area that is not filled with pixel values. In this case, the target area is designated as a first target area, and the target area exceeding the predefined value is set as a second target area, and the second target area can be filled by copying the boundary edge pixel values ​​of the first target area using boundary pixel padding according to the first embodiment.

[0378] Meanwhile, if a block located at the boundary of the (t-1) reference picture is encoded in the IBC mode, a block vector (BV) value instead of a motion vector (MV) value may exist in the MV storage for the block, and the reference picture for the block may be the (t-1) reference picture, which is the same picture.

[0379] For example, referring to (b) of FIG. 24, if the third area (B3) located at the boundary of the (t-1) reference picture is encoded in the IBC mode, after confirming the area (B3') within the (t-1) reference picture pointed to by the block vector (BV) for the third area (B3), the third padding area (P3) can be filled using the target area (P3') having a width less than or equal to a predefined value (k) to the left of the area (B3').

[0380] Unlike the case illustrated in (b) of Fig. 24, when the third area (B3) encoded in the IBC mode is located at the right border of the (t-1) reference picture, the third padding area (P3) can be filled using a target area (P3') having a width less than or equal to a predefined value (k) to the right from the area (B3') pointed to by the block vector (BV).

[0381] Meanwhile, in step S2210 illustrated in FIG. 22, when the first vector information (e.g., motion vector or block vector) does not exist, the surrounding area of ​​the first reference block located outside the picture boundary can be constructed using the samples closest to the picture boundary among the samples in the first reference block.

[0382] For example, if an area located at the boundary of the (t-1) reference picture is encoded in an intra prediction mode other than the IBC mode, both the motion vector (MV) value and the block vector (BV) value for that area may not exist in the MV storage, so the padding area outside the (t-1) reference picture boundary may be filled by copying the pixel value closest to the (t-1) reference picture boundary.

[0383] Additionally, if the motion vector (MV) value of an area located at the boundary of the (t-1) reference picture is (0,0), the motion information in the MV storage for the area may be zero MV, and the padding area outside the boundary of the (t-1) reference picture may be filled by copying the pixel value closest to the boundary of the (t-1) reference picture.

[0384] And if the area located at the boundary of the (t-1) reference picture is encoded with bidirectional prediction, since there are two motion vector (MV) values ​​in the MV storage for the area, the padding area outside the (t-1) reference picture boundary can be filled with the average value of two target areas derived using each of the two motion vector (MV) values, or can be filled with the target area derived using either of the two motion vector (MV) values.

[0385] According to the second embodiment as described above, if there is one of a valid motion vector (MV) value and a block vector (BV) value in the MV storage for an area located at the picture boundary, all or part of the padding area may be filled with a surrounding target area adjacent to the area indicated by the corresponding vector value, and the remaining unfilled padding area may be filled by copying a pixel value located at the outermost end of the filled padding area.

[0386] Meanwhile, if there are no valid motion vector (MV) values ​​or block vector (BV) values ​​in the MV storage for an area located at the picture boundary, the padding area can be filled by copying the pixel value closest to the picture boundary.

[0387] In the above, the embodiment of the present invention has been described with reference to FIG. 24 as an example of a case in which at least a part of the template area of ​​a reference block goes beyond the picture boundary during the template matching process, but the present invention is not limited thereto, and is also applicable to a case in which at least a part of the template area of ​​a current block (Cur. block), which is a target block, goes beyond the picture boundary during the template matching process, and a case in which at least a part of the MC area of ​​a reference block used in a motion compensation process goes beyond the picture boundary.

[0388] And in steps S2210 to S2230 illustrated in FIG. 22, when the target block is located within an I slice (intra slice), a surrounding area of ​​the first reference block located outside the picture boundary may be configured using samples closest to the picture boundary among the samples within the first reference block.

[0389] That is, since the motion compensated padding method according to the second embodiment may have high computational complexity, for the I slice, regardless of whether it is the IBC mode or another intra prediction mode, the padding area can be filled by copying the pixel values ​​closest to the picture boundary according to the boundary pixel padding method of the first embodiment without checking whether a valid block vector (BV) exists in the MV storage.

[0390] Meanwhile, the motion compensated padding function according to the second embodiment can be activated (on) / deactivated (off) with a specific flag (e.g., mc_padding_enabled_flag).

[0391] For example, if the mc_padding_enabled_flag value is “on”, the padding area may be filled using the motion compensation padding method according to the second embodiment, and if the mc_padding_enabled_flag value is “off”, the padding area may be filled using the boundary pixel padding method according to the first embodiment.

[0392] Here, the flag (mc_padding_enabled_flag) is HLS (High-Level Syntax) and can be inserted into at least one of SPS (Sequence Parameter Set), PPS (Picture Parameter Set), PH (Picture Peader), and SH (Slice Header) and transmitted as a bitstream.

[0393]

[0394] As another embodiment, a template matching method may be used to pad areas outside the picture boundary for block areas located at the picture boundary.

[0395] For example, in steps S2210 to S2230 illustrated in FIG. 22, a first matching block area having the highest similarity to the first reference block area may be derived within the same picture, and a surrounding area of ​​the first reference block located outside the picture boundary may be constructed using surrounding samples of the first matching block area. Here, the surrounding samples of the first matching block area may be reconstructed pixel values.

[0396] Referring to FIG. 25, in order to pad a first padding area (P1) outside the picture boundary adjacent to a first area (B1) in a current picture, a matching area (B1') most similar to the first area (B1) is searched for within the same current picture, and the first padding area (P1) can be filled using a target area (P1') adjacent to the matching area (B1').

[0397] Additionally, in order to pad a second padding area (P2) outside the picture boundary adjacent to a second area (B2) within the current picture, an area (B2') most similar to the second area (B2) within the same current picture may be searched, and the second padding area (P2) may be filled using a target area (P2') adjacent to the similar area (B2').

[0398] Here, in order to calculate the similarity between two areas, a function for calculating a difference value or an error value such as SAD (Sum of Absolute Differences), SATD (Sum of Absolute Transformed Differences), MSE (Mean Squared Error), etc. may be used, but the present invention is not limited thereto.

[0399] Meanwhile, in order to search for matching areas most similar to the first and second areas (B1, B2), a search range within the same picture can be set in advance.

[0400] Alternatively, in order to search for matching areas most similar to the first and second areas (B1, B2), a candidate list may be constructed according to a predetermined method, and one candidate with the minimum error value among the candidates in the candidate list may be selected as the most similar matching area.

[0401] For example, the construction of a candidate list for selecting a matching area can be performed in the following manner.

[0402] First, if an area (B1 or B2) located at the picture boundary is encoded in the IBC mode, a reference area indicated by a block vector (BV) value for the area is added to the candidate list, a reference area indicated by a block vector (BV) value for a surrounding IBC block of the area is added to the candidate list, and if the reference area indicated by the block vector (BV) value for the area is encoded in the IBC mode, another reference area indicated by the block vector (BV) value of the reference area can be added to the candidate list.

[0403] Meanwhile, if an area (B1 or B2) located at the picture boundary is encoded in an intra prediction mode other than the IBC mode, and if the intra prediction mode is an angular mode, the area closest to the picture boundary in the angular direction can be added to the candidate list.

[0404] Template matching padding, which processes picture boundary areas using the template matching method described above, can be applied to all slice types, such as I slice, P slice, and B slice, because it searches for the most similar area within the same picture to pad areas outside the picture boundary.

[0405] As illustrated in Fig. 25, the first and second padding areas (P1, P2) outside the picture boundary can be filled using the target areas (P1', P2') within the same picture, and the remaining unfilled padding areas can be filled by copying the pixel values ​​located at the outermost part of the filled padding areas.

[0406] Meanwhile, the template matching padding function as described above can be enabled (on) / disabled (off) using a specific flag (e.g., tm_padding_enabled_flag).

[0407] For example, if the tm_padding_enabled_flag value is “on”, the padding area may be filled using the template matching padding method, and if the tm_padding_enabled_flag value is “off”, the padding area may be filled using the border pixel padding method according to the first embodiment.

[0408] Here, the flag (tm_padding_enabled_flag) is HLS (High-Level Syntax) and can be inserted into at least one of SPS (Sequence Parameter Set), PPS (Picture Parameter Set), PH (Picture Peader), and SH (Slice Header) and transmitted as a bitstream.

[0409]

[0410] As another embodiment, multiple padding methods for handling picture boundary areas may be adaptively applied as needed.

[0411] For example, a motion compensated padding method may be applied to inter-pictures of B or P slices that have motion vector (MV) values, while a different padding method may be required for intra-pictures of I slices that do not have motion vector (MV) values.

[0412] Accordingly, if the current picture is an inter-picture, the area outside the picture boundary is padded using motion compensation padding, and for an intra-picture, template matching padding as described above can be applied.

[0413] As another example, if the current picture is an inter-picture, the area outside the picture boundary may be padded using motion compensation padding, and for an intra-picture, boundary pixel padding as described above may be applied.

[0414] Additionally, while it may be effective to apply motion compensation padding to inter-pictures in the RA (Random Access) structure, it may be effective to apply template matching padding to inter-pictures in the LD (Low Delay) structure.

[0415] As another example, during the template matching process, if at least part of the template area of ​​the current block (or target block) within the current picture extends beyond the picture boundary, boundary pixel padding may be applied.

[0416] Meanwhile, in motion compensation padding or template matching padding, the width (or height) of the target region to fill the padding area outside the picture boundary may not be limited to a specific value (k).

[0417] In this case, a padding area is filled by a predefined number of h pixels (e.g., 16 pixels) from the picture boundary using a motion compensation padding or template matching padding method, and a padding area beyond that can be filled by copying pixel values ​​located at the outermost part of the filled padding area according to the boundary pixel padding method of the first embodiment.

[0418] For example, a padding area of ​​up to 16 pixels to the left, right, top, and bottom from the picture boundary may be filled using a motion compensation padding or template matching padding method in memory, and for the padding area beyond that, the pixel values ​​of the outermost area filled last in memory may be copied and used.

[0419] To this end, in the DPB (Decoded Picture Buffer) that stores restored pictures, the memory space for storing one picture can be explicitly set to a size corresponding to the picture size plus the size of the padding area outside the picture boundary, for example, a width of "picture width + (h*2)" and a height of "picture height + (h*2)".

[0420] As another example, if the padding area extending from the picture boundary to the left, right, top, and bottom within 16 pixels is not filled using the motion compensation padding or template matching padding method, the padding area may be filled within the maximum of 16 pixels using the motion compensation padding or template matching padding method, and the remaining padding area that is not filled may be filled within the maximum of 16 pixels using the boundary pixel padding method of the first embodiment.

[0421] Meanwhile, in order to pad an area exceeding the maximum value of 16 pixels, a pixel value located at the outermost edge of the filled padding area may be used according to the boundary pixel padding method of the first embodiment.

[0422] The above example illustrates a case where the maximum value filling the padding area is 16, but the present invention is not limited thereto.

[0423]

[0424] As a third embodiment of a method for handling a picture boundary area, wraparound padding may be used, which fills the pixel values ​​of an area outside the picture boundary by copying them with pixel values ​​of a corresponding area located on the opposite boundary of the picture.

[0425] Specifically, if some area of ​​the template extends beyond the picture boundary, the pixel values ​​of the corresponding area located on the opposite boundary of the picture can be copied to form the template area extending beyond the picture boundary.

[0426] Additionally, if a part of the MC area of ​​the reference block used for motion compensation extends beyond the picture boundary, the pixel values ​​of the corresponding area located on the opposite boundary of the picture may be copied to construct the MC area extending beyond the picture boundary.

[0427] For example, referring to (a) of Fig. 26, when the left area of ​​the template of the reference block (Ref. block) goes beyond the boundary of the reference picture, the pixel values ​​of the corresponding area (Wr) located at the right boundary of the reference picture can be copied to fill in the pixel values ​​of the left area of ​​the template that goes beyond the picture boundary.

[0428] For example, the left area of ​​the template marked “1234” can be matched with the area (Wr) marked “1234” located at the right boundary of the reference picture, so that pixel values ​​for the left area outside the picture boundary among the templates can be filled in.

[0429] As described above, the method of filling the template area outside the picture boundary using the corresponding area on the opposite boundary can be performed when a specific flag (e.g., wraparound_enabled_flag) is “on”.

[0430] wraparound_enabled_flag is a flag value that indicates related processing for ERP (equirectangular projection) among 360 video formats. When the flag value is “on,” the left boundary area of ​​the picture and the right boundary area of ​​the picture may belong to adjacent areas in the 3D real world as a characteristic of 360 videos according to the ERP video format. Accordingly, information about the left boundary area of ​​the picture and the right boundary area of ​​the picture may be shared with each other.

[0431] Here, the flag (wraparound_enabled_flag) is HLS (High-Level Syntax) and can be inserted into at least one of SPS (Sequence Parameter Set), PPS (Picture Parameter Set), PH (Picture Peader), and SH (Slice Header) and transmitted as a bitstream.

[0432] Meanwhile, the wraparound padding method according to the third embodiment may be the method with the highest priority among the embodiments for processing the picture boundary area.

[0433] Accordingly, even if a method according to another embodiment is applied, if the wraparound_enabled_flag value is “on”, the wraparound padding method according to the third embodiment can be performed.

[0434] In the third embodiment as described above, by using the mathematical expression 2 below, pixel values ​​of a corresponding area located at the opposite boundary of a reference picture as described above can be copied to fill in pixel values ​​of a template that are outside the picture boundary.

[0435]

[0436] In mathematical expression 2, x is the location information of a pixel included in the template, and xA is the updated location information for the pixel.

[0437] Here, if x is less than 0 or greater than (picWidth-1), it may mean that the corresponding pixel in the template is out of the picture boundary.

[0438] In this case, if x is less than 0, you can update the position of the pixel to the right boundary of the picture by adding picWidth to x, and if x is greater than (picWidth-1), you can update the position of the pixel to the left boundary of the picture by subtracting picWidth from x.

[0439] Meanwhile, as described above, pixel values ​​of the template that are outside the picture boundary may be filled in depending on the location of the updated pixel.

[0440]

[0441] As a fourth embodiment of a method for processing a picture boundary area, different padding methods can be used to process the boundary area of ​​a picture separately according to the importance of each boundary area.

[0442] For example, padding for areas beyond the left and right boundaries of a picture and padding for areas beyond the top and bottom boundaries of a picture can be handled separately.

[0443] Considering that the left and right boundary areas of a picture are treated as more important than the upper and lower boundary areas in image processing, motion compensated padding according to the second embodiment may be performed on the left and right boundary areas of the picture, and boundary pixel padding according to the first embodiment may be performed on the upper and lower boundary areas of the picture.

[0444] Additionally, since the importance of areas adjacent to the boundary of a picture and areas far from the boundary are different, padding for areas adjacent to the boundary of a picture and padding for areas far from the boundary of a picture can be processed separately.

[0445] For example, motion compensated padding or template matching padding according to the second embodiment may be performed for an area adjacent to the boundary of the picture, and boundary pixel padding according to the first embodiment may be performed for an area far from the boundary of the picture.

[0446] And, the area adjacent to the boundary of the picture is set to a preset value (m), and motion compensation padding (or template matching padding) is performed in the area below m from the boundary of the picture, and the area exceeding m from the boundary of the picture can be filled by copying pixel values ​​of the area m (located at the outermost part of the filled padding area) from the boundary of the picture using boundary picture padding. Here, m may be 16, for example, but the present invention is not limited thereto.

[0447] Meanwhile, for blocks located at the left border of the picture, the left area of ​​the template can be disabled so that template matching is performed using only the upper area of ​​the template.

[0448] Meanwhile, for blocks located at the upper boundary of the picture, the upper region of the template can be disabled so that template matching is performed using only the left region of the template.

[0449]

[0450] As a fifth embodiment of a method for handling a picture boundary area, sub-picture boundary padding may be used, which constructs an area beyond the boundary of a sub-picture by copying pixel values ​​closest to the boundary of the sub-picture.

[0451] Specifically, if some area of ​​the template extends beyond the subpicture boundary, the pixel values ​​closest to the boundary of the subpicture can be copied to form the template area extending beyond the subpicture boundary.

[0452] Additionally, if some of the MC areas of the reference block used for motion compensation extend beyond the subpicture boundary, the pixel values ​​closest to the boundary of the subpicture may be copied to form the MC area extending beyond the subpicture boundary.

[0453] For example, a template area that extends beyond the boundaries of a subpicture may be filled by copying pixel values ​​located at the very edge of that subpicture, and this may be done when a specific flag (e.g., subpic_treated_as_pic_flag) is “on” while using the subpicture.

[0454] Meanwhile, if the subpic_treated_as_pic_flag value is “off”, the template area outside the boundary of the subpicture can be filled by copying the pixel value located at the edge of another subpicture adjacent to the subpicture.

[0455] Here, subpic_treated_as_pic_flag can indicate whether the boundary of each subpicture can be processed independently when one picture is divided into multiple subpictures, and if the flag value is “on”, the subpicture can be encoded and decoded independently from other subpictures.

[0456] subpic_treated_as_pic_flag is HLS (High-Level Syntax) and can be transmitted as a bitstream by being inserted into at least one of SPS (Sequence Parameter Set), PPS (Picture Parameter Set), PH (Picture Peader), and SH (Slice Header).

[0457] As another example, when the subpic_treated_as_pic_flag value is “on”, the template area outside the boundary of the subpicture may be padded using the motion compensated padding or template matching padding method according to the second embodiment described above.

[0458] In the fifth embodiment, by using the mathematical expression 3 below, pixel values ​​located at the very edge of the subpicture can be copied as described above to fill in pixel values ​​of the template that are outside the subpicture boundary.

[0459]

[0460] In mathematical expression 3, x is the location information of a pixel included in the template, and xA is the updated location information for the pixel.

[0461] Here, if x is less than subPicLeftBoundaryPos, it may mean that the corresponding pixel in the template is outside the subpicture boundary.

[0462] In this case, if x is less than subPicLeftBoundaryPos, the position of the corresponding pixel can be updated to the position of subPicLeftBoundaryPos, and the value of the subPicLeftBoundaryPos-th pixel can be copied and used.

[0463]

[0464] Hereinafter, with reference to FIGS. 27 to 32, embodiments of a method of mixing and using two or more methods for processing a picture boundary area will be described.

[0465] First, the boundary pixel padding method according to the first embodiment and the motion compensated padding method according to the second embodiment may be used in combination depending on the situation.

[0466] Referring to FIG. 27, the mc_padding_enabled_flag value is checked (step S2700), and if it is "1", the area outside the picture boundary is filled using the motion compensation padding method according to the second embodiment (step S2710).

[0467] Meanwhile, if the mc_padding_enabled_flag value is not "1", the area outside the picture boundary is filled using the boundary pixel padding method according to the first embodiment (step S2720).

[0468] Additionally, the border pixel padding method according to the first embodiment and the wraparound padding method according to the third embodiment may be combined and used depending on the situation.

[0469] Referring to FIG. 28, the wraparound_enabled_flag value is checked (step S2800), and if it is "1", the area outside the picture boundary is filled using wraparound padding according to the third embodiment (step S2810).

[0470] Meanwhile, if the wraparound_enabled_flag value is not "1", the area outside the picture boundary is filled using the boundary pixel padding method according to the first embodiment (step S2820).

[0471] And the motion compensation padding method according to the second embodiment and the wraparound padding method according to the third embodiment can be mixed and used depending on the situation.

[0472] Referring to FIG. 29, the wraparound_enabled_flag value is checked (step S2900), and if it is "1", the area outside the picture boundary is filled using wraparound padding according to the third embodiment (step S2910).

[0473] Meanwhile, if the wraparound_enabled_flag value is not "1", the area outside the picture boundary is filled using the motion compensation padding method according to the second embodiment (step S2920).

[0474] The boundary pixel padding method according to the first embodiment, the motion compensation padding method according to the second embodiment, and the wraparound padding method according to the third embodiment may be mixed and used depending on the situation.

[0475] Referring to FIG. 30, the wraparound_enabled_flag value is checked (step S3000), and if it is "1", the area outside the picture boundary is filled using wraparound padding according to the third embodiment (step S3010).

[0476] Meanwhile, if the wraparound_enabled_flag value is not "1", the mc_padding_enabled_flag value is checked (step S3020).

[0477] Here, if the mc_padding_enabled_flag value is "1", the area outside the picture boundary is filled using the motion compensation padding method according to the second embodiment (step S3030), and if it is not "1", the area outside the picture boundary is filled using the boundary pixel padding method according to the first embodiment (step S3040).

[0478] And the boundary pixel padding method according to the first embodiment, the wraparound padding method according to the third embodiment, and the sub-picture boundary padding method according to the fifth embodiment may be mixed and used depending on the situation.

[0479] Referring to FIG. 31, the subpic_treated_as_pic_flag value is checked (step S3100), and if it is "1", the wraparound_enabled_flag value is checked (step S3110).

[0480] Here, if the wraparound_enabled_flag value is "1", the wraparound padding method according to the third embodiment is used (step S3120), and if it is not "1", the sub-picture boundary padding method according to the fifth embodiment is used (step S3130).

[0481] Meanwhile, if the subpic_treated_as_pic_flag value is not "1", the wraparound_enabled_flag value is checked (step S3140), and if the wraparound_enabled_flag value is "1", the wraparound padding method according to the third embodiment is used (step S3120), and if it is not "1", the border pixel padding method according to the first embodiment is used (step S3150).

[0482] As another example, referring to FIG. 32, the wraparound_enabled_flag value is checked (step S3200), and if it is "1", the wraparound padding method according to the third embodiment is used (step S3210).

[0483] If the wraparound_enabled_flag value is not "1", the subpic_treated_as_pic_flag value is checked (step S3220).

[0484] Here, if the subpic_treated_as_pic_flag value is "1", the sub-picture boundary padding method according to the fifth embodiment is used (step S3230), and if it is not "1", the boundary pixel padding method according to the first embodiment is used (step S3240).

[0485]

[0486] In embodiments for padding an area outside the picture boundary area as described above, the padding area may be filled with pixel values ​​of different restoration areas, and thus a blocking phenomenon may occur between areas within the padding area.

[0487] For example, in the case illustrated in FIG. 25, blocking may occur between the first padding area (P1) and the second padding area (P2) that are filled using different areas (P1', P2').

[0488] To reduce blocking of such padding areas, an additional smoothing or blending process may be performed on the boundaries between areas filled with pixel values ​​from different restoration areas.

[0489]

[0490] Meanwhile, the padding process of padding an area outside the picture boundary according to the embodiments described above may be included as part of the picture restoration process.

[0491] Referring to the block diagram of the decoding device illustrated in Fig. 6, a prediction sample value from the prediction unit (260) is added to the residual sample value generated through the entropy decoding unit (210), the inverse quantization unit (220), and the inverse transformation unit (230) processes to generate a primary restored picture, which is a reconstructed pixel value. Here, the size of the primary restored picture may be the same as the width and height of the original picture.

[0492] For the primary restoration picture generated as described above, a padding process according to an embodiment of the present invention for padding an area outside the picture boundary can be applied.

[0493] After the padding process, the size of the restored picture can be set to a size corresponding to the width of “picture width + padding area” and the height of “picture height + padding area”.

[0494] Next, in the filtering unit (240), one or more filters among DBF (De-Blocking Filter), SAO (Sample Adaptive Offset), and ALF (Adaptive Loop Filter) are applied to generate a final restored picture.

[0495] At this time, the filtering unit (240) can apply one or more filters as described above to the padding area outside the picture boundary so as to reduce the blocking phenomenon in the padding area.

[0496] And the final restored picture is stored in DPB (270) and used as prediction information for the next picture.

[0497] As another example, for a primary restored picture in which a prediction sample value is converted to a residual sample value, after the filtering process of the filtering unit (240) is performed, a padding process according to an embodiment of the present invention may be applied so that the final restored picture may be stored in the DPB (270).

[0498] Here, the entropy decoding unit (210) operates at the bitstream level, and the inverse quantization process of the inverse quantization unit (220), the inverse transformation process of the inverse transformation unit (230), the prediction process of the prediction unit (260), and the process of adding the prediction sample value to the residual sample value can operate at the block level.

[0499] Meanwhile, the padding process, the filtering process of the filtering unit (240), and the storage process in the DPB (270) according to the embodiment of the present invention can operate at the picture (or slice) level.

[0500]

[0501] The methods described herein may be performed by a processor of a video encoder or video decoder. Furthermore, the encoder may generate a bitstream that is decoded by a video signal processing method, and the bitstream generated by the encoder may be stored in a computer-readable, non-transitory storage medium (recording medium).

[0502] The embodiments of the present invention described above may be implemented through various means. For example, the embodiments of the present invention may be implemented using hardware, firmware, software, or a combination thereof.

[0503] Some embodiments may also be implemented in the form of a computer-executable storage medium containing computer-executable instructions, such as program modules executed by a computer. Computer-readable media may be any available media that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media.

[0504] Additionally, computer-readable media may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. Communication media typically includes other data, such as computer-readable instructions, data structures, or program modules, in a modulated data signal, or other transport mechanism, and includes any information delivery media.

[0505] In addition, although the preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and various modifications can be made by a person having ordinary skill in the art to which the invention pertains without departing from the gist of the present invention as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present invention.

Claims

1. In a method for processing an area outside the picture boundary, A step of obtaining first vector information for a first reference block when at least a part of a surrounding area of ​​a first reference block used to perform prediction for a target block is located outside a picture boundary; A step of confirming a second reference block referenced by the first reference block according to the first vector information; and A method for processing a picture boundary area, comprising: a step of constructing a surrounding area of ​​a first reference block located outside the picture boundary using surrounding samples of the second reference block; 2. In the first paragraph, the surrounding area of ​​the first reference block A method for processing a picture boundary area including surrounding samples of the first reference block used for template matching with the target block.

3. In the first paragraph, the surrounding area of ​​the first reference block A method for processing a picture boundary region including surrounding samples of the first reference block used to perform motion compensation for the target block.

4. In paragraph 1, A picture boundary area processing method further comprising: a step of constructing a surrounding area of ​​a first reference block located outside the picture boundary by using samples closest to the picture boundary among the samples in the first reference block when the first vector information does not exist; 5. In paragraph 1, A picture boundary area processing method further comprising: a step of constructing a surrounding area of ​​a first reference block located outside the picture boundary by using samples closest to the picture boundary among the samples in the first reference block when the target block is located within an I slice (intra slice); 6. In paragraph 1, A step of deriving a first matching block having the highest similarity to the first reference block within the same picture; and A picture boundary area processing method further comprising: a step of constructing a surrounding area of ​​a first reference block located outside the picture boundary using surrounding samples of the first matching block.

7. An image encoding device that performs the picture boundary area processing method of paragraph 1.

8. In paragraph 7, A video encoding device that inserts a flag (falg) for indicating a method of processing the above picture boundary area into at least one of SPS (Sequence Parameter Set), PPS (Picture Parameter Set), PH (Picture Header), and SH (Slice Header) and transmits it as a bitstream.

9. A bitstream generated by the video encoding device of Article 7.

10. An image decoding device that performs the picture boundary area processing method of paragraph 1.

Images