Method for configuring a candidate list for intra block copy and apparatus using the same

Abstract

A method for configuring a candidate list for intra block copy and an apparatus using the same are disclosed. According to the disclosed method, as part of constructing an IBC candidate list for a current block, a video encoder and a video decoder: divide samples of a template of the current block into a first subset and a second subset; derive potential candidates by performing template matching with the first subset; and select at least one template-based candidate from the potential candidates to include in the IBC candidate list based on differences between samples of the second subset and corresponding samples of a reference template indicated by each potential candidate.

Description

Technical Field

[0001] This invention describes a technique for configuring a merge / AMVP candidate list under intra-block copy (IBC). Block vectors can be predictively encoded and decoded, and the encoding / decoding gain of the block vectors depends on the diversity of candidates in the candidate list. Background Technology

[0002] The statements in this section are provided only as background information in relation to the present invention and do not necessarily constitute prior art.

[0003] Because of the large data volume compared to audio or still image data, unprocessed or uncompressed video data requires significant hardware resources (including memory) to store or transmit it in its raw form.

[0004] Therefore, when storing or transmitting video data, an encoder is typically used to compress the video data for storage or transmission, and a decoder receives the compressed video data and performs decompression and reconstruction. Such video compression techniques include H.264 / AVC, High Efficiency Video Coding (HEVC), and Versatile Video Coding (VVC), with VVC offering approximately 30% or more improvement in coding efficiency compared to HEVC.

[0005] However, as video size, resolution, and frame rate continue to increase, and the amount of data that needs to be encoded and decoded also increases accordingly, a new compression technology is needed that is more efficient in encoding and decoding and more efficient in improving video quality than existing compression technologies. Summary of the Invention

[0006] Technical issues This invention describes a technique for configuring a merge / AMVP candidate list under intra-block copy (IBC). Block vectors can be predictively encoded and decoded, and the encoding / decoding gain of the block vectors depends on the diversity of candidates in the candidate list.

[0007] Technical solution At least one aspect of the present invention provides a method for decoding video data. The method includes: determining a current block to be decoded using intra-block copying (IBC); constructing an IBC candidate list for the current block; decoding an indication from a bitstream representing a candidate selected from the IBC candidate list; and reconstructing the current block using the selected candidate. Constructing the IBC candidate list for the current block includes: dividing samples of a template of the current block into a first subset and a second subset; performing template matching using the first subset to derive potential candidates; and selecting at least one template-based candidate from the potential candidates to be included in the IBC candidate list based on the difference between samples of the second subset and corresponding samples of a reference template indicated by the potential candidates.

[0008] Another aspect of the present invention provides a method for encoding video data. The method includes: determining a current block to be encoded using intra-block copying (IBC); constructing an IBC candidate list for the current block; encoding an indication representing a candidate selected from the IBC candidate list; and encoding the current block using the selected candidate. Constructing the IBC candidate list for the current block further includes: dividing samples of a template of the current block into a first subset and a second subset; performing template matching using the first subset to derive potential candidates; and selecting at least one template-based candidate from the potential candidates to be included in the IBC candidate list based on the difference between samples of the second subset and corresponding samples of a reference template indicated by a potential candidate.

[0009] Another aspect of the present invention provides a method for providing video data to a video decoding apparatus. The method includes: encoding video data into a bitstream; and transmitting the bitstream to the video decoding apparatus. Encoding video data into a bitstream includes: determining a current block to be encoded using intra-block copying (IBC); constructing an IBC candidate list for the current block; encoding an indication of a candidate selected from the IBC candidate list; and encoding the current block using the selected candidate.

[0010] Beneficial effects According to an embodiment of the present invention, the coding efficiency of block vectors can be improved by enhancing the diversity of candidates in the candidate list. Attached Figure Description

[0011] Figure 1 This is a block diagram of a video encoding device that can implement the technology of the present invention.

[0012] Figure 2 This paper demonstrates a method for segmenting blocks using a quadtree plus binary tree ternary tree (QTBTTT) structure.

[0013] Figure 3a and Figure 3b Several intra-prediction modes, including a wide-angle intra-prediction mode, are shown.

[0014] Figure 4 The adjacent blocks of the current block are shown.

[0015] Figure 5 This is a block diagram of a video decoding device that can implement the technology of the present invention.

[0016] Figure 6 This is a flowchart illustrating a method for configuring an IBC candidate list according to an embodiment of the present invention.

[0017] Figure 7 This is a conceptual diagram showing spatially non-adjacent blocks positioned along different directions.

[0018] Figure 8a and Figure 8b This is a conceptual diagram illustrating a method for deriving Automatic Relocation Block Vector Prediction (AR-BVP) that can be used in embodiments of the present invention.

[0019] Figure 9a and Figure 9b This is a conceptual diagram showing the various shapes of templates that can be considered for template matching.

[0020] Figure 10a and Figure 10b It is shown that by for Figure 9a The “L”-shaped template shown can be considered as a conceptual diagram of the partitions or patterns formed by two sample subsets.

[0021] Figure 11 This is a conceptual diagram illustrating a set of boundary-based candidates that can be used in embodiments of the present invention.

[0022] Figure 12 This is a conceptual diagram illustrating the maximum template area that can be configured for the current block according to an embodiment of the present invention.

[0023] Figure 13a , Figure 13b and Figure 13c This is a conceptual diagram showing template-based candidate templates that can be used to derive templates for two sub-regions through geometric partitioning.

[0024] Figure 14 This is a flowchart illustrating a method for a video decoder to decode video data according to an aspect of the present invention. Detailed Implementation

[0025] In the following description, some embodiments of the invention are described in detail with reference to the accompanying illustrative drawings. In the following description, the same reference numerals denote the same elements, although these elements are shown in different drawings. Furthermore, in the following description of some embodiments, detailed descriptions of relevant known components and functions may be omitted for clarity and brevity when they are considered to obscure the subject matter of the invention.

[0026] Figure 1 This is a block diagram of a video encoding apparatus that can implement the technology of the present invention. In the following, reference is made to... Figure 1 The diagram illustrates a video encoding device and its components.

[0027] The encoding device may include: an image segmenter 110, a predictor 120, a subtractor 130, a transformer 140, a quantizer 145, a reordering unit 150, an entropy encoder 155, an inverse quantizer 160, an inverse transformer 165, an adder 170, a loop filter unit 180, and a memory 190.

[0028] Each component of the encoding device can be implemented as hardware or software, or a combination of hardware and software. Furthermore, the function of each component can be implemented as software, and the microprocessor can be implemented to execute the software function corresponding to each component.

[0029] A video consists of one or more sequences of images. Each image is segmented into multiple regions, and encoding is performed on each region. For example, an image is segmented into one or more tiles and / or slices. Here, one or more tiles can be defined as a tile group. Each tile and / or slice is segmented into one or more coding tree units (CTUs). Additionally, each CTU is segmented into one or more coding units (CUs) through a tree structure. Information applied to each coding unit (CU) is encoded as the syntax of the CU, and information commonly applied to CUs included in a CTU is encoded as the syntax of the CTU. Furthermore, information commonly applied to all blocks in a slice is encoded as the syntax of the slice header, and information applied to all blocks constituting one or more images is encoded as a picture parameter set (PPS) or picture header. Additionally, information commonly referenced by multiple images is encoded as a sequence parameter set (SPS). Furthermore, information commonly referenced by one or more SPSs is encoded as a video parameter set (VPS). Furthermore, information commonly applied to a tile or tile group can also be encoded as the syntax of the tile or tile group header. The syntax included in SPS, PPS, slice headers, and tile or tile group headers can be called high-level syntax.

[0030] Image splitter 110 determines the size of the coding tree unit (CTU). Information about the size of the CTU (CTU size) is encoded into SPS or PPS syntax and transmitted to the video decoding device.

[0031] Image segmenter 110 divides each image constituting the video into multiple coding tree units (CTUs) of a predetermined size, and then recursively segments the CTUs using a tree structure. The leaf nodes in the tree structure are called coding units (CUs), which are the basic units of encoding.

[0032] A tree structure can be a quadtree (QT) that divides a higher node (or parent node) into four lower nodes (or child nodes) of the same size. A tree structure can also be a binary tree (BT) that divides a higher node into two lower nodes. A tree structure can also be a ternary tree (TT) that divides a higher node into three lower nodes in a 1:2:1 ratio. A tree structure can also be a combination of two or more of the QT, BT, and TT structures. For example, a quadtree plus binary tree (QTBT) structure can be used, or a quadtree plus binary tree ternary tree (QTBTTT) structure can be used. Here, a binary tree ternary tree (BTTT) is added to the tree structure to form a multiple-type tree (MTT).

[0033] Figure 2 This is a schematic diagram illustrating a method for segmenting blocks using a QTBTT structure.

[0034] like Figure 2 As shown, the CTU can first be segmented into a QT structure. Quadtree segmentation can be recursive until the size of the segmented blocks reaches the minimum block size (MinQTSize) allowed for leaf nodes in the QT. The entropy encoder 155 encodes a first flag (QT_split_flag) indicating whether each node of the QT structure is segmented into the four nodes below it and signals this flag to the video decoding device. When the leaf node of the QT is not larger than the maximum block size (MaxBTSize) allowed for the root node in the BT, the leaf node can be further segmented into at least one of a BT structure or a TT structure. Multiple segmentation directions can exist in the BT structure and / or TT structure. For example, two directions can exist: a horizontal direction for segmenting the blocks of the corresponding node and a vertical direction for segmenting the blocks of the corresponding node. Figure 2 As shown, when MTT splitting begins, the entropy encoder 155 encodes a second flag (mtt_split_flag) indicating whether a node has been split, as well as a flag indicating the splitting direction (vertical or horizontal) and / or the splitting type (binary or ternary) if the node has been split, and signals this to the video decoding device.

[0035] Alternatively, before encoding the first flag (QT_split_flag) indicating whether each node has been split into the four lower-level nodes, the CU split flag (split_cu_flag) indicating whether a node has been split can also be encoded. When the value of the CU split flag (split_cu_flag) indicates that each node has not been split, the block of the corresponding node becomes a leaf node in the split tree structure and becomes a CU, which is the basic unit of encoding. When the value of the CU split flag (split_cu_flag) indicates that each node has been split, the video encoding device begins encoding the first flag using the above scheme.

[0036] When QTBT is used as another example of a tree structure, two types can exist: one where the corresponding node's block is horizontally divided into two blocks of the same size (i.e., symmetrical horizontal division), and another where the corresponding node's block is vertically divided into two blocks of the same size (i.e., symmetrical vertical division). The entropy encoder 155 encodes a split flag (split_flag) indicating whether each node of the BT structure is divided into blocks of the lower layer, and split type information indicating the split type, and transmits this information to the video decoding device. Alternatively, the type where the corresponding node's block is divided into two blocks that are asymmetrically divided can also exist. Asymmetrical forms can include dividing the corresponding node's block into two rectangular blocks with a size ratio of 1:3, or it can also include dividing the corresponding node's block along a diagonal direction.

[0037] The CU can have various sizes depending on the QTBT or QTBTTT segmentation from the CTU. In the following text, the block corresponding to the CU to be encoded or decoded (i.e., the leaf node of the QTBTTT) is called the "current block". When using QTBTTT segmentation, the current block can also be rectangular in shape in addition to a square shape.

[0038] Predictor 120 predicts the current block to generate a prediction block. Predictor 120 includes an intra-frame predictor 122 and an inter-frame predictor 124.

[0039] Typically, each current block in an image can be predictively encoded. This prediction can usually be performed using either intra-frame prediction (using data from the image containing the current block) or inter-frame prediction (using data from an image encoded before the image containing the current block). Inter-frame prediction includes both one-way and two-way prediction.

[0040] Intra-predictor 122 predicts pixels in the current block by utilizing pixels (reference pixels) that are adjacent to the current block in the current image, including the current block. Depending on the prediction direction, there are various intra-prediction modes. For example, such as... Figure 3aAs shown, various intra-frame prediction modes can include two non-directional modes (which include a planar mode and a DC mode) and can include 65 directional modes. The neighboring pixels and algorithm equations to be used are defined differently for each prediction mode.

[0041] To perform efficient orientation prediction for the current block with a rectangular shape, additional methods such as... can be used. Figure 3b The dashed arrows indicate the directional modes (intra-prediction modes #67 to #80, #-1 to #-14). These directional modes can be referred to as "wide-angle intra-prediction modes." Figure 3b In the diagram, the arrows indicate the corresponding reference samples used for prediction, not the prediction direction. The prediction direction is opposite to the direction indicated by the arrow. When the current block has a rectangular shape, the wide-angle intra-frame prediction mode is a mode that performs prediction in the opposite direction to the specific direction mode without additional bit transmission. In this case, in the wide-angle intra-frame prediction mode, some wide-angle intra-frame prediction modes available for the current block can be determined by the ratio of the width to the height of the current block with a rectangular shape. For example, when the current block has a rectangular shape with a height less than its width, wide-angle intra-frame prediction modes with angles less than 45 degrees (intra-frame prediction modes #67 to #80) are available. When the current block has a rectangular shape with a width greater than its height, wide-angle intra-frame prediction modes with angles greater than -135 degrees (intra-frame prediction modes #-1 to #-14) are available.

[0042] Intra-predictor 122 can determine the intra-prediction to be used for encoding the current block. In some examples, intra-predictor 122 can encode the current block using multiple intra-prediction modes, and can also select the appropriate intra-prediction mode to use from test modes. For example, intra-predictor 122 can calculate rate-distortion values ​​by utilizing rate-distortion analysis of multiple test intra-prediction modes, and can also select the intra-prediction mode with the best rate-distortion characteristics from the test modes.

[0043] Intra-predictor 122 selects one intra-prediction mode from a variety of intra-prediction modes and predicts the current block using neighboring pixels (reference pixels) determined according to the selected intra-prediction mode and an algorithm equation. Entropy encoder 155 encodes the information about the selected intra-prediction mode and transmits it to the video decoding device.

[0044] Inter-frame predictor 124 generates a predicted block for the current block by utilizing motion compensation processing. Inter-frame predictor 124 searches for the most similar block to the current block in a reference image that was encoded and decoded earlier than the current image, and uses the searched block to generate the predicted block for the current block. Additionally, a motion vector (MV) is generated, corresponding to the displacement between the current block in the current image and the predicted block in the reference image. Typically, motion estimation is performed on the luminance component, and the motion vector calculated based on the luminance component is used for both the luminance and chrominance components. The motion information, including information about the reference image and information about the motion vector used to predict the current block, is encoded by entropy encoder 155 and transmitted to the video decoding device.

[0045] The inter-frame predictor 124 can also perform interpolation on a reference image or reference block to increase prediction accuracy. In other words, subsamples are interpolated between two consecutive integer samples by applying filter coefficients to multiple consecutive integer samples, including two integer samples. When performing the process of searching for the most similar block to the current block on the interpolated reference image, the motion vector can be represented with fractional-unit precision rather than integer-sample-unit precision. For each target region to be encoded, such as a cell like a slice, tile, CTU, or CU, the precision or resolution of the motion vector can be set differently. When applying this adaptive motion vector resolution (AMVR), information about the motion vector resolution to be applied to each target region should be signaled for each target region. For example, when the target region is a CU, information about the motion vector resolution applied to each CU is signaled. The information about the motion vector resolution can be information representing the precision of the motion vector difference, as described below.

[0046] On the other hand, the inter-frame predictor 124 can perform inter-frame prediction using bidirectional prediction. In the case of bidirectional prediction, two reference images and two motion vectors representing the position of the block most similar to the current block in each reference image are used. The inter-frame predictor 124 selects a first reference image and a second reference image from reference image list 0 (RefPicList0) and reference image list 1 (RefPicList1), respectively. The inter-frame predictor 124 also searches for the block most similar to the current block in each reference image to generate a first reference block and a second reference block. In addition, the predicted block of the current block is generated by averaging or weighted averaging the first reference block and the second reference block. In addition, motion information including information about the two reference images used to predict the current block and information about the two motion vectors is transmitted to the entropy encoder 155. Here, reference image list 0 may consist of images in pre-reconstructed images that are displayed before the current image in display order, and reference image list 1 may consist of images in pre-reconstructed images that are displayed after the current image in display order. However, although not particularly limited to this, pre-reconstructed images that are displayed after the current image in display order may be additionally included in reference image list 0. Conversely, pre-reconstructed images preceding the current image can also be additionally included in the reference image list 1.

[0047] Various methods can be used to minimize the number of bits consumed in encoding motion information.

[0048] For example, when the reference image and motion vector of the current block are the same as those of the adjacent blocks, the information that can identify the adjacent blocks is encoded to transmit the motion information of the current block to the video decoding device. This method is called merge mode.

[0049] In merge mode, the inter-frame predictor 124 selects a predetermined number of merge candidate blocks (hereinafter referred to as "merge candidates") from the neighboring blocks of the current block.

[0050] As adjacent blocks used to derive merge candidates, all or some of the following can be used: left block A0, lower left block A1, upper block B0, upper right block B1, and upper left block B2, which are adjacent to the current block in the current image. Figure 4 As shown. In addition to the current image containing the current block, blocks located within a reference image (which may be the same as or different from the reference image used to predict the current block) can also be used as merge candidates. For example, co-located blocks within the reference image or blocks adjacent to that co-located block can be additionally used as merge candidates. If the number of merge candidates selected using the above method is less than a preset number, a zero vector is added to the merge candidates.

[0051] The inter-frame predictor 124 configures a merge list including a predetermined number of merge candidates by utilizing neighboring blocks. It selects a merge candidate from the merge candidates included in the merge list to be used as motion information for the current block, and generates merge index information for identifying the selected candidate. The generated merge index information is encoded by the entropy encoder 155 and transmitted to the video decoding device.

[0052] The merge skip mode is a special case of the merge mode. After quantization, when all transform coefficients used for entropy coding are close to zero, only the adjacent block selection information is transmitted, and the residual signal is not transmitted. By utilizing the merge skip mode, relatively high coding efficiency can be achieved for images with slight motion, still images, and screen content images.

[0053] From then on, merge mode and merge skip mode were collectively referred to as merge / skip mode.

[0054] Another approach for encoding motion information is the Advanced Motion Vector Prediction (AMVP) model.

[0055] In AMVP mode, the inter-frame predictor 124 derives motion vector prediction candidates for the current block by utilizing neighboring blocks of the current block. As neighboring blocks used to derive motion vector prediction candidates, they can use... Figure 4 The current block in the current image shown can be any or some of the following adjacent blocks: left block A0, lower left block A1, upper block B0, upper right block B1, and upper left block B2. Furthermore, blocks located in a reference image (which may be the same as or different from the reference image used to predict the current block) can also be used as neighboring blocks for deriving motion vector prediction candidates. For example, a co-located block of the current block in the reference image or a block adjacent to that co-located block can be used. If the number of motion vector candidates selected by the above method is less than a preset number, a zero vector is added to the motion vector candidates.

[0056] The inter-frame predictor 124 derives motion vector prediction candidates by utilizing motion vectors from neighboring blocks, and uses these candidates to determine the motion vector prediction for the current block. Furthermore, it calculates the motion vector difference by subtracting the motion vector prediction from the motion vector prediction for the current block.

[0057] Motion vector predictions can be obtained by applying predefined functions (e.g., median and average calculations) to motion vector prediction candidates. In this case, the video decoding device also knows the predefined functions. Furthermore, since the neighboring blocks used to derive the motion vector prediction candidates are blocks that have already been encoded and decoded, the video decoding device may also already know the motion vectors of the neighboring blocks. Therefore, the video encoding device does not need to encode the information used to identify the motion vector prediction candidates. Accordingly, in this case, information about the motion vector difference and information about the reference image used to predict the current block are encoded.

[0058] Alternatively, motion vector prediction can be determined by selecting any of the motion vector prediction candidates. In this case, additional encoding is performed on the information used to identify the selected motion vector prediction candidate, as well as the information about the motion vector difference and the reference image used to predict the current block.

[0059] Predictor 120 can further utilize Intra-Block Copy (IBC) mode. In IBC mode, predictor 120 searches for a prediction block in a frame or picture that includes the current block, as in intra-prediction mode; however, predictor 120 can search a wider search area and adjacent rows and columns of pixels. In IBC mode, predictor 120 can determine a block vector (also called a motion vector) to identify the prediction block within the same frame or picture as the predicted block. The block vector includes an x-component and a y-component, where the x-component identifies the horizontal displacement between the current block being predicted and the prediction block, and the y-component identifies the vertical displacement between the video block being predicted and the prediction block. The determined block vector can be signaled in the bitstream in a manner similar to a motion vector, allowing the video decoding device to identify the same prediction block selected by the video encoding device.

[0060] Subtractor 130 generates a residual block by subtracting the predicted block generated by intra-predictor 122 or inter-predictor 124 from the current block.

[0061] Transformer 140 transforms the residual signal in the residual block, which has pixel values ​​in the spatial domain, into transform coefficients in the frequency domain. Transformer 140 can transform the residual signal in the residual block by using the entire size of the residual block as a transform unit, or it can divide the residual block into multiple sub-blocks and perform the transformation by using the sub-blocks as transform units. Alternatively, the residual block is divided into two sub-blocks (a transformed region and a non-transformed region) to transform the residual signal using only the transformed region sub-block as a transform unit. Here, the transformed region sub-block can be one of two rectangular blocks with a size ratio of 1:1 based on a horizontal axis (or a vertical axis). In this case, the entropy encoder 155 encodes a flag (cu_sbt_flag) indicating that only the transformed sub-block is used, as well as orientation (vertical / horizontal) information (cu_sbt_horizontal_flag) and / or position information (cu_sbt_pos_flag), and signals this information to the video decoding device. Furthermore, the size of the transformed region sub-blocks can be based on a 1:3 size ratio along the horizontal (or vertical) axis. In this case, the entropy encoder 155 additionally encodes the flag (cu_sbt_quad_flag) for dividing the corresponding segment and signals it to the video decoding device.

[0062] On the other hand, the transformer 140 can perform transformations on the residual blocks individually in the horizontal and vertical directions. Various types of transformation functions or transformation matrices can be used for the transformations. For example, a pair of transformation functions for horizontal and vertical transformations can be defined as a multiple transform set (MTS). The transformer 140 can select the pair of transformation functions with the highest transformation efficiency in the MTS and can transform the residual blocks in each of the horizontal and vertical directions. The entropy encoder 155 encodes the information (mts_idx) about the pair of transformation functions in the MTS and signals it to the video decoding device.

[0063] Quantizer 145 quantizes the transform coefficients output from transformer 140 using quantization parameters and outputs the quantized transform coefficients to entropy encoder 155. Quantizer 145 can also quantize the relevant residual blocks immediately without transforming any block or frame. Quantizer 145 can also apply different quantization coefficients (scaling values) based on the position of the transform coefficients in the transform block. The quantization matrix applied to the quantized transform coefficients arranged in two dimensions can be encoded and signaled to the video decoding device.

[0064] The reordering unit 150 can perform a rearrangement of the coefficient values ​​on the quantized residual values.

[0065] The reordering unit 150 can transform a 2D coefficient array into a 1D coefficient sequence using coefficient scanning. For example, the reordering unit 150 can output a 1D coefficient sequence by scanning the DC coefficients to the high-frequency region coefficients using a zig-zag scan or a diagonal scan. Depending on the size of the transform unit and the intra-frame prediction mode, a vertical scan that scans the 2D coefficient array along the column direction and a horizontal scan that scans the 2D block-type coefficients along the row direction can also be used instead of a zig-zag scan. In other words, the scanning method to be used can be determined from zig-zag scan, diagonal scan, vertical scan, and horizontal scan, depending on the size of the transform unit and the intra-frame prediction mode.

[0066] The entropy encoder 155 generates a bitstream by encoding the 1D quantized transform coefficient sequence output from the reordering unit 150 using various encoding schemes, including Context-based Adaptive Binary Arithmetic Code (CABAC) and Exponential Golomb.

[0067] Furthermore, the entropy encoder 155 encodes block-segmentation related information (e.g., CTU size, CTU segmentation flag, QT segmentation flag, MTT segmentation type, MTT segmentation direction, etc.) so that the video decoding device can segment blocks equivalently to the video encoding device. Additionally, the entropy encoder 155 encodes information indicating whether the current block is encoded using intra-frame prediction or inter-frame prediction. The entropy encoder 155 encodes intra-frame prediction information (i.e., information about the intra-frame prediction mode) or inter-frame prediction information (the merge index in the case of merge mode, and information about the reference image index and motion vector difference in the case of AMVP mode) according to the prediction type. Furthermore, the entropy encoder 155 encodes quantization-related information (i.e., information about quantization parameters and information about the quantization matrix).

[0068] Inverse quantizer 160 inverse quantizes the quantized transform coefficients output from quantizer 145 to generate transform coefficients. Inverse transformer 165 transforms the transform coefficients output from inverse quantizer 160 from the frequency domain to the spatial domain to reconstruct the residual block.

[0069] Adder 170 adds the reconstructed residual block to the predicted block generated by predictor 120 to reconstruct the current block. When performing intra-frame prediction for the next sequential block, the pixels in the reconstructed current block can be used as reference pixels.

[0070] The loop filtering unit 180 performs filtering on the reconstructed pixels to reduce blocking artifacts, ringing artifacts, blurring artifacts, etc., that occur due to block-based prediction and transform / quantization. The loop filtering unit 180, as an in-loop filter, may include all or some of the following: a deblocking filter 182, a sample adaptive offset (SAO) filter 184, and an adaptive loop filter (ALF) 186.

[0071] Deblocking filter 182 filters the boundaries between reconstructed blocks to remove blocking artifacts caused by block unit encoding / decoding, and SAO filter 184 and ALF 186 perform additional filtering on the deblocked video. SAO filter 184 and ALF 186 are filters used to compensate for the difference between reconstructed pixels and original pixels caused by lossy coding. SAO filter 184 applies an offset in CTU units to enhance subjective image quality and coding efficiency. On the other hand, ALF 186 performs block unit filtering and applies different filters to compensate for distortion by dividing the boundaries of the corresponding blocks and the degree of variation. Information about the filter coefficients to be used for ALF can be encoded and signaled to the video decoding device.

[0072] The reconstructed blocks filtered by deblocking filter 182, SAO filter 184, and ALF 186 are stored in memory 190. When all blocks in an image are reconstructed, the reconstructed image can be used as a reference image for inter-frame prediction of blocks in subsequent images to be encoded.

[0073] Video encoding devices can store the bitstream of encoded video data in a non-volatile storage medium or send the bitstream to a video decoding device via a communication network.

[0074] Figure 5 This is a functional block diagram of a video decoding device that can implement the technology of the present invention. In the following text, reference is made to... Figure 5 This describes a video decoding device and its components.

[0075] The video decoding device may include: an entropy decoder 510, a reordering unit 515, an inverse quantizer 520, an inverse transformer 530, a predictor 540, an adder 550, a loop filter unit 560, and a memory 570.

[0076] Similar to Figure 1The video encoding device and the video decoding device each component can be implemented as hardware or software, or a combination of hardware and software. Furthermore, the function of each component can be implemented as software, and the microprocessor can also be implemented to execute the software functions corresponding to each component.

[0077] The entropy decoder 510 extracts information related to block segmentation by decoding the bitstream generated by the video encoding device to determine the current block to be decoded, and extracts the prediction information and information about the residual signal required to reconstruct the current block.

[0078] The entropy decoder 510 determines the size of the CTU by extracting information about the CTU size from the sequence parameter set (SPS) or the picture parameter set (PPS), and segments the image into CTUs of a defined size. Furthermore, the CTU is identified as the highest level (i.e., the root node) of the tree structure, and segmentation information for the CTU can be extracted to segment the CTU by utilizing the tree structure.

[0079] For example, when segmenting a CTU using a QTBTT structure, the first flag (QT_split_flag) related to the QT segmentation is extracted to divide each node into four lower-level nodes. Additionally, for nodes corresponding to leaf nodes of the QT, the second flag (mtt_split_flag), segmentation direction (vertical / horizontal), and / or segmentation type (binary / ternary) related to the MTT segmentation are extracted to segment the corresponding leaf node using the MTT structure. Thus, each node below a leaf node of the QT is recursively segmented using either a BT or TT structure.

[0080] As another example, when splitting the CTU using the QTBTTT structure, a CU split flag (split_cu_flag) indicating whether the CU has been split is extracted. A first flag (QT_split_flag) can also be extracted when the corresponding block is split. During the splitting process, for each node, zero or more recursive MTT splits may occur after zero or more recursive QT splits. For example, with respect to the CTU, MTT splits may occur immediately, or conversely, only multiple QT splits may occur.

[0081] As another example, when segmenting a CTU using a QTBT structure, the first flag (QT_split_flag) associated with the QT segmentation is extracted to split each node into four nodes at the lower level. Additionally, a split flag (split_flag) indicating whether the node corresponding to a leaf node of the QT is further segmented using BT, as well as segmentation direction information, is extracted.

[0082] On the other hand, when the entropy decoder 510 determines the current block to be decoded by utilizing the segmentation of the tree structure, the entropy decoder 510 extracts information about the prediction type indicating whether the current block is predicted intra-frame or inter-frame. When the prediction type information indicates intra-frame prediction, the entropy decoder 510 extracts the syntax elements for the intra-frame prediction information (intra-frame prediction mode) for the current block. When the prediction type information indicates inter-frame prediction, the entropy decoder 510 extracts information representing the syntax elements for the inter-frame prediction information, namely, the motion vector and the reference picture for the motion vector reference.

[0083] In addition, the entropy decoder 510 extracts quantization-related information and extracts information about the quantization transform coefficients of the current block as information about the residual signal.

[0084] The reordering unit 515 can again change the 1D quantized transform coefficient sequence entropy decoded by the entropy decoder 510 into a 2D coefficient array (i.e., a block) in the reverse order of the coefficient scan performed by the video encoding device.

[0085] The inverse quantizer 520 performs inverse quantization on the quantized transform coefficients, and performs inverse quantization on the quantized transform coefficients using quantization parameters. The inverse quantizer 520 can also apply different quantization coefficients (scaling values) to the quantized transform coefficients arranged in 2D. The inverse quantizer 520 can perform inverse quantization by applying a matrix of quantization coefficients (scaling values) from the video encoding device to a 2D array of quantized transform coefficients.

[0086] The inverse transformer 530 reconstructs the residual signal by inversely transforming the inverse-quantized transform coefficients from the frequency domain to the spatial domain, thereby generating the residual block of the current block.

[0087] Furthermore, when the inverse transformer 530 performs an inverse transform on a portion (sub-block) of the transform block, it extracts a flag (cu_sbt_flag) indicating that only the sub-blocks of the transform block are transformed, the orientation (vertical / horizontal) information (cu_sbt_horizontal_flag) of the sub-blocks, and / or the position information (cu_sbt_pos_flag) of the sub-blocks. The inverse transformer 530 also inverse transforms the transform coefficients of the corresponding sub-blocks from the frequency domain to the spatial domain to reconstruct the residual signal, and fills the regions not inversely transformed with values ​​of "0" as the residual signal to generate the final residual block of the current block.

[0088] Furthermore, when applying MTS, the inverse transformer 530 determines the transform function or transform matrix to be applied in each of the horizontal and vertical directions by utilizing the MTS information (mts_idx) signaled from the video encoding device. The inverse transformer 530 also performs inverse transform on the transform coefficients in the transform block in both the horizontal and vertical directions using the determined transform function.

[0089] Predictor 540 may include intra-predictor 542 and inter-predictor 544. Intra-predictor 542 is activated when the prediction type of the current block is intra-prediction, and inter-predictor 544 is activated when the prediction type of the current block is inter-prediction.

[0090] Intra-predictor 542 determines the intra-prediction mode for the current block among multiple intra-prediction modes based on the syntax elements for the intra-prediction mode extracted from entropy decoder 510. Intra-predictor 542 also predicts the current block by utilizing neighboring reference pixels of the current block according to the intra-prediction mode.

[0091] The inter-frame predictor 544 determines the motion vector and the reference picture of the motion vector reference for the current block by using the syntax elements for the inter-frame prediction mode extracted from the entropy decoder 510, and predicts the current block by using the motion vector and the reference picture.

[0092] Predictor 540 can further utilize intra-block copy (IBC) mode. In IBC mode, predictor 540 can use block vectors decoded from the bitstream by entropy decoder 510 or derived from previously reconstructed blocks to identify the same prediction blocks selected by the video coding device.

[0093] Adder 550 reconstructs the current block by adding the residual block output from inverse transform 530 to the predicted block output from inter-frame predictor 544 or intra-frame predictor 542. Pixels within the reconstructed current block are used as reference pixels when performing intra-frame prediction on a subsequent block to be decoded.

[0094] The loop filtering unit 560, acting as an in-loop filter, may include a deblocking filter 562, a SAO filter 564, and an ALF 566. The deblocking filter 562 performs deblocking filtering on the boundaries between reconstructed blocks to remove block artifacts caused by block unit decoding. The SAO filter 564 and ALF 566 perform additional filtering on the reconstructed blocks after deblocking to compensate for differences between reconstructed pixels and original pixels caused by lossy encoding. The filter coefficients of the ALF are determined by utilizing information about the filter coefficients decoded from the bitstream.

[0095] The reconstructed blocks filtered by deblocking filter 562, SAO filter 564, and ALF 566 are stored in memory 570. When all blocks in an image are reconstructed, the reconstructed image can be used as a reference image for inter-frame prediction of blocks in subsequent images to be encoded.

[0096] The following describes a method that can be used by the aforementioned video encoder (e.g., Figure 1 The image encoding device shown) or video decoder (e.g., Figure 5The image decoding device shown implements an improved encoding and decoding technique.

[0097] In particular, the present invention relates to a technique for encoding and decoding video data using intra-block copy (IBC). IBC is a prediction technique that uses already decoded (i.e., reconfigured) blocks of the current image as predicted blocks of the current image.

[0098] VVC includes an IBC mode. Since IBC mode is implemented as a block-level encoding / decoding mode, the encoder performs block matching (BM) to find the optimal block vector (BV) for each CU. Here, the block vector represents the displacement within the current frame from the current block to a reconfigured reference block. The luma block vector of a CU encoded with IBC has integer precision. The chroma block vector is also rounded to integer precision. When combined with AMVR, IBC mode can switch between 1-pel and 4-pel motion vector precision. CUs encoded with IBC are processed using a third prediction mode, different from intra-frame or inter-frame prediction modes. In VVC, IBC mode is used for CUs with a width and height less than or equal to 64 luma samples. For blocks encoded with IBC, multi-transform selection (MTS) and low-frequency, non-separable transform (LFNST) are disabled.

[0099] At the CU level, IBC modes can be signaled via flags, and block vectors can be predictively encoded in either IBC AMVP or IBC merge modes. In IBC merge mode, the merge candidate index indicates which block vector in the IBC merge candidate list is used to predict the current block. The merge candidate list includes spatial candidates, history-based block vector predictor (HBVP) candidates, and pairwise average candidates. In IBC AMVP mode, block vector difference (BVD) is encoded and decoded in the same way as motion vector difference (MVD). The block vector prediction process uses two candidates as predictors: one obtained from the left neighbor and the other from the top neighbor. If neither neighbor is available, the default block vector can be used as the predictor. Flags can be signaled to indicate the block vector predictor index.

[0100] The Enhanced Compression Model (ECM), currently undergoing exploratory testing in JVET, features several improvements in IBC compared to VVC.

[0101] Compared to VVC, ECM's IBC has a larger reference area. In VVC's IBC mode, only the reconfigured portion of the predefined region (including the current CTU region and a portion of the left CTU) is allowed as the reference area. In ECM, the IBC reference area extends to two CTU rows above the current CTU. When the CTU size is 256, the reference area is limited to one CTU row above the current CTU.

[0102] ECM includes Reconstruction-Reordered IBC (RR-IBC). RR-IBC mode is allowed for IBC codec blocks. When RR-IBC is applied, samples of the reconstructed block are flipped according to the flip type of the current block. On the encoder side, the original block is flipped before motion estimation and residual calculation, and the predicted block is derived without flipping. On the decoder side, the reconstructed block is flipped again to reconstruct the original block. RR-IBC supports two types of flips: horizontal flips and vertical flips. For blocks encoded and decoded in IBC AMVP mode, a flag indicating whether to flip the reconstruction is first signaled, and if flipped, another flag specifying the flip type is additionally signaled. In the case of IBC merging, the flip type is inherited from adjacent blocks without syntax signaling.

[0103] In ECM, Filtered Intra Block Copy (FIBC) is introduced, and FIBC mode applies filtering to the reference block in IBC mode. The filter coefficients for the filtering process are derived by minimizing the mean square error (MSE) between the template of the current block and the filtered template of the IBC reference block. A flag (FIBC flag) is sent to indicate whether FIBC is used for encoding / decoding blocks in IBC AMVP mode. In the case of IBC merging, the FIBC flag can be inherited from candidates in the merge candidate list.

[0104] In addition, several IBC extensions were introduced in ECM, such as LIC for IBC (IBC-LIC) and geometry partitioning mode for IBC (IBC-GPM), and these tools are similar to the corresponding inter-frame prediction tools.

[0105] As an example, video encoders and video decoders can be configured to perform local illumination compensation (LIC). In ECM-7.0, LIC is an inter-frame prediction technique that models the local illumination variation between the current block and the predicted block as a function of the local illumination variation between the template of the current block and the template of the reference block. The linear function applied to the reference block by LIC can be represented as shown in Equation 1.

[0106] [Equation 1] Here, "predVal" is the output of the LIC operation, RefSample is the reference sample indicated by the motion vector, and α and β are parameters derived by the encoder and decoder from the templates of the current block and the reference block, respectively.

[0107] In ECM-8.0, a LIC for IBC (IBC-LIC) was introduced, which is an extension of the LIC used for regular inter-frame prediction. The LIC utilizes a scaling factor α and an offset β to compensate for differences in local illumination. Since the scaling factor and offset are estimated from the templates of the current block and the reference block, no additional signaling is required for these parameters. A flag (IBC-LIC flag) indicating whether the LIC is used for a block being encoded or decoded in IBC AMVP mode is transmitted via signaling. In the case of IBC merging, the IBC-LIC flag can be inherited from candidates in the merge candidate list.

[0108] Intra-template matching-based intra-frame prediction is called intra-template matching prediction (IntraTMP). In IntraTMP mode, the best prediction block for the current block is determined from the reconfiguration portion of the current image based on template matching of the current block's current template. For a predefined search range, the encoder can search within the reconfiguration portion of the current image for the template most similar to the current block's current template (i.e., the reference template), and can use the corresponding block (i.e., the reference block) as the prediction block for the current block.

[0109] A block vector can represent the displacement between a reference template and the current template. The displacement between the reference block and the current block is the same as the displacement between the reference template and the current template. Therefore, a block vector can represent the displacement between the reference block and the current block. The encoder can use IntraTMP mode with signals, and since the decoder on the decoder side performs the same prediction operation as described above, it is not necessary to send the block vector with signals. The aforementioned LIC is also allowed for IntraTMP.

[0110] In ECM, the block vectors of blocks encoded and decoded in IntraTMP mode can be stored in the IBC block vector buffer, and the stored block vectors can be used to encode and decode another block in IBC mode. For example, the block vectors of blocks encoded and decoded in IntraTMP mode can be used to configure the IBC candidate list of another block. In the following description, the block vectors of blocks encoded and decoded in IntraTMP mode can be referred to as IntraTMP block vectors, and the block vectors derived in IBC mode can be referred to as IBC block vectors.

[0111] In terms of configuring the IBC candidate list, compared with VVC, ECM additionally introduces spatial non-adjacent candidates (JVET-AE0094), auto-relocated block vector prediction (AR-BVP) candidates (JVET-AG0091), and filling candidates (JVET-Z0160).

[0112] This invention improves upon VVC and ECM techniques in several aspects of the configuration of the IBC candidate list.

[0113] As an example, video encoders and decoders can determine multiple latent block vector candidates by performing template matching on some samples or regions of the current template of the current block within the reconfiguration section of the current image. The video encoder and decoder can then include one or more latent block vector candidates that fit the remaining samples or regions of the current template in the IBC candidate list of the current block as template-based candidates. As another example, the pairwise candidates for the VVC generated by averaging predefined candidate pairs can be extended to combinations of two or more candidates.

[0114] Figure 6 This is a flowchart illustrating a method for configuring an IBC candidate list according to an embodiment of the present invention.

[0115] Video encoders and decoders can generate an IBC candidate list by sequentially examining various types of candidates. Candidate types can include spatially adjacent candidates, spatially non-adjacent candidates, HBVP candidates, AR-BVP candidates, mean-based candidates, template-based candidates, boundary-based candidates, and zero-vector candidates.

[0116] exist Figure 6 In the flowchart shown, the order of each step can be changed, or some steps can be omitted. If in Figure 6 In any step of configuring different types of candidates, configuring the candidate list to a predefined maximum number allows subsequent steps for configuring other types of candidates to be omitted.

[0117] When using IBC mode, an IBC candidate list that includes BVP candidates can be configured. When using IBC merge mode, the IBC candidate list can be an IBC merge candidate list. When using IBC AMVP mode, the IBC candidate list can be an IBC AMVP candidate list (or an IBC BVP list).

[0118] In the IBC candidate list, each candidate may include block vector information (e.g., block vector size, block vector precision, etc.) and may further include additional information related to the prediction method. The additional information related to the prediction method may include syntax information related to IBC-based prediction, such as information indicating whether the relevant block is encoded or decoded in IBC mode or IntraTMP mode, Local Illumination Compensation (LIC) flags and / or parameter information, Reconstructed Rearranged IBC (RR-IBC) flags and flipping methods, and Filtered IBC (FIBC) flags. The current block can inherit additional information beyond the block vector information from candidates in the candidate list. For example, syntax information such as the IBC-LIC flag, RR-IBC flag, and FIBC flag can be sent to the current block from candidates in the candidate list.

[0119] For reference Figure 4 For example, five spatially adjacent blocks can be located relative to the current block. These five spatially adjacent blocks can be A0, A1, B0, B1, and B2. The block vector information of one or more spatially adjacent blocks encoded and decoded using IBC mode or IntraTMP mode can be included in the candidate list as spatially adjacent candidates.

[0120] Block vector information of one or more non-adjacent blocks encoded and decoded using IBC mode or IntraTMP mode can be included in the candidate list as spatial non-adjacent candidates. Figure 7 This is a schematic diagram illustrating spatially non-adjacent blocks positioned along different directions. The encoder and decoder can scan these non-adjacent blocks according to a predefined order. They can also scan these blocks alternately. For example, as shown... Figure 7 As shown, sixteen non-adjacent blocks can be scanned from closer to farther distances along various directions with angles of 45 degrees, 90 degrees, 135 degrees, 180 degrees, and 225 degrees. Figure 7 In this context, parameters A and B define the horizontal and vertical spacing between non-adjacent positions. Parameters A and B can be integers of 1 or greater, and can be determined based on the width (w) and height (h) of the current block.

[0121] History-based candidates can be added to the candidate list after spatially adjacent and spatially non-adjacent candidates. History-based candidates can also be called history-based block vector prediction (HBVP) candidates. For example, the block vectors of blocks encoded before the current block are stored in the HBVP table, and at least one of the block vectors stored in the HBVP table can be used as a BVP candidate for the current block. The history buffer can be managed in a first-in-first-out (FIFO) manner. Whenever a predicted block exists under IBC or IntraTMP, the associated block vector information can be added to the last entry of the HBVP table. The last few block vectors in the HBVP table can be inserted into the candidate list after spatially adjacent and spatially non-adjacent candidates. The HBVP table can be initialized for each CTU line, or it can be initialized in units of one or more virtual pipeline data units (VPDUs).

[0122] An auto-relocated block vector predictor (AR-BVP) candidate can be derived as an accumulation of BVs recursively traced from existing block vectors (BVs) in the candidate list. For example, if a first reference block indicated by a block vector in the configured block vector candidates is predicted under IBC-based prediction or IntraTMP prediction and has a block vector, a second reference block indicated by the corresponding block vector can be found. As described above, by recursively repeating the process of finding another reference block referenced by a reference block until the k-th reference block is reached (where k is an integer of 2 or greater), the vector representing the displacement from the current block to the k-th reference block can be defined as the AR-BVP. AR-BVP candidates can be inserted into the candidate list after HBVP candidates.

[0123] The method for deriving AR-BVP is shown. Figure 8a In the example, the bootstrap block vector BV associated with the current block B0 0,1 Indicates reference block B1. Here, the initial guide block vector BV 0,1 It is one of the existing BVP candidates already in the candidate list. When reference block B1 has a BVP indicated by reference block B2. 1,2 When the indicated BV is as shown in Equation 2, it is called BV. 0,2 It can be defined as being composed of BV 0,1 Guided AR-BVP.

[0124] [Equation 2] Similarly, BV 0,n+1 It can be derived as shown in Equation 3.

[0125] [Equation 3] When the derivation is from BV 0,n Guided BV n,n+1 At that time, reference block B can be checked in a predefined order. n The top left position (e.g., Figure 8b LT in the middle), upper right position (e.g., Figure 8b RT in the middle), center position (e.g., Figure 8b C in the middle), the lower left position (for example, Figure 8b (LB in the middle) and the bottom right position (e.g., Figure 8b The five positions of RB in BV are used to find the BV. n,n+1 .

[0126] In some implementations, when the block indicated by the AR-BVP candidate is not predicted via IBC-based prediction, the AR-BVP candidate may include the size of the block vector, but prediction-related syntax information can be disabled (e.g., set to LIC flag=0, FIBC flag=0) or configured by obtaining syntax information determined in units of slices, sequences, images, etc. (e.g., LIC flag=1, FIBC flag=0, etc.). This can also be applied to... Figure 6 Other types of candidates.

[0127] Average-based candidates can be generated by averaging block vectors of predefined combinations of two or more candidates from an existing merge candidate list. Combinations of two or more candidates (e.g., such as {0, 1}, {0, 2}, ..., {0, 1, 2}, ..., {1, 2, 3}) can be predefined in the encoder and decoder as combinations of candidate indices in the candidate list. While pairwise candidates in VVC are generated by averaging predefined candidate pairs, average-based candidates are extended to combinations of two or more candidates.

[0128] The video encoder and decoder can determine multiple latent block vector candidates by performing template matching on some samples or regions of the current template of the current block in the reconfiguration section of the current image. The video encoder and decoder can then include one or more latent block vector candidates that fit the remaining samples or regions of the current template in the IBC candidate list of the current block as template-based candidates.

[0129] Figure 9a and Figure 9bThis illustrates various template shapes that can be considered for template matching. The shape and size of the template can be implicitly determined based on the size and / or aspect ratio of the current block, or the encoder can signal an indication in the bitstream which template to use. For example, template information to be applied to the current block can be inherited from adjacent or non-adjacent blocks in IBC merge mode, and can be explicitly signaled in the bitstream in IBC AMVP mode. Figure 9a and Figure 9b In this context, dimensions a, b, m, and n can be integers of 1 or greater, and can be values ​​that vary depending on the size and / or aspect ratio of the current block.

[0130] The samples used to derive template-based candidate templates can be partitioned or split into two subsets. The "sample subset" of a template can also be called a "sub-template" of the template. Figure 10a It shows that it can be made by Figure 9a An example of a partition (or pattern) formed by two subsets (subset A and subset B) of the “L” shaped template shown.

[0131] In the template samples of the current block, samples belonging to one of two subsets are used to determine or derive "potential" candidates based on template matching in the reconfiguration section of the current image, and samples belonging to the other subset are used to determine or derive "final" template-based candidates from the potential candidates based on comparison with corresponding samples of the reference template indicated by the potential candidates. Figure 10a In the example shown, samples in subset B are located relatively closer to the current block compared to samples in subset A, and therefore it may be advantageous to use subset A to identify potential candidates and subset B to identify the final template-based candidates.

[0132] Figure 10b Other examples of partitions (or patterns) that can be formed from two subsets (subset A and subset B) of an L-shaped template are shown. For Figure 9b Each template shown can also be defined with Figure 10b The two subsets of the partition (or pattern) shown.

[0133] When multiple types of partitions (or patterns) are available for the template of the current block, the encoder can send a signal in the bitstream indicating which type of partition (or pattern) is used for the template.

[0134] In some implementations, information about which of two subsets (subset A and subset B) each sample of a specified template belongs to can be predefined in the encoder and decoder in the form of a matrix. As an example, in the sample positions of the template, a matrix M(i,j) can be defined, where the coefficients corresponding to sample positions belonging to subset A have a value of "1", and the coefficients corresponding to sample positions belonging to subset B have a value of "0". That is, matrix M(i,j) specifies which of the two subsets (subset A and subset B) each sample of the current block's template belongs to. The encoder and decoder can use the matrix M(i,j) of the current block and the template T(i,j) to determine or deduce the values ​​of samples in subset A and subset B, as shown in Equation 4.

[0135] [Equation 4] Here, T(i, j) represents the template of the current block.

[0136] The encoder can send information about matrix M(i,j) in the bitstream via signals. In IBC merge mode, template information and / or matrix information to be applied to the current block can be inherited from adjacent or non-adjacent blocks, or in IBC AMVP mode, it can be sent in the bitstream via signals.

[0137] When two subsets of samples are determined for a template of the current block, the encoder and decoder can deduce potential candidates by performing template matching on the reconfigured portion of the current image using samples from the first subset (e.g., subset A). Potential candidates can be deduced based on template matching cost, where the template matching cost can be calculated based on the difference between samples from the first subset and corresponding samples of the reference template indicated by the potential candidates. To calculate the template matching cost, cost functions such as the sum of absolute differences (SAD), the sum of absolute transformed differences (SATD), the mean squared error (MSE), and the mean absolute error (MAE) can be used. In template matching based on samples from the first subset of the template, k potential candidates with template matching costs less than a predefined threshold can be deduced.

[0138] The encoder and decoder can compute template matching costs for samples of a second subset (e.g., subset B) against k potential candidates. Here, the template matching cost can be computed based on the difference between the samples of the second subset and the corresponding samples of the template of the reference block (indicated by the potential candidate). The encoder and decoder can select at least one as the final template-based candidate from the potential candidates based on the template matching cost of the samples of the second subset, or based on the sum or weighted sum of the template matching costs of the first and second subsets. The cost function used to compute the template matching cost of the second subset can be the same as or different from the cost function used for the first subset.

[0139] In ECM, candidates for zero vectors used to populate the IBC merge / AMVP list are replaced by a set of BVP candidates located in the IBC reference region. This set of BVP candidates is called boundary-based candidates or filling candidates. Since zero vectors are invalid as block vectors in IBC merge mode, they are excluded from the IBC candidate list.

[0140] Figure 11 A schematic diagram illustrating a set of filling candidates is shown, illustrating some examples. Figure 11 In the example, a buffer called the IBC buffer is allocated for reconstruction in IBC mode. During reconstruction, the IBC buffer is allocated as a portion of a buffer image. This portion of the image includes the reconfiguration area (indicated by shading), the current block (displayed as a black area), and the area to be reconfigured (displayed as a white area). Figure 11 In this context, W represents the width of the current block, and H represents the height of the current block.

[0141] exist Figure 11 In the example, a portion of the reconfigured region can be used as a reference for reconstructing the current block, and this portion is called the reference region. The reference region includes sub-regions (e.g., A, B, and C). Figure 11 In this context, three fill candidates are located at the nearest corner of the reference region, and three additional fill candidates are determined at the centers of the three sub-regions (A, B, and C). The coordinates of the fill candidates are determined by the width and height of the current block, as well as the ΔX and ΔY parameters, as shown below. Figure 11 As shown.

[0142] If the candidate list is not completely filled with other types of candidates, a zero-vector candidate can be added to the candidate list. When a zero-vector candidate is used for the current block, the decoder can explicitly or implicitly derive the block vector difference (BVD) from the bitstream and use the block vector obtained by adding the BVD to the zero-vector candidate to perform prediction for the current block.

[0143] In some implementations, one (or other number) of temporal candidates derived from temporal co-occurrence blocks may be added to the candidate list, for example, adding a candidate between spatially non-adjacent candidates and HBVP candidates.

[0144] In the candidate list, candidates can be reordered in ascending order based on the template matching cost of each candidate. The template matching cost can be determined based on the difference between the sample of the template of the current block and the sample of the template of the corresponding reference block. As a cost function, the sum of absolute differences (SAD), the sum of absolute transformation differences (SATD), the mean squared error (MSE), the mean absolute error (MAE), etc., can be used.

[0145] The template of the current block used during the reordering process can include the reconstructed (top and left) sample sets adjacent to the current block. The template of the reference block can be identified from the candidate block vectors based on the template of the current block.

[0146] The template of the current block used during reordering can be the same as the template used to derive template-based candidate current blocks. The template of the current block used during reordering can be the same as a first subset or a second subset of the template used to derive template-based candidate current blocks. Alternatively, the template of the current block used during reordering can have a different shape than the template (or a subset of the template) used to derive template-based candidate current blocks.

[0147] In some implementations, candidates are reordered in ascending order based on the template matching cost of each candidate, and a predetermined number of candidates with the minimum template matching cost can be selected as the final candidates.

[0148] In some implementations, the candidate list reordering process can be performed on all candidates in the candidate list, and alternatively, the candidate list reordering process can be performed only on the first n candidates.

[0149] In some implementations, the encoder and decoder can divide the candidates in the candidate list into m groups and perform a reordering process for each group. Therefore, within each group, the candidates can be reordered in ascending order based on the template matching cost of each candidate. When the block vector information of the current block is signaled, the encoder can encode the group index and the index indicating the reordering candidates within the group into a bitstream.

[0150] The template shape used during reordering can be predefined. The template shape used during reordering can be configured by the encoder. Information about the template shape selected from the available template shapes can be signaled in the bitstream, and the decoder can deduce the template shape based on the signaled information about the template shape, as well as the current block size, aspect ratio, etc. Information about the template shape can be signaled in units of blocks, CTUs, slices, and / or sequences. Information about the template shape is predefined in the encoder and decoder in the form of matrices, so the encoder can signal information about the matrix associated with the selected template shape in the bitstream.

[0151] As an example, you can define the maximum configurable template region Temp for the current block. max (i, j). In Figure 12 In the example, the maximum template region 1202 consists of p rows above the current block 1200 and q columns to the left of the current block 1200. The encoder can be defined by coefficients c. i,j The matrix M formed R (i, j), the coefficient c i,j Each pixel location within the maximum template region 1202 has a value of 1 or 0, corresponding to the selected template shape. That is, matrix M... R (i, j) specifies whether each pixel position within the maximum template region 1202 belongs to the template of the current block 1200. A coefficient c with a value of 1. i,j A coefficient c can be used to indicate that the pixel position (i, j) belongs to template 1200 of the current block and has a value of 0. i,j This can be indicated as a template 1200 where pixel position (i, j) does not belong to the current block. The encoder can send signals about matrix M in the bitstream. R Information about (i, j). The decoder can deduce the template Temp for the current block 1200 to be used in the reordering process. reorder The sample values ​​of (i, j) are shown in Equation 5.

[0152] [Equation 5] On the other hand, geometric partitioning can be used in conjunction with IBC prediction. Here, a block can be divided into two sub-regions by geometrically positioned straight lines (i.e., geometric partition boundaries), and each sub-region can be used for IBC prediction using its own block vector. After predicting each sub-region, the final predicted signal for the block can be generated by a weighted sum of the predicted signals from the two sub-regions. The weights can be adaptively determined based on the distance between the sample location and the partition boundary.

[0153] In some implementations, a separate IBC candidate list can be configured for each sub-region of the block. When configuring the IBC candidate list, a reference can be used. Figure 9a , Figure 9b , Figure 10a and Figure 10b The description describes template-based candidates. Templates for deriving template-based candidates can be determined differently for each sub-region. Templates for each sub-region can be determined based on geometric partition boundaries. For example, as... Figure 13a As shown, the template for each sub-region can be part of the template for the block, which is obtained by dividing the template of the block by the points where the extension of the geometric dividing boundary intersects with the template of the block. Figure 13b and Figure 13c These are other examples that can be used to derive template-based candidate templates for each subregion. Figure 13a , Figure 13b and Figure 13c In the template, template 0 is the template of subregion 0, and template 1 is the template of subregion 1.

[0154] When geometric partitioning is used in conjunction with IBC prediction, specific locations of the geometric partition boundaries allowed for use in blocks can be defined, and template regions for corresponding sub-regions can be defined for each specific location. Once the template is determined for each sub-region, the above process of deriving template-based candidates for each sub-region can be performed separately.

[0155] In some implementations, the block vector for each region of a block can be determined based on a single IBC candidate list configured for the block. Before being used for each sub-region, the single IBC candidate list can be reordered based on template matching using a template for each sub-region.

[0156] Figure 14 This is a flowchart illustrating a method for a video decoder to decode video data according to an aspect of the present invention.

[0157] The video decoder can determine whether the current block is decoded using intra-block copying (S1410). For example, the video encoder can send a flag indicating whether IBC mode is used at the CU level, and the video decoder can determine whether the current block is encoded using intra-block copying based on the flag parsed from the bitstream.

[0158] The video decoder can construct an IBC candidate list for the current block (S1410). The video decoder can be configured to include IBC candidate lists of the various types of candidates described above.

[0159] In the implementation scheme, as part of constructing the IBC candidate list for the current block (S1420), the video decoder can divide samples of the template for the current block into a first subset and a second subset, and perform template matching using the first subset to derive potential candidates. Here, the template matching cost of the first subset is determined based on the difference between the samples of the first subset and the corresponding samples of the reference template indicated by the potential candidates. The template matching cost of each potential candidate in the first subset can have a value less than a predefined threshold. The video decoder can select at least one template-based candidate from the potential candidates and add it to the IBC candidate list based on the difference between the samples of the second subset and the corresponding samples of the reference template indicated by the potential candidates.

[0160] In the implementation, the video decoder can configure an IBC candidate list including at least one type of candidate selected from spatially adjacent candidates, spatially non-adjacent candidates, history-based block vector prediction (HBVP) candidates, and automatic relocation block vector prediction (AR-BVP) candidates, as part of constructing the IBC candidate list for the current block (S1420). Additionally, the video decoder can add average-based candidates to the IBC candidate list, which are generated by averaging the block vectors of combinations of two or more candidates present in the IBC candidate list.

[0161] The video decoder can decode from the bitstream an indication of the candidate selected from the IBC candidate list (S1430), and can reconstruct the current block using the selected candidate (S1440). For example, in IBC merge mode, the video decoder can determine the block vector of the selected candidate for predicting the current block. In IBC AMVP mode, the video decoder can further decode from the bitstream a block vector difference (BVD) representing the difference between the block vector of the current block and the block vector of the selected candidate. The video decoder can determine the block vector of the current block by adding the BVD to the selected candidate.

[0162] When the selected candidate is derived from at least one block decoded in IBC mode prior to the current block, the current block can inherit syntactic information from the selected candidate, including at least one of the IBC-LIC flag, the reconstructed rearranged IBC (RR-IBC) flag, or the filtered IBC (FIBC) flag. When the selected candidate is derived from at least one block decoded in IntraTMP mode prior to the current block, the current block can inherit syntactic information from the selected candidate, including the IBC-LIC flag. Therefore, when reconstructing the current block from a reference block indicated by a block vector, one or more processes can be performed based on the inherited syntactic information.

[0163] Within the candidate list, each candidate may include block vector information indicating the size and resolution of the block vector. Within the candidate list, selected candidates further include prediction-related syntax information, which may include information indicating whether the relevant block is encoded or decoded in IBC mode or IntraTMP mode, as well as syntax information related to IBC-based prediction.

[0164] In some implementations, as part of reconstructing the current block (S1440), the video decoder may perform template matching based on selected candidate block vectors to determine a refined block vector pointing from the current block to a refined reference block in the current image. The video decoder may then reconstruct the current block based on the refined reference block. For example, the video decoder may derive a prediction block from the refined reference block, reconstruct a residual block from the bitstream, and reconstruct the current block by (sample-by-sample) combining the prediction block and the residual block.

[0165] During the template-matching-based refinement process, a refinement offset can be applied to the block vector (BV) (e.g., based on a predefined orientation and positional pattern) to generate a potential refinement BV value for the reference block that indicates potential refinement of the reference block. The reference block for potential refinement has a corresponding reference template for potential refinement. A template matching cost can be calculated between the current template and each reference template for potential refinement. The reference template for potential refinement with the minimum template matching cost is determined, and the potential refinement BV associated with the reference template for potential refinement with the minimum template matching cost can be used as the final block vector for predicting the current block in IBC mode.

[0166] The template-matching-based block vector refinement process can be applied to all or some of the block vector candidates present in the IBC candidate list (e.g., spatial candidates and history-based candidates).

[0167] In some implementations, as part of reconstructing the current block (S1440), the video decoder may decode a refinement execution flag and one or more index information from the bitstream based on selected candidate block vectors. If the corresponding flag is true, the video decoder can derive the block vector difference (BVD) using the decoded index information (e.g., an index indicating direction, an index indicating size), and can determine the refined block vector of the current block by adding the BVD. The video decoder can reconstruct the current block based on the refined reference block. For example, the video decoder can derive a prediction block from the refined reference block, reconstruct a residual block from the bitstream, and reconstruct the current block by combining the prediction block and the residual block (sample-by-sample).

[0168] Although the steps in the various flowcharts are described in sequence, these steps only illustrate the technical ideas of some embodiments of the invention. Therefore, those skilled in the art to which this invention pertains can perform these steps by changing the order described in the various figures or by performing two or more steps in parallel. Thus, the steps in the various flowcharts are not limited to the chronological order shown.

[0169] It should be understood that the above description presents illustrative embodiments that can be implemented in various other ways. The functionality described in some embodiments can be implemented by hardware, software, firmware, and / or combinations thereof. It should also be understood that functional components described in this invention are designated as "...units" to emphasize their potential for independent implementation.

[0170] On the other hand, the various methods or functions described in some embodiments can be implemented as instructions stored in a non-volatile recording medium, which can be read and executed by one or more processors. The non-volatile recording medium can include various types of recording devices, such as those storing data in a computer system-readable form. For example, the non-volatile recording medium can include storage media such as erasable programmable read-only memory (EPROM), flash memory drives, optical disk drives, magnetic hard disk drives, and solid-state drives (SSDs).

[0171] Although embodiments of the invention have been described for illustrative purposes, those skilled in the art will understand that various modifications, additions, and substitutions can be made without departing from the spirit and scope of the invention. Therefore, embodiments of the invention have been described for the sake of brevity and clarity. The scope of the technical concept of the embodiments of the invention is not limited by the examples. Accordingly, those skilled in the art will understand that the scope of the invention should not be limited by the embodiments explicitly described above, but rather by the claims and their equivalents.

[0172] Cross-references to related applications This application claims priority and benefit to Korean Patent Application No. 10-2024-0044381, filed on April 1, 2024, the entire contents of which are incorporated herein by reference.

Claims

1. A method for decoding video data, the method comprising: It was determined that the current block would be decoded using intra-block copy (IBC); Construct a candidate list of IBCs for the current block; Decoding from the bitstream indicates the candidate selected from the IBC candidate list; as well as Reconstruct the current block using the selected candidates. The IBC candidate list for constructing the current block includes: Divide the template samples of the current block into a first subset and a second subset; Template matching is performed using the first subset to derive potential candidates; and Based on the difference between the samples of the second subset and the corresponding samples of the reference template indicated by the potential candidates, at least one template-based candidate is selected from the potential candidates to be included in the IBC candidate list.

2. The method according to claim 1, wherein, The template matching cost for each potential candidate in the first subset has a value less than a predefined threshold.

3. The method according to claim 2, wherein, The template matching cost of the first subset is determined based on the difference between the samples of the first subset and the corresponding samples of the reference template indicated by the potential candidates.

4. The method according to claim 1, wherein, The list of IBC candidates for constructing the current block includes: Construct an IBC candidate list comprising at least one type of candidate selected from spatially adjacent candidates, spatially non-adjacent candidates, history-based block vector prediction (HBVP) candidates, and automatic relocation block vector prediction (AR-BVP) candidates; and A mean-based candidate, generated by averaging the block vectors of two or more candidates existing in the IBC candidate list, is added to the IBC candidate list.

5. The method according to claim 4, wherein, Spatial adjacency candidate, spatial non-adjacency candidate, HBVP candidate, and AR-BVP candidate are derived from at least one block decoded in IBC mode or IntraTMP mode prior to the current block.

6. The method according to claim 5, wherein, The IBC candidate list for constructing the current block further includes: Add one or more filler candidates that indicate a predefined specific location within the IBC reference area to the IBC candidate list.

7. The method according to claim 5, wherein, When the selected candidate is derived from at least one block decoded in IBC mode prior to the current block, the current block inherits syntactic information from the selected candidate, including at least one of the IBC-LIC flag, the reconstructed rearranged IBC (RR-IBC) flag, or the filtered IBC (FIBC) flag.

8. The method according to claim 5, wherein, When the selected candidate is derived from at least one block decoded in IntraTMP mode prior to the current block, the current block inherits syntactic information, including IBC-LIC flags, from the selected candidate.

9. The method according to claim 5, wherein, In the candidate list, each candidate further includes block vector information and prediction-related syntax information, wherein the prediction-related syntax information includes information indicating which mode, IBC mode or IntraTMP mode, the relevant block is encoded or decoded by, and prediction-related syntax information based on IBC.

10. The method according to claim 1, wherein, Decoding the current block further includes: Template matching is performed based on the selected candidate block vectors to determine the thinned block vector pointing from the current block to a thinned reference block within the current image; and Reconstruct the current block based on the refined reference block.

11. A method for encoding video data, the method comprising: It was determined that the current block would be encoded using intra-block copying (IBC); Construct a candidate list of IBCs for the current block; Encode the indication that a candidate is selected from the IBC candidate list; as well as Encode the current block using the selected candidates. The IBC candidate list for constructing the current block further includes: Divide the template samples of the current block into a first subset and a second subset; Template matching is performed using the first subset to derive potential candidates; and Based on the difference between the samples of the second subset and the corresponding samples of the reference template indicated by the potential candidates, at least one template-based candidate is selected from the potential candidates to be included in the IBC candidate list.

12. The method according to claim 11, wherein, The template matching cost for each potential candidate in the first subset has a value less than a predefined threshold.

13. The method according to claim 12, wherein, The template matching cost of the first subset is determined based on the difference between the samples of the first subset and the corresponding samples of the reference template indicated by the potential candidates.

14. A method for providing video data to a video decoding device, the method comprising: Encode video data into a bitstream; as well as Sending the bitstream to the video decoding device, wherein encoding video data into the bitstream includes: It was determined that the current block would be encoded using intra-block copying (IBC); Construct a candidate list of IBCs for the current block; Encode the indication that a candidate is selected from the IBC candidate list; and Encode the current block using the selected candidates. The IBC candidate list for constructing the current block further includes: Divide the template samples of the current block into a first subset and a second subset; Template matching is performed using the first subset to derive potential candidates; and Based on the difference between the samples of the second subset and the corresponding samples of the reference template indicated by the potential candidates, at least one template-based candidate is selected from the potential candidates to be included in the IBC candidate list.

Images