Template matching prediction based on nonlinear modeling

Abstract

The embodiment discloses a video coding and decoding method and device using template matching prediction based on nonlinear modeling. In this embodiment, an image decoding device searches for a reference block based on a block vector of a current block or a template match between a template of a block within a search region and a template of the current block, and determines a prediction block of the current block by means of the found reference block. An image decoding apparatus acquires a form of a filter of a current block, where the form of the filter is a cross-component discrete mapping model or a cross-component nonlinear filter. An image decoding device calculates coefficients of a filter by using a reference block, and applies filtering to a prediction block by using the calculated filter.

Description

TECHNICAL FIELD

[0001] The present application relates to a video coding method and apparatus using a selective multiple syntax. BACKGROUND

[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute the prior art.

[0003] Since video data has a large amount of data compared to audio data or still image data, a large amount of hardware resources including memory are required to store or transmit uncompressed video data.

[0004] Accordingly, an encoder is generally used to compress and store or transmit video data. A decoder receives compressed video data, decompresses the received compressed video data, and plays the decompressed video data. Video compression techniques include H.264 / Advanced Video Coding (AVC), High Efficiency Video Coding (HEVC), and Versatile Video Coding (VVC) which improves coding efficiency by about 30% or more than HEVC.

[0005] However, as the image size, resolution, and frame rate gradually increase, the amount of data to be encoded also increases. Accordingly, there is a need to provide a new compression technique having higher coding efficiency and improved image enhancement effects than existing compression techniques.

[0006] In encoding / decoding using template matching-based prediction, a reference block corresponding to a template most similar to a template of a current block is used as a matching block. During template matching-based decoding, a decoder searches for a matching block based on template matching between a template of a reference block and a template of a current block, and determines a prediction block of the current block with the searched matching block. The decoder uses a sum of absolute difference (SAD) or a sum of squared error (SSE) to calculate a similarity based on template matching. Since template matching is performed by using a plurality of reconstructed reference blocks, template matching-based prediction increases the computational complexity of a video decoding process. In order to reduce the computational complexity, template matching is performed by defining a search area for locating a reference block. As shown in FIG. 6, R1, R2, R3, and R4 correspond to search areas for template search.

[0007] On the other hand, filtering can be applied to intra template matching prediction (intraTMP). A filter is applied to a reference block, and the filtered reference block is used as a prediction block of the current block. For example, the filter is a 6-tap linear filter including five luma taps and one bias term. The filter coefficients of each block are derived by utilizing a regression model that minimizes the mean square error (MSE) between the filter output generated from the reference template and the samples of the current template. A filter flag indicating the use or non-use of the linear filter is transmitted from the encoder to the decoder at the CU level. In addition, the use of the linear filter is considered as a lower level mode of intra template matching prediction. The filter flag is signaled when a template matching flag indicating the use or non-use of intra template matching prediction is true. On the other hand, in order to improve the video coding efficiency and enhance the video quality, there is a need for a method of utilizing a nonlinear model in the above filtering process. SUMMARY

[0008] TECHNICAL PROBLEM The present disclosure is directed to a video coding method and apparatus that utilizes a template matching technique based on a discrete mapping model or a template matching technique based on a nonlinear filter when performing template matching prediction (TMP). The video coding method and apparatus utilizes a cross-component discrete mapping model or a cross-component nonlinear model based on the template matching technique.

[0009] TECHNICAL SOLUTION At least one aspect of the present disclosure provides a method of reconstructing a current block by a video decoding apparatus. The method includes searching for a reference block based on a template matching between a template of a block within a search area and a template of the current block or based on a block vector of the current block. The method further includes determining a prediction block of the current block by utilizing the searched reference block. The method further includes obtaining a type of a filter for the current block. Here, the type of the filter includes a cross-component discrete mapping model or a cross-component nonlinear filter. The method further includes calculating coefficients of the filter by utilizing the reference block. The method further includes applying filtering to the prediction block by utilizing the filter.

[0010] Another aspect of the disclosure provides a method of encoding a current block by a video encoding device. The method includes generating a first prediction block of the current block by utilizing a prediction mode other than template matching and intra block copy (IBC) mode. The method also includes searching a reference block based on a template matching between a template of a block within a search area and a template of the current block or based on a block vector of the current block, and determining a second prediction block of the current block by utilizing the searched reference block. The method further includes obtaining a type of filter for the current block. Here, the type of filter includes a cross-component discrete mapping model or a cross-component nonlinear filter. The method also includes calculating coefficients of the filter by utilizing the reference block. The method further includes generating a filtered second prediction block by applying filtering to the second prediction block by utilizing the filter.

[0011] Yet another aspect of the disclosure provides a method of providing video data to a video decoding device. The method includes encoding the video data into a bitstream, and transmitting the bitstream to the video decoding device. The encoding of the video data includes generating a first prediction block of a current block by utilizing a prediction mode other than template matching and intra block copy (IBC) mode. The encoding of the video data also includes searching a reference block based on a template matching between a template of a block within a search area and a template of the current block or based on a block vector of the current block, and determining a second prediction block of the current block by utilizing the searched reference block. The encoding of the video data further includes obtaining a type of filter for the current block. Here, the type of filter includes a cross-component discrete mapping model or a cross-component nonlinear filter. The encoding of the video data also includes calculating coefficients of the filter by utilizing the reference block. The encoding of the video data further includes generating a filtered second prediction block by applying filtering to the second prediction block by utilizing the filter.

[0012] Advantages As described above, the disclosure provides a video coding method and apparatus that utilizes a template matching technique based on a discrete mapping model or a template matching technique based on a nonlinear filter when performing template matching prediction. The video coding method and apparatus utilizes a cross-component discrete mapping model or a cross-component nonlinear model based on the template matching technique. Accordingly, the video coding method and apparatus improves video coding efficiency and enhances video quality. BRIEF DESCRIPTION OF DRAWINGS

[0013] FIG. 1 is a block diagram of a video encoding device that can implement the techniques of the disclosure.

[0014] FIG. 2 illustrates a method of partitioning a block using a quad tree plus binary tree ternary tree (QTBTTT) structure.

[0015] FIGS. 3a and 3b illustrate a plurality of intra prediction modes including a wide angle intra prediction mode.

[0016] FIG. 4 shows neighboring blocks of a current block.

[0017] FIG. 5 is a block diagram of a video decoding device that can implement the present technology.

[0018] FIG. 6 is a diagram showing a search region for template search according to at least one embodiment of the present disclosure.

[0019] FIG. 7 is a flowchart of reconstructing a current block by a video decoding device according to at least one embodiment of the present disclosure.

[0020] FIG. 8 is a diagram showing a filter for filtering a prediction block based on template matching according to at least one embodiment of the present disclosure.

[0021] FIG. 9 is a diagram showing a multi-model linear model (MMLM) mode.

[0022] FIG. 10 is a diagram showing a Decoder-side Intra Mode Derivation (DIMD) chroma mode.

[0023] FIG. 11 is a diagram showing a reference region for deriving filter coefficients.

[0024] FIG. 12 is a diagram showing a Sobel filter for gradient calculation.

[0025] FIG. 13 is a diagram showing a candidate luma block in a direct block vector (DBV) mode.

[0026] FIG. 14 is a diagram showing prediction of a chroma block in a DBV mode.

[0027] FIG. 15 is a diagram showing a collocated luma sample value resulting from application of a candidate down-sampling filter.

[0028] FIG. 16 is a diagram showing template matching prediction based on non-linear modeling according to at least one embodiment of the present disclosure.

[0029] FIG. 17 is a diagram showing a sequence of luma samples for checking in a discrete mapping model according to at least one embodiment of the present disclosure.

[0030] FIG. 18 is a diagram showing a non-linear filter of an adaptive loop filter (ALF) shape according to at least one embodiment of the present disclosure.

[0031] FIG. 19 is a diagram illustrating template matching prediction based on non-linear modeling according to another embodiment of the present application.

[0032] FIG. 20 is a flowchart of a method of encoding a current block by a video encoding apparatus according to at least one embodiment of the present application.

[0033] FIG. 21 is a flowchart of a method of reconstructing a current block by a video decoding apparatus according to at least one embodiment of the present application. DETAILED DESCRIPTION

[0034] Hereinafter, some embodiments of the present application will be described in detail with reference to the accompanying drawings. In the following description, the same drawing reference numerals are used for the same elements, even in different drawings. Further, in the following description of some embodiments, detailed descriptions of known components and functions incorporated herein are omitted for clarity and conciseness, when deemed that the detailed description of related known components and functions obfuscates the subject matter of the present application.

[0035] FIG. 1 is a block diagram of a video encoding apparatus to which the present application is applicable. Hereinafter, the video encoding apparatus and components of the apparatus are described with reference to the diagram of FIG. 1.

[0036] The encoding apparatus can include a picture partitioner 110, a predictor 120, a subtractor 130, a transformer 140, a quantizer 145, a rearrangement unit 150, an entropy encoder 155, an inverse quantizer 160, an inverse transformer 165, an adder 170, a loop filter unit 180, and a memory 190.

[0037] Each component of the encoding apparatus can be implemented as hardware or software, or a combination of hardware and software. In addition, the function of each component can be implemented as software, and a microprocessor can also be implemented to perform the function of the software corresponding to each component.

[0038] A video is composed of one or more sequences including a plurality of pictures. Each picture is divided into a plurality of regions, and encoding is performed for each region. For example, a picture is divided into one or more tiles or / and slices. Here, one or more tiles can be defined as a tile group. Each tile or / and slice is divided into one or more coding tree units (CTUs). In addition, each CTU is divided into one or more coding units (CUs) by a tree structure. Information applied to each coding unit (CU) is coded as a syntax of the CU, and information commonly applied to CUs included in one CTU is coded as a syntax of the CTU. In addition, information commonly applied to all blocks in one slice is coded as a syntax of a slice header, and information applied to all blocks constituting one or more pictures is coded as a picture parameter set (PPS) or a picture header. Furthermore, information commonly referred to by a plurality of pictures is coded as a sequence parameter set (SPS). In addition, information commonly referred to by one or more SPSs is coded as a video parameter set (VPS). Furthermore, information commonly applied to one tile or tile group can also be coded as a syntax of a tile or tile group header. Syntaxes included in the SPS, PPS, slice header, tile or tile group header can be referred to as high-level syntaxes.

[0039] The picture partitioner 110 determines the size of a coding tree unit (CTU). Information on the size of the CTU (CTU size) is coded as a syntax of the SPS or PPS, and is transmitted to the video decoding apparatus.

[0040] The picture partitioner 110 divides each picture constituting a video into a plurality of coding tree units (CTUs) having a predetermined size, and then recursively divides the CTUs by using a tree structure. A leaf node in the tree structure becomes a coding unit (CU), which is a basic unit of encoding.

[0041] The tree structure can be a quadtree (QT) in which a higher node (or parent node) is split into four lower nodes (or child nodes) having the same size. The tree structure can also be a binary tree (BT) in which a higher node is split into two lower nodes. The tree structure can also be a ternary tree (TT) in which a higher node is split into three lower nodes in a ratio of 1:2:1. The tree structure can also be a structure in which two or more of the QT structure, the BT structure, and the TT structure are mixed. For example, a quadtree plus binary tree (QTBT) structure can be used, or a quadtree plus binary tree ternary tree (QTBTTT) structure can be used. Here, a binary tree ternary tree (BTTT) is added to the tree structure to be called a multiple-type tree (MTT).

[0042] FIG. 2 is a schematic diagram for describing a method of splitting a block by using a QTBTTT structure.

[0043] As shown in FIG. 2, a CTU can be first split into a QT structure. The quadtree splitting can be recursive until the size of the split block reaches a minimum block size of a leaf node allowed in the QT (MinQTSize). A first flag (QT_split_flag) indicating whether each node of the QT structure is split into four nodes of a lower layer is encoded by the entropy encoder 155 and signaled to the video decoding apparatus. When a leaf node of the QT is not greater than a maximum block size of a root node allowed in the BT (MaxBTSize), the leaf node can be further split by at least one of a BT structure or a TT structure. There can be multiple splitting directions in the BT structure and / or the TT structure. For example, there can be two directions, i.e., a direction of splitting a block of a corresponding node horizontally and a direction of splitting the block of the corresponding node vertically. As shown in FIG. 2, when the MTT splitting starts, a second flag (mtt_split_flag) indicating whether a node is split, and a flag additionally indicating a splitting direction (vertical or horizontal) and / or a flag indicating a splitting type (binary or ternary) in the case where the node is split are encoded by the entropy encoder 155 and signaled to the video decoding apparatus.

[0044] Alternatively, a CU split flag (split_cu_flag) indicating whether a node is split or not can also be encoded before a first flag (QT_split_flag) indicating whether each node is split into four nodes of a lower layer is encoded. When a value of the CU split flag (split_cu_flag) indicates that each node is not split, blocks of the corresponding node become leaf nodes in the split tree structure and become a CU, which is a basic unit of encoding. When a value of the CU split flag (split_cu_flag) indicates that each node is split, the video encoding apparatus starts encoding the first flag first in the above-described scheme.

[0045] When QTBT is used as another example of a tree structure, there can be two types, i.e., a type in which a block of a corresponding node is horizontally split into two blocks having the same size (i.e., symmetric horizontal split) and a type in which the block of the corresponding node is vertically split into two blocks having the same size (i.e., symmetric vertical split). A split flag (split_flag) indicating whether each node of the BT structure is split into blocks of a lower layer and split type information indicating a split type are encoded by the entropy encoder 155 and are transmitted to the video decoding apparatus. On the other hand, there can additionally be a type in which the block of the corresponding node is split into two blocks that are asymmetric to each other. The asymmetric form can include a form in which the block of the corresponding node is split into two rectangular blocks having a size ratio of 1:3, or can further include a form in which the block of the corresponding node is split in a diagonal direction.

[0046] A CU can have various sizes according to QTBT or QTBTTT splitting from a CTU. Hereinafter, a block corresponding to a CU to be encoded or decoded (i.e., a leaf node of the QTBTTT) is referred to as a "current block". When QTBTTT splitting is employed, a shape of the current block can be a rectangular shape in addition to a square shape.

[0047] The predictor 120 predicts the current block to generate a prediction block. The predictor 120 includes an intra predictor 122 and an inter predictor 124.

[0048] In general, each of the current blocks in a picture can be predictively encoded. In general, the prediction of the current block can be performed by utilizing an intra prediction technique that utilizes data from the picture including the current block or an inter prediction technique that utilizes data from pictures that are encoded before the picture including the current block. The inter prediction includes both uni-prediction and bi-prediction.

[0049] The intra predictor 122 predicts pixels in the current block by utilizing pixels (reference pixels) located adjacent to the current block in the current picture including the current block. There are a plurality of intra prediction modes according to a prediction direction. For example, as shown in FIG. 3a, the plurality of intra prediction modes can include two non-directional modes including a Planar mode and a DC mode, and can include 65 directional modes. The adjacent pixels and algorithm equations to be used are defined differently according to each prediction mode.

[0050] In order to efficiently perform directional prediction on the current block having a rectangular shape, the directional modes (intra prediction modes #67 to #80, #-1 to #-14) shown by dotted arrows in FIG. 3b can be additionally used. The directional modes can be referred to as "wide angle intra-prediction modes". In FIG. 3b, the arrows indicate respective reference samples for prediction, not representative of the prediction direction. The prediction direction is opposite to the direction indicated by the arrow. When the current block has a rectangular shape, the wide angle intra-prediction mode is a mode in which prediction is performed in a direction opposite to a specific directional mode without additional bit transmission. In this case, in the wide angle intra-prediction mode, some of the wide angle intra-prediction modes available for the current block can be determined by a ratio of the width to the height of the current block having a rectangular shape. For example, when the current block has a rectangular shape having a height smaller than a width, the wide angle intra-prediction modes (intra prediction modes #67 to #80) having an angle smaller than 45 degrees are available. When the current block has a rectangular shape having a width greater than a height, the wide angle intra-prediction modes (intra prediction modes -1 to -14) having an angle greater than -135 degrees are available.

[0051] The intra predictor 122 can determine an intra prediction to be used for encoding the current block. In some examples, the intra predictor 122 can encode the current block by utilizing a plurality of intra prediction modes, and can also select an appropriate intra prediction mode to be used from among the test modes. For example, the intra predictor 122 can calculate rate-distortion values by utilizing rate-distortion analysis on a plurality of test intra prediction modes, and can also select an intra prediction mode having the best rate-distortion characteristics among the test modes.

[0052] The intra predictor 122 selects one of the plurality of intra prediction modes, and predicts the current block by utilizing the adjacent pixels (reference pixels) and algorithm equations determined according to the selected intra prediction mode. Information on the selected intra prediction mode is encoded by the entropy encoder 155 and transmitted to the video decoding apparatus.

[0053] The inter predictor 124 generates a prediction block of the current block by using a motion compensation process. The inter predictor 124 searches for a block most similar to the current block among reference pictures that have been encoded and decoded earlier than the current picture, and generates a prediction block of the current block by using the searched block. In addition, a motion vector (MV) corresponding to a displacement between the current block in the current picture and the prediction block in the reference picture is generated. Generally, motion estimation is performed on a luma component, and the motion vector calculated based on the luma component is used for both the luma component and a chroma component. Motion information including information of the reference picture and information on the motion vector used to predict the current block is encoded by the entropy encoder 155 and transmitted to the video decoding apparatus.

[0054] The inter predictor 124 can also perform interpolation of a reference picture or a reference block to increase the accuracy of prediction. In other words, a sub-sample is interpolated between two consecutive integer samples by applying a filter coefficient to a plurality of consecutive integer samples including the two integer samples. When performing a process of searching for a block most similar to the current block with respect to an interpolated reference picture, a decimal unit precision can be expressed for a motion vector instead of an integer sample unit precision. The precision or resolution of the motion vector can be differently set for each target region to be encoded, for example, a unit such as a slice, a tile, a CTU, a CU, or the like. When such adaptive motion vector resolution (AMVR) is applied, information on the motion vector resolution to be applied to each target region should be signaled for each target region. For example, when the target region is a CU, information on the motion vector resolution applied to each CU is signaled. The information on the motion vector resolution can be information representing the precision of a motion vector difference described below.

[0055] On the other hand, the inter predictor 124 can perform inter prediction by using bi-prediction. In the case of bi-prediction, two reference pictures and two motion vectors representing the most similar block position to the current block in each of the reference pictures are used. The inter predictor 124 selects a first reference picture and a second reference picture from a reference picture list 0 (RefPicListO) and a reference picture list 1 (RefPicListl), respectively. The inter predictor 124 also searches for blocks most similar to the current block in the respective reference pictures to generate a first reference block and a second reference block. Further, a prediction block of the current block is generated by averaging or weighted-averaging the first reference block and the second reference block. In addition, motion information including information on the two reference pictures used for predicting the current block and including information on the two motion vectors is transmitted to the entropy encoder 155. Here, the reference picture list 0 can be constituted by pictures in the pre-reconstructed pictures that precede the current picture in display order, and the reference picture list 1 can be constituted by pictures in the pre-reconstructed pictures that follow the current picture in display order. However, although not particularly limited thereto, a pre-reconstructed picture that follows the current picture in display order can be additionally included in the reference picture list 0. Conversely, a pre-reconstructed picture that precedes the current picture can also be additionally included in the reference picture list 1.

[0056] In order to minimize the amount of bits consumed for encoding the motion information, various methods can be used.

[0057] For example, in the case where the reference picture and the motion vector of the current block are the same as those of a neighboring block, the information of the neighboring block can be identified to encode the motion information of the current block to be transmitted to the video decoding apparatus. This method is referred to as a merge mode.

[0058] In the merge mode, the inter predictor 124 selects a predetermined number of merge candidates (hereinafter, referred to as "merge candidates") from neighboring blocks of the current block.

[0059] As the neighboring blocks used to derive the merge candidates, all or some of a left block A0, a lower-left block Al, an upper block B0, an upper-right block Bl, and an upper-left block B2 adjacent to the current block in the current picture can be used, as illustrated in FIG. 4. In addition, blocks located within a reference picture (which can be the same as or different from the reference picture used to predict the current block) other than the current picture in which the current block is located can also be used as the merge candidates. For example, a co-located block of the current block within the reference picture or a block adjacent to the co-located block can be additionally used as the merge candidates. If the number of the merge candidates selected by the above-described method is less than a predetermined number, a zero vector is added to the merge candidates.

[0060] The inter predictor 124 configures a merge list including a predetermined number of merge candidates by using neighboring blocks. A merge candidate to be used as motion information of the current block is selected from among the merge candidates included in the merge list, and merge index information for identifying the selected candidate is generated. The generated merge index information is encoded by the entropy encoder 155 and transmitted to the video decoding apparatus.

[0061] The merge skip mode is a special case of the merge mode. After quantization, when all transform coefficients for entropy encoding are close to zero, only neighboring block selection information is transmitted without transmitting a residual signal. By using the merge skip mode, relatively high encoding efficiency can be achieved for images with slight motion, still images, screen content images, and the like.

[0062] Hereinafter, the merge mode and the merge skip mode are collectively referred to as a merge / skip mode.

[0063] Another method for encoding motion information is an advanced motion vector prediction (AMVP) mode.

[0064] In the AMVP mode, the inter predictor 124 derives a motion vector prediction candidate for a motion vector of the current block by using neighboring blocks of the current block. As the neighboring blocks for deriving the motion vector prediction candidate, all or some of the left block A0, the lower-left block Al, the upper block B0, the upper-right block Bl, and the upper-left block B2 adjacent to the current block in the current picture shown in FIG. 4 can be used. In addition, blocks located within a reference picture (which can be the same as or different from the reference picture used for predicting the current block) other than the current picture in which the current block is located can also be used as the neighboring blocks for deriving the motion vector prediction candidate. For example, a co-located block or a block adjacent to the co-located block within the reference picture of the current block can be used. If the number of motion vector candidates selected by the above-described method is less than a predetermined number, a zero vector is added to the motion vector candidates.

[0065] The inter predictor 124 derives a motion vector prediction candidate by using a motion vector of a neighboring block, and determines a motion vector prediction of a motion vector of the current block by using the motion vector prediction candidate. In addition, a motion vector difference is calculated by subtracting the motion vector prediction from the motion vector of the current block.

[0066] Motion vector prediction can be obtained by applying a predefined function (e.g., median and average value calculation, etc.) to the motion vector prediction candidates. In this case, the video decoding device also knows the predefined function. In addition, since the neighboring blocks used to derive the motion vector prediction candidates are blocks that have already been encoded and decoded, the video decoding device can also already know the motion vectors of the neighboring blocks. Therefore, the video encoding device does not need to encode information for identifying the motion vector prediction candidates. Accordingly, in this case, the information about the motion vector difference and the information about the reference picture used to predict the current block are encoded.

[0067] On the other hand, the motion vector prediction can also be determined by a scheme of selecting any one of the motion vector prediction candidates. In this case, the information for identifying the selected motion vector prediction candidate is additionally encoded together with the information about the motion vector difference and the information about the reference picture used to predict the current block.

[0068] The subtractor 130 generates a residual block by subtracting the prediction block generated by the intra predictor 122 or the inter predictor 124 from the current block.

[0069] The transformer 140 transforms a residual signal in the residual block having pixel values in a spatial domain into a transform coefficient in a frequency domain. The transformer 140 can transform the residual signal in the residual block by using the entire size of the residual block as a transform unit, or can also divide the residual block into a plurality of sub-blocks and can perform the transformation by using the sub-blocks as transform units. Alternatively, the residual block is divided into two sub-blocks, i.e., a transform region and a non-transform region, to transform the residual signal by using only the transform region sub-block as a transform unit. Here, the transform region sub-block can be one of two rectangular blocks having a size ratio of 1:1 based on a horizontal axis (or a vertical axis). In this case, a flag (cu_sbt_flag) indicating only the transform sub-block, and direction (vertical / horizontal) information (cu_sbt_horizontal_flag) and / or position information (cu_sbt_pos_flag) are encoded by the entropy encoder 155 and signaled to the video decoding device. In addition, the size of the transform region sub-block based on the horizontal axis (or the vertical axis) can have a size ratio of 1:3. In this case, a flag (cu_sbt_quad_flag) indicating division of the corresponding partition is additionally encoded by the entropy encoder 155 and signaled to the video decoding device.

[0070] On the other hand, the transformer 140 can perform a transform of the residual block separately in a horizontal direction and a vertical direction. For the transform, various types of transform functions or transform matrices can be used. For example, a pair of transform functions used for horizontal transform and vertical transform can be defined as a multiple transform set (MTS). The transformer 140 can select one transform function pair having the highest transform efficiency among the MTS, and can transform the residual block in each of the horizontal direction and the vertical direction. Information (mts_idx) about the transform function pair in the MTS is encoded by the entropy encoder 155 and signaled to the video decoding apparatus.

[0071] The quantizer 145 quantizes the transform coefficients output from the transformer 140 with a quantization parameter, and outputs the quantized transform coefficients to the entropy encoder 155. The quantizer 145 can also quantize the relevant residual block immediately without transforming any block or frame. The quantizer 145 can also apply different quantization coefficients (scaling values) according to the positions of the transform coefficients in the transform block. A quantization matrix applied to the quantized transform coefficients arranged in two dimensions can be encoded and signaled to the video decoding apparatus.

[0072] The rearrangement unit 150 can perform a rearrangement of coefficient values on the quantized residual values.

[0073] The rearrangement unit 150 can change a 2D coefficient array into a 1D coefficient sequence by utilizing a coefficient scan. For example, the rearrangement unit 150 can scan a DC coefficient to a coefficient of a high frequency region with a zig-zag scan or a diagonal scan to output a 1D coefficient sequence. According to the size of the transform unit and the intra prediction mode, a vertical scan that scans a 2D coefficient array in a column direction and a horizontal scan that scans a 2D block type coefficient in a row direction can also be utilized instead of the zig-zag scan. In other words, according to the size of the transform unit and the intra prediction mode, a scan method to be used can be determined among the zig-zag scan, the diagonal scan, the vertical scan, and the horizontal scan.

[0074] The entropy encoder 155 encodes the sequence of the 1D quantized transform coefficients output from the rearrangement unit 150 by utilizing various encoding schemes including a Context-based Adaptive Binary Arithmetic Code (CABAC), an Exponential Golomb, etc., to generate a bitstream.

[0075] Further, the entropy encoder 155 encodes information related to block partitioning (e.g., CTU size, CU partition flags, QT partition flags, MTT partition types, and MTT partition directions, etc.) to enable a video decoding device to partition blocks identically to the video encoding device. Further, the entropy encoder 155 encodes information on a prediction type indicating whether the current block is coded by intra prediction or by inter prediction. The entropy encoder 155 encodes intra prediction information (i.e., information on an intra prediction mode) or inter prediction information (a merge index in the case of merge mode, and information on a reference picture index and a motion vector difference in the case of AMVP mode) according to the prediction type. Further, the entropy encoder 155 encodes information related to quantization (i.e., information on a quantization parameter and information on a quantization matrix).

[0076] The inverse quantizer 160 inverse-quantizes quantized transform coefficients output from the quantizer 145 to generate transform coefficients. The inverse transformer 165 transforms the transform coefficients output from the inverse quantizer 160 from a frequency domain to a spatial domain to reconstruct a residual block.

[0077] The adder 170 adds the reconstructed residual block and the prediction block generated by the predictor 120 to reconstruct the current block. Pixels in the reconstructed current block are used as reference pixels when intra-predicting a next block.

[0078] The loop filter 180 performs filtering on reconstructed pixels to reduce blocking artifacts, ringing artifacts, blurring artifacts, etc. that occur due to block-based prediction and transform / quantization. The loop filter 180 as an in-loop filter can include all or some of a deblocking filter 182, a sample adaptive offset (SAO) filter 184, and an adaptive loop filter (ALF) 186.

[0079] The deblocking filter 182 filters boundaries between reconstructed blocks to remove blocking artifacts that occur due to block unit encoding / decoding, and the SAO filter 184 and the ALF 186 additionally filter the deblocking-filtered video. The SAO filter 184 and the ALF 186 are filters for compensating for a difference between reconstructed pixels and original pixels that occurs due to lossy coding. The SAO filter 184 applies an offset in units of CTU to enhance subjective image quality and coding efficiency. On the other hand, the ALF 186 performs block unit filtering, and applies different filters by dividing a boundary of a corresponding block and a degree of change to compensate for distortion. Information on filter coefficients to be used for the ALF can be encoded and signaled to the video decoding apparatus.

[0080] The reconstructed blocks filtered through the deblocking filter 182, the SAO filter 184, and the ALF 186 are stored in the memory 190. When all blocks in one picture are reconstructed, the reconstructed picture can be used as a reference picture for inter-prediction of blocks within a picture to be encoded subsequently.

[0081] The video encoding apparatus can store a bitstream of the encoded video data in a nonvolatile storage medium or transmit the bitstream to the video decoding apparatus through a communication network.

[0082] FIG. 5 is a functional block diagram of a video decoding apparatus that can implement the present technology. Hereinafter, with reference to FIG. 5, a video decoding apparatus and components of the apparatus are described.

[0083] The video decoding apparatus can include an entropy decoder 510, a rearrangement unit 515, an inverse quantizer 520, an inverse transformer 530, a predictor 540, an adder 550, a loop filter 560, and a memory 570.

[0084] Like the video encoding apparatus of FIG. 1, each component of the video decoding apparatus can be implemented as hardware or software, or as a combination of hardware and software. In addition, the function of each component can be implemented as software, and a microprocessor can also be implemented to perform the function corresponding to each component.

[0085] The entropy decoder 510 extracts information related to block partitioning by decoding a bitstream generated by the video encoding apparatus to determine a current block to be decoded, and extracts prediction information required to reconstruct the current block and information on a residual signal.

[0086] The entropy decoder 510 determines the size of a CTU by extracting information on the CTU size from a sequence parameter set (SPS) or a picture parameter set (PPS), and partitions a picture into CTUs having the determined size. Also, the CTU is determined as the highest layer of a tree structure (i.e., a root node), and partitioning information of the CTU can be extracted to partition the CTU by utilizing the tree structure.

[0087] For example, when the CTU is partitioned by utilizing the QTBTTT structure, a first flag (QT_split_flag) related to the partitioning of the QT is first extracted to partition each node into four nodes of a lower layer. In addition, a second flag (mtt_split_flag), a partitioning direction (vertical / horizontal), and / or a partitioning type (binary / ternary) related to the partitioning of the MTT are extracted with respect to a node corresponding to a leaf node of the QT to partition the corresponding leaf node in the MTT structure. As a result, each node below the leaf node of the QT is recursively partitioned in the BT or TT structure.

[0088] As another example, when the CTU is partitioned by utilizing the QTBTTT structure, a CU partitioning flag (split_cu_flag) indicating whether to partition a CU is extracted. When the corresponding block is partitioned, a first flag (QT_split_flag) can also be extracted. During the partitioning process, for each node, 0 or more times of recursive MTT partitioning can occur after 0 or more times of recursive QT partitioning. For example, for a CTU, the MTT partitioning can occur immediately, or vice versa, and only multiple times of QT partitioning can occur.

[0089] As another example, when the CTU is partitioned by utilizing the QTBT structure, a first flag (QT_split_flag) related to the partitioning of the QT is extracted to partition each node into four nodes of a lower layer. In addition, a partitioning flag (split_flag) indicating whether to further partition a node corresponding to a leaf node of the QT in a BT and partitioning direction information are extracted.

[0090] On the other hand, when the entropy decoder 510 determines a current block to be decoded by utilizing the partitioning of the tree structure, the entropy decoder 510 extracts information on prediction type information indicating whether the current block is intra-predicted or inter-predicted. When the prediction type information indicates the intra-prediction, the entropy decoder 510 extracts a syntax element of intra-prediction information (intra-prediction mode) for the current block. When the prediction type information indicates the inter-prediction, the entropy decoder 510 extracts information of a syntax element representing the inter-prediction information, i.e., a motion vector and a reference picture of a motion vector reference.

[0091] Further, the entropy decoder 510 extracts quantization-related information and extracts information on quantized transform coefficients of the current block as information on the residual signal.

[0092] The rearrangement unit 515 can rearrange the sequence of the 1D quantized transform coefficients entropy-decoded by the entropy decoder 510 into a 2D coefficient array (i.e., a block) in the reverse order of the coefficient scan order performed by the video encoding apparatus.

[0093] The inverse quantizer 520 inverse-quantizes the quantized transform coefficients and inverse-quantizes the quantized transform coefficients by utilizing a quantization parameter. The inverse quantizer 520 can also apply different quantization coefficients (scaling values) to the quantized transform coefficients arranged in 2D. The inverse quantizer 520 can perform inverse quantization by applying a matrix of quantization coefficients (scaling values) from the video encoding apparatus to the 2D array of quantized transform coefficients.

[0094] The inverse transformer 530 reconstructs the residual signal by inverse-transforming the inverse-quantized transform coefficients from the frequency domain to the spatial domain to generate a residual block of the current block.

[0095] Further, when the inverse transformer 530 inverse-transforms a partial region (sub-block) of the transform block, the inverse transformer 530 extracts a flag (cu_sbt_flag) to transform only the sub-block of the transform block, directional (vertical / horizontal) information (cu_sbt_horizontal_flag) of the sub-block, and / or position information (cu_sbt_pos_flag) of the sub-block. The inverse transformer 530 also inverse-transforms the transform coefficients of the corresponding sub-block from the frequency domain to the spatial domain to reconstruct the residual signal and fills a region not inverse-transformed with a value "0" as the residual signal to generate a final residual block of the current block.

[0096] Further, when MTS is applied, the inverse transformer 530 determines a transform function or a transform matrix to be applied on each of the horizontal direction and the vertical direction by utilizing MTS information (mts_idx) signaled from the video encoding apparatus. The inverse transformer 530 also performs inverse transformation on the transform coefficients in the transform block in the horizontal direction and the vertical direction by utilizing the determined transform function.

[0097] The predictor 540 can include an intra predictor 542 and an inter predictor 544. The intra predictor 542 is activated when the prediction type of the current block is intra prediction, and the inter predictor 544 is activated when the prediction type of the current block is inter prediction.

[0098] The intra predictor 542 determines an intra prediction mode of the current block from among a plurality of intra prediction modes according to syntax elements of the intra prediction mode extracted from the entropy decoder 510, and predicts the current block by using neighboring reference pixels of the current block according to the intra prediction mode.

[0099] The inter predictor 544 determines a motion vector and a reference picture of a reference of the motion vector of the current block by using syntax elements of the inter prediction mode extracted from the entropy decoder 510, and predicts the current block by using the motion vector and the reference picture.

[0100] The adder 550 reconstructs the current block by adding the residual block output from the inverse transformer 530 to the prediction block output from the inter predictor 544 or the intra predictor 542. Pixels within the reconstructed current block are used as reference pixels when intra predicting a block to be decoded next.

[0101] The loop filter unit 560, which is an in-loop filter, can include a deblocking filter 562, an SAO filter 564, and an ALF 566. The deblocking filter 562 performs deblocking filtering on boundaries between reconstructed blocks in order to remove blocking artifacts occurring due to block-based decoding. The SAO filter 564 and the ALF 566 perform additional filtering on the reconstructed blocks after deblocking filtering in order to compensate for differences between reconstructed pixels and original pixels occurring due to lossy encoding. Filter coefficients of the ALF are determined by using information about the filter coefficients decoded from the bitstream.

[0102] The reconstructed blocks filtered by the deblocking filter 562, the SAO filter 564, and the ALF 566 are stored in the memory 570. When all blocks in one picture are reconstructed, the reconstructed picture can be used as a reference picture for inter predicting blocks within a picture to be encoded next.

[0103] The present invention, in some embodiments thereof, relates to encoding and decoding video images as described above. More specifically, the present invention provides a video coding method and apparatus that utilizes a template matching technique based on a discrete mapping model or a template matching technique based on a non-linear filter when performing template matching prediction. The video coding method and apparatus utilizes a cross-component discrete mapping model or a cross-component non-linear model based on the template matching technique.

[0104] The following embodiments can be performed by the intra predictor 122 in a video encoding apparatus. The following embodiments can also be performed by the intra predictor 542 in a video decoding apparatus.

[0105] A video coding device can generate the signaling information associated with the present embodiment from the perspective of optimizing rate-distortion when encoding a current block. The video coding device can encode the signaling information using the entropy encoder 155 and transmit the encoded signaling information to the video decoding device. The video decoding device can decode the signaling information associated with the decoding of the current block from the bitstream using the entropy decoder 510.

[0106] In the following description, the term "target block" can be used interchangeably with a current block or a coding unit (CU), or can refer to some region of a coding unit.

[0107] In addition, a value of true for a flag indicates a case where the flag is set to 1. Also, a value of false for a flag indicates a case where the flag is set to 0.

[0108] I. Template matching based prediction In encoding / decoding with template matching based prediction, a reference block corresponding to a template most similar to a template of a current block is used as a matching block. In template matching based decoding, a video decoding device searches for a matching block based on template matching between a template of a reference block and a template of a current block, and determines a prediction block of the current block as the searched matching block. In other words, the matching block represents a (selected) reference block according to template matching. Hereinafter, a template of a reference block and a reference template are used interchangeably, and a template of a current block and a current template are used interchangeably.

[0109] A video decoding device uses a metric such as sum of absolute differences (SAD), sum of squared errors (SSE), etc. to calculate a similarity based on template matching. Since template matching is performed by utilizing multiple reconstructed reference blocks, template matching based prediction increases the computational complexity of a video decoding process. To reduce the computational complexity, template matching is performed by defining a search region to search for a reference block. As shown in FIG. 6, R1, R2, R3, and R4 correspond to a search region for template search. In the example of FIG. 6, (x, y) represents a position of a top-left sample of a current block.

[0110] In the following, predicting the mode of the current block based on template matching is referred to as template matching based prediction mode. In addition, the method of performing intra prediction on the current block by utilizing the template matching based prediction mode is referred to as template matching based intra prediction or intra template matching prediction (intra TMP). In the following, the template matching based intra prediction and the template matching based prediction are used interchangeably. The template matching and the intra template matching are used interchangeably.

[0111] Now referring to FIG. 7, a method for predicting and reconstructing the current block by utilizing the template matching based prediction mode is described.

[0112] FIG. 7 is a flowchart of reconstructing the current block performed by the video decoding apparatus according to at least one embodiment of the present application.

[0113] The video decoding apparatus obtains the size of the current block (S700).

[0114] The video decoding apparatus determines whether the size of the current block satisfies the range condition (S702). The video decoding apparatus determines whether the size of the current block satisfies the range condition for decoding on the basis of the template matching based prediction. For example, the range condition refers to the minimum size condition of the current block and / or the maximum size condition of the current block. The reconstruction process is divided into the reconstruction process utilizing the template matching and the reconstruction process not utilizing the template matching according to whether the range condition is satisfied.

[0115] If the range condition is satisfied (S702 is Yes), the video decoding apparatus decodes the template matching flag (S704). The template matching flag indicates whether the intra template matching is to be used for the current block.

[0116] The video decoding apparatus checks the template matching flag (S706).

[0117] If the template matching flag is true (S706 is Yes), the video decoding apparatus generates the prediction block of the current block based on the template matching (S708).

[0118] If the range condition is not satisfied (S702 is No) or the template matching flag is false (S706 is No), the video decoding apparatus generates the prediction block of the current block by utilizing the prediction mode not using the template matching (S720). For example, the video decoding apparatus can utilize the intra prediction mode not using the template matching or can utilize the inter prediction mode.

[0119] Then, the video decoding apparatus can decode the residual block and can sum the residual block and the prediction block to generate the reconstructed block of the current block.

[0120] On the other hand, filtering can be applied to intra template matching prediction. A filter is applied to the matching block, and the resulting filtered matching block is used as the prediction block for the current block. For example, the filter is a 6-tap linear filter of cross shape with five luma taps and one bias term. As shown in FIG. 8, the input to the spatial 5-tap component of the filter consists of the center (C) sample, the upper / north (N) sample, the lower / south (S) sample, the left / west (W) sample, and the right / east (E) sample of the matching block at the positions corresponding to the samples of the current block. In the following, the 6-tap filter is also referred to as 5-tap filter without considering the bias term.

[0121] The output of the filter is calculated as shown in Equation 1.

[0122] [Equation 1] predLumaVal = c0C + c1N + c2S + c3E + c4W + c5B In Equation 1, B denotes the bias term.

[0123] The filter coefficients c0to c5as shown in Equation 1 are derived for each block by utilizing a regression model that minimizes the mean square error (MSE) between the output of the filter and the samples in the template of the current block. The MSE minimization includes computation of the auto-correlation matrix between the reference template and the current template, LDL decomposition of the auto-correlation matrix, and computation of the filter coefficients by back substitution.

[0124] A filter flag indicating whether the filter is to be used is sent from the video encoding device to the video decoding device at the CU level. In addition, the use of the linear filter is considered as a submode of intra template matching prediction. For example, if a template matching flag indicating whether intra template matching prediction is to be used is true, the filter flag is signaled.

[0125] II. Enhanced Compression Model (ECM) chroma prediction mode In ECMs corresponding to Beyond VVC (Versatile Video Coding), a multi-model LM (multi-model LM, MMLM) mode is added, which is an extension of the cross-component linear model (CCLM) included in the existing VVC. The MMLM mode derives a linear model by applying a least mean square (LMS) method to the luminance samples and improves accuracy by adjusting the gradient of the linear model. As shown in FIG. 9, the gradient adjustment is performed by using a linear function that maps a luminance value to a chroma value using a center point determined based on the average luminance value of the reference samples. The linear function of the MMLM mode is derived from the linear function of the CCLM mode as shown in Equation 2.

[0126] [Equation 2] chromaVal = a ·lumaVal + b a = a + u,b = b - u·y r In Equation 2, a and b are parameters representing the linear model of the CCLM mode shown in FIG. 9, and a and b are parameters representing the linear function of the MMLM mode. y r represents the reference luminance sample value corresponding to the center point.

[0127] Decoder-side intra mode derivation (DIMD) chroma determines the intra prediction mode of the current chroma block. As shown in FIG. 10, DIMD chroma mode is used based on the neighboring luminance and chroma samples decoded from the neighboring rows and columns. A histogram of gradients is calculated with horizontal and vertical gradient calculations performed on the luminance samples, Cb and Cr samples decoded at the same position in the current chroma block. The prediction mode with the largest histogram value is used for the intra prediction of the current chroma block. If the intra prediction mode derived according to the DIMD chroma mode is the same as the intra prediction mode derived according to the DM mode, the intra prediction mode with the second largest histogram value is used for the intra prediction of the current chroma block. A CU-level flag can be signaled to indicate the applicability of the DIMD chroma mode.

[0128] In ECMs, two chroma prediction blocks can be fused together to generate a final predicted signal as shown in Equation 3.

[0129] [Equation 3] In Equation 3, one of the two chroma prediction blocks is generated according to a derived mode (DM), a DIMD chroma mode, or one of four base modes (non-LM mode). The other prediction block can be generated according to a cross-component linear prediction mode. For example, pred0(i,j) can be predicted according to a non-linear model mode (non-LM mode), and pred1(i,j) can be predicted according to a MMLM LT mode. The four base modes represent a DC mode, a planar mode, a horizontal mode (No. 18), and a vertical mode (No. 50). The MMLM LT mode represents a MMLM mode that utilizes left and top side samples to derive a linear model.

[0130] In Equation 3, the weights w0and w1are determined based on the intra-prediction mode of the neighboring chroma block, and the offset is set to 2. When both the top and left neighboring blocks are coded in LM mode, {w0, w1} = {1, 3}. When both the top and left neighboring blocks are not coded in LM mode, {w0, w1} = {3, 1}. In all other cases, {w0, w1} = {2, 2}.

[0131] As another example, the final predicted signal can be generated as shown in Equation 4.

[0132] [Equation 4] In Equation 4, pred0(i,j) is predicted according to a non-LM mode, and rec' L(i,j) represents the down-sampled collocated luma samples.

[0133] A convolutional cross-component model mode (CCCM mode) predicts chroma samples from reconstructed luma samples, similar to the CCLM mode. When chroma subsampling is used, the CCCM mode down-samples the luma samples to align their positions with the chroma samples, to compensate for the resolution difference between luma samples and chroma samples. The CCCM mode computes the model by utilizing top, left, or top-left reference samples.

[0134] A 7-tap convolutional filter in the CCCM mode consists of a 5-tap cross-shaped spatial component, a non-linear term, and a bias term. The 5-tap component of the filter can be represented as shown in FIG. 8. For example, the 5-tap component of the filter consists of a center (C) luma sample, a bottom / south (S), a left / west (W), a right / east (E), and a top / north (N) sample.

[0135] The non-linear term P is expressed as a power of the center luma sample C, as shown in Equation 5, and is scaled by a bit-depth.

[0136] [Equation 5] P = ( C · C + 512 ) » 10 For images with 10-bit depth, the non-linear term P is expressed as in Equation 6.

[0137] [Equation 6] P = ( C · C + 512 ) » 10 The bias term B represents a scalar offset between the input and output, and is an intermediate chroma value of 512 for images with 10-bit depth. The output of the convolution filter is computed as the convolution of the filter coefficients c i with the input values, and is clipped to the valid range of chroma samples. The convolution filter ultimately computes the predicted value as shown in Equation 7.

[0138] [Equation 7] predChromaVal = c0C + c1N + c2S + c3E + c4W + c5P + c6B The filter coefficients c i are computed to minimize the mean squared error (MSE) between the predicted chroma samples within the reference region and the reconstructed chroma samples. In the example of FIG. 11, the six lines located above and to the left of the prediction unit (PU) represent the reference region of chroma samples. The reference region extends towards the lower-left and upper-right of the boundaries of the PU by an area equal to the width and height of the PU, respectively. The spatial filter of cross-shape can be derived using the reference region and the shaded region around the PU, and the unavailable region can be padded to make it available.

[0139] The MSE minimization is computed by utilizing the auto-correlation matrix for luma samples and the cross-correlation vector between luma input and chroma output. The auto-correlation matrix undergoes LDL decomposition, and the final filter coefficients are computed via back substitution. The MSE minimization roughly follows the process of computing the ALF filter coefficients in ECM. However, LDL decomposition is used instead of Cholesky decomposition to avoid square root operations.

[0140] When in YUV 4:2:0 color format, a gradient linear model (GLM) is used to predict chroma samples from luma sample gradients. The GLM mode uses two or three variables. When using two variables, the GLM utilizes luma sample gradients to derive a linear model. In this case, the derivation method for CCLM parameters and the linear transformation of the predicted samples remain unchanged, and are used in their original form. In the GLM using two variables, the chroma sample C is expressed as in Equation 8.

[0141] [Equation 8] C = a G + b In Equation 8, a and b represent parameters of a linear model derived according to the CCLM mode, and G represents a gradient.

[0142] The 3-variable GLM mode predicts chroma samples using three variables based on a gradient of luma samples and down-sampled luma values. Based on the LDL decomposition based MSE minimization method used in CCCM, model parameters for the 3-variable GLM are derived by utilizing rows and columns of neighboring six samples. In the 3-variable GLM, a chroma sample C is expressed as in Equation 9.

[0143] [Equation 9] In Equation 9, G and L represent a gradient of luma samples and down-sampled luma values, respectively, and a0, a1, and b represent parameters of the GLM model.

[0144] From a signaling perspective, when the CCLM mode is enabled with a current CU, the present application signals a flag indicating whether GLM is to be enabled for both Cb and Cr components. When GLM is enabled, another flag is signaled indicating one of two GLM modes. In addition, as shown in FIG. 12, the present application also signals a filter index indicating one of four Sobel filters used for gradient calculation. In the example of FIG. 12, the circle at the center of each filter indicates the position of a chroma sample corresponding to six luma samples.

[0145] A direct block vector (DBV) mode is used for chroma blocks in dual tree slices. When dual tree is enabled, a flag is signaled indicating whether a chroma block is coded by utilizing an intra block copy (IBC) mode. When the DBV mode is enabled, the position of the chroma block and the position of the candidate luma block correspond to those shown in FIG. 13. In FIG. 13, the candidate luma blocks are located at the center, top-left, top-right, bottom-left, and bottom-right in the corresponding luma region of the chroma block. When one of the five candidate luma blocks corresponding to the chroma block coded in the DBV mode is coded in the IBC mode or intra TMP mode, the block vector luma (bvL) of the one luma block is used to derive the block vector chroma (bvC) of the chroma block. The block vector of the luma block is scaled and used as the block vector of the chroma block. When template matching is used for block vector scaling and the luma block is coded by utilizing reconstruction reordered IBC (RR-IBC), horizontal or vertical block vector (BV) adjustment is performed on bvL in the same way as for RR-IBC.

[0146] As shown in FIG. 14, the DBV mode uses the location of the current chroma block (xCb, yCb) and bvC to determine a corresponding offset location (xCb + bvC[0], yCb + bvC[1]) and predict the chroma block.

[0147] Screen content coding attempts to improve coding efficiency by employing discrete models compared to natural video where linear filters work efficiently. In contrast to the CCLM mode which uses a continuous linear model, the cross-component discrete mapping model (CCDMM) stores pairs of luma and chroma values from neighboring samples of the reconstruction and derives a discrete mapping that maps based on the stored pairs. The CCDMM mode uses the stored pairs to predict chroma samples from collocated luma samples. In a pair of luma and chroma values, the luma value corresponds to the input and the chroma value corresponds to the output. In the following, the set of pairs of luma and chroma values is defined as the mapping set or the discrete mapping set of the discrete mapping model.

[0148] The CCDMM mode uses the decoded region to derive a mapping function between luma and chroma. In contrast to the continuous model of the CCLM mode or the CCCM mode, the discrete mapping model of the CCDMM mode directly stores luma-chroma pairs without a continuous function. To predict a chroma sample in the current chroma block, the CCDMM mode refers to collocated luma values Y c to derive a chroma prediction value according to the discrete mapping model. When there is no searchable Y c input of the mapping set of the discrete mapping model, the CCDMM mode checks luma-chroma pairs in the order {Y c+1 , Y c-1 , Y c+2 , Y c-2} as shown in FIG. 17. In the example of FIG. 17, c = 3, meaning Y c = Y3. If none of the five values match any input of the discrete mapping set, the CCDMM mode predicts the chroma sample by computing the average of neighboring reconstructed chroma samples.

[0149] For example, to generate the best collocated luma sample value for YCbCr 4:2:0 color format, five candidate down-sampling filters are introduced as shown in FIG. 15. In the example of FIG. 15, the circles represent luma samples and the triangles (Y1 to Y5) represent the collocated luma sample values resulting from applying the candidate down-sampling filters. The video encoding device uses the candidate down-sampling filters to generate the collocated luma sample values to select the index of the best filter. In a similar approach to GLM, the video encoding device signals the index selected to the video decoding device.

[0150] The following embodiments are described in terms of a video decoding device, but they can be implemented in the same or similar manner in a video encoding device.

[0151] In the following description related to the processing after template matching, the matching block as the reference block selected according to the template matching and the reference block can be used interchangeably. Thus, the reference template can represent the template of the matching block. In addition, in the following description related to the application of the IBC mode, the matching block as the reference block selected by the block vector and the reference block can be used interchangeably.

[0152] III. Embodiments according to the present application FIG. 16 is a schematic diagram illustrating template matching prediction based on nonlinear modeling according to at least one embodiment of the present application.

[0153] As shown in FIG. 16, the video decoding device according to the present application uses the current template and the reference template to derive a nonlinear model, and uses the derived model to filter the reference block to generate the final current block. The following describes in turn (Implementation 1) template matching technique based on discrete mapping model, (Implementation 2) template matching technique based on nonlinear filter, (Implementation 3) cross-component discrete mapping model based on template matching technique, and (Implementation 4) cross-component nonlinear model based on template matching technique. The present application can be specifically applied to special video compression, such as screen content. Since it is not a linear filter, the discrete mapping model can be included in the nonlinear filter.

[0154] (Implementation 1) Template matching technique based on discrete mapping model This implementation describes a template matching technique with a discrete mapping model. The video decoding device derives a discrete mapping model between the template of the current block and the template of the reference block, and applies the derived model to the reference block to generate the current block.

[0155] The video decoding device directly stores the luminance pairs between the reference template and the current template to derive the discrete mapping model due to the lack of calculation of continuous functions (e.g., linear functions). That is, the discrete mapping model is implemented as a nonlinear filter. The video decoding device according to the present application derives the discrete mapping model by storing the luminance pairs representing the luminance signal relationship between the template of the reference block and the luminance signal of the template of the current block in the decoded area. This replaces the conventional CCDMM technique of deriving the discrete mapping model by storing the mapping relationship between the luminance signal and the co-located chroma signal. In the luminance pairs, the luminance value of the reference template / reference block is equal to the input, and the luminance value of the current template / current block is equal to the output. In the following, the set containing the luminance pairs is defined as the mapping set of the discrete mapping model, or simply the discrete mapping set.

[0156] The video decoding device predicts samples of the current block by deriving predicted values from luma values of the reference block based on a pre-derived discrete mapping model. When deriving the predicted values, the video decoding device can select one of the following approaches.

[0157] For example, the video decoding device predicts samples of the current block by applying the derived discrete mapping model to samples of the reference block. For example, the video decoding device derives a predicted value Ypredfrom luma values Y c of the reference block at a common location with the current block based on the discrete mapping model. When the input of the discrete mapping set does not have a searchable Y c , the video decoding device either uses Y c as the predicted value without applying the discrete mapping, or uses the value most similar to Y c as the predicted value. The video decoding device uses an arbitrary value (e.g., a median value based on bit-depth) as the predicted value or as the input of the discrete mapping set. Alternatively, the video decoding device uses a luma sample of the nearest reconstructed luma samples that has a suitable value for mapping as the input of the discrete mapping set.

[0158] As another example, as shown in FIG. 17, when the input of the discrete mapping set does not have searchable values of the five samples {Y c , Y c+1 , Y c-1 , Y c+2 , Y c-2}, the video decoding device generates an average, a median, or a predefined weighted sum of the neighboring luma samples as the predicted value. Alternatively, the video decoding device can use the average, the median, or the predefined weighted sum of the neighboring luma samples as the input of the discrete mapping set. Yet alternatively, the video decoding device can generate a preset value as the predicted value. In the example of FIG. 17, c = 3 is suitable, i.e., Y c = Y3. The preset weighted sum represents a predefined weighted sum, where the number of samples subject to the weighted sum and the related weights are predetermined.

[0159] As yet another example, when the input of the discrete mapping set does not have a searchable Y c , the video decoding device generates the predicted value by inserting a preset number of luma samples neighboring Y c .

[0160] Implementation 1 relates to a method of establishing a discrete mapping model between a luma signal of a current block and a luma signal of a reference block during a template matching process. Another aspect can also apply Implementation 1 between a chroma signal of the current block and a chroma signal of the reference block.

[0161] Implementation 1 relates to a method of establishing a discrete mapping model during a template matching process between a luma signal of a current block and a luma signal of a reference block and between a chroma signal of the current block and a chroma signal of the reference block. When implementation 1 is applied to a single tree IBC mode, the reference block can be obtained based on an existing block vector, instead of being obtained during the template matching process. In addition, when implementation 1 is applied to a dual tree IBC mode, the chroma component reference block can be obtained based on a chroma block vector derived from a direct block vector (DBV) mode. Subsequently, the video decoding device uses the current template and the template of the reference block to derive a luma signal relationship, and a chroma signal relationship if applicable to the chroma component.

[0162] (Implementation 2) Template matching technique based on a nonlinear filter This implementation describes a template matching technique with a nonlinear filter. The video decoding device derives a nonlinear model between a template of a current block and a template of a reference block, and applies the derived model to the reference block to predict the current block. Here, existing nonlinear model derivation techniques such as GLM or CCCM will derive a mapping relationship between a luma signal and a co-located chroma signal as a nonlinear function. In contrast, the video decoding device according to the present invention derives a nonlinear model representing a relationship between luma signals of a template of a current block and a template of a reference block within a decoded region, and stores the derived nonlinear model.

[0163] In addition to the derivation method used in existing GLM, CCCM, etc., the video decoding device uses one of the following representation schemes, a part of one representation scheme, or a combination of representation schemes to implement a nonlinear filter.

[0164] To predict a luma sample, the nonlinear filter according to this embodiment can be implemented as shown in Equation 10.

[0165] [Equation 10] In Equation 10, Y c represents a luma sample of a current template or a current block, and Y r represents a luma sample of a reference block or a reference template. (x, y) denotes the position of the current sample, and (x c , y c ) denotes the position of the reference sample corresponding to (x, y). (x0, y0) represents a filter offset adjacent to (x c , y c ). S irepresents a target region for filtering a reference sample. The filter offset can be adjusted by using a block vector used in the co-located luma block or by using a DBV for the chroma block.

[0166] c i (x0, y0) represents a filter coefficient. The shape of the filter can vary with various dimensions. One example can use a filter of the same shape as the ALF filter shown in FIG. 18. Another example can use a filter of the shape shown in FIG. 8. Yet another example can use a nonlinear filter that includes a linear term as shown in Equation 7 in addition to the nonlinear term shown in Equation 10.

[0167] The nonlinear filter filters a luma sample within the target region using nonlinear modeling (NM) to generate a nonlinear modeled value. Subsequently, the nonlinear filter can generate a luma sample value based on a product of the nonlinear modeled value and a nonlinear filter coefficient.

[0168] At this time, one of the following schemes can be used for nonlinear modeling.

[0169] As one example, the nonlinear model can be implemented by using a polynomial function as shown in Equation 11.

[0170] [Equation 11] Another example can use a Michaelis-Menten hyperbolic model as shown in Equation 12.

[0171] [Equation 12] Yet another example can use a Fourier transform-based model as shown in Equation 13.

[0172] [Equation 13] Another example can use an exponential function model as shown in Equation 14.

[0173] [Equation 14] In Equations 11 to 14, Y r represents a luma sample of a reference block or a reference template. (x c , y c ) represents a position of a reference sample corresponding to a position (x, y) of a current sample. (x0, y0) represents a filter coefficient corresponding to (x c , yc ) neighboring filter offset. In addition, a j is a parameter of the nonlinear model and can be a preset value according to the nonlinear model. For example, if the nonlinear model is a polynomial function as in Equation 6, a2 is 1 and a0 is 512.

[0174] As an example, in the proposed nonlinear modeling, the video decoding apparatus sets Y c and Y r as the current template and the reference template, respectively. Then, filter coefficients are calculated in order to minimize the MSE between the predicted value according to Equation 10 and the reconstructed value of the current template. After deriving the nonlinear filter coefficients, the video decoding apparatus predicts the current block based on the derived nonlinear filter. By referring to the luma values in the reference block Y r and co-located with the current block Y c , the video decoding apparatus calculates the final predicted Y c based on the nonlinear filter.

[0175] Implementation 2 relates to a method of establishing a nonlinear relationship between a luma signal of a current block and a luma signal of a reference block during a template matching process. Another aspect can also apply Implementation 2 between a chroma signal of the current block and a chroma signal of the reference block.

[0176] Implementation 2 relates to a method of establishing a nonlinear relationship between a luma signal of a current block and a luma signal of a reference block and between a chroma signal of the current block and a chroma signal of the reference block, and it utilizes template matching in the process of obtaining the reference block. When applying Implementation 2 to the IBC mode of the single tree, the reference block can be obtained based on an existing block vector in addition to utilizing template matching. When applying Implementation 2 to the IBC mode of the dual tree, the chroma component reference block can be obtained based on a chroma block vector derived from the DBV mode. In addition, in Implementation 2, the reference signal can be obtained from a region that holds pre-decoded neighboring samples.

[0177] (Implementation 3) Cross-component discrete mapping model based on template matching technique As described above, the conventional CCDMM mode derives a discrete mapping model between luma and chroma in a reconstructed neighboring region where a linear model is applied, and predicts chroma samples from luma samples within the current block using the derived discrete mapping model.

[0178] FIG. 19 is a schematic diagram illustrating template matching prediction based on nonlinear modeling according to another embodiment of the present application.

[0179] This implementation describes a cross-component discrete mapping model based on template matching. As shown in FIG. 19, during the derivation of the discrete mapping model, the video decoding device according to the present application derives the mapping relationship between luma and chroma within a reference block based on template matching and stores the derived mapping relationship. In the example of FIG. 19, I L represents the luma signal, and I C represents the chroma signal. Omitting the calculation of the continuous function, the discrete mapping model directly stores the luma-chroma pairs from within the reference block obtained through template matching. The video decoding device derives the chroma samples by referring to the luma value at the down-sampled position of the current block based on the discrete mapping model. That is, the cross-component discrete mapping model derived from within the reference block is applied between the luma signal and the chroma signal of the current block. The luma value down-sampled from the luma-chroma pair is equal to the input, and the chroma value is equal to the output. In the following, the set containing the luma-chroma pairs is defined as the mapping set or the discrete mapping set of the discrete mapping model.

[0180] When the input of the mapping set of the discrete mapping model does not have a searchable Y c corresponding to the chroma sample of the current block in position, the video decoding device checks {Y c+1 , Y c-1 , Y c+2 , Y c-2} in a predetermined random order, as shown in FIG. 17. As mentioned above, in the example of FIG. 17, c = 3, i.e., Y c = Y3. In the example of FIG. 17, the sample represents the luma value at the down-sampled position of the current block. When the input of the discrete mapping set does not have a searchable value of the five values, the video decoding device sets the predicted value of the current chroma sample to a value calculated based on the neighboring reconstructed chroma samples, such as an average value, a median value, etc.

[0181] Implementation 3 is related to establishing a discrete mapping model between the luma signal and the chroma signal of the current block, which utilizes a reference block obtained via template matching as reference information to obtain information between the luma signal and the chroma signal. When implementation 3 is applied to the IBC mode of the single tree, the reference block can be obtained according to the existing block vector. In addition, when implementation 3 is applied to the IBC mode of the double tree, the chroma component reference block can be obtained based on the chroma block vector derived from the DBV mode. Furthermore, in implementation 3, the reference information can be obtained from the area where the pre-decoded neighboring samples are located.

[0182] (Implementation 4) Cross-component nonlinear model based on template matching technology In this implementation, a template matching based cross-component nonlinear model is derived. Instead of a template matching based cross-component discrete mapping model, the video decoding device derives a template matching based cross-component nonlinear model. At this time, the above-described schemes for computing a nonlinear model can be utilized. The video decoding device implements the cross-component nonlinear model by utilizing one of the following schemes.

[0183] For example, the video decoding device derives the nonlinear function by applying existing derivation techniques of the nonlinear model used in GLM, CCCM, etc. to the reference block. In other words, the video decoding device derives and stores the nonlinear function representing the relationship between the luma signal and the chroma signal of the reference block.

[0184] Alternatively, the video decoding device implements the nonlinear filter by utilizing one of, a part of, or a combination of the following representation schemes in addition to the existing derivation methods used in GLM, CCCM, etc.

[0185] To predict the chroma samples, the nonlinear filter according to this embodiment can be implemented as shown in Equation 15.

[0186] [Equation 15] In Equation 15, R Y represents the reconstructed luma sample, and R i (i ∈ {Cb, Cr}) represents the predicted chroma sample. (x, y) represents the position of the chroma sample, and (x c , y c ) represents the position of the luma sample corresponding to (x, y). (x0, y0) represents the filter offset around (x c , y c ), and S i represents the target region for filtering the reference samples. c i (x0, y0) represents the filter coefficient.

[0187] c i (x0, y0) represents the filter coefficient. The filter can be various shapes and sizes. For example, a filter of the same shape as the ALF filter shown in FIG. 18 can be used. In the example of FIG. 18, the C2 sample represents the luma sample co-located with the chroma sample to be filtered. As another example, a filter having the shape shown in FIG. 8 can be used. As yet another example, the nonlinear filter can include a linear term as shown in Equation 7 in addition to the nonlinear term shown in Equation 10.

[0188] The nonlinear filter applies nonlinear modeling to the luminance samples within the target region to generate nonlinear modeling values ​​for the luminance samples. Subsequently, the nonlinear filter can generate chromaticity sample values ​​based on the product of the nonlinear modeling values ​​and the nonlinear filter coefficients.

[0189] At this point, one of the following schemes can be used for nonlinear modeling.

[0190] As an example, nonlinear models can be implemented using polynomial functions, as shown in Equation 16.

[0191] [Equation 16] As another example, the Michaelis-Menten hyperbola model can be used as shown in Equation 17.

[0192] [Equation 17] As another example, a model based on the Fourier transform can be used, as shown in Equation 18.

[0193] [Equation 18] As another example, as shown in Equation 19, an exponential function model can be used.

[0194] [Equation 19] In equations 16 to 19, R Y This represents the reconstructed brightness sample. (x) c y c (x0, y0) represents the position of the luminance sample corresponding to the chrominance sample position (x, y). c y c The filter offset of α. Furthermore, α j These are the parameters of the nonlinear model, which can be preset values ​​based on the nonlinear model. For example, if the nonlinear model is a polynomial function as shown in Equation 6, then α2 is 1 and α0 is 512.

[0195] As an example, the video decoding device sets R in the nonlinear modeling of the presentation respectively. Y and R iluma samples and chroma prediction samples of the reference block based on template matching. Then, filter coefficients are calculated in order to minimize the MSE between the predicted values (e.g., according to Equation 15) and the chroma samples of the reference block. After deriving the non-linear filter coefficients, the video decoding device predicts the current chroma block based on the derived non-linear filter. By referring to the reconstructed luma sample values of the current block, the video decoding device calculates the final predicted R i .

[0196] Implementation 4 involves establishing a non-linear relationship between the luma signal and the chroma signal of the current block, and it utilizes a reference block obtained via template matching as reference information to obtain information between the luma signal and the chroma signal. When implementation 4 is applied to the IBC mode of the single tree, the reference block can be obtained based on the existing block vector. In addition, when implementation 4 is applied to the IBC mode of the dual tree, the chroma component reference block can be obtained based on the chroma block vector derived from the DBV mode. In addition, in implementation 4, the reference information can be obtained from the area where the pre-decoded neighboring samples are located.

[0197] On the other hand, when multiple types of filters are used in the template matching techniques according to implementations 1 to 4, the video decoding device can use one of the following methods.

[0198] For example, after applying a filter based on a linear model, a non-linear filter can be applied. For example, after applying a filter based on a linear model, the video decoding device calculates the current block by applying a non-linear model F(·), as shown in Equation 20.

[0199] [Equation 20] In Equation 20, L i curr represents the i-th luma sample of the current block, and L i ref represents the reference sample. a i represents the i-th coefficient of the linear model, and b represents the offset. N represents the number of luma samples included in the linear filtering.

[0200] As another example, as shown in Equation 21, after applying a model G(·) based on the gradient of the luma samples, a non-linear model F(·) can be applied.

[0201] [Equation 21] In Equation 21, a and b represent the parameters implementing the GLM. The gradient G can be calculated by utilizing one pattern of a preset number of Sobel-based gradient patterns.

[0202] In the following, a method of predicting a current block based on template matching is described by utilizing the illustrations of FIG. 20 and FIG. 21.

[0203] FIG. 20 is a flowchart of a method of encoding a current block by a video encoding device according to at least one embodiment of the present disclosure.

[0204] The video encoding device generates a first prediction block of the current block using a prediction mode that does not utilize template matching (S2000). For example, the video encoding device can use an intra prediction mode that does not use template matching, or can use an inter prediction mode.

[0205] The video encoding device obtains a size of the current block and determines whether the size satisfies a range condition for encoding according to template matching based prediction, such as a minimum block size condition and / or a maximum block size condition. If the range condition is satisfied, the video encoding device performs template matching based prediction.

[0206] The video encoding device generates a second prediction block of the current block based on template matching (S2002).

[0207] The video encoding device predicts the current block by utilizing template matching according to a pre-determined order, such as a z-scan order. The video encoding device searches for a reference block based on template matching between a template of a block within a search region and a template of the current block, and uses the searched reference block to determine the second prediction block of the current block.

[0208] The video encoding device determines a type of filter (S2004).

[0209] The filter type can be a linear model, a discrete mapping model, or a non-linear filter. Alternatively, the filter type can be a combination of a linear model or a non-linear filter. The video encoding device can determine the filter type from a rate-distortion optimization perspective. The video encoding device encodes information indicative of the filter type.

[0210] The video encoding device calculates filter coefficients (S2006).

[0211] For example, the video encoding device derives filter coefficients of a linear model by utilizing a regression model that minimizes the MSE between samples within a template of a reference block and samples within a template of the current block. In another example, the video encoding device calculates a discrete mapping model or a non-linear filter between a template of the reference block and a template of the current block. In yet another example, the video encoding device uses luma signals and chroma signals of the reference block to calculate a cross-component discrete mapping model or a cross-component non-linear filter.

[0212] The video coding device applies filtering to the second prediction block based on the filter to generate a filtered second prediction block (S2008).

[0213] For example, the video coding device generates the filtered second prediction block by applying a linear model to the reference block. Alternatively, the video coding device uses a discrete mapping model or a nonlinear filter between the templates to generate the filtered second prediction block from the reference block.

[0214] In another example, the video coding device uses a cross-component discrete mapping model or a cross-component nonlinear filter to predict chroma signals of the second prediction block from a luminance signal of the second prediction block (or the filtered second prediction block). Here, it is assumed that the luminance signal of the prediction block (or the filtered prediction block) is reconstructed.

[0215] The video coding device determines a template matching flag based on the first prediction block, the second prediction block, and the filtered second prediction block (S2010). Here, the template matching flag indicates whether template matching is used for the current block.

[0216] The video coding device determines the template matching flag from a rate-distortion optimization perspective. For example, if the first prediction block is the best, the template matching flag is determined to be false. In contrast, if the second prediction block or the filtered second prediction block is the best, the template matching flag is determined to be true.

[0217] The video coding device encodes the template matching flag (S2012).

[0218] The video coding device checks the template matching flag (S2014).

[0219] If the template matching flag is true (S2014 is Yes), the video coding device performs the following steps.

[0220] The video coding device determines a filter flag based on the second prediction block and the filtered second prediction block (S2016). Here, the filter flag indicates the applicability of filtering of the prediction block according to the template matching.

[0221] The video coding device determines the filter flag from a rate-distortion optimization perspective. For example, if the second prediction block is the best, the filter flag is determined to be false. In contrast, if the filtered second prediction block is the best, the filter flag is determined to be true.

[0222] The video coding device encodes the filter flag (S2018).

[0223] In addition, the video coding device encodes information indicating the type of the filter.

[0224] If the template matching flag is false (S2014 is No), the video encoding device encodes information related to the first prediction block (S2030).

[0225] Subsequently, the video encoding device subtracts the prediction block determined based on the template matching flag and the filter flag from the current block to generate a residual block. Then, the video encoding device applies a transform / quantization to the residual block to generate quantized transform coefficients, and encodes the quantized transform coefficients.

[0226] In the example of FIG. 20, the video encoding device utilizes template matching in obtaining the reference block. As another example, when the IBC mode is applied in a single tree, the video encoding device can obtain the reference block according to an existing block vector. When the IBC mode is applied in a dual tree, the video encoding device can obtain the chroma component reference block based on a chroma block vector derived from the DBV mode. At this time, the template matching flag can be replaced with a flag indicating the application of the IBC mode.

[0227] FIG. 21 is a flowchart of a method of reconstructing a current block by a video decoding device, according to at least one embodiment of the present disclosure.

[0228] The video decoding device obtains the size of the current block and checks whether the size of the current block satisfies a range condition for encoding according to template matching based prediction, such as a minimum block size condition and / or a maximum block size condition.

[0229] When the size satisfies the range condition, the video decoding device decodes a template matching flag from the bitstream (S2100). The template matching flag indicates whether template matching is to be used for the current block.

[0230] The video decoding device checks the template matching flag (S2102).

[0231] If the template matching flag is true (S2102 is Yes), the video decoding device performs the following steps.

[0232] The video decoding device generates a prediction block of the current block based on template matching (S2104).

[0233] The video decoding device predicts the current block by utilizing template matching according to a preset order (e.g., a z-scan order). The video decoding device searches for a reference block based on template matching between a template of a block within a search area and a template of the current block, and determines that the prediction block of the current block is the searched reference block.

[0234] The video decoding device decodes a filter flag from the bitstream (S2106). Here, the filter flag indicates the applicability of filtering of the prediction block based on template matching.

[0235] The video decoding device checks the filter flag (S2108).

[0236] If the filter flag is true (S2108 is Yes), the video decoding device performs the following steps.

[0237] The video decoding device obtains a type of filter for the current block (S2110).

[0238] The type of filter can be a linear model, a discrete mapping model, or a non-linear filter. Alternatively, the type of filter can be a combination of a linear model or a non-linear filter. For example, to obtain the type of filter, the video decoding device can decode information indicating the type of filter from the bitstream.

[0239] The video decoding device calculates filter coefficients (S2112).

[0240] As one example, the video decoding device derives filter coefficients of a linear model by utilizing a regression model that minimizes the MSE between samples within the template of the reference block and samples within the template of the current block. In another example, the video decoding device calculates a discrete mapping model or a non-linear filter between the template of the reference block and the template of the current block. In yet another example, the video decoding device uses the luma signal and the chroma signal of the reference block to calculate a cross-component discrete mapping model or a cross-component non-linear filter.

[0241] The video decoding device uses the filter to apply filtering to the prediction block (S2114).

[0242] For example, the video decoding device generates a filtered prediction block by applying a linear model to the reference block. In another example, the video decoding device uses a discrete mapping model or a non-linear filter between the templates to generate a filtered prediction block from the reference block.

[0243] In another example, the video decoding device uses a cross-component discrete mapping model or a cross-component non-linear filter to predict the chroma signal of the prediction block from the luma signal of the prediction block (or the filtered prediction block). Here, it is assumed that the luma signal of the prediction block (or the filtered prediction block) is reconstructed.

[0244] If the filter flag is false (S2108 is No), the video decoding device does not perform filtering of the prediction block based on template matching.

[0245] If the range condition is not satisfied or the template matching flag is false (S2102 is No), the video decoding device uses a prediction mode other than template matching to generate a prediction block for the current block (S2130). For example, the video decoding device can use an intra prediction mode that does not use template matching, or use an inter prediction mode.

[0246] Subsequently, the video decoding device decodes the quantized transform coefficients and applies inverse quantization / inverse transform to the quantized transform coefficients to generate a residual block. The video decoding device can sum the residual block and the prediction block to generate a reconstructed block of the current block.

[0247] In the example of FIG. 21, the video decoding device utilizes template matching in obtaining the reference block. As another example, when IBC mode is applied in single tree, the video decoding device can obtain the reference block according to the existing block vector. When IBC mode is applied in dual tree, the video decoding device can obtain the chroma component reference block based on the chroma block vector derived from DBV mode. At this time, the template matching flag can be replaced with a flag indicating the application of IBC mode.

[0248] Although the steps in the respective flowcharts are described as being executed in sequence, the steps merely exemplify the technical idea of some embodiments of the present application. Therefore, a person of ordinary skill in the art to which the present application pertains can execute the steps by changing the order described in the respective figures or by executing two or more steps in parallel. Therefore, the steps in the respective flowcharts are not limited to the illustrated time order.

[0249] It should be understood that the above description presents illustrative embodiments that can be implemented in various other ways. The functions described in some embodiments can be implemented by hardware, software, firmware, and / or a combination thereof. It should also be understood that the functional components described in the present application are marked as “... unit” to highlight the possibility of their independent implementation.

[0250] On the other hand, in some embodiments, various methods or functions described therein can be implemented as instructions stored in a non-volatile recording medium, which can be read and executed by one or more processors. The non-volatile recording medium can include various types of recording devices that store data in a form readable by a computer system, for example. For example, the non-volatile recording medium can include storage media such as erasable programmable read-only memory (EPROM), a flash drive, an optical disk drive, a magnetic hard disk drive, and a solid state drive (SSD), etc.

[0251] Although exemplary embodiments of the present application are described for illustrative purposes, those of ordinary skill in the art to which the present application pertains should understand that various modifications, additions, and substitutions can be made without departing from the spirit and scope of the present application. Therefore, embodiments of the present application are described for the sake of brevity and clarity. The scope of the technical idea of the embodiments of the present application is not limited by the examples. Accordingly, those of ordinary skill in the art to which the present application pertains should understand that the scope of the present application should not be limited by the explicitly described embodiments, but by the claims and their equivalents.

[0252] (Reference numeral explanation) 122: Intra predictor 155: Entropy encoder 510: Entropy decoder 542: Intra predictor

[0253] Cross Reference to Related Applications This application claims priority to and the benefit of Korean Patent Application No. 10-2023-0065621, filed May 22, 2023, and Korean Patent Application No. 10-2024-0045374, filed April 3, 2024, the entire contents of each of which are incorporated herein by reference.

Claims

1. A method of reconstructing a current block by a video decoding device, the method comprising: searching for a reference block based on a template match between a template of a block within a search region and a template of the current block or based on a block vector of the current block, and determining a prediction block of the current block by utilizing the searched reference block; obtaining a type of filter for the current block, the type of filter comprising a cross-component discrete mapping model or a cross-component nonlinear filter; calculating coefficients of the filter by utilizing the reference block; and applying filtering to the prediction block by utilizing the filter.

2. The method of claim 1, further comprising: decoding a template match flag from a bitstream, the template match flag indicating whether template matching is to be used for the current block; and checking the template match flag, wherein when the template match flag is true, the method proceeds to determine the prediction block of the current block.

3. The method of claim 1, further comprising: decoding a flag indicating whether an intra block copy (IBC) mode is to be applied; and checking the flag, wherein when the flag is true, the method proceeds to determine the prediction block of the current block. determining the prediction block of the current block comprises: when the IBC mode is applied in a single tree, determining luma samples and chroma samples of the reference block based on the block vector of the current block. determining the prediction block of the current block comprises:

4. The method of claim 3, wherein, when the IBC mode is applied in a dual tree, determining chroma samples of the reference block based on a chroma block vector derived from a direct block vector (DBV) mode.

6. The method of claim 1, further comprising:

5. The method of claim 3, wherein, decoding a filter flag from a bitstream, the filter flag indicating whether filtering is to be applied to the prediction block; and checking the filter flag, wherein when the flag is true, the method proceeds to obtain the type of filter. calculating the coefficients of the filter comprises: inputting luma samples of the reference block to a cross-component nonlinear filter; and calculating coefficients of the cross-component nonlinear filter so as to minimize a difference between an output of the cross-component nonlinear filter and corresponding chroma samples within the reference block. applying the filtering comprises:

7. The method of claim 1, wherein, generating filtered prediction samples of the chroma samples of the current block by applying the coefficients of the cross-component nonlinear filter to luma samples within a luma block of the current block around locations corresponding to the chroma samples. calculating the coefficients of the filter comprises: deriving luma-chroma pairs representing relationships between luma samples and chroma samples of the reference block; and 8. The method of claim 7, wherein, storing the luma-chroma pairs in a mapping set of the cross-component discrete mapping model. applying the filtering comprises:

9. The method of claim 1, wherein, generating filtered chroma prediction samples of the current block from luma samples obtained by downsampling the current block based on the luma-chroma pairs of the cross-component discrete mapping model. applying the filtering comprises: when an input of the mapping set has neither a searchable value of a luma sample corresponding in position to the chroma sample of the current block nor a searchable value of a neighboring sample of the luma sample, generating the filtered chroma prediction sample of the current block by a value calculated based on reconstructed chroma samples neighboring the chroma sample.

10. The method of claim 9, wherein, 12. A method of encoding a current block by a video encoding device, the method comprising: ​ 11. The method of claim 9, wherein, ​ ​ ​ generating a first prediction block of the current block by using a prediction mode other than template matching and intra block copy (IBC) mode; searching for a reference block based on template matching between a template of a block within a search area and a template of the current block or based on a block vector of the current block, and determining a second prediction block of the current block by using the searched reference block; obtaining a type of filter for the current block, the type of filter including a cross-component discrete mapping model or a cross-component non-linear filter; calculating coefficients of the filter by using the reference block; and generating a filtered second prediction block by applying filtering to the second prediction block using the filter.

13. The method of claim 12, further comprising: determining a template matching flag based on the first prediction block, the second prediction block, and the filtered second prediction block, the template matching flag indicating whether template matching is to be used for the current block; encoding the template matching flag; and checking the template matching flag.

14. The method of claim 13, further comprising, when the template matching flag is true: determining a filter flag based on the second prediction block and the filtered second prediction block, the filter flag indicating whether filtering is to be applied to the prediction block; encoding the filter flag; and encoding information indicating the type of filter.

15. The method of claim 12, further comprising: determining a flag indicating whether IBC mode is to be applied based on the first prediction block, the second prediction block, and the filtered second prediction block; and encoding the flag.

16. A method of providing video data to a video decoding device, the method comprising: encoding video data into a bitstream; and sending the bitstream to the video decoding device, wherein encoding the video data comprises: generating a first prediction block of the current block by using a prediction mode other than template matching and intra block copy (IBC) mode; searching for a reference block based on template matching between a template of a block within a search area and a template of the current block or based on a block vector of the current block, and determining a second prediction block of the current block by using the searched reference block; obtaining a type of filter for the current block, the type of filter including a cross-component discrete mapping model or a cross-component non-linear filter; calculating coefficients of the filter by using the reference block; and generating a filtered second prediction block by applying filtering to the second prediction block using the filter. ​ ​ ​ ​

Images