Improvements in intra template matching prediction mode for motion prediction

Abstract

The present application provides a method and system for implementing fusion of an intra TMP mode with other intra prediction modes utilizing neighboring samples to improve prediction accuracy. A VVC standard encoder and a VVC standard decoder may configure one or more processors of a computing system to fuse an intra TMP mode with different prediction modes, such as spatial geometry division mode (SGPM) and inter intra joint prediction (CIIP); refining the intra TMP prediction block based on neighboring samples, such as a method using position-dependent intra prediction combination (PDPC); applying local illumination compensation (LIC) to refine the intra-frame TMP prediction block; and / or sub-pixel positions, including a plurality of different sub-pixel positions, sub-pixel positions combined with a plurality of candidate intra-frame TMPs, or a flipping mode in the intra-frame TMP, are employed to further improve the prediction accuracy.

Description

Related applications

[0001] This application claims the benefit of U.S. patent application No. 63 / 436,826, filed on January 3, 2023, entitled “IMPROVEMENTS TO INTRA TEMPLATE MATCHING PREDICTION MODE FOR MOTION PREDICTION”; U.S. patent application No. 63 / 449,544, filed on March 2, 2023, entitled “IMPROVEMENTS TO INTRA TEMPLATE MATCHING PREDICTION MODE FOR MOTION PREDICTION”; and U.S. patent application No. 18 / 399,650, filed on December 28, 2023, entitled “IMPROVEMENTS TO INTRA TEMPLATE MATCHING PREDICTION MODE FOR MOTION PREDICTION”. The entire contents of the above applications are expressly incorporated into this application by reference. Technical Field

[0002] The present disclosure relates generally to motion prediction, and more particularly to methods and systems for implementing an intra template matching prediction mode. Background Art

[0003] In 2020, the Joint Video Experts Team (JVET) of the ITU-T Video Coding Experts Group (ITU-T VCEG) and the ISO / IEC Moving Picture Experts Group (ISO / IECMPEG) released the final draft of the next-generation video codec specification - Versatile Video Coding (VVC). This specification further improves video coding performance compared to previous standards such as H.264 / AVC (Advanced Video Coding) and H.265 / HEVC (High Efficiency Video Coding). The JVET went on to propose additional technologies beyond the scope of the VVC standard itself, collectively known as the Enhanced Compression Model (ECM).

[0004] According to the VVC standard, the encoder and decoder divide the image data into blocks and perform motion prediction on the luminance and chrominance components of the blocks by selecting one of various intra-frame prediction and inter-frame prediction modes. Among the intra-frame prediction modes provided by the VVC standard, intra template matching prediction (intra TMP) is an intra-frame prediction mode that copies the best prediction block from the reconstructed part of the current frame, and its L-shaped template is matched with the current template. Intra TMP considers the non-local spatial correlation of the current frame for prediction, but does not consider the local spatial correlation of the neighboring samples of the current block.

[0005] In addition, at the time of writing this article, the latest draft of the ECM (submitted at the 32nd JVET meeting in October 2023, entitled "Algorithmic Description of Enhanced Compression Model 11 (ECM 11)") includes proposals for further implementation of intra prediction modes, including angular intra prediction modes beyond those provided by the VVC standard. According to the ECM, intra TMP applies not only to screen content but also to camera-captured content. For camera-captured content, which has richer textures than screen content, intra TMP may not achieve the best results.

[0006] There is a need to further improve the functionality of intra-frame TMP beyond that provided by the VVC standard and ECM. Summary of the Invention

[0007] In a first aspect, an embodiment of the present invention provides a calculation method. The calculation method includes: matching a template of a current block with a template in a searched area; determining an integer-pixel position of a block vector of a matching block corresponding to the matching template; determining fractional-pixel positions around the integer-pixel position; and deriving reconstructed values of the matching block at the fractional-pixel positions around the determined integer-pixel position as predicted sample values of the current block.

[0008] In a second aspect, embodiments of the present disclosure provide a computing system. The computing system includes: a matching module configured to match a template of a current block with a template in a search area; a determination module configured to determine an integer-pixel position of a block vector of a matching block corresponding to the matching template; determine fractional-pixel positions surrounding the integer-pixel position; and a derivation module configured to derive reconstructed values of the matching block at fractional-pixel positions surrounding the determined integer-pixel position as predicted sample values of the current block.

[0009] In a third aspect, an embodiment of the present disclosure provides a computing system comprising: one or more processors, and a computer-readable storage medium communicatively coupled to the one or more processors, wherein the computer-readable storage medium stores computer-readable instructions that can be executed by the one or more processors, and when the computer-readable instructions are executed by the one or more processors, the computing method described in the first aspect is performed.

[0010] In a fourth aspect, an embodiment of the present disclosure provides a non-transitory computer-readable storage medium that stores a bit stream associated with one or more images, and when the bit stream is calculated by the processor, the processor executes the calculation method as described in the first aspect.

[0011] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, comprising: computer program instructions, wherein the computer program instructions enable a computer to execute the computing method as described in the first aspect.

[0012] In a sixth aspect, an embodiment of the present disclosure provides a computer program, which enables a computer to execute the computing method as described in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The detailed description will be set forth with reference to the accompanying drawings. In the drawings, the leftmost digit(s) of a reference number identifies the drawing in which the reference number first appears. The use of the same reference number in different drawings indicates similar or identical items or features.

[0014] Figure 1A and Figure 1B Example block diagrams of an encoding process and a decoding process according to an example embodiment of the present disclosure are respectively shown.

[0015] Figure 2 The reference area of the IBC mode is shown, which shows the available reference samples according to the CTU processing order, where each block represents a 64x64 luma sample unit.

[0016] Figure 3A and Figure 3B BV adjustments for horizontal flipping and vertical flipping are shown separately.

[0017] Figure 4A The search area of the intra-frame TMP is shown; Figure 4B The template used in the alternative proposed intra-frame TMP mode is shown.

[0018] Figure 5 It shows the 67 angular intra prediction modes provided by VVC.

[0019] Figure 6A and Figure 6BA division method of angular intra prediction modes is shown.

[0020] Figure 7A Neighboring samples used for refinement according to an example embodiment of the present disclosure are shown.

[0021] Figure 7B 、 Figure 7C and Figure 7D Interpolation of matching templates according to an example embodiment of the present disclosure is shown.

[0022] Figure 8 Shown are upper, lower, left, and right half-pixel positions around a whole-pixel position according to an example embodiment of the present disclosure.

[0023] Figure 9 Eight half-pixel positions surrounding a whole-pixel position are shown according to an example embodiment of the present disclosure.

[0024] Figure 10 Eight quarter-pixel positions and eight half-pixel positions surrounding a whole-pixel position are shown according to an example embodiment of the present disclosure.

[0025] Figure 11 Eight quarter-pixel positions, eight half-pixel positions, and eight three-quarter-pixel positions around a whole-pixel position are shown according to an example embodiment of the present disclosure.

[0026] Figure 12 Eight quarter-pixel positions, sixteen half-pixel positions, and twenty-four three-quarter-pixel positions around a whole-pixel position are shown according to an example embodiment of the present disclosure.

[0027] Figure 13 Four samples used in interpolation according to an example embodiment of the present disclosure are shown.

[0028] Figure 14 An example system for implementing the processes and methods described herein for implementing intra template matching prediction mode is shown. DETAILED DESCRIPTION

[0029] The systems and methods discussed herein are directed to implementing an intra template matching prediction mode for motion prediction, and more specifically, to integrating the intra TMP mode with other intra prediction modes that utilize neighboring samples to improve prediction accuracy.

[0030] According to the VVC video coding standard (i.e., the "VVC standard") and the motion prediction described therein, a computing system includes at least one or more processors and a computer-readable storage medium communicatively coupled to the one or more processors. The computer-readable storage medium is a non-transitory or non-transitory computer-readable storage medium, as subsequently described with reference to Figure 14As defined, it stores computer-readable instructions. At least some of the computer-readable instructions stored on the computer-readable storage medium are executable by one or more processors of the computing system to configure the one or more processors to perform the associated operations of the computer-readable instructions, including at least the operations of the encoder described by the VVC standard, and the operations of the decoder described by the VVC standard. The operations of these encoders and decoders according to the VVC standard are described in more detail later, however, these subsequent descriptions should not be understood as an exhaustive list of the encoder operations and decoder operations according to the VVC standard. Subsequently, "VVC standard encoder" and "VVC standard decoder" shall describe the corresponding computer-readable instructions stored on the computer-readable storage medium, which configure one or more processors to perform these respective operations (which, as an example, may be referred to as the "reference implementation" of the encoder or decoder).

[0031] In addition, according to an exemplary embodiment of the present disclosure, the VVC standard encoder and the VVC standard decoder further include computer-readable instructions stored on a computer-readable storage medium, which can be executed by one or more processors of a computing system to configure the one or more processors to perform operations not specified by the VVC standard. The VVC standard encoder should not be understood as being limited to the functionality of the reference implementation of the encoder, but rather further includes computer-readable instructions for configuring one or more processors of a computing system to perform further functions as described herein. The VVC standard decoder should not be understood as being limited to the functionality of the reference implementation of the decoder, but rather further includes computer-readable instructions for configuring one or more processors of a computing system to perform further functions as described herein.

[0032] Figure 1A and 1B Example block diagrams of an encoding process 100 and a decoding process 150 are respectively shown according to exemplary embodiments of the present disclosure.

[0033] In the encoding process 100, the VVC standard encoder configures one or more processors of a computing system to receive one or more input images as input from an image source 102. The input image includes a number of pixels sampled by an image acquisition device such as a photosensor array, and includes an uncompressed stream of multiple color channels (such as RGB color channels) storing color data at the image's native resolution, where each channel uses a certain number of bits to store color data for each pixel of the image. The VVC standard encoder configures one or more processors of the computing system to store the uncompressed color data in a compressed format, where the color data is stored at a lower resolution than the image's native resolution and is encoded as a luma ("Y") channel and two chroma ("U" and "V") channels having a lower resolution than the luma channel.

[0034] The VVC standard encoder encodes an image (the image being encoded is referred to as the current image, to be distinguished from any other image received from the image source 102) by configuring one or more processors of the computing system to divide the original image into units and sub-units according to a partitioning structure. The VVC standard encoder configures one or more processors of the computing system to subdivide the image into macroblocks (MBs) each having a dimension of 16x16 pixels, which can be further subdivided into partitions. The VVC standard encoder configures one or more processors of the computing system to subdivide the image into coding tree units (CTUs), whose luminance and chrominance components can be further subdivided into coding tree blocks (CTBs), which are further subdivided into coding units (CUs). Alternatively, the VVC standard encoder configures one or more processors of the computing system to subdivide the image into units of NxN pixels, which are then further subdivided into sub-units. For the purposes of this disclosure, each of these largest subdivision units of an image may generally be referred to as a "block."

[0035] A CU is encoded using one luma sample block and two corresponding chroma sample blocks, where the image is not monochrome, and is encoded using one coding tree.

[0036] The VVC standard encoder configures one or more processors of the computing system to subdivide the block into partitions having a plurality of dimensions of 4x4 pixels. For example, the partitions of the block can have dimensions of 8x4 pixels, 4x8 pixels, 8x8 pixels, 16x8 pixels, or 8x16 pixels.

[0037] The VVC standard encoder configures one or more processors of a computing system to encode the color information of an image at a lower resolution than the input image by encoding the color information of blocks of the image and subdivisions thereof, rather than encoding the color information of pixels of the full-resolution original image, thereby storing the color information in fewer bits than the input image.

[0038] In addition, the VVC standard encoder encodes an image by configuring one or more processors of the computing system to perform motion prediction on a block of the current image. Motion prediction encoding refers to storing image data of a block of the current image using motion information and prediction units (PUs) instead of pixel data based on intra-frame prediction 104 or inter-frame prediction 106 (wherein the block of the original image before encoding is referred to as an "input block").

[0039] Motion information refers to data that describes the motion of a block structure of an image or its units or subunits, such as motion vectors and references to blocks of the current image or a reference image. PUs may refer to a unit or subunits corresponding to one of multiple block structures of an image, such as an MB or CTU, where the blocks are partitioned based on the image data and encoded according to the VVC standard. The motion information corresponding to a PU may describe motion prediction encoded by a VVC standard encoder as described herein.

[0040] The VVC standard encoder configures one or more processors of a computing system to encode motion prediction information on each block of an image in an inter-block coding order, such as a raster scan order, where the first decoded block is the topmost and leftmost block of the image. The block being encoded is called the "current block" and is distinguished from any other block of the same image.

[0041] According to intra prediction 104, one or more processors of the computing system are configured to encode a block by referencing motion information and PUs of one or more other blocks of the same image. According to intra prediction coding, one or more processors of the computing system perform intra prediction 104 (also known as spatial prediction) calculations by encoding motion information of a current block based on spatially neighboring samples from spatially neighboring blocks of the current block.

[0042] According to inter-frame prediction 106, one or more processors of the computing system are configured to encode the block by referencing motion information and PUs of one or more other pictures. The one or more processors of the computing system are configured to store one or more previously encoded and decoded pictures in a reference picture buffer for the purpose of inter-frame prediction encoding; these stored pictures are referred to as reference pictures.

[0043] One or more processors are configured to perform inter-frame prediction 106 (also known as temporal prediction or motion-compensated prediction) calculations by encoding motion information for the current block based on samples from one or more reference images. Inter-frame predictions can also be calculated based on unidirectional prediction or bidirectional prediction: in unidirectional prediction, only one motion vector pointing to a single reference image is used to generate the prediction signal for the current block. In bidirectional prediction, two motion vectors are used to generate the prediction signal for the current block, each motion vector pointing to a corresponding reference image.

[0044] The VVC standard encoder configures one or more processors of the computing system to encode the CU so that it includes a reference index that identifies one or more prediction signals of the current block, thereby providing a reference for the VVC standard decoder. One or more processors of the computing system can encode the CU to include an inter-frame prediction indicator. The inter-frame prediction indicator indicates list 0 prediction with reference to a first reference image list called list 0, list 1 prediction with reference to a second reference image list called list 1, or bidirectional prediction with reference to two reference image lists called list 0 and list 1 respectively.

[0045] When the inter-frame prediction indicator indicates list 0 prediction or list 1 prediction, one or more processors of the computing system are configured to encode a coding unit (CU), the coding unit including a reference index, the reference index respectively referring to a reference picture in a reference picture buffer referenced by list 0 or list 1. When the inter-frame prediction indicator indicates bi-prediction (bi-prediction), the one or more processors of the computing system are configured to encode a coding unit (CU), the coding unit including a first reference index and a second reference index, the first reference index pointing to a first reference picture in a reference picture buffer referenced by list 0, and the second reference index pointing to a second reference picture of the reference pictures referenced by list 1.

[0046] The VVC standard encoder configures one or more processors of the computing system to encode each current block of the image separately, outputting a prediction block for each block. According to the VVC standard, the coding tree unit (CTU) can be a maximum of 128x128 luminance samples (plus corresponding chroma samples depending on the chroma format). The coding tree unit (CTU) can be further divided into coding units (CUs) according to a quadtree, binary tree or ternary tree. One or more processors of the computing system are configured to ultimately record a set of coding parameters at the syntax structure of the leaf node of the partition structure - such as the coding mode (intra mode or inter mode), motion information for inter-coded blocks (reference index, motion vector, etc.) and quantized residual coefficients.

[0047] After outputting the prediction block, the VVC standard encoder configures one or more processors of the computing system to send a set of encoding parameters—such as the encoding mode (e.g., intra or inter prediction), the intra prediction mode or the inter prediction mode, and the motion information—to the entropy encoder 124 (described subsequently).

[0048] The VVC standard specifies semantics for recording coding parameter sets for coding units (CUs). For example, for the above coding parameter sets, the pred_mode_flag of a coding unit (CU) is defined as 0 for inter-frame coded blocks and 1 for intra-frame coded blocks; the general_merge_flag of a coding unit (CU) is defined as indicating whether merge mode is used in inter-frame prediction of the coding unit (CU); the inter_affine_flag and CU_affine_type_flag of a coding unit (CU) are set to indicate whether affine motion compensation is used in inter-frame prediction of the coding unit (CU); the mvp_l0_flag and mvp_l1_flag are set to indicate whether the motion vector index is in list 0 or list 1, respectively; and, ref_idx_l0 and ref_idx_l1 are set to indicate whether the reference picture index is in list 0 or list 1, respectively. It should be understood that the VVC standard includes semantics for recording various other information, flags, and options, which are beyond the scope of this disclosure.

[0049] The VVC standard encoder further implements one or more mode decision and encoder control settings 108, including rate control settings. One or more processors of the computing system are configured to perform mode decision by selecting an optimized prediction mode for the current block based on a rate-distortion optimization method after intra or inter prediction.

[0050] The rate control setting configures one or more processors of the computing system to assign different quantization parameters (QPs) to different images. The magnitude of the quantization parameter (QP) determines the range of scales over which image information is quantized during encoding by the one or more processors (as described below), and thus determines the extent to which encoding process 100 discards image information from a sequence of macroblocks (MBs) during encoding (because the information falls between steps of the range of scales).

[0051] The VVC standard encoder also uses a subtractor 110. One or more processors of the computing system are configured to perform a subtraction operation by calculating the difference between the input block and the prediction block. Based on the optimized prediction mode, the prediction block is subtracted from the input block. The difference between the input block and the prediction block is called the prediction residual, or for brevity, the "residual."

[0052] Based on the prediction residual, the VVC standard encoder also uses transform 112. One or more processors of the computing system are configured to perform a transform operation on the residual through matrix arithmetic operations to calculate a coefficient array (which may be referred to as "residual coefficients", "transform coefficients", etc.) to encode the current block into a transform block (TB). The transform coefficients may represent one of several spatial transforms, such as diagonal flipping, vertical flipping, or rotation that may be applied to the sub-block.

[0053] It will be appreciated that coefficients may be stored as two components, absolute value and sign, as will be described in further detail subsequently.

[0054] The subblocks of coding units (CUs), such as prediction units (PUs) and transform blocks (TBs), can be arranged in any combination of subblock dimensions as described above. The VVC standard encoder configures one or more processors of the computing system to subdivide the coding unit (CU) into a residual quadtree (RQT), which is a hierarchical structure of transform blocks (TBs). The residual quadtree (RQT) provides an order for motion prediction and residual coding for each level of subblocks, and this order is recursively propagated down each level of the RQT.

[0055] The VVC standard encoder also uses quantization 114. One or more processors of the computing system are configured to perform a quantization operation on the residual coefficients using matrix arithmetic operations based on a quantization matrix and a QP as described above. Residual coefficients that fall within the interval are retained, and residual coefficients that fall outside the interval step size are discarded.

[0056] The VVC standard encoder also uses inverse quantization 116 and inverse transform 118. One or more processors of the computing system are configured to perform inverse quantization and inverse transform operations on the quantized residual coefficients using matrix arithmetic operations that are inverse to the quantization and transform operations described above. One or more processors in the computing system are configured to perform inverse quantization and inverse transform operations on the quantized residual coefficients using matrix arithmetic operations that are inverse to the quantization and transform operations described above. The inverse quantization and inverse transform operations produce a reconstructed residual.

[0057] The VVC standard encoder also uses an adder 120. One or more processors of the computing system are configured to perform an addition operation by adding the predicted block and the reconstructed residual, and output a reconstructed block.

[0058] The VVC standard encoder also uses a loop filter 122. One or more processors of the computing system are configured to apply a loop filter, such as a deblocking filter, a sample adaptive offset (SAO) filter, and an adaptive loop filter (ALF), to the reconstructed block and output a filtered reconstructed block.

[0059] The VVC standard encoder also configures one or more processors of the computing system to output the filtered reconstructed blocks to a decoded picture buffer (DPB) 200. The decoded picture buffer 200 (DPB 200) stores reconstructed pictures that the one or more processors of the computing system use as reference pictures when encoding pictures other than the current picture, as described above with respect to inter-frame prediction.

[0060] The VVC standard encoder also uses an entropy encoder 124, and one or more processors of the computing system are configured to perform entropy coding, wherein, according to a context-sensitive binary arithmetic codec (CABAC), the numerical symbols (symbols) constituting the quantized residual coefficients are encoded by mapping to binary strings (hereinafter referred to as "bins"), which can be sent in an output bitstream at a compressed bit rate. The numerical symbols of the encoded quantized residual coefficients include the absolute values of the residual coefficients (these absolute values are hereinafter referred to as "residual coefficient levels").

[0061] Therefore, the entropy encoder configures one or more processors of the computing system to encode the residual coefficient level of the block; bypass encode the residual coefficient symbols and record the residual coefficient symbols with the coding block; record the coding parameter set, such as the coding mode, intra-frame prediction mode or inter-frame prediction mode, and motion information encoded in the coding block syntax structure (such as the picture parameter set (PPS) set in the picture header and the sequence parameter set (SPS) found in a multi-image sequence); and output the coding block.

[0062] The VVC standard encoder configures one or more processors of the computing system to output a coded image composed of coded blocks from the entropy encoder 124. The coded image is output to a transmission buffer and ultimately packaged into a bitstream and output from the VVC standard encoder. The bitstream is written by one or more processors of the computing system to a non-transitory or non-volatile computer-readable storage medium of the computing system for transmission.

[0063] In the decoding process 150 , the VVC standard decoder configures one or more processors of a computing system to receive one or more encoded images from a bitstream as input.

[0064] The VVC standard decoder uses an entropy decoder 152. One or more processors of the computing system are configured to perform entropy decoding, wherein the binary symbols are decoded by reversing the mapping relationship between numerical symbols and binary symbols according to the context-sensitive binary arithmetic codec (CABAC), thereby restoring the entropy-coded quantized residual coefficients. The entropy decoder 152 outputs the quantized residual coefficients, outputs the bypass-coded residual coefficient symbols, and also outputs the syntax structures, such as PPS and SPS.

[0065] The VVC standard decoder also uses inverse quantization 154 and inverse transform 156. One or more processors of the computing system are configured to perform inverse quantization and inverse transform operations on the decoded quantized residual coefficients using inverse matrix arithmetic operations that are inverse operations of the quantization and transform operations described above. The inverse quantization and inverse transform operations produce a reconstructed residual.

[0066] In addition, based on the coding parameter set recorded by the entropy encoder 124 in syntax structures such as PPS and SPS (or, alternatively, received by out-of-band transmission or encoded into the decoder), and the coding mode included in the coding parameter set, the VVC standard decoder determines whether to apply intra-frame prediction 156 (i.e., spatial prediction) or motion compensated prediction 158 (i.e., temporal prediction) to the reconstructed residual.

[0067] In the case where the coding parameter set explicitly specifies intra prediction, the VVC standard decoder configures one or more processors of the computing system to use the prediction information specified in the coding parameter set to perform intra prediction 158. The intra prediction 158 thereby generates a prediction signal.

[0068] Where the coding parameter set specifies inter prediction, the VVC standard decoder configures one or more processors of the computing system to perform motion compensated prediction 160 using reference pictures from the DPB 200. The motion compensated prediction 160 thereby generates a prediction signal.

[0069] The VVC standard decoder further uses an adder 162. The adder 162 configures one or more processors of the computing system to perform an addition operation on the reconstructed residual and the prediction signal, thereby outputting a reconstructed block.

[0070] The VVC standard decoder also uses a loop filter 164. One or more processors of the computing system are configured to apply a loop filter, such as a deblocking filter, a SAO filter, and an ALF, to the reconstructed block, outputting a filtered reconstructed block.

[0071] The VVC standard decoder also configures one or more processors of the computing system to output filtered reconstructed blocks to the DPB 200. As described above, the DPB 200 stores reconstructed images, which the one or more processors of the computing system use as reference images for encoding images other than the current image, as described above with respect to motion compensated prediction.

[0072] The VVC standard decoder further configures one or more processors of the computing system to output the reconstructed image from the DPB to a user-viewable display of the computing system, such as a television display, a personal computing monitor, a smartphone display, or a flat-panel display.

[0073] Therefore, as illustrated by the encoding process 100 and the decoding process 150 described above, the VVC standard encoder and the VVC standard decoder each implement motion prediction encoding according to the VVC specification. The VVC standard encoder and the VVC standard decoder each configure one or more processors of a computing system to generate a reconstructed image based on a previously reconstructed image of the DPB according to motion compensated prediction as described in the VVC standard, wherein the previously reconstructed image is used as a reference image in the motion compensated prediction described herein.

[0074] According to the VVC standard, the coding tree is configured to provide separate block tree structures for the luma and chroma components of an image. A CTU may include three CTBs, which in turn include a luma CTB ("Y") and two chroma CTBs ("Cb" and "Cr").

[0075] For P slices and B slices, the luma and chroma CTBs of a CTU are configured to share the same common coding tree structure. However, for I slices, the luma and chroma CTBs can be configured to have separate block tree structures. When the coding tree is configured to use separate block trees, the luma CTB is divided into CUs using the first coding tree structure, and the chroma CTBs are divided into chroma CUs using the second coding tree structure.

[0076] In other words, while a CU of an I slice may include coded blocks for the luma component or coded blocks for two chroma components, a CU in a P or B slice includes coded blocks for all three color components (unless the video is monochrome).

[0077] According to the VVC standard, the luma component can be predicted using multiple intra prediction modes. These modes include planar intra prediction mode; DC intra prediction mode; angular intra prediction mode; multiple reference line (MRL) prediction mode; intra subpartitioning (ISP) mode; and matrix-based intra prediction (MIP) mode. These modes will be described in further detail below.

[0078] Angular intra prediction is a directional intra prediction method that extends previous implementations according to the HEVC standard. To capture arbitrary edge directions present in natural video, the VVC standard extends the number of angular intra prediction modes from 33 (as used in HEVC) to 65.

[0079] In addition, according to the VVC standard, the intra-block copy (IBC) mode is implemented as a block-level coding mode. Here, the VVC standard encoder configures one or more processors of the computing system to perform block matching (BM) to find the most ideal block vector (or motion vector) for each CU. The block vector indicates the displacement from the current block to the reference block that has been reconstructed within the current image. The luminance block vector of the IBC-encoded coding unit (CU) is integer precision. The chrominance block vector is also rounded to integer precision.

[0080] When combined with Adaptive Motion Vector Resolution (AMVR), IBC mode can switch between 1-pixel and 4-pixel motion vector precision. An IBC-encoded CU is treated as a third prediction mode in addition to intra or inter prediction modes. IBC mode is applicable to coding units (CUs) whose width and height are both less than or equal to 64 luma samples.

[0081] On the encoder side, hash-based motion estimation is performed for IBC. The encoder performs RD checks on blocks with a width or height of no more than 16 luma samples. For non-merge mode, a block vector search is first performed using a hash-based search. If the hash search does not return any valid candidates, a local search based on block matching is performed.

[0082] In a hash-based search, the hash key matching (32-bit CRC) between the current block and the reference blocks is extended to all allowed block sizes. The hash key calculation for each location in the current image is based on a 4x4 sub-block. For larger current block sizes, a hash key is determined to match the hash key of a reference block when all hash keys for all 4x4 sub-blocks match the hash keys in the corresponding reference positions. If multiple reference blocks are found whose hash keys match the hash key of the current block, the block vector cost is calculated for each matching reference block, and the matching reference block with the smallest block vector cost is selected.

[0083] In the block matching search, the search range is set to cover the previous and current CTUs.

[0084] The IBC mode is signaled in the bitstream with a CU-level flag, and can be identified as IBC adaptive motion vector prediction (AMVP) mode or IBC skip / merge mode, as described below.

[0085] IBC skip / merge mode: A merge candidate index is signaled to indicate which block vectors from a list of neighboring candidate IBC coded blocks are used to predict the current block. The merge list consists of spatial candidates, HMVP candidates, and pairwise candidates.

[0086] IBC AMVP mode: Block vector differences are encoded in the same manner as motion vector differences. The block vector prediction method uses two candidates as predictors: one from the left neighbor and one from the top neighbor (if IBC encoding). When no neighboring block is available, a default block vector is used as the predictor. A flag is signaled that specifies the block vector predictor index.

[0087] To reduce memory consumption and decoder complexity, IBC in VVC only allows the reconstruction of a predefined area including the area of the current CTU and some areas of the left CTU. Figure 2 The reference area of the IBC mode is shown, which shows the available reference samples according to the CTU processing order, where each block represents a 64x64 luma sample unit.

[0088] Depending on the position of the CU currently being encoded in the current CTU, the following methods are applicable.

[0089] If the current block falls within the upper left 64x64 block of the current CTU, in addition to the reconstructed samples in the current CTU, the CPR mode may be used to refer to the reference samples in the lower right 64x64 block of the left CTU. The current block may also use the CPR mode to refer to the reference samples in the lower left 64x64 block of the left CTU and the reference samples in the upper right 64x64 block of the left CTU.

[0090] If the current block falls into the upper right 64x64 block of the current CTU, in addition to the reconstructed samples in the current CTU, if the brightness position (0, 64) relative to the current CTU has not been reconstructed, the current block can also adopt the CPR mode and refer to the reference samples in the lower left 64x64 block and the lower right 64x64 block of the left CTU; otherwise, the current block can also refer to the reference samples in the lower right 64x64 block of the left CTU.

[0091] If the current block falls into the lower left 64x64 block of the current CTU, in addition to the reconstructed samples in the current CTU, if the brightness position (64, 0) relative to the current CTU has not been reconstructed, the current block can also use the CPR mode and refer to the reference samples in the upper right 64x64 block and the lower right 64x64 block of the left CTU; otherwise, the current block can also adopt the CPR mode and refer to the reference samples in the lower right 64x64 block of the left CTU.

[0092] If the current block falls into the lower right 64x64 block of the current CTU, the CPR mode can be used to refer only to the reconstructed samples in the current CTU.

[0093] The above-mentioned reference constraints enable the IBC mode to be implemented in hardware using local on-chip memory.

[0094] Furthermore, based on ECM, the reconstruction-reordered IBC (RR-IBC) mode can be applied to IBC-coded blocks. When RR-IBC is applied, samples in the reconstructed block are flipped according to the flip type of the current block. On the encoder side, the original block is flipped before motion search and residual calculation, while the predicted block is derived without flipping. On the decoder side, the reconstructed block is flipped back to restore the original block.

[0095] For RR-IBC coded blocks, two flipping methods are supported, horizontal flipping and vertical flipping. For blocks encoded using IBC AMVP, a syntax flag is first signaled to indicate whether the reconstructed data of the block has been flipped; if it has been flipped, another flag is further signaled to specify the specific flipping type. For IBC merging, the flipping type is inherited from the adjacent block and no syntax elements need to be signaled. Taking into account horizontal or vertical symmetry, the current block and the reference block are usually aligned horizontally or vertically. Therefore, when horizontal flipping is implemented, the vertical component of the BV (block vector) is not signaled and is inferred to be equal to 0. Similarly, when vertical flipping is used, the horizontal component of the BV is not signaled and is inferred to be equal to 0.

[0096] In order to better exploit the symmetry property, a flip-aware BV adjustment method is applied to refine the block vector candidates. Figure 3A and 3B As shown, (x n ,y n ) and (x c ,y c ) represent the coordinates of the center samples of the adjacent blocks and the current block, BV n and BV c Represents the block vector (BV) of the adjacent block and the current block respectively. When the adjacent block adopts horizontal flip coding, BV c The horizontal component of is not directly inherited from the adjacent block, but is obtained by adding BV n The horizontal component of ) is calculated by adding a motion offset, namely: Similarly, in the case where the adjacent blocks are coded with vertical flipping, by replacing BV n The vertical component (denoted as ) is added to a motion offset to calculate BV c The vertical component of

[0097] In addition, the VVC standard and ECM further provide an intra-frame template matching prediction (intra TMP) mode. Here, the VVC standard encoder configures one or more processors of the computing system to derive the best prediction block from the reconstructed portion of the current frame, whose L-shaped template matches the current template. For a predefined search range, the encoder searches for the template most similar to the current template in the reconstructed portion of the current frame, determines a matching block based on the matching template, and uses the matching block as the prediction block. The encoder then signals the use of this mode and performs the same prediction operation on the decoder side.

[0098] The prediction signal is generated by matching the template of the current block, i.e., the L-shaped causal neighbor of the current block, with another template in a predefined search area. In some disclosed embodiments, the predefined search area is such as Figure 4A Shown, including: R1: Current CTU R2: Upper left CTU R3: Upper CTU R4: Left CTU

[0099] The current block's template is matched against templates in the search area by comparing the cost function. The sum of absolute differences (SAD) is used as the cost function. Within each search area, the decoder searches for the template with the minimum SAD relative to the current block template and uses the block corresponding to the matched template (matching block) as the prediction block.

[0100] The sizes of all regions (SearchRange_w, SearchRange_h) are set to be proportional to the block size (i, h) to ensure that each pixel has a fixed number of comparisons of the sum of absolute differences (SAD). That is: SearchRange_w=a*w SearchRange_h=a*h where a is a constant that controls the gain / complexity tradeoff. In practice, a is equal to 5.

[0101] To speed up template matching, the search area (R1 to R4) is subsampled by a factor of 2. This reduces the template matching search to 1 / 4 of its original size. After finding the best match, a refinement process is performed, where another template matching search is performed with a narrowed search range around the best match. The refined search range is defined as min(w, h) / 2, where w and h are the current CU width and height.

[0102] In addition, a multi-candidate intra-frame TMP method for the VVC standard and ECM is proposed. There are usually several blocks similar to the current block with comparable corresponding template matching costs. Therefore, intra-frame template matching prediction (Intra TMP) does not need to select only a single matching block with the smallest sum of absolute differences (SAD), but can be implemented based on multiple prediction block candidates. A candidate list is constructed and the candidate matching blocks are sorted in ascending order of their template matching costs. Thereafter, an index is signaled in the bitstream to indicate which candidate prediction block is actually used.

[0103] Alternatively, according to the intra-frame TMP fusion method proposed for the VVC standard and ECM, N candidate matching blocks corresponding to N minimum template matching costs are fused to derive a prediction block for the current block.

[0104] Alternatively, a linear filter model is applied to intra TMP prediction based on the intra TMP filtering method proposed for the VVC standard and ECM. A 6-tap linear filter is composed of the five spatial luminance samples in the matching block and a bias term. For each block, the filter coefficients are derived using a regression analysis based on the minimum mean square error (MSE) of the samples between the matching template and the current template.

[0105] Instead of the intra-frame TMP mode based on the L-shaped template (left template and upper template), or in addition to the intra-frame TMP mode based on the L-shaped template (left template and upper template), an intra-frame TMP mode using only the left template and an intra-frame TMP mode using only the upper template are proposed, such as Figure 4B shown separately in .

[0106] However, due to the following limitations, the predicted sample values of intra-frame TMP coded blocks may be inaccurate.

[0107] Intra-frame TMP only considers the non-local spatial correlation of the current frame for prediction, but does not consider the local spatial correlation of neighboring samples of the current block.

[0108] Even within the same frame, there may be changes in lighting.

[0109] In ECM, intra-frame TMP can be used not only for screen content but also for camera-captured content. For camera-captured content, which has richer textures than screen content, the restriction of intra-frame TMP to whole-pixel positions may not achieve optimal results.

[0110] Therefore, exemplary embodiments of the present disclosure provide for the fusion of the intra TMP mode with other intra prediction modes that utilize neighboring samples to improve prediction accuracy. According to exemplary embodiments of the present disclosure, the intra TMP mode is fused with different prediction modes, such as spatial geometric partitioning mode (SGPM) and inter-intra joint prediction (CIIP). According to further exemplary embodiments of the present disclosure, the intra TMP prediction block is refined based on neighboring samples, such as using a method of position-dependent intra prediction combination (PDPC). According to further exemplary embodiments of the present disclosure, LIC is applied to refine the intra TMP prediction block. According to further exemplary embodiments of the present disclosure, sub-pixel position is implemented in the intra TMP to further improve prediction accuracy.

[0111] According to an exemplary embodiment of the present disclosure, the intra TMP mode is merged with an intra prediction mode. The intra prediction modes enabled for the luma component in VVC are planar mode, DC mode, angular intra prediction mode, multiple reference line (MRL) prediction mode, intra sub-partitioning (ISP) mode, and matrix-based intra prediction (MIP) mode.

[0112] Angular intra prediction is a directional intra prediction method supported in HEVC and is also part of VVC. In order to capture arbitrary edge directions present in natural videos, the number of angular intra prediction modes in VVC is extended from 33 used in HEVC to 65. Figure 5 New angular intra prediction modes that are not in HEVC are depicted as dashed arrows in FIG.

[0113] As in HEVC, VVC also supports two non-angular intra prediction modes, DC mode and planar mode. The DC intra prediction mode uses the average sample value of the reference samples of the block for prediction generation. VVC only uses reference samples along the longer sides of rectangular blocks to calculate the average value, while for square blocks, reference samples from the left and above are used. In planar mode, the predicted sample value is obtained by weighted average of the four reference sample values. Here, reference samples in the same row or column as the current sample and reference samples located at the lower left and upper right positions relative to the block are used.

[0114] In VVC, the intra prediction results for DC mode, planar mode, and several angular modes are further modified by a position-dependent intra prediction combination (PDPC) method. Without the need for signaling, PDPC is applied to the following intra modes: planar mode, DC mode, intra angle mode less than or equal to horizontal, and intra angle mode greater than or equal to vertical and less than or equal to index 80.

[0115] The prediction sample pred(x', y') is predicted using a linear combination of the intra prediction mode (DC, planar, angular) and the reference samples according to the following equation: pred(x′,y′)=Clip(0,(1< <BitDepth)-1,(wL×R -1，y′ +wT×R x′,-1 +(64-wL-wT)×pred(x′,y′)+32)>>6) where R x′,-1 , R -1，y′ Respectively represent the reference samples located at the upper and left boundaries of the current sample (x, y).

[0116] The PDPC process is identical for DC and planar modes. For angular modes, if the current angular mode is HOR_IDX or VER_IDX, the left or top reference samples, respectively, are not used. The PDPC weights and scaling factors depend on the prediction mode and block size. PDPC is applied to blocks with width and height both greater than or equal to 4.

[0117] For MRL mode, in addition to the directly adjacent neighboring sample rows, one of the two non-adjacent reference rows can be used as input for intra prediction in VVC.

[0118] ISP divides the luma intra prediction block into 2 or 4 sub-partitions vertically or horizontally according to the block size. For each sub-partition, prediction and transform coding operations are performed separately, but the intra prediction mode is shared among all sub-partitions.

[0119] In addition, for VVC, MIP is a new intra-frame prediction technique. To predict samples for a block of width W and height H, MIP takes as input a row of H reconstructed adjacent boundary samples to the left of the block and a row of W reconstructed adjacent boundary samples above the block. The generation of the prediction signal is based on three steps: downsampling of the reference samples, matrix-vector multiplication, and linear upsampling of the results.

[0120] ECM further proposes luma intra prediction modes: decoder-side intra mode derivation (DIMD) mode and template-based intra mode derivation (TIMD) mode. When DIMD is applied, two intra prediction modes are derived from the reconstructed neighboring samples in 65 angular modes, and these two predictions are combined with the planar mode prediction using weights derived from the gradient. When TIMD is applied, for each intra prediction mode in the list, the SATD between the predicted sample and the reconstructed sample of the template is calculated. The two intra prediction modes with the smallest SATD are first selected and fused with weights derived from the SATD.

[0121] According to an exemplary embodiment of the present disclosure, the VVC standard encoder and the VVC standard decoder configure one or more processors of the computing system to fuse the prediction of the intra TMP mode and the prediction of another intra prediction mode (which may be any intra prediction mode as described above) to generate a motion prediction, as shown below: pred(i, j) = w0·pred intraTMP (i, j)+w1·pred intra (i, j) where pred intraTMP (i, j) represents the predicted value of the current sample generated by the intra-frame TMP mode; pred intra (i, j) represents the predicted value of the current sample generated by another intra-frame prediction mode; pred(i, j) represents the final predicted value of the current sample generated by fusing the intra-frame TMP mode and the intra-frame prediction mode; (i, j) is the coordinate of the current sample in the current block: w0 and w1 are two weights, and the sum of the two weights should be equal to 1.

[0122] Used to generate pred intra The intra-frame prediction mode of (i, j) may be a predefined intra-frame prediction mode. For example, the intra-frame prediction mode is a planar mode. As another example, the intra-frame prediction mode is a DC mode. As another example, the intra-frame prediction mode is an angular mode derived by the DIMD method, i.e., derived from gradient information of adjacent samples. As another example, the intra-frame prediction mode is an intra-frame prediction mode derived by the TIMD method, i.e., derived from a template.

[0123] Used to generate pred intra The intra prediction mode of (i, j) can be identified by a syntax element signaled in the bitstream. For example, for a block indicated to be predicted by a fusion intra TMP mode and another intra prediction mode, an intra prediction mode list can be constructed, which contains several intra prediction modes in the plane mode, DC mode, and 65 angular modes. In the bitstream, an index is signaled to indicate which intra prediction mode is selected.

[0124] The two weights w0 and w1 can be two predefined values: for example, the value of w0 can be equal to 0.5 and the value of w1 can be equal to 0.5. The two weights w0 and w1 can be determined based on the intra prediction mode of the adjacent blocks. For example, when there are more blocks encoded in the intra TMP mode in the adjacent blocks, the value of w0 is larger: when there are fewer blocks encoded in the intra TMP mode in the adjacent blocks, the value of w1 is larger. The two weights w0 and w1 can also be identified by a syntax element signaled in the bitstream. For example, a list of two weights is constructed, and the index of the selected weight is signaled in the bitstream.

[0125] The use of the intra-frame TMP mode for a block to be merged with another intra-frame prediction mode can be indicated by a flag signaled in the bitstream. Specifically, for a block, when a flag is present in the bitstream to indicate that the intra-frame TMP mode is used for prediction, another flag is further signaled in the bitstream to indicate whether to merge with another intra-frame prediction mode. In some embodiments, there is no additional flag to indicate merging: that is, for a block, when a flag is present in the bitstream to indicate that the intra-frame TMP mode is used for prediction, it is always merged with another intra-frame prediction mode.

[0126] In some embodiments, the template matching process is also adjusted for blocks predicted by a fusion of an intra-frame TMP mode and another intra-frame prediction mode. In the original intra-frame TMP template matching, the SAD value between the reconstructed value of the template of the current block and the reconstructed value of the matching template is calculated to find a matching block. In contrast, according to an exemplary embodiment of the present disclosure, for blocks predicted by a fusion mode, the intra-frame prediction mode is used to predict the template of the current block through its neighboring samples, and the predicted sample value of the template will also affect the template matching process. As an example, when performing template matching, the reconstructed value of the matching template is fused with the predicted sample value of the template of the current block obtained using the intra-frame prediction mode, and then the SAD value between the fused value and the reconstructed value of the template of the current block is calculated for template matching. As another example, when performing template matching, the reconstructed value of the template of the current block is subtracted from the predicted sample value obtained using the intra-frame prediction mode, and then the SAD value between the adjusted value and the reconstructed value of the matching template is calculated for template matching.

[0127] According to an exemplary embodiment of the present disclosure, the VVC standard encoder and the VVC standard decoder configure one or more processors of the computing system to fuse the intra-frame TMP mode with the spatial geometric partitioning mode (SGPM). In VVC, the use of the SGPM mode for inter-frame prediction is supported. The geometric partitioning mode is a type of merge mode and is identified by a CU-level flag. Other merge modes include a normal merge mode, an MMVD mode, a CIIP mode, and a sub-block merge mode. In general, for each possible coding unit (CU) size, 64 partitions are supported by the geometric partitioning mode.

[0128] When using this mode, the CU is divided into two parts by a geometrically positioned straight line. The position of the dividing line is mathematically derived from the angle and offset parameters of the specific partition. Each part of the geometric partition in the CU is inter-predicted using its own motion: for each partition, only unidirectional prediction is allowed, that is, each part has one motion vector and one reference index. The unidirectional prediction motion constraints are applied to ensure that, as with conventional bidirectional prediction, only two motion compensated predictions are required for each CU. After predicting each part of the geometric partition, a hybrid process with adaptive weights is used to adjust the sample values along the edges of the geometric partition. This is the prediction signal for the entire CU, and the transformation and quantization process will be applied to the entire CU, the same as in other prediction modes.

[0129] The spatial GPM method is adopted in ECM, in which the geometric partitioning mode (GPM) is used in intra prediction. This new intra coding tool partitions the coding block into two parts and generates two corresponding intra prediction modes. To efficiently express the partitions and associated prediction information in the bitstream, the method uses a template-reordered candidate list, where each candidate in the list includes a combination of the partition mode and two intra prediction modes, and only the index of the candidate is signaled.

[0130] For each partition mode, the VVC standard encoder and the VVC standard decoder derive an IPM list for each part. The size of this IPM list is 3. The IPM list is derived as follows: the TIMD mode, the DIMD mode, the intra mode of the neighboring blocks. The possible combinations of one geometric partition mode and two intra prediction modes are arranged in ascending order based on their SAD between the prediction and reconstruction of the template. Depending on the partition mode, the weights in the template are 1 or 0. The template size is set equal to 1, i.e. the height of the upper template part is 1, the width of the left template part is 1, and the length of the combined candidate list is set equal to 16.

[0131] According to an exemplary embodiment of the present disclosure, the VVC standard encoder and the VVC standard decoder configure one or more processors of the computing system to include the intra TMP mode in the IPM list of the SGPM so that the intra TMP mode can be merged with another intra prediction mode in the geometrically divided blocks.

[0132] As an example, the intra TMP mode is always included in the IPM list of the SGPM. For example, the number of intra prediction modes in the IPM list is expanded from 3 to 4, the first 3 modes are constructed in the original way, and the fourth mode is set to the intra TMP mode.

[0133] As another example, when at least one neighboring block is encoded in the intra TMP mode, the intra TMP mode is included in the IPM list of the SGPM.

[0134] As another example, when at least one neighboring block is encoded in intra-frame TMP mode and the block vector of the neighboring block is to be used to predict the current block, the intra-frame TMP mode is included in the IPM list of the SGPM. Specifically, when constructing the IPM list in the SGPM for the current block, the neighboring blocks are traversed in a specific order, and the block vector indicating the position of the matching block of the first traversed intra-frame TMP-coded neighboring block is stored. Then, when predicting the current block in intra-frame TMP mode, the stored block vector is used instead of searching for another block vector for the current block through template matching.

[0135] According to an exemplary embodiment of the present disclosure, the VVC standard encoder and the VVC standard decoder configure one or more processors of the computing system to fuse the intra-frame TMP mode and the inter-frame intra-frame joint prediction mode (CIIP). According to VVC, when a CU is encoded in merge mode, if the CU contains at least 64 luma samples (i.e., the CU width multiplied by the CU height is equal to or greater than 64), and if the CU width and the CU height are both less than 128 luma samples, an additional flag is signaled to indicate whether the inter-frame intra-frame joint prediction (CIIP) mode is applied to the current coding unit (CU). As the name implies, the CIIP prediction combines an inter-frame prediction signal with an intra-frame prediction signal. The inter-frame prediction signal P_inter in the CIIP mode is derived using the same inter-frame prediction process applied to the conventional merge mode; and the derivation of the intra-frame prediction signal P_intra follows the conventional intra-frame prediction process with the planar mode. Then, the intra-frame and inter-frame prediction signals are combined using weighted averaging, where the weight values are calculated according to the coding modes of the upper and left neighboring blocks as follows:

[0136] If the upper neighbor is available and is intra-coded, set isIntraTop to 1, otherwise set isIntraTop to 0:

[0137] If the left neighbor is available and is intra-coded, set isIntraLeft to 1, otherwise set isIntraLeft to 0:

[0138] If (isIntraLeft + isIntraTop) is equal to 2, then set wt to 3:

[0139] Otherwise, if (isIntraLeft + isIntraTop) equals 1, set wt to 2:

[0140] Otherwise, set wt to 1

[0141] The CIIP forecast is calculated as follows: P CIIP =((4-wt)*P inter +wt*P intra +2)>>2

[0142] According to ECM, the TIMD derivation method is used to derive the intra prediction mode in CIIP. Specifically, the intra prediction mode with the smallest SATD value in the TIMD mode list is selected and mapped to one of the 67 conventional intra prediction modes.

[0143] In addition, if the derived intra prediction mode is an angular mode, it is also recommended to adjust the weights. For near-horizontal modes (2 <= angular mode index < 34), the current block is split vertically, such as Figure 6A As shown; for the near vertical mode (34 <= angle mode index <= 66), the current block is divided horizontally, as Figure 6B shown.

[0144] The (wIntra, wInter) of different sub-blocks are shown in Table 1 below Sub-block index (wIntra, wInter) 0 (6，2) 1 (5，3) 2 (3，5) 3 (2，6)

[0145] According to an exemplary embodiment of the present disclosure, a VVC standard encoder and a VVC standard decoder configure one or more processors of a computing system to include an intra-frame TMP mode in a CIIP mode. The CIIP prediction may combine an inter-frame prediction signal with an intra-frame prediction signal. For example, the CIIP prediction may further combine an intra-frame TMP prediction signal with another intra-frame prediction signal. As another example, the CIIP prediction may further combine an intra-frame TMP prediction signal with an inter-frame prediction signal.

[0146] In addition, according to an exemplary embodiment of the present disclosure, the VVC standard encoder and the VVC standard decoder configure one or more processors of the computing system to refine the predicted sample value of the intra-frame TMP mode by neighboring samples. For a block encoded by the intra-frame TMP mode, each sample in the block can be refined using its corresponding neighboring sample's reconstructed value and its corresponding left neighboring sample's reconstructed value, such as Figure 7A For example, the same method used for PDPC in planar mode can be used to refine the predicted sample values.

[0147] The use of refinement by neighboring samples for a coding block using intra TMP mode (intra TMP) can be indicated by a flag bit transmitted in the bitstream. Specifically, for a block, when a flag bit is present in the bitstream indicating that intra TMP mode is used for prediction, another flag bit is further signaled in the bitstream to indicate whether refinement by neighboring samples is to be performed. In some embodiments, there is no additional flag bit for indicating refinement: that is, for a block, when a flag bit is present in the bitstream indicating that intra TMP mode is used for prediction of the block, it is always refined by the neighboring samples.

[0148] In addition, local illumination compensation (LIC) is an inter-frame prediction technique that models the local illumination variation between the current block and its predicted block as a function of the illumination variation between the current block template and the reference block template. The parameters of the function can be represented by a scaling factor α and an offset β, which compensate for the illumination variation according to the linear equation α*p[x]+β, where p[x] is the reference sample pointed to by the motion vector (MV) at position x on the reference image. When surround motion compensation is used, the motion vector (MV) should be removed to introduce surround offset. Since α and β can be derived based on the current block template and the reference block template, they do not require signaling overhead. Unlike the AMVP mode, the LIC flag (LICflag) is set to indicate the activation of LIC.

[0149] Local illumination compensation for unidirectionally predicted inter CUs includes the following modifications.

[0150] Neighboring samples within a frame can be used in LIC parameter derivation;

[0151] Disable LIC for blocks with less than 32 luma samples;

[0152] For non-subblock mode and affine mode, LIC parameter derivation is performed based on the template block samples corresponding to the current CU instead of the local template block samples corresponding to the first upper left 16x16 unit; and

[0153] The samples of the reference block template are generated by using motion compensation (MC) based on the block motion vector (MV) that is not rounded to integer-pel positions.

[0154] According to an exemplary embodiment of the present disclosure, the VVC standard encoder and the VVC standard decoder configure one or more processors of the computing system to perform LIC on the predicted sample values of the intra-frame TMP mode coded block. Specifically, when the intra-frame TMP mode predicted block finds a matching block through template matching, first, a linear model is constructed between the matched template and the template of the current block, and then the linear model is applied to the reconstructed samples in the matching block to generate the predicted sample values of the current block, rather than directly copying from the matching block.

[0155] The activation of LIC for blocks coded in intra-frame TMP mode can be indicated by a flag signaled in the bitstream. Specifically, for a block, when a flag indicating the use of intra-frame TMP mode for prediction is present in the bitstream, another flag is further signaled in the bitstream to indicate whether LIC is to be performed. In some embodiments, no additional flag indicates the execution of LIC: that is, for a block, when a flag indicating the use of intra-frame TMP mode for prediction is present in the bitstream, LIC is always performed. In some embodiments, an implicit method is used to determine whether LIC is to be performed for blocks coded in intra-frame TMP mode. This determination can be made using information from the template. For example, when the mean of the current block template differs significantly from the mean of the matching block template, LIC is performed; otherwise, LIC is not performed.

[0156] In some embodiments, for blocks predicted using intra-frame TMP mode, the template matching process is modified when performing LIC. First, a linear model is constructed between the template of the current block and the matching template. Then, the reconstructed values of the matching template are adjusted using this linear model. Finally, the SAD value between the modified values and the reconstructed values of the template of the current block is calculated for use in template matching.

[0157] According to an example embodiment of the present disclosure, the VVC standard encoder and the VVC standard decoder configure one or more processors of the computing system to implement sub-pixel positions around integer pixel positions in the intra-frame TMP, or in other words, fractional-pixel positions around integer pixel positions. For example, half-pixel positions (i.e., 1 / 2 pixel positions) around integer pixel positions can be implemented in the intra-frame TMP, but are not limited thereto. As another example, quarter-pixel positions (i.e., 1 / 4 pixel positions) around integer pixel positions can be implemented in the intra-frame TMP, but are not limited thereto. As yet another example, finer-grained sub-pixel positions around integer pixel positions can be implemented in the intra-frame TMP.

[0158] Currently, intra-frame TMP only supports integer pixel precision: through template matching, the position of the block vector indicating the matching block position is determined with integer pixel precision. Then, the reconstructed value of the integer pixel position indicated by the block vector is directly copied to the corresponding position of the current block as the predicted sample value of the current block.

[0159] According to the present disclosure, it should be understood that the sub-pixel positions around the integer pixel positions supported by the intra-frame TMP mode can be expressed as coordinates along the horizontal and vertical axes in the (x, y) format (i.e., without reference to the integer pixel position), but can also be expressed as two decomposed components: precision and direction. The sub-pixel precision should be understood as the amount by which the sub-pixel position deviates from the integer pixel position, where the offset can be further decomposed into a horizontal precision offset component and a vertical precision offset component, which are then expressed in the (o1, o2) format. The horizontal precision offset component describes the horizontal offset of the sub-pixel position from the best integer pixel position. The vertical precision offset component describes the vertical offset of the sub-pixel position from the best integer pixel position. Both the horizontal precision offset and the vertical precision offset can be expressed as absolute sub-pixel values (no negative values) or as sub-pixel values along a positive-negative axis centered on the best integer pixel position.

[0160] The direction of the fractional pixel positions around the integer pixel position should be understood as one of any number of different directions offset from the integer pixel position, which may include eight cardinal directions, and which may further include additional intermediate directions between the cardinal directions, as described below.

[0161] According to one embodiment, the VVC standard encoder and the VVC standard decoder configure one or more processors of the computing system to determine the best integer pixel position of the matching block based on intra-frame TMP template matching, and further configure the one or more processors to then compare the integer pixel position with the half-pixel positions surrounding the integer pixel position based on the template matching to determine whether to refer to the fractional pixel positions surrounding the integer pixel position for the intra-frame TMP.

[0162] According to another embodiment, the VVC standard encoder and the VVC standard decoder configure one or more processors of the computing system to determine the best integer pixel position of the matching block based on intra TMP template matching, and further configure the one or more processors to subsequently decide whether to perform intra TMP with reference to fractional pixel positions surrounding the integer pixel based on syntax elements signaled in the bitstream.

[0163] The VVC standard encoder and the VVC standard decoder configure the template matching process of the intra-frame TMP in a basically similar manner. However, the VVC standard encoder also configures the signaling of syntax elements, and the VVC standard decoder is further configured by these syntax elements. The VVC standard encoder also configures one or more processors of the computing system to predict the current block by the integer pixel position and a group of pixel positions around the integer pixel position after the template matching of the intra-frame TMP determines the matching block at the best integer pixel position. For example, template matching is performed at the four half-pixel positions of the top, bottom, left and right around the integer pixel position, and the best prediction is determined. Then, a flag bit is signaled in the bitstream to identify whether the integer pixel position or the sub-pixel position is used: when the flag bit indicates that the sub-pixel position is used, another syntax element is further signaled to identify which of a group of possible sub-pixel positions around the integer pixel position is selected.

[0164] When using integer pixel positions for prediction, the reconstructed value of the matching block is copied to the corresponding position of the current block as the predicted sample value; when using fractional pixel positions around the integer pixel position for prediction, the matching block at the integer pixel position is offset according to the fractional pixel position, and then the matching blocks at the fractional pixel position around the integer pixel position are interpolated to obtain the predicted sample value of the current block.

[0165] The VVC standard decoder further configures one or more processors of the computing system to determine, for an intra-frame TMP mode coding block, whether to refer to an integer pixel position or a sub-pixel position by a flag signaled in the bitstream, and to use a syntax element signaled in the bitstream to determine which sub-pixel position around the integer pixel position to use.

[0166] According to another embodiment, the VVC standard encoder and the VVC standard decoder configure one or more processors of the computing system to determine the most ideal integer pixel position of the matching block by template matching, and further configure the one or more processors to decide whether to perform intra-frame TMP with reference to sub-pixel positions around the integer pixel position based on a flag signaled in the bitstream and based on the template.

[0167] The VVC standard encoder also configures one or more processors of the computing system to predict the current block by the integer pixel position and a group of pixel positions around the integer pixel position after the template matching of the intra-frame TMP determines the matching block at the best integer pixel position. For example, template matching is performed at each of the four half-pixel positions of the top, bottom, left, and right around the integer pixel position, and then the best prediction is determined. Next, a flag indicating whether the integer pixel position or the sub-pixel position is used will be signaled in the bitstream.

[0168] The VVC standard decoder also configures one or more processors of the computing system to use intra-frame TMP mode coding blocks to determine whether to refer to integer pixel positions or fractional pixel positions by using a flag signaled in the bitstream. In the same way, the coordinates of the half-pixel position are determined by having templates at both the encoder and the decoder.

[0169] According to some embodiments, a high-level syntax structure flag is signaled to indicate whether the intra-frame TMP supports split-pixel positioning: for example, as described above, this flag can be signaled in the SPS syntax structure in the multi-image sequence. Based on this high-level flag, split-pixel positioning may only be enabled in certain circumstances. For example, for camera-captured content, setting this flag to true configures the intra-frame TMP to support split-pixel positioning for camera-captured content; while for screen content, setting this flag to false configures the intra-frame TMP to not support split-pixel positioning for screen content.

[0170] After the template matching of the intra-frame TMP determines the most ideal matching block at the integer pixel position, the fractional pixel positions surrounding the integer pixel position are referenced (regardless of how the one or more processors determine that they will be referenced), and the one or more processors continue to perform template matching at a group of pixel positions surrounding the integer pixel position. For example, template matching is performed at four half-pixel positions: top, bottom, left, and right around the integer pixel position. Figure 8 An upper half-pixel position, a lower half-pixel position, a left half-pixel position, and a right half-pixel position around the integer pixel position are shown.

[0171] When template matching is performed on sub-pixel positions around an integer-pixel position, for each of a group of pixel positions around the integer-pixel position, the matching template is interpolated using an interpolation filter, and then the SAD between the interpolated matching template and each of a group of sub-pixel positions around the integer-pixel position is calculated.

[0172] Interpolating a matching template should be understood as, for each sample of the matching template, inputting the sample and some adjacent samples into an interpolation filter to output the corresponding interpolated sample. In other words, all samples of the matching template and some adjacent samples are input into the interpolation filter to output the interpolated sample of the entire matching template.

[0173] The present disclosure does not specifically limit the type and number of taps of the interpolation filter used for matching template interpolation.

[0174] In any direction (up, down, left, and right) of the matching template, adjacent samples from rows 0 to 2 can be input into the interpolation filter. In addition, the adjacent samples can be offset in a certain offset direction from the integer pixel position to the fractional pixel position.

[0175] For example, Figure 7B The matching template can be interpolated in the following way: for the sub-pixel position that is offset to the left of the integer pixel position, and using a 4-tap interpolation filter, Figure 7B The samples a to n shown in are input to the interpolation filter, where, to output each individual sample, the left-shifted adjacent samples of the individual sample are input to the interpolation filter. In other words, to output sample c', the left-shifted samples a, b, c, and d are input to the interpolation filter; to output sample d', the left-shifted samples b, c, d, and e are input to the interpolation filter; to output sample m', the left-shifted samples k, l, m, and n are input to the interpolation filter; and so on.

[0176] As another example, Figure 7C The matching template can be interpolated as follows: for the sub-pixel position offset to the right of the integer pixel position, and using a 4-tap interpolation filter, such as Figure 7C The interpolation filter is input to samples a through n, where to output each individual sample, the right-shifted adjacent samples of the individual sample are input to the interpolation filter. In other words, to output sample b', the interpolation filter is input to the right-shifted samples a, b, c, and d; to output sample c', the interpolation filter is input to the right-shifted samples b, c, d, and e; to output sample l', the interpolation filter is input to the right-shifted samples k, l, m, and n; and so on.

[0177] As another example, Figure 7D The matching template can be interpolated as follows: for the upper left corner of the integer pixel position, the sub-pixel position is offset and a 4-tap interpolation filter is used, such as Figure 7D The samples a to ai shown in the figure are first input to the horizontal interpolation filter, wherein, in order to output each individual sample, the left-offset adjacent sample of the individual sample is input to the interpolation filter. The output samples c' to ah' are sequentially input to the vertical interpolation filter, wherein, in order to output each individual sample, the upper-offset adjacent sample of the individual sample is input to the interpolation filter.

[0178] In other words, to output sample c', left-biased samples a, b, c, and d are input to the interpolation filter; to output sample j', left-biased samples h, i, j, k are input to the interpolation filter; to output sample q', left-biased samples o, p, q, r are input to the interpolation filter; to output sample x', left-biased samples v, w, x, y are input to the interpolation filter, etc. Then, to output sample q', up-biased samples c', j', q', and x' are input to the interpolation filter, and so on.

[0179] If the SAD value of at least one of the sub-pixel positions surrounding the integer-pixel position is less than the SAD value of the optimal integer-pixel position, the sub-pixel position is used. The sub-pixel position with the smallest SAD value in the sub-pixel position surrounding the integer-pixel position is determined for motion prediction. During prediction, the matching block at the integer-pixel position is offset according to the determined sub-pixel position, and the predicted sample value of the current block is obtained by interpolating the matching blocks at the sub-pixel positions surrounding the determined integer-pixel position.

[0180] According to some embodiments, the position of the sub-pixel may be eight positions relative to the best integer pixel position, such as Figure 9 As shown in FIG. 1 , relative to the optimal integer pixel position, the upper half-pixel position, the lower half-pixel position, the left half-pixel position, the right half-pixel position, the upper left half-pixel position, the upper right half-pixel position, the lower left half-pixel position, and the lower right half-pixel position. For the four half-pixel positions in the diagonal direction, horizontal and vertical interpolation can be performed separately during interpolation. For example, horizontal interpolation is performed first, followed by vertical interpolation. As another example, vertical interpolation is performed first, followed by horizontal interpolation.

[0181] Each of the eight positions is offset in precision and direction from the optimal whole-pixel position by a half-pixel position, and each is offset in a different one of the eight directions relative to the optimal whole-pixel position.

[0182] According to some embodiments, not only half-pixel positions may be considered in the intra TMP, but also other fractional pixel positions, such as 1 / 4 pixel positions, 1 / 8 pixel positions, and 1 / 16 pixel positions. For different levels of precision, the interpolation filter coefficients may be different.

[0183] According to a further embodiment, multiple sub-pixel positions of different precisions can be implemented in a combination in the intra-frame TMP. That is, different combinations of sub-pixel precisions and directions can be supported in the intra-frame TMP.

[0184] A combination of two sub-pixel precisions and eight directions can be supported in intra-frame TMP. For example, 1 / 2 pixel precision and 1 / 4 pixel precision are supported, and eight directions are supported for each precision, such as Figure 10 As shown in FIG, which illustrates a total of sixteen sub-pixel positions in eight directions, wherein blue circles represent integer pixel positions, gray circles represent adjacent integer pixel positions, red circles represent 1 / 2 pixel precision positions in eight directions, and yellow circles represent 1 / 4 pixel precision positions in eight directions.

[0185] Alternatively, a combination selected from three sub-pixel precisions and eight directions can be supported in intra-frame TMP. For example, 1 / 2 pixel precision, 1 / 4 pixel precision, and 3 / 4 pixel precision are supported, and eight directions are supported for each precision, such as Figure 11 As shown, Figure 11 A total of 24 sub-pixel positions in eight directions are shown. Among them, blue circles represent integer pixel positions, gray circles represent adjacent integer pixel positions, red circles represent 1 / 2 pixel precision positions in eight directions, yellow circles represent 1 / 4 pixel precision positions in eight directions, and green circles represent 3 / 4 pixel precision positions in eight directions.

[0186] Alternatively, three sub-pixel precisions and up to twenty-four directions can be supported in an intra-TMP, where additional directions are supported in addition to the eight basic directions due to the presence of sub-pixel granularity. For example, 1 / 2 pixel precision, 1 / 4 pixel precision, and 3 / 4 pixel precision are supported; for 1 / 4 pixel precision, eight directions are supported, for 1 / 2 pixel precision, 16 directions of greater granularity are supported, and for 3 / 4 pixel precision, 24 directions of greater granularity are supported, such as Figure 12 As shown in FIG, 48 fractional pixel positions are shown. Blue circles represent integer pixel positions, gray circles represent adjacent integer pixel positions, red circles represent 1 / 2 pixel precision positions in 16 directions, yellow circles represent 1 / 4 pixel precision positions in eight directions, and green circles represent 3 / 4 pixel precision positions in twenty-four directions.

[0187] Figure 11 and 12 Each sub-pixel position is further shown having a coordinate position decomposed into a horizontal offset component and a vertical offset component (o1, o2) (using absolute values) as described above; the direction component as described above is not shown. Figure 11 As shown, those sub-pixel positions in cardinal directions from the most ideal integer pixel position have integer pixel precision along one of the two offset directions, while those sub-pixel positions in ordinal directions from the most ideal integer pixel position have fractional pixel precision along both offsets, where the two fractional pixel precisions have the same magnitude. Figure 12 As shown, these additional sub-pixel positions are located between the cardinal direction and the oblique ordinal direction; these intermediate sub-pixel positions have different sub-pixel precisions in the two offset directions.

[0188] The multiple sub-pixel precisions and directional decompositions provided above can be combined with the aforementioned embodiments.

[0189] In some embodiments, the sub-pixel precision and direction are derived by template matching, and the template matching is configured in a substantially similar manner by the VVC standard encoder and the VVC standard decoder. As an example, 1 / 2 pixel precision, 1 / 4 pixel precision, and 3 / 4 pixel precision are supported, as well as eight directions (such as Figure 11 (as shown in FIG. 1 ) implements intra-frame TMP, and after determining the most ideal integer-pixel position of the matching block through template matching based on the intra-frame TMP, it further determines whether to refer to sub-pixel positions around the integer-pixel position and which sub-pixel position to use (with which precision and in which direction) based on the template. That is, the position with the smallest template SAD value is used.

[0190] In some embodiments, both the sub-pixel precision and direction are identified by syntax elements signaled in the bitstream. A flag is first signaled to indicate whether to reference the best integer-pixel position derived from template matching or the sub-pixel positions surrounding the integer-pixel position. If the flag indicates the use of one of the sub-pixel positions, one or more syntax elements are further signaled to indicate which of the surrounding sub-pixel positions is used. As one example, the sub-pixel position is signaled by one syntax element. As another example, the selected precision is first signaled by a syntax element, followed by the selected direction by another syntax element. As yet another example, the selected direction is first signaled by a syntax element, followed by the selected precision by another syntax element. Each of the aforementioned syntax elements can be signaled using fixed-length coding, truncated unary coding, truncated binary coding, Exponential-Golomb coding, or any other binary coding method.

[0191] In some embodiments, the sub-pixel precision is indicated by a syntax element signaled in the bitstream; the direction is not signaled in the bitstream; and the encoder and decoder are each configured to derive the direction by template matching. That is, if a flag indicates that one of the sub-pixel positions is to be used, a syntax element is further signaled to indicate which sub-pixel precision is to be selected. Template matching is then performed in the supported directions with the selected precision, and the direction with the smallest SAD value is selected.

[0192] In some embodiments, the direction is indicated by a syntax element signaled in the bitstream; the sub-pixel precision is not signaled in the bitstream; and the encoder and decoder are each configured to derive the sub-pixel precision by template matching. That is, if a flag indicates that a particular sub-pixel position is to be used, a syntax element is further signaled to indicate which direction is to be selected. Template matching is then performed in the directions with the supported precisions, and the precision with the smallest SAD value is selected.

[0193] In some embodiments, the sub-pixel precision is indicated by a syntax element signaled in the bitstream; the direction is not signaled in the bitstream; and the direction is derived from the syntax element signaled in the bitstream and the template matching method. That is, if a flag indicates the use of one of the sub-pixel positions, a syntax element is further signaled to indicate which sub-pixel precision is selected. The supported directions can then be divided into C categories, and another syntax element is further signaled to indicate which category of directions is selected. The value of C should be less than the number of supported directions, and each category should have at least one direction. For the selected direction category, if there is more than one direction, template matching is performed on these directions with the selected precision, and the direction with the smallest SAD value is selected.

[0194] As an example, in an implementation supporting 1 / 2 pixel precision, 1 / 4 pixel precision and 3 / 4 pixel precision and eight directions (such as Figure 11 In the intra TMP (as shown in FIG. 1 ), a syntax element is signaled to indicate which of the three sub-pixel precisions is selected. Furthermore, the eight directions can be divided into four categories: {top, bottom}, {left, right}, {upper left, lower right}, and {upper right, lower left} (i.e., every two opposite directions are classified into the same category). Furthermore, another syntax element is signaled to indicate which category of direction is selected. For the two opposite directions in the category, template matching is performed at the selected precision to select a direction.

[0195] In some embodiments, when selecting a direction within a category by template matching, template matching can be performed with a fixed precision for the direction within that category, rather than using the selected precision. For example, regardless of the selected precision, when selecting a direction within a category by template matching, template matching is performed with 1 / 2 pixel precision.

[0196] In some embodiments, for each intra-frame TMP coded block, if one of the sub-pixel positions is selected, template matching is used to reorder the list of sub-pixel positions.

[0197] As an example, all sub-pixel positions are sorted by template matching based on the SAD value. Then, the best N positions are used to construct a sub-pixel position list, and an index is signaled to indicate which sub-pixel position is selected to predict the current block. The value of N can be any positive integer less than or equal to the number of supported sub-pixel positions. For example, for Figure 11 The supported sub-pixel positions in , N is less than or equal to 24.

[0198] As another example, the pixel precisions are sorted by template matching based only on the SAD value. The direction can be determined by the syntax element, and then the pixel precisions in that direction are sorted by template matching based on the SAD value, and the pixel precision list is constructed using the best P precision, and an index is signaled to indicate which pixel precision is selected. The value of P can be any positive integer less than or equal to the number of pixel precisions supported in that direction. For example, for Figure 11 The supported sub-pixel positions in , P is less than or equal to 3.

[0199] As another example, the sub-pixel directions are sorted by template matching based only on the SAD value. The precision can be determined by a syntax element signaled in the bitstream, and then the sub-pixel directions with that precision are sorted using template matching based on the SAD value. The best D direction is used to construct a sub-pixel direction list, and an index is signaled to indicate which sub-pixel direction is selected. The value of D can be any positive integer less than or equal to the number of supported sub-pixel directions with that precision. For example, for Figure 11 The supported sub-pixel positions associated with D are less than or equal to 8. In some embodiments, the same sub-pixel direction list can be used for each precision, and the list can be derived with a certain precision. Figure 11 The associated supported sub-pixel position and sub-pixel direction lists are constructed with 1 / 2 pixel precision and can be used for 1 / 4 pixel precision and 4 / 3 pixel precision.

[0200] All mentioned indices can be signaled via fixed length coding, truncated unary coding, truncated binary coding, Exponential-Golomb coding, or any other binary coding method.

[0201] In some embodiments, the template matching cost may be the Sum of Absolute Transformed Difference (SATD), so that the above-mentioned SAD cost function may be replaced by a SATD cost function.

[0202] The present invention does not impose any specific limitation on the type and number of taps of the interpolation filter used for half-pixel interpolation.

[0203] For example, for horizontal half-pixel interpolation, a two-tap interpolation filter [32 32] may be used. As another example, a 4-tap DCT-IF interpolation filter [-4 36 36 -4] may be used. As another example, a 4-tap DCT-IF interpolation filter [-16 144 144 -16] may be used. As another example, a 4-tap weak DCT-IF interpolation filter [-5 37 37 -5] may be used. As another example, a 4-tap Gaussian interpolation filter [8 24 24 8] may be used. As another example, a 6-tap flat interpolation filter [3 9 20 20 9 3] may be used. As another example, a 6-tap DCT-IF interpolation filter [12 -44 160 160 -44 12] may be used. As another example, an 8-tap DCT-IF interpolation filter [-4 16 -44 160 160 -44 16 -4] may be used. As another example, a 12-tap DCT-IF interpolation filter [-2 6 13 25 -50 162 162 -50 25 13 6 -2] may be used. For vertical half-pixel interpolation, the interpolation filter is the transpose of the interpolation filter used for horizontal half-pixel interpolation.

[0204] For example, for horizontal quarter-pixel interpolation in the right direction (when the quarter-pixel position is to the right of the best integer-pixel position), a two-tap bilinear interpolation filter [48 16] may be used. As another example, a 4-tap DCT-IF interpolation filter [-16 216 64 -8] may be used. As another example, a 6-tap DCT-IF interpolation filter [12 -40 232 68 -20 4] may be used. As another example, an 8-tap DCT-IF interpolation filter [-4 16 -40 32368 -20 4 0] may be used. As another example, a 12-tap DCT-IF interpolation filter [-2 5 -11 21 -43230 75 -2915 -8 4 -1] may be used. For horizontal quarter-pixel interpolation in the left direction (when the quarter-pixel position is to the left of the best integer-pixel position), the interpolation filter is the horizontal flip of the interpolation filter for the horizontal quarter-pixel in the right direction. For vertical quarter-pixel interpolation in the top direction (when the quarter-pixel position is at the top of the best integer-pixel position), the interpolation filter is the transpose of the interpolation filter used for horizontal quarter-pixel interpolation in the left direction. For vertical quarter-pixel interpolation in the bottom direction (when the quarter-pixel position is at the bottom of the best integer-pixel position), the interpolation filter is the transpose of the interpolation filter used for horizontal quarter-pixel interpolation in the right direction.

[0205] For horizontal three-quarter pixel interpolation, the interpolation filter is the horizontal flip of the interpolation filter used for horizontal quarter pixel interpolation; and for vertical three-quarter pixel interpolation, the interpolation filter is the vertical flip of the interpolation filter used for horizontal quarter pixel interpolation.

[0206] Interpolation is performed using neighboring samples adjacent to the current block. According to some embodiments, when pixel-wise interpolation is performed on a matching block (see the definition of the intra TMP technique above), and at least one sample outside the matching block is input to the interpolation filter, the input sample outside the matching block will be filled by copying the closest reconstructed sample in the matching block. For example, Figure 13 As shown, a half-pixel position is used; the half-pixel position is between sample b and sample c; a 4-tap interpolation filter is used for interpolation; input sample a, input sample b, input sample c, and input sample d are each input to the interpolation filter. Input sample d is outside the matching block, so input sample c, which is the closest reconstructed sample to input sample d within the matching block, is used for filling: the value of input sample c is copied to input sample d.

[0207] According to some embodiments, when an input sample outside the matching block is not available, only padding is performed on the input sample during the interpolation process. In the case where the matching block is located at the boundary of an image, slice, or tile, there may be no adjacent samples on the entire side. In addition, although the upper adjacent samples and the left adjacent samples have been encoded or decoded before the matching block, the right adjacent samples and the lower adjacent samples may not have been encoded or decoded before the current coding block in the raster scan order. Other possible coding orders may also change the availability of adjacent samples at the entire upper, left, right, or lower edge. Therefore, the present disclosure regards non-existent or uncoded and undecoded adjacent samples along the edge as "unavailable". For example, as Figure 13 As shown, if the input sample d is available, there is no padding; if the input sample d is not available, it will be filled by copying the input sample c.

[0208] According to some embodiments, when determining half-pixel positions, in addition to the half-pixel positions directly adjacent to the ideal integer pixel, other half-pixel positions may also be considered. For example, all fractional pixel positions within a region surrounding the ideal integer pixel may be considered. The extent of this region may be related to the width and height of the current block.

[0209] For example, all half-pixel positions within a range of no more than 2 whole pixels from the best whole-pixel position obtained by template matching are supported. Thus, 1 / 2 pixel positions and 3 / 2 pixel positions are supported.

[0210] In some embodiments, integer pixel positions within a region surrounding the most ideal integer pixel position may also be considered. In some embodiments, a syntax element is first signaled to indicate which integer pixel position within the region surrounding the most ideal integer pixel position is selected, and then another syntax element is further signaled to indicate which fractional pixel position surrounding the integer pixel position is used.

[0211] The proposed sub-pixel position based intra TMP can be combined with other intra TMP tools.

[0212] In some embodiments, when combined with multi-candidate intra-frame TMP, fractional pixel position-based intra-frame TMP and multi-candidate intra-frame TMP are treated as two independent modes. That is, the multi-candidate index is signaled only when integer pixel positions are used; or, a flag is first signaled to indicate whether the multi-candidate method is used, and the fractional pixel position-based intra-frame TMP method can be used only when the multi-candidate method is not used.

[0213] In some embodiments, when combined with multiple candidate intra TMP, if multiple candidate intra TMP is used, the sub-pixel position related parameters identifying the sub-pixel position are signaled or derived using template matching after signaling the multiple candidate indexes.

[0214] In some embodiments, when combined with intra-frame TMP blending, fractional-pixel position-based intra-frame TMP and intra-frame TMP blending are treated as two independent modes. That is, the blending method is signaled only when integer-pixel positions are used; or the fractional-pixel position-based intra-frame TMP method is signaled only when intra-frame TMP blending is not used.

[0215] In some embodiments, when combined with intra-frame TMP fusion, the sub-pixel position related parameters identifying the sub-pixel position of each matching block used in the fusion method are signaled or derived by template matching. In some embodiments, the same sub-pixel position is used for all matching blocks used in the fusion method.

[0216] In some embodiments, when combined with the intra TMP filter, fractional pixel position-based intra TMP and the intra TMP filter are treated as two separate modes. That is, the filter method is signaled only when integer pixel positions are used, or the fractional pixel position-based intra TMP method is signaled only when the intra TMP filter is not used.

[0217] In some embodiments, when combined with an intra-frame TMP filter, if an intra-frame TMP filter is used, sub-pixel position-dependent parameters identifying sub-pixel positions are further signaled or derived through template matching. In some embodiments, the filtering process is performed after the sub-pixel position interpolation; in some embodiments, the filtering process is performed before the sub-pixel position interpolation.

[0218] In some embodiments, when only the left template intra frame TMP or only the upper template intra frame TMP is combined, the sub-pixel position-based intra frame TMP and the multi-mode intra frame TMP are treated as two separate modes. That is, only when integer pixel positions are used, is the left template intra frame TMP or only the upper template intra frame TMP signaled; or when an L-shaped template intra frame TMP is used, only the sub-pixel position-based intra frame TMP is signaled.

[0219] In some embodiments, when only the left template frame TMP or only the upper template frame TMP is combined, if only the left template frame TMP or only the upper template frame TMP is used, the sub-pixel position related parameters identifying the sub-pixel position are further transmitted by signaling or derived by template matching.

[0220] In some embodiments, the methods mentioned above for intra-frame TMP can be used in IBC.

[0221] Furthermore, according to exemplary embodiments of the present disclosure, the VVC standard encoder and the VVC standard decoder configure one or more processors of the computing system to perform the intra TMP mode according to one of two flipping modes: a horizontal flipping mode and a vertical flipping mode.

[0222] In horizontal flip mode, only the top template is used when performing template matching. Before template matching, the top template of the current block is horizontally flipped. Then, the template matching process is performed to find a template that matches the horizontally flipped top template. The corresponding matching block is then horizontally flipped and used as the prediction sample value for the current block.

[0223] In vertical flip mode, only the left template is used when performing template matching. Before template matching, the left template of the current block is vertically flipped, and then the template matching process is performed to find a template that matches the vertically flipped left template. The corresponding matching block is then vertically flipped and used as the prediction sample value for the current block.

[0224] According to another embodiment, a different flipping method is implemented, as described below.

[0225] In horizontal flip mode, only the top template is used when performing template matching. Before performing template matching, the top template of the current block is horizontally flipped. Then, the template matching process is performed to find a template that matches the horizontally flipped top template. The VVC standard encoder also configures one or more processors of the computing system to horizontally flip the original values of the current block before calculating the residual between the original values and the predicted sample values of the current block. After the current block is reconstructed, the reconstructed values are horizontally flipped.

[0226] In vertical flip mode, only the left template is used when performing template matching. Before performing template matching, the left template of the current block is vertically flipped. Then, the template matching process is performed to find a template that matches the vertically flipped left template. The VVC standard encoder also configures one or more processors of the computing system to vertically flip the original values of the current block before calculating the residual between the original values and the predicted sample values of the current block. After the current block is reconstructed, the reconstructed values are vertically flipped.

[0227] The effectiveness of flipping on intra-frame TMP-mode coded blocks can be signaled by a flag bit in the bitstream. Specifically, for a block, when a flag bit in the bitstream indicates that intra-frame TMP mode is used for prediction, another flag bit is signaled in the bitstream to indicate whether flipping is performed. Then, another flag bit is signaled to indicate whether horizontal flipping or vertical flipping is selected.

[0228] Those skilled in the art will understand that the above-mentioned aspects of the present disclosure may be implemented in any combination thereof, and the combination of all aspects of the present disclosure may be implemented as yet another embodiment of the present disclosure.

[0229] Figure 14 An example system 1400 is shown for implementing the above-described processes and methods for implementing intra template matching prediction mode.

[0230] The techniques and mechanisms described herein may be implemented by multiple instances of the system 1400, as well as any other computing devices, systems, and / or environments. Figure 14 The system 1400 shown is only one example of a system and is not intended to limit the scope of use or functionality of any computing device used to perform the above-described processes and / or procedures. Other well-known computing devices, systems, environments, and / or configurations that may be suitable for the embodiments include, but are not limited to, personal computers, server computers, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, game consoles, programmable consumer electronics devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, implementations using field programmable gate arrays (FPGAs) and application-specific integrated circuits (ASICs), and the like.

[0231] System 1400 may include one or more processors 1402 and a system memory 1404 communicatively coupled to the processor(s) 1402. The processor(s) 1402 may execute one or more modules and / or processes to enable the processor(s) 1402 to perform various functions. In some embodiments, the processor(s) 1402 may include a central processing unit (CPU), a graphics processing unit (GPU), a CPU and a GPU, or other processing units or components known in the art. In addition, each processor 1402 may have its own local memory, which may also store program modules, program data, and / or one or more operating systems.

[0232] System memory 1404 may be volatile, such as RAM, non-volatile, such as ROM, flash memory, a micro hard drive, a memory card, etc., or some combination thereof, depending on the exact configuration and type of system 1400. System memory 1404 may include one or more computer-executable modules 1406 that are executable by processor 1402.

[0233] Module 1406 may include, but is not limited to, one or more of an encoder 1408 and a decoder 1410 .

[0234] The encoder 1408 may be a VVC standard encoder implementing any, some or all aspects of the example embodiments of the present disclosure as described above, and may be executed by the processor(s) 1402 to configure the processor(s) 1402 to perform the operations described above.

[0235] The decoder 1410 may be a VVC standard encoder of an exemplary embodiment of the present disclosure implementing any, some or all aspects as described above, which may be executed by the processor(s) 1402 to configure the processor(s) 1402 to perform the operations described above.

[0236] The system 1400 may also include an input / output (I / O) interface 1440 for receiving image source data and bitstream data and for outputting the reconstructed image to a reference image buffer or a DPB (decoded image buffer) and / or a display buffer. The system 1400 may also include a communication module 1450 that allows the system 1400 to communicate with other devices (not shown) via a network (not shown). The network may include the Internet, wired media such as a wired network or a direct wired connection, and wireless media such as acoustic, radio frequency (RF), infrared, and other wireless media.

[0237] In one implementation, a calculation method is provided, comprising: Match the current block's template with a template in a search region: Determine the integer pixel position of the block vector corresponding to the matching block of the matching template: Determining fractional pixel positions around the integer pixel position; and The reconstructed values of the matching blocks at fractional pixel positions around the determined integer pixel position are derived as predicted sample values of the current block.

[0238] In one implementation, deriving the reconstructed values of the matching blocks at fractional pixel positions around the determined integer pixel positions as the predicted sample values of the current block includes: The reconstructed values of the matching blocks at the determined sub-pixel positions are derived by applying an interpolation filter.

[0239] In one implementation, deriving the reconstructed value of the matching block at the determined sub-pixel position by applying an interpolation filter includes: The reconstructed values of the matching block are interpolated at half-pixel positions by applying a 4-tap DCT-IF interpolation filter [-16 144 144 -16].

[0240] In one implementation, deriving the reconstructed value of the matching block at the determined sub-pixel position by applying an interpolation filter includes: The reconstructed values of the matching blocks are interpolated at quarter-pixel positions by applying a 4-tap DCT-IF interpolation filter [-16 216 64 -8].

[0241] In one implementation, the calculation method further includes: filling unavailable input samples by copying the closest reconstructed samples in the matching block.

[0242] In one implementation, the fractional pixel positions around the integer pixel position include precision and direction; wherein the precision includes the magnitude of the offset from the integer pixel position; and The direction comprises one of a plurality of cardinal directions starting from the integer-pixel position.

[0243] In one implementation, the set of pixel positions surrounding the integer pixel position includes twenty-four combinations of precision and direction, each precision is selected from three fractional pixel precisions, and each direction is selected from eight cardinal directions.

[0244] In one implementation, the three sub-pixel precisions include 1 / 4 pixel precision, 1 / 2 pixel precision, and 3 / 4 pixel precision.

[0245] In one implementation, the calculation method further includes: By interpolating the matching template of each sub-pixel position, template matching is performed between the integer pixel position and the sub-pixel positions around the integer pixel position.

[0246] In one implementation, the calculation method further includes: A first flag is sent in the bitstream, which is used to identify fractional pixel positions around the reference integer pixel position in intra template matching prediction (ie, intra TMP).

[0247] In one implementation, the calculation method further includes: A second flag bit is sent in the bitstream and is used to identify one of a group of pixel positions surrounding the integer pixel position.

[0248] In one implementation, sending a second flag bit in the bitstream for identifying one of a group of pixel positions surrounding the integer pixel position includes: One of a set of pixel positions is sent in a syntax element of the bitstream.

[0249] In one implementation, sending a second flag bit in the bitstream for identifying one of a group of pixel positions surrounding the integer pixel position includes: signaling a precision of one of a set of pixel positions in a first syntax element of the bitstream; and In a second syntax element of the bitstream following the first syntax element, a direction of one of the set of pixel positions is signaled.

[0250] In one implementation, sending a second flag bit in the bitstream for identifying one of a group of pixel positions surrounding the integer pixel position includes: sending a direction of one of a set of pixel positions in a first syntax element of the bitstream; and The precision of one of the set of pixel positions is signaled in a second syntax element of the bitstream following the first syntax element.

[0251] In one implementation, sending a second flag bit in the bitstream for identifying one of a group of pixel positions surrounding the integer pixel position includes: One of the group of pixel positions is transmitted according to a binarization code selected from a fixed length code, a truncated unary code, a truncated binary code, and an exponential Golomb code.

[0252] In one implementation, the calculation method further includes: Reading a first flag bit from the bit stream; and The deriving of the reconstructed values of the matching blocks at fractional pixel positions around the determined integer pixel positions as the predicted sample values of the current block includes: Based on the first flag bit, reconstructed values of matching blocks at fractional pixel positions around the determined integer pixel position are interpolated as predicted sample values of the current block.

[0253] In one implementation, the calculation method further includes: Reading a second flag bit from the bit stream: and The determined sub-pixel positions around the integer pixel position include: The sub-pixel positions around the integer pixel position are determined based on the second flag bit that identifies the sub-pixel positions around the integer pixel position.

[0254] In one implementation, a computing system is provided, comprising: a matching module configured to match the template of the current block with a template in a search area; a determination module configured to determine an integer pixel position of a block vector corresponding to a matching block of the matching template; and determine a fractional pixel position around the integer pixel position; and The deriving module is configured to derive the reconstructed values of the matching block at fractional pixel positions around the determined integer pixel position as the predicted sample values of the current block.

[0255] In one implementation, the deriving module is configured to derive the reconstructed value of the matching block at the determined sub-pixel position by applying an interpolation filter.

[0256] In one implementation, the deriving module is configured to interpolate the reconstructed values of the matching block at half-pixel positions by applying a 4-tap DCT-IF interpolation filter [-16 144 144 -16].

[0257] In one implementation, the deriving module is configured to interpolate the reconstructed values of the matching block at quarter-pixel positions by applying a 4-tap DCT-IF interpolation filter [-16216 64 -8].

[0258] In one implementation, the computing system further includes: The filling module is configured to fill in the unavailable input samples by copying the closest reconstructed samples in the matching block.

[0259] In one implementation, the fractional pixel positions around the integer pixel position include precision and direction; wherein the precision includes a magnitude of an offset from the integer pixel position; and The direction comprises one of a plurality of cardinal directions starting from an integer pixel position.

[0260] In one implementation, the set of pixel positions surrounding the integer pixel position includes twenty-four combinations of precision and direction, each precision is selected from three fractional pixel precisions, and each direction is selected from eight cardinal directions.

[0261] In one implementation, the three sub-pixel precisions include 1 / 4 pixel precision, 1 / 2 pixel precision, and 3 / 4 pixel precision.

[0262] In one implementation, the computing system further includes: The template matching module is configured to perform template matching between the integer pixel position and the sub-pixel positions around the integer pixel position by interpolating the matching template of each sub-pixel position respectively.

[0263] In one implementation, the computing system further includes: The sending module is configured to send a first flag in a bit stream, which is used to identify a certain sub-pixel position around the reference integer pixel position in the intra-frame template matching prediction (ie, intra-frame TMP).

[0264] In one implementation, the sending module is further configured to send a second flag bit in the bit stream, which is used to identify one of a group of pixel positions surrounding the integer pixel position.

[0265] In one implementation, the sending module is further configured to send one of a set of pixel positions in a syntax element of the bitstream.

[0266] In one implementation, the sending module is configured to send a precision of one of a group of pixel positions in a first syntax element of the bitstream, and to send a direction of one of the group of pixel positions in a second syntax element of the bitstream following the first syntax element.

[0267] In one implementation, the sending module is configured to send the direction of one of the set of pixel positions in a first syntax element of the bitstream, and to send the precision of one of the set of pixel positions in a second syntax element following the first syntax element of the bitstream.

[0268] In one implementation, the sending module is configured to send one of the group of pixel positions according to a binarization code selected from fixed-length coding, truncated unary coding, truncated binary coding, and exponential Golomb coding.

[0269] In one implementation, the computing system further includes: a reading module configured to read a first flag bit from the bit stream; and The export module is configured to interpolate the reconstructed values of the matching blocks at the fractional pixel positions around the determined integer pixel position based on the first flag bit as the predicted sample values of the current block.

[0270] In one implementation, the computing system further includes: a reading module configured to read a second flag bit from the bit stream; and The derivation module is configured to determine the sub-pixel positions around the integer pixel position based on the second flag bit that identifies the sub-pixel positions around the integer pixel position.

[0271] In one implementation, a computing system is provided, comprising: one or more processors, and A computer-readable storage medium is communicatively coupled to one or more processors, wherein the computer-readable storage medium stores computer-readable instructions executable by the one or more processors. When executed by the one or more processors, the computing method described above is performed.

[0272] In one implementation, a non-transitory computer-readable storage medium storing a bitstream associated with one or more images is provided. When the bitstream is calculated by the processor, the processor is caused to execute the calculation method described above.

[0273] In one implementation, a computer program product is provided, comprising: Computer program instructions, wherein the computer program instructions enable a computer to execute the computing method as described above.

[0274] In one implementation, a computer program is provided, wherein The computer program enables a computer to execute the calculation method described above.

[0275] Some or all of the operations of the above methods may be performed by executing computer-readable instructions stored on a computer-readable storage medium, as defined below. As used in the specification and claims, the term "computer-readable instructions" includes routines, applications, application modules, program modules, programs, components, data structures, algorithms, and the like. Computer-readable instructions may be implemented on a variety of system configurations, including single-processor or multi-processor systems, minicomputers, mainframe computers, personal computers, handheld computing devices, microprocessor-based programmable consumer electronic devices, and combinations thereof.

[0276] Computer-readable storage media may include volatile memory (such as random access memory (RAM)) and / or non-volatile memory (such as read-only memory (ROM), flash memory, etc.). Computer-readable storage media may also include additional removable and / or non-removable storage, including but not limited to flash memory, magnetic storage, optical storage, and / or tape storage, which may provide non-volatile storage of computer-readable instructions, data structures, program modules, etc.

[0277] Non-transitory or non-temporary computer-readable storage media are examples of computer-readable media. Computer-readable media include at least two types of computer-readable media: computer-readable storage media and communication media. Computer-readable storage media include volatile and non-volatile, removable and non-removable media implemented in any process or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc-read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device. In contrast, communication media may embody computer-readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism. The computer-readable storage medium used herein should not be interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (such as light pulses through a fiber optic cable), or an electrical signal propagating through a wire.

[0278] When executed by one or more processors, the computer readable instructions stored on one or more non-transitory or non-transitory computer readable storage media may perform the above-referenced Figure 1A-10 The operations described, generally, are computer-readable instructions including routines, programs, objects, components, data structures, etc. that perform specific functions or implement specific abstract data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations can be combined in any order and / or in parallel to implement the process.

[0279] Although the subject matter has been described in language specific to structural features and / or methodological steps, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts disclosed are example forms of implementing the claims.

Claims

1. A calculation method comprising: Matching the template of the current block with the template in a search area; determining an integer pixel position of a block vector corresponding to a matching block of the matching template; Determine fractional pixel positions around the integer pixel position; as well as The reconstructed values of the matching blocks at fractional pixel positions around the determined integer pixel position are derived as prediction sample values of the current block.

2. The calculation method according to claim 1, wherein: The deriving, as the prediction sample value of the current block, the reconstructed value of the matching block at the fractional pixel position around the determined integer pixel position comprises: The reconstructed values of the matching blocks at the determined sub-pixel positions are derived by applying an interpolation filter.

3. The calculation method according to claim 2, wherein: The deriving the reconstructed value of the matching block at the determined sub-pixel position by applying an interpolation filter comprises: The reconstructed values of the matching block are interpolated at half-pixel positions by applying a 4-tap DCT-IF interpolation filter [-16 144 144-16].

4. The calculation method according to claim 2, wherein: The deriving the reconstructed value of the matching block at the determined sub-pixel position by applying an interpolation filter comprises: The reconstructed values of the matching blocks are interpolated at quarter-pixel positions by applying a 4-tap DCT-IF interpolation filter [-16 216 64-8].

5. The calculation method according to any one of claims 2 to 4, further comprising: Unavailable input samples are filled in by copying the closest reconstructed samples in the matching block.

6. The calculation method according to any one of claims 1 to 5, wherein: The fractional pixel positions around the integer pixel position include precision and direction; wherein the precision includes a magnitude of an offset from the integer pixel position; and The direction comprises one of a plurality of cardinal directions starting from the integer-pixel position.

7. The calculation method according to claim 6, wherein: A group of pixel positions around the integer pixel position includes twenty-four combinations of precision and direction, each precision is selected from three sub-pixel precisions, and each direction is selected from eight cardinal directions.

8. The calculation method according to claim 7, wherein: The three sub-pixel precisions include 1 / 4 pixel precision, 1 / 2 pixel precision, and 3 / 4 pixel precision.

9. The calculation method according to any one of claims 1 to 8, further comprising: By interpolating the matching template of each sub-pixel position, template matching is performed between the integer pixel position and the sub-pixel positions around the integer pixel position.

10. The calculation method according to any one of claims 1 to 9, further comprising: The first flag is sent in the bitstream, which is used to identify a certain sub-pixel position around the integer pixel position to be referenced in the intra-frame template matching prediction, wherein the intra-frame template matching prediction is intra-frame TMP.

11. The calculation method according to claim 10, further comprising: A second flag bit is sent in the bitstream, identifying one of a group of pixel positions surrounding the integer pixel position.

12. The calculation method according to claim 11, wherein: The sending of a second flag bit in the bitstream, which is used to identify one of a group of pixel positions surrounding the integer pixel position, includes: One of a set of pixel positions is sent in a syntax element of the bitstream.

13. The calculation method according to claim 11, wherein: The sending of a second flag bit in the bitstream for identifying one of a group of pixel positions surrounding the integer pixel position includes: signaling a precision of one of a set of pixel positions in a first syntax element of the bitstream; and In a second syntax element of the bitstream following the first syntax element, a direction of one of the set of pixel positions is signaled.

14. The calculation method according to claim 11, wherein: The sending of a second flag bit in the bitstream for identifying one of a group of pixel positions surrounding the integer pixel position includes: sending a direction of one of a set of pixel positions in a first syntax element of the bitstream; and In a second syntax element following the first syntax element in the bitstream, a precision of one of the set of pixel positions is signaled.

15. The calculation method according to claim 11, wherein: The sending of a second flag bit in the bitstream, which is used to identify one of a group of pixel positions surrounding the integer pixel position, includes: One of the group of pixel positions is transmitted according to a binarization code selected from fixed length coding, truncated unary coding, truncated binary coding, and exponential Golomb coding.

16. The calculation method according to claim 1, further comprising: Read the first flag bit from the bit stream; as well as The deriving of the matching block reconstruction value at a fractional pixel position around the determined integer pixel position as the prediction sample value of the current block includes: Based on the first flag bit, reconstructed values of matching blocks at fractional pixel positions around the determined integer pixel position are interpolated as predicted sample values of the current block.

17. The calculation method according to claim 16, further comprising: Reading a second flag bit from the bit stream; as well as The determined sub-pixel positions around the integer pixel position include: The sub-pixel positions around the integer pixel position are determined based on the second flag bit that identifies the sub-pixel positions around the integer pixel position.

18. A computing system comprising: a matching module configured to match the template of the current block with a template in a search area; a determination module configured to determine an integer pixel position of a block vector corresponding to a matching block of the matching template; and determine a fractional pixel position around the integer pixel position; as well as The deriving module is configured to derive the reconstructed values of the matching block at fractional pixel positions around the determined integer pixel position as the predicted sample values of the current block.

19. The computing system of claim 18, wherein The deriving module is configured to derive the reconstructed value of the matching block at the determined sub-pixel position by applying an interpolation filter.

20. The computing system of claim 19, wherein The derivation module is configured to interpolate the reconstructed values of the matching block at half-pixel positions by applying a 4-tap DCT-IF interpolation filter [-16 144 144-16].

21. The computing system of claim 19, wherein The derivation module is configured to interpolate the reconstructed values of the matching block at quarter-pixel positions by applying a 4-tap DCT-IF interpolation filter [-16 216 64-8].

22. The computing system of any one of claims 19 to 21, further comprising: The filling module is configured to fill in the unavailable input samples by copying the closest reconstructed samples in the matching block.

23. The computing system of any one of claims 18 to 22, wherein The fractional pixel positions around the integer pixel position include precision and direction; in, The precision includes the magnitude of the offset from the integer pixel location; and The direction comprises one of a plurality of cardinal directions starting from an integer pixel position.

24. The computing system of claim 23, wherein A group of pixel positions around the integer pixel position includes twenty-four combinations of precision and direction, each precision is selected from three sub-pixel precisions, and each direction is selected from eight cardinal directions.

25. The computing system of claim 24, wherein The three sub-pixel precisions include 1 / 4 pixel precision, 1 / 2 pixel precision, and 3 / 4 pixel precision.

26. The computing system of any one of claims 18 to 25, further comprising: The template matching module is configured to perform template matching between the integer pixel position and the sub-pixel positions around the integer pixel position by interpolating the matching template of each sub-pixel position.

27. The computing system of any one of claims 18 to 26, further comprising: The sending module is configured to send a first flag in a bit stream, which is used to identify a certain sub-pixel position around the reference integer pixel position in the intra-frame template matching, and the intra-frame template matching prediction is the intra-frame TMP.

28. The computing system of claim 27, wherein The sending module is further configured to send a second flag bit in the bit stream, which is used to identify one of a group of pixel positions surrounding the integer pixel position.

29. The computing system of claim 28, wherein The sending module is further configured to send one of a set of pixel positions in a syntax element of the bitstream.

30. The computing system of claim 28, wherein The sending module is configured to send a precision of one of a group of pixel positions in a first syntax element of the bitstream and to send a direction of one of the group of pixel positions in a second syntax element of the bitstream following the first syntax element.

31. The computing system of claim 28, wherein The sending module is configured to send a direction of one of a group of pixel positions in a first syntax element of the bitstream and to send a precision of one of the group of pixel positions in a second syntax element following the first syntax element of the bitstream.

32. The computing system of claim 28, wherein The sending module is configured to send one of the group of pixel positions according to a binarization code selected from fixed-length coding, truncated unary coding, truncated binary coding, and exponential Golomb coding.

33. The computing system of claim 18, further comprising: A reading module configured to read a first flag bit from a bit stream; as well as The export module is configured to interpolate the reconstructed values of the matching blocks at the fractional pixel positions around the determined integer pixel position based on the first flag bit as the predicted sample values of the current block.

34. The computing system of claim 33, further comprising: A reading module, configured to read a second flag bit from the bit stream; as well as The deriving module is configured to determine sub-pixel positions around the integer pixel position based on the second flag bit identifying the sub-pixel positions around the integer pixel position.

35. A computing system comprising: one or more processors, and A computer-readable storage medium is communicatively coupled to one or more processors, wherein the computer-readable storage medium stores computer-readable instructions executable by the one or more processors, and when executed by the one or more processors, performs the computing method according to any one of claims 1 to 17.

36. A non-transitory computer-readable storage medium storing a bitstream associated with one or more images, wherein the bitstream, when calculated by the processor, causes the processor to perform the calculation method according to any one of claims 1 to 17.

37. A computer program product comprising: Computer program instructions, wherein the computer program instructions enable a computer to perform the computing method according to any one of claims 1 to 17.

38. A computer program, wherein The computer program enables a computer to execute the computing method according to any one of claims 1 to 17.

Images