Similarity thresholds for candidate manifests

Abstract

The video codec generates a candidate prediction vector list of the current block. The video codec cuts down the candidate prediction vector manifest by removing the first prediction vector located within a threshold of the second prediction vector from the list. When the optimization is enabled, the threshold has a first value, and when the optimization is not enabled, the threshold has a second value. The video codec selects candidate prediction vectors from the reduced candidate prediction vector list. When optimization is enabled, the video codec optimizes the selected prediction vector and reconstructs the current block by encoding or decoding the current block using the optimized prediction vector. When the optimization is not enabled, the video codec encodes or decodes the current block by using the selected prediction vector to reconstruct the current block.

Description

[0001] Cross-references

[0002] This application is part of a non-provisional application claiming priority to U.S. Provisional Patent Application No. 63 / 339,534 filed on May 9, 2022. The above application is hereby incorporated by reference in its entirety. Technical Field

[0003] The present invention generally relates to video encoding and decoding, and more particularly to a method for encoding and decoding pixel blocks by creating a candidate list. Background Art

[0004] Unless otherwise indicated herein, the methods described in this section are not prior art in the scope of the following patent claims and are not admitted to be prior art by inclusion of this section.

[0005] High-Efficiency Video Coding (HEVC) is an international video codec standard developed by the Joint Collaborative Team on Video Coding (JCT-VC). HEVC is based on a hybrid block-based motion compensated DCT-like transform codec architecture. The basic unit of compression is called a coding unit (CU), which is a 2N×2N square pixel block. Each CU can be recursively divided into four smaller CUs until a preset minimum size is reached. Each CU contains one or more prediction units (PUs).

[0006] Versatile video coding (VVC) is the latest international video codec standard developed by ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11 Joint Video Expert Team (JVET). The input video signal is predicted from the reconstructed signal, which is derived from the coded picture region. The prediction residual signal is processed by block transform. The transform coefficients are quantized and entropy coded along with other side information in the bitstream. The reconstructed signal is generated from the prediction signal and the reconstructed residual signal after inverse transforming the de-quantized transform coefficients. The reconstructed signal is also processed by in-loop filtering to remove coding artifacts. The decoded picture is stored in a buffer and used to predict future pictures in the input video signal.

[0007] In VVC, the coded image is divided into non-overlapping square block areas represented by related coding tree units (CTUs). The leaf nodes of the coding tree correspond to coding units (CUs). The coded image can be represented by a set of slices, each slice including an integer number of CTUs. Each CTU in a slice is processed in raster scan order. A bi-predictive slice (B slice) can be decoded using intra prediction or inter prediction with up to two motion vectors and reference indices to predict the sample values ​​of each block. A predictive slice (P slice) is decoded using intra prediction or inter prediction with up to one motion vector and reference index to predict the sample values ​​of each block. Intra slices are decoded using only intra prediction.

[0008] Using a quadtree (QT) with a nested multi-type-tree (MTT) structure, the CTU can be divided into one or more non-overlapping codec units (CUs) to accommodate various local motion and texture features. The CU is further divided into smaller CUs using one of five partition types: quadtree partition, vertical binary tree partition, horizontal binary tree partition, vertical center-side ternary tree partition, and horizontal center-side ternary tree partition.

[0009] Each CU contains one or more prediction units (PUs). The prediction unit together with the related CU syntax serves as the basic unit for signaling the predictor information. The specified prediction process is used to predict the values ​​of the relevant pixel samples in the PU. Each CU may contain one or more transform units (TUs) representing the prediction residual blocks. The transform unit (TU) includes a transform block (TB) of luma samples and two corresponding transform blocks of chroma samples, each TB corresponding to a residual block sample of a color component. An integer transform is applied to the transform block. The level value of the quantized coefficient is entropy encoded and decoded in the bitstream together with other side information. The terms coding tree block (CTB), coding block (CB), prediction block (PB), transform block (TB) are defined as 2D sample arrays (syntax elements) that indicate a color component associated with a CTU, CU, PU, ​​or TU, respectively. Therefore, a CTU includes a luma CTB, two chroma CTBs, and related syntax elements. Similar relationships apply to CU, PU, ​​and TU.

[0010] For each inter-predicted CU, motion parameters including motion vector, reference picture index and reference picture list usage index and additional information are used to generate inter-predicted samples. Motion parameters can be indicated in an explicit or implicit manner. When the CU is encoded and decoded using skip mode, the CU is associated with one PU and has no valid residual coefficients, no coded motion vector delta or reference picture index. When merge mode is specified, the motion parameters of the current CU are obtained from neighboring CUs including spatial candidates and temporal candidates, as well as the additional schedule introduced in VVC. Merge mode can be applied to any inter-predicted CU. An alternative to merge mode is the explicit transmission of motion parameters, where the motion vector, the corresponding reference picture index and reference picture list usage flag for each reference picture list and other required information are explicitly indicated for each CU. Summary of the invention

[0011] The following summary is illustrative only and is not intended to be limiting in any way. That is, the following summary is provided to introduce the concepts, highlights, benefits, and advantages of the new and non-obvious technologies described herein. Selected but not all embodiments are further described in the detailed description below. Therefore, the following summary is not intended to determine the essential characteristics of the claimed subject matter, nor is it intended to determine the scope of the claimed subject matter.

[0012] Some embodiments provide a method for pruning a list of prediction candidates using a similarity threshold before refining a selected candidate. A video codec generates a list of candidate prediction vectors for a current block. The video codec generates a list of candidate prediction vectors for the current block. The video codec prunes the list of candidate prediction vectors by removing a first prediction vector from the list that is within a threshold of a second prediction vector. The threshold has a first value when refinement is enabled and a second value when refinement is not enabled. The video codec selects a candidate prediction vector from the pruned list of candidate prediction vectors.

[0013] When optimization is enabled, the video codec optimizes the selected prediction vector and reconstructs the current block by encoding or decoding the current block using the optimized prediction vector. When optimization is not enabled, the video codec reconstructs the current block by encoding or decoding the current block using the selected prediction vector. The list of candidate prediction vectors may be generated based on a prediction mode used to encode or decode the current block. The prediction mode may be a merge mode, a combined intra-inter prediction mode (CIIP), a geometric prediction mode (GPM), an intra block copy (IBC), or another prediction mode. The selected prediction vector may be a motion vector that references a sample in a reference image or a block vector that references a sample in the current image.

[0014] In some embodiments, the first value of the threshold when optimization is enabled is greater than the second value of the threshold when optimization is disabled. In some embodiments, the first threshold is the same regardless of the prediction mode used to encode the current block. In some embodiments, the first threshold has different values ​​for different prediction modes. In some embodiments, the video codec performs optimization through a template matching (TM) process, i.e., the selected prediction vector is optimized to minimize the cost between template pixels adjacent to the current block and corresponding pixels referenced by the optimized prediction vector. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The drawings are included to provide a further understanding of the present invention and are incorporated into and constitute a part of the present invention. The drawings illustrate embodiments of the present invention and together with the description are used to explain the principles of the present invention. It should be noted that the drawings are not necessarily drawn to scale, as some components may be shown out of proportion to the size in the actual embodiment in order to clearly illustrate the concepts of the present invention.

[0016] Figure 1 It is conceptually shown that template matching is performed based on a search area around an initial motion vector (MV).

[0017] Figure 2 The CU is partitioned by the geometric partitioning mode (GPM).

[0018] Figure 3 An example uni-prediction candidate list for GPM segmentation and selection of the uni-prediction MV for the GPM are shown.

[0019] Figure 4 Example GPM partition edge blending for a CU is shown.

[0020] Figure 5A -C conceptually illustrates the use of a similarity threshold to prune the candidate list for subsequent optimization based on whether the selected candidate will be optimized.

[0021] Figure 6 An example video encoder is shown that can prune a list of prediction candidates based on a similarity threshold.

[0022] Figure 7 The portion of the video encoder that implements candidate list reduction and prediction vector optimization is marked.

[0023] Figure 8 The process of pruning a list of candidate prediction vectors based on a similarity threshold when encoding a pixel block is conceptually illustrated.

[0024] Fig. 9 An example video decoder is shown that can prune a list of prediction candidates based on a similarity threshold.

[0025] Fig.10 The portion of the video decoder that implements candidate list reduction and prediction vector optimization is highlighted.

[0026] Fig.11 The process of pruning a list of candidate prediction vectors based on a similarity threshold when encoding a pixel block is conceptually illustrated.

[0027] Fig.12 An electronic system for implementing some embodiments of the present invention is conceptually illustrated. DETAILED DESCRIPTION

[0028] In the following detailed description, a large number of specific details are explained by way of example in order to provide a thorough understanding of the relevant teachings. Any changes, derivations and / or extensions based on the teachings described herein are within the scope of protection of the present invention. In some examples, in order to avoid unnecessary confusion of aspects of the teachings of the present invention, known methods, procedures, components and / or circuits related to one or more exemplary embodiments disclosed herein are described at a relatively high level without details.

[0029] I. MV Optimization Based on Template Matching (TM)

[0030] Template Matching (TM) is a decoder-side MV derivation method that refines the motion information of the current CU by finding the closest match between the template of the current CU in the current image (e.g., the top neighboring block and / or the left neighboring block of the current CU) and the set of pixels in the reference image (i.e., the same size as the template).

[0031] Figure 1The performance of template matching based on a search area around an initial motion vector (MV) is conceptually illustrated. As shown, for a current CU 105 in a current image 100, the video codec searches a reference image or frame 101 within a [-8, +8] pixel search range around the initial MV 110 for a better or refined MV 111. The search is based on minimizing the difference (or cost) between a current template 120 adjacent to the current CU 105 and a reference template 121 identified by the refined MV 111. Template matching may be performed using a search step size determined based on an adaptive motion vector resolution mode (AMVR). The template matching process may be cascaded with a bilateral matching process in a merge mode.

[0032] In the advanced motion vector prediction (AMVP) mode, the candidate MVP is determined based on the template matching error to select the one with the smallest difference between the current block template and the reference block template, and then TM is performed only on this specific candidate MVP for MV refinement. The TM process refines the MVP candidate by using iterative diamond search, starting with full-pixel MVD accuracy (or 4 pixels in 4-pixel AMVR mode) within the [-8, +8] pixel search range. The AMVP candidate can be further refined by using a cross search with full-pixel MVD accuracy (or 4 pixels in 4-pixel AMVR mode), and then performing half-pixel and quarter-pixel searches sequentially according to the AMVR mode search model, as shown in Table 1 below.

[0033] Table 1: Search model of AMVR and merging mode with AMVR

[0034]

[0035] This search process ensures that the MVP candidate still maintains the same MV accuracy as indicated by the AMVR mode after the TM process. During the search process, if the difference between the previous minimum cost and the current minimum cost in the iteration is less than a threshold, which is equal to the block area, the search process terminates.

[0036] In some embodiments, when the merge mode is used, the video codec can apply a similar TM search method to optimize the merge candidates indicated by the merge index. As shown in Table 1 above, TM can be performed all the way to 1 / 8 pixel MVD accuracy or skip MVD accuracy beyond half pixel, depending on whether an alternative interpolation filter is used based on the merged motion information (which is used when AMVR is in half pixel mode). In addition, when TM mode is enabled, template matching can be used as an independent process, or as an additional MV optimization process between the block-based bilateral matching (BM) method and the sub-block-based bilateral matching (BM) method, depending on whether BM can be enabled based on its enabling condition.

[0037] II. Multi-split prediction mode with TM

[0038] A. Combined inter and intra prediction (CIIP)

[0039] When a CU is encoded and decoded in merge mode, if the CU contains at least 64 luma samples (i.e., the CU width multiplied by the CU height is equal to or greater than 64), and if both the CU width and the CU height are less than 128 luma samples, an additional flag may be indicated to indicate whether the combined intra-inter prediction (CIIP) mode is applied to the current CU. The CIIP prediction combines the inter-picture prediction signal and the frame intra-picture prediction signal. In some embodiments, the CIIP mode P inter The inter-frame prediction signal in is derived using the same inter-frame prediction process as applied to the normal merge mode; the intra-frame prediction signal P intra It is derived through a conventional intra prediction process with one or more intra prediction modes derived from the planar mode or from the default mechanism. The intra prediction signal and the inter prediction signal are combined using a weighted average method, where the weights are calculated based on the codec modes of the top neighbor and the left neighbor. CIIP predicts P CIIP is formed as follows: (wt is the weight value)

[0040] P CIIP =((4–wt)*P inter +wt*P intra +2)>>2

[0041] In some embodiments, the motion candidates selected from candidate list I are used to generate Pinter, and the intra-picture prediction mode selected from candidate list II is used to generate Pintra. In some embodiments, candidate list I is the same as the merge candidate list of the current block. In some embodiments, candidate list I is a subset of the merge candidate list.

[0042] B. Template-based Intra Mode Derivation (TIMD)

[0043] Template-based intra mode derivation (TIMD) is a coding method in which the intra prediction mode of a CU is implicitly derived by using neighboring templates at both the encoder and decoder sides, instead of the encoder signaling the exact intra prediction mode to the decoder.

[0044] In some embodiments, for each intra-picture prediction mode in the MPM list, the SATD between the predicted sample and the reconstructed sample of the template is calculated. The first two intra-picture prediction modes with the smallest SATD are selected as TIMD modes. After applying the PDPC process, the two TIMD modes are fused with weight values, and this weighted intra-picture prediction is used to encode and decode the current CU. Position-dependent intra-picture prediction combination (PDPC) is included in the derivation of the TIMD mode.

[0045] The costs of the two selected modes (mode 1 and mode 2) are compared with the threshold. In the test, a cost factor of 2 is applied as follows:

[0046] costMode2<2*costMode1

[0047] If this condition is true, prediction fusion is applied, otherwise only mode 1 is used. The weight of a mode is calculated from its SATD cost as follows:

[0048] weight1=costMode2 / (costMode1+costMode2)

[0049] weight2=1-weight1

[0050] In some embodiments, for CIIP mode, the prediction sample is an inter-picture prediction signal (P inter ) and the intra-frame prediction signal (P intra In some embodiments, the method is only applicable to codec blocks with an area less than or equal to 1024.

[0051] C. CIIP with TIMD and TM combined patterns

[0052] In some embodiments, the TIMD derivation method is used to derive the intra-picture prediction mode in CIIP. Specifically, the intra-picture prediction mode with the smallest SATD value in the TIMD mode list is selected and mapped to one of the 67 conventional intra-picture prediction modes. In some embodiments, if the intra-picture prediction mode derived by TIMD is an angle mode, the two CIIP predictions (P intra and P inter In some embodiments, the weights (wIntra, wInter) for different sub-blocks are modified for the angle mode, as shown in Table 2:

[0053] Table 2:

[0054] Sub-block index (wIntra,wInter) 0 (6,2) 1 (5,3) 2 (3,5) 3 (2,6)

[0055] Using CIIP-TM, a CIIP-TM merge candidate list is created for use in CIIP-TM mode. The merge candidates in the CIIP-TM merge candidate list are optimized through template matching. Using the template matching method, the CIIP-TM merge candidates can also be reordered as regular merge candidates through adaptive reordering of merge candidates (ARMC). (ARMC is a method in which merge candidates are divided into several subgroups and the merge candidates in the subgroups are reordered based on the cost value based on the template matching.) In some embodiments, the maximum number of CIIP-TM merge candidates is 2.

[0056] D. Geometric Prediction Mode (GPM)

[0057] In VVC, geometric partitioning modes are supported for inter-picture prediction. Geometric partitioning modes (GPM) are indicated using the CU layer flag as a merge mode, along with other merge modes including normal merge mode, MMVD mode, CIIP mode, and sub-block merge mode. A total of 64 partitions are supported for each possible CU size w×h=2 m ×2 n Where m,n∈{3…6}, excluding 8x64 and 64x8.

[0058] Figure 2The partitioning of a CU by a geometric partitioning pattern (GPM) is indicated. Each GPM partition or GPM division is characterized by a distance-angle pairing that defines a bisector. The figure shows an example of a GPM partition grouped by the same angle. As shown in the figure, when using GPM, the CU is divided into two parts by a straight line at a geometric position. The position of the dividing line is mathematically derived from the angle and offset parameters of the specific partition.

[0059] Each part of the geometric partition in the CU is inter-picture predicted using its own motion (vector). Each partition only allows uni-prediction, that is, each part has one motion vector and one reference index. Uni-prediction motion constraints are applied to ensure that only two motion compensated predictions are performed for each CU, similar to traditional bi-prediction.

[0060] If GPM is used for the current CU, a geometric partitioning index indicating the partitioning mode of the geometric partitioning (angle and offset) and two merge indices (one for each partition) are also indicated. The merge index of the geometric partitioning is used to select candidates from a uni-prediction candidate list (also called a GPM candidate list). The maximum number of candidates in the GPM candidate list is explicitly indicated in the SPS to specify the syntactic binarization of the GPM merge index. After predicting each part of the geometric partition, the sample values ​​along the edges of the geometric partition are adjusted using a blending process with adaptive weight values. This is the prediction signal for the entire CU, and the transform and quantization process will be applied to the entire CU as in other prediction modes. The motion field of the CU predicted by the GPM is then stored.

[0061] The single prediction candidate list of the GPM partition (GPM candidate list) can be directly derived from the merge candidate list of the current CU. Figure 3 An example uni-prediction candidate list 300 for GPM segmentation and the selection of the uni-prediction MV for the GPM is shown. The GPM candidate list 300 is constructed in an even-odd fashion using only uni-prediction candidates that alternate between L0 MV and L1 MV. Let n be the index of the uni-prediction motion in the uni-prediction candidate list for the GPM. The LX (i.e., L0 or L1) motion vector of the nth extended merge candidate, where X is equal to the parity of n, is used as the nth uni-prediction motion vector for the GPM. (These motion vectors are marked with "x" in the figure.) If there is no corresponding LX motion vector for the nth extended merge candidate, the L(1-X) motion vector of the same candidate is used as the uni-prediction motion vector for the GPM.

[0062] As mentioned before, the sample values ​​along the edges of geometric partitions are adjusted using a blending process with adaptive weights. Specifically, after predicting each part of the geometric partition by its own motion, blending is applied to the two prediction signals to derive samples around the edges of the geometric partitions. The blending weights for each position of the CU are derived based on the distance between the respective position and the partition edge. Figure 4 An example GPM segmentation edge blending is shown for CU 400. In the figure, the blending weights are generated based on the initial blending weights w0.

[0063] A block coded by GPM may have one partition coded in inter-mode and one partition coded in intra-mode. This GPM mode may be referred to as GPM with inter and intra, or GPM-intra. In some embodiments, each GPM partition has a corresponding flag in the bitstream to indicate whether the GPM partition is coded by intra prediction or by inter prediction. For a GPM partition coded by inter prediction, the prediction signal is generated by MVs in the merge candidate list from the CU. For a GPM partition coded by intra prediction, the prediction signal is generated from neighboring pixels of the intra prediction mode specified by an index from the encoder. The possible variants of the intra prediction mode may be restricted by geometry. As in conventional GPM mode, the final prediction of a GPM-coded CU is generated by combining (blending at the partition edge) the prediction of the inter prediction partition and the prediction of the intra prediction partition (i.e., a partition with two inter predictions).

[0064] In some embodiments, by reusing a merge candidate list, a bi-prediction candidate list is allowed into a GPM candidate list. In some embodiments, a merge candidate list (which includes uni-prediction candidates and / or bi-prediction candidates) is used as a GPM candidate list. In some embodiments, a GPM candidate list that may include bi-prediction candidates (e.g., reusing a merge candidate list) is only allowed for smaller CUs (having a size less than a threshold) and / or when GPM-intra (e.g., a GPM mode that combines inter- and intra-prediction) is enabled, in order to constrain the motion compensation bandwidth. Otherwise (CUs greater than or equal to the threshold), only uni-prediction is allowed and the GPM candidate list will be constructed in an even-odd manner.

[0065] E. GPM with TM

[0066] In some embodiments, template matching (TM) can be applied to optimize the MV of the GPM partition. When the GPM mode is enabled for a CU, a CU layer flag is indicated to indicate whether TM is applied to both geometric partitions. Using TM, the motion information of each geometric partition is optimized. When TM is selected, a template is constructed using at least one of the left neighboring sample and the top neighboring sample according to the partition angle. Table 3 below shows the templates for the first geometric partition and the second geometric partition, where A indicates using the top sample, L indicates using the left sample, and L+A indicates using the left sample and the top sample.

[0067] Table 3:

[0068]

[0069] Then, motion is optimized by minimizing the difference between the current template and the template in the reference image using the same merge mode search model while disabling the half-pixel interpolation filter. The GPM candidate list is constructed as follows: (1) The video codec directly derives interlaced List-0 MV candidates and List-1 MV candidates from the merge candidate list, where List-0 MV candidates have a higher priority than List-1 MV candidates. A reduction method with an adaptive threshold based on the current CU size is applied to remove redundant candidate MVs; (2) The video codec further directly derives interlaced List-1 MV candidates and List-0 MV candidates from the regular merge candidate list, where List-1 MV candidates have a higher priority than List-0 MV candidates. The same reduction method with an adaptive threshold is also applied to remove redundant candidate MVs; (3) The video codec fills the GPM candidate list with zero MV candidates until the GPM candidate list is complete.

[0070] In some embodiments, GPM-MMVD and GPM-TM are enabled only for one CU using a GPM. In some embodiments, this is accomplished by first indicating the GPM-MMVD syntax. When both GPM-MMVD control flags are set to false (i.e., GPM-MMVD is disabled for both GPM partitions), the GPM-TM flag is indicated to indicate whether template matching is applied to both GPM partitions. Otherwise (at least one GPM-MMVD flag is set to true), the value of the GPM-TM flag is inferred to be false.

[0071] III. Intra Block Copy (IBC) Mode

[0072] Intra-picture block copy (IBC) is also called current picture reference (CPR). An IBC (or CPR) motion vector refers to one of the reconstructed reference samples in the current picture. For some embodiments, the IBC prediction mode is considered as a third prediction mode in addition to the inter-picture prediction mode or intra-picture prediction mode for encoding and decoding a CU.

[0073] Since the IBC mode is implemented as a block-level codec mode, block matching (BM) is performed at the encoder to find the best block vector (or motion vector) for each CU. Here, the block vector (BV) is used to indicate the displacement from the current block to the reference block, which has been reconstructed in the current image. The luma block vector of the IBC-coded CU is integer precision.

[0074] Chroma block vectors are also rounded to integer precision. When combined with AMVR, IBC mode can switch between 1-pixel motion vector precision and 4-pixel motion vector precision. IBC mode is applicable to CUs with width and height both less than or equal to 64 luma samples.

[0075] On the encoder side, hash-based motion estimation is performed for IBC. The encoder performs RD checks on blocks with width or height no greater than 16 luma samples. For non-merge mode, a block vector search is performed using a hash-based search first. If the hash search does not return a valid candidate, a local search based on block matching is performed. In the hash-based search, the hash key match (32-bit CRC) between the current block and the reference block is extended to all allowed block sizes. The hash key calculation for each position in the current image is based on 4×4 sub-blocks. For current blocks of larger sizes, the hash key is determined to match the hash key of the reference block when all hash keys of all 4×4 sub-blocks match the hash keys of the corresponding reference positions. If the hash keys of multiple reference blocks are found to match the hash key of the current block, the block vector cost of each matching reference block is calculated, and the matching reference block with the minimum cost is selected.

[0076] In block matching search, the search range is set to cover the previous CTU and the current CTU. At the CU level, the IBC mode is indicated by a flag, which can be marked as IBC AMVP mode or IBC skip / merge mode.

[0077] IV. Narrowing the TM Candidate List

[0078] Some embodiments of the present invention provide a method for improving coding efficiency of prediction vector optimization through TM. A prediction vector may refer to a motion vector (MV) that refers to a pixel in a reference image, or a block vector (BV) that refers to a pixel in a current image. Specifically, when TM is used to optimize a prediction vector (e.g., MV or BV) to generate a prediction (for skip, merge, direct, IBC, CIIP, GPM, etc.), the candidate list from which the selected prediction vector comes is reduced by applying a similarity check. The similarity check removes candidates from the candidate list that are too similar to at least one other candidate in the list. In some embodiments, the similarity check is based on a similarity threshold, so that candidate prediction vectors that are within a similarity threshold from another prediction vector are removed from the candidate list. In some embodiments, the similarity threshold is set to prevent the candidate list from having two or more candidates that are optimized to the same prediction vector by an optimization process (e.g., a TM process, a boundary matching process, or another optimization process).

[0079] In some embodiments, the video codec applies a similarity threshold regardless of whether TM optimization is enabled for the current block. The threshold may be preset or signaled by a syntax element in the bitstream.

[0080] Figure 5A -C conceptually illustrates the use of a similarity threshold to prune a list of candidates for subsequent optimization based on whether the selected candidate is to be optimized. These figures illustrate a current block 500 in a current image 510. The current block 500 is encoded or decoded by a prediction mode or codec that uses a prediction vector to obtain pixels as predictors or prediction samples of the current block. Such a codec or prediction mode may be any of merge mode, CIIP, GPM, IBC, etc.

[0081] The codec selected to encode or decode the current block uses a candidate list 520, which initially includes candidates A through H. Candidates A through H include predictor vectors (MVs and / or BVs) that reference samples in the current image 510 and reference images 511 and 512. Specifically, candidates A and B are MVs that reference samples in reference image 511. Candidates C, D, and E are MVs that reference samples in reference image 512. Candidates F, G, and H are BVs that reference samples in the current image 510.

[0082] Figure 5A It is indicated that the video codec performs prediction based on the candidate list 520 without optimization. The video codec does not perform pruning on the candidate list 520 (or performs pruning with a very small threshold), and candidate D is selected from the candidate list 520. Candidate D is used to obtain prediction sample 530 without optimization. Predictor 530 is then used to encode the current block 500.

[0083] Figure 5B It is indicated that the video codec prunes the candidate list 520 when optimization is enabled. Since optimization is enabled, the video codec applies a similarity threshold 505 to prune the candidate list 520. In some embodiments, the threshold 505 is set to identify prediction vectors that are too similar and can be optimized to be the same. In this example, the difference between MV C and MV D is less than the threshold 505, so candidate D is pruned from the candidate list 520. The difference between BV F and BV G is also less than the threshold 505, so candidate G is also pruned from the candidate list 520. A pruned candidate list 521 is created without candidate D and candidate G. The prediction of the current block 500 will be based on the candidates selected from the pruned candidate list 521.

[0084] Figure 5C It is indicated that the video codec performs prediction based on the reduced candidate list 521 with optimization enabled. In this example, candidate C is selected from the reduced candidate list 521. The video codec performs a template matching (TM) process to optimize MVC to MVC' based on a template 540 of pixels above and to the left of the current block 500. The optimized MVC' is then used to obtain prediction samples 531 from the reference image 512 for encoding and decoding the current block 500.

[0085] In some embodiments, when TM is used, the similarity threshold is the same for different prediction modes, for example, the threshold is the same for a regular merge mode with TM, a CIIP mode with TM, a GPM mode with TM, etc.

[0086] In some embodiments, the similarity threshold of the candidates of the coding / prediction mode with TM enabled is different from the similarity threshold without TM enabled. For example, the threshold of the conventional merge mode (or CIIP mode or GPM mode) without TM enabled is different from the threshold of the conventional merge mode (or CIIP mode or GPM mode) with TM enabled. Or the threshold of the conventional merge mode without TM enabled is less than the threshold of the conventional merge mode with TM enabled. Or the threshold of the conventional merge mode without TM enabled is greater than the threshold of the conventional merge mode with TM enabled. In some embodiments, when TM is not enabled, the similarity threshold is set to zero or a very small value, so that no candidates are cut off.

[0087] The proposed method may be implemented in an encoder and / or a decoder. For example, the proposed method may be implemented in an intra prediction module and / or an intra block copy prediction module of an encoder and / or an intra prediction module (and / or a frame block copy prediction module) of a decoder.

[0088] V. Example Video Codec

[0089] Figure 6 An exemplary video encoder 600 is shown that can reduce a list of prediction candidates based on a similarity threshold. As shown, the video encoder 600 receives an input video signal from a video source 605 and encodes the signal into a bitstream 695. The video encoder 600 has several components or modules for encoding the signal from the video source 605, including at least some components selected from a transform module 610, a quantization module 611, an inverse quantization module 614, an inverse transform module 615, an intra-image estimation module 620, an intra-image prediction module 625, a motion compensation module 630, a motion estimation module 635, a loop filter 645, a reconstructed image buffer 650, a motion vector (MV) buffer 665, and a motion vector prediction module 675, and an entropy encoder 690. The motion compensation module 630 and the motion estimation module 635 are part of the inter-image prediction module 640.

[0090] In some embodiments, modules 610-690 are modules of software instructions being executed by one or more processing units (e.g., processors) of a computing device or electronic device. In some embodiments, modules 610-690 are modules of hardware circuits implemented by one or more integrated circuits (ICs) of an electronic device. Although modules 610-690 are shown as separate modules, some of these modules can be combined into a single module.

[0091] The video source 605 provides a raw video signal, which represents pixel data for each video frame without compression. The subtractor 608 calculates the difference between the raw video pixel data of the video source 605 and the predicted pixel data 613 from the motion compensation module 630 or the intra-frame image prediction module 625 as a prediction residual 609. The transform module 610 transforms the difference (or residual pixel data or residual signal 608) into transform coefficients (e.g., by performing a discrete cosine transform (DCT)). The quantization module 611 quantizes the transform coefficients into quantized data (or quantized coefficients) 612, which are encoded into the bitstream 695 by the entropy encoder 690.

[0092] The inverse quantization module 614 dequantizes the quantized data (or quantized coefficients) 612 to obtain transform coefficients, and the inverse transform module 615 inversely transforms the transform coefficients to generate a reconstructed residual 619. The reconstructed residual 619 is added to the predicted pixel data 613 to generate reconstructed pixel data 617. In some embodiments, the reconstructed pixel data 617 is temporarily stored in a line buffer (not shown) for intra-frame image prediction and spatial MV prediction. The reconstructed pixels are filtered by the in-loop filter 645 and stored in a reconstructed image buffer 650. In some embodiments, the reconstructed image buffer 650 is a storage external to the video codec 600. In some embodiments, the reconstructed image buffer 650 is a storage internal to the video encoder 600.

[0093] The image intra-picture estimation module 620 performs intra-picture prediction based on the reconstructed pixel data 617 to generate intra-picture prediction data. The intra-picture prediction data is provided to the entropy encoder 690 to be encoded into a bitstream 695. The intra-picture prediction data is also used by the intra-picture prediction module 625 to generate the predicted pixel data 613.

[0094] The motion estimation module 635 performs inter-picture prediction by generating motion vectors to reference pixel data of previously decoded frames stored in the reconstructed picture buffer 650. These motion vectors are provided to the motion compensation module 630 to generate predicted pixel data.

[0095] Instead of encoding the complete actual MV in the bitstream, the video codec 600 uses MV prediction to generate a predicted MV, and the difference between the MV used for motion compensation and the predicted MV is encoded as residual motion data and stored in the bitstream 695.

[0096] The motion vector prediction module 675 generates a predicted motion vector, i.e., a motion compensated motion vector used to perform motion compensation, based on the reference motion vector generated for encoding the previous video information frame. The motion vector prediction module 675 retrieves the reference motion vector from the previous video information frame from the motion vector buffer 665. The video encoder 600 stores these motion vectors generated for the current video information frame in the motion vector buffer 665 as reference motion vectors used to generate the predicted motion vector.

[0097] The motion vector prediction module 675 uses the reference motion vector to create a predicted motion vector. The predicted motion vector can be calculated by spatial motion vector prediction or temporal motion vector prediction. The difference (residual motion data) between the predicted motion vector and the motion compensation motion vector (MC MV) of the current information frame is encoded into a bitstream 695 by the entropy encoder 690.

[0098] Using entropy coding techniques, such as context-adaptive binary arithmetic coding (CABAC) or Huffman coding, the entropy encoder 690 encodes various parameters and data into a bitstream 695. The entropy encoder 690 encodes various header elements, flags, and quantized transform coefficients 612 and residual motion data as syntax elements into the bitstream 695. In turn, the bitstream 695 is stored in a storage device or transmitted to a decoder via a communication medium such as a network.

[0099] The loop filter 645 performs filtering or smoothing operations on the reconstructed pixel data 617 to reduce encoding and decoding artifacts, especially artifacts located at the boundaries of pixel blocks. In some embodiments, the filtering operation or smoothing operation performed by the loop filter 645 includes a deblocking filter (DBF), a sample adaptive offset (SAO) and / or an adaptive loop filter (ALF).

[0100] Figure 7 Portions of the video encoder 600 that implement candidate list reduction and prediction vector optimization are labeled. Specifically, the figure illustrates components of the motion compensation module 630 of the video encoder 600.

[0101] The motion compensation module 630 retrieves (at a "candidate fetch" block 710) existing motion vectors and / or block vectors from the motion vector buffer 665 to form an initial candidate list 715 for a prediction mode 712. The prediction mode (e.g., merge mode, CIIP, GPM, IBC, etc.) is specified by the motion estimation module 635 and indicated in the bitstream 695 by the entropy encoder 690.

[0102] The motion compensation module 630 prunes the candidate list 715 (at a "prune" block 720) based on the similarity threshold 722, thereby pruning the prediction vectors in the candidate list that are within the similarity threshold of another prediction vector from the candidate list. The result of the pruning operation is a pruned candidate list 725. In some embodiments, the value of the similarity threshold 722 is determined based on whether optimization is to be performed on the candidate prediction vector to be selected. For example, when TM optimization is to be performed on the selected candidate, the value of the threshold may be greater than the value when TM optimization is not to be performed.

[0103] The motion compensation module 630 selects (at a "candidate selection" block 730) a candidate prediction vector 735 from the reduced candidate list 725. The candidate selection 732 may be provided by the motion estimation module 635. The motion compensation module 630 optimizes (at an "optimization" block 740) the selected prediction vector 735 into an optimized prediction vector 745 by searching the contents 742 of the reconstructed image buffer 650. The optimization may be based on a TM optimization that minimizes the difference between the current template neighboring the current block and a reference template in a reference image referenced by the prediction vector. The TM optimization is performed by referring to the above Figure 1 Describing.

[0104] The motion compensation module 630 uses the optimized prediction vector 745 to retrieve (at a "prediction retrieval" block 750) prediction samples 752 from the reconstructed image buffer 650. Depending on the prediction mode 712 (e.g., by generating a weighted sum of inter-frame prediction samples and intra-frame prediction samples for CIIP mode), the motion compensation module 630 generates predicted pixel data 613 based on the retrieved prediction samples 752.

[0105] Figure 8 The process 800 of pruning a list of candidate prediction vectors based on a similarity threshold when encoding a pixel block is conceptually illustrated. In some embodiments, one or more processing units (e.g., processors) of a computing device implementing the encoder 600 perform the process 800 by executing instructions stored in a computer-readable medium. In some embodiments, an electronic device implementing the encoder 600 performs the process 800.

[0106] The encoder receives (at block 810) data for a current pixel block in a current picture to be encoded as a video. The encoder generates (at block 820) a list of candidate prediction vectors for the current block. The list of candidate prediction vectors may be generated based on a prediction mode used to encode or decode the current block. The prediction mode may be a merge mode, a combined intra-inter prediction mode (CIIP), a geometric prediction mode (GPM), an intra-block copy (IBC), or other prediction modes.

[0107] The encoder prunes (at block 830) the list of candidate prediction vectors by removing from the list a first prediction vector that is within a threshold of a second prediction vector. The threshold has a first value when optimization is enabled and a second value when optimization is not enabled. In some embodiments, the first value of the threshold when optimization is enabled is greater than the second value of the threshold when optimization is disabled. In some embodiments, the first threshold is the same regardless of the prediction mode used to encode the current block. In some embodiments, the first threshold has different values ​​for different prediction modes. (In some embodiments, the enabled optimization is a TM optimization process.)

[0108] The encoder selects (at block 840) a candidate prediction vector from the pruned list of candidate prediction vectors. The selected prediction vector may be a motion vector that references samples in a reference picture, or a block vector that references samples in the current picture.

[0109] The encoder determines (at block 850) whether optimization is enabled for the current block. If optimization is enabled, the process continues to 870. If optimization is not enabled, the encoder encodes (at block 860) the current block using the selected prediction vector to reconstruct the current block.

[0110] At block 870, the encoder optimizes the selected prediction vector. In some embodiments, the encoder performs the optimization via a TM process, i.e., the selected prediction vector is optimized to minimize the cost between the template pixels adjacent to the current block and the corresponding pixels referenced by the optimized prediction vector. The encoder then encodes the current block (at block 880) using the optimized prediction vector to reconstruct the current block and generate a prediction residual. The selected prediction vector may be used to construct a predictor for the current block according to the prediction mode of the current block.

[0111] VI. Sample Video Decoder

[0112] In some embodiments, the encoder may indicate (or generate) one or more syntax elements in the bitstream, so that the decoder may parse the one or more syntax elements from the bitstream.

[0113] Fig. 9 An example video decoder 900 is shown that can prune a list of prediction candidates based on a similarity threshold. As shown, the video decoder 900 is an image decoding or video decoding circuit that receives a bitstream 995 and decodes the contents of the bitstream into pixel data of a video information box for display. The video decoder 900 has several components or modules for decoding the bitstream 995, including some components selected from an inverse quantization module 911, an inverse transform module 910, an intra-picture prediction module 925, a motion compensation module 930, a loop filter 945, a decoded image buffer 950, a motion vector buffer 965, a motion vector prediction module 975, and a parser 990. The motion compensation module 930 is part of the inter-picture prediction module 940.

[0114] In some embodiments, modules 910-990 are modules of software instructions executed by one or more processing units (e.g., processors) of a computing device. In some embodiments, modules 910-990 are hardware circuit modules implemented by one or more ICs of an electronic device. Although modules 910-990 are illustrated as independent modules, some of these modules may be combined into a single module.

[0115] The parser 990 (or entropy decoder) receives the bitstream 995 and performs initial parsing according to the syntax defined by the video codec or image codec standard. The parsed syntax elements include various header elements, flags, and quantized data (or quantized coefficients) 912. The parser 990 parses out the various syntax elements by using entropy coding techniques such as context-adaptive binary arithmetic coding (CABAC) or Huffman coding.

[0116] The inverse quantization module 911 dequantizes the quantized data (or quantized coefficients) 912 to obtain transform coefficients, and the inverse transform module 910 inversely transforms the transform coefficients 916 to generate a reconstructed residual 919. The reconstructed residual 919 is added to the predicted pixel data 913 from the intra-picture prediction module 925 or the motion compensation module 930 to generate decoded pixel data 917. The decoded pixel data is filtered by the in-loop filter 945 and stored in the decoded image buffer 950. In some embodiments, the decoded image buffer 950 is a storage outside the video decoder 900. In some embodiments, the decoded image buffer 950 is a storage inside the video decoder 900.

[0117] The intra-picture prediction module 925 receives the intra-picture prediction data from the bitstream 995 and generates predicted pixel data 913 based on the intra-picture prediction data from the decoded pixel data 917 stored in the decoded image buffer 950. In some embodiments, the decoded pixel data 917 is also stored in a line buffer (not shown) for intra-picture prediction and spatial MV prediction.

[0118] In some embodiments, the contents of the decoded image buffer 950 are used for display. The display device 955 retrieves the contents of the decoded image buffer 950 for direct display, or retrieves the contents of the decoded image buffer to a display buffer. In some embodiments, the display device receives pixel values ​​from the decoded image buffer 950 via pixel transfer.

[0119] Based on motion compensated MV (MC MV), the motion compensation module 930 generates predicted pixel data 913 from the decoded pixel data 917 stored in the decoded picture buffer 950. These motion compensated MVs are decoded by adding the residual motion data received from the bitstream 995 to the predicted MV received from the motion vector prediction module 975.

[0120] The motion vector prediction module 975 generates a predicted MV, for example, a motion compensated MV for performing motion compensation, based on a reference MV generated for decoding a previous video information frame. The motion vector prediction module 975 retrieves the reference motion vector of the previous video information frame from the motion vector buffer 965. The video decoder 900 also stores the motion compensated motion vector generated for decoding the current video information frame in the motion vector buffer 965 as a reference motion vector for generating a predicted motion vector.

[0121] The loop filter 945 performs filtering or smoothing operations on the decoded pixel data to reduce encoding and decoding artifacts, especially artifacts located at the boundaries of pixel blocks. In some embodiments, the filtering or smoothing operations performed by the loop filter 945 include a deblock filter (DBF), a sample adaptive offset (SAO), and / or an adaptive loop filter (ALF).

[0122] Fig.10 Portions of the video decoder 900 that implement candidate list pruning and prediction vector optimization are labeled. Specifically, the figure illustrates components of the motion compensation module 930 of the video decoder 900.

[0123] The motion compensation module 930 retrieves (at a "candidate fetch" block 1010) existing motion vectors and / or block vectors from the motion vector buffer 965 to form an initial candidate list 1015 for a prediction mode 1012. The prediction mode (e.g., merge mode, CIIP, GPM, IBC, etc.) is specified by the entropy decoder 990 based on syntax elements in the bitstream 995.

[0124] The motion compensation module 930 prunes the candidate list 1015 (at a "prune" block 1020) based on the similarity threshold 1022, thereby pruning the prediction vectors in the candidate list that are within the similarity threshold of another prediction vector from the candidate list. The result of the pruning operation is a pruned candidate list 1025. In some embodiments, the value of the similarity threshold 1022 is determined based on whether optimization is to be performed on the candidate prediction vector to be selected. For example, when TM optimization is to be performed on the selected candidate, the value of the threshold may be greater than the value when TM optimization is not to be performed.

[0125] The motion compensation module 930 selects (at a "candidate selection" block 1030) a candidate prediction vector 1035 from the reduced candidate list 1025. The candidate selection 1032 may be provided by the entropy decoder 990. The motion compensation module 930 optimizes (at an "optimization" block 1040) the selected prediction vector 1035 into an optimized prediction vector 1045 by searching the contents 1042 of the reconstructed image buffer 950. The optimization may be based on a TM optimization that minimizes the difference between the current template neighboring the current block and a reference template in a reference image referenced by the prediction vector. The TM optimization is performed by referring to the above Figure 1 Describing.

[0126] The motion compensation module 930 uses the optimized prediction vector 1045 to retrieve (at a "prediction retrieval" block 1050) prediction samples 1052 from the decoded picture buffer 950. The motion compensation module 930 generates predicted pixel data 913 based on the retrieved prediction samples 1052 according to the prediction mode 1012 (e.g., by generating a weighted sum of inter-frame prediction samples and intra-frame prediction samples for CIIP mode).

[0127] Fig.11 Conceptually, a process 1100 of pruning a list of candidate prediction vectors based on a similarity threshold when encoding a pixel block is shown. In some embodiments, one or more processing units (e.g., processors) of a computing device implementing decoder 900 perform process 1100 by executing instructions stored in a computer-readable medium. In some embodiments, an electronic device implementing decoder 900 performs process 1100.

[0128] The decoder receives (at block 1110) data for a current pixel block in a current picture to be decoded as a video. The decoder generates (at block 1120) a list of candidate prediction vectors for the current block. The list of candidate prediction vectors may be generated based on a prediction mode used to encode and decode the current block. The prediction mode may be a merge mode, a combined intra-inter prediction mode (CIIP), a geometric prediction mode (GPM), an intra-block copy mode (IBC), or other prediction modes.

[0129] The encoder prunes (at block 1130) a list of candidate prediction vectors by removing from the list a first prediction vector that is within a threshold of a second prediction vector. The threshold has a first value when optimization is enabled and a second value when optimization is not enabled. In some embodiments, the first value of the threshold when optimization is enabled is greater than the second value of the threshold when optimization is disabled. In some embodiments, the first threshold is the same regardless of the prediction mode used to decode the current block. In some embodiments, the first threshold has different values ​​for different prediction modes. (In some embodiments, the enabled optimization is a TM optimization process.)

[0130] The decoder selects (at block 1140) a candidate prediction vector from the reduced list of candidate prediction vectors. The selected prediction vector may be a motion vector that references samples in a reference picture, or a block vector that references samples in the current picture.

[0131] The decoder determines (at block 1150) whether optimization is enabled for the current block. If optimization is enabled, the process continues to 1170. If optimization is not enabled, the decoder decodes (at block 1160) the current block by using the selected prediction vector to reconstruct the current block.

[0132] At block 1170, the decoder optimizes the selected prediction vector. In some embodiments, the decoder performs the optimization via a TM process, i.e., the selected prediction vector is optimized to minimize the cost between a template pixel adjacent to the current block and a corresponding pixel referenced by the optimized prediction vector. The decoder then decodes the current block (at block 1180) using the optimized prediction vector to reconstruct the current block and generate a prediction residual. The selected prediction vector may be used to construct a predictor for the current block based on a prediction mode for the current block. The decoder may then provide the reconstructed current block as part of a reconstructed current image for display.

[0133] VII. Example Electronic System

[0134] Many of the above features and applications can be implemented as software processes, which are specified as sets of instructions recorded on a computer readable storage medium (also referred to as a computer readable medium). When these instructions are executed by one or more computing units or processing units (e.g., one or more processors, processor cores, or other processing units), the instructions cause the processing units to perform the actions represented by the instructions. Examples of computer readable media include, but are not limited to, CD-ROMs, flash drives, random access memory (RAM) chips, hard disks, erasable programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), etc. Computer readable media do not include carriers and telecommunications signals connected wirelessly or wired.

[0135] In this specification, the term "software" is meant to include firmware in a read-only memory or an application stored in a magnetic storage device, which can be read into the memory for processing by the processor. At the same time, in some embodiments, multiple software inventions can be implemented as sub-parts of a larger program, while retaining different software inventions. In some embodiments, multiple software inventions can be implemented as independent programs. Finally, any combination of independent programs that implement the software inventions described herein together is within the scope of the present invention. In some embodiments, when installed to operate on one or more electronic systems, the software program defines one or more specific machine implementations that execute and implement the operations of the software program.

[0136] Fig.12An electronic system 1200 is conceptually shown as being implemented in some embodiments of the present invention. The electronic system 1200 may be a computer (e.g., a desktop computer, a personal computer, a tablet computer, etc.), a phone, a PDA, or other types of electronic devices. The electronic system includes various types of computer-readable media and interfaces for various other types of computer-readable media. The electronic system 1200 includes a bus 1205, a processing unit 1210, a graphics-processing unit (GPU) 1215, a system memory 1220, a network 1225, a read-only memory (ROM) 1230, a permanent storage device 1235, an input device 1240, and an output device 1245.

[0137] The bus 1205 collectively represents all system buses, peripheral buses, and chipset buses that are communicatively coupled to a number of internal devices of the electronic system 1200. For example, the bus 1205 is communicatively coupled to the processing unit 1210 via the graphics processing unit 1215, the read-only memory 1230, the system memory 1220, and the permanent storage device 1235.

[0138] From these various memory units, the processing unit 1210 retrieves instructions to execute and data to process in order to perform the processes of the present invention. In various embodiments, the processing unit may be a single processor or a multi-core processor. Certain instructions are transmitted to and executed by the image processing unit 1215. The image processing unit 1215 may offload various calculations or supplement the image processing provided by the processing unit 1210.

[0139] The read-only memory 1230 stores static data and instructions required by the processing unit 1210 or other modules of the electronic system. On the other hand, the permanent storage device 1235 is a read-and-write memory device. This device is a non-volatile memory unit that stores instructions and data even when the electronic system 1200 is turned off. Some embodiments of the present invention use a large-capacity storage device (such as a magnetic disk or optical disk and its corresponding magnetic disk drive) as the permanent storage device 1235.

[0140] Other embodiments use removable storage devices (such as floppy disks, flash memory devices, etc., and their corresponding disk drives) as permanent storage devices. Like permanent storage device 1235, system memory 1220 is a read-write memory device. However, unlike storage device 1235, system memory 1220 is a volatile read-write memory, such as random access memory. System memory 1220 stores instructions and data that some processors need at runtime. In some embodiments, processing according to the present invention is stored in system memory 1220, permanent storage device 1235 and / or read-only memory 1230. For example, various memory units include instructions for processing multimedia clips according to some embodiments. For these various memory units, processing unit 1210 retrieves executed instructions and processed data in order to perform processing of certain embodiments.

[0141] The bus 1205 is also connected to input devices 1240 and output devices 1245. The input devices 1240 enable a user to communicate information and select commands to the electronic system. The input devices 1240 include an alphanumeric keyboard and pointing device (also known as a "cursor control device"), a camera (such as a webcam), a microphone or similar device for receiving voice commands, etc. The output devices 1245 display images generated by the electronic system or data output in other ways. The output devices 1245 include printers and display devices, such as cathode ray tubes (CRT) or liquid crystal displays (LCD), and speakers or similar audio output devices. Some embodiments include devices such as touch screens that function as both input devices and output devices.

[0142] Finally, if Fig.12 As shown, bus 1205 also couples electronic system 1200 to a network 1225 via a network adapter (not shown). In this manner, the computer may be part of a computer network (e.g., a local area network (LAN), a wide area network (WAN), or an intranet) or a network of networks (e.g., the Internet). Any or all of the components of electronic system 1200 may be used in conjunction with the present invention.

[0143] Some embodiments include electronic components, such as a microprocessor, a storage device, and a memory that stores computer program instructions to a machine-readable medium or a computer-readable medium (optionally referred to as a computer-readable storage medium, a machine-readable medium, or a machine-readable storage medium). Some examples of computer-readable media include RAM, ROM, read-only compact discs (CD-ROM), recordable compact discs (CD-R), rewritable compact discs (CD-RW), read-only digital versatile discs (e.g., DVD-ROM, dual-layer DVD-ROM), various recordable / rewritable DVDs (e.g., DVD RAM, DVD-RW, DVD+RW, etc.), flash memory (e.g., SD card, mini SD card, micro SD card, etc.), magnetic and / or solid-state hard drives, read-only and recordable Computer readable media may store computer programs executed by at least one processing unit and include sets of instructions for performing various operations. Examples of computer programs or computer code include machine code, such as that produced by a compiler, and files containing high-level code executed by a computer, electronic component, or microprocessor using an interpreter.

[0144] While the above discussion primarily refers to microprocessors or multi-core processors that execute software, many of the above functions and applications are performed by one or more integrated circuits, such as application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions stored on the circuits themselves. In addition, some embodiments execute software stored in programmable logic devices (PLDs), ROM, or RAM devices.

[0145] As used in the specification and any claims of the present invention, the terms "computer", "server", "processor" and "memory" refer to electronic devices or other technical devices. These terms do not include people or groups. For the purpose of this specification, the term display or display device refers to displaying on an electronic device. As used in the specification and any claims of the present invention, the terms "computer-readable medium", "computer-readable medium" and "machine-readable medium" are entirely limited to tangible, physical objects that store information in a computer-readable form. These terms do not include any wireless signals, wired download signals and any other transient signals.

[0146] While the present invention has been described in conjunction with many specific details, those skilled in the art will recognize that the present invention may be implemented in other specific forms without departing from the spirit of the present invention. Figure 8 and Fig.11 ) conceptually illustrates the processes. The specific operations of these processes may not be performed in the exact order shown and described. These specific operations may not be performed in a continuous series of operations, and different specific operations may be performed in different embodiments. In addition, the processes are implemented using several sub-processes, or as part of a larger macro process. Therefore, it will be understood by those skilled in the art that the present invention is not limited by the foregoing illustrative details, but is defined by the scope of the claims.

[0147] Additional Notes

[0148] The subject matter described herein sometimes represents different components, which are contained in or connected to other different components. It is understood that the described structure is only an example and can actually be implemented by many other structures to implement the same function. Conceptually, any arrangement of components that implement the same function is actually "associated" in order to implement the desired function. Therefore, regardless of the structure or intermediate components, any two components combined to implement a specific function are considered to be "interrelated" to implement the desired function. Similarly, any two associated components are considered to be "operably connected" or "operably coupled" to each other to implement a specific function. Any two components that can be associated with each other are also considered to be "operably coupled" to each other to implement a specific function. Specific examples of operable connections include, but are not limited to, physically pairable and / or physically interacting components, and / or wirelessly interactive and / or wirelessly interactive components, and / or logically interactive and / or logically interactive components.

[0149] Furthermore, with respect to the use of substantially any plural and / or singular terms, those of ordinary skill in the art can translate from the plural to the singular and / or from the singular to the plural as appropriate to the context and / or application. For clarity, the various singular / plural permutations are expressly provided herein.

[0150] In addition, it will be understood by those skilled in the art that, in general, the terms used in the present invention, especially in the claims, such as the subject matter of the claims, are generally used as "open" terms, for example, "including" should be interpreted as "including but not limited to", "having" should be interpreted as "at least", "including" should be interpreted as "including but not limited to", etc. It will be further understood by those skilled in the art that if a specific number of claimed contents are intended to be introduced, it will be explicitly indicated in the claims, and will not be displayed in the absence of such contents. For example, to aid understanding, the claims may include the phrases "at least one" and "one or more" to introduce the contents of the claims. However, the use of these phrases should not be construed as implying the use of the indefinite article "a" or "an" to introduce the contents of the claims, thereby limiting any particular patent scope. Even when the same claim includes the introductory phrases "one or more" or "at least one", the indefinite article, such as "a" or "an", should be interpreted to mean at least one or more, and the same is true for the use of explicit descriptions used to introduce claims. Furthermore, even when an introductory content explicitly refers to a specific number, one of ordinary skill in the art will recognize that such content should be interpreted as indicating the number of references, for example, "two references" without other modifications means at least two references, or two or more references. Furthermore, where a statement similar to "at least one of A, B, and C" is used, it is generally stated so that one of ordinary skill in the art can understand the statement, for example, "a system includes at least one of A, B, and C" will include but is not limited to a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B, and C, etc. One of ordinary skill in the art will further understand that any separated words and / or phrases represented by two or more alternative terms, whether in the specification, claims, or drawings, should be understood to include the possibility of one of these terms, one of them, or both of these terms. For example, "A or B" should be understood as the possibility of "A", or "B", or "A and B".

[0151] As can be seen from the foregoing, various embodiments have been described herein for illustrative purposes and that various modifications may be made without departing from the scope and spirit of the invention. Therefore, the various embodiments disclosed herein are not intended to be limiting, with the claims indicating the true scope and spirit.

Claims

1. A video encoding and decoding method, include: receiving data for a block of pixels of a current block of a current picture to be encoded or decoded as a video; Generate a candidate prediction vector list for the current block; pruning the list of candidate prediction vectors by removing from the list first prediction vectors that are within a threshold of a second prediction vector, wherein the threshold has a first value when optimization is enabled and a second value when optimization is not enabled; selecting a candidate prediction vector from the reduced list of candidate prediction vectors; When optimization is enabled: optimizing the selected prediction vector; By using the optimized prediction vector, encoding or decoding the current block to reconstruct the current block; When optimization is not enabled: The current block is encoded or decoded by using the selected prediction vector to reconstruct the current block.

2. The method according to claim 1, It is characterized in that in, The selected prediction vector is a motion vector of a reference sample in a reference picture.

3. The method according to claim 1, It is characterized in that in, The selected prediction vector is a block vector that refers to a reference sample in the current picture.

4. The method according to claim 1, It is characterized in that in, The selected prediction vector is optimized to minimize the cost between template pixels adjacent to the current block and corresponding pixels referenced by the optimized prediction vector.

5. The method according to claim 1, It is characterized in that in, The selected prediction vector is used to construct a predictor for the current block according to the prediction mode.

6. The method according to claim 5, It is characterized in that in, The prediction mode is one of a merge mode, a combined intra-inter prediction mode (CIIP), a geometric prediction mode (GPM), and an intra block copy (IBC).

7. The method according to claim 6, It is characterized in that in, The first threshold is the same regardless of the prediction mode used to encode or decode the current block.

8. The method according to claim 6, It is characterized in that in, The first threshold has different values ​​for different prediction modes.

9. The method according to claim 1, It is characterized in that in, The first value of the threshold when optimization is enabled is greater than the second value of the threshold when optimization is disabled.

10. An electronic device, include: A video codec circuit is configured to perform a plurality of operations, including: receiving data for a block of pixels of a current block of a current picture to be encoded or decoded as a video; Generate a candidate prediction vector list for the current block; pruning the list of candidate prediction vectors by removing from the list first prediction vectors that are within a threshold of a second prediction vector, wherein the threshold has a first value when optimization is enabled and a second value when optimization is not enabled; selecting a candidate prediction vector from the reduced list of candidate prediction vectors; When optimization is enabled: optimizing the selected prediction vector; By using the optimized prediction vector, encoding or decoding the current block to reconstruct the current block; When optimization is not enabled: The current block is encoded or decoded by using the selected prediction vector to reconstruct the current block.

11. A video decoding method, include: receiving data for a block of pixels of a current block of a current picture to be decoded as video; Generate a candidate prediction vector list for the current block; pruning the list of candidate prediction vectors by removing from the list first prediction vectors that are within a threshold of a second prediction vector, wherein the threshold has a first value when optimization is enabled and a second value when optimization is not enabled; selecting a candidate prediction vector from the reduced list of candidate prediction vectors; When optimization is enabled: optimizing the selected prediction vector; Reconstructing the current block by using the optimized prediction vector to generate a prediction block; When optimization is not enabled: The current block is reconstructed using the selected prediction vector to generate a prediction block.

12. A video encoding method, include: receiving data for a block of pixels of a current block of a current picture to be encoded as video; Generate a candidate prediction vector list for the current block; pruning the list of candidate prediction vectors by removing from the list first prediction vectors that are within a threshold of a second prediction vector, wherein the threshold has a first value when optimization is enabled and a second value when optimization is not enabled; selecting a candidate prediction vector from the reduced list of candidate prediction vectors; When optimization is enabled: Optimize the selected prediction vector; By using the optimized prediction vector, encoding the current block to generate a prediction block; When optimization is not enabled: The current block is encoded using the selected prediction vector to generate a prediction block.

Images