Methods and apparatus with similarity check in video coding system

Abstract

A method and apparatus for generating a more diverse merge list are disclosed. According to the method, input data related to the current block is received, wherein the input data includes pixel data to be encoded on the encoder side or encoded data related to the current block and to be decoded on the decoder side. A candidate to be included and a current intra-frame merge candidate list are determined. Whether to insert the candidate to be included into the current intra-frame merge candidate list to generate an updated intra-frame merge candidate list is determined by examining the similarity between the candidate to be included and existing candidates in the current intra-frame merge candidate list, wherein the similarity includes mode type similarity. The current block is encoded or decoded using encoding / decoding information including the updated intra-frame merge candidate list.

Description

[Technical Field]

[0001] This invention relates to merging modes in video encoding and decoding systems. In particular, this invention relates to increasing the diversity of candidates in a merge list by examining the similarity between candidates to be included and existing candidates in the merge list. [Previous Technology]

[0002] Versatile Video Coding (VVC) is the latest international video coding standard developed by the Video Coding Experts Group (VCEG) of the International Telecommunication Union-Telecommunication Standardization Sector (ITU-T) and the Joint Video Experts Team (JVET) of the Moving Picture Experts Group of the International Organization for Standardization / International Electrotechnical Commission (ISO / IEC). This standard has been published as an ISO standard: ISO / IEC 23090-3:2021, Information Technology – Encoded Representation of Immersive Media – Part 3: Versatile Video Coding, published in February 2021. VVC was developed based on its predecessor, HEVC (High Efficiency Video Coding), by adding more coding and decoding tools to improve coding and decoding efficiency and handle various types of video sources containing three-dimensional (3D) video signals.

[0003] Figure 1A illustrates an exemplary adaptive inter-frame / intra-frame video coding system incorporating loop processing. For intra-frame prediction, the prediction data is derived from previously encoded video data in the current frame. For inter-frame prediction 112, motion estimation (ME) is performed on the encoder side, and motion compensation (MC) is performed based on the results of ME to provide prediction data and motion data derived from other frames. Switch 114 selects either intra-frame prediction 110 or inter-frame prediction 112 and provides the selected prediction data to adder 116 to form a prediction error, also known as a residual. The prediction error is then processed by transform (T) 118, followed by quantization (Q) 120. The residuals from the transform and quantization are then encoded by entropy encoder 122 to be included in the video bitstream corresponding to the compressed video data. The bitstream associated with the transform coefficients is then packaged with side information, such as motion and encoding / decoding modes associated with intra-frame and inter-frame prediction, and parameters associated with loop filters applied to the underlying image regions. Side information related to intra-frame prediction 110, inter-frame prediction 112, and loop filter 130, as shown in Figure 1A, is provided to entropy encoder 122. When using inter-prediction mode, the reference picture or image must also be reconstructed at the encoder. Therefore, the residuals from transform and quantization are processed by inverse quantization (IQ) 124 and inverse transform (IT) 126 to recover the residuals. The residuals are then added back to the prediction data 136 to reconstruct the video data at reconstruction (REC) 128. The reconstructed video data can be stored in the reference picture buffer 134 and used for prediction in other frames.

[0004] As shown in Figure 1A, the incoming video data undergoes a series of processes in the encoding system. The reconstructed video data from REC 128 may be subject to various degradations due to these processes. Therefore, a loop filter 130 is typically applied to the reconstructed video data to improve video quality before it is stored in the reference picture buffer 134. For example, a deblocking filter (DF), sample adaptive offset (SAO), and adaptive loop filter (ALF) may be used. It may be necessary to incorporate loop filter information into the bitstream so that the decoder can correctly recover the required information. Therefore, loop filter information is also provided to the entropy encoder 122 for incorporation into the bitstream. In Figure 1A, the loop filter 130 is applied to the reconstructed video, and then the reconstructed samples are stored in the reference picture buffer 134. The system in Figure 1A is intended to illustrate an exemplary structure of a typical video encoder. It may correspond to a High Efficiency Video Codec (HEVC) system, VP8, VP9, ​​H.264, or VVC.

[0005] As shown in Figure 1B, the decoder can use some of the same functional blocks as the encoder. For example, the decoder can reuse inverse quantization 124 and inverse transform 126; however, transform 118 and quantization 120 are not required at the decoder. The decoder uses entropy decoder 140 instead of entropy encoder 122 to decode the movie bitstream into quantized transform coefficients and the required encoding / decoding information (e.g., ILPF information, intra-frame prediction information, and inter-frame prediction information). Intra-frame prediction 150 on the decoder side does not require mode search. Instead, the decoder only needs to generate intra-frame predictions based on the intra-frame prediction information received from entropy decoder 140. Furthermore, for inter-frame prediction, the decoder only needs to perform motion compensation (MC 152) based on the inter-frame prediction information received from entropy decoder 140, without motion estimation.

[0006] According to VVC, the input image is divided into non-overlapping square block regions called CTUs (Coding Tree Units), similar to HEVC. Each CTU can be divided into one or more smaller coding units (CUs). The resulting CU segmentation can be square or rectangular. In addition, VVC divides CTUs into prediction units (PUs) as units for applying prediction processes (such as inter-frame prediction, intra-frame prediction, etc.).

[0007] The VVC standard incorporates various new encoding and decoding tools to further improve the encoding and decoding efficiency of the HEVC standard. The following is a review of some tools related to this invention.

[0008] 1.1 Intra-frame mode encoding and decoding, with 67 intra-frame prediction modes.

[0009] In VVC, several traditional angle intra-prediction modes are adaptively replaced with wide-angle intra-prediction modes for non-square blocks.

[0010] 1.2 Intra-frame mode encoding and decoding

[0011] The Most Probable Mode (MPM) list is generated by considering two available neighboring intra-frame modes.

[0012] 1.3 Decoder-side Intra Mode Derivation (DIMD)

[0013] When DIMD is applied, two intra-frame modes are derived from the reconstructed neighbor samples (templates). These two predictors are combined with the planar mode predictor, and the weights are derived from the gradient.

[0014] Texture gradient analysis is performed on both the encoder and decoder sides. This process begins with an empty Histogram of Gradients (HoG) with 65 entries, corresponding to 65 angular patterns. The magnitudes of these entries are determined during texture gradient analysis.

[0015] For more details, please refer to JVET-O0449 “Non-CE3: Decoder-side Intra Mode Derivation (DIMD) with prediction fusion using Planar” and / or “Algorithm description of enhanced compression model” (e.g., JVET-AG2025).

[0016] Figures 2A-2C show an example of the HoG calculated after applying the above operations at all pixel positions of the template. Figure 2A shows an example of a template 220 for a selection of a current block 210. Template 220 includes T rows above the current block and T columns to the left of the current block. For intra-frame prediction of the current block, the areas 230 above and to the left correspond to a reconstructed region, while the areas 240 below and to the right of the block correspond to an unusable region. Figure 2B shows an example with T=3, and the HoG is calculated for pixels 260 in the middle row and pixels 262 in the middle column. For example, for pixel 252, a 3x3 window 250 is used. Figure 2C shows an example of the amplitude (ampl) calculated for the angular intra-frame prediction mode (i.e., HoG270).

[0017] 1.4 Template-based Intra Mode Derivation (TIMD)

[0018] Template-based intra-frame mode derivation (TIMD) implicitly derives the intra-frame prediction mode of the CU at both ends of the encoder and decoder using neighboring templates, rather than providing the exact intra-frame prediction mode bits to the decoder signal. Predicted samples of the template are generated for each candidate mode using reference samples of the template. The cost is calculated as the SATD between the predicted and reconstructed samples of the template. The two intra-frame prediction modes with the minimum SATD are selected as TIMD modes. These two TIMD modes are fused with weights to generate the prediction for the current CU.

[0019] For more details, please refer to the "Algorithm Description of Enhanced Compression Model" (e.g., JVET-AG2025).

[0020] 1.5 Extrapolation filter-based Intra Prediction (EIP) mode

[0021] In EIP mode, samples are predicted from the upper left to the lower right position of the CU by applying an extrapolation filter to neighboring reconstructed or predicted samples. EIP mode uses a 15-tap filter for prediction as follows: where is the predicted value at position (x, y) in the CU, is the filter coefficient, and is the reconstructed or predicted sample.

[0022] The EIP filter can be derived from neighboring reconstructed samples or inherited from previous EIP coding blocks. ECM supports three EIP filter shapes and three reconstruction region types, as shown in Figures 3 and 4. In Figure 3, the three filter shapes correspond to square 310, horizontal bar 320, and vertical bar 330. In Figure 4, the three reconstruction regions correspond to the upper left region (Figure 4A), the upper region (Figure 4B), and the left side region (Figure 4C).

[0023] For an EIP-coded CU, an EIP merging flag is signaled to indicate whether the EIP filter is inherited from a previous EIP-coded block. When the EIP merging flag is true, an EIP merging list is constructed from spatially adjacent, spatially non-adjacent, temporal, and historical candidates. The positions and inclusion order of these candidates are the same as in the CCP merging mode. An EIP merging index is further signaled to indicate which EIP merging candidate was selected.

[0024] When the EIP merging flag is false, the EIP filter is derived from neighboring reconstructed samples, and the associated syntax elements are signaled to indicate which of the three reconstructed region types and which of the three filter shapes is used for the CU. The selected filter moves horizontally or vertically within the selected reconstructed region, one pixel step at a time, to construct the autocorrelation matrix and cross-correlation vector. The method for calculating the coefficients from the autocorrelation matrix and cross-correlation vector is the same as in CCCM.

[0025] 1.6 Template-based multi-reference line intra-frame prediction

[0026] The Template-Based Multiple Reference Line Intra Prediction (TMRL) mode combines reference lines and prediction modes, and uses a template matching method to construct a candidate combination list. An index to the candidate combination list is signaled.

[0027] The extended reference lines begin with reference line 1. Reference line 0 is used for template matching. The SAD cost (TMRL cost) between the prediction (generated from 50 combinations) and reconstruction is calculated over the template region (see Figure 5). The 20 combinations with the lowest SAD cost are selected in ascending order to form a TMRL candidate list.

[0028] 1.7 Intra-frame prediction using merged gradient histograms

[0029] JVET-AG0078 proposes adding a new intra-prediction mode called Merged Intra Mode Derivation (MIMD), based on the calculation of the Merged Histogram of Gradients (MHoG). Similar to DIMD, up to five MIMD modes are derived from MHoG and then mixed together. The derivation process of modes and mixing weights is the same as that of deriving DIMD modes and mixing weights from HoG. However, unlike DIMD, MHoG is not calculated by directly analyzing template samples, but is based on information extracted from neighboring blocks.

[0030] In particular, N neighboring blocks are considered. A neighboring block is considered if it is encoded using at least one directional intra-prediction mode. refers to the amplitude of directional mode m in the HoG, where m can take values ​​from 0 to , and is the maximum number of intra-prediction modes. A normalization process can be used when considering this.

[0031] Then, the MHoG can be calculated using all HoGs extracted from available neighboring blocks, as follows:

[0032] Finally, MHoG is used to calculate the MIMD modes and weights. The directional modes corresponding to the five highest amplitudes in the MHoG and their weights are selected as the directional modes and weights of the MIMD. More details can be found in JVET-AG0078.

[0033] 1.8 Occurrence-Based Intra Coding (OBIC)

[0034] On-Bound Intra-Encoding / Decoding (OBIC) derives the intra-prediction mode of the current block based on the sample-level occurrence of intra-modes in the spatial neighborhood of the block. To do this, adjacent and non-adjacent spatial neighbor blocks are examined, and the intra-prediction modes of the blocks are collected into an occurrence histogram. Unlike the gradient histogram (HoG) in DIMD, the OBIC method uses an occurrence histogram (HoC), which consists of intra-modes and their sample-level occurrence counts. The occurrence value is calculated based on the number of samples encoded / decoded in that neighborhood using a certain intra-prediction mode.

[0035] Select the five most frequent angular patterns from HoC, as well as planar patterns or block vector-based predictions (same as DIMD), and use the predictions of the selected patterns for the final prediction.

[0036] Some of the blocks mentioned below use more than one intra-frame mode for prediction. In this case, when creating the OBIC histogram, all intra-frame modes of these blocks are selected and used: ● DIMD: up to 5 angle modes ● TIMD: up to 2 modes ● SGPM: 2 modes ● OBIC: up to 5 angle modes

[0037] In addition, when creating the histogram of the OBIC mode, the following blocks are considered in the interleaved slices as virtual intra prediction mode (VIPM): ● MIP (Matrix-Based Intra Prediction) block ● Intra TMP block ● IBC block ● EIP block.

[0038] The calculation of weights for the hybrid weight and planar mode is similar to that of the DIMD mode, but OBIC uses the generated values ​​instead of gradient values ​​from the template. More details can be found in JVET-AH0076.

[0039] In this invention, a method and apparatus for increasing the diversity of merged lists are disclosed. [Summary of the Invention]

[0040] A method and apparatus for generating a more diverse merge list are disclosed. According to the method, input data related to a current block is received, wherein the input data includes pixel data to be encoded at the encoder side or encoded data related to the current block and to be decoded at the decoder side. A candidate to be included and a current intra-frame merge candidate list are determined. Whether to insert the candidate to be included into the current intra-frame merge candidate list to generate an updated intra-frame merge candidate list is determined by checking the similarity between the candidate to be included and existing candidates in the current intra-frame merge candidate list, wherein the similarity includes mode type similarity. The current block is encoded or decoded using encoding / decoding information including the updated intra-frame merge candidate list.

[0041] In one embodiment, similarity checking is also applied when constructing a history-based buffer array, wherein the similarity is checked between the historical candidate to be included and the existing historical candidates in the history-based buffer array.

[0042] In one embodiment, the pattern type similarity is determined when the candidate to be included and the existing candidate in the current intra-frame merge candidate list have the same pattern type.

[0043] In one embodiment, when the candidate to be included or the existing candidate in the current intra-frame merging candidate list is an OBIC candidate or a MIMD candidate, the candidate to be included or the existing candidate is considered to have a DIMD mode type with similarity to the mode type in the current intra-frame merging candidate list.

[0044] In one embodiment, if the current intra-frame merging candidate list already contains more than a threshold number of candidates of the same mode type as the candidate to be included, then the candidate to be included is not included in the current intra-frame merging candidate list. In one embodiment, the threshold is 0 or any positive integer.

[0045] In one embodiment, the mode type similarity is checked to determine whether the candidate to be included and existing candidates in the current intra-merge candidate list belong to the same mode type group. In one embodiment, a mode type group is defined as one or more intra-prediction methods used by the candidate to be included and existing candidates in the current intra-merge candidate list. In one embodiment, a mode type group is defined as one or more learning-based and / or angle-based intra-prediction methods used by the candidate to be included and existing candidates in the current intra-merge candidate list.

[0046] In one embodiment, the similarity further includes intra-frame prediction mode similarity, prediction fusion weight similarity, block vector (BV) usage information similarity, or a combination thereof.

[0047] In one embodiment, the intra-prediction mode similarity is checked to determine whether the intra-prediction modes of the candidate to be included and the existing candidates in the current intra-merge candidate list are the same. In one embodiment, only the intra-prediction mode similarity of the predefined intra-prediction modes of the candidate to be included and the existing candidates in the current intra-merge candidate list is checked. In one embodiment, the intra-prediction mode similarity check is applied only when the candidate to be included and the existing candidates in the current intra-merge candidate list belong to the same mode type group or the same mode type.

[0048] In one embodiment, the intra-prediction mode similarity is checked to determine whether the internal mode difference between the candidate to be included and the existing candidates in the current intra-merge candidate list is less than a predefined threshold. In one embodiment, for the intra-prediction mode similarity check, the domain and / or granularity of the intra-prediction modes of the candidate to be included and the existing candidates in the current intra-merge candidate list are aligned.

[0049] In one embodiment, if the first number of intra-prediction modes are different from the second number of intra-prediction modes, the candidate to be included and the existing candidate are determined to be dissimilar candidates in the current intra-merge candidate list.

[0050] In one embodiment, intra-prediction mode similarity is checked to determine whether the intra-prediction modes of the candidate to be included and existing candidates in the current intra-merging candidate list are the same. In one embodiment, the intra-prediction mode similarity check applies to all or part of the intra-prediction modes of the candidate to be included and existing candidates in the current intra-merging candidate list. In one embodiment, the intra-prediction mode similarity check applies to the first N intra-prediction modes of the candidate to be included and existing candidates in the current intra-merging candidate list. In one embodiment, the mode types of both the candidate to be included and existing candidates in the current intra-merging candidate list are extrapolation filter-based intra-prediction (EIP) or matrix-based intra-prediction (MIP), and the intra-prediction mode similarity check corresponds to checking whether the candidate to be included and existing candidates in the current intra-merging candidate list have the same EIP filter shape or the same MIP matrix index, respectively.

[0051] In one embodiment, the fusion weight similarity check is used to determine whether the fusion weights of the candidate to be included and the existing candidates in the current intra-frame merging candidate list are the same for all or part of the intra-frame modes of the candidate to be included and the existing candidates in the current intra-frame merging candidate list. In one embodiment, the fusion weight similarity check is performed on the fusion weights between the corresponding intra-frame prediction modes of the candidate to be included and the existing candidates in the current intra-frame merging candidate list. In one embodiment, the fusion weight similarity check is performed with low precision. In one embodiment, if the mode type of both the candidate to be included and the existing candidates in the current intra-frame merging candidate list is EIP mode, then the fusion weight similarity check is performed to determine whether all or part of the EIP model parameters of the candidate to be included and the existing candidates in the current intra-frame merging candidate list are completely identical.

[0052] In one embodiment, BV uses information similarity checking to determine whether the candidate to be included and the existing candidates in the current frame's merge candidate list both use BV to generate predictions. In one embodiment, when both the candidate to be included and the existing candidates in the current frame's merge candidate list use BV to generate predictions, BV uses information similarity checking is further performed to determine whether the candidate to be included and the existing candidates in the current frame's merge candidate list use the same BV. In one embodiment, BV uses information similarity checking with low precision.

[0053] In one embodiment, after checking the similarity between the candidate to be included and the existing candidates in the current frame merge candidate list, if no similar candidate is found, the candidate to be included is added to the current frame merge candidate list.

Implementation Method

[0055] The components of the present invention, as generally described and depicted in the figures herein, can be arranged and designed in various different configurations. Therefore, the following more detailed description of embodiments of the systems and methods of the present invention, as shown in the figures, is not intended to limit the scope of the invention, as requested, but merely represents selected embodiments of the invention. References to “one embodiment,” “an embodiment,” or similar language in this specification mean that a particular function, structure, or feature associated with an embodiment may be included in at least one embodiment of the invention. Therefore, the phrases “in one embodiment” or “in a kind of embodiment” appearing throughout this specification do not necessarily refer to the same embodiment.

[0056] Furthermore, the described functions, structures, or features can be combined in one or more embodiments in any suitable manner. However, those skilled in the art will recognize that the invention can be practiced without using one or more specific details, or using other methods, components, etc. In other instances, well-known structures or operations are not shown or described in detail to avoid obscuring aspects of the invention. Embodiments of the invention can be best understood by referring to the drawings, in which like parts are designated by like numbers. The following description is by way of example only, illustrating only apparatus and methods of certain selected embodiments consistent with the invention stated herein.

[0057] In this disclosure, an intra-block merging scheme is proposed to improve intra-block prediction using a merging scheme. This intra-block merging mode can also be called a decoder-derived intra-prediction (DIP) merging mode. The concept of the merging scheme is to inherit (or reference) information from previous codec blocks and use the inherited information to predict the current block. The main differences between the proposed intra-block merging scheme and conventional intra-block MPMs are as follows. For intra-blocks, conventional intra-block MPMs can only inherit intra-prediction modes marked in previous codec blocks. For example, the mode index of the intra-prediction mode points to the angular prediction mode, DC, or plane from the upper neighboring block and / or the left neighboring block. The proposed intra-block merging scheme brings a more flexible inheritance process and / or more efficient inheritance information in the following aspects.

[0058] In a first aspect, in certain embodiments specified in the “Inheritance Information Settings” section, the inheritance information may be any mode information, any sample information, any block information, any model information and / or any information related to prediction generation, depending on the proposed intra-frame merging scheme.

[0059] In a second aspect, in some embodiments specified in the “Inheritance Block Settings” section, the current block can find the previous coded block containing inheritance information by several proposed merging methods, according to the proposed intra-frame merging scheme.

[0060] In a third aspect, in the "Target Mode Settings and Shortcut Syntax Settings" section, for some embodiments, the proposed merging scheme uses the target mode to obtain information related to the prediction generation of the current block. For some embodiments, the proposed method is enabled or disabled according to the shortcut syntax settings.

[0061] When encoding the current block using the target pattern to generate a prediction, one or more inheriting blocks are selected from a list of merge candidates or several merge candidates (e.g., a set of merge candidates). Then, the inheritance information of the inheriting blocks is used to generate a prediction for the target pattern.

[0062] In some embodiments, the inheritance information of each merge candidate may belong to a different target mode, i.e., a hybrid mode. For example, the inheritance information of a merge candidate may be associated with DIMD, TIMD, MRL, MIP, EIP, or variants thereof. A merge list may contain merge candidates with different target modes (hybrid modes). Depending on which merge candidate is used, the current block can be predicted based on DIMD, TIMD, MRL, MIP, EIP, or variants thereof.

[0063] In one sub-implementation, a merge candidate list or set is established only for the current block. For example, the generation of intra-frame luminance prediction for the current block using a unified list or set and / or the target mode applied to the current block depends on the candidates selected from the list or set. In one case, the similarity check can be different for the candidate lists. The similarity check of the lists can vary with the mode type of the candidates or any implicit rules, such as the block width, block height, and / or block area of ​​the current block or the candidate's inheritable block. In another case, the similarity check can be different for the candidate lists. The similarity check of the lists can vary with explicit syntax elements.

[0064] In one sub-implementation, at least one or at least two candidate lists or sets for merging are established for the current block. For example, for each mode category, a candidate list or set containing one or more candidates belonging to the same mode category is established. In one case, the similarity check may differ for different candidate lists. The similarity check of the list may vary with the mode category of the list or any implicit rules, such as the block width, block height, and / or block area of ​​the current block or the candidate's block to be inherited. For example, for a list of mode categories related to DIMD and / or TIMD-related modes, the similarity check refers to intra-frame mode checking and / or fusion weight checking, for example, the methods in Section 2.2.2 "Similarity Check of Candidates to be Included" (which can be applied to candidates of mode types related to DIMD / TIMD). For example, for a list of mode categories related to EIP and / or MIP-related modes, the similarity check refers to intra-frame mode checking and / or fusion weight checking, for example, the methods specified in Section 2.2.2 "Similarity Check of Candidates to be Included" (which can be applied to candidates of mode types related to EIP / MIP). In another scenario, the similarity check can differ for different candidate lists. The similarity check of lists can vary depending on the explicit syntax element.

[0065] In one embodiment, the maximum number of candidates allowed in the merged candidate list is limited.

[0066] In one sub-implementation, the maximum number is 12.

[0067] 2.1 Inheritance Information Settings

[0068] This section defines the inheritance information for generating the prediction of the current block using the target pattern.

[0069] 2.1.1 DIMD Inheritance Information Settings

[0070] For DIMD mode, inherited information includes list items (a), (b), (c), (d) or a combination thereof. (a) Histogram (bar) values ​​of one or more available DIMD intra-prediction modes (e.g., DC, planar and / or directional prediction modes), (b) N intra-prediction modes suggested by the histogram values ​​(with the highest N histogram bars), (c) DIMD weighting information and / or whether fusion is used, (d) reference line information and / or wide-angle conditions.

[0071] 2.1.2 TIMD Inheritance Information Settings

[0072] For TIMD modes, the inherited information includes list items (a), (b), (c), (d) or a combination thereof. (a) TIMD cost values ​​of one or more available TIMD intra-prediction modes (e.g., DC, planar, and / or directional prediction modes), (b) N intra-prediction modes suggested by TIMD costs (with a minimum of N TIMD costs), (c) TIMD weighting information and / or whether fusion is used, and (d) reference line information and / or wide-angle conditions.

[0073] 2.1.3 MRL Inheritance Information Settings

[0074] For MRL mode, the inherited information includes list items (a), (b), (c), (d) or a combination thereof. (a) One or more reference lines common to the intra-prediction mode (e.g., DC, planar and / or directional prediction modes), (b) N intra-prediction modes recommended by TIMD cost (with a minimum of N TIMD costs), (c) weighting information and / or whether fusion is used, (d) reference line information and / or wide-angle conditions.

[0075] 2.1.4 EIP Inheritance Information Settings

[0076] For EIP mode, the inherited information includes list items (a), (b), (c) or a combination thereof. (a) Filter shape, (b) All or some of the filter coefficients, (c) Template used to derive the filter coefficients.

[0077] 2.1.5 Inheritance Information Settings for Intra-Frame Merging Mode

[0078] When the target mode is intra-frame merging mode, it can also be called decoder-derived intra-frame prediction (DIP) merging mode, and the inherited information includes all or any subset of the following items. Item 1: (a), (b), (c), (d) or a combination thereof in Section 2.1.1 (DIMD Inheritance Information Settings) Item 2: (a), (b), (c), (d) or a combination thereof in Section 2.1.2 (TIMD Inheritance Information Settings) Item 3: (a), (b), (c), (d) or a combination thereof in Section 2.1.3 (MRL Inheritance Information Settings) Item 4: Codec mode (e.g., DIMD-related or unrelated modes, TIMD-related or unrelated modes, MRL-related or unrelated modes, SGPM-related or unrelated modes, ISP-related or unrelated modes, intra-frame TMP-related or unrelated modes, MPM-related or unrelated modes, MIP-related or unrelated modes, EIP-related or unrelated modes, and / or any modes related to predefined intra-frame modes) and / or corresponding information for the codec mode. Item 5: (a), (b), (c) or a combination thereof in Section 2.1.4 (EIP Inheritance Information Settings)

[0079] In one embodiment, for items 1, 2, 3, and 5, the inheritance information can be obtained using the embodiments in Sections 2.1.1, 2.1.2, 2.1.3, 2.1.4, or combinations thereof.

[0080] In another embodiment, item 4 is included in the inheritance information, and / or the encoding / decoding mode in item 4 determines the corresponding information to be further included in the DIP merge inheritance information. For example, the encoding / decoding mode in item 4 is an MRL-related mode, and only the corresponding information from item 3 is included in the DIP merge inheritance information.

[0081] In one embodiment, the encoding / decoding mode in item 4 is derived from a previous encoding block specified in Section 2.2, “Inheritance Block Setting.” The previous encoding block may store hybrid mode inheritance information. The stored hybrid mode inheritance information of the previous encoding block can be referenced by subsequent encoding / decoding blocks (e.g., the current block). For example, for each block containing hybrid mode information, the hybrid mode information is stored and / or referenced by subsequent encoding / decoding blocks. For example, for each predefined cell containing hybrid mode information, the hybrid mode information is stored and / or referenced by subsequent encoding / decoding blocks. This cell can be any predefined region or kxk grid, where k can be 2, 4, 8, 16, or any predefined positive integer.

[0082] In one sub-implementation, the encoding / decoding mode is EIP. The previous coded block generates corresponding EIP inheritance information (e.g., filter shape and / or filter coefficients) and / or stores DIP merge inheritance information (encoding / decoding mode and / or corresponding information).

[0083] In one sub-implementation, the encoding / decoding mode is SGPM (Spatial Geometric Partitioning Mode). This mode generates multiple prediction hypotheses from different intra-frame prediction modes and combines these prediction hypotheses using weights based on geometric partition lines to form a final prediction. Previous coded blocks generate corresponding SGPM inheritance information (e.g., intra-frame prediction modes and / or partition lines and / or weights) and / or store DIP merge inheritance information (encoding / decoding modes and / or corresponding information).

[0084] In one sub-implementation, the encoding / decoding mode is Intra Sub-Partition (ISP). This mode divides the current block into multiple sub-partitions and uses reconstructed samples adjacent to the current sub-partition (possibly from the previous sub-partition) as reference samples to generate predictions for each sub-partition. Previously encoded blocks generate corresponding ISP inheritance information (e.g., one or more intra-prediction modes and / or segmentation methods) and / or store DIP merge inheritance information (encoding / decoding modes and / or corresponding information).

[0085] In one sub-implementation, the encoding / decoding mode is intra-TMP (intra-temporal matching prediction). This mode uses template matching to find a displacement (i.e., block vector) to reference a reference block within a predetermined range in the current image, and generates predictions using reconstructed samples of the reference block. Previously encoded blocks generate corresponding intra-TMP inheritance information (e.g., one or more block vectors) and / or store DIP merge inheritance information (encoding / decoding mode and / or corresponding information).

[0086] In another embodiment, the encoding / decoding mode is MIP (Matrix-Based Intra Prediction). This mode uses predefined matrices and predefined neighboring or non-neighboring reference samples to generate predictions. Previously encoded blocks generate corresponding MIP inheritance information (e.g., one or more matrix indicators and / or one or more matrix coefficients of the indicated matrix, block width, block height, block area, block position, or combinations thereof) and / or store DIP merge inheritance information (encoding / decoding mode and / or corresponding information).

[0087] In another embodiment, to reduce storage, instead of storing all DIP merge information (possibly for reference), only a subset of the DIP merge information is stored. For example, only the top 3 items or any predefined positive number are stored. For example, the subset is the first 3 items. The first 3 always include item 4.

[0088] In one embodiment, the inheritance information of the target mode depends on the inheritance codec mode value of the selected merge candidate.

[0089] In one sub-implementation, if the selected merge candidate codec mode is DIMD, TIMD, EIP or MRL, the inherited codec mode value is the same as the codec mode (i.e., DIMD, TIMD, EIP or MRL respectively).

[0090] In this application, “encoding / decoding mode” and “mode type” are used interchangeably in some paragraphs, but they have the same meaning.

[0091] In one sub-implementation, for a block encoded in DIP merge mode, the inheritance mode type value is set as follows: when the current block is encoded in DIP merge mode, the block will select a merge candidate to generate a prediction; if the selected merge candidate of the current block is encoded in DIMD, TIMD, MRL or EIP, the inheritance mode type value of the current block is set to DIMD, TIMD, MRL or EIP respectively; if the selected merge candidate of the current block is encoded in DIP merge mode, the inheritance mode type value of the current block is set to the inheritance mode type value of the selected merge candidate.

[0092] In one sub-implementation, if the inheritance mode type is DIMD, the inheritance information settings follow the description in Section 2.1.1.

[0093] In one sub-implementation, if the inheritance mode type is TIMD, the inheritance information settings follow the description in Section 2.1.2.

[0094] In one sub-implementation, if the inheritance mode type is MRL, the inheritance information settings follow the description in Section 2.1.3.

[0095] In one sub-implementation, if the inheritance mode type is EIP, the inheritance information settings follow the description in Section 2.1.4.

[0096] 2.2 Inheritance Block Settings

[0097] 2.2.1 Inheritance information obtained from previous coding blocks

[0098] Inherited information is obtained from previously encoded blocks.

[0099] In one embodiment, one or more spatially adjacent candidates and / or non-adjacent candidates, historical candidates, time candidates, default candidates, or any subset of the above candidates provide inheritance information from previous coded blocks.

[0100] In one embodiment, a merge candidate list containing inheritance information is established for the current block. Similar to the interactive merge mode candidate list, the merge candidate list includes spatially adjacent candidates and / or non-adjacent candidates, historical candidates, time-based candidates, default candidates, or any subset of the above candidates.

[0101] In one sub-implementation, after establishing a list of merge candidates, one or more candidates are selected from the list for the current block. Selection depends on one or more (promising) candidates that are explicitly indicated by a signaling index or implicit selection. For example, the first or more candidates with the lowest cost are selected. For example, an explicit index indicates that one or more candidates in the list are reordered as selected candidates. Cost calculation and / or list reordering may depend on a template matching process that calculates the cost based on the distortion between the prediction on the template (using a certain candidate) and the reconstruction on the template. Candidates in the list are reordered in ascending order based on cost.

[0102] 2.2.1.1 Spatial Adjacent Candidates and Non-Adjacent Candidates

[0103] Spatial adjacency candidates come from the adjacent neighboring blocks of the current block. The adjacent neighboring blocks can be the same as the five spatial neighboring blocks in the interactive merge mode, or any subset of the adjacent neighboring blocks of the current block. For example, to add spatial adjacency candidates to the merge list, as shown in Figure 6, the inclusion order can be A1 → B1 → A0 → B0 → B2 or B1 → A1 → B0 → A0 → B2. Non-adjacency candidates come from the search range surrounding (but not adjacent to) the current block. The search range can be the same as the search range for non-adjacency candidates in the interactive merge mode. Non-adjacency candidates can come from predefined locations and be added to the merge list in a predefined inclusion order. For example, the predefined locations and inclusion order are the same as for non-adjacency candidates in the interactive merge mode.

[0104] 2.2.1.2 History candidate

[0105] Historical candidates are selected from a history-based buffer array. In the history-based buffer array, inheritance information for each valid previously encoded block is stored, where a valid previously encoded block refers to any block containing inheritance information.

[0106] 2.2.1.3 Time Candidates

[0107] The time candidate is obtained from inherited information stored in one or more previously encoded images. The time candidate can be obtained when the current slice / image is a non-intra-frame slice / image.

[0108] In one embodiment, time candidates may come from blocks of previously encoded slices / images at some predefined locations.

[0109] In one sub-implementation, the location is within the corresponding region of the current coding block.

[0110] In one sub-implementation, the predefined location is located outside the corresponding region of the current coding block.

[0111] In one sub-implementation, the predefined position can be determined based on the position, width, and height of the current block.

[0112] In one sub-implementation, the predefined location can be determined based on the location and some predefined fixed xy distance.

[0113] In one embodiment, the previously encoded image is one of the images in the reference list.

[0114] In one embodiment, the previously encoded image is the same as the co-image of the regular interactive merging mode.

[0115] In one embodiment, there may be more than one selected / predefined previously encoded image.

[0116] In one embodiment, the previously encoded image may be marked in the image / slice header. Reference lists and reference indices are marked in the image / slice header. For example, L0[0] is marked. Another example is L1[0] being marked.

[0117] In one embodiment, the previously encoded image is selected from an image set with some predefined rules.

[0118] In one embodiment, the previously encoded image is selected from images in a reference list. Selection may be determined based on POC, and / or POC distance, and / or QP, and / or QP difference. Another example is selecting the image with the smaller POC. Another example is selecting the image with the larger POC. For example, selecting the image with the smallest POC distance from the current image. Another example is selecting the image with the smaller QP. Another example is selecting the image with the larger QP. Another example is selecting the image with the smallest QP difference from the current image.

[0119] In one embodiment, the previously encoded image is the most recently encoded I image.

[0120] In one embodiment, the rules for selecting / not selecting previously encoded images described in the preceding paragraphs can be used in combination. For example, the image with the smallest QP among the unscaled images in the reference list can be selected.

[0121] 2.2.1.4 Preset Candidates

[0122] Preset candidates are candidates that include preset information, and / or the preset information is derived from candidates already included in the merged candidate list. In one embodiment, the preset mode for the preset candidates can be derived using the MIMD scheme and / or the OBIC scheme and / or the EIP derived mode scheme and / or any existing or mentioned intra-frame scheme (e.g., TIMD, DIMD, EIP and / or MRL) and / or any variant of the existing or mentioned intra-frame scheme.

[0123] 2.2.2 Similarity check for candidates

[0124] In order to increase the diversity of candidates in the merge list, when adding a candidate to be included to the current merge list, a similarity check can be applied to compare the candidate to be included with the existing candidates in the merge list.

[0125] In one embodiment, similarity checking may include, but is not limited to, checking the similarity of mode types, intra-frame prediction modes, prediction fusion weights, EIP filter shape similarity, MIP matrix index similarity and / or BV usage information.

[0126] In one sub-implementation, when checking the similarity of pattern types, the equality of the pattern type of the candidate to be included and the pattern type of the candidate to be checked in the merge list is evaluated. For example, if the pattern type of the candidate to be included is DIMD and the pattern type of the candidate to be checked is also DIMD, they are similar candidates. For an example of the pattern types of an encoding / decoding tool, an OBIC candidate and / or a MIMD candidate belong to the DIMD pattern type. For another example of the similarity check in this embodiment, if the merge list already contains a number (the same as the pattern type of the candidate to be included) greater than a threshold, the similarity check will indicate that the candidate to be included is not included in the merge list. The threshold can be 0 or any positive integer.

[0127] In another sub-implementation, the mode type check may be to check whether the mode type of the candidate to be included and the mode type of the candidate to be checked in the merge list belong to the same mode type group. A mode type group is defined as one or more intra-prediction methods used by the candidate to be included and existing candidates in the current merge candidate list. For example, a group may be defined as candidates that use one or more intra-prediction methods (e.g., DIMD, TIMD, MRL, and TMRL) to generate the final intra-prediction. Another group may be defined as candidates that use a learning / non-angle-based method (e.g., EIP or MIP) to generate the final intra-prediction. As an example of the similarity check in this embodiment, if the merge list already contains a number (the same as the group of candidates to be included) greater than a threshold, the similarity check indicates that the candidate to be included is not to be included in the merge list. The threshold may be 0 or any positive integer.

[0128] In one sub-implementation, the intra-prediction mode check may be to check whether the intra-mode of the candidate to be included is the same as the intra-mode of the candidate to be checked in the merge list. For example, the intra-mode check is applicable to both a predefined intra-mode of the candidate to be included and a predefined intra-mode of the candidate to be checked. The predefined intra-mode may be the first intra-mode of the candidate. Another example, this embodiment is used only when the candidate to be included belongs to the same mode type group (or the same mode type) as the candidate to be checked. Another example, this embodiment can be used when the candidate to be included belongs to the same or different mode type group (or the same mode type) as the candidate to be checked.

[0129] In another sub-implementation, the intra-prediction mode check may be to check whether the difference between the intra-mode of the candidate to be included and the intra-mode of the candidate to be checked in the merge list is greater than a predefined threshold N. For example, if N is set to 3, the intra-mode of the candidate to be included is 50, and the intra-mode of the candidate to be checked is 52, they are considered similar candidates.

[0130] In one embodiment, when checking intra-prediction mode similarity, the domains / granularities of the intra-prediction modes should be aligned. For example, if the candidate mode type to be included is TIMD, the intra-prediction mode comparison used in the intra-prediction mode check should be performed in domain 131.

[0131] In one sub-implementation, the aforementioned intra-prediction mode check can be applied to all or part of the intra-modes of the candidates to be included and all or part of the intra-modes of the candidates to be checked in the merge list. For example, if all the checked intra-modes are similar modes, they are considered similar candidates. Otherwise, they are not similar candidates.

[0132] Another example: Suppose there are N intra-prediction modes in the candidate to be included, and M intra-prediction modes in the candidate to be checked in the merge list, where M is not equal to N. If N > M, and the M intra-prediction modes in the candidate to be checked are a subset of the N intra-prediction modes in the candidate to be included, then they are considered similar candidates. Similarly, if N < M, and the N intra-prediction modes in the candidate to be included are a subset of the M intra-prediction modes in the candidate to be checked, then they are considered similar candidates.

[0133] In another sub-implementation, the aforementioned intra-prediction mode check can be applied to the first N intra-modes to be included as candidates and the first N intra-modes to be checked in the merging list, where N is a positive integer value. If the first N intra-modes are similar modes, they are considered similar candidates. Otherwise, they are not similar candidates.

[0134] In another sub-implementation, the aforementioned intra-prediction mode check can be applied to the N intra-modes to be included as candidates with a first maximum weight and the N intra-modes to be checked as candidates with a first maximum weight in the merging list. If the N intra-modes are similar modes, they are considered similar candidates. Otherwise, they are not similar candidates.

[0135] In one embodiment, if the number of intra-mode candidates to be included is different from the number of intra-mode candidates to be checked in the merge list, they are not similar candidates.

[0136] In another embodiment, if the number of intra-mode candidates to be included is different from the number of intra-mode candidates to be checked in the merge list, the similarity of only N intra-modes should be checked, where N is less than or equal to the smaller number of intra-modes among the two candidates.

[0137] 2.3 Target Mode Settings and Shortcut Syntax Settings

[0138] 2.3.1 Target Mode Setting for DIMD (DIMD Merge Mode)

[0139] This section specifies a DIMD merging mode. When using the DIMD merging mode, one or more predefined candidate DIMD inheritance information are used, depending on the inherited block settings, to determine the prediction information required to generate the prediction for the current block. For example, inheritance information refers to histogram values ​​from previous coded blocks and is used to determine one or more intra-prediction modes (a type of prediction information) and / or the corresponding weights for the current block as regular DIMD. Then, in unification with regular DIMD, the prediction assumptions from each derived intra-prediction mode are combined using a mixing process to form the final prediction for the current block.

[0140] 2.3.2 Target Mode Setting for TIMD (TIMD Merge Mode)

[0141] This section specifies a TIMD merging mode. When using the TIMD merging mode, the prediction information required to generate the prediction for the current block is determined based on the inherited block settings, using TIMD inheritance information from one or more predefined candidate blocks (e.g., each candidate provides N intra-prediction modes for available TIMD intra-prediction modes and TIMD weight information, such as {IPM1, IPM2, W1, W2}, and / or merging or non-merging, and wide-angle conditions). For example, the merge candidate list is established and reordered based on the cost during the template matching process. Template size and template cost calculations may be consistent with regular TIMD. After reordering the TIMD merge candidates, the top 2 or any predefined number of candidates are retained from the list for signaling. Then, consistent with regular TIMD, the prediction assumptions from each inherited intra-prediction mode are combined using a mixing process to form the final prediction for the current block.

[0142] 2.3.3 Target Mode Setting for MRL (MRL Merge Mode)

[0143] This section specifies the MRL merging mode. When using the MRL merging mode, inherited information from one or more predefined candidates (e.g., each candidate provides one or more reference lines with intra-prediction modes (IPM1 and MRL1) and / or merge or not merge, and wide-angle conditions) is used to determine the prediction information required to generate the prediction for the current block, depending on the inherited block settings. For example, the merge candidate list is built and reordered based on the cost in the template matching process. Template size and template cost calculations can be unified with regular TMRL. After reordering the MRL merge candidates, the first two or any predefined number of candidates in the list are reserved for signaling. Then, unified with regular intra-prediction, such as inherited IPM1 and MRL1, the predictions used from the inherited intra-prediction modes and inherited reference lines are used to form the final prediction for the current block. For the examples of inherited IPM1 with MRL1 and IPM2 with MRL2, the prediction assumptions of the inherited intra-prediction modes and the corresponding inherited reference lines are combined through a mixing process to form the final prediction for the current block. For the examples of inheriting IPM1 with MRL1 / W1 and IPM2 with MRL2 / W2, the prediction assumptions of the inherited intra-prediction mode and the corresponding inherited reference line are combined through a mixing process with W1 and W2 to form the final prediction of the current block.

[0144] 2.3.4 Target Mode Settings for Hybrid Mode

[0145] This section specifies the hybrid mode merging mode. When using the hybrid mode merging mode, inherited hybrid mode information is used to generate the prediction for the current block. Candidate inherited information may refer to encoding / decoding modes and / or corresponding information. The current block obtains its own prediction information from the inherited information.

[0146] The encoding / decoding mode in the inherited hybrid mode information is an example related to DIMD, as shown below:

[0147] In some cases, DIMD merge mode (Section 2.3.1) or regular DIMD is applied to the current block.

[0148] The encoding / decoding mode in the inherited hybrid mode information is an example related to TIMD, as shown below:

[0149] In some cases, TIMD merge mode (Section 2.3.2) or regular TIMD is applied to the current block.

[0150] The encoding / decoding mode in the inherited hybrid mode information is an example related to MRL, as shown below:

[0151] In some cases, MRL merging mode (Section 2.3.3) or regular intra-frame prediction using MRL or TMRL is applied to the current block.

[0152] An example of the encoding / decoding mode in the inherited hybrid mode information related to EIP is shown below:

[0153] In some cases, the EIP merge mode or the EIP derivative mode is applied to the current block.

[0154] In another embodiment, only one candidate (e.g., the first available candidate) is used to determine the prediction information for the current block based on the inherited block settings.

[0155] In another embodiment, one or more candidates are used to determine the prediction information for the current block. For example, all available candidates are used to determine the prediction information for the current block. For example, assuming a predefined maximum number of N, the first N available candidates according to the inspection order are used to determine the prediction information for the current block. If the total number of available candidates is less than N, then all available candidates are used to determine the prediction information for the current block.

[0156] In one embodiment, an additional flag is signaled to indicate whether DIP merge mode is used before the DIMD flag. If DIP merge mode is used, an additional candidate index is signaled.

[0157] In one embodiment, the candidate index uses truncated unary encoding, and each binary bit of the candidate index is context-encoded using a separate context.

[0158] In this invention, the term "block" may refer to TU / TB, CU / CB, PU / PB, predefined area, or CTU / CTB.

[0159] Any combination of the methods proposed in this invention can be applied.

[0160] The method proposed in this invention can be enabled and / or disabled based on implicit rules (e.g., block width, height, or area) or explicit rules (e.g., syntax regarding blocks, tiles, slices, pictures, SPS, or PPS levels). For example, the proposed method is applied when the block area is less than / greater than a threshold.

[0161] The aforementioned methods for merging list derivation and candidate diversity can be implemented in the encoder and / or decoder. For example, any of the proposed methods can be implemented in the interactive / intra-frame / IBC / prediction / transform module of the encoder, and / or in the interactive / intra-frame / IBC / prediction / transform module of the decoder. Alternatively, any of the proposed methods can be implemented as circuitry connected to the interactive / intra-frame / IBC / prediction / transform module of the encoder and / or the interactive / intra-frame / IBC / prediction / transform module of the decoder to provide the information required by the interactive / intra-frame / IBC / prediction / transform module.

[0162] Referring to the exemplary encoder and decoder in Figures 1A and 1B, the proposed method for merging list derivation and candidate diversity can be implemented in an intra-frame codec module in the decoder (e.g., intra-frame prediction 150 in Figure 1B) or in an intra-frame codec module in the encoder (e.g., intra-frame prediction 110 / in Figure 1A). However, the decoder or encoder may also use additional processing units to implement the required cross-component prediction processing. Although the intra-frame prediction units (e.g., unit 110 in Figure 1A and unit 150 in Figure 1B) are shown as separate processing units, they may correspond to executable software or firmware code of a CPU (Central Processing Unit) or a programmable device (e.g., a DSP (Digital Signal Processor) or an FPGA (Field Programmable Gate Array)).

[0163] Figure 7 illustrates a flowchart of an exemplary video encoding / decoding system that, according to an embodiment of the invention, derives a merge list with candidate diversity. The steps shown in the flowchart can be implemented as executable program code on one or more processors (e.g., one or more CPUs) for the encoder side. The steps shown in the flowchart can also be implemented based on hardware, such as one or more electronic devices or processors configured to perform the steps in the flowchart. According to the method, in step 710, input data associated with the current block is received, wherein the input data includes pixel data to be encoded on the encoder side or encoded data associated with the current block and to be decoded on the decoder side. In step 720, a candidate to be included and a current intra-frame merge candidate list are determined. In step 730, it is determined whether to insert the candidate to be included into the current intra-frame merge candidate list to generate an updated intra-frame merge candidate list by checking the similarity between the candidate to be included and existing candidates in the current intra-frame merge candidate list, wherein the similarity includes mode type similarity. In step 740, the current block is encoded or decoded using encoding / decoding information including the updated intra-frame merge candidate list.

[0164] The flowchart shown is intended to illustrate an example of video encoding / decoding according to the present invention. Those skilled in the art can modify each step, rearrange the steps, split the steps, or combine the steps to practice the invention without departing from its spirit. Specific syntax and semantics are used in the disclosure to illustrate examples of implementing the invention. Those skilled in the art can practice the invention by substituting equivalent syntax and semantics without departing from its spirit.

[0165] The foregoing description is intended to enable those skilled in the art to practice the invention in the context of a particular application and its requirements. Various modifications will be seen by those skilled in the art regarding the described embodiments, and the general principles defined herein may be applied to other embodiments. Therefore, the invention is not intended to be limited to the specific embodiments shown and described, but rather to be given the broadest scope consistent with the principles and novel features disclosed herein. In the foregoing detailed description, various specific details have been shown to provide a thorough understanding of the invention. However, those skilled in the art will understand that the invention can be practiced.

[0166] As described above, embodiments of the present invention can be implemented in various hardware, software code, or combinations thereof. For example, one embodiment of the invention may be one or more circuits integrated into a video compression chip, or program code integrated into video compression software to perform the processes described herein. Another embodiment of the invention may be program code to be executed on a digital signal processor (DSP) to perform the processes described herein. The invention may also relate to multiple functions executed by a computer processor, digital signal processor, microprocessor, or field-programmable gate array (FPGA). These processors may be configured to perform specific tasks according to the invention, defining specific methods embodied in the invention by executing machine-readable software code or firmware code. The software code or firmware code may be developed in different programming languages ​​and different formats or styles. The software code may also be compiled for different target platforms. However, different code formats, styles, and languages ​​of the software code, as well as other means of configuring the code to perform tasks consistent with the invention, do not depart from the spirit and scope of the invention.

[0167] The present invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The examples described are to be regarded in all respects as illustrative only and not restrictive. Therefore, the scope of the invention should be indicated by the appended claims rather than the foregoing description. All variations within the meaning and scope of the claims should be included within its scope. [Simplified Explanation of the Diagram]

[0054] Figure 1A illustrates an exemplary adaptive Inter / Intra movie coding system incorporating loop processing. Figure 1B shows the corresponding decoder for the encoder in Figure 1A. Figure 2A shows an example of a selected template for a current block, wherein the template comprises T rows above the current block and T columns to the left of the current block. Figure 2B shows an example with T=3, and gradient histograms (HoGs) are computed for the pixels in the middle rows and middle columns. Figure 2C shows an example of the amplitude (ampl) of the angular intra-prediction mode. Figure 3 shows three filter shapes with fifteen inputs and generating one output for the EIP process. Figures 4A-C show three types of reconstructed regions for deriving the filter coefficients for the EIP (Figure 4A: upper left region, Figure 4B: upper region, and Figure 4C: left region). Figure 5 shows an example of a template region with multiple reference lines. Figure 6 shows the location of spatial merging candidates. Figure 7 shows a flowchart of an exemplary movie codec system that derives a merging list with candidate diversity according to an embodiment of the present invention.

Claims

1. A method for video encoding and decoding, the method comprising: Receive input data related to the current block, wherein the input data includes pixel data to be encoded on the encoder side or encoded data related to the current block and to be decoded on the decoder side; determine a candidate to be included and a current intra-frame merge candidate list; determine whether to insert the candidate to be included into the current intra-frame merge candidate list to generate an updated intra-frame merge candidate list by checking the similarity between the candidate to be included and existing candidates in the current intra-frame merge candidate list, wherein the similarity includes mode type similarity; encode or decode the current block using encoding / decoding information including the updated intra-frame merge candidate list.

2. The method as described in claim 1, wherein a similarity check is also applied when constructing the history-based buffer array, and wherein the similarity is checked between the history candidate to be included and the existing history candidates in the history-based buffer array.

3. The method as described in claim 1, wherein the mode type similarity is determined when the candidate to be included and an existing candidate in the current intra-frame merge candidate list have the same mode type.

4. The method as described in claim 1, wherein when the candidate to be included or the existing candidate in the current intra-frame merge candidate list is an OBIC candidate or a MIMD candidate, the candidate to be included or the existing candidate is considered to have a DIMD mode type with similarity to the mode type in the current intra-frame merge candidate list.

5. The method as described in request item 1, wherein if the current intra-frame merge candidate list already contains more than a threshold number of candidates of the same mode type as the candidate to be included, then the candidate to be included is not included in the current intra-frame merge candidate list.

6. The method as described in request item 5, wherein the threshold is 0 or any positive integer.

7. The method as described in request item 1, wherein the mode type similarity is checked to determine whether the candidate to be included and the existing candidates in the current intra-frame merge candidate list belong to the same mode type group.

8. The method as described in claim 7, wherein the mode type group is defined as one or more intra-prediction methods used by the candidate to be included and existing candidates in the current intra-merge candidate list.

9. The method as described in claim 7, wherein one mode type group is defined as one or more learning-based and / or angle-based intra-prediction methods used by the candidate to be included and existing candidates in the current intra-merge candidate list.

10. The method of claim 1, wherein the similarity further comprises intra-frame prediction mode similarity, prediction fusion weight similarity, block vector (BV) usage information similarity, EIP filter shape similarity, MIP matrix index similarity, or a combination thereof.

11. The method of claim 10, wherein the intra-prediction mode similarity is checked to determine whether the intra-prediction modes of the candidate to be included and the existing candidates in the current intra-merge candidate list are the same.

12. The method as described in request 11, wherein only the intra-prediction mode similarity of the predefined intra-prediction modes of the candidate to be included and the existing candidates in the current intra-merge candidate list is checked.

13. The method as described in request 11, wherein when the candidate to be included and the existing candidates in the current intra-fragment merge candidate list belong to the same mode type group or the same mode type, only the intra-fragment prediction mode similarity check is applied.

14. The method of claim 11, wherein the intra-predictive mode similarity is checked to determine whether the internal mode difference between the candidate to be included and an existing candidate in the current intra-merge candidate list is less than a predefined threshold.

15. The method of claim 11, wherein for the intra-prediction mode similarity check, the domain and / or granularity of the intra-prediction modes of the candidate to be included and the existing candidates in the current intra-merge candidate list are aligned.

16. The method of claim 11, wherein if a first number of intra-prediction modes differ from a second number of intra-prediction modes, the candidate to be included and the existing candidate are determined to be dissimilar candidates in the current intra-merge candidate list.

17. The method of claim 11, wherein the intra-prediction mode similarity check applies to all or part of the candidates to be included and the intra-prediction modes of existing candidates in the current intra-merge candidate list.

18. The method of claim 11, wherein the intra-prediction mode similarity check is applied to the top N intra-prediction modes of the candidate to be included and the existing candidates in the current intra-merge candidate list.

19. The method of claim 11, wherein the mode types of the candidate to be included and the existing candidate in the current intra-frame merging candidate list are both extrapolation filter-based intra-frame prediction (EIP) or matrix-based intra-frame prediction (MIP), and the intra-frame prediction mode similarity check corresponds to checking whether the candidate to be included and the existing candidate in the current intra-frame merging candidate list have the same EIP filter shape or the same MIP matrix index, respectively.

20. The method of claim 10, wherein the fusion weight similarity check is used to determine whether the fusion weights of the candidate to be included and the existing candidates in the current intra-frame merge candidate list are the same for all or part of the intra-frame modes of the candidate to be included and the existing candidates in the current intra-frame merge candidate list.

21. The method of claim 20, wherein the fusion weights between the candidate to be included and the corresponding intra-prediction modes of existing candidates in the current intra-merge candidate list are subjected to a fusion weight similarity check.

22. The method as described in request item 20, wherein the fusion weight similarity check is performed with low precision.

23. The method of claim 20, wherein if the mode type of the candidate to be included and the existing candidate in the current frame merge candidate list are both EIP mode, then a fusion weight similarity check is performed to determine whether all or part of the EIP model parameters of the candidate to be included and the existing candidate in the current frame merge candidate list are completely identical.

24. The method of claim 10, wherein BV uses an information similarity check to determine whether both the candidate to be included and existing candidates in the merged candidate list in the current frame use BV to generate predictions.

25. The method of claim 24, wherein when both the candidate to be included and the existing candidate in the current frame merge candidate list use BV to generate predictions, a BV usage information similarity check is further performed to determine whether the candidate to be included and the existing candidate in the current frame merge candidate list use the same BV.

26. The method as described in request 24, wherein BV uses information similarity checking with low precision.

27. The method of claim 1, wherein after checking the similarity between the candidate to be included and existing candidates in the current frame merge candidate list, if no similar candidate is found, the candidate to be included is added to the current frame merge candidate list.

28. An apparatus for video encoding / decoding, the apparatus comprising one or more electronic means or processors configured to: receive input data associated with a current block, wherein the input data includes pixel data to be encoded at the encoder side or encoded data associated with the current block and to be decoded at the decoder side; determine candidates to be included and a current intra-merge candidate list; determine whether to insert the candidate to be included into the current intra-merge candidate list to generate an updated intra-merge candidate list by examining the similarity between the candidate to be included and existing candidates in the current intra-merge candidate list, wherein the similarity includes mode type similarity; and encode or decode the current block using encoding / decoding information including the updated intra-merge candidate list.

Images