Coordination between different video processing modes

Abstract

This application relates to coordination between different video processing modes. A method of video processing, comprising: during a conversion between a first block in video data and a bitstream representation of the first block, determining one or more coding modes of one or more second blocks; based on the one or more coding modes of the one or more second blocks, determining a coding mode constraint for the first block; and performing the conversion at least by applying the coding mode constraint for the first block; wherein the one or more second blocks comprise at least one of a neighboring block, a non-neighboring block, and a reference block of the first block.

Description

TECHNICAL FIELD

[0001] This document relates to video and image coding and decoding techniques. BACKGROUND

[0002] Digital video accounts for the largest bandwidth use on the internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, it is expected that the bandwidth demand for digital video usage will continue to grow. SUMMARY

[0003] The disclosed techniques can be used by video, image decoder or encoder embodiments in which combined inter-intra prediction is used.

[0004] In one example aspect, a method of processing a video is disclosed. The method includes determining a coding mode of a first video block, constraining one or more flags to an operational state based on the determination of the coding mode of the first video block, the operational state being false or true, and performing further processing of a second video block according to the operational state of the one or more flags, wherein the first video block is a neighboring video block or a reference video block related to the second video block.

[0005] In another example aspect, a method of processing a video is disclosed. The method includes determining to use paired prediction or combined bi-prediction for a first video block, determining an operational state of combined inter-intra prediction (CIIP) based on the determination to use paired prediction or combined bi-prediction, wherein the operational state is enabled or disabled, and performing further processing of the first video block according to the operational state of the CIIP.

[0006] In another example aspect, another method of video processing is disclosed. The method includes performing a conversion between a video block of a video and a coded representation of the video block using a combined inter-intra prediction mode, wherein the coded representation corresponds to a weighted average of an intra and an inter prediction result of the video block using a pair of weights from a set of pairs of weights, the set of pairs of weights containing less than three pairs.

[0007] In another example aspect, another method of video processing is disclosed. The method includes performing a conversion between a video block of a video and a coded representation of the video block using a combined inter-intra prediction mode, wherein the coded representation corresponds to a weighted average of an intra and an inter prediction result of the video block using a pair of weights from a set of pairs of weights, the set of pairs of weights being determined from coding information of one or more neighboring blocks.

[0008] In another example aspect, another method of video processing is disclosed. The method includes determining one or more coding modes of one or more second blocks during a conversion between a first block in video data and a bitstream representation of the first block; determining a coding mode constraint of the first block based on the one or more coding modes of the one or more second blocks; and performing the conversion at least by applying the coding mode constraint of the first block; wherein the one or more second blocks comprise at least one of a neighboring block, a non-neighboring block, and a reference block of the first block.

[0009] In another example aspect, another method of video processing is disclosed. The method includes performing a conversion between a current block in video data and a bitstream representation of the current block by using at least one of combined inter-intra prediction (CIIP), diffusion filtering, bilateral filtering, transform domain filtering, or another type of post-reconstruction filtering different from diffusion filtering, bilateral filtering, and transform domain filtering, wherein reconstructed neighboring samples of the current block used in the at least one of CIIP, diffusion filtering, bilateral filtering, transform domain filtering, or another type of post-reconstruction filtering are replaced with approximation samples generated from corresponding samples of reconstructed neighboring samples in one or more reference pictures.

[0010] In another example aspect, another method of video processing is disclosed. The method includes storing, in a history-based motion vector prediction (HMVP) table with motion information, a CIIP flag of a combined inter-intra prediction (CIIP) mode and / or an intra mode; and performing a conversion between a current block in video data and a bitstream representation of the current block based at least on the HMVP table.

[0011] In another example aspect, another method of video processing is disclosed. The method includes determining a prediction mode of a current block during a conversion between the current block in video data and a bitstream representation of the current block; determining applicability of a combined inter-intra prediction (CIIP) mode, the applicability indicating that the CIIP mode is enabled for the current block in response to determining that the current block is coded using an advanced motion vector prediction (AMVP) mode or a Merge mode; and performing the conversion based on the applicability of the CIIP mode.

[0012] In another example aspect, another method of video processing is disclosed. The method includes determining a type of a selected Merge candidate of a current block during a conversion between the current block in video data and a bitstream representation of the current block; determining applicability of combined inter-intra prediction (CIIP) for the current block according to the type of the selected Merge candidate, wherein the current block is coded in a Merge mode.

[0013] In another example aspect, the above method can be implemented by a video decoder apparatus comprising a processor.

[0014] In yet another example aspect, the methods can be implemented in the form of processor-executable instructions and stored on a computer-readable program medium.

[0015] These and other aspects are further described below. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 An example of the derivation process for Merge candidate list construction is shown.

[0017] Figure 2 An example of the location of the spatial Merge candidate is shown.

[0018] Figure 3 An example of a pair of candidates considering the redundancy check for the spatial Merge candidate is shown.

[0019] Figure 4 An example of the location of the second PU for Nx2N and 2NxnN partitions is shown.

[0020] Figure 5 An example of the motion vector scaling for temporal Merge candidate is shown.

[0021] Figure 6 An example of the candidate locations C0 and C1 for temporal Merge candidate is shown.

[0022] Figure 7 An example of the combined bi-predictive Merge candidate is shown.

[0023] Figure 8 An example of the derivation process for motion vector prediction candidate is shown.

[0024] Figure 9 An example of the motion vector scaling for spatial motion vector candidate is shown.

[0025] Figure 10 An example of the optional AT M VP for a coding unit (CU) is shown.

[0026] Figure 11 An example of the neighboring samples used to derive IC parameters is shown.

[0027] Figure 12 An example of the UMVE search process is shown.

[0028] Figure 13 An example of the UMVE search point is shown.

[0029] Figure 14 An example of a decoding flow diagram using the proposed HMVP method is shown.

[0030] Figure 15 An example of updating the table in the proposed HMVP method is shown.

[0031] Figure 16 An example of symmetric mode is shown.

[0032] Figure 17 An example of neighboring samples of the current block in the reference picture and their corresponding samples is shown.

[0033] Figure 18 A block diagram of an example of a video processing apparatus.

[0034] Figure 19 A block diagram of an example implementation of a video encoder is shown.

[0035] Figure 20 A flowchart of an example of a video processing method.

[0036] Figure 21 A flowchart of an example of a video processing method.

[0037] Figure 22 An example of top and left neighboring blocks used in CIIP weight evaluation is shown.

[0038] Figure 23 A flowchart of an example of a video processing method.

[0039] Figure 24 A flowchart of an example of a video processing method.

[0040] Figure 25 A flowchart of an example of a video processing method.

[0041] Figure 26 A flowchart of an example of a video processing method.

[0042] Figure 27 A flowchart of an example of a video processing method. DETAILED DESCRIPTION

[0043] Various techniques are provided herein that can be used by a decoder of an image or video bitstream to improve the quality of decompressing or decoding digital video or images. For brevity, the term “video” is used herein to include sequences of pictures (commonly referred to as video) and single images. Also, a video encoder can implement these techniques during the encoding process as well to reconstruct decoded frames for further encoding.

[0044] For ease of understanding, section headings are used in the following disclosure and the embodiments and techniques are not limited to the sections in which they are described. Thus, embodiments from one section can be combined with embodiments from other sections.

[0045] 1. SUMMARY

[0046] This document relates to video coding techniques. In particular, it relates to combined inter-intra prediction (CIIP) in video encoding or decoding. It can be applied to existing video coding standards such as HEVC, or to pending standards (Versatile Video Coding). It can also be applicable to future video coding standards or video codecs.

[0047] 2. BACKGROUND

[0048] Video coding standards have evolved primarily through the development of the well-known ITU-T and ISO / IEC standards. The ITU-T produced H.261 and H.263, ISO / IEC produced MPEG-1 and MPEG-4 Visual, and the two organizations jointly produced the H.262 / MPEG-2 Visual and H.264 / MPEG-4 Advanced Video Coding (AVC) and H.265 / HEVC [1] standards. Since H.262, the video coding standards are based on the hybrid video coding structure, where temporal prediction plus transform coding are utilized. To explore future video coding technologies beyond HEVC, the Joint Video Exploration Team (JVET) was founded by VCEG and MPEG in 2015. Since then, many new methods have been adopted by the JVET and applied to the reference software named Joint Exploration Model (JEM). In April 2018, the Joint Video Team (JVT) between VCEG (Q6 / 16) and ISO / IEC JTC1 SC29 / WG11 (MPEG) was formed to work on the VVC standard with the goal of 50% bitrate reduction compared to HEVC.

[0049] Figure 19 is a block diagram of an exemplary implementation of a video encoder. Figure 19 The encoder implementation is shown with a built-in feedback path, where the video encoder also performs the video decoding function (reconstructing the compressed representation of the video data for use in the encoding of the next video data).

[0050] 2.1 INTER PREDICTION IN HEVC / H.265

[0051] Each inter predicted PU has one or two motion parameters for the reference picture list. The motion parameters include a motion vector and a reference picture index. The use of one of the two reference picture lists can also be signaled using inter_pred_idc. The motion vector can be explicitly coded as a delta relative to the predictor.

[0052] When a CU is coded with the skip mode, one PU is associated with the CU and there is no significant residual coefficient, no coded motion vector delta or reference picture index. A Merge mode is specified by which the motion parameters of the current PU can be derived from neighboring PUs, including spatial and temporal candidates. Merge mode can be applied to any inter predicted PU, not only to the skip mode. Another option for the Merge mode is the explicit signaling of the motion parameters, where the motion vector (more precisely, the motion vector difference compared to the motion vector predictor), the corresponding reference picture index for each reference picture list, and the usage of the reference picture list are explicitly signaled per PU. In this disclosure, such a mode is named as the advanced motion vector prediction (AMVP).

[0053] When the signaling indicates that one of the two reference picture lists is to be used, the PU is generated from one sample of the samples. This is called "uni-prediction". Uni-prediction is available for P slices and B slices.

[0054] When the signaling indicates that both reference picture lists are to be used, the PU is generated from two blocks of samples. This is called "bi-prediction". Bi-prediction is only available for B slices.

[0055] Details on the inter prediction modes specified in HEVC are provided below. The description will start from the Merge mode.

[0056] 2.1.1 Merge mode

[0057] 2.1.1.1 Derivation of candidates for the Merge mode

[0058] When a PU is predicted using the Merge mode, an index pointing to an entry in the list of Merge candidates is parsed from the bitstream and the motion information is retrieved using this index. The construction of this list is specified in the HEVC standard and can be summarized in the following steps order:

[0059] Step 1 : Initial candidate derivation

[0060] Step 1.1 : Spatial candidate derivation

[0061] Step 1.2 : Spatial candidate redundancy check

[0062] Step 1.3 : Temporal candidate derivation

[0063] Step 2 : Additional candidate insertion

[0064] Step 2.1 : Creation of bi-prediction candidates

[0065] Step 2.2 : Insertion of zero motion candidate

[0066] These steps are also illustrated in Figure 1 For spatial Merge candidate derivation, up to four Merge candidates are selected among the candidates located in five different positions. For temporal Merge candidate derivation, up to one Merge candidate is selected among two candidates. Since the number of candidates per PU is assumed to be constant at the decoder, additional candidates are generated when the number of candidates obtained from step 1 does not reach the maximum number of Merge candidates (maxNumMergeCand) signaled in the slice header. Since the number of candidates is constant, the index of the best Merge candidate is coded using truncated unary (TU). If the size of the CU is equal to 8, all PUs of the current CU share one Merge candidate list, which is the same as the Merge candidate list for 2Nx2N prediction units.

[0067] In the following, the operations associated with the aforementioned steps are described in detail.

[0068] 2.1.1.2 Spatial candidate derivation

[0069] In the derivation of spatial Merge candidates, up to four Merge candidates are selected among the candidates located in the positions Figure 2 A1, B1, B0, A0 and B2. Position B2 is only considered if any of the positions A1, B1, B0, A0 is not available (e.g. because it belongs to another slice or tile) or is intra coded. After adding the candidate of position A1, a redundancy check is performed for the remaining additions, which ensures that candidates with the same motion information are excluded from the list, thus improving the coding. In order to reduce the complexity of the calculations, not all possible pairs of candidates are considered in the mentioned redundancy check. Instead, only the pairs linked by the arrows in Figure 3 are considered and a candidate is only added to the list if the corresponding candidate used for the redundancy check does not have the same motion information. Another source of copied motion information is the "second PU" related to the partitioning different from 2Nx2N. For example, Figure 4 The second PU for the N x 2N and 2N x N cases is described separately. When the current PU is partitioned as N x 2N, the candidate of position A1 is not considered for the list construction. Indeed, adding this candidate could lead to two prediction units with the same motion information, which is redundant as only one PU is present in the coded unit. Similarly, when the current PU is partitioned as 2N x N, position B1 is not considered.

[0070] 2.1.1.3 Temporal candidate derivation

[0071] In this step, only one candidate is added to the list. In particular, in the derivation of this temporal Merge candidate, the scaled motion vector is derived based on the collocated PU belonging to the picture having the smallest POC difference with the current picture in the given reference picture list. The reference picture list used for deriving the collocated PU is explicitly signaled in the slice header. The scaled motion vector of the temporal Merge candidate is obtained (as shown by the dashed line in the middle) using the POC distances tb and td from the motion vector of the collocated PU, where tb is defined as the POC difference between the reference picture of the current picture and the current picture, and td is defined as the POC difference between the reference picture of the collocated picture and the collocated picture. The reference picture index of the temporal Merge candidate is set to zero. The actual implementation of the scaling process is described in the HEVC specification. For B slices, two motion vectors are obtained (one for reference picture list 0 and the other for reference picture list 1) and combined to make it a bi-predictive Merge candidate. Figure 5

[0072] In the collocated PU (Y) belonging to the reference frame, the position of the temporal candidate is chosen between the candidates C0 and C1 as shown in Figure 6 If the PU at position C0 is not available, intra coded or outside the current CTU row, position C1 is used. Otherwise, position C0 is used for the derivation of the temporal Merge candidate.

[0073] 2.1.1.4 Additional candidate insertion

[0074] In addition to the spatial and temporal Merge candidates, there are two additional types of Merge candidates: combined bi-predictive Merge candidates and zero Merge candidates. Combined bi-predictive Merge candidates are generated with the spatial and temporal Merge candidates. Combined bi-predictive Merge candidates are only used for B slices. A combined bi-predictive candidate is generated by combining the first reference picture list motion parameters of an initial candidate with the second reference picture list motion parameters of another candidate. If these two tuples provide different motion hypotheses, they will form a new bi-predictive candidate. As an example, Figure 7 This situation is shown in which two candidates with MVL0 and refldxL0 or MVL1 and refldxL1 in the original list (on the left) are used to create a combined bi-predictive Merge candidate that is added to the final list (on the right). Many rules are defined regarding the combinations that are considered to generate these additional Merge candidates.

[0075] ​Zero motion candidates are inserted to fill the rest of the entries in the Merge candidate list up to the capacity of MaxNumMergeCand. These candidates have zero spatial displacement and a reference picture index that starts from zero and is incremented each time a new zero motion candidate is added to the list. The number of reference frames used by these candidates is 1 and 2 for uni- and bi-prediction, respectively. Finally, no redundancy check is performed for these candidates.

[0076] 2.1.1.5 Motion estimation region for parallel processing

[0077] To speed up the encoding process, motion estimation can be performed in parallel, thus deriving the motion vectors of all prediction units within a given region at the same time. Deriving Merge candidates from spatial neighbors can interfere with parallel processing, as a prediction unit cannot derive motion parameters from neighboring PUs until their associated motion estimation is completed. To mitigate the trade-off between coding efficiency and processing delay, HEVC uses the "log2_parallel_merge_level_minus2" syntax element to define in the picture parameter set the size of the motion estimation region (MER) whose dimensions are signaled. When a MER is defined, Merge candidates falling into the same region are marked as unavailable and thus not considered in the list construction.

[0078] 2.1.2 AMVP

[0079] AMVP exploits the spatio-temporal correlation of motion vectors with neighboring PUs, which are used for the explicit transmission of motion parameters. For each reference picture list, a motion vector candidate list is first constructed by checking the availability of the top-left spatio-temporal neighbor PU position, removing redundant candidates and adding zero vectors to make the candidate list length constant. The encoder can then select the best predictor from the candidate list and send the corresponding index indicating the selected candidate. Similar to the Merge index signaling, the index of the best motion vector candidate is coded using truncated unary. In this case, the maximum value to be coded is 2 (see Figure 8 ). In the following sections, details about the derivation process of motion vector prediction candidates are provided.

[0080] 2.1.2.1 Derivation of AMVP candidates

[0081] Figure 8 The derivation process of motion vector prediction candidates is summarized.

[0082] In motion vector prediction, two types of motion vector candidates are considered: spatial motion vector candidates and temporal motion vector candidates. For the derivation of spatial motion vector candidates, the motion vectors of each PU located in five different positions shown in Figure 2 are finally derived.

[0083] For temporal motion vector candidates, one motion vector candidate is selected from two candidates that are derived based on two different collocated positions. After the first temporal motion vector list is made, duplicate motion vector candidates in the list are removed. If the number of potential candidates is greater than two, the motion vector candidates with reference picture list index greater than one in the associated reference picture list are removed from the list. If the number of temporal motion vector candidates is less than two, additional zero motion vector candidates are added in the list.

[0084] 2.1.2.2 Spatial motion vector candidates

[0085] In deriving spatial motion vector candidates, at most two candidates are considered from five potential candidates, which are derived from the PUs in the Figure 2 positions that are the same as the position of the motion Merge. The derivation order for the left side of the current PU is defined as A0, A1, and scaled A0, scaled A1. The derivation order for the top side of the current PU is defined as B0, B1, B2, scaled B0, scaled B1, scaled B2. Thus, there are four cases for each side that can be used as motion vector candidates, two of which do not need to use spatial scaling, and two of which use spatial scaling. The four different cases are summarized as follows:

[0086] -- No spatial scaling

[0087] (1) Same reference picture list, and same reference picture index (same POC)

[0088] (2) Different reference picture list, but same reference picture (same POC)

[0089] -- Spatial scaling

[0090] (3) Same reference picture list, but different reference picture (different POC)

[0091] (4) Different reference picture list, and different reference picture (different POC)

[0092] The no spatial scaling case is checked first, and then the spatial scaling allowed case is checked. Spatial scaling is considered for all cases when the POC is different between the reference picture of the neighboring PU and the reference picture of the current PU, regardless of the reference picture list. If all PUs of the left side candidate are not available or intra coded, scaling of the above motion vector is allowed to help parallel derivation of the left and top MV candidates. Otherwise, spatial scaling of the above motion vector is not allowed.

[0093] In the spatial scaling process, the motion vector of the neighboring PU is scaled in a similar way as temporal scaling, as shown in Figure 9The main difference is that the reference picture list and index of the current PU are given as input; the actual scaling process is the same as the temporal scaling process.

[0094] 2.1.2.3 Temporal motion vector candidate

[0095] Except for the derivation of the reference picture index, all derivation processes of the temporal Merge candidate are the same as the derivation processes of the spatial motion vector candidate (see Figure 6 ). The reference picture index is signaled to the decoder.

[0096] 2.2 New inter prediction method

[0097] 2.2.1 Sub-CU based motion vector prediction

[0098] In JEM with QTBT, each CU can have at most one set of motion parameters for each prediction direction. By dividing a large CU into sub-CUs and deriving the motion information for all sub-CUs of the large CU, two sub-CU level motion vector prediction methods are considered in the encoder. The alternative temporal motion vector prediction (ATMVP) method allows each CU to extract multiple sets of motion information from multiple blocks smaller than the current CU in the collocated reference picture. In the spatial-temporal motion vector prediction (STMVP) method, the motion vector of a sub-CU is derived recursively by using a temporal motion vector predictor and a spatial neighboring motion vector.

[0099] To keep more accurate motion field for sub-CU motion prediction, the motion compression of the reference frame is currently disabled.

[0100] 2.2.1.1 Alternative temporal motion vector prediction

[0101] In the alternative temporal motion vector prediction (ATMVP) method, the motion vector temporal motion vector prediction (TMVP) is modified by extracting multiple sets of motion information (including motion vector and reference index) from blocks smaller than the current CU. As shown in Figure 10 , the sub-CU is a square NxN block (default N is set to 4).

[0102] ATMVP predicts the motion vector of a sub-CU within a CU in two steps. The first step is to identify the corresponding block in the reference picture with a so-called temporal vector. The reference picture is called the motion source picture. The second step is to divide the current CU into sub-CUs and obtain the reference index and motion vector for each sub-CU from the corresponding block of each sub-CU, as shown in Figure 10 .

[0103] In the first step, the reference picture and the corresponding block are determined by the motion information of the spatial neighboring blocks of the current CU. To avoid the repeated scanning process of the neighboring blocks, the first Merge candidate in the Merge candidate list of the current CU is used. The first available motion vector and its associated reference index are set to the temporal vector and the index to the motion source picture. In this way, in ATMVP, the corresponding block, sometimes referred to as the collocated block, can be identified more accurately compared to TMVP, which is always located at the right bottom or center position relative to the current CU.

[0104] In the second step, the corresponding block of the sub-CU is identified by the temporal vector in the motion source picture by adding the temporal vector to the coordinates of the current CU. For each sub-CU, the motion information of its corresponding block (covering the smallest motion grid of the center sample) is used to derive the motion information of the sub-CU. After the motion information of the corresponding NxN block is identified, it is converted to the motion vector and the reference index of the current sub-CU, the same as the TMVP method of HEVC, where motion scaling and other processes are applied. For example, the decoder checks whether the low-delay condition is satisfied (e.g., the POCs of all reference pictures of the current picture are smaller than the POC of the current picture) and possibly uses the motion vector MV x (corresponding to the reference picture list X) to predict the motion vector MV y (X equals 0 or 1 and Y equals 1-X).

[0105] 2.2.2 Pairwise average candidate

[0106] The pairwise average candidate is generated by averaging the predefined pairs of candidates in the current Merge candidate list, and the predefined pairs are defined as {(0, 1), (0, 2), (1, 2), (0, 3), (1, 3), (2, 3)}, where the numbers represent the Merge indices to the Merge candidate list. The average motion vector is calculated for each reference list separately. If both motion vectors are available in one list, they are averaged even if they point to different reference pictures; if only one motion vector is available, it is used directly; if no motion vector is available, this list is kept invalid. The pairwise average candidate replaces the combined candidate in the HEVC standard.

[0107] The complexity analysis of the pair-wise average candidate is summarized in Table 1. For the worst case of additional computation for the average (last column of Table 1), 4 additions and 4 shifts are needed per pair (MVx and MVy in L0 and L1), and 4 reference index comparisons are needed per pair (refldxO valid in L0 and refldxl valid in L1). There are 6 pairs, resulting in 24 additions, 24 shifts and 24 reference index comparisons in total. The combined candidate in HEVC standard uses 2 reference index comparisons per pair (refldxO valid in L0 and refldxl valid in L1), and there are 12 pairs, resulting in 24 reference index comparisons in total.

[0108] Table 1: Operation analysis of pair-wise average candidate

[0109]

[0110] 2.2.3 Local Illumination Compensation

[0111] Local Illumination Compensation (LIC) is based on a linear model for illumination changes, using a scale factor a and an offset b. And it is adaptively enabled or disabled for each inter mode coded Coding Unit (CU).

[0112] When LIC is applied to a CU, the least square error method is employed to derive the parameters a and b by using the neighboring samples of the current CU and their corresponding reference samples. More specifically, as shown in Figure 11 sub-sampled (2:1 sub-sampling) neighboring samples of the CU in the reference picture and the corresponding samples (identified by the motion information of the current CU or sub-CU) are used. The IC parameters are derived and applied to each prediction direction respectively.

[0113] When a CU is coded using Merge mode, the LIC flag is copied from the neighboring block in a similar way as the motion information copy in Merge mode; otherwise, the LIC flag will be signaled for the CU to indicate whether LIC is applicable.

[0114] When LIC is enabled for a picture, additional CU-level RD check is needed to determine whether to apply LIC for a CU. When LIC is enabled for a CU, mean-removed sum of absolute difference (MR-SAD) and mean-removed sum of absolute Hadamard-transformed difference (MR-SATD) are used for integer pixel motion search and fractional pixel motion search respectively, instead of SAD and SATD.

[0115] To reduce coding complexity, the following coding scheme is applied in JEM:

[0116] If there is no significant change in illumination between the current image and its reference images, LIC is disabled for the entire image. To identify this situation, the histogram of the current image and the histogram of each reference image of the current image are calculated at the encoder. If the histogram difference between the current image and each reference image of the current image is less than a given threshold, LIC is disabled for the current image; otherwise, LIC is enabled for the current image.

[0117] 2.2.4 Combined Inter-Frame and Intra-Frame Prediction

[0118] In some embodiments, multiple hypothesis prediction is proposed, wherein combining intra-frame and inter-frame prediction is one way to generate multiple hypotheses.

[0119] When multiple hypothesis prediction is applied to improve intra mode, multiple hypothesis prediction combines one intra prediction and one Merge index prediction. In Merge CU, when the flag is true, one flag is signaled for Merge mode to select the intra mode from the intra candidate list. For luma component, the intra candidate list is derived from 4 intra prediction modes including DC, Planar, Horizontal and Vertical modes, and the size of the intra candidate list can be 3 or 4 depending on the block shape. When the CU width is more than twice the CU height, the horizontal mode is not included in the intra mode list, and when the CU height is more than twice the CU width, the vertical mode is removed from the intra mode list. One intra prediction mode selected by the intra mode index and one Merge index prediction selected by the Merge index are combined using weighted average. For chroma component, DM is always applied without extra signaling. The weights used for combining the predictions are described as follows. When DC or Planar mode is selected, or the CB width or height is less than 4, equal weights are applied. For CBs with CB width and height greater than or equal to 4, when horizontal / vertical mode is selected, a CB is first divided into four equal regions vertically / horizontally. Each weight set denoted as (w_intrai, w_interi) is applied to the corresponding region, where i is from 1 to 4, and (w_intra1, w_inter1) = (6, 2), (w_intra2, w_inter2) = (5, 3), (w_intra3, w_inter3) = (3, 5), and (w_intra4, w_inter4) = (2, 6). (w_intra1, w_inter1) is used for the region closest to the reference samples, and (w_intra4, w_inter4) is used for the region farthest from the reference samples. Then, the combined prediction can be computed by adding the two weighted predictions and right shifting by 3 bits. In addition, the intra prediction mode of the intra hypothesis of the predictor can be saved for subsequent neighboring CUs to reference.

[0120] 2.2.5 Generalized bi-prediction

[0121] In traditional bi-prediction, the predictors from L0 and L1 are averaged using equal weights 0.5 to generate the final predictor. The predictor generation formula is shown in Equation (1):

[0122] P TraditionalBiPred = (P L0 + P L1 + RoundingOffset) >> shiftNum, (1)

[0123] In Equation (1), P TraditionalBiPred is the final predictor of traditional bi-prediction, P L0 and PL1 are predictors from L0 and L1 respectively, and RoundingOffset and shiftNum are used to normalize the final predictor.

[0124] Generalized bi-prediction (GBI) is proposed to allow different weights to be applied to the predictors from L0 and L1. The predictor is generated as shown in equation (2).

[0125] P GBi = ((1 - w1) * P L0 + w1 * P L1 + RoundingOffset GBi ) » shiftNum GBi , (2)

[0126] In equation (2), P GBi is the final predictor for GBI. (1 - w1) and w1 are the selected GBI weights applied to the predictors from L0 and L1 respectively. RoundingOffset GBi and shiftNum GBi are used to normalize the final predictor in GBI.

[0127] The supported w1 weight table is {-1 / 4, 3 / 8, 1 / 2, 5 / 8, 5 / 4}. One equal-weight set and four unequal-weight sets are supported. For the equal-weight case, the process to generate the final predictor is exactly the same as in the conventional bi-prediction mode. For the true bi-prediction case under random access (RA) conditions, the number of candidate weight sets is reduced to three.

[0128] For the advanced motion vector prediction (AMVP) mode, the weight selection in GBI is explicitly signaled at the CU level if the CU is coded with bi-prediction. For the Merge mode, the weight selection is inherited from the Merge candidate. In this proposal, GBI supports DMVR to generate the weighted average of the templates and the final predictor of BMS-1.0.

[0129] 2.2.6 Final motion vector expression

[0130] In some embodiments, a final motion vector expression (UMVE) is proposed. UMVE is used for skip or Merge mode with the proposed motion vector expression method.

[0131] UMVE is reused as in VVC. In Merge candidate, a candidate can be selected and further extended by the proposed motion vector expression method.

[0132] UMVE provides a new motion vector expression with simplified signaling. The expression method includes a starting point, a motion magnitude, and a motion direction.

[0133] This proposed technique uses the Merge candidate list as is. But only the candidates of default Merge type (MRG_TYPE_DEFAULT_N) are considered for the extension of UMVE.

[0134] The base candidate index defines the starting point. The base candidate index indicates the best candidate among the candidates in the list as follows.

[0135] Table 1 Base candidate index IDX

[0136]

[0137] If the number of base candidates is equal to 1, the base candidate IDX is not signaled.

[0138] The distance index is the motion magnitude information. The distance index indicates a predefined distance from the starting point information. The predefined distance is as follows:

[0139] Table 2 Distance index IDX

[0140]

[0141] The direction index represents the direction of the MVD relative to the starting point. The direction index can represent four directions as follows.

[0142] Table 3 Direction index IDX

[0143] Direction IDX 00 01 10 11 x-axis + – N / A N / A y-axis N / A N / A + –

[0144] The UMVE flag is signaled immediately after the signaling of the skip flag and the Merge flag. If the skip and Merge flags are true, the UMVE flag is parsed. If the UMVE flag is equal to 1, the UMVE syntax is parsed. But if it is not 1, the affine flag is parsed. If the affine flag is equal to 1, it is the affine mode, but if it is not 1, the skip / Merge index of the skip / Merge mode of VTM is parsed.

[0145] The additional line buffer due to the UMVE candidate is not needed. Because the software’s skip / Merge candidate is directly used as the base candidate. By using the input UMVE index, the MV’s complement is directly decided before the motion compensation. No long line buffer is needed for this.

[0146] 2.2.7 History-based motion vector prediction

[0147] In our previous invention P180502841H, one or more lookup tables storing at least one motion candidate are used to predict the motion information of a block.

[0148] A history-based MVP (HMVP) method is proposed, where a HMVP candidate is defined as the motion information of a previously coded block. A table with multiple HMVP candidates is maintained during the encoding / decoding process. When a new slice is encountered, the table is emptied. Whenever there is an inter-coded block, the associated motion information is added as a new HMVP candidate to the last entry of the table. The whole coding flow is shown as Figure 13

[0149] In one example, the table size is set to L (e.g., L = 16 or 6, or 44), which indicates that at most L HMVP candidates can be added to the table.

[0150] In one embodiment, if there are more than L HMVP candidates from previously coded blocks, a first-in-first-out (FIFO) rule is applied so that the table always contains the L most recent previously coded motion candidates. Figure 14 An example is described, where a HMVP candidate is removed according to the FIFO rule, and a new HMVP candidate is added to the table used by the proposed method.

[0151] In another embodiment, whenever a new motion candidate is added (such as the current block is inter-coded and non-affine mode), a redundancy check process is first applied to identify whether there is an identical or similar motion candidate in the LUT.

[0152] 2.2.8 Symmetric motion vector difference

[0153] In some embodiments, a symmetric motion vector difference (SMVD) is proposed to more efficiently code the MVD.

[0154] First, at slice level, the variables BiDirPredFlag, RefIdxSymL0 and RefIdxSymL1 are derived as follows:

[0155] Search the forward reference picture in reference picture list 0 that is closest to the current picture. If found, set RefIdxSymL0 equal to the reference index of the forward picture.

[0156] Search the backward reference picture in reference picture list 1 that is closest to the current picture. If found, set RefIdxSymL1 equal to the reference index of the backward picture.​

[0157] If both the forward and backward pictures are found, set BiDirPredFlag equal to 1.

[0158] Otherwise, the following applies:

[0159] Search for the closest backward reference picture in reference picture list 0 to the current reference picture. If found, set RefIdxSymL0 equal to the reference index of the backward picture.

[0160] Search for the closest forward reference picture in reference picture list 1 to the current reference picture. If found, set RefIdxSymL1 equal to the reference index of the forward picture.

[0161] If both the backward and forward pictures are found, set BiDirPredFlag equal to 1. Otherwise, set BiDirPredFlag equal to 0.

[0162] Secondly, at the CU level, if the prediction direction of the CU is bi-prediction and BiDirPredFlag is equal to 1, a symmetric mode flag indicating whether to use the symmetric mode is explicitly signaled.

[0163] When the flag is true, only mvp_l0_flag, mvp_l1_flag and MVD0 are explicitly signaled. The reference indices of list 0 and list 1 are set equal to RefIdxSymL0, RefIdxSymL1, respectively. MVD1 is set equal to -MVD0 only. The final motion vector is shown in the following equation.

[0164]

[0165] The modifications in the coding unit syntax are shown in Table 2.

[0166] Table 2: Modifications in the coding unit syntax

[0167]

[0168]

[0169] 2.2.9 CIIP in VTM4

[0170] In VTM4, when a CU is coded in Merge mode, and if the CU contains at least 64 luma samples (i.e., the CU width times the CU height is equal to or greater than 64), an additional flag is signaled to indicate whether to apply the combined inter / intra prediction (CIIP) mode to the current CU.

[0171] To form the CIIP prediction, first the intra prediction mode is derived from two additional syntax elements. Up to four possible intra prediction modes can be used: DC, PLANAR, HORIZONTAL or VERTICAL. Then, the inter and intra prediction signals are derived using the regular intra and inter decoding process. Finally, the inter and intra prediction signals are weighted averaged to obtain the CIIP prediction.

[0172] 1. Intra prediction mode derivation

[0173] Up to 4 intra prediction modes (including DC mode, PLANAR mode, HORIZONTAL mode and VERTICAL mode) can be used to predict the luma component in the CIIP mode. If the CU shape is very wide (i.e. the width is more than twice the height), the HORIZONTAL mode is not allowed. If the CU shape is very narrow (i.e. the height is more than twice the width), the VERTICAL mode is not allowed. In these cases, only 3 intra prediction modes are allowed.

[0174] The CIIP mode uses 3 most probable modes (MPM) for intra prediction. The CIIP MPM candidate list is formed as follows:

[0175] - Set the left and top neighboring blocks as A and B, respectively.

[0176] - The intra prediction modes of block A and block B (denoted as intraModeA and intraModeB, respectively) are derived as follows:

[0177] o Let X be A or B;

[0178] o If 1) block X is not available; or 2) block X is not predicted using CIIP mode or intra mode; 3) block B is out of the current CTU, set intraModeX as DC;

[0179] o Otherwise, 1) if the intra prediction mode of block X is DC or PLANAR, set intraModeX as DC or PLANAR; or, 2) if the intra prediction mode of block X is “vertical-like” angular mode (greater than 34), set intraModeX as VERTICAL; or, 3) if the intra prediction mode of block X is “horizontal-like” angular mode (less than or equal to 34), set intraModeX as HORIZONTAL;

[0180] - If intraModeA and intraModeB are the same, then:

[0181] o If intraModeA is PLANAR or DC, set the three MPMs as {PLANAR, DC, VERTICAL} in that order;

[0182] o Otherwise, set the three MPMs in order to {intraModeA, Planar, DC};

[0183] - Otherwise (intraModeA and intraModeB are different):

[0184] o Set the first two MPMs in order to {intraModeA, intraModeB};

[0185] o Check the uniqueness of Planar, DC and Vertical in the first two MPM candidate modes in order; once a unique mode is found, add it as the third MPM.

[0186] If the CU shape is very wide or very narrow as defined above, the MPM flag is inferred to be 1 without signaling. Otherwise, the MPM flag is signaled to indicate whether the CIIP intra prediction mode is one of the CIIP MPM candidate modes.

[0187] If the MPM flag is 1, the MPM index is further signaled to indicate which MPM candidate mode is used in the CIIP intra prediction. Otherwise, if the MPM flag is 0, the intra prediction mode is set to the "missing" mode in the MPM candidate list. For example, if the Planar mode is not in the MPM candidate list, Planar is the missing mode, and the intra prediction mode is set to Planar. Since 4 possible intra prediction modes are allowed in CIIP, and the MPM candidate list contains only 3 intra prediction modes, one of the 4 possible modes must be the missing mode.

[0188] For chroma components, DM mode is always applied without additional signaling; that is, chroma uses the same prediction mode as luma.

[0189] The intra prediction mode of a CU in CIIP coding will be saved and used for the intra mode coding of future neighboring CUs.

[0190] 2.2.9.2 Combining inter and intra prediction signals

[0191] The inter prediction signal P in CIIP mode is derived using the same inter prediction process applied to regular Merge mode inter ; and after the regular intra prediction process, the intra prediction signal P is derived using the CIIP intra prediction mode intra . Then, the intra and inter prediction signals are combined using a weighted average, where the weight values depend on the intra prediction mode and the position of the sample in the coded block, as follows:

[0192] - If the intra prediction mode is DC mode or planar mode, or if the block width or height is smaller than 4, equal weights are applied to the intra and inter prediction signals.

[0193] - Otherwise, the weights are determined based on the intra prediction mode (in this case, horizontal or vertical mode) and the sample positions in the block. Take the horizontal prediction mode as an example (similarly derive the weights for the vertical mode, but in the orthogonal direction): denote W as the width of the block, and H as the height of the block. The coded block is first divided into four equally-sized parts, each part having dimensions of (W / 4) x H. Starting from the part closest to the intra prediction reference sample, and ending with the part furthest away from the intra prediction reference sample, the weights wtfor each of the 4 regions are set to 6, 5, 3, and 2, respectively. The final CIIP prediction signal is derived using:

[0194] P CIIP = ((8 - wt) * P inter + wt * P intra + 4) » 3

[0195] 2. CIIP in VTM5

[0196] In VTM5, when a CU is coded in Merge mode, an additional flag is signaled to indicate whether the Combined Inter / Intra Prediction (CIIP) mode is applied to the current CU if the CU contains at least 64 luma samples (i.e., the CU width times the CU height is equal to or larger than 64), and if both the CU width and the CU height are smaller than 128 luma samples. As the name suggests, CIIP prediction combines an inter prediction signal with an intra prediction signal. The inter prediction signal P inter in CIIP mode is derived using the same inter prediction process as applied to regular Merge mode; and the intra prediction signal P intra is derived according to the same regular intra prediction process as for the planar mode. Then, the intra and inter prediction signals are combined using a weighted average, where the weighting values are computed depending on the coding modes of the top and left neighboring blocks as follows (depicted in Figure 16

[0197] - If the top neighbor is available and is intra coded, set isIntraTop to 1, otherwise set it to 0;

[0198] - If the left neighbor is available and is intra coded, set islntraLeft to 1, otherwise set it to 0;

[0199] - If (islntraLeft + islntraTop) is equal to 2, set wt to 3;​

[0200] - Otherwise, if (isIntraLeft + islntraTop) is equal to 1, set wt to 2;

[0201] - Otherwise, set wt to 1.

[0202] CIIP prediction is formed as follows:

[0203] P CIIP = ((4 - wt) * P inter + wt * P intra + 2) » 2

[0204] Figure 16 Top and left neighboring blocks are shown for CIIP weight derivation.

[0205] 2.3 Intra mode derivation process

[0206] In current VTM-4.0, when coding an intra coded block, a most probable mode (MPM) flag is first signaled. If the MPM flag is true, the index of the MPM list is further signaled. Otherwise, the index of a non-MPM mode is signaled.

[0207] The related syntax elements, symmetries and decoding process in the latest VVC specification (JVET-M1001_v7) are as follows:

[0208] Related syntax coding unit syntax

[0209]

[0210] Semantics

[0211] The syntax elements intra_luma_mpm_flag[ x0 ][ y0 ], intra_luma_mpm_idx[ x0 ][ y0 ] and intra_luma_mpm_remainder[ x0 ][ y0 ] specify the intra prediction mode for luma samples. The array indices x0, y0 specify the position (x0, y0) of the top-left luma sample of the considered coding block relative to the top-left luma sample of the picture. When intra_luma_mpm_flag[ x0 ][ y0 ] is equal to 1, the intra prediction mode is inferred from neighboring intra prediction coding units according to clause 8.4.2.

[0212] If intra_luma_mpm_flag[ x0 ][ y0 ] is not present, it is inferred to be equal to 1.

[0213] Decoding process

[0214] Derivation process of luma intra prediction mode

[0215] The inputs of this process are:

[0216] a luma position (xCb, yCb) specifying the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture,

[0217] a variable cbWidth specifying the width of the current coding block in luma samples,

[0218] a variable cbHeight specifying the height of the current coding block in luma samples.

[0219] In this process, the luma intra prediction mode IntraPredModeY[xCb][yCb] is derived.

[0220] Table 8-1 specifies the values of the intra prediction mode IntraPredModeY[xCb][yCb] and the related names.

[0221] Specification of the intra prediction mode and related names of Table 8-1

[0222] Intra prediction mode Correlation name 0 INTRA_PLANAR 1 INTRA_DC 2..66 INTRA_ANGULAR2..INTRA_ANGULAR66 81..83 INTRA_LT_CCLM, INTRA_L_CCLM, NTRA_T_CCLM

[0223] NOTE - The intra prediction modes INTRA_LT_CCLM, INTRA_L_CCLM and INTRA_T_CCLM are only applicable for chroma components.

[0224] IntraPredModeY[xCb][yCb] is derived by the following ordered steps:

[0225] 1. Set the neighbouring positions (xNbA, yNbA) and (xNbB, yNbB) equal to (xCb - 1, yCb + cbHeight - 1) and (xCb + cbWidth - 1, yCb - 1), respectively.

[0226] 2. For the case where X is replaced by A or B, the variable candIntraPredModeX is derived as follows:

[0227] - Set the position (xCurr, yCurr) equal to (xCb, yCb) and the neighbouring position (xNbY, yNbY) equal to (xNbX, yNbX) as inputs, invoke the availability derivation process of a block as specified in clause 6.4.X [Ed.(BB): pending neighbouring block availability check process] and assign the output to availableX.

[0228] - The candidate intra prediction mode candIntraPredModeX is derived as follows:

[0229] - candlntraPredModeX is set equal to INTRA PLANAR if one or more of the following conditions are true.

[0230] - the variable availableX is equal to FALSE.

[0231] - CuPredMode[ xNbX ][ yNbX ] is not equal to MODE INTRA and ciip_flag[ xNbX ][ yNbX ] is not equal to 1.

[0232] - pcm_flag[ xNbX ][ yNbX ] is equal to 1.

[0233] - X is equal to B and yCb - 1 is less than ( yCb » CtbLog2SizeY)

[0234] < CtbLog2SizeY ).

[0235] - Otherwise, candlntraPredModeX is set equal to IntraPredModeY[ xNbX ][ yNbX ].

[0236] 3. The variables ispDefaultMode1 and ispDefaultMode2 are defined as follows:

[0237] - If IntraSubPartitionsSplitType is equal to ISP_HOR_SPLIT, ispDefaultMode1 is set equal to INTRA ANGULAR18 and ispDefaultMode2 is set equal to INTRA ANGULAR5.

[0238] - Otherwise, ispDefaultMode1 is set equal to INTRA ANGULAR50 and ispDefaultMode2 is set equal to INTRA ANGULAR63.

[0239] 4. candModeList[ x ] ( x = 0...5 ) is derived as follows:

[0240] - If candlntraPredModeB is equal to candlntraPredModeA and candlntraPredModeA is greater than INTRA DC, candModeList[ x ] ( x = 0...5 ) is derived as follows:

[0241] – If IntraLumaRefLineIdx[ xCb ][ yCb ] is equal to 0 and IntraSubPartitionsSplitType is equal to ISP_NO_SPLIT, the following applies:

[0242] candModeList[ 0 ] = candIntraPredModeA (8-9) candModeList[ 1 ] = INTRA_PLANAR (8-10)

[0243] candModeList[ 2 ] = INTRA_DC (8-11)

[0244] candModeList[3] = 2 + ((candIntraPredModeA + 61) % 64) (8-12)

[0245] candModeList[ 4 ] = 2 + ((candIntraPredModeA - 1) % 64) (8-13)

[0246] candModeList[5] = 2 + ((candIntraPredModeA + 60) % 64) (8-14)

[0247] – Otherwise (IntraLumaRefLineIdx[ xCb ][ yCb ] is not equal to 0 or IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT), the following applies:

[0248] candModeList[ 0 ] = candIntraPredModeA (8-15)

[0249] candModeList[1] = 2 + ((candIntraPredModeA + 61) % 64) (8-16)

[0250] candModeList[ 2 ] = 2 + ((candIntraPredModeA - 1) % 64) (8-17) - If one of the following conditions is true,

[0251] - IntraSubPartitionsSplitType is equal to ISP_HOR_SPLIT and candIntraPredModeA is less than INTRA_ANGULAR34,

[0252] - IntraSubPartitionsSplitType is equal to ISP_VER_SPLIT and candIntraPredModeA is greater than or equal to INTRA_ANGULAR34,

[0253] - IntraLumaRefLineIdx[ xCb ][ yCb ] is not equal to 0,

[0254] then the following applies:

[0255] candModeList[ 3 ] = 2 + ( candIntraPredModeA + 60 ) % 64 (8-18) candModeList[ 4 ] = 2 + ( candIntraPredModeA % 64 ) (8-19) candModeList[ 5 ] = 2 + ( candIntraPredModeA + 59 ) % 64 (8-20)

[0256] - Otherwise, the following applies:

[0257] candModeList[ 3 ] = ispDefaultMode1 (8-21)

[0258] candModeList[ 4 ] = ispDefaultMode2 (8-22)

[0259] candModeList[ 5 ] = INTRA_PLANAR (8-23)

[0260] - Otherwise, if candIntraPredModeB is not equal to candIntraPredModeA and either candIntraPredModeA or candIntraPredModeB is greater than INTRA_DC, the following applies:

[0261] - The variables minAB and maxAB are derived as follows:

[0262] minAB = Min( candIntraPredModeA, candIntraPredModeB ) (8-24)

[0263] maxAB = Max( candIntraPredModeA, candIntraPredModeB ) (8-25)

[0264] - If candIntraPredModeA and candIntraPredModeB are both greater than INTRA DC, candModeList[ x ] ( x = 0...5 ) is derived as follows:

[0265] candModeList[ 0 ] = candIntraPredModeA (8-26)

[0266] candModeList[ 1 ] = candIntraPredModeB (8-27)

[0267] - If IntraLumaRefLineIdx[ xCb ][ yCb ] is equal to 0 and IntraSubPartitionsSplitType is equal to ISP_NO_SPLIT, the following applies:

[0268] candModeList[ 2 ] = INTRA_PLANAR (8-28)

[0269] candModeList[ 3 ] = INTRA_DC (8-29) - If maxAB - minAB is in the range of 2 to 62, inclusive, the following applies:

[0270] candModeList[ 4 ] = 2 + ( ( maxAB + 61 ) % 64 ) (8-30)

[0271] candModeList[ 5 ] = 2 + ( ( maxAB - 1 ) % 64 ) (8-31) - Otherwise, the following applies:

[0272] candModeList[ 4 ] = 2 + ( ( maxAB + 60 ) % 64 ) (8-32)

[0273] candModeList[ 5 ] = 2 + ( ( maxAB ) % 64 ) (8-33)

[0274] - Otherwise (IntraLumaRefLineIdx[ xCb ][ yCb ] is not equal to 0 or IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT), the following applies:

[0275] – If IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT, and

[0276] abs( candIntraPredModeB - ispDefaultMode1 ) is less than

[0277] abs( candIntraPredModeA - ispDefaultMode1 ), the following applies:

[0278] candModeList[ 0 ] = candIntraPredModeB (8-34)

[0279] candModeList[ 1 ] = candIntraPredModeA (8-35) – If maxAB - minAB is equal to 1, the following applies:

[0280] candModeList[ 2 ] = 2 + ( ( minAB + 61 ) % 64 ) (8-36)

[0281] candModeList[ 3 ] = 2 + ( ( maxAB - 1 ) % 64 ) (8-37)

[0282] candModeList[ 4 ] = 2 + ( ( minAB + 60 ) % 64 ) (8-38)

[0283] candModeList[ 5 ] = 2 + ( maxAB % 64 ) (8-39) – Else if maxAB - minAB is equal to 2, the following applies:

[0284] candModeList[ 2 ] = 2 + ( ( minAB - 1 ) % 64 ) (8-40)

[0285] candModeList[ 3 ] = 2 + ( ( minAB + 61 ) % 64 ) (8-41)

[0286] candModeList[ 4 ] = 2 + ( ( maxAB - 1 ) % 64 ) (8-42)

[0287] candModeList[ 5 ] = 2 + ( ( maxAB + 60 ) % 64 ) (8-47) - Otherwise, the following applies:

[0288] candModeList[ 2 ] = 2 + ( ( minAB - 1 ) % 64 ) (8-49)

[0289] candModeList[ 3 ] = 2 + ( ( maxAB + 61 ) % 64 ) (8-50)

[0290] candModeList[ 4 ] = 2 + ( ( maxAB - 1 ) % 64 ) (8-51)

[0291] candModeList[ 5 ] = 2 + ( ( maxAB + 60 ) % 64 ) (8-47) - Otherwise, the following applies:

[0292] candModeList[ 2 ] = 2 + ( ( minAB + 61 ) % 64 ) (8-48)

[0293] candModeList[ 3 ] = 2 + ( ( minAB - 1 ) % 64 ) (8-49)

[0294] candModeList[ 4 ] = 2 + ( ( maxAB + 61 ) % 64 ) (8-50)

[0295] candModeList[ 5 ] = 2 + ( ( maxAB - 1 ) % 64 ) (8-51)

[0296] - Otherwise ( candIntraPredModeA or candIntraPredModeB is greater than INTRA DC ), candModeList[ x ] ( x = 0... 5 ) is derived as follows:

[0297] - If IntraLumaRefLineIdx[ xCb ][ yCb ] is equal to 0 and IntraSubPartitionsSplitType is equal to ISP_NO_SPLIT, the following applies:

[0298] candModeList[ 0 ] = candIntraPredModeA (8-52)

[0299] candModeList[ 1 ] = candIntraPredModeB (8-53)

[0300] candModeList[ 2 ] = 1 - minAB (8-54)

[0301] candModeList[ 3 ] = 2 + ( ( maxAB + 61 ) % 64 ) (8-55)

[0302] candModeList[ 4 ] = 2 + ( ( maxAB - 1 ) % 64 ) (8-56)

[0303] candModeList[ 5 ] = 2 + ( ( maxAB + 60 ) % 64 ) (8-57) - Otherwise, if IntraLumaRefLineIdx[ xCb ][ yCb ] is not equal to 0, then

[0304] the following applies:

[0305] candModeList[ 0 ] = maxAB (8-58)

[0306] candModeList[ 1 ] = 2 + ( ( maxAB + 61 ) % 64 ) (8-59)

[0307] candModeList[ 2 ] = 2 + ( ( maxAB - 1 ) % 64 ) (8-60)

[0308] candModeList[ 3 ] = 2 + ( ( maxAB + 60 ) % 64 ) (8-61)

[0309] candModeList[ 4 ] = 2 + ( maxAB % 64 ) (8-62)

[0310] candModeList[ 5 ] = 2 + ( ( maxAB + 59 ) % 64 ) (8-63)

[0311] - Otherwise (IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT), the following applies:

[0312] candModeList[ 0 ] = INTRA_PLANAR (8-64)

[0313] candModeList[ 1 ] = maxAB (8-65)

[0314] candModeList[ 2 ] = 2 + ( ( maxAB + 61 ) % 64 ) (8-66)

[0315] candModeList[ 3 ] = 2 + ( ( maxAB - 1 ) % 64 ) (8-67)

[0316] candModeList[ 4 ] = 2 + ( ( maxAB + 60 ) % 64 ) (8-68)

[0317] candModeList[ 5 ] = 2 + ( maxAB % 64 ) (8-69)

[0318] - Otherwise, the following applies:

[0319] - If IntraLumaRefLineIdx[ xCb ][ yCb ] is equal to 0 and IntraSubPartitionsSplitType is equal to ISP_NO_SPLIT, the following applies:

[0320] candModeList[ 0 ] = candIntraPredModeA (8-70)

[0321] candModeList[1] = ( candModeList[0] == INTRA_PLANAR )?

[0322] INTRA_DC: INTRA_PLANAR (8-71)

[0323] candModeList[ 2 ] = INTRA_ANGULAR50 (8-72)

[0324] candModeList[ 3 ] = INTRA_ANGULAR18 (8-73)

[0325] candModeList[ 4 ] = INTRA_ANGULAR46 (8-74)

[0326] candModeList[ 5 ] = INTRA_ANGULAR54 (8-75)

[0327] - Otherwise, if IntraLumaRefLineIdx[ xCb ][ yCb ] is not equal to 0, the following apply:

[0328] candModeList[ 0 ] = INTRA_ANGULAR50 (8-76)

[0329] candModeList[ 1 ] = INTRA_ANGULAR18 (8-77)

[0330] candModeList[ 2 ] = INTRA_ANGULAR2 (8-78)

[0331] candModeList[ 3 ] = INTRA_ANGULAR34 (8-79)

[0332] candModeList[ 4 ] = INTRA_ANGULAR66 (8-80)

[0333] candModeList[ 5 ] = INTRA_ANGULAR26 (8-81)

[0334] - Otherwise, if IntraSubPartitionsSplitType is equal to ISP_HOR_SPLIT, the following apply:

[0335] candModeList[ 0 ] = INTRA_PLANAR (8-82)

[0336] candModeList[ 1 ] = INTRA_ANGULAR18 (8-83)

[0337] candModeList[ 2 ] = INTRA_ANGULAR25 (8-84)

[0338] candModeList[ 3 ] = INTRA_ANGULAR10 (8-85)

[0339] candModeList[ 4 ] = INTRA_ANGULAR65 (8-86)

[0340] candModeList[5]=INTRA_ANGULAR50 (8-87)

[0341] – Otherwise, if IntraSubPartitionsSplitType is equal to ISP_VER_SPLIT, the following applies:

[0342] candModeList[0]=INTRA_PLANAR (8-88)

[0343] candModeList[1]=INTRA_ANGULAR50 (8-89)

[0344] candModeList[2]=INTRA_ANGULAR43 (8-90)

[0345] candModeList[3]=INTRA_ANGULAR60 (8-91)

[0346] candModeList[4]=INTRA_ANGULAR3 (8-92)

[0347] candModeList[5]=INTRA_ANGULAR18 (8-93)

[0348] 5. Derive IntraPredModeY[ xCb ][ yCb ] by applying the following procedure:

[0349] – If intra_luma_mpm_flag[ xCb ][ yCb ] is equal to 1, set IntraPredModeY[ xCb ][ yCb ] equal to candModeList[ intra_luma_mpm_idx[ xCb ][ yCb ] ].

[0350] – Otherwise, derive IntraPredModeY[ xCb ][ yCb ] by applying the following sequential steps:

[0351] 1. If for i = 0…4 and for each i, j = (i+1)…5, candModeList[ i ]

[0352] is greater than candModeList[ j ], swap the two values as follows:

[0353] (candModeList[i], candModeList[j]) = Swap(candModeList[i], candModeList[j]) (8-94)

[0354] 2. Derive IntraPredModeY[ xCb ][ yCb ] by applying the following sequential steps:

[0355] i. Set IntraPredModeY[ xCb ][ yCb ] equal to

[0356] intra_luma_mpm_remainder[ xCb ][ yCb ].

[0357] ii. For i equal to 0 to 5, inclusive, if

[0358] IntraPredModeY[ xCb ][ yCb ] is greater than or equal to candModeList[ i ], then increase the value of IntraPredModeY[ xCb ][ yCb ] by one.

[0359] Set the variable IntraPredModeY[ x ][ y ] (x = xCb... xCb + cbWidth - 1 and y = yCb... yCb + cbHeight - 1) equal to IntraPredModeY[ xCb ][ yCb ].

[0360] 3. Examples of problems solved by embodiments

[0361] In LIC, two parameters including a scaling parameter and an offset b need to be derived by using neighboring reconstructed samples, which can cause a latency issue.

[0362] The set of allowed weight factors used in GBI is fixed, which can be less efficient.

[0363] CIIP mode is only applicable to non-skip Merge mode, which can be less efficient.

[0364] In the current design, the CIIP mode flag should be stored because the CIIP mode flag of the neighboring block is used for both intra mode derivation process for intra coded blocks and intra mode derivation process for CIIP coded blocks.

[0365] In the current design, the CIIP weight derivation can be further improved, for example, the checking of two neighboring blocks and three different weight pairs can be simplified.

[0366] In the current design, during CIIP weight derivation process, if the neighboring block is coded using BDPCM mode, it is considered as intra mode. However, it can be more reasonable to consider it as non-intra mode, since BDPCM is designed for screen content coding.

[0367] 4. Examples of embodiments

[0368] In the following, a block is used to represent a transform unit (TU) / prediction unit (PU) / coding unit (CU) / sub-block within a transform unit (TU) / prediction unit (PU) / coding unit (CU) / sub-block.

[0369] The following detailed technologies should be considered as examples to explain general concepts. These technologies should not be interpreted narrowly. Furthermore, these technologies can be combined in any manner.

[0370] Let (xCb, yCb) be the position of the top-left sample of the current block relative to the top-left sample of the current picture, and cbWidth and cbHeight specify the width and height of the current block.

[0371] 1. The CIIP flag or / and the LIC flag or / and the diffusion filter flag or / and the bilateral filter flag or / and the transform domain filter flag or / and the enabling flag of other types of post-reconstruction filter can be constrained to false (CIIP or / and LIC or / and diffusion filter or / and bilateral filter or / and transform domain filter can be implicitly disabled), depending on the coding mode of other blocks (such as neighboring or non-neighboring neighboring blocks and / or reference blocks).

[0372] a. When a flag is derived to false, the corresponding method can not be applied.

[0373] b. When a flag is derived to false, the signaling of the indication of the use of such mode can be skipped.

[0374] c. When a flag is derived to false, the indication of the use of such mode can still be signaled, but is constrained to false in a conforming bitstream, and the such mode is not applied.

[0375] d. In one example, whether to apply one or more proposed constraints (such as 1.a, 1.b and 1.c) can depend on the coding mode of all or some neighboring and / or non-neighboring neighboring rows or columns.

[0376] i. Alternatively, whether to apply such constraint can depend on at least N (N >= 1) samples in the neighboring or non-neighboring neighboring rows or columns that are not coded using a particular mode.

[0377] e. In one example, the neighboring and / or non-neighboring neighboring rows can include the above row and / or the top-right row.

[0378] f. In one example, the adjacent and / or non-adjacent neighboring columns can include the left column and / or the lower-left corner and / or the upper-left corner.

[0379] g. In one example, the particular mode of the block that is not the current block can include the intra mode and / or the CIIP mode and / or the CPR mode.

[0380] h. In one example, if any of the neighboring / non-adjacent blocks in the neighboring and / or non-adjacent row or column are coded with the particular mode (e.g., intra and / or CIIP mode and / or CPR mode), one or more of the proposed constraints (such as 1.a, 1.b, and 1.c) are applied.

[0381] i. In one example, if all of the neighboring / non-adjacent blocks in the neighboring and / or non-adjacent row or column are coded with the particular mode (e.g., intra and / or CIIP and / or CPR mode), one or more of the proposed constraints are applied.

[0382] j. In one example, if at least N of the neighboring / non-adjacent blocks in the neighboring or non-adjacent row or column are not coded with the particular mode (e.g., intra and / or CIIP mode and / or CPR mode), one or more of the proposed constraints (such as 1.a, 1.b, and 1.c) are not applied.

[0383] k. In one example, whether one or more of the proposed constraints (such as 1.a, 1.b, and 1.c) are applied can depend on the location of the current block.

[0384] i. In one example, if the current block is located at the top of the current CTU (the current block and its above neighboring block belong to different CTUs), the proposed constraints are not applied.

[0385] ii. In one example, if the current block is located at the left of the current CTU (the current block and its left neighboring block belong to different CTUs), the proposed constraints are not applied.

[0386] 2. The CIIP flag in the CIIP mode and / or the intra mode can be stored in a history-based motion vector prediction (HMVP) table together with the motion information.

[0387] a. In one example, when comparing the motion information of two candidates (such as the pruning process), the CIIP flag or / and the intra mode in the CIIP mode is not considered in the comparison.

[0388] b. In one example, when comparing the motion information of two candidates, the CIIP flag or / and the intra mode in the CIIP mode is considered in the comparison.

[0389] c. In one example, when the Merge candidate is from an entry in the HMVP table, the CIIP flag of that entry is also copied to the Merge candidate.

[0390] d. In one example, when the Merge candidate is from an entry in the HMVP table, the CIIP flag and the intra mode of that entry are also copied to the Merge candidate.

[0391] 3. CIIP can be performed for AMVP mode (AMVP mode or / and AMVP with SMVD mode)

[0392] a. In Merge mode or / and UMVE (also known as Merge with Motion Vector Difference, MMVD for short) mode of the current block, the CIIP flag and / or the intra mode in the CIIP mode of the neighboring / non-adjacent block can be inherited.

[0393] b. The CIIP flag and / or the intra mode in the CIIP mode can be signaled for non-skipped Merge mode or / and non-skipped UMVE mode, and in skipped Merge mode or / and skipped UMVE mode, the CIIP flag and / or the intra mode in the CIIP mode of the neighboring / non-adjacent block can be inherited.

[0394] c. The CIIP flag and / or the intra mode in the CIIP mode can be signaled for skipped Merge mode or / and skipped UMVE mode, and in non-skipped Merge mode or / and non-skipped UMVE mode, the CIIP flag and / or the intra mode in the CIIP mode of the neighboring / non-adjacent block can be inherited.

[0395] d. In one example, the CIIP flag and the intra mode can be signaled for AMVP mode.

[0396] e. In one example, in Merge mode or / and UMVE mode, the CIIP flag can be inherited, and if the CIIP flag is true, the intra mode can be further signaled if multiple intra modes are allowed in the CIIP.

[0397] f. In one example, the CIIP flag and the intra mode in the neighboring / non-adjacent block in Merge mode or / and UMVE mode can be inherited by the current block.

[0398] g. In one example, CIIP can be disabled for skipped mode.

[0399] h. In one example, when comparing two Merge candidates, the CIIP flag or / and the intra mode in the CIIP can not be considered in the comparison.

[0400] i. In one example, the CIIP flag in CIIP or / and the intra mode in the comparison can be considered in the comparison when comparing two Merge candidates.

[0401] 4. CIIP can be disabled for pair-wise prediction or combined bi-prediction or other kinds of virtual / artificial candidates (e.g., zero motion vector candidates).

[0402] a. Alternatively, CIIP can be enabled for pair-wise or combined bi-predictive Merge candidates if one of the two candidates contained in the pair-wise prediction or combined bi-prediction employs CIIP prediction.

[0403] i. In one example, the intra mode of the CIIP mode candidate can be inherited.

[0404] ii. In one example, the intra mode can be explicitly signaled.

[0405] b. Alternatively, CIIP can be enabled for pair-wise or combined bi-predictive Merge candidates if both of the two candidates contained in the pair-wise prediction or combined bi-prediction employ CIIP prediction.

[0406] i. In one example, the intra mode of one of the two candidates can be inherited.

[0407] ii. In one example, the intra mode can be derived from the intra modes of the two candidates, and

[0408] for pair-wise or combined bi-predictive Merge candidates.

[0409] iii. In one example, the intra mode can be explicitly signaled.

[0410] 5. The neighboring and / or non-adjacent spatial reconstructed samples required for CIIP mode or / and diffusion filter or / and bilateral filter or / and transform domain filter or / and other types of post-reconstruction filters can be replaced by the corresponding samples in the reference picture (such as the neighboring and / or non-adjacent spatial samples of the reference blocks, as shown in Figure 17 ).

[0411] a. In one example, if the current block is bi-predicted, the neighboring samples of the two reference blocks can be averaged to generate the final neighboring samples.

[0412] b. In one example, if the current block is bi-predicted, the neighboring samples of one of the two reference blocks can be used as the final neighboring samples.

[0413] c. In one example, if the current block is bi-predicted, if unequal-weighted GBI or weighted prediction or LIC is applied to the current block, a weighted average of the neighboring samples of the two reference blocks can be generated to generate the final neighboring samples.

[0414] d. In one example, the corresponding samples in the reference picture (e.g., the neighboring samples of the reference block) can be identified by the motion information of the current block.

[0415] e. In one example, the corresponding samples in the reference picture (e.g., the neighboring samples of the reference block) can be identified by the modified motion information of the current block. For example, the motion vector can be rounded to integer precision before used to identify the neighboring samples.

[0416] f. In one example, the corresponding samples in the reference picture (e.g., the neighboring samples of the reference block) are identified via motion vector in integer pixel precision.

[0417] i. In one example, the MV of the reference block is first rounded to integer pixel. The neighboring samples of the reference block are identified with the rounded MV.

[0418] ii. In one example, the proposed replacement is applied when the current block is coded with integer pixel motion information. Thus, no rounding is needed.

[0419] g. In one example, the proposed replacement can be applied only to certain color components, such as the luma component.

[0420] i. Alternatively, the proposed replacement can be applied to all color components.

[0421] 6. Propose to prohibit the use of previously coded CIIP flag for coding subsequent blocks.

[0422] a. In one example, the checking process of CIIP flag from a previously coded block is skipped to improve throughput.

[0423] b. In one example, the checking process of CIIP flag from a neighboring block in the intra mode derivation process is skipped.

[0424] i. In one example, for a neighboring block, if it is coded with CIIP mode, the associated intra mode can be set to a given mode (such as planar mode). Alternatively, in addition, the associated intra mode can be used in the MPM list derivation process.

[0425] ii. In one example, for a neighboring block, if it is coded with CIIP mode or regular inter mode, the associated intra mode can be set to a given mode (such as planar mode). Alternatively, the associated intra mode can be used for MPM list derivation process.

[0426] iii. In one example, the intra mode of a CIIP coded block can not be stored and can be considered as not available in the decoding process. Alternatively, the CIIP coded block can be treated in the same way as regular inter coded block.

[0427] c. Alternatively, it is also proposed to remove the CIIP flag from the memory to save the memory size needed to store the mode information.

[0428] 7. It is proposed to code the CIIP mode flag with bypass coding.

[0429] a. Alternatively, the CIIP mode flag can be context coded but not referring to the CIIP mode flag of the neighboring block.

[0430] 8. It is proposed that the method of weighting and in CIIP can depend on the color component.

[0431] a. For example, the weighting values in CIIP are different for the primary color component (e.g. G component) and other color components (e.g. B and R components).

[0432] b. For example, the weighting values in CIIP are different for the luma component and chroma components.

[0433] c. For example, the weighting values are equal on chroma components for inter prediction and intra prediction.

[0434] 9. It is proposed that when the weighting factor selection is according to the neighboring block, for a neighboring block, if coded with CIIP mode, it can be considered as inter coded block.

[0435] a. Alternatively, when the weighting factor selection is according to the neighboring block, for a neighboring block, if coded with CIIP mode, it can be considered as intra coded block.

[0436] 10. The proposed methods can be applied under certain conditions, such as block size, slice / picture / tile type or motion information.

[0437] a. In one example, the proposed methods are not allowed when the block size contains less than MxH samples (e.g. 16, 32 or 64 luma samples).

[0438] b. Alternatively, the proposed method is not allowed when the minimum dimension of the width or / and height of the block is less than or not greater than X. In one example, X is set to 8.

[0439] c. Alternatively, the proposed method is not allowed when the minimum dimension of the width or / and height of the block is not less than X. In one example, X is set to 8.

[0440] d. Alternatively, the proposed method is not allowed when the width of the block is >th1 or >=th1 and / or the height of the block is >th2 or >=th2. In one example, th1 and / or th2 is set to 8.

[0441] e. Alternatively, the proposed method is not allowed when the width of the block is <th1 or <=th1 and / or the height of the block is <th2 or <=th2. In one example, th1 and / or th2 is set to 8.

[0442] 11. The number of allowed weight pairs (wIntra, wInter) used in CIIP can be reduced from 3 to 2, denoted as (a, b) and (c, d).

[0443] a. In one example, the two pairs are defined as {(1, 3) and (3, 1)}; or {(1, 3) and (2, 2)}; or {(3, 1) and (2, 2)}; or {(3, 5) and (4, 4)} or {(5, 3) and (4, 4)}, or {(1, 7) and (4, 4)} or {(7, 1) and (4, 4)}.

[0444] b. In one example, the weight pair is determined based on only one neighboring block A.

[0445] i. In one example, if one neighboring block A is available and is intra coded, then (wIntra, wInter) is set to be equal to (a, b); otherwise, (wIntra, wInter) is set to be equal to (c, d).

[0446] ii. In one example, the block A is a left neighboring block.

[0447] iii. In one example, the block A is an above neighboring block.

[0448] c. In one example, the weight pair is determined based on two or more neighboring blocks, and at least one of the neighboring blocks satisfies certain conditions.

[0449] i. In one example, if at least one of the neighboring blocks A and B is available and is intra coded, then (wIntra, wInter) is set equal to (A, B); otherwise, (wIntra, wInter) is set equal to (c, d).

[0450] 1. In one example, block A is the left neighboring block and B is the above neighboring block.

[0451] d. In one example, the weight pair is determined based on two or more neighboring blocks, and each neighboring block satisfies the same condition.

[0452] i. In one example, if both neighboring blocks A and B are available and are intra coded, then (wIntra, wInter) is set equal to (A, B); otherwise, (wIntra, wInter) is set equal to (c, d).

[0453] 1. In one example, block A is the left neighboring block and B is the above neighboring block.

[0454] e. In one example, (a, b) and (c, d) can be set as follows:

[0455] 1. In one example, (a, b) is set to (2, 2) or (3, 1).

[0456] 2. In one example, (c, d) is set equal to (1, 3).

[0457] 3. In one example, at least one of the following two conditions is true: a is not equal to c, or b is not equal to d.

[0458] 4. (a, b) is not equal to (c, d).

[0459] f. In one example, the aforementioned neighboring block (e.g., A or B) is a spatial neighboring block or a temporal neighboring block.

[0460] i. In one example, the neighboring block A or B is a left (or top) neighboring block.

[0461] ii. In one example, the neighboring blocks A and B are the left and top neighboring blocks, respectively.

[0462] iii. In one example, the left neighboring block covers the position (xCb-1, yCb+cbHeight-1).

[0463] iv. In one example, the top neighboring block covers the position (xCb+cbWidth-1, yCb-1).

[0464] v. In one example, the left neighboring block covers the position (xCb-1, yCb).

[0465] vi. In one example, the top neighboring block covers the position (xCb, yCb-1).

[0466] g. For the above example, the CIIP prediction is formed as follows:

[0467] P CIIP = (wInter * P inter + wIntra * P intra + offset) » N

[0468] where offset is set to (1 « (N-1)) or 0, and N can be set to log2(wIntra + wInter).

[0469] 12. The number of allowed weight pairs (wIntra, wInter) used in CIIP can be reduced from 3 to 1.

[0470] a. In one example, one weight pair is defined as (1, 3) or (2, 2) or (1, 7) or (2, 6) or (3, 5) or (4, 4).

[0471] b. In one example, (wIntra, wInter) is set to equal (2, 2).

[0472] c. In one example, (wIntra, wInter) is set to equal (1, 3).

[0473] d. For the above example, the CIIP prediction is formed as follows:

[0474] P CIIP = (wInter * P inter + wIntra * P intra + offset) » N

[0475] where offset is set to (1 « (N-1)) or 0, and N can be set to log2(wIntra + wInter).

[0476] 13. The (wIntra, wInter) weight pair can be derived based on whether one or more neighboring blocks are coded using CIIP mode.

[0477] a. The weight pair for the current block can be derived from the weight pair used for a previous coded block.

[0478] b. In one example, the weight pair is determined based on only one neighboring block A.

[0479] i. In one example, if one neighboring block A is coded with CIIP mode, then (wIntra, wInter) for the current block is set equal to (wIntra, wInter) of neighboring block A.

[0480] Otherwise, if neighboring block A is available and is intra coded, then (wIntra, wInter) is set equal to (a, b); otherwise, (wIntra, wInter) is set equal to (c, d).

[0481] ii. In one example, if one neighboring block A is coded with CIIP mode, then (wIntra, wInter) for the current block is set equal to (wIntra, wInter) of neighboring block A; otherwise, (wIntra, wInter) is set equal to (a, b).

[0482] iii. For the above example, the following can apply:

[0483] 1. In one example, (a, b) is set equal to (2, 2) or (3, 1).

[0484] 2. In one example, (c, d) is set equal to (1, 3).

[0485] 3. In one example, at least one of the following two conditions is true: a is not equal to c; b is not equal to d.

[0486] c. In one example, the weights pairs are determined based on two or more neighboring blocks, and at least one of the neighboring blocks satisfies a certain condition.

[0487] i. Multiple neighboring blocks can be checked in a given checking order (e.g., first check the left block, then check the top block) to identify the use of CIIP mode.

[0488] ii. In one example, if at least one of neighboring blocks A and B is coded with CIIP mode, then (wIntra, wInter) for the current block is set equal to (wIntra, wInter) of the first CIIP coded neighboring block in a given checking order; otherwise, if at least one of A

[0489] and B is available and is intra coded, then (wIntra, wInter) is set equal to (a, b); otherwise, (wIntra, wInter) is set equal to (c, d).

[0490] iii. For the above example, the following can apply:

[0491] 1. In one example, (a, b) is set equal to (2, 2) or (3, 1).

[0492] 2. In one example, (c, d) is set equal to (1, 3).

[0493] 3. In one example, at least one of the following two conditions is true: a is not equal to c; b is not equal to d.

[0494] iv. In one example, if at least one of the neighboring blocks A and B is coded with CIIP mode, then (wIntra, wInter) for the current block is set equal to (wIntra, wInter) of the first CIIP coded neighboring block in the given checking order; otherwise, if both neighboring blocks A and B are available and are intra coded, then (wIntra, wInter) is set equal to (a, b); otherwise, (wIntra, wInter) is set equal to (c, d).

[0495] v. In one example, if at least one of the neighboring blocks A and B is coded with CIIP mode, then (wIntra, wInter) for the current block is set equal to (wIntra, wInter) of the first CIIP coded neighboring block in the given checking order; otherwise, (wIntra, wInter) is set equal to (c, d).

[0496] vi. For the above example, the following can apply:

[0497] 1. In one example, (c, d) is set equal to (2, 2) or (1, 3).

[0498] d. In one example, the above neighboring block A or B is a spatial or temporal neighboring block that is adjacent or non-adjacent.

[0499] i. In one example, the neighboring block A or B is a left (or top) neighboring block.

[0500] ii. In one example, the neighboring blocks A and B are a left neighboring block and a top neighboring block, respectively.

[0501] iii. In one example, the left neighboring block covers the position (xCb - 1, yCb + cbHeight - 1).

[0502] iv. In one example, the top neighboring block covers the position (xCb + cbWidth - 1, yCb - 1).

[0503] v. In one example, the left neighboring block covers the position (xCb - 1, yCb).

[0504] vi. In one example, the top neighboring block covers the position (xCb, yCb-1).

[0505] e. For the above example, the CIIP prediction is formed as follows:

[0506] P CIIP = (winter*P inter + wInter*P intra + offset) » N

[0507] where offset is set to (1 « (N-1)) or 0, and N can be set to log2(wIntra + wInter).

[0508] 14. The selection of the neighboring block used in CIIP can depend on coding information such as block dimension / block shape / low delay check flag / reference picture information / motion information of the current block and the neighboring block / intra prediction mode of the neighboring block.

[0509] 15. The weight pair used in CIIP can depend on coding information such as block dimension / block shape / low delay check flag / reference picture information / motion information of the current block and the neighboring block / intra prediction mode of the neighboring block.

[0510] a. In one example, a set of weight pairs is further signaled.

[0511] b. In one example, the selection of the weight pair can depend on whether the neighboring block is inter coded or intra coded.

[0512] 16. In determining the CIIP information (e.g., for the previous designs and the above items, in determining the weight applied to the intra prediction signal or the inter prediction signal), if the neighboring block is not coded with MODE_INTRA (i.e., intra mode), it can be treated as intra coded.

[0513] a. In one example, if the neighboring block is IBC coded.

[0514] b. In one example, if the neighboring block is CIIP coded.

[0515] c. In one example, if the neighboring block is TPM coded.

[0516] d. In one example, if the neighboring block is Palette coded.

[0517] e. In one example, if the neighboring block is RDPCM coded.

[0518] f. In one example, if the neighboring block is coded without applying transform (e.g., transform skip mode) and / or quantization (e.g., transform bypass quantization mode).

[0519] g. Alternatively, if the neighboring block is coded with the modes mentioned in the sub-items above (e.g., IBC, RDPCM, palette), the neighboring block can be considered as non-intra coded (e.g., inter coded).

[0520] h. In one example, if the neighboring block is intra coded but not coded with a specific prediction mode (e.g., planar), the neighboring block can be considered as non-intra coded (e.g., inter coded).

[0521] i. Alternatively, if the neighboring block is coded with matrix-based intra prediction method and / or multi-reference line intra prediction method (where the reference line index is not equal to K, e.g., K=0) and / or BDPCM, the neighboring block can be considered as non-intra coded (e.g., inter coded).

[0522] j. The proposed method can be applied to other coding tools, depending on whether the neighboring block is coded in intra mode or not.

[0523] k. The proposed method can be applied to other coding tools, depending on whether the neighboring block is coded in inter mode or not.

[0524] 17. Whether to enable or disable the above method can be signaled in SPS / PPS / VPS / sequence header / picture header / slice header / tile group header / CTU group, etc.

[0525] a. Alternatively, which method to use can be signaled in SPS / PPS / VPS / sequence header / picture header / slice header / tile group header / CTU group, etc.

[0526] b. Alternatively, whether to enable or disable the above method and / or which method to apply can depend on block dimension, video processing data unit (VPDU), picture type, low delay check flag, coding information of the current block or previous coded blocks (e.g., reference picture, uni-prediction or bi-prediction).

[0527] 5. Embodiments

[0528] 5.1 Embodiment 1

[0529] Examples of CIIP flag skipping checking previous coded blocks are given below.

[0530] 8.4.2 Derivation process of luma intra prediction mode

[0531] The input of this process is:

[0532] luma position ( xCb, yCb ) specifying the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture,

[0533] a variable cbWidth specifying the width of the current coding block in luma samples,

[0534] a variable cbHeight specifying the height of the current coding block in luma samples.

[0535] In this process, the luma intra prediction mode IntraPredModeY[ xCb ][ yCb ] is derived.

[0536] Table 8-1 specifies the values of the intra prediction mode IntraPredModeY[ xCb ][ yCb ] and the related names.

[0537] Table 8-1 Specification of intra prediction modes and related names

[0538] Intra prediction mode Correlation name 0 INTRA_PLANAR 1 INTRA_DC 2..66 INTRA_ANGULAR2..INTRA_ANGULAR66 81..83 INTRA_LT_CCLM, INTRA_L_CCLM, NTRA_T_CCLM

[0539] NOTE: The intra prediction modes INTRA LT CCLM, INTRA L CCLM and INTRA T CCLM are only applicable for chroma components.

[0540] IntraPredModeY[ xCb ][ yCb ] is derived by the following ordered steps:

[0541] 1. Set the neighbouring positions ( xNbA, yNbA ) and ( xNbB, yNbB ) equal to ( xCb - 1, yCb + cbHeight - 1 ) and ( xCb + cbWidth - 1, yCb - 1 ), respectively.

[0542] 2. For the case where X is replaced by A or B, the variable candIntraPredModeX is derived as follows:

[0543] - Set the position ( xCurr, yCurr ) equal to ( xCb, yCb ) and the neighbouring position ( xNbY, yNbY ) equal to ( xNbX, yNbX ) as inputs, invoke the availability derivation process of a block as specified in clause 6.4.X [Ed.(BB): pending neighbouring block availability check process] and assign the output to availableX.

[0544] - The candidate intra prediction mode candIntraPredModeX is derived as follows:

[0545] - candIntraPredModeX is set equal to INTRA_PLANAR if one or more of the following conditions are true.

[0546] - variable availableX is equal to FALSE.

[0547] - CuPredMode[ xNbX ][ yNbX ] is not equal to MODE_INTRA.

[0548] - pcm_flag[ xNbX ][ yNbX ] is equal to 1.

[0549] - X is equal to B and yCb - 1 is less than (( yCb » CtbLog2SizeY)

[0550] < CtbLog2SizeY ).

[0551] - Otherwise, candIntraPredModeX is set equal to IntraPredModeY[ xNbX ][ yNbX ].

[0552] 3. Variables ispDefaultMode1 and ispDefaultMode2 are defined as follows:

[0553] - If IntraSubPartitionsSplitType is equal to ISP_HOR_SPLIT, ispDefaultMode1 is set equal to INTRA_ANGULAR18 and ispDefaultMode2 is set equal to INTRA_ANGULAR5.

[0554] - Otherwise, ispDefaultMode1 is set equal to INTRA_ANGULAR50 and ispDefaultMode2 is set equal to INTRA_ANGULAR63.

[0555] 4. candModeList[ x ] ( x = 0...5 ) is derived as follows:

[0556] …

[0557] Embodiment 2

[0558] If the left neighbor is available and is intra coded, ( wIntra, wInter ) is set equal to ( 2, 2 ); otherwise, ( wIntra, wInter ) is set equal to ( 1, 3 ).

[0559] CIIP prediction is formed as follows:

[0560] PCIIP = (wInter * P inter + wIntra * P intra + 2) » 2

[0561] Example 3

[0562] If the left neighbor is available and is intra coded, then set (wIntra, wInter) to be equal to (3, 1); otherwise, set (wIntra, wInter) to be equal to (1, 3).

[0563] The CIIP prediction is formed as follows:

[0564] P CIIP = (wInter * P inter + wIntra * P intra + 2) » 2

[0565] Example 4

[0566] If the top neighbor is available and is intra coded, then set (wIntra, wInter) to be equal to (2, 2); otherwise, set (wIntra, wInter) to be equal to (1, 3).

[0567] The CIIP prediction is formed as follows:

[0568] P CIIP = (wInter * P inter + wIntra * P intra + 2) » 2

[0569] Example 5

[0570] If the top neighbor is available and is intra coded, then set (wIntra, wInter) to be equal to (3, 1); otherwise, set (wIntra, wInter) to be equal to (1, 3).

[0571] The CIIP prediction is formed as follows:

[0572] P CIIP = (wIntra * P inter + wIntra * P intra + 2) » 2

[0573] Example 6

[0574] If the left neighbor and the top neighbor are available and are intra coded, then set (wIntra, wInter) to be equal to (2, 2); otherwise, set (wIntra, wInter) to be equal to (1, 3).

[0575] The CIIP prediction is formed as follows:

[0576] P CIIP = (wInter * P inter + wIntra * P intra + 2) » 2

[0577] Example 7

[0578] If the left and top neighbors are available and intra coded, set (wIntra, wInter) to equal (3, 1); otherwise, set (wIntra, wInter) to equal (1, 3).

[0579] The CIIP prediction is formed as follows:

[0580] P CIIP = (wInter * P inter + wIntra * P intra + 2) » 2

[0581] Example 8

[0582] If the left or top neighbor is available and intra coded, set (wIntra, wInter) to equal (2, 2); otherwise, set (wIntra, wInter) to equal (1, 3).

[0583] The CIIP prediction is formed as follows:

[0584] P CIIP = (wInter * P inter + wIntra * P intra + 2) » 2

[0585] Example 9

[0586] If the left or top neighbor is available and intra coded, set (wIntra, wInter) to equal (3, 1); otherwise, set (wIntra, wInter) to equal (1, 3).

[0587] The CIIP prediction is formed as follows:

[0588] P CIIP = (wInter * P inter + wIntra * P intra + 2) » 2

[0589] Example 10

[0590] Set (wIntra, wInter) to equal (2, 2).

[0591] The CIIP prediction is formed as follows:

[0592] P CIIP = (wInter * P inter + wIntra * P intra + 2) » 2

[0593] Example 11

[0594] Set (wIntra, wInter) to be equal to (1, 3).

[0595] The CIIP prediction is formed as follows:

[0596] P CIIP = (wInter * P inter + wIntra * P intra + 2) » 2

[0597] Example 12

[0598] If the left neighbor is CIIP coded, set (wIntra, wInter) for the current block to be equal to (wIntra, wInter) of the left neighbor; otherwise, if the left neighbor is available and intra coded, set (wIntra, wInter) to be equal to (3, 1); otherwise, set (wIntra, wInter) to be equal to (1, 3).

[0599] The CIIP prediction is formed as follows:

[0600] P CIIP = (wInter * P inter + wInter * P inter + 2) » 2

[0601] Example 13

[0602] If the left neighbor is CIIP coded, set (wIntra, wInter) for the current block to be equal to (wIntra, wInter) of the left neighbor; otherwise, if the left neighbor is available and intra coded, set (wIntra, wInter) to be equal to (2, 2); otherwise, set (wIntra, wInter) to be equal to (1, 3).

[0603] The CIIP prediction is formed as follows:

[0604] P CIIP = (wInter * P inter + wIntra * P intra + 2) » 2

[0605] Example 14

[0606] If the top neighbor is CIIP coded, set (wIntra, wInter) for the current block equal to (wIntra, wInter) of the top neighbor; otherwise, if the top neighbor is available and intra coded, set (wIntra, wInter) equal to (3, 1); otherwise, set (wIntra, wInter) equal to (1, 3).

[0607] The CIIP prediction is formed as follows:

[0608] P CIIP = (wInter * P inter + wIntra * P intra + 2) » 2

[0609] Example 15

[0610] If the top neighbor is CIIP coded, set (wIntra, wInter) for the current block equal to (wIntra, wInter) of the top neighbor; otherwise, if the top neighbor is available and intra coded, set (wIntra, wInter) equal to (2, 2); otherwise, set (wIntra, wInter) equal to (1, 3).

[0611] The CIIP prediction is formed as follows:

[0612] P CIIP = (wInter * P inter + wIntra * P intra + 2) » 2

[0613] Example 16

[0614] If the left or / and top neighbor is CIIP coded, set (wIntra, wInter) for the current block equal to (wIntra, wInter) of the first CIIP coded neighbor (from left to top); otherwise, if the left or / and top neighbor is available and intra coded, set (wIntra, wInter) equal to (3, 1); otherwise, set (wIntra, wInter) equal to (1, 3).

[0615] The CIIP prediction is formed as follows:

[0616] P CIIP = (wInter * P inter + wIntra * P intra + 2) » 2

[0617] Example 17

[0618] If the left or / and top neighbor is CIIP coded, set (wIntra, wInter) of the current block equal to (wIntra, wInter) of the first CIIP coded neighbor (from left to top); otherwise, if the left or / and top neighbor is available and is intra coded, set (wIntra, wInter) equal to (2, 2); otherwise, set (wIntra, wInter) equal to (1, 3).

[0619] The CIIP prediction is formed as follows:

[0620] P CIIP = (wInter * P inter + wIntra * P intra + 2) » 2

[0621] Example 18

[0622] If the left neighbor is CIIP coded, set (wIntra, wInter) of the current block equal to (wIntra, wInter) of the left neighbor; otherwise, set (wIntra, wInter) equal to (1, 3).

[0623] The CIIP prediction is formed as follows:

[0624] P CIIP = (wIntra * P inter + wIntra * P intra + 2) » 2

[0625] Example 19

[0626] If the left neighbor is CIIP coded, set (wIntra, wInter) of the current block equal to (wIntra, wInter) of the left neighbor; otherwise, set (wIntra, wInter) equal to (2, 2).

[0627] The CIIP prediction is formed as follows:

[0628] P CIIP = (wInter * P inter + wIntra * P intra + 2) » 2

[0629] Example 20

[0630] If the top neighbor is CIIP coded, set (wIntra, wInter) for the current block equal to (wIntra, wInter) of the top neighbor; otherwise, set (wIntra, wInter) equal to (1, 3).

[0631] CIIP prediction is formed as follows:

[0632] P CIIP = (wInter * P inter + wIntra * P intra + 2) » 2

[0633] Example 21

[0634] If the top neighbor is CIIP coded, set (wIntra, wInter) for the current block equal to (wIntra, wInter) of the top neighbor; otherwise, set (wIntra, wInter) equal to (2, 2).

[0635] CIIP prediction is formed as follows:

[0636] P CIIP = (wIntra * P inter + wIntra * P intra + 2) » 2

[0637] Example 22

[0638] If the left or / and top neighbor is CIIP coded, set (wIntra, wInter) for the current block equal to (wIntra, wInter) of the first CIIP coded neighbor (from left to top); otherwise, set (wIntra, wInter) equal to (1, 3).

[0639] CIIP prediction is formed as follows:

[0640] P CIIP = (wInter * P inter + wIntra * P intra + 2) » 2

[0641] Example 23

[0642] If the left or / and top neighbor is / are CIIP coded, set (wIntra, wInter) for the current block equal to (wIntra, wInter) of the first CIIP coded neighbor (from left to top); otherwise, set (wIntra, wInter) equal to (2, 2).

[0643] The CIIP prediction is formed as follows:

[0644] P CIIP = (wInter * P inter + wIntra * P intra + 2) » 2

[0645] Embodiment 24

[0646] 8.5.6 Decoding process of inter-block

[0647] 8.5.6.1 Overview

[0648] This process is invoked when decoding a coding unit coded in inter-prediction mode.

[0649] When ciip_flag[ xCb ][ yCb ] is equal to 1, the array of prediction samples, predSamples, is modified as follows:

[0650] - If cldx is equal to 0, the following applies:

[0651] - Set the position ( xTbCmp, yTbCmp ) to be equal to ( xCb, yCb ), the intra-prediction mode predModeIntra to be equal to IntraPredModeY[ xCb ][ yCb ], the transform block width nTbW and height nTbH to be equal to cbWidth and cbHeight, the coding block width nCbW and height nCbH to be equal to cbWidth and cbHeight, the variable cldx as input, invoke the regular intra-sample prediction process specified in clause 8.4.5.2.5 and assign the output to the ( cbWidth ) x ( cbHeight ) array predSamplesIntra L .

[0652] - Set the position ( xTbCmp, yTbCmp ) to be equal to ( xCb, yCb ), the coding block width cbWidth, the coding block height cbHeight, the array of samples predSamplesInter and predSamplesIntra to be equal to predSamples and predSamplesIntra L , the intra-prediction mode predModeIntra to be equal to IntraPredModeY[ xCb ][ yCb ], and the color component index cldx as input, invoke the combined Merge and intra-prediction weighted sample prediction process specified in clause 8.5.6.7 and assign the output to the ( cbWidth ) x ( cbHeight ) array predSamples.

[0653] …

[0654] 8.5.6.7 Weighted sample prediction process combining Merge and intra prediction

[0655] The input of this process is:

[0656] - the luma position (xCb, yCb) specifying the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture,

[0657] - the width cbWidth of the current coding block,

[0658] - the height cbHeight of the current coding block,

[0659] - two (cbWidth) x (cbHeight) arrays predSamplesInter and predSamplesIntra,

[0660] - the variable cldx specifying the colour component index.

[0661] The output of this process is a (cbWidth) x (cbHeight) array predSamplesComb of predicted sample values.

[0662] The variable bitDepth is derived as follows:

[0663] - if cldx is equal to 0, bitDepth is set equal to BitDepth Y .

[0664] - otherwise, bitDepth is set equal to BitDepth C .

[0665] The variable callFact is derived as follows:

[0666] scallFact = ( cldx = = 0 )? 0 : 1. (8-838)

[0667] The neighbouring luma positions (xNbA, yNbA) and (xNbB, yNbB) are set equal to (xCb - 1, yCb - 1 + (cbHeight « scallFact)) and (xCb - 1 + (cbWidth « scallFact), yCb - 1), respectively.

[0668] For the case where X is replaced by A or B, the variables availableX and isIntraCodedNeighbourX are derived as follows:

[0669] - Set the position (xCurr, yCurr) equal to (xCb, yCb), the neighbouring position (xNbY, yNbY) equal to (xNbX, yNbX), checkPredModeY to FALSE, and cIdx equal to 0 as inputs, invoke the derivation process of the availability of a neighbouring block as specified in clause 6.4.4 and assign the output to availableX.

[0670] - The variable isIntraCodedNeighbourX is derived as follows:

[0671] - If availableX is equal to TRUE, and CuPredMode[0][xNbX][yNbX] is equal to MODE_INTRA, and BdpcmFlag[xNbX][yNbX] is equal to 0, then set isIntraCodedNeighbourX equal to TRUE.

[0672] - Otherwise, set isIntraCodedNeighbourX equal to FALSE.

[0673] The weight w is derived as follows:

[0674] - If both isIntracodedNeighbourA and isIntraCodedNeighbourB are equal to TRUE, then set w equal to 3.

[0675] - Otherwise, if both isIntracodedNeighbourA and isIntraCodedNeighbourB are equal to FALSE, then set w equal to 1.

[0676] - Otherwise, set w equal to 2.

[0677] When cIdx is equal to 0 and slice_lmcs_enabled_flag is equal to 1, predSamplesInter[ x ][ y ] (x = 0..cbWidth - 1 and y = 0..cbHeight - 1) is modified as follows:

[0678] Figure 18is a block diagram of a video processing apparatus 1800. The apparatus 1800 can be used to implement one or more methods described herein. The apparatus 1800 can be implemented in a smartphone, a tablet computer, a computer, an Internet of Things (IoT) receiver, etc. The apparatus 1800 can include one or more processors 1802, one or more memories 1804, and video processing hardware 1806. The processor(s) 1802 can be configured to implement one or more methods described herein. The memory (memories) 1804 can be used for storing data and code used during the operation of the apparatus 1800. The video processing hardware 1806 can be used to implement, in hardware circuitry, some of the techniques described herein.

[0679] Figure 20 is a flowchart of a method 2000 of processing a video. The method 2000 includes determining (2005) a coding mode of a first video block, constraining (2110) one or more flags to an operational state based on the determination of the coding mode of the first video block, the operational state being false or true, and performing (2015) further processing of a second video block according to the operational state of the one or more flags, wherein the first video block is a neighboring video block or a reference video block related to the second video block.

[0680] Figure 21 is a flowchart of a method 2100 of processing a video. The method 2100 includes determining (2105) to use paired prediction or combined bi-prediction for a first video block, determining (2110) an operational state of combined inter-intra prediction (CIIP) based on the determination to use paired prediction or combined bi-prediction, the operational state being enabled or disabled, and performing (2115) further processing of the first video block according to the operational state of the CIIP.

[0681] Figure 23 is a flowchart of a method 2300 of processing a video. The method 2300 includes determining (2305) one or more coding modes of one or more second blocks during a conversion between a first block in video data and a bitstream representation of the first block, determining (2310) a coding mode constraint of the first block based on the one or more coding modes of the one or more second blocks, performing (2315) the conversion at least by applying the coding mode constraint of the first block, wherein the one or more second blocks include at least one of a neighboring block, a non-neighboring block, and a reference block of the first block.

[0682] Figure 24is a flowchart of a method 2400 of processing video. The method 2400 includes performing (2405) a conversion between a current block in video data and a bitstream representation of the current block using at least one of combined inter-intra prediction (CIIP), diffusion filtering, bilateral filtering, transform domain filtering, or another type of post-reconstruction filtering different from diffusion filtering, bilateral filtering, and transform domain filtering, wherein reconstructed neighboring samples of the current block used in the at least one of combined inter-intra prediction (CIIP), diffusion filtering, bilateral filtering, transform domain filtering, or another type of post-reconstruction filtering are replaced with approximated samples generated from corresponding samples of reconstructed neighboring samples in one or more reference pictures.

[0683] Figure 25 is a flowchart of a method 2500 of processing video. The method 2500 includes storing (2505), in a history-based motion vector prediction (HMVP) table with motion information, a CIIP flag of a combined inter-intra prediction (CIIP) mode and / or an intra mode; and performing (2510) a conversion between a current block in video data and a bitstream representation of the current block based at least on the HMVP table.

[0684] Figure 26 is a flowchart of a method 2600 of processing video. The method 2600 includes, during a conversion between a current block in video data and a bitstream representation of the current block, determining (2605) a prediction mode of the current block; determining (2610) an applicability of a combined inter-intra prediction (CIIP) mode, the applicability indicating that the CIIP mode is enabled for the current block in response to determining that the current block is coded using an advanced motion vector prediction (AMVP) mode or a Merge mode; and performing (2615) the conversion based on the applicability of the CIIP mode.

[0685] Figure 27 is a flowchart of a method 2700 of processing video. The method 2700 includes, during a conversion between a current block in video data and a bitstream representation of the current block, determining (2705) a type of a selected Merge candidate of the current block; determining (2710) an applicability of combined inter-intra prediction (CIIP) for the current block according to the type of the selected Merge candidate, wherein the current block is coded in a Merge mode.

[0686] With reference to the methods 2000 and 2100, some examples of combined inter-intra prediction for encoding and their use are described in Section 4 herein. For example, as described in Section 4, a video block can be processed according to a combined intra-inter prediction use.

[0687] Referring to the methods 2000 and 2100, a video block can be encoded in a video bitstream, where bit efficiency can be achieved by using bitstream generation rules related to combined inter-intra prediction.

[0688] The method can include where the one or more flags include a combined inter-intra prediction (CIIP) flag, a local illumination compensation (LIC) flag, a diffusion filtering flag, a bilateral filtering flag, a transform domain filtering flag, or another type of post-reconstruction filtering flag.

[0689] The method can include where the one or more flags include a first flag related to the first operation, the first flag is derived to be false, and the first operation is not applied.

[0690] The method can include where use of the first operation is signaled as skipped.

[0691] The method can include where use of the first operation is signaled as constrained to be false in a conforming bitstream.

[0692] The method can include where the first video block is within a proximate row or column relative to the second video block.

[0693] The method can include where the proximate row includes an above row or an above-right row.

[0694] The method can include where the proximate column includes a left column, a below-left column, or a top-left column.

[0695] The method can include where the coding mode includes an intra mode, a CIIP mode, or a CPR mode.

[0696] The method can include determining a position of the second video block, and where constraining the one or more flags to a false state is also based on the determination of the position of the second video block.

[0697] The method can include where the position is at a top of a current coding tree unit (CTU) and an above proximate block of the second video block is within a different CTU.

[0698] The method can include where the position is at a left side of a current coding tree unit (CTU) and a left proximate block of the second video block is within a different CTU.

[0699] The method can include where the one or more flags include a CIIP flag or an intra mode of a CIIP mode flag, and data related to the one or more flags is stored in a history-based motion vector prediction (HMVP) table along with motion information.

[0700] The method may include: comparing two candidate motion information, wherein the CIIP flag or intra-frame mode CIIP mode flag is not used in the comparison of motion information.

[0701] The method may include comparing two candidate motion information, wherein the comparison of motion information uses a CIIP flag or an intra-frame mode CIIP mode flag.

[0702] The method may include: determining that the Merge candidate comes from an entry in the HMVP table; and copying the CIIP flag of the entry to the Merge candidate.

[0703] The method may include: determining that the Merge candidate is an entry from the HMVP table; and copying the CIIP flag and intra-frame mode of that entry to the Merge candidate.

[0704] The method may include: wherein one or more flags are associated with CIIP, and CIIP is performed for Advanced Motion Vector Prediction (AMVP).

[0705] The method may include: wherein, AMVP mode signaling notifies the CIIP flag and the intra-frame mode flag.

[0706] The method may include: inheriting the CIIP flag in Merge mode or UMVE mode, the CIIP flag including an operation state of true, multiple intra-frame modes allowed in CIIP, and signaling the intra-frame mode based on the multiple intra-frame modes allowed in CIIP.

[0707] The method may include: wherein the second video block inherits the CIIP flag and intra-frame mode of adjacent or non-adjacent video blocks in Merge mode or UMVE mode.

[0708] The method may include: wherein CIIP is disabled for skip mode.

[0709] The method may include: comparing information related to two candidates, wherein the CIIP flag or intra-frame mode CIIP mode flag is not used in the comparison of information.

[0710] The method may include: comparing information associated with two candidates, wherein the comparison of information uses a CIIP flag or an intra-frame mode CIIP mode flag.

[0711] The method may include: wherein one or more candidates involved in pairwise prediction or combined bidirectional prediction are predicted using CIIP, and wherein the operation state is enabled.

[0712] The method may include: wherein the intra-frame mode of one of the two candidates is inherited.

[0713] The method can include wherein the intra mode is signaled.

[0714] The method can include wherein the pair-wise prediction or the combined bi-prediction involves two candidates.

[0715] The method can include wherein the reconstructed samples used in the CIIP mode, the diffusion filter, the bilateral filter, the transform domain filter, or other types of post-reconstruction filters are replaced with samples in the reference picture.

[0716] The method can include wherein the second video block is bi-predicted and the neighboring samples of the two reference blocks are averaged to generate the final neighboring samples.

[0717] The method can include wherein the second video block is bi-predicted and the neighboring samples of the two reference blocks are used as the final neighboring samples.

[0718] The method can include wherein the second video block is bi-predicted and the neighboring samples of the two reference blocks are weighted averaged based on unequal weights generalized bi-prediction (GBI) to generate the final neighboring samples or weighted prediction LIC is applied to the second video block.

[0719] The method can include wherein the samples in the reference picture are identified with motion information of the second video block.

[0720] The method can include wherein the samples in the reference picture are identified with motion information of the second video block.

[0721] The method can include wherein the motion vector is rounded to integer precision and used to identify the samples.

[0722] The method can include wherein the samples are in the reference picture identified with the motion vector at integer pixel precision.

[0723] The method can include determining a characteristic of the second video block, the characteristic including one or more of a block size, a slice type, a picture type, a tile type, or motion information, and wherein the operational state is constrained based on the determination of the characteristic.

[0724] The method can include wherein the method is signaled in a sequence parameter set (SPS), a view parameter set (VPS), a picture parameter set (PPS), a sequence header, a picture header, a slice header, a tile group header, a tile, or a CTU group.

[0725] The following list of solutions further provides embodiments and variants of the items listed in the previous sections (e.g., items 11-16).

[0726] 40. A method of video processing, comprising performing a conversion between a video block of a video and a coded representation of the video block using a combined inter-intra prediction mode, wherein the coded representation corresponds to a weighted average of intra and inter prediction results of the video block using a pair of weights from a set of pairs of weights, the set of pairs of weights containing less than three pairs.

[0727] 41. The method of solution 40, wherein the set of pairs of weights corresponds to two pairs.

[0728] 42. The method of any of solutions 40 to 41, wherein the pair of weights is determined based on a single neighboring block.

[0729] 43. The method of any of solutions 40 to 41, wherein the pair of weights is determined based on multiple neighboring blocks.

[0730] 44. The method of solution 40, wherein the set of pairs of weights includes exactly one pair.

[0731] 45. The method of solution 44, wherein the one pair is one of (1, 3) or (2, 2) or (1, 7) or (2, 6) or (3, 5) or (4, 4).

[0732] 46. A method of video processing, comprising performing a conversion between a video block of a video and a coded representation of the video block using a combined inter-intra prediction mode, wherein the coded representation corresponds to a weighted average of intra and inter prediction results of the video block using a pair of weights from a set of pairs of weights, the set of pairs of weights being determined from coded information of one or more neighboring blocks.

[0733] 47. The method of solution 46, wherein the set of pairs of weights is determined from exactly one neighboring block.

[0734] 48. The method of solution 46, wherein the set of pairs of weights is determined from two or more neighboring blocks.

[0735] 49. The method of any of solutions 40 to 48, wherein the conversion includes encoding the video to generate the coded representation.

[0736] 50. The method of any of solutions 40 to 48, wherein the conversion includes decoding the coded representation to generate the video.

[0737] Various other embodiments and additional features of these methods can be described using the following clause-based recitations.

[0738] 1. A method of video processing, comprising:

[0739] determining one or more coding modes of one or more second blocks during a conversion between a first block in video data and a bitstream representation of the first block;

[0740] determining a coding mode constraint of the first block based on the one or more coding modes of the one or more second blocks; and

[0741] performing the conversion at least by applying the coding mode constraint of the first block;

[0742] wherein the one or more second blocks comprise at least one of a neighboring block, a non-neighboring block, and a reference block of the first block.

[0743] 2. The method of clause 1, wherein determining the coding mode constraint of the first block comprises:

[0744] determining one or more flags respectively corresponding to one or more coding modes of the first block based on the one or more coding modes of the one or more second blocks.

[0745] 3. The method of clause 2, wherein the one or more coding modes comprise:

[0746] a combined inter-intra prediction (CIIP) mode, a local illumination compensation (LIC) mode, a diffusion filtering mode, a bilateral filtering mode, a transform domain filtering mode, or a post-reconstruction filtering mode different from the diffusion filtering mode, the bilateral filtering mode, and the transform domain filtering mode.

[0747] 4. The method of any of clauses 2-3, wherein the coding mode constraint of the first block comprises:

[0748] when a first flag corresponding to a first coding mode is derived to be false, the first coding mode of the one or more coding modes of the first block is disabled.

[0749] 5. The method of any of clauses 2-4, wherein the coding mode constraint of the first block comprises:

[0750] when a second flag corresponding to a second coding mode is derived to be false, information indicating whether the second coding mode of the one or more coding modes of the first block is enabled is not signaled.

[0751] 6. The method of any of clauses 2-5, wherein the coding mode constraint of the first block comprises:

[0752] When a third flag corresponding to a third coding mode is derived as false, signaling information indicating whether the third coding mode of the one or more coding modes of the first block is enabled and constraining it to be false.

[0753] 7. The method of any of clauses 1-6, further comprising:

[0754] determining whether to apply the coding mode constraint of the first block.

[0755] 8. The method of clause 7, wherein determining whether to apply the coding mode constraint of the first block comprises:

[0756] determining whether to apply the coding mode constraint of the first block according to coding modes of neighboring rows or columns of the first block.

[0757] 9. The method of clause 7, wherein determining whether to apply the coding mode constraint of the first block comprises:

[0758] determining whether to apply the coding mode constraint of the first block according to at least N samples of neighboring rows or neighboring columns of the first block that are not coded with the fourth mode, and N > 1.

[0759] 10. The method of clause 8 or 9, wherein the neighboring rows comprise an above row and an upper-right row of the first block.

[0760] 11. The method of any of clauses 8-10, wherein the neighboring columns comprise a left column, a lower-left column, and an upper-left column of the first block.

[0761] 12. The method of any of clauses 7-11, wherein determining whether to apply the coding mode constraint of the first block comprises:

[0762] determining to apply the coding mode constraint of the first block when any neighboring non-adjacent block is coded with the fourth mode.

[0763] 13. The method of any of clauses 7-11, wherein determining whether to apply the coding mode constraint of the first block comprises:

[0764] determining to apply the coding mode constraint of the first block when all neighboring non-adjacent blocks are coded with the fourth mode.

[0765] 14. The method of any of clauses 7-11, wherein determining whether to apply the coding mode constraint of the first block comprises:

[0766] determining not to apply the coding mode constraint for the first block when at least M neighboring non-adjacent blocks are not coded with the fourth mode, and M is a predefined first threshold.

[0767] 15. The method of any of clauses 9-14, wherein the fourth coding mode comprises at least one of an intra prediction mode, a combined inter-intra prediction (CIIP) mode, and a current picture reference (CPR) mode.

[0768] 16. The method of any of clauses 7-14, wherein determining whether to apply the coding mode constraint for the current block comprises:

[0769] determining whether to apply the coding mode constraint for the first block according to a position of the first block.

[0770] 17. The method of clause 16, wherein determining whether to apply the coding mode constraint for the first block according to a position of the first block comprises:

[0771] determining not to apply the coding mode constraint for the first block when the position is at the top of a current coding tree unit (CTU), and an above neighboring block and the first block belong to different CTUs.

[0772] 18. The method of clause 16, wherein determining whether to apply the coding mode constraint for the first block according to a position of the first block comprises:

[0773] determining not to apply the coding mode constraint for the first block when the position is at the left of a current coding tree unit (CTU), and a left neighboring block and the first block belong to different CTUs.

[0774] 19. The method of any of clauses 1-18, further comprising:

[0775] determining a characteristic of the current block; and

[0776] determining to apply the coding mode constraint for the current block when the characteristic of the current block satisfies a predefined condition.

[0777] 20. The method of any of clauses 1-18, further comprising:

[0778] determining a characteristic of the current block and / or a previously coded block; and

[0779] determining whether to apply the coding mode constraint for the current block according to the characteristic of the current block and / or a previously coded block.

[0780] 21. A method of video processing, comprising:

[0781] performing a conversion between a current block in video data and a bitstream representation of the current block using at least one of combined inter-intra prediction (CIIP), diffusion filtering, bilateral filtering, transform domain filtering, or another type of post-reconstruction filtering different from the diffusion filtering, the bilateral filtering, and the transform domain filtering,

[0782] wherein the reconstructed neighboring samples of the current block used in the at least one of combined inter-intra prediction (CIIP), diffusion filtering, bilateral filtering, transform domain filtering, or another type of post-reconstruction filtering are replaced with an approximate sample generated from a corresponding sample of reconstructed neighboring samples in one or more reference pictures.

[0783] 22. The method of clause 21, wherein when the current block is bi-predicted, neighboring samples of two reference blocks of the current block are averaged to generate the approximate sample.

[0784] 23. The method of clause 21, wherein when the current block is bi-predicted or uni-predicted, a neighboring sample of one reference block of the current block is used as the approximate sample.

[0785] 24. The method of clause 21, wherein when the current block is bi-predicted and at least one of unequal-weighted generalized bi-prediction (GBI), weighted prediction, and local illumination compensation (LIC) is applied to the current block, neighboring samples of two reference blocks of the current block are weighted averaged to generate the approximate sample.

[0786] 25. The method of clauses 22-24, wherein the neighboring samples of the reference blocks of the current block are identified by motion information of the current block.

[0787] 26. The method of clauses 22-24, wherein the neighboring samples of the reference blocks of the current block are identified by modified motion information of the current block.

[0788] 27. The method of clause 24, wherein the modified motion information of the current block is a modified motion vector that is rounded to integer precision.

[0789] 28. The method of clause 21, wherein the neighboring samples of the reference blocks of the current block are identified by motion vectors in integer pixel precision.

[0790] 29. The method of clause 28, wherein motion vectors of the reconstructed neighboring samples are rounded to integer precision to identify the corresponding samples.

[0791] 30. The method of any of clauses 21-29, wherein the reconstructed neighboring samples of the current block are replaced with the approximate samples only when the current block is coded with integer pixel precision.

[0792] 31. The method of any of clauses 21-30, wherein the reconstructed neighboring samples of the current block are replaced with the approximate samples only for luma components or chroma components.

[0793] 32. The method of any of clauses 19-31, further comprising:

[0794] determining a characteristic of the current block; and

[0795] determining to replace the reconstructed neighboring samples with the approximate samples when the characteristic of the current block satisfies a predefined condition.

[0796] 33. The method of clause 19 or 32, wherein the characteristic of the current block comprises at least one of a block size, a slice type, a picture type, a tile type, and motion information.

[0797] 34. The method of clause 33, wherein the predefined condition is that the current block contains no less than a predefined second threshold of samples.

[0798] 35. The method of clause 33, wherein the predefined condition is that a width and a height of the current block are greater than a predefined third threshold.

[0799] 36. The method of clause 33, wherein the predefined condition is that a width and a height of the current block are less than a predefined fourth threshold.

[0800] 37. The method of clause 33, wherein the predefined condition is that a width of the current block is less than a predefined fifth threshold and / or a height of the current block is less than a predefined sixth threshold.

[0801] 38. The method of clause 33, wherein the predefined condition is that a width of the current block is greater than a predefined seventh threshold and / or a height of the current block is greater than a predefined eighth threshold.

[0802] 39. The method of clause 38, wherein information indicating whether the coding mode constraint of the current block is applied and / or whether the reconstructed neighboring samples are replaced with the approximate samples is signaled in a sequence parameter set (SPS), a view parameter set (VPS), a picture parameter set (PPS), a sequence header, a picture header, a slice header, a tile group header, a tile, or a CTU group.

[0803] 40. The method of any of clauses 21-39, further comprising:

[0804] determining a characteristic of the current block and / or a previously coded block; and

[0805] determining whether to replace the reconstructed neighboring sample with the approximated sample based on the characteristic of the current block and / or a previously coded block.

[0806] 41. The method of clause 20 or 40, wherein the characteristic of the current block and / or a previously coded block comprises at least one of a block dimension, a video processing data unit (VPDU), a picture type, a low delay check flag, coding information of the current block and / or a previously coded block.

[0807] 42. The method of clause 41, wherein the coding information of the current block and / or a previously coded block indicates a reference picture, a uni-prediction, or a bi-prediction.

[0808] 43. A video processing apparatus comprising a processor configured to implement a method recited by any of clauses 1-42.

[0809] 44. The apparatus of clause 43, wherein the apparatus is a video encoder.

[0810] 45. The apparatus of clause 43, wherein the apparatus is a video decoder.

[0811] 46. A computer-readable recording medium having recorded thereon a program comprising code that causes a processor to execute a method recited by any of clauses 1-42.

[0812] Various other embodiments and additional features of these methods can be described using the following clause-based recitations.

[0813] 1. A method of video processing, comprising:

[0814] storing, in a history-based motion vector prediction (HMVP) table with motion information, a combined inter-intra prediction (CIIP) flag and / or an intra mode of a CIIP mode; and

[0815] performing a conversion between a current block in video data and a bitstream representation of the current block based at least on the HMVP table.

[0816] 2. The method of clause 1, further comprising:

[0817] comparing motion information of two candidates,

[0818] wherein the CIIP flag and / or the intra mode of the CIIP mode are not used in the comparison of the two candidates of motion information.

[0819] 3. The method of clause 1, further comprising:

[0820] comparing two candidates of motion information,

[0821] wherein the CIIP flag and / or the intra mode of the CIIP mode are used in the comparison of the two candidates of motion information.

[0822] 4. The method of any of clauses 1 to 3, wherein the performing comprises:

[0823] when a Merge candidate is from an entry in the HMVP table, copying the CIIP flag of the entry to the Merge candidate.

[0824] 5. The method of any of clauses 1 to 3, wherein the performing comprises:

[0825] when a Merge candidate is from an entry in the HMVP table, copying the CIIP flag and the intra mode of the entry to the Merge candidate.

[0826] 6. A method of video processing, comprising:

[0827] during a conversion between a current block in video data and a bitstream representation of the current block, determining a prediction mode of the current block;

[0828] determining applicability of a combined inter-intra prediction (CIIP) mode, the applicability indicating that the CIIP mode is enabled for the current block in response to determining that the current block is coded in an advanced motion vector prediction (AMVP) mode or a Merge mode; and

[0829] performing the conversion based on the applicability of the CIIP mode.

[0830] 7. The method of clause 6, wherein when the current block is coded in the AMVP mode, a CIIP flag and / or an intra mode of the CIIP mode are signaled.

[0831] 8. The method of clause 6 or 7, wherein in a Merge mode and / or a Merge with motion vector difference (MMVD) mode of the current block, a CIIP flag and / or an intra mode of the CIIP mode of a neighboring neighboring block and / or a non-neighboring neighboring block is inherited.

[0832] 9. The method of any of clauses 6-8, wherein the CIIP flag and / or the intra mode of the CIIP mode is signaled for non-skip Merge mode and / or non-skip MMVD mode and in skip Merge mode and / or skip MMVD mode, the CIIP flag and / or the intra mode of the CIIP mode of neighboring neighboring blocks and / or non- neighboring neighboring blocks is inherited.

[0833] 10. The method of any of clauses 6-8, wherein the CIIP flag and / or the intra mode of the CIIP mode is signaled for skip Merge mode and / or skip MMVD mode and in non-skip Merge mode and / or non-skip MMVD mode, the CIIP flag and / or the intra mode of the CIIP mode of neighboring neighboring blocks and / or non- neighboring neighboring blocks is inherited.

[0834] 11. The method of any of clauses 6-10, wherein the CIIP flag is inherited in Merge mode and / or skip MMVD mode.

[0835] 12. The method of clause 11, wherein the intra mode is signaled when the CIIP flag is true and multiple intra modes are allowed in the CIIP mode.

[0836] 13. The method of any of clauses 6-12, wherein the CIIP flag and / or the intra mode of the CIIP mode of neighboring neighboring blocks and / or non- neighboring neighboring blocks coded in Merge mode and / or MMVD mode is inherited by the current block.

[0837] 14. The method of any of clauses 6-13, further comprising:

[0838] disabling the CIIP mode for skip mode.

[0839] 15. The method of any of clauses 6-13, further comprising:

[0840] comparing information of two Merge candidates,

[0841] wherein the CIIP flag and / or the intra mode of the CIIP mode is not used in the comparison of the information of the two Merge candidates.

[0842] 16. The method of any of clauses 6-13, further comprising:

[0843] comparing information of two Merge candidates,

[0844] wherein the CIIP flag and / or the intra mode of the CIIP mode are used in the comparison of the information of the two Merge candidates.

[0845] 17. A method of video processing, comprising:

[0846] during a conversion between a current block in video data and a bitstream representation of the current block, determining a type of a selected Merge candidate of the current block; and

[0847] determining, for the current block, a combinational inter-intra prediction (CIIP) applicability according to the type of the selected Merge candidate,

[0848] wherein the current block is coded in a Merge mode.

[0849] 18. The method of clause 17, wherein the CIIP is disabled when the selected Merge candidate of the current block is at least one of a paired Merge candidate, a combined bi-directional Merge candidate, a zero motion Merge candidate, or a virtual or artificial Merge candidate.

[0850] 19. The method of clause 17 or 18, wherein the CIIP mode is enabled for a paired Merge candidate or the combined bi-directional Merge candidate when one of the candidates involved in the paired Merge candidate or the combined bi-directional Merge candidate employs the CIIP mode.

[0851] 20. The method of clause 19, wherein an intra mode of the involved CIIP mode candidate is inherited.

[0852] 21. The method of clause 19, wherein an intra mode of the involved CIIP mode is signaled for the paired Merge candidate or the combined bi-directional Merge candidate.

[0853] 22. The method of clause 17 or 18, wherein the CIIP mode is enabled for a paired Merge candidate or the combined bi-directional Merge candidate when both of the candidates involved in the paired Merge candidate or the combined bi-directional Merge candidate employ the CIIP mode.

[0854] 23. The method of clause 22, wherein an intra mode of one of the two involved candidates is inherited.

[0855] 24. The method of clause 22, wherein the involved CIIP mode intra mode is derived from intra modes of the two involved candidates and used for the paired Merge candidate or the combined bi-directional Merge candidate.

[0856] 25. The method of clause 22, wherein the pair-wise Merge candidate or the combined bi-directional Merge candidate is signaled with an intra mode of the involved CIIP mode.

[0857] 26. A video processing apparatus comprising a processor configured to implement a method recited in any of clauses 1 to 25.

[0858] 27. The apparatus of clause 26, wherein the apparatus is a video encoder.

[0859] 28. The apparatus of clause 26, wherein the apparatus is a video decoder.

[0860] 29. A computer-readable recording medium having recorded thereon a program including code, the program causing a processor to execute a method recited in any of clauses 1 to 25.

[0861] It should be understood that the disclosed technology can be implemented in a video encoder or decoder to improve compression efficiency using hash-based motion estimation.

[0862] The disclosed and other solutions, examples, embodiments, modules and functional operations described herein can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this document and their structural equivalents, or in combinations of one or more of them. The disclosed and other embodiments can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer readable medium for execution by, or to control the operation of, data processing apparatus. The computer readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter effecting a machine-readable propagated signal, or a combination of one or more of them. The term “data processing apparatus” encompasses all apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can also include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them. The propagated signal is an artificially generated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus.

[0863] A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub programs, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and are interconnected by a communication network.

[0864] The processes and logic flows described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by special purpose logic circuitry, and that circuitry can be

[0865] Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for performing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. However, a computer need not have such devices. Computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0866] While this patent document contains many details, it should not be construed to limit any subject matter or claims to the precise embodiments described, but rather should be understood to include all features and variations that can come within the subject matter or claims. Some features of the embodiments described in this patent document can be combined, substituted, or deleted without departing from the general nature of these embodiments. Further, although the features described above can be described as acting in particular combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination and the claimed combination can be directed to a sub-combination or variation of a sub-combination.

[0867] Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring any particular order among the operations or that all illustrated operations be performed, to achieve desirable results. Additionally, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

[0868] Only a few implementations and examples are described and other implementations, enhancements and variations can be made based on what is described and illustrated in this patent document.

Claims

1. A video processing method, comprising: During a first conversion between a first block of video data and the bitstream representation of the first block, one or more encoding / decoding modes for one or more second blocks are determined; Based on the one or more encoding / decoding modes of the one or more second blocks, determine the encoding / decoding mode constraints of the first block; as well as The first conversion is performed at least by applying the codec mode constraints of the first block, wherein the codec mode constraints (a) are determined based on one or more codec modes of the one or more second blocks, and (b) specify whether at least one of combined inter-frame intra-frame prediction (CIIP), local illumination compensation (LIC), spread filtering, bilateral filtering or transform domain filtering, or a reconstructed filtering different from the spread filtering, the bilateral filtering and the transform domain filtering is applied to the first block; The one or more second blocks include at least one of the adjacent blocks, non-adjacent blocks, and reference blocks of the first block; Wherein, the reconstructed neighbor samples of the first block of the first image used in at least one of CIIP, the diffusion filter, the bilateral filter, or the transform domain filter are replaced with approximate samples generated from corresponding samples of the reconstructed neighbor samples in one or more reference images of the first image only when the first block is encoded and decoded with integer pixel precision.

2. The method according to claim 1, wherein determining the encoding / decoding mode constraint of the first block includes: Based on the one or more codec modes of the one or more second blocks, determine one or more flags corresponding to the one or more codec modes of the first block.

3. The method according to claim 2, wherein the encoding / decoding mode constraint of the first block includes: When the first flag corresponding to the first codec mode is exported as false, the first codec mode in the one or more codec modes of the first block is disabled.

4. The method according to claim 2, wherein the encoding / decoding mode constraint of the first block includes: When the second flag corresponding to the second codec mode is exported as false, the no signaling notification indicates whether the information of the second codec mode in the one or more codec modes of the first block is enabled.

5. The method according to claim 2, wherein the encoding / decoding mode constraint of the first block includes: When the third flag corresponding to the third codec mode is exported as false, the signaling notification indicates whether to enable the information of the third codec mode in the one or more codec modes of the first block and constrain it to false.

6. The method according to claim 1, further comprising: Determine whether to apply the encoding / decoding mode constraints of the first block.

7. The method of claim 6, wherein determining whether to apply the encoding / decoding mode constraint of the first block comprises: Whether to apply the encoding / decoding mode constraint of the first block is determined based on the encoding / decoding mode of the adjacent rows or columns of the first block.

8. The method of claim 6, wherein determining whether to apply the encoding / decoding mode constraint of the first block comprises: Whether to apply the encoding / decoding mode constraint of the first block is determined based on at least N samples in the adjacent rows or columns of the first block that are not encoded / decoded using the fourth mode, and N≥1.

9. The method of claim 7 or 8, wherein the adjacent row includes the row above the first block and the row to the upper right.

10. The method of claim 7, wherein the adjacent column includes the left column, the column below the left, and the column at the top left of the first block.

11. The method of claim 6, wherein determining whether to apply the encoding / decoding mode constraint of the first block comprises: When any adjacent non-adjacent block is encoded and decoded using the fourth mode, the encoding and decoding mode constraint of the first block is determined to be applied.

12. The method of claim 6, wherein determining whether to apply the encoding / decoding mode constraint of the first block comprises: When all adjacent non-adjacent blocks are encoded and decoded using the fourth mode, the encoding and decoding mode constraint of the first block is determined to be applied.

13. The method of claim 6, wherein determining whether to apply the encoding / decoding mode constraint of the first block comprises: When at least M neighboring non-adjacent blocks are not encoded and decoded using the fourth mode, it is determined that the encoding and decoding mode constraint of the first block is not applied, and M is a predefined first threshold.

14. The method of claim 8, wherein the fourth encoding / decoding mode includes at least one of an intra-frame prediction mode, a combined inter-frame intra-frame prediction (CIIP) mode, and a current picture reference (CPR) mode.

15. The method of claim 6, wherein determining whether to apply the encoding / decoding mode constraint of the first block comprises: Whether to apply the encoding / decoding mode constraint of the first block is determined based on the position of the first block.

16. The method of claim 15, wherein determining whether to apply the encoding / decoding mode constraint of the first block based on the position of the first block comprises: When the position is at the top of the current codec tree unit (CTU) and the adjacent block above it belongs to a different CTU from the first block, it is determined that the codec mode constraint of the first block is not applied.

17. The method of claim 15, wherein determining whether to apply the encoding / decoding mode constraint of the first block based on the position of the first block comprises: When the position is to the left of the current codec tree unit (CTU) and the left neighboring block and the first block belong to different CTUs, it is determined that the codec mode constraint of the first block is not applied.

18. The method according to claim 1, further comprising: Determine the characteristics of the first block; as well as When the characteristics of the first block meet the predefined conditions, the encoding / decoding mode constraints of the first block are determined to be applied.

19. The method according to claim 1, further comprising: Determine the characteristics of the first block and / or the previously encoded / decoded block; as well as Whether to apply the encoding / decoding mode constraint of the first block is determined based on the characteristics of the first block and / or the previous encoding / decoding blocks.

20. The method of claim 1, wherein when the first block is a bidirectional prediction, the neighboring samples of the two reference blocks of the first block are averaged to generate the approximate samples.

21. The method of claim 1, wherein when the first block is a bidirectional prediction or a unidirectional prediction, a neighboring sample of a reference block of the first block is used as the approximate sample.

22. The method of claim 1, wherein when the first block is bidirectionally predicted and at least one of unequal-weighted generalized bidirectional prediction (GBI), weighted prediction, and local illumination compensation (LIC) is applied to the first block, a weighted average of neighboring samples of two reference blocks of the first block is performed to generate the approximate samples.

23. The method of claim 20, wherein the neighboring sample points of the reference block of the first block are identified by the motion information of the first block.

24. The method of claim 20, wherein the neighboring sample points of the reference block of the first block are identified by the modified motion information of the first block.

25. The method of claim 24, wherein the modified motion information of the first block is a modified motion vector rounded to integer precision.

26. The method of claim 1, wherein the neighboring samples of the reference block of the first block are identified using motion vectors with integer pixel precision.

27. The method of claim 26, wherein the motion vectors of the reconstructed neighboring samples are rounded to integer precision to identify the corresponding samples.

28. The method of claim 1, wherein the reconstructed neighboring samples of the first block are replaced with the approximate samples only for the luminance component or the chrominance component.

29. The method of claim 18, further comprising: Determine the characteristics of the first block; as well as When the characteristics of the first block meet the predefined conditions, it is determined to replace the reconstructed neighboring sample points with the approximate sample points.

30. The method of claim 18, wherein the characteristics of the first block include at least one of block size, strip type, image type, slice type, and motion information.

31. The method of claim 30, wherein the predefined condition is that the first block contains samples not less than a predefined second threshold.

32. The method of claim 30, wherein the predefined condition is that the width and height of the first block are greater than a predefined third threshold.

33. The method of claim 30, wherein the predefined condition is that the width and height of the first block are less than a predefined fourth threshold.

34. The method of claim 30, wherein the predefined condition is: the width of the first block is less than a predefined fifth threshold and / or the height of the first block is less than a predefined sixth threshold.

35. The method of claim 30, wherein the predefined condition is: the width of the first block is greater than a predefined seventh threshold and / or the height of the first block is greater than a predefined eighth threshold.

36. The method of claim 35, wherein The signaling notification in the Sequence Parameter Set (SPS), View Parameter Set (VPS), Picture Parameter Set (PPS), Sequence Header, Picture Header, Strip Header, Slice Group Header, Slice or CTU Group indicates whether the encoding / decoding mode constraints of the first block are applied and / or whether the information of the reconstructed neighboring samples is replaced with the approximate samples.

37. The method according to claim 1, further comprising: Determine the characteristics of the first block and / or the previously encoded / decoded block; as well as Whether to replace the reconstructed neighboring sample with the approximate sample is determined based on the characteristics of the first block and / or the previous codec block.

38. The method of claim 19, wherein the characteristics of the first block and / or the previous codec block include at least one of the block dimension, video processing data unit (VPDU), picture type, low latency check flag, and codec information of the first block and / or the previous codec block.

39. The method of claim 38, wherein the encoding / decoding information of the first block and / or the previous encoding / decoding block indicates a reference picture, one-way prediction, or two-way prediction.

40. A video processing apparatus comprising a processor configured to implement the method of any one of claims 1 to 39.

41. The apparatus of claim 40, wherein the apparatus is a video encoder.

42. The apparatus of claim 40, wherein the apparatus is a video decoder.

43. A computer-readable recording medium having a program including code recorded thereon, the program causing a processor to perform the method of any one of claims 1 to 39.

Images