Coordination between different video processing modes

Abstract

This application relates to coordination between different video processing modes. A video processing method includes: determining one or more codec modes for one or more second blocks during a conversion between a first block of video data and a bitstream representation of the first block; determining codec mode constraints for the first block based on the one or more codec modes of the one or more second blocks; and performing the conversion by at least applying the codec mode constraints of the first block; wherein the one or more second blocks include at least one of adjacent blocks, non-adjacent blocks, and reference blocks of the first block.

Description

[0001] Cross-references to related applications

[0002] This application is a divisional application of Chinese Patent Application No. 202080009022.8, filed on February 2, 2020, which aims to promptly claim priority and benefits to International Patent Application No. PCT / CN2019 / 074425, filed on February 1, 2019; International Patent Application No. PCT / CN2019 / 079148, filed on March 21, 2019; and International Patent Application No. PCT / CN2019 / 100616, filed on August 14, 2019. For all purposes required by law, the entire disclosure of the foregoing applications is incorporated herein by reference as part of the disclosure of this patent. Technical Field

[0003] This document covers video and image encoding and decoding technologies. Background Technology

[0004] Digital video consumes the most bandwidth in the internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, the bandwidth demand for digital video is expected to continue to grow. Summary of the Invention

[0005] The disclosed techniques can be used by video, image decoder or encoder embodiments in which combined inter-frame and intra-frame prediction is used.

[0006] In one example aspect, a method for processing video is disclosed. The method includes: determining a encoding / decoding mode of a first video block; constraining one or more flags to an operation state, said operation state being false or true, based on the determination of the encoding / decoding mode of the first video block; and performing further processing on a second video block according to the operation state of the one or more flags, wherein the first video block is a neighboring video block or a reference video block associated with the second video block.

[0007] In another example aspect, a method for processing video is disclosed. The method includes: determining whether to use pairwise prediction or combined bidirectional prediction for a first video block; determining an operational state of combined inter-frame intra-frame prediction (CIIP) based on the determination of using pairwise prediction or combined bidirectional prediction, wherein the operational state is enabled or disabled; and performing further processing on the first video block according to the operational state of CIIP.

[0008] In another example, a different method for video processing is disclosed. The method includes: performing a conversion between video blocks and codec representations of the video blocks using a combined inter-frame and intra-frame prediction mode, wherein the codec representation corresponds to a weighted average of intra-frame and inter-frame prediction results of the video blocks using weight pairs from a set of weight pairs containing fewer than three pairs.

[0009] In another example, a different method for video processing is disclosed. The method includes: performing a conversion between video blocks and codec representations of the video blocks using a combined inter-frame and intra-frame prediction mode, wherein the codec representation corresponds to a weighted average of intra-frame and inter-frame prediction results of the video blocks using weighted pairs from a set of weighted pairs determined by codec information of one or more neighboring blocks.

[0010] In another example aspect, a different method for video processing is disclosed. The method includes: determining one or more codec modes for one or more second blocks during a conversion between a first block of video data and a bitstream representation of the first block; determining codec mode constraints for the first block based on the one or more codec modes of the first block; and performing the conversion at least by applying the codec mode constraints of the first block; wherein the one or more second blocks include at least one of adjacent blocks, non-adjacent blocks, and reference blocks of the first block.

[0011] In another example aspect, a different method for video processing is disclosed. The method includes performing a conversion between a current block in video data and a bitstream representation of the current block using at least one of combined inter-frame intra-frame prediction (CIIP), spread filtering, bilateral filtering, transform domain filtering, or a post-reconstruction filtering of another type different from spread filtering, bilateral filtering, and transform domain filtering. The reconstructed neighboring samples of the current block used in at least one of combined inter-frame intra-frame prediction (CIIP), spread filtering, bilateral filtering, transform domain filtering, or another type of post-reconstruction filtering are replaced with approximate samples generated from corresponding samples of reconstructed neighboring samples from one or more reference images.

[0012] In another example aspect, a different method for video processing is disclosed. The method includes: storing, along with motion information, a CIIP flag and / or an intra-frame mode combining inter-frame intra-frame prediction (CIIP) modes in a history-based motion vector prediction (HMVP) table; and performing a conversion between the current block in the video data and its bitstream representation, at least based on the HMVP table.

[0013] In another example aspect, a different method for video processing is disclosed. The method includes: determining a prediction mode for the current block during a conversion between the current block in video data and its bitstream representation; determining the suitability of a combined inter-frame intra-frame prediction (CIIP) mode, the suitability indicating whether the CIIP mode is enabled for the current block in response to determining whether the current block is encoded using an Advanced Motion Vector Prediction (AMVP) mode or a Merge mode; and performing a conversion based on the suitability of the CIIP mode.

[0014] In another example aspect, a different method for video processing is disclosed. The method includes: determining the type of a selected merge candidate for the current block during a conversion between a current block in video data and a bitstream representation of the current block; and determining the suitability of combined inter-frame intra-frame prediction (CIIP) for the current block based on the type of the selected merge candidate, wherein the current block is encoded in a merge mode.

[0015] In another example, the above method can be implemented by a video decoder device that includes a processor.

[0016] In yet another example, these methods can be implemented as processor-executable instructions and stored on a computer-readable program medium.

[0017] This article further describes these and other aspects. Attached Figure Description

[0018] Figure 1 An example of the derivation process for constructing the Merge candidate list is shown.

[0019] Figure 2 An example of the location of the spatial merge candidate is shown.

[0020] Figure 3 An example of candidate pairs is shown that take into account for redundancy checks for spatial merge candidates.

[0021] Figure 4 An example of the location of the second PU divided into N×2N and 2N×N segments is shown.

[0022] Figure 5 An example of motion vector scaling for a temporal Merge candidate is shown.

[0023] Figure 6 Examples of candidate positions C0 and C1 for time-domain Merge candidates are shown.

[0024] Figure 7 An example of combined bidirectional prediction of Merge candidates is shown.

[0025] Figure 8An example of the derivation process for motion vector prediction candidates is shown.

[0026] Figure 9 An example of motion vector scaling for spatial motion vector candidates is shown.

[0027] Figure 10 An example of optional temporal motion vector prediction (ATMVP) for the codec unit (CU) is shown.

[0028] Figure 11 An example of neighboring samples used to derive IC parameters is shown.

[0029] Figure 12 An example of the final motion vector representation (UMVE) search process is shown.

[0030] Figure 13 An example of a UMVE search point is shown.

[0031] Figure 14 An example of a decoding flowchart using the proposed HMVP method is shown.

[0032] Figure 15 An example of updating a table in the proposed HMVP method is shown.

[0033] Figure 16 An example of a symmetric pattern is shown.

[0034] Figure 17 An example of the neighboring samples and their corresponding samples of the current block in the reference image is shown.

[0035] Figure 18 This is a block diagram of an example video processing device.

[0036] Figure 19 A block diagram of an example implementation of a video encoder is shown.

[0037] Figure 20 This is a flowchart illustrating an example of a video processing method.

[0038] Figure 21 This is a flowchart illustrating an example of a video processing method.

[0039] Figure 22 An example of the upper and left adjacent blocks used in CIIP weight evaluation is shown.

[0040] Figure 23 This is a flowchart illustrating an example of a video processing method.

[0041] Figure 24 This is a flowchart illustrating an example of a video processing method.

[0042] Figure 25This is a flowchart illustrating an example of a video processing method.

[0043] Figure 26 This is a flowchart illustrating an example of a video processing method.

[0044] Figure 27 This is a flowchart illustrating an example of a video processing method. Detailed Implementation

[0045] This article provides various techniques that can be used by decoders of image or video bitstreams to improve the quality of decompressing or decoding digital video or images. For the sake of brevity, this article uses the term "video" to include both sequences of pictures (conventionally referred to as video) and individual images. Furthermore, video encoders can also implement these techniques during the encoding process to reconstruct decoded frames for further encoding.

[0046] For ease of understanding, chapter headings are used herein, and the embodiments and techniques are not limited to the corresponding chapters. Therefore, embodiments from one chapter can be combined with embodiments from other chapters.

[0047] 1. Overview

[0048] This article relates to video coding and decoding techniques. Specifically, it relates to Combined Inter-Frame Intra-Frame Prediction (CIIP) in video coding or decoding. It can be applied to existing video coding and decoding standards such as HEVC, or to pending standards (Multi-Functional Video Coding). It may also be applicable to future video coding and decoding standards or video codecs.

[0049] 2. Background

[0050] Video coding standards have been developed primarily through the development of well-known ITU-T and ISO / IEC standards. ITU-T produced H.261 and H.263, ISO / IEC produced MPEG-1 and MPEG-4 video, and the two organizations jointly produced the H.262 / MPEG-2 video and H.264 / MPEG-4 Advanced Video Coding (AVC) and H.265 / HEVC[1] standards. Starting with H.262, video coding standards are based on a hybrid video coding architecture that utilizes temporal prediction plus transform coding. To explore future video coding technologies beyond HEVC, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET) in 2015. Since then, JVET has adopted many new approaches and applied them to a reference software called the Joint Exploration Model (JEM). In April 2018, the Joint Video Experts Team (JVET) was established between VCEG (Q6 / 16) and ISO / IEC JTC1 SC29 / WG11 (MPEG) to work on the VVC standard, which aims to reduce the bit rate by 50% compared to HEVC.

[0051] Figure 19 This is a block diagram of an exemplary implementation of a video encoder. Figure 19 The encoder implementation is shown to have a built-in feedback path, in which the video encoder also performs video decoding (reconstructing the compressed representation of the video data for encoding the next video data).

[0052] 2.1 Inter-frame prediction in HEVC / H.265

[0053] Each inter-frame predicted PU has motion parameters for one or two lists of reference images. The motion parameters include motion vectors and reference image indices. The use of one of the two reference image lists can also be notified via inter_pred_idc signaling. The motion vectors can be explicitly encoded as increments relative to the predictor.

[0054] When a CU uses skip mode encoding / decoding, a PU is associated with the CU and has no significant residual coefficients, no encoding / decoding motion vector increments, or a reference picture index. A Merge mode is specified, through which the motion parameters of the current PU can be obtained from neighboring PUs (including spatial and temporal candidates). The Merge mode can be applied to any PU for inter-frame prediction, not just skip mode. Another option for the Merge mode is explicit transmission of motion parameters, where the motion vectors (more precisely, the difference in motion vectors compared to the motion vector predictor), the corresponding reference picture index for each reference picture list, and the use of the reference picture list are explicitly signaled according to each PU. In this disclosure, such a mode is named Advanced Motion Vector Prediction (AMVP).

[0055] When signaling indicates that one of two reference image lists should be used, a PU is generated from one of the sample points. This is called "one-way prediction". One-way prediction is available for both P-slices and B-slices.

[0056] When signaling indicates that two reference image lists should be used, a PU is generated from the two sample blocks of the sample points. This is called "bidirectional prediction". Bidirectional prediction is only available for B-strips.

[0057] The following section provides details regarding inter-frame prediction modes as defined in HEVC. The description will begin with Merge mode.

[0058] 2.1.1 Merge Mode

[0059] 2.1.1.1 Derivation of Merge Pattern Candidates

[0060] When predicting the PU using the Merge mode, the index pointing to the entry in the Merge candidate list is parsed from the bitstream, and motion information is retrieved using this index. The construction of this list is specified in the HEVC standard and can be summarized in the following steps:

[0061] Step 1: Initial Candidate Derivation

[0062] Step 1.1: Spatial Candidate Derivation

[0063] Step 1.2: Spatial Candidate Redundancy Check

[0064] Step 1.3: Time-domain candidate derivation

[0065] Step 2: Add candidate insertions

[0066] Step 2.1: Create bidirectional prediction candidates

[0067] Step 2.2: Insert zero-motion candidates

[0068] exist Figure 1 These steps are also illustrated schematically. For spatial merge candidate derivation, at most four merge candidates are selected from five candidates located at different positions. For temporal merge candidate derivation, at most one merge candidate is selected from two candidates. Since the number of candidates per PU is assumed to be constant at the decoder, additional candidates are generated when the number of candidates obtained from step 1 does not reach the maximum number of merge candidates (maxNumMergeCand) notified by the signaling in the stripe header. Because the number of candidates is constant, the index of the best merge candidate is encoded using truncated univariate binarization (TU). If the size of the CU is equal to 8, all PUs of the current CU share a single merge candidate list, which is the same as the merge candidate list of a 2N×2N prediction unit.

[0069] The operations associated with the foregoing steps are described in detail below.

[0070] 2.1.1.2 Derivation of Airspace Candidates

[0071] In the derivation of the spatial Merge candidate, located in Figure 2 At most four Merge candidates can be selected from the candidates at the indicated positions. The derivation order is A. 1， B 1， B 0， A0 and B2. Only when position A 1， B 1， B 0， Position B2 is considered only if any PU of A0 is unavailable (e.g., because it belongs to another stripe or slice) or if it is intra-frame encoding / decoding. After adding candidates for position A1, a redundancy check is performed on the addition of the remaining candidates, which ensures that candidates with the same motion information are excluded from the list, thereby improving encoding / decoding. To reduce computational complexity, not all possible candidate pairs are considered in the aforementioned redundancy check. Instead, only those with... Figure 3 Only pairs of arrow links in the code are considered, and a candidate is added to the list only if the corresponding candidate used for redundancy checking does not have the same motion information. Another source of duplicated motion information is the "second PU" associated with different 2Nx2N partitions. For example, Figure 4 The second prediction unit (PU) is described for N×2N and 2N×N cases respectively. When the current PU is segmented into N×2N, candidates for position A1 are not considered during list construction. In fact, adding this candidate could result in two prediction units with the same motion information, which is redundant for an encoding / decoding unit with only one PU. Similarly, when the current PU is segmented into 2N×N, position B1 is not considered.

[0072] 2.1.1.3 Time-domain candidate derivation

[0073] In this step, only one candidate is added to the list. Specifically, in the derivation of this temporal merge candidate, the scaling motion vector is derived based on the juxtaposed PU of the image that has the smallest POC difference with the current image in the given list of reference images. The list of reference images used to derive the juxtaposed PU is explicitly signaled in the strip header. Obtain the scaling motion vector of the temporal merge candidate (e.g., Figure 5 (As shown by the dashed line), it scales the motion vectors from the juxtaposed PU using POC distances tb and td, where tb is defined as the POC difference between the current image and the reference image, and td is defined as the POC difference between the juxtaposed image and the reference image. The reference image index for the temporal merge candidate is set to zero. The actual implementation of the scaling process is described in the HEVC specification. For the B-strip, two motion vectors are obtained (one for reference image list 0 and the other for reference image list 1) and combined to form a bidirectional predictive merge candidate.

[0074] In the juxtaposed PU(Y) belonging to the reference frame, the position of the temporal candidate is selected between candidate C0 and C1, as follows: Figure 6 As shown. If the PU at position C0 is unavailable, intra-frame encoding / decoding is used, or it is outside the current CTU line, then position C1 is used. Otherwise, position C0 is used for the derivation of the temporal merge candidate.

[0075] 2.1.1.4 Additional Candidate Insertion

[0076] In addition to spatial and temporal merge candidates, there are two additional types of merge candidates: combined bidirectional prediction merge candidates and zero merge candidates. Combined bidirectional prediction merge candidates are generated using both spatial and temporal merge candidates. Combined bidirectional prediction merge candidates are only used for B-strips. They are generated by combining the motion parameters of the first reference image list of the initial candidate with the motion parameters of the second reference image list of another candidate. If these two tuples provide different motion hypotheses, they will form new bidirectional prediction candidates. As an example, Figure 7 This scenario is illustrated, where two candidates from the original list (on the left) with MVL0 and refIdxL0 or MVL1 and refIdxL1 are used to create combined bidirectional prediction merge candidates added to the final list (on the right). Several rules are defined regarding the combinations considered to generate these additional merge candidates.

[0077] Zero-motion candidates are inserted to populate the remaining entries in the Merge candidate list, thus reaching the capacity of MaxNumMergeCand. These candidates have zero spatial displacement and a reference image index that starts from zero and increases each time a new zero-motion candidate is added to the list. The number of reference frames used for these candidates is 1 and 2, respectively, for unidirectional and bidirectional prediction. Finally, no redundancy checks are performed on these candidates.

[0078] 2.1.1.5 Motion estimation region for parallel processing

[0079] To accelerate encoding processing, motion estimation can be performed in parallel, thereby simultaneously deriving motion vectors for all prediction units within a given region. Deriving merge candidates from spatial neighbors can interfere with parallel processing because a prediction unit cannot derive motion parameters from neighboring PUs until its associated motion estimation is complete. To mitigate the trade-off between encoding / decoding efficiency and processing latency, HEVC uses the "log2_parallel_merge_level_minus2" syntax element to define the motion estimation region (MER) whose size is signaled in the image parameter set. When defining the MER, merge candidates falling into the same region are marked as unavailable and therefore not considered in the list construction.

[0080] 2.1.2 AMVP

[0081] AMVP utilizes the spatiotemporal correlation between motion vectors and neighboring PUs for explicit transmission of motion parameters. For each list of reference images, a motion vector candidate list is first constructed by checking the availability of temporally neighboring PU locations in the upper left, removing redundant candidates, and adding zero vectors to keep the candidate list length constant. The encoder can then select the best predictor from the candidate list and send the corresponding index indicating the selected candidate. Similar to the Merge index signaling, the index of the best motion vector candidate is encoded using truncated unary values. In this case, the maximum value to be encoded is 2 (see [link to documentation]). Figure 8 The following sections provide details on the derivation of motion vector prediction candidates.

[0082] 2.1.2.1 Derivation of AMVP Candidates

[0083] Figure 8 The derivation and processing of motion vector prediction candidates are summarized.

[0084] In motion vector prediction, two types of motion vector candidates are considered: spatial motion vector candidates and temporal motion vector candidates. The derivation of the spatial motion vector candidate is based on the fact that... Figure 2 The motion vectors of each PU at the five different locations shown ultimately yield two motion vector candidates.

[0085] For the derivation of temporal motion vector candidates, one motion vector candidate is selected from two candidates derived from two different juxtaposition positions. After creating the first spatiotemporal candidate list, duplicate motion vector candidates are removed from the list. If the number of potential candidates is greater than two, motion vector candidates with reference image indices greater than 1 in the associated reference image list are removed from the list. If the number of spatiotemporal motion vector candidates is less than two, additional zero motion vector candidates are added to the list.

[0086] 2.1.2.2 Candidate Spatial Motion Vectors

[0087] When deriving the candidates for spatial motion vectors, at most two candidates are considered from the five potential candidates. These five candidates are derived from... Figure 2 The positions of the PUs shown are the same as those of the motion merge. The derivation order to the left of the current PU is defined as A0, A1, and scaled A0, scaled A1. The derivation order above the current PU is defined as B0, B1, B2, scaled B0, scaled B1, scaled B2. Therefore, there are four cases on each side that can be used as motion vector candidates, two of which do not require spatial scaling, and two of which do. The four different cases are summarized as follows:

[0088] -- No spatial scaling

[0089] (1) Same list of reference images, and same reference image index (same POC)

[0090] (2) Different lists of reference images, but the same reference image (same POC)

[0091] -- Spatial scaling

[0092] (3) Same list of reference images, but different reference images (different POCs)

[0093] (4) Different lists of reference images, and different reference images (different POCs)

[0094] First, we check the case where spatial scaling is not allowed, then we check the case where spatial scaling is allowed. Spatial scaling is considered when the POC differs between the reference image of the neighboring PU and the reference image of the current PU, regardless of the list of reference images. If all PUs in the left-hand candidate pool are unavailable or are intra-frame encoded / decoded, scaling of the above motion vectors is allowed to aid in the parallel derivation of the left-hand and top-hand MV candidates. Otherwise, spatial scaling of the above motion vectors is not allowed.

[0095] In spatial scaling, the motion vectors of neighboring PUs are scaled in a manner similar to temporal scaling, such as... Figure 9 As shown. The main difference is that a list of reference images and their indices for the current PU are given as input; the actual scaling process is the same as the temporal scaling process.

[0096] 2.1.2.3 Candidate Motion Vectors in the Time Domain

[0097] Except for the derivation of the reference image index, all derivation processes for the temporal merge candidate are the same as those for the spatial motion vector candidate (see [link]). Figure 6 The reference image index signaling is sent to the decoder.

[0098] 2.2 New Inter-Frame Prediction Method

[0099] 2.2.1 Motion Vector Prediction Based on Sub-CU

[0100] In a JEM with QTBT, each CU can have at most one set of motion parameters for each prediction direction. The encoder considers two sub-CU level motion vector prediction methods by dividing a large CU into sub-CUs and deriving the motion information of all sub-CUs within that large CU. The Optional Temporal Motion Vector Prediction (ATMVP) method allows each CU to extract multiple sets of motion information from multiple smaller blocks in a juxtaposed reference image. In the Spatial-Temporal Motion Vector Prediction (STMVP) method, the motion vectors of sub-CUs are recursively derived using a temporal motion vector predictor and spatially neighboring motion vectors.

[0101] To maintain a more accurate motion field for sub-CU motion prediction, motion compression of the reference frame is currently disabled.

[0102] 2.2.1.1 Optional temporal motion vector prediction

[0103] In the optional temporal motion vector prediction (ATMVP) method, the temporal motion vector prediction (TMVP) is modified by extracting multiple sets of motion information (including motion vectors and reference indices) from blocks smaller than the current CU. For example... Figure 10 As shown, the sub-CU is a square N×N block (N is set to 4 by default).

[0104] ATMVP predicts the motion vectors of sub-CUs within a CU in two steps. The first step is to identify corresponding blocks in a reference image using so-called temporal vectors. This reference image is called the motion source image. The second step is to divide the current CU into sub-CUs and obtain the reference index and motion vector of each sub-CU from the corresponding block, such as... Figure 10 As shown.

[0105] In the first step, the reference image and the corresponding block are determined by the motion information of the spatial neighboring blocks of the current CU. To avoid repeated scanning of neighboring blocks, the first merge candidate in the current CU's merge candidate list is used. The first available motion vector and its associated reference index are set as a temporal vector and an index to the motion source image. In this way, in ATMVP, compared to TMVP, the corresponding block can be identified more accurately, where the corresponding block (sometimes called the juxtaposed block) is always located at the lower right or center position relative to the current CU.

[0106] In the second step, the corresponding block of the sub-CU is identified by adding the temporal vector to the coordinates of the current CU, using the temporal vector in the motion source image. For each sub-CU, the motion information of the sub-CU is derived using the motion information of its corresponding block (the smallest motion grid covering the center sample). After identifying the motion information of the corresponding N×N block, it is converted into the motion vector and reference index of the current sub-CU, similar to the TMVP method of HEVC, where motion scaling and other processing are applied. For example, the decoder checks whether a low-latency condition is met (e.g., the POC of all reference images of the current image is smaller than the POC of the current image), and may use the motion vector MV. x (The motion vector corresponding to the reference image list X) is used to predict the motion vector MV for each sub-CU. y (X equals 0 or 1 and Y equals 1-X).

[0107] 2.2.2 Pairwise Average Candidates

[0108] Pairwise averaged candidates are generated by averaging predefined candidate pairs in the current Merge candidate list, and the predefined pairs are defined as {(0,1), (0,2), (1,2), (0,3), (1,3), (2,3)}, where the numbers represent the Merge indices to the Merge candidate list. The average motion vector for each reference list is calculated separately. If two motion vectors are available in a list, they are averaged even if they point to different reference images; if only one motion vector is available, that motion vector is used directly; if no motion vector is available, the list remains invalid. Pairwise averaged candidates replace combined candidates in the HEVC standard.

[0109] The complexity analysis of pairwise averaging candidates is summarized in Table 1. For the worst-case scenario of additional computations used for averaging (last column of Table 1), each pair (MVx and MVy in L0 and L1) requires 4 additions and 4 shifts, and each pair requires 4 reference index comparisons (refIdx0 is valid in L0 and refIdx1 is valid in L1). There are 6 pairs, resulting in a total of 24 additions, 24 shifts, and 24 reference index comparisons. The combined candidate pairs in the HEVC standard use 2 reference index comparisons per pair (refIdx0 is valid in L0 and refIdx1 is valid in L1), and there are 12 pairs, resulting in a total of 24 reference index comparisons.

[0110] Table 1: Operational Analysis of Paired Average Candidates

[0111]

[0112] 2.2.3 Local lighting compensation

[0113] Local illumination compensation (LIC) is based on a linear model for illumination variations, using a scaling factor a and an offset b. Furthermore, it is adaptively enabled or disabled for each inter-frame mode encoding / decoding unit (CU).

[0114] When LIC is applied to CU, the least squares error method is used to derive parameters a and b by using the nearest samples of the current CU and their corresponding reference samples. More specifically, as... Figure 11 As shown, neighboring and corresponding samples (identified by the motion information of the current CU or sub-CU) of the CU in the reference image are used as subsamples (2:1 subsamples). IC parameters are derived and applied to each prediction direction.

[0115] When encoding and decoding a CU using Merge mode, the LIC flag is copied from the neighboring block in a manner similar to motion information copying in Merge mode; otherwise, the CU signaling will notify the LIC flag to indicate whether LIC is applicable.

[0116] When LIC is enabled for an image, an additional CU-level RD check is required to determine whether LIC is applied to the CU. When LIC is enabled for the CU, for integer pixel motion search and fractional pixel motion search, the mean-removed sum of absolute difference (MR-SAD) and the mean-removed sum of absolute Hadamard-transformed difference (MR-SATD) are used respectively, instead of SAD and SATD.

[0117] To reduce coding complexity, the following coding scheme is applied in JEM:

[0118] If there is no significant change in illumination between the current image and its reference images, LIC is disabled for the entire image. To identify this situation, the histogram of the current image and the histogram of each reference image of the current image are calculated at the encoder. If the histogram difference between the current image and each reference image of the current image is less than a given threshold, LIC is disabled for the current image; otherwise, LIC is enabled for the current image.

[0119] 2.2.4 Combined Inter-Frame and Intra-Frame Prediction

[0120] In some embodiments, multiple hypothesis prediction is proposed, wherein combining intra-frame and inter-frame prediction is one way to generate multiple hypotheses.

[0121] When multiple hypothesis prediction is applied to improved intra-mode, it combines an intra-mode prediction with a Merge index prediction. In the Merge CU, when a flag is true, a flag is signaled for the Merge mode to select an intra-mode from the intra-candidate list. For the luma component, an intra-candidate list is derived from four intra-mode prediction options: DC, planar, horizontal, and vertical. The size of the intra-candidate list can be 3 or 4, depending on the block shape. The horizontal mode is not included in the intra-mode list when the CU width is greater than twice the CU height, and the vertical mode is removed when the CU height is greater than twice the CU width. A weighted average is used to combine an intra-mode prediction selected by the intra-mode index with a Merge index prediction selected by the Merge index. For the chroma component, a DM is always applied without additional signaling. The weights used for combining predictions are described below. Equal weights are applied when DC or planar modes are selected, or when the CB width or height is less than 4. For a CB with a width and height greater than or equal to 4, when selecting the horizontal / vertical mode, a CB is first vertically / horizontally divided into four equal regions. Each weight set, denoted as (w_intrai, w_interi), is applied to the corresponding region, where i ranges from 1 to 4, and (w_intra1, w_inter1) = (6, 2), (w_intra2, w_inter2) = (5, 3), (w_intra3, w_inter3) = (3, 5), and (w_intra4, w_inter4) = (2, 6). (w_intra1, w_inter1) is used for the region closest to the reference sample, and (w_intra4, w_inter4) is used for the region furthest from the reference sample. The combined prediction can then be calculated by adding the two weighted predictions and right-shifting them by 3 bits. Furthermore, the intra-prediction mode of the predictor's intra-frame hypothesis can be saved for subsequent reference by neighboring CUs.

[0122] 2.2.5 Generalized Two-Way Forecasting

[0123] In traditional bidirectional forecasting, the predictors from L0 and L1 are averaged with an equal weight of 0.5 to generate the final predictor. The predictor generation formula is shown in equation (1):

[0124]

[0125] In equation (1), P TraditionalBiPred It is the final predictor of traditional bidirectional forecasting, P L0 and P L1These are the predictors from L0 and L1, respectively, and RoundingOffset and shiftNum are used to normalize the final predictor.

[0126] A generalized bidirectional prediction (GBI) is proposed to allow different weights to be applied to the predictors from L0 and L1. The predictor is generated as shown in equation (2).

[0127]

[0128] In equation (2), P GBi This is the final prediction of GBi. (1-w1) and w1 are the selected GBi weights applied to the L0 and L1 predictors, respectively. RoundingOffset GBi and shiftNum GBi It is used to normalize the final predictor in GBi.

[0129] The supported w1 weight tables are {-1 / 4, 3 / 8, 1 / 2, 5 / 8, 5 / 4}. One equal-weight set and four unequal-weight sets are supported. For the equal-weight case, the process of generating the final predictor is exactly the same as in the traditional bidirectional prediction mode. For true bidirectional prediction under random access (RA) conditions, the number of candidate weight sets is reduced to three.

[0130] For Advanced Motion Vector Prediction (AMVP) mode, if the CU is encoded and decoded using bidirectional prediction, the weight selection in GBI is explicitly signaled at the CU level. For Merge mode, the weight selection is inherited from the Merge candidate. In this proposal, GBI supports DMVR to generate a weighted average of templates and the final predictor of BMS-1.0.

[0131] 2.2.6 Final Motion Vector Expression

[0132] In some embodiments, a Final Motion Vector Expression (UMVE) is proposed. UMVE, along with the proposed motion vector expression method, is used in skip or merge modes.

[0133] UMVE reuses the Merge candidate as it does in VVC. Within the Merge candidate, candidates can be selected and further extended using the proposed motion vector representation method.

[0134] UMVE provides a new motion vector representation with simplified signaling. The representation includes the starting point, motion amplitude, and motion direction.

[0135] This proposed technique uses the Merge candidate list as is. However, only candidates of the default Merge type (MRG_TYPE_DEFAULT_N) are considered for UMVE extensions.

[0136] The base candidate index defines the starting point. The base candidate index indicates the best candidate among the candidates in the list, as shown below.

[0137] Table 1 Basic Candidate Indexes (IDX)

[0138]

[0139] If the number of basic candidates is equal to 1, then no signaling is sent to the basic candidate IDX.

[0140] The distance index is motion amplitude information. The distance index indicates a predefined distance from the starting point. The predefined distances are shown below:

[0141] Table 2 Distance Index IDX

[0142]

[0143] The direction index represents the direction of MVD relative to the starting point. The direction index can represent the four directions shown below.

[0144] Table 3 Direction Index IDX

[0145]

[0146] Immediately after sending the skip and merge flags, signal the UMVE flag. If the skip and merge flags are true, the UMVE flag is resolved. If the UMVE flag is equal to 1, the UMVE syntax is resolved. However, if it is not 1, the affine flag is resolved. If the affine flag is equal to 1, it is an affine pattern; however, if it is not 1, the skip / merge index of the VTM's skip / merge pattern is resolved.

[0147] The additional line buffer required for UMVE candidates is unnecessary because the software's skip / merge candidates are used directly as the base candidates. MV supplementation is determined directly before motion compensation using the input UMVE index. Therefore, there is no need to reserve a long line buffer for this.

[0148] 2.2.7 Historical Motion Vector Prediction

[0149] In our previous invention P180502841H, one or more lookup tables containing at least one motion candidate were stored to predict motion information of a block.

[0150] A history-based MVP (HMVP) method is proposed, where HMVP candidates are defined as motion information from previous codec blocks. A table with multiple HMVP candidates is maintained during encoding / decoding. The table is cleared when a new stripe is encountered. Whenever an inter-frame codec block exists, the associated motion information is added as a new HMVP candidate to the last entry in the table. The entire encoding / decoding process is as follows: Figure 13 As shown.

[0151] In one example, the table size is set to L (e.g., L = 16 or 6 or 44), which indicates that up to L HMVP candidates can be added to the table.

[0152] In one embodiment, if there are more than L HMVP candidates from previous codec blocks, a first-in-first-out (FIFO) rule is applied so that the table always contains the latest L motion candidates from previous codecs. Figure 14 An example is described where a FIFO rule is applied to remove HMVP candidates and add new HMVP candidates to the table used by the proposed method.

[0153] In another embodiment, whenever a new motion candidate is added (such as when the current block is inter-frame encoded and non-affine mode), a redundancy check is first applied to identify whether the same or similar motion candidates exist in the LUT.

[0154] 2.2.8 Symmetrical Motion Vector Difference

[0155] In some embodiments, Symmetric Motion Vector Difference (SMVD) is proposed to encode MVD more efficiently.

[0156] First, at the stripe level, the variables BiDirPredFlag, RefIdxSymL0, and RefIdxSymL1 are derived as follows:

[0157] Search for the closest preceding reference image in reference image list 0. If found, set RefIdxSymL0 to the reference index of the preceding image.

[0158] Search for the closest backward reference image in reference image list 1. If found, set RefIdxSymL1 to the reference index of the backward image.

[0159] If both the forward and backward images are found, set BiDirPredFlag to 1.

[0160] Otherwise, the following applies:

[0161] Search for the closest backward reference image in reference image list 0. If found, set RefIdxSymL0 to the reference index of the backward image.

[0162] Search for the closest preceding reference image in reference image list 1. If found, set RefIdxSymL1 to the reference index of the preceding image.

[0163] If both backward and forward images are found, set BiDirPredFlag to 1. Otherwise, set BiDirPredFlag to 0.

[0164] Secondly, at the CU level, if the CU's prediction direction is bidirectional and BiDirPredFlag equals 1, then an explicit signaling notification indicates whether to use the symmetric mode flag of the symmetric mode.

[0165] When the flag is true, only mvp_l0_flag, mvp_l1_flag, and MVD0 are explicitly signaled. The reference indices of list 0 and list 1 are set to equal RefIdxSymL0 and RefIdxSymL1, respectively. MVD1 is set to equal only –MVD0. The final motion vector is shown in the formula below.

[0166]

[0167] The modifications to the encoding / decoding unit syntax are shown in Table 2.

[0168] Table 2: Modifications in the Encoder / Decoder Unit Syntax

[0169]

[0170]

[0171] 2.2.9 CIIP in VTM4

[0172] In VTM4, when a CU is encoded or decoded in Merge mode, and if the CU contains at least 64 luma samples (i.e., the CU width multiplied by the CU height is equal to or greater than 64), a signaling flag is sent to indicate whether the combined inter-frame / intra-frame prediction (CIIP) mode is applied to the current CU.

[0173] To generate CIIP predictions, the intra-prediction mode is first derived from two additional syntax elements. Up to four possible intra-prediction modes can be used: DC, planar, horizontal, or vertical. Then, the inter-prediction and intra-prediction signals are derived using regular intra- and inter-prediction processing. Finally, a weighted average of the inter- and intra-prediction signals is taken to obtain the CIIP prediction.

[0174] 1. Derivation of Intra-Frame Prediction Mode

[0175] Up to four intra-prediction modes (DC mode, PLANA mode, HORIZONTAL mode, and VERTICAL mode) can be used to predict the luma component in CIIP mode. Horizontal mode is not allowed if the CU shape is very wide (i.e., width greater than twice its height). Vertical mode is not allowed if the CU shape is very narrow (i.e., height greater than twice its width). In these cases, only three intra-prediction modes are allowed.

[0176] CIIP mode uses three most probable modes (MPMs) for intra-frame prediction. The CIIP MPM candidate list is formed as follows:

[0177] Set the left and top adjacent blocks to A and B respectively.

[0178] The intra-prediction modes for block A and block B (denoted as intraModeA and intraModeB, respectively) are derived as follows:

[0179] Let X be either A or B;

[0180] If 1) block X is unavailable; or 2) block X is not predicted using CIIP mode or intra-frame mode; or 3) block B is outside the current CTU, then set intraModeX to DC;

[0181] Otherwise, 1) if the intra-prediction mode of block X is DC or planar, then set intraModeX to DC or planar; or 2) if the intra-prediction mode of block X is "vertical" angular mode (greater than 34), then set intraModeX to vertical; or 3) if the intra-prediction mode of block X is "horizontal" angular mode (less than or equal to 34), then set intraModeX to horizontal.

[0182] -If intraModeA and intraModeB are the same, then:

[0183] If intraModeA is planar or DC, then set the three MPMs to {planar, DC, vertical} in that order;

[0184] Otherwise, set the three MPMs to {intraModeA, plane, DC} in sequence;

[0185] -Otherwise (intraModeA and intraModeB are different):

[0186] o Set the first two MPMs to {intraModeA, intraModeB} in sequence;

[0187] o sequentially checks the uniqueness of the planar, DC, and vertical patterns in the first two MPM candidate patterns; once a unique pattern is found, it is added as the third MPM.

[0188] If the CU shape is very wide or very narrow as defined above, the MPM flag is inferred to be 1 without signaling notification. Otherwise, the MPM flag is signaled to indicate whether the CIIP intra-prediction mode is one of the CIIP MPM candidate modes.

[0189] If the MPM flag is 1, further signaling is used to inform the MPM index, indicating which MPM candidate mode to use in CIIP intra-prediction. Otherwise, if the MPM flag is 0, the intra-prediction mode is set to the "missing" mode in the MPM candidate list. For example, if the planar mode is not in the MPM candidate list, then planar is the missing mode, and the intra-prediction mode is set to planar. Since CIIP allows four possible intra-prediction modes, and the MPM candidate list only contains three, one of the four possible modes must be the missing mode.

[0190] For the chromaticity component, the DM mode is always applied without additional signaling; that is, chromaticity uses the same prediction mode as luminance.

[0191] The intra-prediction mode of the CU encoded and decoded by CIIP will be saved and used for the intra-mode encoding and decoding of future adjacent CUs.

[0192] 2.2.9.2 Combining inter-frame and intra-frame prediction signals

[0193] The inter-frame prediction signal in CIIP mode is derived using the same inter-frame prediction processing applied to the regular Merge mode. Furthermore, after conventional intra-prediction processing, the intra-prediction signal is derived using the CIIP intra-prediction mode. Then, a weighted average is used to combine the intra-frame and inter-frame prediction signals, where the weight values ​​depend on the intra-frame prediction mode and the location of the sample within the codec block, as shown below:

[0194] - If the intra-prediction mode is DC mode or planar mode, or if the block width or height is less than 4, then equal weights are applied to the intra-prediction and inter-prediction signals.

[0195] - Otherwise, weights are determined based on the intra-prediction mode (horizontal or vertical in this case) and the sample positions within the block. Taking the horizontal prediction mode as an example (the weights for the vertical mode are derived similarly, but in the orthogonal direction): Let W represent the width of the block, and H the height of the block. The codec block is first divided into four equal-area regions, each with dimensions (W / 4) x H. Starting from the region closest to the intra-prediction reference sample and ending at the region furthest from the intra-prediction reference sample, the weights wt for each of the four regions are set to 6, 5, 3, and 2, respectively. The final CIIP prediction signal is derived using the following:

[0196]

[0197] 2. CIIP in VTM5

[0198] In VTM5, when a CU is encoded or decoded in Merge mode, if the CU contains at least 64 luma samples (i.e., the CU width multiplied by the CU height is equal to or greater than 64), and if both the CU width and CU height are less than 128 luma samples, a signaling flag is sent to indicate whether to apply Combined Inter-Frame / Intra-Frame Prediction (CIIP) mode to the current CU. As the name suggests, CIIP prediction combines inter-frame prediction signals with intra-frame prediction signals. The inter-frame prediction signal in CIIP mode is derived using the same inter-frame prediction processing applied to the regular Merge mode. Furthermore, the intra-prediction signal is derived using the same conventional intra-prediction processing as the planar mode. Then, a weighted average is used to combine the intra-frame prediction signal and the inter-frame prediction signal, where the weights are calculated based on the encoding / decoding modes of the top and left neighboring blocks as follows ( Figure 16 (as depicted in the text)

[0199] - If the top neighbor is available and is intra-coded, set isIntraTop to 1; otherwise, set isIntraTop to 0.

[0200] - If the left neighbor is available and is intra-coded, set isIntraLeft to 1; otherwise, set isIntraLeft to 0.

[0201] - If (isIntraLeft+isIntraTop) equals 2, then set wt to 3;

[0202] Otherwise, if (isIntraLeft+isIntraTop) equals 1, then set wt to 2;

[0203] Otherwise, set wt to 1.

[0204] The CIIP predictions are as follows:

[0205]

[0206] Figure 16 The top and left neighboring blocks used for CIIP weight derivation are shown.

[0207] 2.3 Intra-frame mode derivation processing

[0208] In the current VTM-4.0, when encoding or decoding an intra-frame codec block, a Most Probable Mode (MPM) flag is first signaled. If the MPM flag is true, the index of the MPM list is then signaled. Otherwise, the index of a non-MPM mode is signaled.

[0209] The relevant syntax elements, symmetries, and decoding processes in the latest VVC specification (JVET-M1001_v7) are as follows:

[0210] Related syntax

[0211] Encoder / decoder unit syntax

[0212]

[0213]

[0214] Semantics

[0215] The syntax elements intra_luma_mpm_flag[x0][y0], intra_luma_mpm_idx[x0][y0], and intra_luma_mpm_remainder[x0][y0] specify the intra-prediction mode for the luma sample. The array indices x0 and y0 specify the position (x0, y0) of the top-left luma sample of the codec block under consideration relative to the top-left luma sample of the image. When intra_luma_mpm_flag[x0][y0] equals 1, the intra-prediction mode is inferred from the nearest intra-prediction codec unit according to section 8.4.2.

[0216] If intra_luma_mpm_flag[x0][y0] does not exist, it is inferred that it is equal to 1.

[0217] Decoding Processing

[0218] Derivation and processing of the lumen intra-frame prediction mode

[0219] The input for this process is:

[0220] The luminance position (xCb, yCb) specifies the top-left sample of the current luminance block relative to the top-left luminance sample of the current image.

[0221] The variable cbWidth specifies the width of the current codec block in the luminance sample.

[0222] The variable cbHeight specifies the height of the current codec block in the luminance sample.

[0223] In this process, the luminance intra-prediction mode IntraPredModeY[ xCb ][ yCb ] is derived.

[0224] Table 8-1 specifies the values ​​and related names of the IntraPredModeY [xCb] [yCb].

[0225] Table 8-1 Specifications of Intra-Frame Prediction Modes and Related Names

[0226]

[0227] Note: The intra-prediction modes INTRA_LT_CCLM, INTRA_L_CCLM, and INTRA_T_CCLM apply only to the chroma component.

[0228] The IntraPredModeY[xCb][yCb] is derived by the following sequential steps:

[0229] 1. Set the neighboring positions (xNbA, yNbA) and (xNbB, yNbB) to equal (xCb -1, yCb + cbHeight - 1) and (xCb + cbWidth - 1, yCb - 1) respectively.

[0230] 2. For cases where X is replaced by A or B, the variable candIntraPredModeX is exported as follows:

[0231] – Take the position (xCurr, yCurr) as input and set it to be equal to (xCb, yCb), and the neighboring position (xNbY, yNbY) as equal to (xNbX, yNbX), and call the availability derivation process for the block as specified in Section 6.4.X [Ed. (BB): Pending Neighboring Block Availability Check Processing], and assign the output to availableX.

[0232] – The derivation of the candidate intra-prediction mode candIntraPredModeX is as follows:

[0233] – If one or more of the following conditions are true, then set candIntraPredModeX to be equal to INTRA_PLANAR.

[0234] – The variable availableX is equal to FALSE.

[0235] – CuPredMode[ xNbX ][ yNbX ] is not equal to MODE_INTRA and ciip_flag[ xNbX ][yNbX ] is not equal to 1.

[0236] – pcm_flag[ xNbX ][ yNbX ] equals 1.

[0237] – X equals B and yCb - 1 is less than ((yCb >> CtbLog2SizeY) << CtbLog2SizeY).

[0238] – Otherwise, set candIntraPredModeX to equal IntraPredModeY[ xNbX ][ yNbX ].

[0239] 3. The variables ispDefaultMode1 and ispDefaultMode2 are defined as follows:

[0240] – If IntraSubPartitionsSplitType is equal to ISP_HOR_SPLIT, then set ispDefaultMode1 to equal INTRA_ANGULAR18 and set ispDefaultMode2 to equal INTRA_ANGULAR5.

[0241] Otherwise, set ispDefaultMode1 to equal INTRA_ANGULAR50 and ispDefaultMode2 to equal INTRA_ANGULAR63.

[0242] 4. The candModeList[x] (x = 0…5) is exported as follows:

[0243] – If candIntraPredModeB equals candIntraPredModeA and candIntraPredModeA is greater than INTRA_DC, then candModeList[x] (x = 0…5) is exported as follows:

[0244] – If IntraLumaRefLineIdx[xCb][yCb] equals 0 and IntraSubPartitionsSplitType equals ISP_NO_SPLIT, then the following applies:

[0245] candModeList[ 0 ] = candIntraPredModeA (8-9)

[0246] candModeList[ 1 ] = INTRA_PLANAR (8-10)

[0247] candModeList[ 2 ] = INTRA_DC (8-11)

[0248] candModeList[ 3 ] = 2 + ( ( candIntraPredModeA + 61 ) % 64 )(8-12)

[0249] candModeList[ 4 ] = 2 + ( ( candIntraPredModeA - 1 ) % 64 ) (8-13)

[0250] candModeList[ 5 ] = 2 + ( ( candIntraPredModeA + 60 ) % 64 )(8-14)

[0251] – Otherwise (if IntraLumaRefLineIdx[xCb][yCb] is not equal to 0 or IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT), then the following applies:

[0252] candModeList[ 0 ] = candIntraPredModeA (8-15)

[0253] candModeList[ 1 ] = 2 + ( ( candIntraPredModeA + 61 ) % 64 )(8-16)

[0254] candModeList[ 2 ] = 2 + ( ( candIntraPredModeA - 1 ) % 64 ) (8-17)

[0255] –If one of the following conditions is true

[0256] –IntraSubPartitionsSplitType equals ISP_HOR_SPLIT and candIntraPredModeA is less than INTRA_ANGULAR34,

[0257] – IntraSubPartitionsSplitType equals ISP_VER_SPLIT and candIntraPredModeA is greater than or equal to INTRA_ANGULAR34.

[0258] – IntraLumaRefLineIdx[xCb][yCb] is not equal to 0,

[0259] The following applies:

[0260] candModeList[ 3 ] = 2 + ( ( candIntraPredModeA + 60 ) % 64 )(8-18)

[0261] candModeList[ 4 ] = 2 + ( candIntraPredModeA % 64 ) (8-19)

[0262] candModeList[ 5 ] = 2 + ( ( candIntraPredModeA + 59 ) % 64 )(8-20)

[0263] – Otherwise, the following applies:

[0264] candModeList[ 3 ] = ispDefaultMode1 (8-21)

[0265] candModeList[4] = ispDefaultMode2 (8-22)

[0266] candModeList[ 5 ] = INTRA_PLANAR (8-23)

[0267] Otherwise, if candIntraPredModeB is not equal to candIntraPredModeA and either candIntraPredModeA or candIntraPredModeB is greater than INTRA_DC, then the following applies:

[0268] – The variables minAB and maxAB are derived as follows:

[0269] minAB = Min( candIntraPredModeA, candIntraPredModeB ) (8-24)

[0270] maxAB = Max( candIntraPredModeA, candIntraPredModeB ) (8-25)

[0271] – If both candIntraPredModeA and candIntraPredModeB are greater than INTRA_DC, then candModeList[x] (x = 0…5) is exported as follows:

[0272] candModeList[ 0 ] = candIntraPredModeA (8-26)

[0273] candModeList[ 1 ] = candIntraPredModeB (8-27)

[0274] – If IntraLumaRefLineIdx[xCb][yCb] equals 0 and IntraSubPartitionsSplitType equals ISP_NO_SPLIT, then the following applies:

[0275] candModeList[ 2 ] = INTRA_PLANAR (8-28)

[0276] candModeList[ 3 ] = INTRA_DC (8-29)

[0277] – If maxAB - minAB is in the range of 2 to 62 (inclusive), then the following applies:

[0278] candModeList[ 4 ] = 2 + ( ( maxAB + 61 ) % 64 ) (8-30)

[0279] candModeList[ 5 ] = 2 + ( ( maxAB - 1 ) % 64 ) (8-31)

[0280] – Otherwise, the following applies:

[0281] candModeList[ 4 ] = 2 + ( ( maxAB + 60 ) % 64 ) (8-32)

[0282] candModeList[ 5 ] = 2 + ( ( maxAB ) % 64 ) (8-33)

[0283] – Otherwise (if IntraLumaRefLineIdx[xCb][yCb] is not equal to 0 or IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT), then the following applies:

[0284] – If IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT, and abs(candIntraPredModeB - ispDefaultMode1) is less than abs(candIntraPredModeA - ispDefaultMode1), then the following applies:

[0285] candModeList[ 0 ] = candIntraPredModeB (8-34)

[0286] candModeList[ 1 ] = candIntraPredModeA (8-35)

[0287] –If maxAB - minAB equals 1, then the following applies:

[0288] candModeList[ 2 ] = 2 + ( ( minAB + 61 ) % 64 ) (8-36)

[0289] candModeList[ 3 ] = 2 + ( ( maxAB - 1 ) % 64 ) (8-37)

[0290] candModeList[ 4 ] = 2 + ( ( minAB + 60 ) % 64 ) (8-38)

[0291] candModeList[ 5 ] = 2 + ( maxAB % 64 ) (8-39)

[0292] Otherwise, if maxAB - minAB equals 2, then the following applies:

[0293] candModeList[ 2 ] = 2 + ( ( minAB - 1 ) % 64 ) (8-40)

[0294] candModeList[ 3 ] = 2 + ( ( minAB + 61 ) % 64 ) (8-41)

[0295] candModeList[ 4 ] = 2 + ( ( maxAB - 1 ) % 64 ) (8-42)

[0296] candModeList[ 5 ] = 2 + ( ( minAB + 60 ) % 64 ) (8-43)

[0297] Otherwise, if maxAB - minAB is greater than 61, then the following applies:

[0298] candModeList[ 2 ] = 2 + ( ( minAB - 1 ) % 64 ) (8-44)

[0299] candModeList[ 3 ] = 2 + ( ( maxAB + 61 ) % 64 ) (8-45)

[0300] candModeList[ 4 ] = 2 + ( minAB % 64 ) (8-46)

[0301] candModeList[ 5 ] = 2 + ( ( maxAB + 60 ) % 64 ) (8-47)

[0302] – Otherwise, the following applies:

[0303] candModeList[ 2 ] = 2 + ( ( minAB + 61 ) % 64 ) (8-48)

[0304] candModeList[ 3 ] = 2 + ( ( minAB - 1 ) % 64 ) (8-49)

[0305] candModeList[ 4 ] = 2 + ( ( maxAB + 61 ) % 64 ) (8-50)

[0306] candModeList[ 5 ] = 2 + ( ( maxAB - 1 ) % 64 ) (8-51)

[0307] Otherwise (if candIntraPredModeA or candIntraPredModeB is greater than INTRA_DC), candModeList[x] (x = 0…5) is exported as follows:

[0308] – If IntraLumaRefLineIdx[xCb][yCb] equals 0 and IntraSubPartitionsSplitType equals ISP_NO_SPLIT, then the following applies:

[0309] candModeList[ 0 ] = candIntraPredModeA (8-52)

[0310] candModeList[ 1 ] = candIntraPredModeB (8-53)

[0311] candModeList[ 2 ] = 1 - minAB (8-54)

[0312] candModeList[ 3 ] = 2 + ( ( maxAB + 61 ) % 64 ) (8-55)

[0313] candModeList[ 4 ] = 2 + ( ( maxAB - 1 ) % 64 ) (8-56)

[0314] candModeList[ 5 ] = 2 + ( ( maxAB + 60 ) % 64 ) (8-57)

[0315] Otherwise, if IntraLumaRefLineIdx[xCb][yCb] is not equal to 0, then the following applies:

[0316] candModeList[ 0 ] = maxAB (8-58)

[0317] candModeList[ 1 ] = 2 + ( ( maxAB + 61 ) % 64 ) (8-59)

[0318] candModeList[ 2 ] = 2 + ( ( maxAB - 1 ) % 64 ) (8-60)

[0319] candModeList[ 3 ] = 2 + ( ( maxAB + 60 ) % 64 ) (8-61)

[0320] candModeList[ 4 ] = 2 + ( maxAB % 64 ) (8-62)

[0321] candModeList[ 5 ] = 2 + ( ( maxAB + 59 ) % 64 ) (8-63)

[0322] – Otherwise (IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT), the following applies:

[0323] candModeList[ 0 ] = INTRA_PLANAR (8-64)

[0324] candModeList[ 1 ] = maxAB (8-65)

[0325] candModeList[ 2 ] = 2 + ( ( maxAB + 61 ) % 64 ) (8-66)

[0326] candModeList[ 3 ] = 2 + ( ( maxAB - 1 ) % 64 ) (8-67)

[0327] candModeList[ 4 ] = 2 + ( ( maxAB + 60 ) % 64 ) (8-68)

[0328] candModeList[ 5 ] = 2 + ( maxAB % 64 ) (8-69)

[0329] – Otherwise, the following applies:

[0330] – If IntraLumaRefLineIdx[xCb][yCb] equals 0 and IntraSubPartitionsSplitType equals ISP_NO_SPLIT, then the following applies:

[0331] candModeList[ 0 ] = candIntraPredModeA (8-70)

[0332] candModeList[ 1 ] = ( candModeList[0] = = INTRA_PLANAR )? INTRA_DC:INTRA_PLANAR (8-71)

[0333] candModeList[ 2 ] = INTRA_ANGULAR50 (8-72)

[0334] candModeList[ 3 ] = INTRA_ANGULAR18 (8-73)

[0335] candModeList[ 4 ] = INTRA_ANGULAR46 (8-74)

[0336] candModeList[ 5 ] = INTRA_ANGULAR54 (8-75)

[0337] Otherwise, if IntraLumaRefLineIdx[xCb][yCb] is not equal to 0, then the following applies:

[0338] candModeList[ 0 ] = INTRA_ANGULAR50 (8-76)

[0339] candModeList[ 1 ] = INTRA_ANGULAR18 (8-77)

[0340] candModeList[ 2 ] = INTRA_ANGULAR2 (8-78)

[0341] candModeList[ 3 ] = INTRA_ANGULAR34 (8-79)

[0342] candModeList[ 4 ] = INTRA_ANGULAR66 (8-80)

[0343] candModeList[ 5 ] = INTRA_ANGULAR26 (8-81)

[0344] Otherwise, if IntraSubPartitionsSplitType equals ISP_HOR_SPLIT, then the following applies:

[0345] candModeList[ 0 ] = INTRA_PLANAR (8-82)

[0346] candModeList[ 1 ] = INTRA_ANGULAR18 (8-83)

[0347] candModeList[ 2 ] = INTRA_ANGULAR25 (8-84)

[0348] candModeList[ 3 ] = INTRA_ANGULAR10 (8-85)

[0349] candModeList[ 4 ] = INTRA_ANGULAR65 (8-86)

[0350] candModeList[ 5 ] = INTRA_ANGULAR50 (8-87)

[0351] Otherwise, if IntraSubPartitionsSplitType equals ISP_VER_SPLIT, then the following applies:

[0352] candModeList[ 0 ] = INTRA_PLANAR (8-88)

[0353] candModeList[ 1 ] = INTRA_ANGULAR50 (8-89)

[0354] candModeList[ 2 ] = INTRA_ANGULAR43 (8-90)

[0355] candModeList[ 3 ] = INTRA_ANGULAR60 (8-91)

[0356] candModeList[ 4 ] = INTRA_ANGULAR3 (8-92)

[0357] candModeList[ 5 ] = INTRA_ANGULAR18 (8-93)

[0358] 5. Export IntraPredModeY[xCb][yCb] using the following procedure:

[0359] – If intra_luma_mpm_flag[xCb][yCb] equals 1, then set IntraPredModeY[xCb][yCb] to equal candModeList[intra_luma_mpm_idx[xCb][yCb]].

[0360] – Otherwise, export IntraPredModeY[xCb][yCb] by applying the following sequential steps:

[0361] 1. If for i = 0…4 and for each i, j = (i + 1)…5, candModeList[i] is greater than candModeList[j], then swap the two values ​​as follows:

[0362] ( candModeList[ i ], candModeList[ j ] ) = Swap( candModeList[ i ], candModeList[ j ] ) (8-94)

[0363] 2. Export IntraPredModeY[xCb][yCb] by applying the following sequential steps:

[0364] i. Set IntraPredModeY[xCb][yCb] to be equal to intra_luma_mpm_remainder[xCb][yCb].

[0365] ii. For i equal to 0 to 5 (inclusive), if IntraPredModeY[xCb][yCb] is greater than or equal to candModeList[i], then increment the value of IntraPredModeY[xCb][yCb] by one.

[0366] Set the variable IntraPredModeY[x][y] (x = xCb…xCb + cbWidth - 1 and y = yCb…yCb + cbHeight - 1) to be equal to IntraPredModeY[xCb][yCb].

[0367] 3. Examples of problems solved by the embodiments

[0368] In LIC, two parameters need to be derived using nearby reconstructed samples, including the scaling parameter and the offset b, which may cause latency issues.

[0369] The set of allowed weighting factors used in GBI is fixed, which may be inefficient.

[0370] CIIP mode is only available in non-skip Merge mode, which may be less efficient.

[0371] In the current design, CIIP mode flags should be stored because the CIIP mode flags of neighboring blocks are used for intra-mode derivation processing of intra-codec blocks and intra-mode derivation processing of CIIP codec blocks.

[0372] In the current design, the CIIP weight derivation can be further improved, for example, by simplifying the checking of two neighboring blocks and three different weight pairs.

[0373] In the current design, during CIIP weight derivation processing, if a neighboring block is encoded in BDPCM mode, it is treated as an intra-frame mode. However, it might be more reasonable to treat it as a non-intra-frame mode, since BDPCM is designed for encoding screen content.

[0374] 4. Examples of Implementation Methods

[0375] In the following text, the term "block" is used to refer to the transform unit (TU), predictor unit (PU), codec unit (CU), or sub-block within the transform unit (TU), predictor unit (PU), codec unit (CU), etc.

[0376] The detailed techniques described below should be considered as examples to illustrate general concepts. These techniques should not be interpreted narrowly. Furthermore, these techniques can be combined in any way.

[0377] Assuming (xCb, yCb) is the position of the top-left sample point of the current block relative to the top-left sample point of the current image, cbWidth and cbHeight specify the width and height of the current block.

[0378] 1. The CIIP flag and / or LIC flag and / or spread filter flag and / or bilateral filter flag and / or transform domain filter flag and / or other types of reconstructed filter enable flags can be constrained to false (CIIP and / or LIC and / or spread filter and / or bilateral filter and / or transform domain filter can be implicitly disabled), depending on the encoding / decoding mode of other blocks (such as adjacent or non-adjacent neighboring blocks and / or reference blocks).

[0379] a. When a flag is exported as false, the corresponding method may not be applied.

[0380] b. When a flag is exported as false, skip the signaling notification for the use indication of this type of pattern.

[0381] c. When a flag is exported as false, signaling may still be used to indicate the use of such a mode, but it is constrained to false in the consistency bitstream and such a mode is not applied.

[0382] d. In one example, whether one or more of the proposed constraints (such as 1.a, 1.b, and 1.c) are applied may depend on the encoding / decoding patterns of all or some adjacent and / or non-adjacent rows or columns.

[0383] i. Alternatively, whether to apply this constraint may depend on at least N (N>=1) samples in adjacent or non-adjacent rows or columns that are not encoded using a specific mode.

[0384] e. In one example, adjacent and / or non-adjacent rows may include the row above and / or the row to the upper right.

[0385] f. In one example, adjacent and / or non-adjacent adjacent columns may include the left column and / or the bottom left and / or top left columns.

[0386] g. In one example, the specific mode of a block that is not the current block may include intra-frame mode and / or CIIP mode and / or CPR mode.

[0387] h. In one example, if any of the adjacent / non-adjacent blocks in adjacent and / or non-adjacent rows or columns are encoded or decoded using a specific mode (e.g., intra-frame and / or CIIP mode and / or CPR mode), then one or more proposed constraints (such as 1.a, 1.b and 1.c) are applied.

[0388] i. In one example, if all adjacent / non-adjacent blocks in adjacent and / or non-adjacent adjacent rows or columns are encoded and decoded using a specific mode (e.g., intra-frame and / or CIIP and / or CPR modes), then one or more of the proposed constraints are applied.

[0389] j. In one example, if at least N adjacent / non-adjacent blocks in adjacent or non-adjacent rows or columns are not encoded or decoded using a specific mode (e.g., intra-frame and / or CIIP mode and / or CPR mode), then one or more of the proposed constraints (such as 1.a, 1.b and 1.c) shall not be applied.

[0390] k. In one example, whether one or more of the proposed constraints (such as 1.a, 1.b, and 1.c) are applied may depend on the position of the current block.

[0391] i. In one example, if the current block is at the top of the current CTU (the current block and its adjacent blocks above it belong to different CTUs), the proposed constraint is not applied.

[0392] ii. In one example, if the current block is to the left of the current CTU (the current block and its left neighboring block belong to different CTUs), the proposed constraint is not applied.

[0393] 2. The CIIP flag and / or intra-frame mode of CIIP mode can be stored together with motion information in a history-based motion vector prediction (HMVP) table.

[0394] a. In one example, when comparing two candidate motion information (such as trimming), the CIIP flag in CIIP mode or / and intra-frame mode are not considered in the comparison.

[0395] b. In one example, when comparing two candidate motion information, the CIIP flag in the CIIP mode and / or the intra-frame mode are considered in the comparison.

[0396] c. In one example, when a Merge candidate comes from an entry in the HMVP table, the CIIP flag of that entry is also copied to the Merge candidate.

[0397] d. In one example, when a Merge candidate comes from an entry in the HMVP table, the CIIP flag and intra-frame mode of that entry are also copied to the Merge candidate.

[0398] 3. CIIP can be executed for AMVP mode (AMVP mode or / and AMVP with SMVD mode).

[0399] a. In the Merge mode and / or UMVE (also known as Merge with Motion Vector Difference, or MMVD) mode of the current block, the CIIP flags and / or intra-frame modes in the CIIP mode of adjacent / non-adjacent blocks may be inherited.

[0400] b. Signaling can be used to notify the CIIP flag and / or intra-frame mode in the CIIP mode for non-skip Merge mode or / and non-skip UMVE mode, and in skip Merge mode or / and skip UMVE mode, the CIIP flag and / or intra-frame mode in the CIIP mode of adjacent / non-adjacent blocks can be inherited.

[0401] c. The CIIP flag and / or intra-frame mode in the CIIP mode can be notified by the skip Merge mode or / and skip UMVE mode signaling, and in the non-skip Merge mode or / and non-skip UMVE mode, the CIIP flag and / or intra-frame mode in the CIIP mode of adjacent / non-adjacent blocks can be inherited.

[0402] d. In one example, CIIP flags and intra-frame mode can be notified for AMVP mode signaling.

[0403] e. In one example, in Merge mode and / or UMVE mode, the CIIP flag can be inherited, and if the CIIP flag is true, then if multiple intra-frame modes are allowed in CIIP, further signaling can be used to notify the intra-frame modes.

[0404] f. In one example, the CIIP flags and intra-frame modes of neighboring / non-neighboring blocks in Merge mode and / or UMVE mode can be inherited by the current block.

[0405] g. In one example, CIIP can be disabled for skip mode.

[0406] h. In one example, when comparing two Merge candidates, the CIIP flags and / or intra-frame modes in CIIP may not be considered in the comparison.

[0407] i. In one example, when comparing two Merge candidates, the CIIP flag and / or intra-frame mode in CIIP may be considered in the comparison.

[0408] 4. CIIP can be disabled for pairwise predictions or combined bidirectional predictions or other types of virtual / artificial candidates (e.g., zero motion vector candidates).

[0409] a. Alternatively, CIIP can be enabled for pairwise or combined bidirectional Merge candidates if one of the two candidates included in the pairwise or combined bidirectional prediction uses CIIP prediction.

[0410] i. In one example, the intra-frame mode of a CIIP mode candidate can be inherited.

[0411] ii. In one example, the intra-frame mode can be explicitly signaled.

[0412] b. Alternatively, CIIP can be enabled for pairwise or combined bidirectional merge candidates if both candidates included in the pairwise or combined bidirectional predictions are predicted using CIIP.

[0413] i. In one example, the intra-frame mode of one of the two candidates can be inherited.

[0414] ii. In one example, an intra-mode can be derived from two candidate intra-modes and used for pairwise or combined bidirectional merge candidates.

[0415] iii. In one example, the intra-frame mode can be explicitly signaled.

[0416] 5. The neighboring and / or non-adjacent spatial reconstruction samples required for CIIP mode or / and diffused filters or / and bilateral filters or / and transform domain filters or / and other types of reconstructed filters can be replaced with corresponding samples in the reference image (such as neighboring and / or non-adjacent spatial samples of the reference block, e.g.) Figure 17 (As shown).

[0417] a. In one example, if the current block is bidirectionally predicted, the nearest samples of the two reference blocks can be averaged to generate the final nearest samples.

[0418] b. In one example, if the current block is bidirectionally predicted, the nearest sample from one of the two reference blocks can be used as the final nearest sample.

[0419] c. In one example, if the current block is bidirectionally predicted, then if unequal weighted GBI or weighted prediction or LIC is applied to the current block, the neighboring samples of the two reference blocks can be weighted to generate the final neighboring samples.

[0420] d. In one example, the motion information of the current block can be used to identify the corresponding sample point in the reference image (e.g., the neighboring sample point of the reference block).

[0421] e. In one example, corresponding samples in a reference image (e.g., neighboring samples of the reference block) can be identified using the modified motion information of the current block. For example, the motion vector can be rounded to integer precision before being used to identify neighboring samples.

[0422] f. In one example, corresponding samples in a reference image (e.g., neighboring samples of a reference block) are identified via motion vectors with integer pixel precision.

[0423] i. In one example, the reference block's MV is first rounded to an integer pixel. The rounded MV is then used to identify neighboring samples of the reference block.

[0424] ii. In one example, the proposed replacement is applied when the current block is encoded and decoded with integer pixel motion information. Therefore, rounding is not required.

[0425] g. In one example, the proposed substitution may be applied only to specific color components, such as the luminance component.

[0426] i. Alternatively, the proposed substitutions can be applied to all color components.

[0427] 6. Propose prohibiting the use of previously encoded / decoded CIIP flags for encoding / decoding subsequent blocks.

[0428] a. In one example, skipping the inspection of CIIP flags from previous codec blocks improves throughput.

[0429] b. In one example, skip the checking process for CIIP flags from neighboring blocks during the intra-frame mode derivation process.

[0430] i. In one example, for a neighboring block, if it is encoded / decoded in CIIP mode, the associated intra-frame mode can be set to a given mode (such as planar mode). Alternatively, the associated intra-frame mode can also be used for MPM list derivation processing.

[0431] ii. In one example, for a neighboring block, if it is encoded or decoded in CIIP mode or regular inter-frame mode, the associated intra-frame mode can be set to a given mode (such as planar mode). Alternatively, the associated intra-frame mode can be used for MPM list derivation processing.

[0432] iii. In one example, the intra-frame mode of the CIIP codec block may not be stored, and the CIIP codec block may be considered unavailable in the decoding process. Alternatively, the CIIP codec block can be treated in the same way as a regular inter-frame mode.

[0433] c. Alternatively, it is also proposed to remove the CIIP flag from memory to save memory size required for storing mode information.

[0434] 7. A method for encoding and decoding CIIP mode flags using bypass encoding and decoding is proposed.

[0435] a. Alternatively, the CIIP mode flag can be context-coded but does not reference the CIIP mode flag of neighboring blocks.

[0436] 8. A method for weighted summation in CIIP is proposed that can depend on the color components.

[0437] a. For example, the weighting values ​​in CIIP are different for the primary color component (e.g., the G component) and other color components (e.g., the B and R components).

[0438] b. For example, the weighting values ​​in CIIP are different for the luminance component and the chrominance component.

[0439] c. For example, for inter - frame prediction and intra - frame prediction, the weighting values are equal for chrominance components.

[0440] 9. It is proposed that when the weighting factor selection is based on neighboring blocks, for a neighboring block, if it is encoded and decoded in the CIIP mode, it can be regarded as an inter - frame encoded and decoded block.

[0441] a. Alternatively, when the weighting factor selection is based on neighboring blocks, for a neighboring block, if it is encoded and decoded in the CIIP mode, it can be regarded as an intra - frame encoded and decoded block.

[0442] 10. The method proposed above can be applied under specific conditions, such as block size, slice / picture / tile type, or motion information.

[0443] a. In one example, when the block size contains less than M×H samples (e.g., 16, 32, or 64 luma samples), the proposed method is not allowed.

[0444] b. Alternatively, when the minimum size of the width or / and height of the block is less than or not greater than X, the proposed method is not allowed. In one example, X is set to 8.

[0445] c. Alternatively, when the minimum size of the width or / and height of the block is not less than X, the proposed method is not allowed. In one example, X is set to 8.

[0446] d. Alternatively, when the width of the block > th1 or >= th1 and / or the height of the block > th2 or >= th2, the proposed method is not allowed. In one example, th1 and / or th2 are set to 8.

[0447] e. Alternatively, when the width of the block < th1 or <= th1 and / or the height of the block < th2 or < a = th2, the proposed method is not allowed. In one example, th1 and / or th2 are set to 8.

[0448] 11. The number of allowed weight pairs (wIntra, wInter) used in CIIP can be reduced from 3 to 2, denoted as (a, b) and (c, d).

[0449] a. In one example, the two pairs are defined as {(1, 3) and (3, 1)}; or {(1, 3) and (2, 2)}; or {(3, 1) and (2, 2)}; or {(3, 5) and (4, 4)} or {(5, 3) and (4, 4)}, or {(1, 7) and (4, 4)} or {(7, 1) and (4, 4)}.

[0450] b. In one example, the weight pair is determined based only on one neighboring block A.

[0451] i. In one example, if a neighboring block A is available and is intra-coded, then (wIntra, wInter) is set to equal (a, b); otherwise, (wIntra, wInter) is set to equal (c, d).

[0452] ii. In one example, block A is the left neighboring block.

[0453] iii. In one example, block A is the block above it.

[0454] c. In one example, weight pairs are determined based on two or more neighboring blocks, and at least one of the neighboring blocks satisfies certain conditions.

[0455] i. In one example, if at least one of the neighboring blocks A and B is available and is intra-coded, then (wIntra, wInter) is set to equal (A, B); otherwise, (wIntra, wInter) is set to equal (c, d).

[0456] 1. In one example, block A is the left neighboring block, and block B is the top neighboring block.

[0457] d. In one example, weight pairs are determined based on two or more neighboring blocks, and each neighboring block satisfies the same conditions.

[0458] i. In one example, if neighboring blocks A and B are both available and are both intra-coded, then (wIntra, wInter) is set to equal (A, B); otherwise, (wIntra, wInter) is set to equal (c, d).

[0459] 1. In one example, block A is the left neighboring block, and block B is the top neighboring block.

[0460] e. In one example, the settings for (a, b) and (c, d) can be as follows:

[0461] 1. In one example, (a, b) is set to (2, 2) or (3, 1).

[0462] 2. In one example, (c, d) is set to equal (1, 3).

[0463] 3. In one example, at least one of the following two conditions is true: a is not equal to c, or b is not equal to d.

[0464] 4. (a, b) is not equal to (c, d).

[0465] f. In one example, the aforementioned neighboring blocks (e.g., A or B) are adjacent or non-adjacent spatial or temporal neighboring blocks.

[0466] i. In one example, neighboring block A or B is the left (or top) neighboring block.

[0467] ii. In one example, neighboring blocks A and B are the left neighboring block and the top neighboring block, respectively.

[0468] iii. In one example, the left neighboring block covers the location (xCb-1, yCb+cbHeight-1).

[0469] iv. In one example, the top adjacent block covers the location (xCb+cbWidth-1, yCb-1).

[0470] v. In one example, the left adjacent block covers the location (xCb-1, yCb).

[0471] vi. In one example, the top adjacent block covers the location (xCb, yCb-1).

[0472] g. For the example above, the CIIP prediction is formed as follows:

[0473]

[0474] The offset is set to (1<<(N-1)) or 0, and N can be set to log2(wIntra+wInter).

[0475] 12. The number of allowed weight pairs (wIntra, wInter) used in CIIP can be reduced from 3 to 1.

[0476] a. In one example, a weight pair is defined as (1,3) or (2,2) or (1,7) or (2,6) or (3,5) or (4,4).

[0477] b. In one example, (wIntra, wInter) is set to equal (2, 2).

[0478] c. In one example, (wIntra, wInter) is set to equal (1, 3).

[0479] d. For the example above, the CIIP prediction is formed as follows:

[0480]

[0481] The offset is set to (1<<(N-1)) or 0, and N can be set to log2(wIntra+wInter).

[0482] 13. Weight pairs (wIntra, wInter) can be derived based on whether one or more neighboring blocks are encoded or decoded using CIIP mode.

[0483] a. Weight pairs for the current block can be derived from weight pairs used for previous codec blocks.

[0484] b. In one example, weight pairs are determined based on only one neighboring block A.

[0485] i. In one example, if a neighboring block A is encoded in CIIP mode, then the (wIntra, wInter) of the current block is set to be equal to the (wIntra, wInter) of the neighboring block A; otherwise, if the neighboring block A is available and is intra-frame encoded, then (wIntra, wInter) is set to be equal to (a, b); otherwise, (wIntra, wInter) is set to be equal to (c, d).

[0486] ii. In one example, if a neighboring block A is encoded and decoded in CIIP mode, then the (wIntra, wInter) of the current block is set to be equal to the (wIntra, wInter) of the neighboring block; otherwise, (wIntra, wInter) is set to be equal to (a, b).

[0487] iii. For the above example, the following may apply:

[0488] 1. In one example, (a, b) is set to be equal to (2, 2) or (3, 1).

[0489] 2. In one example, (c, d) is set to equal (1, 3).

[0490] 3. In one example, at least one of the following two conditions is true: a is not equal to c; b is not equal to d.

[0491] c. In one example, weight pairs are determined based on two or more neighboring blocks, and at least one of the neighboring blocks satisfies a specific condition.

[0492] i. Multiple neighboring blocks can be checked in a given inspection order (e.g., check the left block first, then check the top block) to identify the use of the CIIP pattern.

[0493] ii. In one example, if at least one of the neighboring blocks A and B is encoded in CIIP mode, then the (wIntra, wInter) of the current block is set to be equal to the (wIntra, wInter) of the first CIIP-encoded neighboring block in the given inspection order; otherwise, if at least one of the neighboring blocks A and B is available and is intra-frame encoded, then (wIntra, wInter) is set to be equal to (a, b); otherwise, (wIntra, wInter) is set to be equal to (c, d).

[0494] iii. For the above example, the following may apply:

[0495] 1. In one example, (a, b) is set to be equal to (2, 2) or (3, 1).

[0496] 2. In one example, (c, d) is set to equal (1, 3).

[0497] 3. In one example, at least one of the following two conditions is true: a is not equal to c; b is not equal to d.

[0498] iv. In one example, if at least one of the neighboring blocks A and B is encoded in CIIP mode, then the (wIntra, wInter) of the current block is set to be equal to the (wIntra, wInter) of the first CIIP-encoded neighboring block in the given inspection order; otherwise, if both neighboring blocks A and B are available and are intra-frame encoded, then (wIntra, wInter) is set to be equal to (a, b); otherwise, (wIntra, wInter) is set to be equal to (c, d).

[0499] v. In one example, if at least one of the neighboring blocks A and B is encoded in CIIP mode, then the (wIntra, wInter) of the current block is set to be equal to the (wIntra, wInter) of the first CIIP-encoded neighboring block in the given inspection order; otherwise, (wIntra, wInter) is set to be equal to (c, d).

[0500] vi. For the above example, the following may apply:

[0501] 1. In one example, (c, d) is set to be equal to (2, 2) or (1, 3).

[0502] d. In one example, the aforementioned neighboring blocks A or B are adjacent or non-adjacent spatial or temporal neighboring blocks.

[0503] i. In one example, neighboring block A or B is the left (or top) neighboring block.

[0504] ii. In one example, neighboring blocks A and B are the left neighboring block and the top neighboring block, respectively.

[0505] iii. In one example, the left neighboring block covers the location (xCb-1, yCb+cbHeight-1).

[0506] iv. In one example, the top adjacent block covers the location (xCb+cbWidth-1, yCb-1).

[0507] v. In one example, the left adjacent block covers the location (xCb-1, yCb).

[0508] vi. In one example, the top adjacent block covers the location (xCb, yCb-1).

[0509] e. For the example above, the CIIP prediction is formed as follows:

[0510]

[0511] The offset is set to (1<<(N-1)) or 0, and N can be set to log2(wIntra+wInter).

[0512] 14. The selection of neighboring blocks used in CIIP can depend on encoding and decoding information such as block dimension / block shape / low latency check flag / reference picture information / motion information of the current block and neighboring blocks / intra-prediction mode of neighboring blocks.

[0513] 15. The weight pairs used in CIIP can depend on encoding and decoding information such as block dimension / block shape / low latency check flag / reference picture information / motion information of the current block and neighboring blocks / intra-prediction mode of neighboring blocks.

[0514] a. In one example, further signaling is given to the set of weight pairs.

[0515] b. In one example, the selection of weight pairs may depend on whether neighboring blocks are inter-frame encoded rather than intra-frame encoded.

[0516] 16. When determining CIIP information (e.g., for the previous design and the above project, when determining the weights applied to intra-prediction signals or inter-prediction signals), if a neighboring block is not encoded with MODE_INTRA (i.e., intra-mode), it can be considered as intra-coded.

[0517] a. In one example, if the neighboring block is IBC encoded / decoded.

[0518] b. In one example, if the neighboring block is CIIP encoded or decoded.

[0519] c. In one example, if the neighboring block is TPM encoded / decoded.

[0520] d. In one example, if the adjacent blocks are encoded in Palette.

[0521] e. In one example, if the neighboring block is RDPCM encoded or decoded.

[0522] f. In one example, if the adjacent block is encoded or decoded without applying transforms (e.g., transform skip mode) and / or quantization (e.g., transform bypass quantization mode).

[0523] g. Alternatively, if the neighboring block is encoded or decoded using a mode mentioned in the above sub-items (e.g., IBC, RDPCM, Palette), then the neighboring block may be considered as non-intra-frame encoded or decoded (e.g., inter-frame encoded).

[0524] h. In one example, if a neighboring block is intra-coded but not encoded with a specific prediction mode (e.g., plane), then the neighboring block can be considered non-intra-coded (e.g., inter-coded).

[0525] i. Alternatively, if the neighboring block is encoded using a matrix-based intra-prediction method and / or a multi-reference line intra-prediction method (where the reference line index is not equal to K, e.g., K=0) and / or BDPCM, then the neighboring block can be considered as non-intra-codec (e.g., inter-codec).

[0526] j. The proposed method can be applied to other encoding and decoding tools, depending on whether neighboring blocks are encoded and decoded in intra-frame mode.

[0527] k. The proposed method can be applied to other encoding and decoding tools, depending on whether neighboring blocks are encoded and decoded in inter-frame mode.

[0528] 17. Signaling notifications in SPS / PPS / VPS / sequence header / image header / strip header / fragment group header / CTU group, etc., can be enabled or disabled using the above methods.

[0529] a. Alternatively, the signaling method to be used can be indicated in SPS / PPS / VPS / sequence header / picture header / strip header / fragmentation header / CTU group, etc.

[0530] b. Alternatively, whether to enable or disable the above methods and / or which method to apply may depend on the block dimension, Video Processing Data Unit (VPDU), picture type, low latency check flag, and codec information of the current block or previous codec blocks (e.g., reference picture, one-way or two-way prediction).

[0531] 5. Examples

[0532] 5.1 Example 1

[0533] Below is an example of skipping the inspection of the CIIP flags of previous codec blocks.

[0534] 8.4.2 Derivation and processing of the lumen intra-frame prediction mode

[0535] The input for this process is:

[0536] The luminance position (xCb, yCb) specifies the top-left sample of the current luminance block relative to the top-left luminance sample of the current image.

[0537] The variable cbWidth specifies the width of the current codec block in the luminance sample.

[0538] The variable cbHeight specifies the height of the current codec block in the luminance sample.

[0539] In this process, the luminance intra-prediction mode IntraPredModeY[ xCb ][ yCb ] is derived.

[0540] Table 8-1 specifies the values ​​and related names of the IntraPredModeY [xCb] [yCb].

[0541] Table 8-1 Specifications of Intra-Frame Prediction Modes and Related Names

[0542]

[0543] Note: The intra-prediction modes INTRA_LT_CCLM, INTRA_L_CCLM, and INTRA_T_CCLM apply only to the chroma component.

[0544] The IntraPredModeY[xCb][yCb] is derived by the following sequential steps:

[0545] 1. Set the neighboring positions (xNbA, yNbA) and (xNbB, yNbB) to equal (xCb -1, yCb + cbHeight - 1) and (xCb + cbWidth - 1, yCb - 1) respectively.

[0546] 2. For cases where X is replaced by A or B, the variable candIntraPredModeX is exported as follows:

[0547] – Take the position (xCurr, yCurr) as input and set it to be equal to (xCb, yCb), and the neighboring position (xNbY, yNbY) as equal to (xNbX, yNbX), invoke the availability derivation process for the block as specified in Section 6.4.X [Ed. (BB): Pending Neighboring Block Availability Check Processing], and assign the output to availableX.

[0548] – The derivation of the candidate intra-prediction mode candIntraPredModeX is as follows:

[0549] – If one or more of the following conditions are true, then set candIntraPredModeX to be equal to INTRA_PLANAR.

[0550] – The variable availableX is equal to FALSE.

[0551] – CuPredMode[ xNbX ][ yNbX ] is not equal to MODE_INTRA.

[0552] –pcm_flag[ xNbX ][ yNbX ] equals 1.

[0553] – X equals B and yCb - 1 is less than ((yCb >> CtbLog2SizeY) < <CtbLog2SizeY )。

[0554] Otherwise, set candIntraPredModeX to be equal to IntraPredModeY[ xNbX ][ yNbX ].

[0555] 3. The variables ispDefaultMode1 and ispDefaultMode2 are defined as follows:

[0556] – If IntraSubPartitionsSplitType is equal to ISP_HOR_SPLIT, then set ispDefaultMode1 to equal INTRA_ANGULAR18 and set ispDefaultMode2 to equal INTRA_ANGULAR5.

[0557] Otherwise, set ispDefaultMode1 to equal INTRA_ANGULAR50 and ispDefaultMode2 to equal INTRA_ANGULAR63.

[0558] 4. The candModeList[x] (x = 0…5) is exported as follows:

[0559] …

[0560] Example 2

[0561] If the left neighbor is available and is intra-coded, then set (wIntra, wInter) to (2, 2); otherwise, set (wIntra, wInter) to (1, 3).

[0562] The CIIP predictions are as follows:

[0563]

[0564] Example 3

[0565] If the left neighbor is available and is intra-coded, then set (wIntra, wInter) to (3, 1); otherwise, set (wIntra, wInter) to (1, 3).

[0566] The CIIP predictions are as follows:

[0567]

[0568] Example 4

[0569] If the top neighbor is available and is intra-coded, then set (wIntra, wInter) to (2, 2); otherwise, set (wIntra, wInter) to (1, 3).

[0570] The CIIP predictions are as follows:

[0571]

[0572] Example 5

[0573] If the top neighbor is available and is intra-coded, then set (wIntra, wInter) to equal (3, 1); otherwise, set (wIntra, wInter) to equal (1, 3).

[0574] The CIIP predictions are as follows:

[0575]

[0576] Example 6

[0577] If the left and top neighbors are available and are intra-coded, then set (wIntra, wInter) to (2, 2); otherwise, set (wIntra, wInter) to (1, 3).

[0578] The CIIP predictions are as follows:

[0579]

[0580] Example 7

[0581] If the left and top neighbors are available and are intra-coded, then set (wIntra, wInter) to equal (3, 1); otherwise, set (wIntra, wInter) to equal (1, 3).

[0582] The CIIP predictions are as follows:

[0583]

[0584] Example 8

[0585] If the left or top neighbor is available and is intra-coded, then set (wIntra, wInter) to equal (2, 2); otherwise, set (wIntra, wInter) to equal (1, 3).

[0586] The CIIP predictions are as follows:

[0587]

[0588] Example 9

[0589] If the left or top neighbor is available and is intra-coded, then set (wIntra, wInter) to equal (3, 1); otherwise, set (wIntra, wInter) to equal (1, 3).

[0590] The CIIP predictions are as follows:

[0591]

[0592] Example 10

[0593] Set (wIntra, wInter) to equal (2, 2).

[0594] The CIIP predictions are as follows:

[0595]

[0596] Example 11

[0597] Set (wIntra, wInter) to equal (1, 3).

[0598] The CIIP predictions are as follows:

[0599]

[0600] Example 12

[0601] If the left neighbor is CIIP encoded, then set (wIntra, wInter) of the current block to be equal to (wIntra, wInter) of the left neighbor; otherwise, if the left neighbor is available and is intra-coded, then set (wIntra, wInter) to be equal to (3, 1); otherwise, set (wIntra, wInter) to be equal to (1, 3).

[0602] The CIIP predictions are as follows:

[0603]

[0604] Example 13

[0605] If the left neighbor is CIIP-coded, then set (wIntra, wInter) of the current block to be equal to (wIntra, wInter) of the left neighbor; otherwise, if the left neighbor is available and is intra-coded, then set (wIntra, wInter) to be equal to (2, 2); otherwise, set (wIntra, wInter) to be equal to (1, 3).

[0606] The CIIP predictions are as follows:

[0607]

[0608] Example 14

[0609] If the top neighbor is CIIP-coded, then set (wIntra, wInter) of the current block to be equal to (wIntra, wInter) of the top neighbor; otherwise, if the top neighbor is available and is intra-coded, then set (wIntra, wInter) to be equal to (3, 1); otherwise, set (wIntra, wInter) to be equal to (1, 3).

[0610] The CIIP predictions are as follows:

[0611]

[0612] Example 15

[0613] If the top neighbor is CIIP-coded, then set (wIntra, wInter) of the current block to be equal to (wIntra, wInter) of the top neighbor; otherwise, if the top neighbor is available and is intra-coded, then set (wIntra, wInter) to be equal to (2, 2); otherwise, set (wIntra, wInter) to be equal to (1, 3).

[0614] The CIIP predictions are as follows:

[0615]

[0616] Example 16

[0617] If the left and / or top neighbor is CIIP-coded, then the (wIntra, wInter) of the current block is set to be equal to the (wIntra, wInter) of the first CIIP-coded neighbor (from left to top); otherwise, if the left and / or top neighbor is available and is intra-coded, then (wIntra, wInter) is set to be equal to (3, 1); otherwise, (wIntra, wInter) is set to be equal to (1, 3).

[0618] The CIIP predictions are as follows:

[0619]

[0620] Example 17

[0621] If the left and / or top neighbor is CIIP encoded, then the (wIntra, wInter) of the current block is set to be equal to the (wIntra, wInter) of the first CIIP encoded neighbor (from left to top); otherwise, if the left and / or top neighbor is available and is intra-frame encoded, then (wIntra, wInter) is set to be equal to (2, 2); otherwise, (wIntra, wInter) is set to be equal to (1, 3).

[0622] The CIIP predictions are as follows:

[0623]

[0624] Example 18

[0625] If the left neighbor is CIIP encoded or decoded, then set (wIntra, wInter) of the current block to be equal to (wIntra, wInter) of the left neighbor; otherwise, set (wIntra, wInter) to be equal to (1, 3).

[0626] The CIIP predictions are as follows:

[0627]

[0628] Example 19

[0629] If the left neighbor is CIIP encoded, then set (wIntra, wInter) of the current block to be equal to (wIntra, wInter) of the left neighbor; otherwise, set (wIntra, wInter) to be equal to (2, 2).

[0630] The CIIP predictions are as follows:

[0631]

[0632] Example 20

[0633] If the top neighbor is CIIP encoded, then set (wIntra, wInter) of the current block to be equal to (wIntra, wInter) of the top neighbor; otherwise, set (wIntra, wInter) to be equal to (1, 3).

[0634] The CIIP predictions are as follows:

[0635]

[0636] Example 21

[0637] If the top neighbor is CIIP encoded, then set (wIntra, wInter) of the current block to be equal to (wIntra, wInter) of the top neighbor; otherwise, set (wIntra, wInter) to be equal to (2, 2).

[0638] The CIIP predictions are as follows:

[0639]

[0640] Example 22

[0641] If the left and / or top neighbor is CIIP encoded, then set (wIntra, wInter) of the current block to be equal to (wIntra, wInter) of the first CIIP encoded neighbor (from left to top); otherwise, set (wIntra, wInter) to be equal to (1, 3).

[0642] The CIIP predictions are as follows:

[0643]

[0644] Example 23

[0645] If the left and / or top neighbor is / is CIIP encoded, then set (wIntra, wInter) of the current block to be equal to (wIntra, wInter) of the first CIIP encoded neighbor (from left to top); otherwise, set (wIntra, wInter) to be equal to (2, 2).

[0646] The CIIP predictions are as follows:

[0647]

[0648] Example 24

[0649] 8.5.6 Decoding of Inter-Frame Blocks

[0650] 8.5.6.1 Overview

[0651] This process is invoked when decoding a codec unit encoded in inter-frame prediction mode.

[0652] When ciip_flag[xCb][yCb] equals 1, the array predSamples of the predicted samples is modified as follows:

[0653] – If cIdx equals 0, then the following applies:

[0654] – Set the position (xTbCmp, yTbCmp) to equal (xCb, yCb), the intra-prediction mode predModeIntra to equal IntraPredModeY[xCb][yCb], the transform block width nTbW and height nTbH to equal cbWidth and cbHeight, the codec block width nCbW and height nCbH to equal cbWidth and cbHeight, and the variable cIdx as input. Call the regular intra-sample prediction process specified in Section 8.4.5.2.5, and assign the output to the (cbWidth) x (cbHeight) array predSamplesIntraL .

[0655] – Set the position (xTbCmp, yTbCmp) to equal to (xCb, yCb), the codec block width cbWidth, the codec block height cbHeight, and set the sample arrays predSamplesInter and predSamplesIntra to equal predSamples and predSamplesIntra respectively. L Set the intra-prediction mode predModeIntra to equal IntraPredModeY[xCb][yCb] and the color component index cIdx as input, call the combined Merge and weighted sample prediction processing of intra-prediction as specified in Section 8.5.6.7, and assign the output to the (cbWidth)x(cbHeight) array predSamples.

[0656] …

[0657] 8.5.6.7 Weighted Sample Prediction Processing Combining Merge and Intra-Frame Prediction

[0658] The input for this process is:

[0659] – Luminance position (xCb, yCb), which specifies the top-left sample of the current luminance block relative to the top-left luminance sample of the current image.

[0660] – The width of the current codec block, cbWidth

[0661] – The height of the current codec block, cbHeight

[0662] – Two (cbWidth) x (cbHeight) arrays, predSamplesInter and predSamplesIntra.

[0663] – The variable cIdx specifies the color component index.

[0664] The output of this process is an array of (cbWidth) x (cbHeight) predicted sample values, named predSamplesComb.

[0665] The derivation of the variable bitDepth is as follows:

[0666] – If cIdx equals 0, then set bitDepth to BitDepth. Y .

[0667] Otherwise, set bitDepth to BitDepth. C .

[0668] The variable callFact is derived as follows:

[0669] scallFact = ( cIdx == 0 ) ? 0 : 1. (8-838)

[0670] Set the adjacent luminance positions (xNbA, yNbA) and (xNbB, yNbB) to be equal to (xCb-1, yCb-1+(cbHeight<<scallFact)) and (xCb-1+(cbWidth<<scallFact), yCb-1), respectively.

[0671] For the case where X is replaced by A or B, the variables availableX and isIntraCodedNeighbourX are derived as follows:

[0672] – Set the position (xCurr, yCurr) to be equal to (xCb, yCb), the adjacent position (xNbY, yNbY) to be equal to (xNbX, yNbX), checkPredModeY to FALSE, and set cIdx to be equal to 0 as input, call the derivation process of adjacent block availability specified in Article 6.4.4, and assign the output to availableX.

[0673] – The derivation of the variable isIntraCodedNeighbourX is as follows:

[0674] – If availableX is equal to TRUE, and CuPredMode[0][xNbX][yNbX] is equal to MODE_INTRA, and BdpcmFlag[xNbX][yNbX] is equal to 0, then set isIntraCodedNeighbourX to be equal to TRUE.

[0675] – Otherwise, set isIntraCodedNeighbourX to be equal to FALSE.

[0676] The weight w is derived as follows:

[0677] – If both isIntracodedNeighbourA and isIntraCodedNeighbourB are equal to TRUE, then set w to be equal to 3.

[0678] – Otherwise, if both isIntracodedNeighbourA and isIntraCodedNeighbourB are equal to FALSE, then set w to be equal to 1.

[0679] Otherwise, set w to equal 2.

[0680] When cIdx equals 0 and slice_lmcs_enabled_flag equals 1, predSamplesInter[x][y](x=0..cbWidth-1 and y=0..cbHeight-1) is modified as follows:

[0681] Figure 18 This is a block diagram of a video processing apparatus 1800. Apparatus 1800 can be used to implement one or more methods described herein. Apparatus 1800 can be implemented in smartphones, tablets, computers, Internet of Things (IoT) receivers, etc. Apparatus 1800 may include one or more processors 1802, one or more memories 1804, and video processing hardware 1806. Processor 1802 can be configured to implement one or more methods described herein. Memory 1804 can be used to store data and code for implementing the methods and techniques described herein. Video processing hardware 1806 can be used to implement some of the techniques described herein in hardware circuitry.

[0682] Figure 20 This is a flowchart of a video processing method 2000. Method 2000 includes: determining (2005) a encoding / decoding mode of a first video block; constraining (2110) one or more flags to an operation state, said operation state being false or true, and performing (2015) further processing of a second video block according to the operation state of the one or more flags, wherein the first video block is a neighboring video block or a reference video block associated with the second video block.

[0683] Figure 21 This is a flowchart of a method 2100 for processing video. Method 2100 includes: determining (2105) to use pairwise prediction or combined bidirectional prediction for a first video block; determining (2110) an operational state of combined inter-frame intra-frame prediction (CIIP) based on the determination of using pairwise prediction or combined bidirectional prediction, wherein the operational state is enabled or disabled; and performing (2115) further processing of the first video block according to the operational state of CIIP.

[0684] Figure 23This is a flowchart of a method 2300 for processing video. Method 2300 includes: determining (2305) one or more codec modes of one or more second blocks during a conversion between a first block in video data and a bitstream representation of the first block; determining (2310) codec mode constraints of the first block based on the one or more codec modes of the first block; performing (2315) a conversion by at least applying the codec mode constraints of the first block; wherein the one or more second blocks include at least one of a neighboring block, a non-neighboring block, and a reference block of the first block.

[0685] Figure 24 This is a flowchart of a method 2400 for processing video. Method 2400 includes: performing (2405) a conversion between a current block in video data and a bitstream representation of the current block by using at least one of combined inter-frame intra-frame prediction (CIIP), spread filtering, bilateral filtering, transform domain filtering, or another type of post-reconstruction filtering different from spread filtering, bilateral filtering, and transform domain filtering, wherein the reconstructed neighboring samples of the current block used in at least one of combined inter-frame intra-frame prediction (CIIP), spread filtering, bilateral filtering, transform domain filtering, or another type of post-reconstruction filtering are replaced with approximate samples generated from corresponding samples of reconstructed neighboring samples in one or more reference pictures.

[0686] Figure 25 This is a flowchart of a method 2500 for processing video. Method 2500 includes: storing (2505) CIIP flags and / or intra-frame modes of a combined inter-frame intra-frame prediction (CIIP) mode along with motion information in a history-based motion vector prediction (HMVP) table; and performing (2510) a conversion between the current block in the video data and the bitstream representation of the current block, at least based on the HMVP table.

[0687] Figure 26 This is a flowchart of a method 2600 for processing video. Method 2600 includes: determining (2605) a prediction mode for the current block during a conversion between the current block in video data and its bitstream representation; determining (2610) the suitability of a combined inter-frame intra-frame prediction (CIIP) mode, the suitability indicating that CIIP mode is enabled for the current block in response to determining whether the current block uses Advanced Motion Vector Prediction (AMVP) mode or Merge mode encoding / decoding; and performing (2615) a conversion based on the suitability of the CIIP mode.

[0688] Figure 27This is a flowchart of a method 2700 for processing video. Method 2700 includes: during a conversion between a current block in video data and a bitstream representation of the current block, determining (2705) the type of a selected merge candidate for the current block; and determining (2710) the suitability of combined inter-frame intra-frame prediction (CIIP) for the current block based on the type of the selected merge candidate, wherein the current block is encoded and decoded in merge mode.

[0689] Referring to methods 2000 and 2100, Section 4 of this paper describes some examples of combined inter-frame and intra-frame prediction for encoding and their usage. For example, as described in Section 4, video blocks can be processed based on the combined intra-frame and inter-frame prediction usage.

[0690] Reference methods 2000 and 2100 allow for the encoding of video blocks within a video bitstream, where bit efficiency can be achieved by using bitstream generation rules associated with combined inter-frame and intra-frame prediction.

[0691] The method may include: wherein one or more flags include a combined inter-frame intra-frame prediction (CIIP) flag, a local illumination compensation (LIC) flag, a spread filter flag, a bilateral filter flag, a transform domain filter flag, or another type of reconstructed post-filter flag.

[0692] The method may include: wherein one or more flags include a first flag associated with a first operation, the first flag being derived as false, and the first operation not being applied.

[0693] The method may include: wherein the use of the first operation is signaled to be skipped.

[0694] The method may include: wherein, in a consistent bitstream, the use of a first operation is signaled as a constraint being false.

[0695] The method may include: wherein the first video block is in an adjacent row or column relative to the second video block.

[0696] The method may include: wherein the adjacent row includes the row above or the row to the upper right.

[0697] The method may include: wherein the adjacent column includes the left column, the column below the left, or the column above the left.

[0698] The method may include: wherein the encoding / decoding mode includes intra-frame mode, CIIP mode, or CPR mode.

[0699] The method may include: determining the location of a second video block, wherein constraining one or more flags to a false state is also based on the determination of the location of the second video block.

[0700] The method may include: wherein the position is located at the top of the current codec tree unit (CTU), and the adjacent blocks above the second video block are located in different CTUs.

[0701] The method may include: wherein the position is located to the left of the current codec tree unit (CTU), and the left adjacent block of the second video block is located in a different CTU.

[0702] The method may include: wherein one or more flags include an intra-frame mode of a CIIP flag or a CIIP mode flag, and data associated with the one or more flags are stored together with motion information in a history-based motion vector prediction (HMVP) table.

[0703] The method may include: comparing two candidate motion information, wherein the CIIP flag or intra-frame mode CIIP mode flag is not used in the comparison of motion information.

[0704] The method may include comparing two candidate motion information, wherein the comparison of motion information uses a CIIP flag or an intra-frame mode CIIP mode flag.

[0705] The method may include: determining that the Merge candidate comes from an entry in the HMVP table; and copying the CIIP flag of the entry to the Merge candidate.

[0706] The method may include: determining that the Merge candidate is an entry from the HMVP table; and copying the CIIP flag and intra-frame mode of that entry to the Merge candidate.

[0707] The method may include: wherein one or more flags are associated with CIIP, and CIIP is performed for advanced motion vector prediction (AMVP).

[0708] The method may include: wherein, AMVP mode signaling notifies the CIIP flag and the intra-frame mode flag.

[0709] The method may include: inheriting the CIIP flag in Merge mode or UMVE mode, the CIIP flag including an operation state of true, multiple intra-frame modes allowed in CIIP, and signaling the intra-frame mode based on the multiple intra-frame modes allowed in CIIP.

[0710] The method may include: wherein the second video block inherits the CIIP flag and intra-frame mode of adjacent or non-adjacent video blocks in Merge mode or UMVE mode.

[0711] The method may include: wherein CIIP is disabled for skip mode.

[0712] The method may include: comparing information related to two candidates, wherein the CIIP flag or intra-frame mode CIIP mode flag is not used in the comparison of information.

[0713] The method may include: comparing information associated with two candidates, wherein the comparison of information uses a CIIP flag or an intra-frame mode CIIP mode flag.

[0714] The method may include: wherein one or more candidates involved in pairwise prediction or combined bidirectional prediction are predicted using CIIP, and wherein the operation state is enabled.

[0715] The method may include: wherein the intra-frame mode of one of the two candidates is inherited.

[0716] The method may include: wherein signaling notification of intra-frame mode.

[0717] The method may include: wherein pairwise prediction or combined bidirectional prediction involves two candidates.

[0718] The method may include replacing reconstructed samples used in CIIP mode, diffusion filter, bilateral filter, transform domain filter or other types of reconstructed filters with samples in a reference image.

[0719] The method may include: wherein the second video block is bidirectionally predicted, and the neighboring samples of the two reference blocks are averaged to generate the final neighboring samples.

[0720] The method may include: wherein the second video block is bidirectionally predicted, and the neighboring samples of the two reference blocks are used as the final neighboring samples.

[0721] The method may include: wherein the second video block is bidirectionally predicted, and the neighboring samples of the two reference blocks are weighted and averaged based on unequal-weighted generalized bidirectional prediction (GBI) to generate the final neighboring samples, or weighted prediction LIC is applied to the second video block.

[0722] The method may include: using motion information from a second video block to identify sample points in a reference image.

[0723] The method may include: using motion information from a second video block to identify sample points in a reference image.

[0724] The method may include: rounding the motion vector to integer precision and using it to identify sample points.

[0725] The method may include: wherein the sample points are in a reference image identified using motion vectors with integer pixel precision.

[0726] The method may include: determining characteristics of a second video block, the characteristics including one or more of block size, strip type, image type, slice type, or motion information, and wherein an operational state is constrained based on the determination of the characteristics.

[0727] The method may include: wherein the method is signaled in the Sequence Parameter Set (SPS), View Parameter Set (VPS), Picture Parameter Set (PPS), Sequence Header, Picture Header, Strip Header, Slice Group Header, Slice or CTU Group.

[0728] The following list of solutions further provides embodiments and variations of the items listed in the previous chapters (e.g., items 11 to 16).

[0729] 40. A video processing method comprising: performing a conversion between video blocks and codec representations of the video blocks using a combined inter-frame and intra-frame prediction mode, wherein the codec representations correspond to a weighted average of intra-frame and inter-frame prediction results of the video blocks using weight pairs from a set of weight pairs, the set of weight pairs containing fewer than three pairs.

[0730] 41. The method according to solution 40, wherein the set of weight pairs corresponds to two pairs.

[0731] 42. The method according to any one of solutions 40 to 41, wherein the weight pair is determined based on a single neighboring block.

[0732] 43. The method according to any one of solutions 40 to 41, wherein the weight pairs are determined based on a plurality of neighboring blocks.

[0733] 44. The method according to solution 40, wherein the set of weight pairs comprises exactly one pair.

[0734] 45. The method according to solution 44, wherein the pair is one of (1,3) or (2,2) or (1,7) or (2,6) or (3,5) or (4,4).

[0735] 46. ​​A video processing method comprising: performing a conversion between video blocks and codec representations of the video blocks using a combined inter-frame and intra-frame prediction mode, wherein the codec representations correspond to a weighted average of intra-frame and inter-frame prediction results of the video blocks using weight pairs from a set of weight pairs, the set of weight pairs being determined by codec information of one or more neighboring blocks.

[0736] 47. The method according to solution 46, wherein the set of weight pairs is determined by exactly one neighboring block.

[0737] 48. The method according to solution 46, wherein the set of weight pairs is determined by two or more neighboring blocks.

[0738] 49. The method according to any one of solutions 40 to 48, wherein the conversion includes encoding the video to generate the codec representation.

[0739] 50. The method according to any one of solutions 40 to 48, wherein the conversion includes decoding the codec representation to generate the video.

[0740] Various other embodiments and additional features of these methods can be described using the following clause-based description.

[0741] 1. A video processing method, comprising:

[0742] During the conversion between a first block of video data and the bitstream representation of the first block, one or more encoding / decoding modes for one or more second blocks are determined;

[0743] Based on the one or more encoding / decoding modes of the one or more second blocks, determine the encoding / decoding mode constraints of the first block; and

[0744] The conversion is performed at least by applying the encoding / decoding mode constraints of the first block;

[0745] The one or more second blocks include at least one of the adjacent blocks, non-adjacent blocks, and reference blocks of the first block.

[0746] 2. The method according to Clause 1, wherein determining the encoding / decoding mode constraints of the first block includes:

[0747] Based on the one or more codec modes of the one or more second blocks, determine one or more flags corresponding to the one or more codec modes of the first block.

[0748] 3. The method according to Clause 2, wherein the one or more encoding / decoding modes include:

[0749] The combined inter-frame intra-frame prediction (CIIP) mode, local illumination compensation (LIC) mode, spread filtering mode, bilateral filtering mode, transform domain filtering mode, or a reconstructed filtering mode different from the spread filtering mode, the bilateral filtering mode, and the transform domain filtering mode.

[0750] 4. The method according to any one of clauses 2 to 3, wherein the encoding / decoding mode constraint of the first block includes:

[0751] When the first flag corresponding to the first codec mode is exported as false, the first codec mode in the one or more codec modes of the first block is disabled.

[0752] 5. The method according to any one of clauses 2 to 4, wherein the encoding / decoding mode constraint of the first block includes:

[0753] When the second flag corresponding to the second codec mode is exported as false, the no signaling notification indicates whether the information of the second codec mode in the one or more codec modes of the first block is enabled.

[0754] 6. The method according to any one of clauses 2 to 5, wherein the encoding / decoding mode constraint of the first block includes:

[0755] When the third flag corresponding to the third codec mode is exported as false, the signaling notification indicates whether to enable the information of the third codec mode in the one or more codec modes of the first block and constrain it to false.

[0756] 7. The method according to any one of clauses 1 to 6 further includes:

[0757] Determine whether to apply the encoding / decoding mode constraints of the first block.

[0758] 8. The method according to Clause 7, wherein determining whether to apply the encoding / decoding mode constraint of the first block includes:

[0759] Whether to apply the encoding / decoding mode constraint of the first block is determined based on the encoding / decoding mode of the adjacent rows or columns of the first block.

[0760] 9. The method according to Clause 7, wherein determining whether to apply the encoding / decoding mode constraint of the first block includes:

[0761] Whether to apply the encoding / decoding mode constraint of the first block is determined based on at least N samples in the adjacent rows or columns of the first block that are not encoded using the fourth mode, and N≥1.

[0762] 10. The method according to Clause 8 or 9, wherein the adjacent row includes the row above and the row to the upper right of the first block.

[0763] 11. The method according to any one of Clauses 8 to 10, wherein the adjacent column includes the left column, the column below the left, and the column at the top left of the first block.

[0764] 12. The method according to any one of clauses 7 to 11, wherein determining whether to apply the encoding / decoding mode constraint of the first block comprises:

[0765] When any adjacent non-adjacent block is encoded and decoded using the fourth mode, the encoding and decoding mode constraint of the first block is determined to be applied.

[0766] 13. The method according to any one of clauses 7 to 11, wherein determining whether to apply the encoding / decoding mode constraint of the first block comprises:

[0767] When all adjacent non-adjacent blocks are encoded and decoded using the fourth mode, the encoding and decoding mode constraint of the first block is determined to be applied.

[0768] 14. The method according to any one of clauses 7 to 11, wherein determining whether to apply the encoding / decoding mode constraint of the first block comprises:

[0769] When at least M neighboring non-adjacent blocks are not encoded and decoded using the fourth mode, it is determined that the encoding and decoding mode constraint of the first block is not applied, and M is a predefined first threshold.

[0770] 15. The method according to any one of Clauses 9 to 14, wherein the fourth encoding / decoding mode includes at least one of an intra-frame prediction mode, a combined inter-frame intra-frame prediction (CIIP) mode, and a current picture reference (CPR) mode.

[0771] 16. The method according to any one of clauses 7 to 14, wherein determining whether to apply the encoding / decoding mode constraint of the first block comprises:

[0772] Whether to apply the encoding / decoding mode constraint of the first block is determined based on the position of the first block.

[0773] 17. The method according to Clause 16, wherein determining whether to apply the codec mode constraint of the first block based on the position of the first block comprises:

[0774] When the position is at the top of the current codec tree unit (CTU) and the adjacent block above it belongs to a different CTU from the first block, it is determined that the codec mode constraint of the first block is not applied.

[0775] 18. The method according to Clause 16, wherein determining whether to apply the encoding / decoding mode constraint of the first block based on the position of the first block comprises:

[0776] When the position is to the left of the current codec tree unit (CTU) and the left neighboring block and the first block belong to different CTUs, it is determined that the codec mode constraint of the first block is not applied.

[0777] 19. The method according to any one of clauses 1 to 18 further comprises:

[0778] Determine the characteristics of the current block; and

[0779] When the characteristics of the current block meet the predefined conditions, the encoding / decoding mode constraints of the current block are determined to be applied.

[0780] 20. The method according to any one of clauses 1 to 18 further comprises:

[0781] Determine the characteristics of the current block and / or the previous codec block; and

[0782] Whether to apply the encoding / decoding mode constraint of the current block is determined based on the characteristics of the current block and / or the previous encoding / decoding block.

[0783] 21. A video processing method, comprising:

[0784] A conversion between the current block in the video data and the bitstream representation of the current block is performed by using at least one of combined inter-frame intra-frame prediction (CIIP), spread filtering, bilateral filtering, transform domain filtering, or a reconstructed post-filtering of a different type than said spread filtering, said bilateral filtering, and said transform domain filtering.

[0785] Specifically, approximate samples generated from corresponding samples of reconstructed neighboring samples in one or more reference images are used to replace the reconstructed neighboring samples of the current block used in at least one of the combined inter-frame intra-frame prediction (CIIP), the spread filtering, the bilateral filtering, the transform domain filtering, or the other type of post-reconstruction filtering.

[0786] 22. The method according to Clause 21, wherein when the current block is a bidirectional prediction, the neighboring samples of the two reference blocks of the current block are averaged to generate the approximate sample.

[0787] 23. The method according to Clause 21, wherein when the current block is a bidirectional or unidirectional prediction, a neighboring sample of a reference block of the current block is used as the approximate sample.

[0788] 24. The method according to Clause 21, wherein when the current block is bidirectionally predicted and at least one of unequal-weighted generalized bidirectional prediction (GBI), weighted prediction, and local illumination compensation (LIC) is applied to the current block, a weighted average of neighboring samples of two reference blocks of the current block is performed to generate the approximate sample.

[0789] 25. The method according to clauses 22 to 24, wherein the neighboring samples of the reference block of the current block are identified by the motion information of the current block.

[0790] 26. The method according to clauses 22 to 24, wherein the neighboring samples of the reference block of the current block are identified by the modified motion information of the current block.

[0791] 27. The method according to Clause 24, wherein the modified motion information of the current block is a modified motion vector rounded to integer precision.

[0792] 28. The method according to Clause 21, wherein the neighboring samples of the reference block of the current block are identified using motion vectors with integer pixel precision.

[0793] 29. The method according to Clause 28, wherein the motion vectors of the reconstructed neighboring samples are rounded to integer precision to identify the corresponding samples.

[0794] 30. The method according to any one of clauses 21 to 29, wherein the reconstructed neighboring sample of the current block is replaced with the approximate sample only when the current block is encoded and decoded with integer pixel precision.

[0795] 31. The method according to any one of clauses 21 to 30, wherein the reconstructed neighboring samples of the current block are replaced with the approximate samples only for the luminance component or the chrominance component.

[0796] 32. The method according to any one of clauses 19 to 31 further comprises:

[0797] Determine the characteristics of the current block; and

[0798] When the characteristics of the current block meet the predefined conditions, it is determined to replace the reconstructed neighboring sample with the approximate sample.

[0799] 33. The method according to Clause 19 or 32, wherein the characteristics of the current block include at least one of block size, strip type, image type, slice type, and motion information.

[0800] 34. The method according to Clause 33, wherein the predefined condition is that the current block contains samples not less than a predefined second threshold.

[0801] 35. The method according to Clause 33, wherein the predefined condition is that the width and height of the current block are greater than a predefined third threshold.

[0802] 36. The method according to Clause 33, wherein the predefined condition is that the width and height of the current block are less than a predefined fourth threshold.

[0803] 37. The method according to Clause 33, wherein the predefined condition is that the width of the current block is less than a predefined fifth threshold and / or the height of the current block is less than a predefined sixth threshold.

[0804] 38. The method according to Clause 33, wherein the predefined condition is that the width of the current block is greater than a predefined seventh threshold and / or the height of the current block is greater than a predefined eighth threshold.

[0805] 39. The method according to Clause 38, wherein the signaling notification in the Sequence Parameter Set (SPS), View Parameter Set (VPS), Picture Parameter Set (PPS), Sequence Header, Picture Header, Strip Header, Slice Group Header, Slice or CTU Group indicates whether the encoding / decoding mode constraint of the current block is applied and / or whether the information of the reconstructed neighboring sample is replaced with the approximate sample.

[0806] 40. The method according to any one of clauses 21 to 39 further comprises:

[0807] Determine the characteristics of the current block and / or the previous codec block; and

[0808] Whether to replace the reconstructed neighboring sample with the approximate sample is determined based on the characteristics of the current block and / or the previous codec block.

[0809] 41. The method according to Clause 20 or 40, wherein the characteristics of the current block and / or the previous codec block include at least one of the following: block dimension, video processing data unit (VPDU), picture type, low latency check flag, and codec information.

[0810] 42. The method according to Clause 41, wherein the encoding / decoding information of the current block and / or the previous encoding / decoding block indicates a reference picture, one-way prediction, or two-way prediction.

[0811] 43. A video processing apparatus comprising a processor configured to implement the method of any one of claims 1 to 42.

[0812] 44. The apparatus as described in Clause 43, wherein the apparatus is a video encoder.

[0813] 45. The apparatus as described in Clause 43, wherein the apparatus is a video decoder.

[0814] 46. ​​A computer-readable recording medium having a program including code recorded thereon, the program causing a processor to perform any one of the methods described in clauses 1 to 42.

[0815] Various other embodiments and additional features of these methods can be described using the following clause-based description.

[0816] 1. A video processing method, comprising:

[0817] The history-based motion vector prediction (HMVP) table stores the CIIP flag and / or intra-frame mode, along with motion information, combining inter-frame intra-frame prediction (CIIP) modes; and

[0818] At least based on the HMVP table, perform the conversion between the current block in the video data and the bitstream representation of the current block.

[0819] 2. The method described in Clause 1 further includes:

[0820] Compare the motion information of the two candidates.

[0821] In the comparison of the two candidate motion information, the CIIP flag of the CIIP mode and / or the intra-frame mode are not used.

[0822] 3. The method described in Clause 1 further includes:

[0823] Compare the motion information of the two candidates.

[0824] The CIIP flag of the CIIP mode and / or the intra-frame mode are used in the comparison of the two candidate motion information.

[0825] 4. The method according to any one of clauses 1 to 3, wherein the execution comprises:

[0826] When a Merge candidate comes from an entry in the HMVP table, the CIIP flag of that entry is copied to the Merge candidate.

[0827] 5. The method according to any one of clauses 1 to 3, wherein the execution comprises:

[0828] When a Merge candidate comes from an entry in the HMVP table, the CIIP flag and the intra-frame mode of that entry are copied to the Merge candidate.

[0829] 6. A video processing method, comprising:

[0830] During the conversion between the current block in the video data and the bitstream representation of the current block, the prediction mode of the current block is determined;

[0831] Determine the suitability of the combined inter-frame intra-frame prediction (CIIP) mode, the suitability indicating whether the CIIP mode is enabled for the current block in response to determining whether the current block is encoded and decoded in Advanced Motion Vector Prediction (AMVP) mode or Merge mode; and

[0832] The conversion is performed based on the applicability of the CIIP mode.

[0833] 7. The method according to Clause 6, wherein when the current block is encoded or decoded in the AMVP mode, signaling notifies the CIIP flag of the CIIP mode and / or the intra-frame mode.

[0834] 8. The method according to Clause 6 or 7, wherein in the Merge mode and / or Merge (MMVD) mode with motion vector difference of the current block, the CIIP flags and / or intra-frame modes of the CIIP mode of adjacent neighboring blocks and / or non-adjacent neighboring blocks are inherited.

[0835] 9. The method according to any one of Clauses 6 to 8, wherein the CIIP flag and / or intra-frame mode of the CIIP mode are signaled for non-skip Merge mode and / or non-skip MMVD mode, and in skip Merge mode and / or skip MMVD mode, the CIIP flag and / or intra-frame mode of the CIIP mode of adjacent neighboring blocks and / or non-adjacent neighboring blocks are inherited.

[0836] 10. The method according to any one of Clauses 6 to 8, wherein the CIIP flag and / or intra-frame mode of the CIIP mode are signaled for skipping Merge mode and / or skipping MMVD mode, and in non-skip Merge mode and / or non-skip MMVD mode, the CIIP flag and / or intra-frame mode of the CIIP mode of adjacent neighboring blocks and / or non-adjacent neighboring blocks are inherited.

[0837] 11. The method according to any one of Clauses 6 to 10, wherein the CIIP flag is inherited in Merge mode and / or Skip MMVD mode.

[0838] 12. The method according to Clause 11, wherein when the CIIP flag is true and multiple intra-frame modes are allowed in the CIIP mode, the signaling notifies the intra-frame mode.

[0839] 13. The method according to any one of clauses 6 to 12, wherein the CIIP flags and / or intra-frame modes of the CIIP mode of adjacent and / or non-adjacent adjacent blocks encoded and decoded in Merge mode and / or MMVD mode are inherited by the current block.

[0840] 14. The method according to any one of clauses 6 to 13 further comprises:

[0841] Disable the CIIP mode for skip mode.

[0842] 15. The method according to any one of clauses 6 to 13 further comprises:

[0843] Compare the information of the two Merge candidates.

[0844] In the comparison of the information of the two Merge candidates, the CIIP flag of the CIIP mode and / or the intra-frame mode are not used.

[0845] 16. The method according to any one of clauses 6 to 13 further comprises:

[0846] Compare the information of the two Merge candidates.

[0847] The comparison of the information of the two Merge candidates uses the CIIP flag of the CIIP mode and / or the intra-frame mode.

[0848] 17. A video processing method, comprising:

[0849] During the conversion between the current block in the video data and the bitstream representation of the current block, the type of the selected merge candidate for the current block is determined; and

[0850] The suitability of combined inter-frame intra-frame prediction (CIIP) for the current block is determined based on the type of the selected Merge candidate.

[0851] The current block is encoded and decoded in Merge mode.

[0852] 18. The method according to Clause 17, wherein the CIIP is disabled when the selected Merge candidate of the current block is at least one of a pair of Merge candidates, a combined bidirectional Merge candidate, a zero-motion Merge candidate, or a virtual or artificial Merge candidate.

[0853] 19. The method according to Clause 17 or 18, wherein the CIIP mode is enabled for the pair of Merge candidates or the combined bidirectional Merge candidate when one of the candidates involved in the pair of Merge candidates or the combined bidirectional Merge candidate adopts the CIIP mode.

[0854] 20. The method according to Clause 19, wherein the intra-frame mode of the CIIP mode candidate involved is inherited.

[0855] 21. The method according to Clause 19, wherein the intra-frame mode of the relevant CIIP mode is notified for the pairwise Merge Candidate or combined bidirectional Merge Candidate signaling.

[0856] 22. The method according to Clause 17 or 18, wherein the CIIP mode is enabled for the pairwise Merge candidate or the combined bidirectional Merge candidate when both candidates involved in the pairwise Merge candidate or the combined bidirectional Merge candidate are in the CIIP mode.

[0857] 23. The method according to Clause 22, wherein the intra-frame mode of one of the two candidates involved is inherited.

[0858] 24. The method according to Clause 22, wherein the intra-frame mode of the involved CIIP mode is derived from the intra-frame modes of the two involved candidate modes and is used for the paired Merge candidate or the combined bidirectional Merge candidate.

[0859] 25. The method according to Clause 22, wherein the intra-frame mode of the relevant CIIP mode is notified for the paired Merge Candidate or the combined bidirectional Merge Candidate signaling.

[0860] 26. A video processing apparatus comprising a processor configured to implement the method of any one of claims 1 to 25.

[0861] 27. The apparatus as described in Clause 26, wherein the apparatus is a video encoder.

[0862] 28. The apparatus as described in Clause 26, wherein the apparatus is a video decoder.

[0863] 29. A computer-readable recording medium having a program including code recorded thereon, the program causing a processor to perform any one of the methods described in any one of clauses 1 to 25.

[0864] It should be understood that the disclosed techniques can be implemented in video encoders or decoders to improve compression efficiency using hash-based motion estimation.

[0865] The disclosed and other solutions, examples, embodiments, modules, and functional operations described herein can be implemented in digital electronic circuits or computer software, firmware, or hardware, including the structures disclosed herein and their structural equivalents, or combinations thereof. The disclosed embodiments and other embodiments can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer-readable medium for execution by or control of a data processing device. The computer-readable medium can be a combination of a machine-readable storage device, a machine-readable storage substrate, a storage device, a substance that influences machine-readable propagated signals, or one or more combinations thereof. The term "data processing device" includes all devices, apparatuses, and machines for processing data, including, for example, a programmable processor, a computer, or multiple processors or computers. In addition to hardware, the device may also include code that creates an execution environment for the computer program, for example, code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination thereof. The propagated signals are artificially generated signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode and decode information for transmission to a suitable receiver device.

[0866] Computer programs (also known as programs, software, software applications, scripts, or code) can be written in any programming language, including compiled or interpreted languages, and can be deployed in any form, including as standalone programs or as modules, components, subroutines, or other units suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to that program, or in multiple coordinating files (e.g., a file storing one or more modules, subroutines, or portions of code). Computer programs can be deployed to execute on one or more computers located at a single site or distributed across multiple sites interconnected by a communication network.

[0867] The processing and logic flows described in this specification can be executed by one or more programmable processors that execute one or more computer programs to perform functions by manipulating input data and generating outputs. The processing and logic flows can also be executed by special-purpose logic circuitry, and the device can be implemented as special-purpose logic circuitry, such as an FPGA (Field-Programmable Gate Array) or an ASIC (Application-Specific Integrated Circuit).

[0868] For example, processors suitable for executing computer programs include general-purpose and special-purpose microprocessors, as well as one or more of any type of digital computer. Typically, the processor receives instructions and data from read-only memory or random access memory, or both. The basic components of a computer are a processor that executes instructions and one or more storage devices that store the instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or receive data from or transfer data to one or more mass storage devices via operative coupling, or both. However, a computer does not necessarily have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable hard disks; magneto-optical disks; and CD-ROMs and DVD-ROMs. The processor and memory may be supplemented by or incorporated into special-purpose logic circuitry.

[0869] While this patent document contains numerous details, it should not be construed as limiting any subject matter or scope of the claims, but rather as a description of features of specific embodiments of a particular technology. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various functions described in the context of a single embodiment may also be implemented individually in multiple embodiments, or in any suitable sub-combination. Furthermore, although the foregoing features may be described as functioning in certain combinations, or even initially claimed to be so, in some cases one or more features from a combination of claims may be removed from the combination, and a combination of claims may refer to a sub-combination or a variation of a sub-combination.

[0870] Similarly, although the operations are described in a specific order in the accompanying drawings, this should not be construed as requiring the specific order or sequence shown to perform such operations, or all the described operations, in order to obtain the desired result. Furthermore, the separation of various system components in the embodiments described in this patent document should not be construed as requiring such separation in all embodiments.

[0871] Only some implementations and examples are described; other implementations, enhancements, and variations can be made based on the content described and illustrated in this patent document.

Claims

1. A method for processing video data, comprising: The conversion between the current block of the video and the bitstream of the video is performed by using at least one of inter-frame and intra-frame joint prediction (CIIP), spread filtering, bilateral filtering, transform domain filtering, or another type of post-reconstruction filtering different from said spread filtering, said bilateral filtering, and said transform domain filtering. Wherein, the reconstruction neighbor samples of the current block used in at least one of the CIIP, the diffusion filter, the bilateral filter, the transform domain filter, or the other type of post-reconstruction filter are replaced with approximate samples generated from corresponding samples of the reconstruction neighbor samples in one or more reference images.

2. The method according to claim 1, further comprising: Determine the characteristics of the current block; as well as When the characteristics of the current block meet the predefined conditions, it is determined to replace the reconstructed neighboring sample points with the approximate sample points.

3. The method according to claim 2, wherein, The characteristics of the current block include at least one of block size, strip type, image type, slice type, and motion information.

4. The method according to claim 3, wherein, The predefined condition is that the current block includes samples with a number not less than a predefined second threshold.

5. The method according to claim 3, wherein, The predefined condition is that the width and height of the current block are greater than a predefined third threshold.

6. The method according to claim 3, wherein, The predefined condition is that the width and height of the current block are less than a predefined fourth threshold.

7. The method according to claim 3, wherein, The predefined conditions are that the width of the current block is less than a predefined fifth threshold and / or the height of the current block is less than a predefined sixth threshold.

8. The method according to claim 3, wherein, The predefined conditions are that the width of the current block is greater than the predefined seventh threshold and / or the height of the current block is greater than the predefined eighth threshold.

9. The method according to claim 1, further comprising: The encoding / decoding mode constraints of the current block are determined, wherein the sequence parameter set SPS, view parameter set VPS, picture parameter set PPS, sequence header, picture header, strip header, slice group header, slice or CTU group indicate by signal transmission whether the encoding / decoding mode constraints of the current block are applied and / or whether the information of the reconstructed neighboring samples is replaced with the approximate samples.

10. The method according to claim 1, further comprising: Determine the characteristics of the previous encoded / decoded blocks of the current block; as well as Based on the characteristics of the previous codec block, determine whether to replace the reconstructed neighboring sample with the approximate sample.

11. The method according to claim 10, wherein, The characteristics of the previous codec block include at least one of block dimension, video processing data unit (VPDU), picture type, low latency check flag, and codec information of the previous codec block.

12. The method according to claim 11, wherein, The encoding / decoding information of the previous encoding / decoding block indicates a reference picture, one-way prediction, or two-way prediction.

13. The method according to claim 1, wherein, When the current block is a bidirectional prediction, the neighboring samples of the two reference blocks of the current block are averaged to generate the approximate sample; or, Wherein, when the current block is a bidirectional prediction or a unidirectional prediction, a neighboring sample point of a reference block of the current block is used as the approximate sample point.

14. The method according to claim 1, wherein, When the current block is bidirectionally predicted, and at least one of unequal-weight generalized bidirectional prediction (GBi), weighted prediction, and local illumination compensation (LIC) is applied to the current block, a weighted average of the neighboring samples of the two reference blocks of the current block is performed to generate the approximate sample.

15. The method according to claim 13, wherein, The motion information of the current block is used to identify the neighboring sample points of the reference block of the current block, or... The modified motion information of the current block identifies neighboring sample points of the reference block, and the modified motion information of the current block is a modified motion vector rounded to integer precision, or... The motion vectors of the reference block of the current block are identified using motion vectors with integer pixel precision, and the motion vectors of the reconstructed neighboring samples are rounded to integer precision to identify the corresponding samples.

16. The method according to claim 1, wherein, For either the luminance or chrominance component, the reconstructed neighboring samples of the current block are replaced with the approximate samples.

17. The method according to claim 1, wherein, The conversion includes encoding the current block into the bit stream.

18. The method according to claim 1, wherein, The conversion includes decoding the current block from the bitstream.

19. A video processing apparatus for processing video data, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to: The conversion between the current block of the video and the bitstream of the video is performed by using at least one of inter-frame and intra-frame joint prediction (CIIP), spread filtering, bilateral filtering, transform domain filtering, or another type of post-reconstruction filtering different from said spread filtering, said bilateral filtering, and said transform domain filtering. in, The reconstructed neighbor samples of the current block used in at least one of CIIP, the diffusion filter, the bilateral filter, the transform domain filter, or the other type of post-reconstruction filter are replaced with approximate samples generated from corresponding samples of the reconstructed neighbor samples in one or more reference images.

20. A non-transitory computer-readable recording medium having stored thereon a computer program / instructions and a bit stream, wherein, When the computer program / instructions are executed by the processor, they implement a method for processing video data to generate the bitstream, wherein the method for processing video data includes: The bitstream is generated for the current block of the video by using at least one of inter-frame and intra-frame joint prediction (CIIP), spread filtering, bilateral filtering, transform domain filtering, or a post-reconstruction filtering of a different type than said spread filtering, said bilateral filtering, and said transform domain filtering. Wherein, the reconstruction neighbor samples of the current block used in at least one of the CIIP, the diffusion filter, the bilateral filter, the transform domain filter, or the other type of post-reconstruction filter are replaced with approximate samples generated from corresponding samples of the reconstruction neighbor samples in one or more reference images.

21. A non-transitory computer-readable storage medium storing instructions that cause the processor to: The conversion between the current block of the video and the bitstream of the video is performed by using at least one of inter-frame and intra-frame joint prediction (CIIP), spread filtering, bilateral filtering, transform domain filtering, or another type of post-reconstruction filtering different from said spread filtering, said bilateral filtering, and said transform domain filtering. in, The reconstructed neighbor samples of the current block used in at least one of CIIP, the diffusion filter, the bilateral filter, the transform domain filter, or the other type of post-reconstruction filter are replaced with approximate samples generated from corresponding samples of the reconstructed neighbor samples in one or more reference images.

22. A method for storing a video bitstream, comprising: The bitstream is generated for the current block of the video by using at least one of inter-frame and intra-frame joint prediction (CIIP), spread filtering, bilateral filtering, transform domain filtering, or a post-reconstruction filtering of another type different from the spread filtering, the bilateral filtering, and the transform domain filtering. as well as The bitstream is stored in a non-transitory computer-readable recording medium. Wherein, the reconstruction neighbor samples of the current block used in at least one of the CIIP, the diffusion filter, the bilateral filter, the transform domain filter, or the other type of post-reconstruction filter are replaced with approximate samples generated from corresponding samples of the reconstruction neighbor samples in one or more reference images.

Images