Method and apparatus for combined prediction for video coding

Abstract

A combined prediction method and apparatus for video coding. According to the method, a combined prediction is derived, where the combined prediction includes a weighted combination of one or more source items. The weighting of the one or more source terms is derived using a regression technique based on relationship estimation of the one or more templates. The one or more source items correspond to different prediction modes of the current block, and the different prediction modes include at least one non-intra mode. Alternatively, one or more source items correspond to member samples determined in a mode associated with a sample to be predicted of the current block, and each member sample corresponds to an initial prediction sample of a combined prediction mode. The current block is encoded or decoded using information including the combined prediction.

Description

[0001] Cross-referencing

[0002] This invention is a non-provisional application of U.S. Provisional Patent Application No. 63 / 490,807, filed on March 17, 2023, and claims priority thereto. The entire U.S. Provisional Patent Application is incorporated herein by reference. [Technical Field]

[0003] This invention relates to video encoding and decoding systems. In particular, this invention relates to combined prediction, which improves the performance of intra-frame or inter-frame predictive encoding and decoding by using regression-based techniques to determine weights. [Background Technology]

[0004] Versatile Video Coding (VVC) is the latest international video coding standard developed by the ITU-T Video Coding Experts Group (VCEG) and the Joint Video Experts Team (JVET) of the International Organization for Standardization / International Electrotechnical Commission (ISO / IEC Moving Picture Experts Group). This standard has been published as an ISO standard: ISO / IEC 23090-3:2021, Information technology—Encoding representation of immersive media—Part 3: Versatile Video Coding, published in February 2021. VVC is developed based on its predecessor, High Efficiency Video Coding (HEVC), by adding more coding and decoding tools to improve coding and decoding efficiency, while handling various types of video sources, including three-dimensional (3D) video signals.

[0005] Figure 1AAn exemplary adaptive inter-frame / intra-frame video coding system incorporating loop processing is illustrated. For intra-frame prediction, the prediction data is derived from previously encoded video data in the current frame. For inter-frame prediction 112, motion estimation (ME) is performed at the encoder, and motion compensation (MC) is performed based on the results of ME to provide prediction data derived from other frames and motion data. Switch 114 selects either intra-frame prediction 110 or inter-frame prediction 112, and the selected prediction data is provided to adder 116 to form a prediction error, also known as a residual. The prediction error is then processed by transform (T) 118 and then quantized (Q) 120. The transformed and quantized residual is then encoded by entropy encoder 122 to be included in the video bitstream corresponding to the compressed video data. The bitstream associated with the transform coefficients is then packaged with additional information, such as the motion and encoding / decoding modes associated with intra-frame and inter-frame prediction, as well as other information, such as loop filter parameters applied to the underlying image regions. Additional information associated with intra-frame prediction 110, inter-frame prediction 112, and loop filter 130, such as Figure 1A The data is provided to the entropy encoder 122. When using inter-frame prediction mode, the reference picture or image must also be reconstructed at the encoder. Therefore, the transformed and quantized residuals are processed by inverse quantization (IQ) 124 and inverse transformation (IT) 126 to recover the residuals. The residuals are then added back to the prediction data 136 at reconstruction (REC) 128 to reconstruct the video data. The reconstructed video data can be stored in the reference picture buffer (RPB) 134 and used for prediction of other frames.

[0006] like Figure 1AAs shown, the input video data undergoes a series of processes in the encoding system. The reconstructed video data from REC 128 may be subject to various degradations due to these processes. Therefore, a loop filter 130 is typically applied to the reconstructed video data before it is stored in the reference picture buffer 134 to improve video quality. For example, a deblocking filter (DF), Sample Adaptive Offset (SAO), and Adaptive Loop Filter (ALF) may be used. Loop filter information may need to be included in the bitstream so that the decoder can correctly recover the required information. Therefore, loop filter information is also provided to the entropy encoder 122 for inclusion in the bitstream. Figure 1A In the process, the loop filter 130 is applied to the reconstructed video before the reconstructed samples are stored in the reference image buffer 134. Figure 1A The system described is intended to demonstrate an exemplary architecture of a typical video encoder. It may correspond to a High Efficiency Video Coding (HEVC) system, VP8, VP9, ​​H.264, or VVC.

[0007] like Figure 1B As shown, the decoder can use functional modules similar to or partially the same as the encoder, except for transform 118 and quantization 120, since the decoder only needs inverse quantization 124 and inverse transform 126. The decoder uses an entropy decoder (ED) 140 to decode the video bitstream into quantized transform coefficients and the required encoding / decoding information (e.g., ILPF information, intra-frame prediction information, and inter-frame prediction information). Intra-frame prediction 150 at the decoder end does not require mode search. Instead, the decoder only needs to generate intra-frame predictions based on the intra-frame prediction information received from the entropy decoder 140. Furthermore, for inter-frame prediction, the decoder only needs to perform motion compensation (MC 152) based on the inter-frame prediction information received from the entropy decoder 140, without needing to perform motion estimation.

[0008] According to VVC, the input image is divided into non-overlapping block regions called Coding Tree Units (CTUs), similar to HEVC. Each CTU can be further divided into one or more smaller Coding Units (CUs). The resulting CU partitions can be square or rectangular in shape. Furthermore, VVC divides the CTUs into Prediction Units (PUs), which serve as units for applying prediction processes, such as inter-frame prediction and intra-frame prediction.

[0009] CTU partitioning using a tree structure

[0010] In VVC, a quadtree partitioning structure using nested multi-type trees with binary and ternary partitions replaces the concept of multiple partition unit types. This removes the separation between CU, PU, ​​and TU concepts unless the CU size is excessively large, exceeding the maximum transform length, and allows for greater flexibility in CU partition shapes. In the codec tree structure, CUs can be square or rectangular. Codec tree units (CTUs) are first partitioned using a quadtree (also known as a quaternion tree) structure. Then, the leaf nodes of the quadtree can be further partitioned using multi-type tree structures. In most cases, CUs, PUs, and TUs have the same block size in a nested multi-type tree codec block structure. Exceptions occur when the maximum supported transform length is less than the width or height of the CU's color components.

[0011] In VVC, the codec tree scheme supports independent block tree structures for luma and chroma. For P and B slices, the luma and chroma CTBs within a single CTU must share the same codec tree structure. However, for I slices, luma and chroma can have independent block tree structures. When the independent block tree mode is applied, the luma CTB is divided into CUs via one codec tree structure, while the chroma CTB is divided into chroma CUs via another codec tree structure. This means that a CU in an I slice may consist of codec blocks for the luma component or codec blocks for the two chroma components, while a CU in a P or B slice always consists of codec blocks for all three color components, unless the video is monochrome.

[0012] Intra-mode encoding and decoding with 67 intra-prediction modes

[0013] To capture arbitrary edge orientations presented in natural video, the number of directional intra-frame modes in VVC is expanded from 33 used in HEVC to 65. In VVC, several traditional angular intra-frame prediction modes are adaptively replaced with wide-angle intra-frame prediction modes for non-square blocks. To keep the generation of the Most Probable Mode (MPM) list relatively low in complexity, an intra-frame mode encoding / decoding method with 6 MPMs is used, considering two available adjacent intra-frame modes. The following three aspects are considered when constructing the MPM list:

[0014] - Default intra-frame mode

[0015] -Intra-adjacent frame mode

[0016] - Export in-frame mode.

[0017] As described in JVET-D0114 (Seregin et al., “Block Shape Dependent Intra-Mode Coding and Decoding,” Joint Video Exploration Group (JVET) of ITU-T SG 16WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, Fourth Meeting: Chengdu, China, October 15-21, 2016, document JVET-D0114), a secondary MPM list is introduced. The existing primary MPM (PMPM) list contains 6 entries, and the secondary MPM (SMPM) list contains 16 entries. First, a general MPM list containing 22 entries is constructed. Then, the first 6 entries in this general MPM list are included in the PMPM list, and the remaining entries form the SMPM list. The first entry in the general MPM list is the planar mode. The remaining entries consist of the intra-modes of the left (L), top (A), bottom left (BL), top right (AR), and top left (AL) adjacent blocks, as shown below, with directional modes having offsets added from the first two available directional modes of the adjacent blocks, and a default mode.

[0018] Decoder-side Intra-Frame Mode Export (DIMD)

[0019] When DIMD is applied, multiple intra-frame patterns are derived from the reconstructed neighboring samples (e.g., pattern 1 and pattern 2 suggested by the DIMD derivation process), and these two predictions are combined with a weighted planar pattern prediction derived from the gradient. The DIMD pattern is used as an alternative prediction pattern and is always checked in the high-complexity RDO pattern.

[0020] To implicitly derive the intra-prediction mode of the block, texture gradient analysis is performed on both the encoder and decoder sides. This process begins with an empty gradient histogram (HoG) with 65 entries, corresponding to 65 angular modes. The magnitudes of these entries are determined during texture gradient analysis.

[0021] In the first step, DIMD selects a template of T=3 columns and 3 rows from the left and top sides of the current block, respectively. This region is used as a reference for gradient-based intra-prediction mode derivation.

[0022] In the second step, horizontal and vertical Sobel filters are applied to all 3×3 window locations, centered on the pixels along the midline of the template. At each window location, the Sobel filter calculates the intensity G in the purely horizontal and vertical directions, respectively. x and G y Then, the texture angle of the window is calculated as follows:

[0023] angle = arctan(G) x / G y ), (1)

[0024] This can be converted to one of 65 intra-prediction modes with different angles. Once the intra-prediction mode index idx for the current window is exported, the magnitude of its entry in HoG[idx] is updated as follows:

[0025] ampl=|G x |+|G y | (2)

[0026] Figure 2A -C shows an example of HoG calculated after applying the above operations to all pixel positions in the template. Figure 2A An example of template 220 selected for the current block 210 is shown. Template 220 includes T rows above the current block and T columns to the left of the current block. For intra-frame prediction of the current block, the regions 230 above and to the left of the current block correspond to the reconstruction region, and the regions 240 below and to the right of the block correspond to the unavailable region. Figure 2B An example with T=3 is shown, and HoG is calculated for pixel 260 in the middle line and pixel 222 in the middle column. For example, for pixel 252, a 3x3 window of 250 is used. Figure 2C An example is shown of the amplitude (abbreviated as Ampl in the figure) calculated based on equation (2) for the intra-frame prediction mode of the angle determined from equation (1).

[0027] Once the Histogram of Oriented Gradients (HoG) is calculated, if two intra-prediction modes are derived, the indices of the two highest histogram bars are selected as the two implicitly derived intra-prediction modes for that block, and further combined with the planar mode as the prediction for the DIMD mode. The prediction combination is applied as a weighted average of the three predictions. For this purpose, the weight of the planar mode is fixed at 21 / 64 (approximately 1 / 3). The remaining 43 / 64 (approximately 2 / 3) weight is distributed to the two HoG intra-prediction modes (IPM) according to the magnitude ratio of their HoG bars. Figure 3 An example of the mixing process is shown. For example... Figure 3 As shown, two intra-frame modes (M1 312 and M2 314) are selected based on the indices of the two highest bars in histogram bar 310. These three predictions (340, 342, and 344) are used to form a combined prediction. These three predictions correspond to applying M1, M2, and the planar intra-frame modes (320, 322, and 324, respectively) to reference pixel 330 to form the corresponding predictions. These three predictions are weighted by corresponding weighting factors (ω1, ω2, and ω3) 350. The weighted predictions are added by adder 352 to generate a combined prediction 360. It should be noted that if there is only one mode in the histogram (i.e., a single mode), no mixing process is performed, and there is no second DIMD mode.

[0028] Furthermore, these two implicitly exported intra-modes are included in the Most Probable Mode (MPM) list so that the DIMD process is performed before the MPM list is built. The primary exported intra-mode of the DIMD block is stored with the block and used for the construction of the MPM list of neighboring blocks.

[0029] Template-based Intra Mode Derivation (TIMD)

[0030] Template-based intra-mode derivation (TIMD) mode implicitly derives the intra-prediction mode of coding units (CUs) using neighboring templates in both the encoder and decoder, instead of explicitly signaling the intra-prediction mode to the decoder. For example... Figure 4 As shown, the predicted samples of the templates (412 and 414) for the current block 410 are generated by applying each candidate mode to the reference samples (420 and 422) of the template. The SATD (Sum of Absolute Transformed Differences) between the predicted samples and the reconstructed samples of the template is then calculated as the cost. The intra-prediction mode with the lowest cost is selected as the DIMD mode and used for intra-prediction of the CU. The candidate modes can be the same 67 intra-prediction modes as in the VVC, or expanded to 131 intra-prediction modes. Typically, the MPM can provide clues indicating CU orientation information. Therefore, to reduce the intra-prediction mode search space and utilize the characteristics of the CU, intra-prediction modes can be implicitly derived from the MPM list.

[0031] For each intra-prediction mode in the MPM, the SATD (TIMD cost) between the predicted and reconstructed samples of the template is calculated. The two intra-prediction modes with the minimum SATD are selected as the TIMD modes proposed in the TIMD derivation process. These two TIMD modes are weighted and combined after applying the Position Dependent IntraPrediction Combination (PDPC) process, and this weighted intra-prediction is used to encode the current CU. The PDPC process is included in the TIMD mode derivation.

[0032] Compare the costs of two selected models to a threshold. The following cost factor of 2 is applied during the test:

[0033] costMode2 < 2 * costMode1

[0034] If the condition is true, then combination is performed; otherwise, only Mode 1 (i.e., the single-mode case) is used. The weighting of modes is calculated based on their SATD costs as follows:

[0035] Weighted 1 = costMode2 / (costMode1 + costMode2)

[0036] Weighted 2 = 1 - Weighted 1

[0037] Template Matching Prediction (TMP)

[0038] Template matching prediction (TMP) is a special intra-frame prediction mode that copies the best prediction block from the reconstructed portion of the current frame, whose L-shaped template matches the current template.

[0039] Cross-Component Linear Model (CCLM)

[0040] The main idea of ​​the CCLM model (sometimes simply called the LM model) is as follows: the chromaticity components of a block can be predicted from the corresponding reconstructed luminance samples using a linear model whose parameters are derived from the reconstructed luminance and chromaticity samples of the block (which are located in adjacent positions of the block).

[0041] In VVC, the CCLM mode leverages inter-channel dependencies by predicting chromaticity samples from reconstructed luminance samples. This prediction is performed using a linear model of the following form:

[0042] P(i,j)=a·rec′ L (i,j)+b (3)

[0043] Where P(i,j) represents the predicted chromaticity sample in CU, rec′ L (i,j) represent reconstructed luminance samples from the same CU. In the case of a non-4:4:4 color format, these luminance samples are downsampled. Model parameters a and b are derived based on the reconstructed neighboring luminance and chrominance samples from both the encoder and decoder sides without explicit signaling.

[0044] In VVC, three CCLM modes are specified: CCLM_LT, CCLM_L, and CCLM_T. These three modes differ in the location of the reference samples used for model parameter derivation. In CCLM_T mode, only samples from the top boundary are used; in CCLM_L mode, only samples from the left boundary are used; and in CCLM_LT mode, samples from both the top and left boundaries are used.

[0045] Overview of Multimodal Linear Models (MMLM)

[0046] As the name suggests, the original CCLM model uses a linear model to predict chromaticity samples from luminance samples for the entire CU, while in MMLM (Multi-Model CCLM), there can be two models. In MMLM, the neighboring luminance samples and neighboring chromaticity samples of the current block are divided into two groups, each used as a training set to derive a linear model (i.e., deriving specific α and β for a specific group). Furthermore, the samples of the current luminance block are also classified based on the same rules used to classify neighboring luminance samples.

[0047] The threshold is calculated as the average value of the neighboring reconstructed brightness samples. Neighboring samples satisfy Rec′. L [x,y]<=threshold are classified as group 1; while neighboring samples satisfy Rec′ L The thresholds [x,y] are classified as group 2.

[0048] Accordingly, a linear model is used to obtain chromaticity prediction:

[0049]

[0050] Convolutional Cross Component Model (CCCM)

[0051] In CCCM, a convolutional model is applied to improve chromaticity prediction performance. The convolutional model has a 7-tap filter consisting of a 5-tap plus shapespace component, a nonlinear term, and a bias term.

[0052] The filter output is calculated as the convolution between the filter coefficients and the input values, and then cropped to the range of valid chromaticity samples.

[0053] The filter coefficients are calculated by minimizing the MSE between the predicted and reconstructed chromaticity samples in the reference region.

[0054] MSE minimization is performed by calculating the autocorrelation matrix of the luminance input and the cross-correlation vector between the luminance input and chrominance output. The autocorrelation matrix is ​​decomposed using LDL, and the final filter coefficients are calculated by back substitution. This process roughly follows the calculation of ALF filter coefficients in ECM; however, LDL decomposition is chosen instead of Cholesky decomposition to avoid the use of square root operations.

[0055] Gradient Linear Model (GLM)

[0056] Compared to CCLM, GLM does not use downsampled luminance values; instead, it uses luminance sample gradients to derive a linear model. Specifically, when applying GLM, the input to the CCLM process, i.e., the downsampled luminance sample L, is replaced by the luminance sample gradient G. Other parts of CCLM (e.g., parameter derivation, linear transformation of predicted samples) remain unchanged.

[0057] C = α·G + β

[0058] For signaling, when the current CU has CCLM mode enabled, two flags are issued for the Cb and Cr components respectively to indicate whether GLM is enabled for each component. If GLM is enabled for a component, a syntax element is further issued to select one of 16 gradient filters for gradient calculation. Figure 5 (See 510-540 in the original text). GLM can be used in conjunction with an existing CCLM by emitting an additional flag in the bitstream. When this combination is applied, the filter coefficients used to derive the linear model input luminance samples are calculated as a combination of the gradient filter selected by the GLM and the downsampling filter of the CCLM.

[0059] Space GPM

[0060] JVET-Z0124 (Fan Wang et al., “Non-EE2: Spatial GPM”, Joint Video Exploration Team (JVET) of ITU-T SG 16WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, 26th Meeting, via Teleconference, April 20-29, 2022, document JVET-Z0124) recommends extending GPM to intra-frame prediction. Similar to inter-frame GPM, SGPM includes one partitioned mode and two associated intra-frame prediction modes.

[0061] This candidate list was generated using a template. The template has the following shape: Figure 6 As shown. For each possible combination of a partitioned mode and two intra-frame prediction modes, a template prediction is generated, and the partition weights are extended to the template, as follows. Figure 6 As shown. These combinations are sorted in ascending order based on the SATD between prediction and template reconstruction. The candidate list is set to a length of 16, and these candidates are considered the most likely SGPM combinations for the current block. Both the encoder and decoder construct the same candidate list based on the template.

[0062] Matrix-weighted intra-frame prediction

[0063] Matrix-weighted intra-prediction (MIP) is a new intra-prediction technique added to VVC. To predict samples from a rectangular block of width W and height H, MIP uses H reconstructed neighbor boundary samples from the left side of the block and W reconstructed neighbor boundary samples from the top of the block as input. If reconstructed samples are unavailable, they are generated using the traditional intra-prediction method. The generation of the predicted signal is based on three steps: averaging, matrix-vector multiplication, and linear interpolation.

[0064] LFNST modification of MIP encoded blocks

[0065] In JVET-AB0067 (Junyan Huo et al., “EE2-4.1: Modification of LFNST for MIPcoded block”, ITU-T SG 16WP 3 and ISO / IEC JTC 1 / SC 29 Joint Video Experts Group (JVET), 28th meeting, Mainz, Germany, 21-28 October 2022, document: JVET-AB0067), it is proposed to use DIMD to derive the LFNST transform set and determine the LFNST transpose flag.

[0066] exist Figure 7 In this method, matrix-vector multiplication 710 is applied to the input data to generate MIP prediction output 720. MIP prediction with upsampling 730 is then applied to the MIP prediction output 720 to generate upsampled MIP prediction 740. The proposed method uses DIMD 750 to derive the intra-prediction mode of the current block based on the MIP prediction samples prior to upsampling. Specifically, the horizontal and vertical gradients are computed for each prediction sample to construct HoG 760, as shown below. Figure 7 As shown. Then, the intra-prediction mode with the largest histogram magnitude is used to determine the LFNST transform set and the LFNST transpose flag.

[0067] In addition, LFNST is enabled for MIP encoded blocks with a width and height greater than or equal to 4.

[0068] Intra-block copy

[0069] Intra-block copy (IBC) is a tool used in HEVC Extended Screen Content Coding (SCC). It is well-known for significantly improving the encoding and decoding efficiency of screen content material. Since IBC mode is implemented as a block-level encoding / decoding mode, block matching (BM) is performed in the encoder to find the optimal block vector (or motion vector) for each CU. Here, the block vector is used to indicate the displacement from the current block to a reference block that has already been reconstructed in the current image. The luma block vector of the IBC-encoded CU has integer precision. The chroma block vector is also rounded to integer precision. When used in conjunction with Adaptive Motion Vector Resolution (AMVR), IBC mode can switch between 1-pixel and 4-pixel motion vector precision. IBC-encoded CUs are considered a third prediction mode in addition to intra-frame or inter-frame prediction modes. IBC mode is suitable for CUs with a width and height of 64 luma samples or less.

[0070] Direct Block Vector (DBV) Mode for Chromatography Prediction

[0071] Direct Block Vectors (DBVs) are used for chroma blocks. A flag indicates whether the chroma block is encoded using IBC mode. If a luma block at a predefined location is encoded using IBC or intra-frame TMP mode, its block vector is scaled and used as the block vector for the chroma block. Block vector scaling is performed through template matching.

[0072] Inter-frame prediction overview

[0073] According to Section 3.4 of JVET-T2002 (Jianle Chen et al., “Algorithm description for Versatile Video Coding and Decoding and Test Model 11 (VTM 11)”, ITU-T SG 16 WP 3 and ISO / IEC JTC1 / SC 29 Joint Video Experts Group (JVET), 20th Meeting, held remotely from October 7 to 16, 2020, document: JVET-T2002), for each inter-frame predicted CU, motion parameters include motion vectors, reference picture indexes, and reference picture list usage indexes, as well as additional information required for generating inter-frame prediction samples for new VVC coding and decoding features. Motion parameters can be passed explicitly or implicitly. When a CU is coded in skip mode, the CU is associated with a PU and has no significant residual coefficients, coded motion vector increments, or reference picture indexes. A merging mode is specified in the VVC, where the motion parameters of the current CU are obtained from neighboring CUs, including spatial and temporal candidates, as well as additional plans introduced in the VVC. Merging mode can be applied to any inter-frame predicted CU, not just skip mode. An alternative to merging mode is to explicitly pass motion parameters, where the motion vector for each CU, the reference image index of the corresponding reference image list, the reference image list usage flag, and other necessary information are explicitly passed.

[0074] In addition to the inter-frame encoding and decoding features in HEVC, VVC also includes several new and improved inter-frame prediction encoding and decoding tools, as listed below:

[0075] - Extended merge forecast

[0076] - Merge pattern with MVD (MMVD)

[0077] -Symmetric MVD (SMVD) signaling

[0078] -Affine Motion Compensation Prediction

[0079] - Sub-block-based temporal motion vector prediction (SbTMVP)

[0080] - Adaptive Motion Vector Resolution (AMVR)

[0081] -Sports Field Storage: 1 / 16 th Luminance sample motion vector storage and 8x8 motion field compression

[0082] - Dual prediction (BCW) with CU-level weighting

[0083] - Bidirectional optical flow (BDOF)

[0084] -Decoder-side motion vector optimization (DMVR)

[0085] - Geometric Partitioning (GPM)

[0086] - Combination of inter-frame and intra-frame prediction (CIIP)

[0087] The following description provides detailed information about these inter-frame prediction methods specified in VVC.

[0088] Extended merge forecast

[0089] In VVC, the merge candidate list is constructed by sequentially including the following five types of candidates:

[0090] -Space MVP from adjacent CU

[0091] -Time MVP from Co-located CU

[0092] - Historical MVP from FIFO table

[0093] - Paired average MVP

[0094] -Zero motion vector

[0095] The size of the merge list is indicated in the Sequence Parameter Set (SPS) header, and the maximum allowed size of the merge list is 6. For each code unit (CU) encoded in merge mode, the index of the best merge candidate is encoded using truncated unary binarization (TU). The first bit of the merge index is context-coded, and the remaining bits are bypassed.

[0096] Dual prediction with CU-level weighting (BCW)

[0097] In High-Efficiency Video Coding (HEVC), the dual prediction signal P bi-pred The dual prediction mode is generated by averaging two prediction signals, P0 and P1, obtained from two different reference images. In video coding (VVC), the dual prediction mode extends beyond simple averaging, allowing a weighted average of the two prediction signals.

[0098] Pbi-pred =((8-w)*P0+w*P1+4)>>3 (4)

[0099] In weighted average biprediction, five weights w∈{-2,3,4,5,10} are allowed. For each bipredicted CU, the weight w is determined by one of two methods: 1) for non-merged CUs, the weighted index is labeled after the motion vector difference; 2) for merged CUs, the weighted index is inferred from neighboring blocks based on merge candidate indices. BCW is applied only to CUs with 256 or more luminance samples (i.e., CU width multiplied by CU height is greater than or equal to 256). For low-latency images, all 5 weights are used. For non-low-latency images, only 3 weights (w∈{3,4,5}) are used. In the encoder, fast search algorithms are applied to find the weighted indices without significantly increasing the complexity of the encoder. These algorithms are summarized below. Detailed information is disclosed in the VTM software and document JVET-L0646 (Yu-Chi Su et al., “CE4-related: Generalized bi-prediction improvements combined from JVET-L0197 and JVET-L0296”, Joint Video Expert Group (JVET) of ITU-T SG 16WP 3 and ISO / IEC JTC 1 / SC 29, 12th Meeting: Macau, China, CN, October 3-12, 2018, document: JVET-L0646).

[0100] The BCW weighted index is encoded using a context-encoded bit followed by a bypass-encoded bit. The first context-encoded bit indicates whether equal weighting is used; if unequal weighting is used, the bypass codec marks additional bits to indicate which unequal weighting is used.

[0101] In VVC, CIIP and BCW cannot be used together on a single CU. When a CU is encoded in CIIP mode, the BCW index of the current CU is set to 2 (i.e., w=4 indicates equal weighting). Equal weighting implies the default value of the BCW index.

[0102] Geometric Partitioning (GPM)

[0103] In VVC, geometric partitioning modes are supported for inter-frame prediction. Geometric partitioning modes are marked as a merging mode using a CU-level flag. Other merging modes include regular merging mode, MMVD mode, CIIP mode, and sub-block merging mode. A total of 64 partitions are supported for each possible CU size w×h=2. m ×2 n, where m,n∈{3…6} excludes 8x64 and 64x8.

[0104] When using this mode, the CU is divided into two parts by a geometrically positioned straight line (e.g., Figure 8 (As shown). The position of the segmentation line is mathematically derived from the angle and offset parameters of the specific partition. Each part of the geometric segmentation in the CU uses its own motion for inter-frame prediction; each partition is allowed only a single prediction, i.e., each part has one motion vector and one reference index. Single prediction motion constraints are applied to ensure that, as with traditional dual prediction, only two motion-compensated predictions are needed per CU. The single prediction motion is derived for each partition.

[0105] If the current CU uses a geometric segmentation pattern, a geometric segmentation index is further labeled, indicating the partitioning pattern of the geometric segmentation (angle and offset), and two merge indices (one for each partition). The number of maximum GPM candidate sizes is explicitly labeled in the SPS, and the syntax binarization of the GPM merge indices is specified. After predicting each part of the geometric segmentation, the sample values ​​along the geometric segmentation edges are adjusted using a mixing process with adaptive weighting. This is the predicted signal for the entire CU, and the transformation and quantization processes are applied to the entire CU as with other prediction patterns. Finally, the motion field of the CU predicted using the geometric segmentation pattern is stored.

[0106] Single prediction candidate list construction

[0107] The single prediction candidate list is directly derived from the merge candidate list constructed according to the extended merge prediction process. Let n be the index of the single prediction motion in the geometric single prediction candidate list. The LX motion vector (X = 0 or 1, i.e., LX = L0 or L1) of the nth extended merge candidate, where X equals the parity of n, is used as the nth single prediction motion vector for the geometric segmentation pattern. If the corresponding LX motion vector of the nth extended merge candidate does not exist, the L(1-X) motion vector of the same candidate is used as the single prediction motion vector for the geometric segmentation pattern.

[0108] Blending along geometrically segmented edges

[0109] After using its own motion prediction for each part of the geometric segmentation, a blend is applied to the two prediction signals to derive samples around the geometric segmentation edges. The blend weights at each location of the CU are derived based on the distance between each location and the segmentation edge.

[0110] Combined Inter-Frame and Intra-Frame Prediction (CIIP)

[0111] In VVC, when a CU is encoded in merged mode, if the CU contains at least 64 luma samples (i.e., the CU's width multiplied by its height is equal to or greater than 64), and if both the CU's width and height are less than 128 luma samples, an additional flag is sent to indicate whether Combined Inter-Frame / Intra-Frame Prediction (CIIP) mode is applied to the current CU. As the name suggests, CIIP prediction combines inter-frame prediction signals with intra-frame prediction signals. The inter-frame prediction signal P in CIIP mode... inter It is derived using the same inter-frame prediction process applied to the regular merging mode; the intra-frame prediction signal P intra It is derived according to the conventional intra-frame prediction process and planar mode. Then, the intra-frame and inter-frame prediction signals are combined using a weighted average, where the weight wt is based on the encoding / decoding mode of the top and left neighboring blocks of the current CU 910 (e.g., ...). Figure 9 The calculation is as follows (as shown):

[0112] - If the top neighbor is available and intra-frame encoding is used, set isIntraTop to 1; otherwise, set isIntraTop to 0.

[0113] - If the left neighbor is available and intra-frame coding is used, set isIntraLeft to 1; otherwise, set isIntraLeft to 0.

[0114] - If (isIntraLeft+isIntraTop) equals 2, then set wt to 3;

[0115] Otherwise, if (isIntraLeft+isIntraTop) equals 1, then set wt to 2;

[0116] Otherwise, set wt to 1.

[0117] The CIIP predictions are as follows:

[0118] P CIIP =((4-wt)*P inter +wt*P intra +2)>>2 (5)

[0119] Multiple Hypothesis Prediction (MHP) (More details can be found in JVET-W2025.)

[0120] In the multi-hypothesis inter-frame prediction mode (JVET-M0425), in addition to the traditional dual prediction signal, one or more additional motion-compensated prediction signals are transmitted. The final overall prediction signal is obtained by weighted superposition of samples. The dual prediction signal p... bi With the first additional inter-frame prediction signal / hypothesis h3, the final prediction signal p3 is obtained as follows:

[0121] p3=(1-α)p bi +αh3

[0122] The weighting factor α is specified by the new syntax element add_hyp_weight_idx, according to the mapping in Table 1.

[0123] Table 1. Mapping between weighting factor α and add_hyp_weight_idx.

[0124] add_hyp_weight_idx α 0 1 / 4 1 -1 / 8

[0125] Similar to the above, multiple additional prediction signals can be used. The final overall prediction signal is accumulated iteratively through each additional prediction signal.

[0126] p n+1 =(1-α) n+1 )p n +α n+1 h n+1

[0127] The final overall prediction signal is the last p n The obtained (i.e., p with the largest index n) n For example, up to two additional prediction signals can be used (i.e., n is limited to 2).

[0128] The motion parameters for each additional prediction hypothesis can be sent explicitly by specifying the reference index, motion vector prediction index, and motion vector difference, or implicitly by specifying the merge index. A separate multi-hypothesis merge flag distinguishes between these two sending modes.

[0129] This invention discloses a method for improving the performance of combined prediction. [Summary of the Invention]

[0130] A video encoding / decoding method and apparatus using combined prediction are disclosed. According to the method, input data associated with a current block is received, wherein the input data includes pixel data to be encoded at the encoder end or encoded data associated with the current block to be decoded at the decoder end. A combined prediction is derived, wherein the combined prediction includes a weighted combination of one or more source items, and wherein the weighting of the one or more source items is derived using regression techniques based on relation estimates evaluated on one or more templates, and the one or more source items correspond to different prediction modes of the current block, and the different prediction modes include at least one non-intra-frame mode, or the one or more source items correspond to member samples determined in modes associated with the samples to be predicted of the current block. The current block is then encoded or decoded using information including the combined prediction.

[0131] In one embodiment, each member sample corresponds to an initial prediction sample of the combined prediction pattern. In one embodiment, the one or more templates include the upper reference region of the current block, the left reference region of the current block, the upper left reference region of the current block, or a combination thereof.

[0132] In one embodiment, the combined prediction corresponds to a target prediction generated based on Inter and IntraPrediction (CIIP). In one embodiment, each of the one or more source items corresponds to one or more prediction samples associated with an inter-mode index from an inter-candidate list or an intra-mode index from an intra-candidate list. In another embodiment, the one or more source items correspond to one or more combined prediction samples derived from the CIIP.

[0133] In one embodiment, the combined prediction corresponds to a target prediction generated based on Multiple-Hypothesis Prediction (MHP). In one embodiment, each of the one or more source items corresponds to one or more prediction samples associated with an inter-frame mode index from an inter-frame candidate list. In another embodiment, the one or more source items correspond to one or more combined prediction samples derived from the MHP.

[0134] In one embodiment, the combined prediction corresponds to a target prediction generated based on bi-prediction with CU-level weight (BCW). In one embodiment, each of the one or more source items corresponds to one or more prediction samples associated with an inter-frame mode index from an inter-frame candidate list. In another embodiment, the one or more source items correspond to one or more combined prediction samples derived from the BCW.

[0135] In one embodiment, the regression technique corresponds to the Linear Minimum Mean Square Error (LMMSE) technique. In another embodiment, the regression technique corresponds to the LDL decomposition technique. In one embodiment, the regression technique is unified with a technique used for cross-component chromaticity modes. In another embodiment, the regression technique corresponds to Gaussian elimination. In one embodiment, the relation estimation corresponds to the distortion between the predicted samples and the reconstructed samples on the one or more templates. [Attached Image Description]

[0136] Figure 1AAn exemplary adaptive inter-frame / intra-frame video coding system is shown, which includes loop processing.

[0137] Figure 1B It shows Figure 1A The corresponding decoder for the encoder.

[0138] Figure 2A An example of a selected template for the current block is shown, where the template includes T rows above the current block and T columns to the left of the current block.

[0139] Figure 2B An example with T=3 is shown, and gradient histograms (HoG) are calculated for the pixels in the middle row and the middle column.

[0140] Figure 2C An example of the amplitude (ampl) of the angular intra-frame prediction mode is shown.

[0141] Figure 3 An example of the blending process is shown, in which two intra-frame modes (M1 and M2) are selected based on the two highest bars in the histogram.

[0142] Figure 4 An example of a template-based intra-mode derivation (TIMD) mode is shown, where TIMD implicitly derives the intra-prediction mode of the CU at the encoder and decoder using a neighboring template.

[0143] Figure 5 Sixteen gradient modes of the gradient linear model (GLM) are shown.

[0144] Figure 6 An example of a spatial GPM (SGPM) template is shown.

[0145] Figure 7 An example of LFNST modification of a MIP coded block is shown, which uses DIMD to derive the LFNST transform set and determine the LFNST transpose flag.

[0146] Figure 8 An example of GPM segmentation grouped by the same angle is shown.

[0147] Figure 9 An example of deriving the weighted values ​​of combined inter-frame and intra-frame prediction (CIIP) based on the encoding and decoding modes of the top and left neighboring blocks is shown.

[0148] Figure 10A -C shows an example of a pattern for selecting a source item at the center. Figure 10A ), where the crossover pattern is displayed for the luminance component ( Figure 10B ) and diamond pattern ( Figure 10C ).

[0149] Figure 11A -C shows an example of a pattern for selecting a source item at the center. Figure 11A ), where the cross mode is displayed for the chromaticity components ( Figure 11B ) and diamond pattern ( Figure 11C ).

[0150] Figure 12 An example of a current block reference region is shown for deriving combined, inter-frame, or intra-frame predictions using template-based regression techniques.

[0151] Figure 13 A flowchart of an exemplary video codec system based on template regression technology to derive combined predictions according to an embodiment of the present invention is shown.

Detailed Implementation Methods

[0152] It will be readily understood that the components of the present invention, as described and illustrated in the figures, can be arranged and designed in a variety of different configurations. Therefore, the following more detailed description of embodiments of the systems and methods of the present invention, as shown in the figures, is not intended to limit the scope of the claimed invention, but is merely representative of selected embodiments of the invention. Throughout this specification, references to “an embodiment,” “one embodiment,” or similar language mean that a particular feature, structure, or characteristic associated with that embodiment may be included in at least one embodiment of the invention. Therefore, the phrases “in one embodiment” or “in one embodiment” appearing in various places in this specification do not necessarily refer to the same embodiment.

[0153] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. However, those skilled in the art will recognize that the invention can be practiced without one or more specific details, or using other methods, components, etc. In other instances, well-known structures or operations have not been shown or described in detail to avoid obscuring various aspects of the invention. Embodiments of the invention will be best understood by referring to the accompanying drawings, in which like parts are labeled with the same numerals throughout. The following description is by way of example only and merely illustrates embodiments of certain selected devices and methods consistent with the invention claimed herein.

[0154] This invention discloses a template regression-based method for deriving combined predictions for the current block.

[0155] In this invention, an innovative mechanism for improving prediction accuracy is proposed. The prediction of the current block is formed by combining one or more proposed source terms and a weighted setting of a proposal. As shown in Equation (6), pred(i,j) is a target (predicted) sample in the current block, which can be obtained after the proposal mechanism. sourceTermSet0 includes one or more source terms from the luma component, sourceTermSet1 includes one or more source terms from the chroma component, and biasTermSet includes one or more bias terms. One or more target samples can be regarded as information, including combined predictions and / or one or more target samples used for encoding or decoding the current block.

[0156] pred(i,j) = (sourceTermSet0(i,j) + sourceTermSet1(i,j) + ... + biasTermSet) has the proposed weighted setting (6).

[0157] Where (i,j) is a sample position in the current block.

[0158] Equation (6) is merely an example; the proposed mechanism can use any subset or extension of sourceTermSet0, sourceTermSet1, and biasTermSet. Each sample or any subset of samples in the current block obtains its target (predicted) sample according to Equation (6). In the following sections, the contents of sourceTermSet0 are described in Section I, the contents of sourceTermSet1 in Section II, the contents of biasTermSet in Section III, and the prediction derivation using the proposed source terms and the proposed weighting settings is described in Section IV. Several encoding / decoding tools, including combinations of MHP and / or BCW and / or CIIP and / or chroma cross-component prediction modes (e.g., TIMD-CCM), are presented in Section IV along with our proposed mechanism.

[0159] The content of I.sourceTermSet0(i,j)

[0160] SourceTermSet0(i,j) includes one or more luminance source terms, denoted as sourceTerm00, sourceTerm01, ... and / or sourceTerm0 n-1The value of n represents the number of taps in the source itemset. In one embodiment, the source items can be linear items and / or nonlinear items, linear items only, and / or nonlinear items only. In another embodiment, n is a predefined value, such as 1, 2, ... or any positive integer. For example, the predefined value is fixed in the standard. Another example is that the predefined value is less than or equal to the maximum threshold indicated by the syntax in the bitstream, where the syntax is at the block, CTU, CTB, slice, tile, image, SPS, PPS, image and / or sequence level. In another embodiment, n is determined based on the codec information and / or sample position (i,j) of the current block. For example, when the current block is encoded using a specific codec tool, n is (1) fixed at a predefined value, (2) determined based on the block width, block height, block area, codec information and / or sample information of the current block, (3) determined based on the codec information and / or sample information of the adjacent / non-adjacent spatial neighbor reference regions of the current block, and / or (4) determined based on the codec information and / or sample information of the temporal reference region of the current block. In another embodiment, the pattern of n taps refers to the pattern defined as surrounding / containing the location (i L j L The pattern of any subset of the window region M x N. That is, (i L j L () is used to export windows and / or modes, which means one or more locations to be used. For some examples, (i L j L (i) refers to the center of the window and / or mode. However, (i) L j L ) is not limited to referring to the center of the window and / or pattern. If the target sample is brightness, (i L j L (i,j) is the target sample. If the target sample is chroma (cb or cr), (i) is the target sample. L j L ) is the corresponding brightness position from (i,j).

[0161] For an example, (i L ,j L ) refers to the center of the window, and only the center of the window (i) is used. L ,j L ),like Figure 10A As shown, the center is represented by a gray square C. For another example, (i L ,j L () refers to the center of the window; the pattern is a 5x5 cross, which may or may not include the area located at (i). L ,j L The center of ) such as Figure 10BAs shown, the sample used as the source item is displayed as a dot-filled square. For another example, (i L ,j L (i) refers to the center of the window; the pattern is a 5x5 rhombus, which may or may not include the center. L ,j L ),like Figure 10C As shown, the sample used as the source item is displayed as a dot-filled square.

[0162] In another embodiment, different taps refer to source items from different prediction modes or different mode types. In one sub-implementation, one or more taps come from mode type intra-frame, another or more taps come from mode type inter-frame, and / or another or more taps come from mode type IBC. In another sub-implementation, one or more taps come from MIP intra-frame prediction modes, and another or more taps come from non-MIP intra-frame prediction modes.

[0163] For a source item in the source item set, the following example is used to determine the generation of source content.

[0164] In one embodiment, the source content is based on a prediction sample generated by a prediction model and / or a reconstruction sample generated by a prediction sample generated by a prediction model and a reconstruction residual.

[0165] In one sub-implementation, the prediction mode belongs to the mode type intra, mode type inter, or a third mode type (e.g., mode type IBC). For example, a prediction mode belonging to mode type intra refers to planar, DC, horizontal, vertical, other angular (directional) prediction modes, any intra prediction mode specified in the 67 / 131 intra prediction mode domain, wide-angle intra prediction (WAIP) mode, TIMD derived mode, DIMD derived mode, intra TMP, and / or any intra prediction mode specified in the standard. Another example of a prediction mode belonging to mode type inter refers to skip mode, regular merge mode, MMVD mode, affine mode, SbTMVP, AMVR, any merge mode specified in the standard, any AMVP (Advanced MVP, where AMVP can be called non-merged inter) mode specified in the standard, or any inter mode specified in the standard. Another example of a prediction mode belonging to mode type IBC refers to IBC merge, IBC AMVP, or any IBC mode specified in the standard. Note that any possible combination between prediction modes and mode types is supported in this invention. In other words, according to the standard definition, any mentioned prediction mode can belong to any mode type. For example, according to the standard definition, if the IBC mode belongs to the mode type inter-frame, then in the embodiment, the prediction mode belonging to the mode type inter-frame can refer to the IBC mode.

[0166] In another sub-implementation, the source content is a filtered source or a source with any preprocessing. For example, the source content is a predicted / reconstructed sample filtered by a predefined model or filter.

[0167] In another sub-implementation, the source content is gradient information from the predicted and / or reconstructed samples. If the target sample (i,j) belongs to chromaticity, and any of the following Sobel filters (e.g.) are used... Figure 5 The gradient information of the target sample (i,j) is calculated using the gradient filter shown or any predefined filter. Each value around the center circle is multiplied with the corresponding predicted / reconstructed sample in the corresponding brightness block, and then summed to form the gradient information of the source term of the target sample (i,j).

[0168] In another sub-implementation, if the target sample belongs to the luma sample, the predicted and / or reconstructed samples are located within the current block; otherwise (if the target sample belongs to the chroma sample (cb or cr)), the predicted and / or reconstructed samples are located within the corresponding (luma) block from the current (chroma) block. The predicted and / or reconstructed samples are treated as initial samples and used as source content to generate the target sample.

[0169] In another embodiment, the value of the source item is further adjusted (increased or decreased) by a predefined offset. If the target sample refers to luminance, several embodiments are used to generate the offset of the source item. In one sub-embodiment, the offset is determined as the average (or any subset) of predicted or reconstructed samples in the current block or a reference region of the current block. In another sub-embodiment, the offset is determined as a sample value of a predefined predicted or reconstructed sample in the current block or a reference region of the current block. For example, the sample value comes from the top-left position of the current block (just outside the top-left corner of the current block). If the target sample refers to chroma, several embodiments are used to generate the offset of the source item. In one sub-embodiment, the offset is determined as the average (or any subset) of predicted or reconstructed samples from the corresponding luminance block or a reference region of the corresponding luminance block. In another sub-embodiment, the offset is determined as a sample value of a predefined predicted or reconstructed sample in the corresponding luminance block or a reference region of the corresponding luminance block. For example, the sample value comes from the top-left position of the corresponding luminance block (just outside the top-left corner of the corresponding luminance block).

[0170] In another embodiment, the source item may further include location information. For example, if the target sample refers to luminance, then the horizontal position (i) of (i,j) is used for the source item, and the vertical position (j) of (i,j) is used for the source item; otherwise, the horizontal position of the co-located luminance block from sample (i,j) is used for the source item, and the vertical position of the co-located luminance block from sample (i,j) is used for the source item.

[0171] In another embodiment, the source term may further include location information. For example, if the target sample refers to brightness, then the horizontal position (i) of (i,j) is used for the source term, and the vertical position (j) of (i,j) is used for the source term; otherwise, the horizontal position of the co-located brightness from sample (i,j) is used for the source term, and the vertical position of the co-located brightness from sample (i,j) is used for the source term.

[0172] II. Contents of SourceTermSet1(i,j)

[0173] SourceTermSet1(i,j) includes one or more chrominance (cb or cr) source terms, denoted as sourceTerm10, sourceTerm11, ..., and / or sourceTerm1 m-1 The value of m represents the number of taps in the source itemset. In one embodiment, the source items can be linear items and / or nonlinear items, linear items only, and / or nonlinear items only. In another embodiment, m is a predefined value, such as 1, 2, ... or any positive integer. For example, the predefined value is fixed in the standard. Another example is that the predefined value is less than or equal to the maximum threshold indicated by the syntax in the bitstream, where the syntax is located at the block, CTU, CTB, slice, tile, image, SPS, PPS, image and / or sequence level. In another embodiment, m is determined by the codec information and / or sample position (i,j) of the current block. For example, when the current block is encoded by a specific codec tool, m is (1) fixed at the predefined value, (2) determined based on the block width, block height, block area, codec information and / or sample information of the current block, (3) determined based on the codec information and / or sample information of the adjacent / non-adjacent spatial neighbor reference region of the current block, and / or (4) determined based on the codec information and / or sample information of the temporal reference region of the current block. In another embodiment, the pattern of the m-tap refers to the location defined as the area surrounding / containing the window region M2 x N2 (i C ,j C The pattern of any subset of ). That is, (i C ,j C () is used to export windows and / or modes, which means one or more locations to be used. For some examples, (i C ,j C (i) refers to the center of the window and / or mode. However, (i) C ,j C ) is not limited to the center of the window and / or pattern. If the target sample is chroma (cb or cr), (i C ,j C (i,j) is the target sample. If the target sample is brightness, (i) is the target sample. C ,j C) is the colocalized chromaticity position from (i,j).

[0174] For example, (i C ,j C ) refers to the center of the window, and only the center of the window (i) is used. C ,j C ),like Figure 11A As shown, the center is indicated by a gray square C. Another example is, (i C ,j C () refers to the center of the window, which is a 5x5 cross shape, and may or may not include the area located at (i C ,j C The center of ) such as Figure 11B As shown, the sample used as the source item is displayed as a dot-filled square. Another example is, (i C ,j C () refers to the center of the window, which is a 5x5 rhombus shape, and may or may not include the area located at (i). C ,j C The center of ) such as Figure 11C As shown, the sample used as the source item is displayed as a dot-filled square.

[0175] In another embodiment, different taps refer to source items from different prediction modes or different mode types. In one sub-implementation, one or more taps are from mode type intra-frame, another one or more taps are from mode type inter-frame, and / or another one or more taps are from mode type IBC. In another sub-implementation, one or more taps are from MIP intra-frame prediction modes, and another one or more taps are from non-MIP intra-frame prediction modes.

[0176] For source items in the source item set, the following examples are used to determine the generation of source content.

[0177] In one embodiment, the source content is based on a prediction sample generated by a prediction model and / or a reconstruction sample generated from a prediction sample obtained by a prediction model and a reconstruction residual.

[0178] In one sub-implementation, the prediction mode belongs to mode type intra, mode type inter, or a third mode type (e.g., mode type IBC). For an example where the prediction mode belongs to mode type intra, the prediction mode refers to planar, DC, horizontal, vertical, other angular (directional) prediction modes, any intra prediction mode specified in the 67 / 131 intra prediction mode domain, wide-angle intra prediction (WAIP) mode, TIMD-derived mode, DIMD-derived mode, intra TMP, DBV, any cross-component mode (CCLM (including CCLM_LT, CCLM_L and / or CCLM_T), MMLM (including MMLM_LT, MMLM_L and / or MMLM_T), CCCM (including CCCM_LT, CCCM_L and / or CCCM_T), GLM and / or any variant / extension of the above modes), and / or any intra prediction mode specified in the standard. For another example where the prediction mode belongs to the mode type inter-frame, the prediction mode refers to skip mode, regular merge mode, MMVD mode, affine mode, SbTMVP, AMVR, any merge mode specified in the standard, any AMVP mode specified in the standard, or any inter-frame mode specified in the standard. For another example where the prediction mode belongs to the mode type IBC, the prediction mode refers to IBC merge, IBC AMVP (Advanced MVP, where AMVP can be referred to as non-merged inter-frame), or any IBC mode specified in the standard. Note that any possible combination between prediction mode and mode type is supported in this invention. That is, according to the standard definition, any mentioned prediction mode can belong to any mode type. For example, according to the standard definition, if the IBC mode belongs to the mode type inter-frame, then in this embodiment, "predicting mode belonging to the mode type inter-frame" can refer to the IBC mode. In one embodiment, DBV can be considered as using IBC to generate chroma prediction samples.

[0179] In another sub-implementation, the source content is a filtered source or a source that has undergone any preprocessing. For example, the source content is a predicted / reconstructed sample filtered using a predefined model or filter.

[0180] In another sub-implementation, the source content is gradient information from the predicted and / or reconstructed samples. If the target sample (i,j) belongs to luminance, the gradient information of its corresponding chrominance sample is calculated using any Sobel filter, any gradient filter, or any predefined filter.

[0181] In another sub-implementation, if the target sample is a chroma sample, the predicted and / or reconstructed samples are located within the current block; otherwise (if the target sample is a luma sample), the predicted and / or reconstructed samples are located within the corresponding chroma block from the current (luma) block. The predicted and / or reconstructed samples are treated as initial samples and used as source content to generate the target sample.

[0182] In another embodiment, the value of the source item is further adjusted (increased or decreased) by a predefined offset. If the target sample refers to chroma, the offset of the source item is generated using several embodiments. In one sub-embodiment, the offset is determined as the average (or any subset) of predicted or reconstructed samples in the current block or current block reference region. In another sub-embodiment, the offset is determined as a sample value of a predefined predicted or reconstructed sample in the current block or current block reference region. For example, the sample value comes from the top-left position (just outside the top-left corner of the current block). If the target sample refers to luma, the offset of the source item is generated using several embodiments. In one sub-embodiment, the offset is determined as the average (or any subset) of predicted or reconstructed samples from the chroma block or corresponding chroma block reference region corresponding to the current (luma) block. In another sub-embodiment, the offset is determined as a sample value of a predefined predicted or reconstructed sample from the chroma block or corresponding chroma block reference region corresponding to the current (luma) block. For example, the sample value comes from the top-left position (just outside the top-left corner of the corresponding chroma block).

[0183] In another embodiment, the source term may further include positional information. For example, if the target sample refers to chroma, then the horizontal position (i) of (i,j) is used as the source term, and the vertical position (j) of (i,j) is used as the source term; otherwise, the horizontal position of the chroma block corresponding to sample (i,j) is used as the source term, and the vertical position of the chroma block corresponding to sample (i,j) is used as the source term.

[0184] III. Contents of the Bias Term Set

[0185] The bias term can be any predefined value. In one embodiment, the bias term is the midValue based on the bit depth specified in the standard. For example, the bias term is set to (1 << (bitDepth-1)). In another embodiment, the bias term is the same for every sample in the current block. That is, the bias term is independent of position (i,j).

[0186] IV. Prediction Derivation of Sample (i,j)

[0187] IV.1. Proposed Weighting Settings

[0188] The proposed weighting setup involves estimating the relationship between the combined results of those source terms and the reconstructed samples on the reference region of the current block using a predefined regression method (e.g., minimizing distortion) to generate weights (referring to model parameters) based on the regression method. These weights are then applied to the source terms to obtain the target (predicted) samples in the current block. In one embodiment, the predefined regression method could be a Linear Least Mean Squared Error (LMMSE) method as a cross-component pattern, such as CCLM, or any method consistent with the regression method used for cross-component patterns, such as CCLM. In another embodiment, the predefined regression method could be an LDL decomposition method as CCCM, or any method consistent with the regression method used for CCCM. In yet another embodiment, the predefined regression method could be Gaussian elimination.

[0189] In one embodiment, the reference region of the current block is as follows: Figure 12 The spatially adjacent or non-adjacent neighborhood regions of the current block 1210 are shown. The spatial neighborhood regions of the current block (serving as the template for the current block) include the upper reference region 1220, the left reference region 1230, the upper-left reference region 1240, and / or any subset thereof. The size of the upper reference region is A. w xA H The size of the reference region on the left is L. w xL H The size of the upper left reference region is AL. W xAL H ,in

[0190] -A w = Current block width (W), k*W, W+Current block height (H), any predefined value, or any adaptive value, depending on the current block's block position, block width, block height, and / or block area.

[0191] -A H or AL H =H, any predefined value (1,2,4,…), or any adaptive value, depending on the current block's position, width, height, and / or area.

[0192] -L w or AL W =W, any predefined value (1,2,4,…), or any adaptive value, depending on the current block's position, width, height, and / or area.

[0193] -L H =H,k*H,H+W, any predefined value, or any adaptive value, depending on the current block's position, width, height, and / or area.

[0194] IV.2. Different Example Expressions

[0195] IV.2.1:pred C (i,j)=α0·G(i,j)+α1·rec′ L (i,j)+α2·bias

[0196] In the expression where the target sample is chroma, sourceTermSet0 includes two taps, G(i,j) and rec'L(i,j), sourceTermSet1 is unused, and biasTerm refers to another tap, midValue. G(i,j) is the gradient information generated from the selected gradient filter, and rec'L(i,j) is the gradient value. L (i,j) are downsampled reconstructed brightness samples. The weighted model parameters (a0,a1, anda2) are derived from the following:

[0197] - Use six adjacent rows and columns as the reference region for the current block.

[0198] - Use LDL decomposition as the regression method

[0199] IV.2.2.pred C (i,j)=α0·C+α1·Gy(i,j)+α2·Gx(i,j)+α3·Y(i,j)+α4·X(i,j)+α5·P(i,j)+α6·bias

[0200] In this expression where the target sample is chroma (similar to JVET-AC0054), sourceTermSet0 includes six taps: C (corresponding luminance reconstructed sample), Gy(i,j), Gx(i,j), Y, X, and P (e.g., as a non-linear term in CCCM), sourceTermSet1 is unused, and biasTerm refers to another tap, midValue.

[0201] -Gy(i,j) is the gradient information generated from the vertical gradient filter.

[0202] -Gx(i,j) is the gradient information generated from the horizontal gradient filter.

[0203] -Y and X are the vertical and horizontal positions of the same brightness sample.

[0204] - Use six adjacent rows and columns as the reference region for the current block.

[0205] - Use LDL decomposition as the regression method

[0206] IV.2.3.pred L(i,j)=α0·Pmode_0(i,j)+α1·Pmode_1(i,j)+…+α s-1 ·Pmode_s-1(i,j)+α s ·bias

[0207] In this representation, the target sample is the luminance used for the inter-frame coded block, sourceTermSet0 includes s taps, denoted as Pmode_0 to Pmode_s-1, sourceTermSet1 is unused, and biasTerm refers to another tap called midValue. Each or any subset of Pmode_0(i,j) to Pmode_s-1(i,j) is a predicted sample from the mode indicated by the inter-frame mode index. For example, Pmode_0(i,j) is a predicted sample from the first mode. For example, Pmode_s-1(i,j) is a predicted sample from the s-th mode (or the (s-1)-th mode if the first mode is the 0th mode). In one embodiment, an inter-frame candidate list is first constructed, and s inter-frame motion candidates from the inter-frame candidate list are used. For example, the inter-frame candidate list refers to the merge / AMVP candidate list, which may be the same as or different from the merge / AMVP candidate list of the regular merge / AMVP mode. For example, the merge candidate list of the regular merge mode is reused to derive the merge candidate list of the proposed mechanism. In another embodiment, only single-prediction, only double-prediction, or single-prediction and / or double-prediction inter-frame motion candidates are inserted into the inter-frame candidate list. In another embodiment, the maximum number of candidates in the inter-frame candidate list is specified as a fixed number in the standard or as syntax in block-level, CTU-level, SPS-level, PPS-level, slice-level, tile-level, image-level, and / or sequence-level signaling. In another embodiment, the inter-frame mode index is signaled / parsed to indicate each mode (mode 0 to s or s-1). In another embodiment, the inter-frame mode index is signaled / parsed to indicate one mode (e.g., mode_0), and the remaining modes are selected based on mode 0. In another embodiment, two candidate lists are constructed. One or more modes can be selected from one or both of the two candidate lists. One list is an inter-frame candidate list containing one or more inter-frame motion candidates, and the other is an intra-frame candidate list containing one or more intra-frame candidates. If one list contains only one candidate, the unique candidate is inferred to be used without signaling. Parameters a0 to a s-1This can be viewed as a weighted sum of predictions from each mode to be combined. In one embodiment, the proposed mechanism is considered as an optional mode (e.g., an optional mode of MHP). That is, a flag is signaled / resolved at the encoder / decoder to indicate whether the proposed mechanism is used for the current block encoded by MHP. In one sub-implementation, this flag is located at the block level, CTU level, slice level, SPS level, tile level, PPS level, and / or image level. In another embodiment, the flag is context-encoded. For example, only one context is used to signal the flag. For another example, the context selection of the flag depends on the encoding / decoding information of the current block, block width, block height, and / or block area, and / or the encoding / decoding information, block width, block height, and / or block area of ​​adjacent blocks. In another embodiment, the proposed mechanism is an alternative approach. When generating predictions for the current MHP-encoded block that support the proposed mechanism, the generation of inferred predictions follows the proposed mechanism. In another embodiment, s is a predefined value. For example, s = 1, 2, 3, or any positive integer. In one sub-implementation, s is fixed to a predefined value in the standard. In another sub-implementation, s is adaptive based on the codec information, block width, block height, and / or block area of ​​the current block and / or the codec information, block width, block height, and / or block area of ​​adjacent blocks. For example, s is a larger number if the block width, height, or area is greater than a predefined threshold; otherwise, s is a smaller number. In another embodiment, when model parameters are derived on the reference region of the current block using a regression method, the distortion to be minimized is a combination of results on the reference region of the current block, including (1) the predictions generated from mode 0 to s-1 and (2) the bias and (3) the weighting settings, with the distortion between the reconstructed samples on the reference region of the current block. In another embodiment, sourceTermSet1 can be used in the expression. That is, the target luminance sample can be generated using the corresponding chroma information. In another embodiment, the term "MHP" can be replaced with "BCW", "GPM", "CIIP", and / or any luminance / chroma codec tool, and when the current block uses a certain codec tool, multiple hypothesis predictions are used to form the final prediction for the current block to apply the proposed mechanism. When the proposed mechanism is applied to BCW, each prediction hypothesis refers to a single prediction from a different list (list 0 or list 1), and the expression generating the final prediction is shown below. The prediction hypotheses from list 0 and list 1 are indicated by a shared inter-frame mode index, such as a merged candidate index or an AMVP candidate index. L (i,j)=α0·Plist_0(i,j)+α1·Plist_1(i,j)+α2·bias

[0208] IV.2.4.pred L(i,j)=α0·Ppat_0(i,j)+α1·Ppat_1(i,j)+…+α s-1 ·Ppat_s-1(i,j)+α s ·bias

[0209] In this expression, the target sample is the luminance used for the inter-frame coded block, sourceTermSet0 includes s taps as Ppat_0 to Ppat_s-1, sourceTermSet1 is unused, and biasTerm refers to another tap as midValue.

[0210] Each or any subset of Ppat_0(i,j) to Ppat_s-1(i,j) is a combined prediction sample that follows the rules of the inter-frame codec tool. Each combined prediction sample can be considered as a member sample of the initial prediction sample corresponding to the combined prediction mode (e.g., MHP, BCW, and / or CIIP). For the example where the inter-frame codec tool is MHP, the combined prediction samples are formed by MHP weighted averaging, with each prediction generated from the MHP prediction mode to be combined. 0 to s-1 represent the s-tap mode of the source item (any predefined mode in Section I).

[0211] In one embodiment, the proposed mechanism is considered as an optional mode (e.g., an optional mode of MHP). That is, a flag is emitted / resolved at the encoder / decoder to indicate whether the proposed mechanism is used for the current block (encoded by MHP). In one sub-implementation, the flag is located at the block level, CTU level, slice level, SPS level, tile level, PPS level, and / or image level. In another embodiment, the flag is context-encoded. For example, only one context is used to emit the flag. For another example, the context selection of the flag depends on the encoding / decoding information of the current block, block width, block height, and / or block area, and / or the encoding / decoding information, block width, block height, and / or block area of ​​neighboring blocks.

[0212] In another embodiment, the proposed mechanism is an alternative approach. When generating predictions for the current MHP coding block that support the proposed mechanism, the generation of the predictions is inferred to follow the proposed mechanism.

[0213] In another embodiment, s is a predefined value. For example, s = 1, 2, 3, or any positive integer. In one sub-embodiment, s is fixed to a predefined value in the standard. In another sub-embodiment, s is adaptive based on the codec information of the current block, the block width, block height, and / or block area, and / or the codec information, block width, block height, and / or block area of ​​neighboring blocks. For example, if the block width, height, or area is greater than a predefined threshold, s is a larger number; otherwise, s is a smaller number.

[0214] In another embodiment, when model parameters are derived on the reference region of the current block using a regression method, the distortion to be minimized is the combined result on the reference region of the current block, including (1) the combined predictions generated from mode taps 0 to s-1 on the reference region and (2) the bias and (3) the weighting settings, the distortion between the reconstructed samples on the reference region of the current block.

[0215] In another embodiment, sourceTermSet1 can be used in an expression. That is, the target luminance sample can be generated using the corresponding chromaticity information.

[0216] In another embodiment, the term "MHP" can be replaced with "BCW," "GPM," "CIIP," and / or any luma codec tool to apply the proposed mechanism when a specific codec tool is used in the current block. When the specific codec tool is GPM, combined predictions are generated in the reference region of the current block or within the current block, with the GPM weighted average following the GPM dividing line. That is, following the GPM mixing rules, for samples close to the dividing line, equally weighted predictions from the GPM modes to be combined are used; for samples far from the dividing line, if the current sample is located in a prediction unit belonging to one of the GPM modes to be mixed (but not to another GPM mode to be mixed), then a larger weight is used for the predictions from one GPM mode to be mixed, and a smaller weight is used for the predictions from the other GPM mode to be mixed.

[0217] IV.2.5

[0218] pred C (i,j)=α0·P_CCM_0(i,j)+…+α s-1 ·P_CCM s-1 (i,j)+α s ·bias

[0219] In this expression, the target sample is the chroma of a block encoded by cross-component mode (CCM), sourceTermSet1 includes s taps as P_CCM_0 to P_CCM_s-1, sourceTermSet0 is unused, and biasTerm refers to another tap as midValue.

[0220] P_CCM_0(i,j) to P_CCM_s-1(i,j) for each or any subset are prediction samples generated from the modes selected for the codec mode from all or any subset of candidate cross-component prediction modes. For example, P_CCM_0(i,j) is the prediction sample generated from the first mode. For example, P_CCM_s-1(i,j) is the prediction sample generated from the s-th mode (or if the first mode is 0). thThe pattern is the predicted sample generated by the (s-1) pattern.

[0221] In one embodiment, s is set to at least 2. For an example where s is 2, when the codec mode is CCM, one prediction comes from CCM_0 and another from CCM_1, used to generate the final prediction. In one sub-implementation, CCM_0 is indicated by a mode index, and CCM_1 is derived through a TIMD derivation process. That is, CCM_0 is selected based on the chroma prediction mode index of the signaling, while CCM_1 is the mode with the minimum TIMD cost from a predefined set of candidate CCMs on the reference region (template) of the current block. For example, the predefined set of candidate CCMs includes MMLM_L, MMLM_T, and / or MMLM_LT. In another example, CCM_0 is one of CCLM_L, CCLM_T, and CCLM_LT.

[0222] In another embodiment, the proposed mechanism is considered as an optional mode (e.g., an optional mode for CCM encoded blocks). That is, a flag is signaled / resolved in the encoder / decoder to indicate whether the proposed mechanism is used for the current block. In one sub-implementation, the flag is located at the block level, CTU level, fragment level, SPS level, tile level, PPS level, and / or image level. In another embodiment, the flag is context-encoded. For example, only one context is used to signal the flag. In another example, the context selection of the flag depends on the encoding / decoding information, block width, block height, and / or block area of ​​the current block, and / or the encoding / decoding information, block width, block height, and / or block area of ​​adjacent blocks.

[0223] In another embodiment, the proposed mechanism is an alternative approach. When generating predictions for the current CCM coding block that support the proposed mechanism, the generation of the predictions is inferred to follow the proposed mechanism.

[0224] In another embodiment, s is a predefined value. For example, s = 1, 2, 3, or any positive integer. In one sub-embodiment, s is fixed to a predefined value in the standard. In another sub-embodiment, s is adaptive based on the codec information, block width, block height, and / or block area of ​​the current block, and / or the codec information, block width, block height, and / or block area of ​​adjacent blocks. For example, if the block width, height, or area is greater than a predefined threshold, then s is a larger number; otherwise, s is a smaller number.

[0225] In another embodiment, when model parameters are derived on the reference region of the current block using a regression method, the distortion to be minimized is the distortion between the combined result (including (1) the prediction generated from mode 0 to s-1 on the reference region of the current block, (2) the bias and (3) the weighting settings) and the reconstructed sample on the reference region of the current block.

[0226] In another embodiment, sourceTermSet0 can be used in the expression. That is, the target chromaticity sample can be generated using the corresponding luminance information. For example, rec′ L (i,j), G(i,j) and / or Gy(i,j), Gx(i,j) are added as source items in sourceTermSet0.

[0227] IV.2.6pred C (i,j)=α0·Ppat_0(i,j)+α1·Ppat_1(i,j)+…+α s-1 ·Ppat_s-1(i,j)+α s ·bias

[0228] In this expression, the target sample is the chroma of the CCM coded block, sourceTermSet1 includes s taps as Ppat_0 to Ppat_s-1, sourceTermSet0 is not used, and biasTerm refers to another tap as midValue.

[0229] Ppat_0(i,j) to Ppat_s-1(i,j) of each or any subset are combined prediction samples that follow the rules of the CCM codec tool for the current block. For example, when the CCM codec tool is TIMD-CCM, the combined prediction samples are formed by each prediction generated from the TIMD-CCM prediction pattern to be combined using a TIMD-CCM weighted average. 0 to s-1 represent the s-tap pattern of the source item (any predefined pattern in Part I or II). Below is an example of a TIMD-CCM weighted average.

[0230] In one scenario, both cross-component modes (CCMs) to be blended are selected by one or more signaling mode indices. In another scenario, only one of the two CCMs to be blended is selected by a signaling mode index, while the other is determined through a TIMD derivation procedure. The TIMD derivation procedure means that for all candidate CCMs of the other of the two CCMs to be blended, the TIMD cost (or template cost) is calculated by comparing the distortion between the reconstructed samples of the template and the predicted samples of the template, and the candidate CCM with the minimum TIMD cost is determined as the other of the two CCMs to be blended, where the predicted samples of the template are derived from the model of the current candidate CCM, with inputs being (1) the template reference region of the current chroma block and (2) the template reference region of the corresponding luma block. The derived model of the current candidate CCM is applied to the reconstructed samples in the template of the corresponding luma block to obtain the template predicted samples of the current (chroma) block. In another scenario, both CCMs to be blended are determined through a TIMD derivation procedure. After determining the two CCMs to be blended, the weighting of the two CCMs depends on the TIMD costs of the two CCMs to be blended. Patterns with lower TIMD costs receive higher weighting when performing TIMD-CCM (or template-based CCM) weighted averages.

[0231] In one embodiment, the proposed mechanism is considered as an optional mode (e.g., an optional mode of TIMD-CCM). That is, a flag is signaled / resolved at the encoder / decoder to indicate whether the proposed mechanism is used for the current block (encoded by TIMD-CCM). In one sub-implementation, the flag is located at the block level, CTU level, slice level, SPS level, tile level, PPS level, and / or image level. In another embodiment, the flag is context-encoded. For example, only one context is used to signal the flag. For yet another example, the context selection of the flag depends on the encoding / decoding information of the current block, block width, block height, and / or block area, and / or the encoding / decoding information, block width, block height, and / or block area of ​​neighboring blocks.

[0232] In another embodiment, the proposed mechanism is an alternative approach. When generating predictions for the current TIMD-CCM coding block that support the proposed mechanism, the generation of the predictions is inferred to follow the proposed mechanism.

[0233] In another embodiment, s is a predefined value. For example, s = 1, 2, 3, or any positive integer. In one sub-embodiment, s is fixed to a predefined value in the standard. In another sub-embodiment, s is adaptive based on the encoding / decoding information of the current block, the block width, the block height, and / or the block area, and / or the encoding / decoding information, the block width, the block height, and / or the block area of ​​neighboring blocks. For example, if the block width, height, or area is greater than a predefined threshold, s is a larger number; otherwise, s is a smaller number.

[0234] In another embodiment, when model parameters are derived on the reference region of the current block using a regression method, the distortion to be minimized is between the combined results, including (1) the combined predictions generated from mode tap 0 to s-1 of the reference region of the current block, (2) the bias and (3) the weighting settings, and between the reconstructed samples of the reference region of the current block.

[0235] In another embodiment, sourceTermSet0 can be used in an expression. That is, the target chromaticity sample can be generated using the corresponding luminance information.

[0236] In another embodiment, the term “TIMD-CCM” can be replaced with any cross-component tool that uses multiple predictive hypotheses from multiple CCM models to form a final prediction to apply the proposed mechanism when a specific codec tool is used for the current block.

[0237] In another embodiment, a long-tap post-filter is applied when generating target predictions for the current block and / or generating template predictions over a reference region of the current block. The filter shape can be any of the patterns proposed in the above invention.

[0238] The method proposed in this invention can be enabled and / or disabled based on implicit rules (e.g., block width, height, or area) or explicit rules (e.g., syntax for blocks, tiles, slices, images, SPS, or PPS levels). For example, the proposed method is applied when the block area is less than / greater than a threshold.

[0239] In this invention, the term "block" can refer to TU / TB, CU / CB, PU / PB, a predefined region, or CTU / CTB.

[0240] Any combination of regression-based combined prediction methods proposed in this invention can be applied. In one embodiment, for chroma, more primary transform candidates are allowed when using the proposed mechanism. Any of the regression-based combined prediction methods proposed above can be implemented in the encoder and / or decoder. For example, any of the proposed methods can be implemented in the encoder's inter-frame / intra-frame / IBC / prediction / transform module (e.g., Figure 1A Intra-frame prediction 110 / inter-frame prediction 112) and / or the decoder's inter-frame / intra-frame / IBC / prediction / transform module (e.g., Figure 1B The method is implemented in the intra-frame prediction 150 / MC 152. Alternatively, any proposed method can be implemented as a circuit coupled to the inter-frame / intra-frame / IBC / prediction / transform module of the encoder and / or the inter-frame / intra-frame / IBC / prediction / transform module of the decoder to provide the information required by the inter-frame / intra-frame / IBC / prediction / transform module.

[0241] Figure 13 A flowchart of an exemplary video encoding / decoding system is shown, which, according to an embodiment of the invention, derives a combined prediction based on template regression. The steps shown in the flowchart can be implemented as program code executable on one or more processors (e.g., one or more CPUs) at the encoder and / or decoder ends. The steps shown in the flowchart can also be implemented in hardware, such as one or more electronic devices or processors arranged to perform the steps in the flowchart. According to the method, in step 1310, input data associated with the current block is received, wherein the input data includes pixel data to be encoded at the encoder end or encoded data associated with the current block to be decoded at the decoder end. In step 1320, a combined prediction is derived, wherein the combined prediction includes a weighted combination of one or more source items, and wherein the weighting of the one or more source items is derived using a regression technique based on relation estimation of one or more templates, and the one or more source items correspond to different prediction modes of the current block, and the different prediction modes include at least one non-intra-frame mode, or the one or more source items correspond to member samples determined in modes associated with the samples to be predicted of the current block. In step 1330, the current block is encoded or decoded using information including the combined prediction.

[0242] The flowchart shown is intended to illustrate an example of video encoding and decoding according to the present invention. Those skilled in the art can modify each step, rearrange the steps, split the steps, or combine the steps to practice the invention without departing from its spirit. Specific syntax and semantics are used in this disclosure to illustrate examples of implementing the invention. Those skilled in the art can practice the invention by using equivalent syntactic and semantic substitutions without departing from its spirit.

[0243] The foregoing description is intended to enable those skilled in the art to practice the invention in the context of specific applications and their requirements. Various modifications to the described embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments. Therefore, the invention is not intended to be limited to the specific embodiments shown and described, but is to be given the broadest scope consistent with the principles and novel features disclosed herein. Various specific details have been shown in the foregoing detailed description to provide a thorough understanding of the invention. However, those skilled in the art will understand that the invention can be practiced.

[0244] The embodiments of the invention described above can be implemented in various hardware, software code, or combinations thereof. For example, one embodiment of the invention may be program code integrated into a video compression chip or into video compression software to perform the processes described herein. Another embodiment of the invention may be program code to be executed on a digital signal processor (DSP) to perform the processes described herein. The invention may also relate to multiple functions performed by a computer processor, digital signal processor, microprocessor, or field-programmable gate array (FPGA). These processors can be configured to perform specific tasks by executing machine-readable software code or firmware code that defines the specific methods embodied in the invention. The software code or firmware code may be developed in different programming languages ​​and different formats or styles. The software code may also be compiled for different target platforms. However, different code formats, styles, and languages ​​of the software code, as well as other configuration codes to perform the tasks consistent with the invention, do not depart from the spirit and scope of the invention.

[0245] The invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described examples are to be regarded in all respects as illustrative rather than restrictive. Therefore, the scope of the invention is indicated by the appended claims rather than the foregoing description. All variations within the meaning and equivalence of the claims should be included within their scope.

Claims

1. A video encoding / decoding method, the method comprising: Receive input data associated with the current block, wherein the input data includes pixel data to be encoded at the encoder end or encoded data associated with the current block to be decoded at the decoder end; Derive a combined prediction, wherein the combined prediction comprises a weighted combination of one or more source items, and wherein the weighting of the one or more source items is derived using regression techniques based on relation estimates evaluated on one or more templates, and the one or more source items correspond to different prediction modes of the current block, and the different prediction modes include at least one non-intra-frame mode, or the one or more source items correspond to member samples determined in a mode associated with the sample to be predicted in the current block. And use the information including the combined predictions to encode or decode the current block.

2. The method as described in claim 1, characterized in that, Each member sample corresponds to the initial prediction sample of the combined prediction model.

3. The method as described in claim 1, characterized in that, The one or more templates include the upper reference region of the current block, the left reference region of the current block, the upper left reference region of the current block, or a combination thereof.

4. The method as described in claim 1, characterized in that, This combined prediction corresponds to the target prediction generated based on inter-frame and intra-frame prediction (CIIP).

5. The method as described in claim 4, characterized in that, Each of the one or more source terms corresponds to one or more prediction samples associated with an inter-frame mode index from an inter-frame candidate list or an intra-frame mode index from an intra-frame candidate list.

6. The method as described in claim 4, characterized in that, The one or more source terms correspond to one or more combined prediction samples derived from the CIIP.

7. The method as described in claim 1, characterized in that, This combined prediction corresponds to the target prediction generated based on multiple hypothesis prediction (MHP).

8. The method as described in claim 7, characterized in that, Each of the one or more source items corresponds to one or more predicted samples associated with an inter-frame mode index from an inter-frame candidate list.

9. The method as described in claim 7, characterized in that, The one or more source terms correspond to one or more combined prediction samples derived from the MHP.

10. The method as described in claim 1, characterized in that, This combined prediction corresponds to the target prediction generated based on the dual prediction (BCW) with CU level weighting.

11. The method as described in claim 10, characterized in that, Each of the one or more source items corresponds to one or more predicted samples associated with an inter-frame mode index from an inter-frame candidate list.

12. The method as described in claim 10, characterized in that, The one or more source terms correspond to one or more combined prediction samples derived from the BCW.

13. The method as described in claim 1, characterized in that, This regression technique corresponds to the Linear Minimum Mean Square Error (LMMSE) technique.

14. The method as described in claim 1, characterized in that, This regression technique corresponds to the LDL decomposition technique.

15. The method as described in claim 1, characterized in that, This regression technique is consistent with techniques used for cross-component chromaticity modes.

16. The method as described in claim 1, characterized in that, This regression technique corresponds to Gaussian elimination.

17. The method as described in claim 1, characterized in that, This relationship estimate corresponds to the distortion between the predicted sample and the reconstructed sample on the one or more templates.

18. An apparatus for video encoding and decoding, the apparatus comprising one or more electronic devices or processors configured to: receive input data associated with a current block, wherein the input data includes pixel data to be encoded at an encoder end or encoded data associated with the current block to be decoded at a decoder end; derive a combined prediction, wherein the combined prediction includes a weighted combination of one or more source items, and wherein one or more weights of the weighted combination of the one or more source items are evaluated on one or more templates using a regression technique based on relation estimation, and wherein the one or more source items correspond to different prediction modes of the current block, and the different prediction modes include at least one non-intra-frame mode, or the one or more source items correspond to member samples determined in a mode associated with a sample to be predicted of the current block; and encode or decode the current block using information including the combined prediction.

Images