Methods and apparatus of combined prediction mode with more inter modes for video coding

Abstract

A method and apparatus for video encoding and decoding are disclosed. According to the method, input data associated with a current block is received, wherein the input data includes pixel data to be encoded at the encoder side or encoded data associated with the current block to be decoded at the decoder side. A combined prediction including a target inter-frame prediction and at least one target intra-frame prediction and a neighboring reconstructed sample is determined, wherein the target inter-frame prediction refers to a sub-block-based inter-frame prediction or a prediction associated with GPM (Geometric Partition Mode). The current block is encoded or decoded using the prediction data including the combined prediction.

Description

Combined prediction mode method and device combining more inter-frame modes in video coding and decoding The present invention relates to a video coding and decoding method, and in particular to a derivative combined prediction method, wherein the inter-frame prediction may refer to a sub-block-based prediction or a prediction related to a GPM. Versatile Video Coding (VVC) is the latest international video codec standard developed by the Video Coding Experts Group (VCEG) of the International Telecommunication Union-Telecommunication Standardization Sector (ITU-T) and the Joint Video Experts Group (JVET) of the International Organization for Standardization / International Electrotechnical Commission Moving Picture Experts Group (ISO / IEC MPEG). The standard was published as an international standard: ISO / IEC 23090-3:2021, Information technology — Coded immersive media representation — Part 3: Versatile video codec, in February 2021. VVC builds on its predecessor, High Efficiency Video Coding (HEVC), by adding additional codec tools to improve codec efficiency and handle various video sources, including three-dimensional (3D) video signals. Figure 1A illustrates an exemplary adaptive intra / inter video coding system that includes loop processing. For intra prediction 110, prediction information is derived based on previously encoded video data from the current picture. For inter prediction 112, motion estimation (ME) is performed on the encoder side, and motion compensation (MC) is performed based on the ME results to provide prediction information derived from other pictures and motion data. Switch 114 selects either intra prediction 110 or inter prediction 112 and provides the selected prediction data to adder 116 to form a prediction error, also known as a residual. The prediction error is then transformed (T) 118 and subsequently quantized (Q) 120. The transformed and quantized residual is then encoded by entropy encoder 122 for inclusion in a video bitstream corresponding to the compressed video data. The bitstream associated with the transform coefficients is then packaged with side information such as the motion and codec modes associated with the intra and inter predictions, as well as parameters associated with the loop filters applied to the underlying image regions. Side information related to intra-frame prediction 110, inter-frame prediction 112, and loop filter 130, as shown in Figure 1A, is provided to entropy encoder 122. When using cross-prediction mode, the reference picture or pictures must also be reconstructed at the encoder end. Therefore, the transformed and quantized residual is processed through inverse quantization (IQ) 124 and inverse transform (IT) 126 to recover the residual. The residual is then added back to the prediction data 136 to reconstruct the video data at reconstruction (REC) 128. The reconstructed video data can be stored in the reference picture buffer 134 and used to predict other frames. As shown in Figure 1A , incoming video data undergoes a series of processing within the encoding system. The reconstructed video data from REC 128 may be subject to various impairments due to this processing. Therefore, a loop filter 130 is typically applied to the reconstructed video data before it is stored in a reference picture buffer 134 to improve video quality. For example, a deblocking filter (DF), sample adaptive offset (SAO), and an adaptive loop filter (ALF) may be used. Loop filter information may need to be included in the bitstream so that the decoder can correctly recover the required information. Therefore, loop filter information is also provided to the entropy encoder 122 for inclusion in the bitstream. In Figure 1A , loop filter 130 is applied to the reconstructed video, and the reconstructed samples are then stored in a reference picture buffer 134. The system shown in Figure 1A is intended to illustrate an exemplary structure of a typical video encoder. It may correspond to a High Efficiency Video Codec (HEVC) system, VP8, VP9, ​​H.264, or VVC. The decoder, as shown in Figure 1B , can utilize the same or partially identical functional blocks as the encoder, with the exception of transform 118 and quantization 120, as the decoder only requires inverse quantization 124 and inverse transform 126. Instead of using the entropy encoder 122, the decoder uses an entropy decoder 140 to decode the video bitstream into quantized transform coefficients and the required codec information (e.g., ILPF information, intra-frame prediction information, and inter-frame prediction information). Intra-frame prediction 150 on the decoder side does not require a mode search. Instead, the decoder only needs to generate intra-frame predictions based on the intra-frame prediction information received from the entropy decoder 140. Furthermore, for inter-frame prediction, the decoder only needs to perform motion compensation (MC 152) based on the inter-frame prediction information received from the entropy decoder 140, without requiring motion estimation. Intra-mode codec with 67 Intra prediction modes To capture arbitrary edge directions present in natural videos, the number of directional intra patterns in VVC is expanded to 65 from the 33 used in HEVC. In VVC, several conventional angular intra prediction modes are adaptively replaced with wide-angle intra prediction modes for non-square blocks. Intra-mode codec In order to keep the complexity of the Most Probable Mode (MPM) list generation low, an intra-mode codec approach using 6 MPMs (or primary MPMs) is used, taking into account two available neighboring intra modes. The following three aspects are considered when building the MPM list: – Preset intra modes – Neighboring intra modes – Derived intra modes Wide Angle Intra Prediction for Non-square Blocks (WAIP ) The traditional angular intra prediction direction is defined from 45 degrees clockwise to -135 degrees. In VVC, several conventional angular intra prediction modes are adaptively replaced with wide-angle intra prediction modes for non-square blocks. To support these prediction directions, a top reference of length 2W+1 and a left reference of length 2H+1 are defined as shown in Figures 2A and 2B. Decoder-side intra mode derivation (DIMD ) When DIMD is applied, two intra modes are derived from the reconstructed neighboring samples (templates). These two predictors are combined with the planar mode predictor with weights derived from the gradients. DIMD mode is used as an alternative prediction mode and is always checked in high-complexity RDO mode. To implicitly derive the intra prediction mode for a block, texture gradient analysis is performed on both the encoder and decoder sides. The process starts with an empty Histogram of Gradients (HoG) containing 65 entries corresponding to 65 angular modes. The magnitudes of these entries are determined during texture gradient analysis. In the first step, DIMD selects T=3 columns and rows from the left and above the current block as a template. This region is used as a reference for gradient-based intra prediction mode derivation. In the second step, horizontal and vertical Sobel filters are applied to all 3×3 window positions, centered on the pixels in the middle row of the template. At each window position, the Sobel filter calculates the intensity in the pure horizontal and vertical directions respectively as and . Then calculate the texture angle of the window as: (1) It can be converted into one of the 65 angular intra prediction modes. Once the intra prediction mode index of the current window is derived as idx, the magnitude of its entry in HoG[idx] is updated by adding: (2) FIG3 shows an example of a HoG calculated after applying the above operations to all pixel positions in the template. FIG3A illustrates an example of a selected template 320 for a current block 310. The template 320 contains T rows above the current block and T columns to the left of the current block. For intra prediction of the current block, the reconstruction area 330 corresponds to a reconstruction area above and to the left of the current block, and the unavailable area 340 corresponds to an unavailable area below and to the right of the block. FIG3B illustrates an example where T=3, and the HoGs are calculated for the pixel 360 in the middle row and the pixel 362 in the middle column. For example, for pixel 352, a 3x3 window 350 is used. FIG3C illustrates an example of the amplitude (ampl) calculated based on equation (2) for the angular intra prediction mode determined from equation (1). Once the HoG is calculated, the indices with the two highest histogram bins are selected as the two implicitly derived intra prediction modes (IPMs) of the block and further combined with the planar mode as the prediction for DIMD mode. Prediction fusion is applied as a weighted average of the above three predictors. Template-based Intra Mode Derivation , referred to as TIMD ) The Template-Based Intra Mode Derivation (TIMD) mode implicitly uses neighboring templates to derive the intra prediction mode for a CU at both the encoder and decoder, rather than signaling the intra prediction mode to the decoder. As shown in Figure 4, prediction samples from a template (412 and 414) for the current block (current CU) 410 are generated for each candidate mode using the template's reference samples (420 and 422). The SATD (Sum of Absolute Transformed Differences) between the prediction samples and the reconstructed samples from the template is calculated as the cost. The intra prediction mode with the lowest cost is selected as the TIMD mode (similar to the derivative of the DIMD mode) and used for intra prediction of the CU. Candidate modes can be any of the 67 intra prediction modes in VVC, or an expanded list of 131 intra prediction modes. Generally speaking, multi-mode prediction (MPMs) can provide clues indicating the CU's directional information. Therefore, to reduce the intra mode search space and exploit CU characteristics, intra prediction modes can be implicitly derived from the MPM list. For each intra prediction mode in the MPMs, the SATD between the predicted and reconstructed samples of the template is calculated. The first two intra prediction modes with the smallest SATD are selected as TIMD modes. These two TIMD modes are fused with weights after applying the PDPC process, and this weighted intra prediction is used to encode the current CU. Position dependent intra prediction combination (PDPC) is included in the derivation of TIMD modes. The costs of the two selected modes are compared with a threshold. In the test, a cost factor of 2 is applied as follows: costMode2 < 2*costMode1. If this condition is true, fusion is applied, otherwise only mode1 is used. The weights of the modes are calculated according to their SATD costs as follows: weight1 = costMode2 / (costMode1+ costMode2) weight2 = 1 - weight1. Intra Sub-Partitions (ISP) Intra sub-partitioning (ISP) divides the luma intra prediction block into two or four sub-partitions vertically or horizontally, depending on the block size. For each sub-partition, reconstructed samples are obtained by adding the residual signal to the prediction signal. Here, the residual signal is generated through processes such as entropy decoding, inverse quantization, and inverse transform. Therefore, the reconstructed sample values ​​of each sub-partition are used to generate the prediction for the next sub-partition, and each sub-partition is processed repeatedly. In addition, the sub-partition processed first is the sub-partition containing the sample in the top-left corner of the CU, and then continues downward (horizontal partitioning) or to the right (vertical partitioning). Therefore, the reference samples used to generate the sub-partition prediction signal are only located to the left and above the line. All sub-partitions share the same intra mode. Template Matching Prediction , referred to as TMP ) JVET-V0130 and JVET-U0048 disclose Template Matching Prediction (TMP). TMP is a special intra-frame prediction mode that copies the best prediction block from the reconstructed portion of the current frame, whose L-shaped template matches the current template. This is illustrated in Figure 5, where block 510 is a current block and block 520 is a prediction block. Within a predefined search range 530, the encoder searches for the template 522 most similar to the current template 512 in the reconstructed portion 550 of the current frame 540 and uses the corresponding block 520 as the prediction block (as a reference block). The encoder then signals the use of this mode, and the decoder performs the inverse operation. Chroma Intra Mode Codec Chroma mode encoding and decoding directly depends on the intra prediction mode of the corresponding luma block. Since separate block partitioning structures for luma and chroma components are enabled in I slices, a chroma block may correspond to multiple luma blocks. Therefore, for chroma DM mode, the intra prediction mode of the corresponding luma block covering the center position of the current chroma block is directly inherited. An extrapolation filter based on intra prediction , referred to as EIP )model In JVET-AF0080 (Luhang Xu et al., "EE2-2.7: An Extrapolation Filter Based on Intra Prediction Modes," Joint Video Experts Group (JVET) ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29, 32nd Meeting, Hannover, Germany, October 13–20, 2023, Archives: JVET-AF0080), an extrapolation filter based on intra prediction is presented. EIP prediction is performed in three steps. First, extrapolation filter coefficients are inherited from the neighboring reconstructed area of ​​the current block or from the previous EIP block. Second, the extrapolation process generates a prediction signal from the top-left corner to the bottom-right corner within the current block. Third, the intra prediction angle is derived by analyzing the gradient of the prediction block, and then transformed using the corresponding intra mode selection MTS (Multiple Transform Selection), NSPT (Non-Separable Primary Transform), and LFNST (Low Frequency Non-Separable Transform) cores. Obtaining EIP filter Three EIP filter shapes are shown in FIG. 6 , where the three filter shapes correspond to square 610 , horizontal bar 620 , and vertical bar 630 . According to JVET-AF0080, there are two ways to obtain the filter coefficients of the current CU. First, the coefficients can be derived from neighboring reconstructed pixels, and second, they can also be inherited from previously decoded blocks. Derivative EIP coefficient The decoder decodes the relevant syntax elements to determine the selected reconstruction region type and filter shape for the current block. The selected filter is shifted horizontally or vertically within the selected reconstruction region, one pixel at a time, to construct the autocorrelation matrix and cross-correlation vector. The coefficients are calculated from the autocorrelation matrix and cross-correlation vector in the same way as in the convolutional cross-component model (CCCM). Three types of reconstruction regions are defined, as shown in Figure 7. These regions correspond to the upper left region (Figure 7A), the upper region (Figure 7B), and the left region (Figure 7C). The size of the reconstruction region depends on min(block width, block height) and the selected filter shape. For example, when the current block is an 8x16 block and the selected filter shape is 4x4, the upper size (aboveSize) of the reconstruction region is min(8, 16) + 4 – 1 = 11, while the left size (leftSize) of the reconstruction region is min(8, 16) + 4 – 1 = 11. EIP Filter inheritance EIP merge mode is also disclosed in JVET-AF0080. The filter shapes and filter coefficients can be inherited from a previously decoded block with EIP or EIP merge mode. Prediction of the current block The EIP mode generates prediction values ​​for the current block in a diagonal prediction order from the upper left position to the lower right position, as shown in Figure 8. The calculation of the predicted value in JVET-AF0080 is as follows, in is the predicted value at (x, y) in the current block, is the selected EIP filter Coefficients, the coefficient index ranges from 0 to 14, is the reconstructed or predicted value used for the current position prediction. and The x and y offsets from the current position. Enhanced MTS For intra-frame encoding and decoding In the current VVC design, for MTS, only DST7 and DCT8 transform cores are used, which are used for intra-frame and inter-frame coding and decoding. Additional major transforms including DCT5, DST4, DST1, and identity transform (IDT) are adopted. In addition, the MTS set is determined based on the TU size and intra-frame mode information. For blocks predicted by IntraTMP (intra-frame template matching prediction), the prediction block is processed using the DIMD process to derive the intra-frame mode for transform selection. Specifically, the horizontal and vertical gradients are calculated for each prediction sample to establish the HoG. The intra-frame prediction mode with the largest histogram amplitude value is then used to select the MTS transform set. Secondary Transformation: LFNST Expansion and large core The LFNST design in VVC is extended as follows: ● The number of LFNST sets (S) and candidates (C) is extended to S=35 and C=3, and the LFNST set (lfnstTrSetIdx) for a given intra mode (predModeIntra) is derived according to the following formula: o For predModeIntra < 2, lfnstTrSetIdx is equal to 2 o lfnstTrSetIdx = predModeIntra, in the range [0,34] for predModeIntra o lfnstTrSetIdx = 68 – predModeIntra, in the range [35,66] for predModeIntra ● Three different kernels are defined, LFNST4, LFNST8 and LFNST16, which are applied to 4xN / Nx4 (Nx4) respectively. 4), 8xN / Nx8 (N 8) and MxN (M, N 16). The mapping of these sets to intra prediction modes is shown in Table 1. Table 1. Mapping of intra prediction modes to LFNST set indices For blocks predicted using MIP (Matrix-based Intra Prediction) or IntraTMP, the LFNST set index is derived as follows: DIMD is used to derive the intra mode of the current block based on MIP or IntraTMP prediction samples. Non-Separable Primary Transform , referred to as NSPT ) for intra-frame encoding and decoding The separable DCT-II plus LFNST transform combination is replaced by NSPT for block shapes 4x4, 4x8, 8x4 and 8x8, 4x16, 16x4, 8x16 and 16x8. All NSPTs consist of 35 sets and 3 candidates (similar to the current LFNST). The core of NSPT has the following shapes: ● NSPT4x4: 16x16 ● NSPT4x8 / NSPT8x4: 32x20 ● NSPT8x8: 64x32 ● NSPT4x16 / NSPT16x4: 64x24 ● NSPT8x16 / NSPT16x8: 128x40 ● NSPT4x32 / NSPT32x4: 128x20 ● NSPT8x32 / NSPT32x8: 256x24 Therefore, 12, 32, 40, and 88 coefficients are zeroed for NSPT4x8 / NSPT8x4, NSPT8x8, NSPT4x16 / NSPT16x4, and NSPT8x16 / NSPT16x8, respectively. For NSPT4x32 / NSPT32x4 and NSPT8x32 / NSPT32x8, the remaining 108 and 232 locations in each transform block are zeroed, respectively. Inter-frame prediction overview According to JVET-T2002 (Jianle Chen et al., "Algorithmic Description of the Versatile Video Codec and Test Model 11 (VTM 11)", Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29, 20th Meeting, held by teleconference, October 7–16, 2020, document: JVET-T2002), for each inter-predicted codec unit (CU), motion parameters include motion vectors, reference picture indices, and reference picture list usage indices, as well as additional information required to utilize the new codec features of VVC for inter-prediction sample generation. Motion parameters can be explicitly or implicitly signaled. When a CU is coded in skip mode, the CU is associated with a single prediction unit (PU) and has no significant residual coefficients, no coded motion vector differences, or reference picture indices. A merge mode is specified, in which the motion parameters of the current CU are obtained from neighboring CUs, including spatial and temporal candidates, as well as additional schemes introduced in VVC. Merge mode can be applied to any inter-predicted CU, not just skip mode. An alternative to merge mode is explicit transmission of motion parameters, where each CU is explicitly marked with the motion vector, the reference picture index for each reference picture list, the reference picture list usage flag, and other necessary information. In addition to the inter codec features in HEVC, VVC includes many new and improved inter prediction codec tools listed below: – Extended merge prediction – Merge mode with MVD (MMVD) – Symmetric MVD (SMVD) signal – Affine motion compensated prediction – Sub-block based temporal motion vector prediction (SbTMVP) – Adaptive motion vector resolution (AMVR) – Motion field storage: 1 / 16 th Luma sample MV storage and 8x8 motion field compression – Bi-prediction with CU-level weights (BCW) – Bidirectional optical flow (BDOF) – Decoder-side motion vector refinement (DMVR) – Geometric partitioning mode (GPM) – Combined inter- and intra-frame prediction (CIIP) The following description provides details of the inter prediction method specified in VVC. Extended Merger Forecast In VVC, the merge candidate list is constructed by including the following five types of candidates in order: 1) spatial MVP from spatially neighboring CUs 2) temporal MVP from co-located CUs 3) history-based MVP from FIFO table 4) pairwise average MVP 5) zero MV. The size of the merge list is signaled in the sequence parameter set (SPS) header, and the maximum allowed size of the merge list is 6. For each CU coded in merge mode, the index of the best merge candidate is encoded using truncated unary binarization (TU). The first binary bit of the merge index is context-coded, and the remaining binary bits are decoded using bypass codecs. A derivation process for each merge candidate class is provided. As in HEVC, VVC also supports parallel derivation of merge candidate lists (or merge candidate lists) for all CUs in a certain size region. Spatial candidate derivatives The derivation of spatial merge candidates in VVC is the same as in HEVC, except that the positions of the first two merge candidates are swapped. In the positions shown in Figure 9, up to four merge candidates (B 0, A 0, B 1 and A 1). The order of derivation is B 0, A 0, B 1, A 1 and B 2. Only when one or more neighboring CU positions B 0, A 0, B 1, A Position B is considered only when 1 is not available (e.g., belongs to another slice or tile) or is intra-coded 2. Time candidate derivatives In this step, only one candidate is added to the list. Specifically, when deriving this temporal merge candidate for the current CU 1010, a scaled motion vector is derived from the collocated CU 1020 in the collocated reference picture shown in FIG10 . The reference picture list and reference index used to derive the collocated CU are explicitly marked in the slice header. The scaled motion vector 1030 of the temporal merge candidate is scaled from the motion vector 1040 of the collocated CU, as shown by the dashed line in FIG10 , using the POC (Picture Order Count) distances tb and td, where tb is defined as the POC difference between the current picture and the reference picture, and td is defined as the POC difference between the reference picture and the collocated picture. The reference picture index of the temporal merge candidate is set to zero. History-based Merge Candidate Derivation After spatial MVP and TMVP, history-based MVP (HMVP) merge candidates are added to the merge list. In this method, the motion information of previously coded blocks is stored in a table and used as the MVP for the current CU. A table containing multiple HMVP candidates is maintained during the encoding and decoding process. When a new CTU row is encountered, the table is reset (cleared). Whenever a non-subblock inter-coded CU is encountered, the associated motion information is added to the last entry of the table as a new HMVP candidate. With MVD (MMVD Merge Mode with MVD In addition to the merge mode, in which the implicitly derived motion information is directly used for prediction sample generation of the current CU, VVC introduces a merge mode with motion vector difference (MMVD). After sending the regular merge flag, the MMVD flag is sent immediately to specify whether to use the MMVD mode for the CU. In MMVD, after a merge candidate is selected (referred to in this disclosure as the base merge candidate), it is further refined by signaling MVD information. This further information includes a merge candidate flag, an index specifying the motion magnitude, and an index indicating the motion direction. In MMVD mode, one of the first two candidates in the merge list is selected as the MV base. The MMVD candidate flag is sent to specify which of the first and second merge candidates to use. The distance index specifies the motion magnitude information and indicates the predefined offsets from the starting point (1112 and 1122) for the L0 reference block 1110 and the L1 reference block 1120. As shown in Figure 11, the offset is added to the horizontal component or vertical component of the starting MV, where the small circles of different patterns correspond to different offsets from the center. The relationship between the distance index and the predefined offset is specified in Table 2. Table 2 - Relationship between distance index and predefined offset The direction index indicates the MVD direction relative to the starting point. The direction index can indicate four directions as shown in Table 3. Table 3 - MV offset symbols specified by the direction index Affine motion compensated prediction In HEVC, only the translational motion model is used for motion-compensated prediction (MCP). However, in the real world, there are many types of motion, such as zooming in / out, rotation, perspective motion, and other irregular motion. In VVC, block-based affine transformation motion compensation prediction is applied. As shown in Figures 12A-B, the affine motion field of block 1210 is described by motion information, namely, two control point motion vectors (4 parameters) in Figure 12A or three control point motion vectors (6 parameters) in Figure 12B. For the 4-parameter affine motion model, the motion vector for a sample position (x, y) in the block is derived as: (3) For the 6-parameter affine motion model, the motion vector for a sample position (x, y) in a block is derived as: (4) Among them (mv 0x , mv 0y ) is the motion vector of the upper left control point, (mv 1x , mv 1y ) is the motion vector of the upper right control point, and (mv 2x , mv 2y ) is the motion vector of the lower left control point. To simplify motion compensated prediction, block-based affine transform prediction is applied. To derive the motion vector for each 4×4 luma subblock, the motion vector of the center sample of each subblock is calculated, as shown in Figure 13, according to the above equation and rounded to 1 / 16 fractional precision. Then, a motion compensated interpolation filter is applied to generate a prediction for each subblock using the derived motion vector. The subblock size of the chroma component is also set to 4×4. The MV of the 4×4 chroma subblock is calculated as the average of the MVs of the upper left and lower right luma subblocks located in the same 8x8 luma region. Like translational motion inter prediction, there are two affine motion inter prediction modes: affine merge mode and affine AMVP (Advanced MVP) mode. Affine Merge Predictions Affine merge mode can be applied to coding units (CUs) with width and height both greater than or equal to 8. In this mode, the control point motion vector (CPMV) of the current CU is generated based on the motion information of spatially neighboring CUs. There can be up to five control point motion vector prediction (CPMVP) candidates, and an index is sent to indicate the candidate for the current CU. The following three types of CPMVP candidates are used to form the affine merge candidate list: – Inherited affine merge candidates derived from the CPMVs of neighboring CUs; ​​– Constructed affine merge candidate CPMVPs derived using the translated MVs of neighboring CUs; ​​– Zero MV; In VVC, there are at most two inherited affine candidates, which are derived from the affine motion model of the neighboring blocks, one from the left neighboring CU and one from the upper neighboring CU. The candidate blocks are the same as those shown in Figure 9. For the left predictor, the scanning order is A 0->A 1. For the upper predictor, the scanning order is B 0->B 1->B 2. Only the first successor candidate is selected on each side. No pruning check is performed between two successor candidates. When a neighboring affine CU is identified, its control point motion vector is used to derive the CPMVP candidate in the affine merge list of the current CU. As shown in Figure 14, if the neighboring lower left block A of the current block 1410 is coded in affine mode, the motion vectors of the upper left corner, upper right corner and lower left corner of block A in CU 1420 are 、 and When block A is encoded with a 4-parameter affine model, the two CPMVs of the current CU (i.e. and )according to and If block A is encoded with a 6-parameter affine model, the three CPMVs of the current CU are calculated according to 、 and calculate. Constructing affine candidates means that candidates are constructed by combining the neighboring translational motion information of each control point. The motion information of the control point is derived from the specified spatial neighbors and the temporal neighbors of the current block 1510, as shown in Figure 15. CPMV k (k=1, 2, 3, 4) represents the kth control point. For CPMV 1. Check B 2->B 3->A 2 blocks, and use the MV of the first available block. For CPMV 2. Check B 1->B 0 blocks for CPMV 3. Check A 1->A 0 blocks. For CPMV 4. If available, use TMVP as CPMV 4. After obtaining the MVs of the four control points, an affine merge candidate is constructed based on the motion information. The following combinations of control point MVs are used in order for construction: {CPMV1, CPMV2, CPMV3}, {CPMV1, CPMV2, CPMV4}, {CPMV1, CPMV3, CPMV4}, {CPMV2, CPMV3, CPMV4}, {CPMV1, CPMV2}, {CPMV1, CPMV3} A combination of 3 CPMVs constructs a 6-parameter affine merge candidate, and a combination of 2 CPMVs constructs a 4-parameter affine merge candidate. To avoid motion scaling, if the reference indices of the control points are different, the related control point MV combination is discarded. After checking inherited affine merge candidates and constructing affine merge candidates, if the list is still not full, a zero MV is inserted to the end of the list. Affine AMVP predict Affine AMVP mode can be applied to CUs with width and height both greater than or equal to 16. An affine flag is sent in the bitstream at the CU level to indicate whether to use the affine AMVP mode, and then another flag is sent to indicate whether to use 4-parameter affine or 6-parameter affine. In this mode, the difference between the CPMV of the current CU and its predictor CPMVPs is sent in the bitstream. The affine AMVP candidate list size is 2 and is generated using the following four types of CPMVP candidates in the following order: – Inherited affine AMVP candidates derived from the CPMVs of neighboring CUs; ​​– Constructed affine AMVP candidate CPMVPs derived using the translated MVs of neighboring CUs; ​​– Shifted MVs from neighboring CUs; ​​– Zero MVs; Geometric Partitioning Mode (GPM ) In VVC, Geometric Partition Mode (GPM) for inter-frame prediction is supported. The geometric partition mode is marked by a merge mode using a CU level flag. Other merge modes include regular merge mode, MMVD mode, CIIP mode, and sub-block merge mode. Each possible CU size supports 64 partitions. and , excluding 8x64 and 64x8. When using this mode, the CU is split into two parts along a straight line at a specific geometric angle. The position of the split line is mathematically derived from the angle and offset parameters of the specific partition. Each geometric partition in the CU uses its own motion for inter prediction; only a single prediction is allowed for each partition, i.e., one motion vector and one reference index per partition. A single-prediction motion constraint is applied to ensure that only two motion-compensated predictions are required for each CU, the same as with traditional dual prediction. A single-prediction motion is derived for each partition. If the current CU uses geometric partitioning mode, a geometric partitioning index is further transmitted, indicating the selected partitioning mode for the geometric partitioning (angle and offset), along with two merge indices (one for each partition region). The maximum number of GPM candidate sizes is explicitly transmitted in the SPS (Sequence Parameter Set), and the syntax for the GPM merge indices is binary. After predicting each part of the geometric partitioning, a blending process with adaptive weights is used to adjust the sample values ​​along the edges of the geometric partitioning. This is the prediction signal for the entire CU, and the transform and quantization processes are applied to the entire CU as in other prediction modes. Finally, the motion field of the CU predicted using the geometric partitioning mode is stored. Combined inter-frame and intra-frame prediction (CIIP ) In VVC, when a codec unit (CU) is encoded in merge mode, if the CU contains at least 64 luma samples (i.e., the CU width multiplied by the CU height is equal to or greater than 64), and if both the CU width and the CU height are less than 128 luma samples, an additional flag is sent to indicate whether the combined inter / intra prediction (CIIP) mode is applied to the current CU. As the name suggests, CIIP prediction combines an inter prediction signal with an intra prediction signal. Inter prediction signal in CIIP mode is derived using the same inter-frame prediction process used in regular merge mode; the intra-frame prediction signal It is derived following the conventional intra prediction process using planar mode. The intra and inter prediction signals are then combined using a weighted average, where the weights wt are calculated based on the codec modes of the upper and left neighboring blocks of the current CU 1610 (as shown in Figure 16) as follows: – If the upper neighbor is available and is intra-coded, isIntraTop is set to 1, otherwise isIntraTop is set to 0; – If the left neighbor is available and is intra-coded, isIntraLeft is set to 1, otherwise isIntraLeft is set to 0; – If (isIntraLeft + isIntraTop) is equal to 2, wt is set to 3; – Otherwise, if (isIntraLeft + isIntraTop) is equal to 1, wt is set to 2; – Otherwise, wt is set to 1. The CIIP forecast is formed as follows: GPM Extensions At the JVET-V conference, several variants were proposed to improve the encoding and decoding efficiency of the GPM mode in VVC. These methods were included in the Exploration Experiment (EE2) for further evaluation, and their main technical aspects are described as follows: EE2-3.3 Regarding GPM with MMVD (GPM-MMVD): 1) Add additional MVD to the existing GPM merge candidates; 2) MVD is sent in the same way as MMVD in VVC, that is, a distance index plus a direction index; 3) Two flags are sent to control whether MMVD is applied to each GPM segmentation area. EE2-3.4-3.5 Regarding GPM with template matching (GPM-TM): 1) Extend template matching to GPM mode by refining GPM MVs based on the left and above neighboring samples of the current CU; 2) Select template samples based on the GPM split direction; 3) Send a single flag to jointly control whether template matching is applied to MVs of two GPM splits. In JVET-Y0065 (Yoshitaka Kidani et al., "EE2-3.1: GPM with inter and intra prediction (JVET-X0166)", Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29, 25th Meeting, held by teleconference, January 12-21, 2022, document: JVET-Y0065), in GPM with inter and intra prediction (or GPM intra or GPM-intra), the final prediction samples are generated by weighting the inter prediction samples and intra prediction samples for each GPM partition. The inter prediction samples are derived according to the same scheme as GPM in the current Enhanced Compression Model (ECM), while the intra prediction samples are derived from an intra prediction mode (IPM) candidate list and an index sent from the encoder. The IPM candidate list size is predetermined to be 3. Available IPM candidates are angular modes parallel to GPM block boundaries (parallel mode), angular modes perpendicular to GPM block boundaries (vertical mode), and planar modes, as shown in Figures 17A-C, respectively. Furthermore, as shown in Figure 17D, to reduce the signal overhead of IPMs and avoid increasing the size of the intra prediction circuitry in the hardware decoder, the proposed method restricts GPMs with intra and intra prediction. Furthermore, direct motion vectors and IPM storage are introduced in the GPM hybrid region to further improve codec performance. Non-adjacent spatial candidates As described in JVET-L0399 (Yu Han et al., "CE4.4.6: Improvements to Merge / Skip Mode," ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 of the Joint Video Exploration Team (JVET), 12th Meeting: Macau, CN, October 3-12, 2018, document: JVET-L0399), non-adjacent spatial merging candidates are inserted after the TMVP in the regular merging candidate list. The pattern of spatial merging candidates is shown in Figure 18. The distance between non-adjacent spatial candidates and the current codec block is based on the width and height of the current codec block. No line buffer restrictions apply. Template matching for MV Refinement Template Matching (TM) is a decoder-side MV derivation method for refining the motion information of the current CU 1912 in the current picture 1910 by finding the closest match between the neighboring blocks above 1914 and / or to the left 1916 of the current CU and blocks in the reference picture 1920 (i.e., blocks 1924 and 1926 of the same size as the template), as shown in Figure 19. In Figure 19, a better MV is searched within a [–8, +8]-pixel search range 1922 around a position 1928 in the reference picture 1920 around the initial motion 1930 of the current CU 1912. Adaptive Reorder Merging Candidate and Template Matching (ARMC-TM , Adaptive Reordering of Merge Candidates with Template Matching (ARMC-TM) Adaptively reorders merge candidates based on their costs evaluated using Template Matching (TM). The reordering method can be applied to regular merge mode, Template Matching (TM) merge mode, and affine merge mode (except for SbTMVP candidates). For TM merge mode, merge candidates are reordered before the refinement process. During the reordering process, the template matching cost of a merge candidate is measured by the SAD between samples of the current block's template and its corresponding reference samples. The template consists of a set of reconstructed samples adjacent to the current block. The template's reference samples are located using the merge candidate's motion information. When the merge candidate uses bi-prediction, the reference samples of the merge candidate's template are also generated using bi-prediction, as shown in Figure 20. In Figure 20, block 2012 corresponds to the current block in current picture 2010, and blocks 2022 and 2032 correspond to reference blocks in reference pictures 2020 and 2030, respectively, in List 0 and List 1. Templates 2014 and 2016 are used for current block 2012, templates 2024 and 2026 are used for reference block 2022, and templates 2034 and 2036 are used for reference block 2032. The merge candidate's motion vectors 2040, 2042, and 2044 are in List 0, and the merge candidate's motion vectors 2060, 2062, and 2064 are in List 1. For subblock-based merge candidates with subblock size equal to Wsub × Hsub, the above template includes several subblocks of size Wsub × 1, while the left template includes several subblocks of size 1 × Hsub. As shown in Figure 21, the reference samples for each subblock are derived using the motion information of the subblocks in the first row and first column of the current block. In Figure 21, block 2112 corresponds to the current block in current image 2110, and block 2122 corresponds to the co-located block in reference image 2120. Each small square in the current block and the co-located block corresponds to a subblock. The dot-filled areas to the left and top of the current block correspond to the template of the current block. Boundary subblocks are labeled A through G. The arrow associated with each subblock corresponds to the subblock's motion vector. The reference subblocks (labeled Aref through Gref) are positioned based on the motion vectors of the boundary subblocks. Intra-block copy (IBC ) Intra Block Copy (IBC) is a tool implemented in the HEVC extension of the Screen Content Codec (SCC). It's known to significantly improve the codec efficiency of screen content material. Implemented as a block-level codec, IBC performs block matching (BM) to find the optimal block vector (or motion vector) for each CU. A block vector indicates the displacement from the current block to a reference block reconstructed within the current picture. CUs encoded with IBC are treated as a third prediction mode, in addition to intra or inter prediction. IBC applies to CUs with a width and height less than or equal to 64 luma samples. Intra-frame template matching Intra-frame template matching prediction (IntraTMP, similar to or identical to the previously mentioned TMP) is a special intra-frame prediction mode that copies the best prediction block from the reconstructed portion of the current frame, whose L-shaped template matches the current template. For a predetermined search range, the encoder searches for the template most similar to the current template in the reconstructed portion of the current frame and uses the corresponding block as the prediction block. The encoder then signals the use of this mode, and the same prediction operation is performed on the decoder side. In the present invention, a method and apparatus for derived combined prediction are disclosed, wherein inter-frame prediction may refer to sub-block-based prediction or GPM-related prediction. The present invention discloses a method and apparatus for video encoding and decoding. According to the method, input data related to a current block is received, wherein the input data includes pixel data to be encoded at the encoder end or coded data related to the current block to be decoded at the decoder end. A combined prediction comprising a target inter-frame prediction and at least one target intra-frame prediction or adjacent reconstructed samples is determined, wherein the target inter-frame prediction is a sub-block-based inter-frame prediction or a prediction associated with a Geometric Partitioning Mode (GPM). The current block is encoded or decoded using the prediction data comprising the combined prediction. In one embodiment, the explicit signal is used to indicate whether to use sub-block based combined inter and intra prediction (CIIP) for the current block or whether to use GPM based CIIP for the current block. In one embodiment, the subblock-based inter prediction is generated using one or more subblock-based motion candidates from a subblock merge candidate list. In one embodiment, the subblock merge candidate list is unified with the subblock merge candidates of the affine and subblock-based temporal motion vector prediction (SbTMVP) modes. In one embodiment, sub-block based CIIP is supported for CIIP-Template Matching (CIIP-TM), CIIP-Position Dependent intra Prediction Combination (CIIP-PDPC), or both. In one embodiment, the GPM-related prediction is generated using GPM-TM, GPM-Merge mode with Motion Vector Differences (GPM-MMVD), GPM-Intra, or any combination or extension thereof. In one embodiment, the GPM-based CIIP supports CIIP-TM, CIIP-PDPC, or both. In one embodiment, the combined prediction is formed using a mixture of the specified neighboring reconstructed samples and the target inter-frame prediction. In one embodiment, the target inter-frame prediction is generated using one or more inter-frame motion candidates refined by template matching. In one embodiment, the one or more inter-frame motion candidates correspond to one or more merge candidates after the template matching. In one embodiment, the target intra prediction and the target inter prediction are mixed using a mixing weight, and a plurality of mixing methods with different weights are considered as a plurality of selectable modes of the combined prediction mode. In one embodiment, one or more intra modes are determined using a histogram analysis process. In one embodiment, the one or more intra modes correspond to a maximum histogram count, which is used to determine a transform set for the current block and / or a coding process for a subsequent coding block. In one embodiment, the transform process for the current block corresponds to a Low Frequency Non-Separable Transform (LFNST), a Non-Separable Primary Transform (NSPT), a Multiple Transform Selection (MTS), or a combination thereof. The components of the present invention, as generally described and shown in the figures, may be arranged and designed in a variety of different configurations. Therefore, the following more detailed description of the embodiments of the present systems and methods, as shown in the figures, is not intended to limit the scope of the invention, as requested, but is merely representative of selected embodiments of the invention. References in this specification to "one embodiment," "an embodiment," or similar language mean that a particular feature, structure, or characteristic described in connection with that embodiment may be included in at least one embodiment of the present invention. Therefore, the phrases "in one embodiment" or "in an embodiment" appearing throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. However, a person skilled in the relevant art will recognize that the present invention may be practiced without one or more of the specific details, or with other methods, components, and the like. In other cases, well-known structures or operations are not shown or described in detail to avoid obscuring aspects of the present invention. The embodiments of the present invention may be best understood by reference to the drawings, in which like parts are designated by like numbers. The following description is for example only and simply illustrates certain selected embodiments of apparatus and methods consistent with the invention claimed herein. In order to improve the encoding and decoding performance of combined inter-frame and intra-frame prediction (CIIP), the inter-frame prediction part is extended to refer to sub-block-based inter-frame prediction or prediction related to geometric partitioning mode (GPM). This invention proposes the following aspects to improve combined prediction, which is formed by using mode-type (e.g., intra) prediction and inter prediction with blending weights. This combined prediction is referred to as CIIP in this invention. In one embodiment, the inter portion of CIIP uses one or more conventional inter motion candidates to generate an inter prediction, where one or more inter motion candidates refers to conventional merge candidates without template matching (TM). In another embodiment, the intra portion of CIIP uses the intra prediction mode as a TIMD to generate an intra predictor. For example, intra prediction using TIMD-derived intra prediction modes and / or this method is only applicable to blocks with an area less than or equal to a threshold (e.g., 1024). The TIMD-derived method is used to derive intra prediction modes in CIIP. Specifically, the intra prediction mode with the smallest SATD value in the TIMD mode list is selected and mapped to one of the 67 conventional intra prediction modes. In another embodiment (for CIIP-PDPC), the intra portion of CIIP directly blends with the inter predictor using neighboring reconstructed samples, with blending weights dependent on the PDPC weights. PDPC weights measure the relationship between the current sample position (x, y) to be predicted and the corresponding adjacent sample above (R x, -1 ) and / or the current sample position to be predicted (x, y) and the corresponding adjacent sample on the left (R -1, y ) as shown in Figure 22. For example, the PDPC weights (wT and wL) are calculated as follows: – = 32 >> ((y<<1) >> nScale) – = 32 >> ((x<<1) >> nScale) – nScale = (floorLog2(width) + floorLog2(height) - 2) >> 2; In this example, the combined prediction (Pred(x, y)) for each sample (x, y) is formed as follows. In another embodiment (for CIIP-TM which may be labeled as CIIP TM), the inter portion of the CIIP may be generated using one or more inter motion candidates refined by template matching. For example, the inter merge candidate used is a merge candidate after template matching (TM). In this case, the merge candidates of the CIIP-TM are reordered according to the ARMC method as regular non-CIIP merge candidates. The maximum number of CIIP-TM merge candidates is equal to two or any predefined positive integer. Different variants of CIIP may be viewed as different optional modes of CIIP. For example, when the CIIP flag indicates that CIIP is applied to the current block, additional syntax elements are signaled to indicate the selected CIIP variant. One possible signaling method includes a flag (e.g., a CIIP-PDPC flag) to indicate whether PDPC is used for the intra portion of the CIIP and / or another flag (e.g., a CIIP-TM flag) to indicate whether TM is used for the inter portion of the CIIP. One syntax case (e.g. CIIP-PDPC flag = true and CIIP TM flag = false) refers to using PDPC as the intra part of CIIP and regular inter-motion candidates (without TM) as the inter part of CIIP. Another syntax case (e.g. CIIP-PDPC flag = false and CIIP TM flag = false) refers to using TIMD or a predefined regular intra prediction mode as the intra part of CIIP and regular inter-motion candidates (without TM) as the inter part of CIIP. Another syntax case (e.g. CIIP-PDPC flag = true and CIIP TM flag = true) refers to using PDPC as the intra part of CIIP and inter-motion candidates with TM as the inter part of CIIP. Another syntax case (e.g. CIIP-PDPC flag = false and CIIP TM flag = true) refers to using TIMD or a predefined regular intra prediction mode as the intra part of CIIP and inter-motion candidates with TM as the inter part of CIIP. In another embodiment, the mode type prediction is generated using extrapolation filter-based intra prediction (EIP). For example, the mode type prediction is generated in extrapolation filter-based intra prediction using any predefined mode, such as EIP normal mode (i.e., non-merge EIP mode), with the EIP mode index equal to a predefined value, where the predefined value is fixed to 0 or is adaptive based on the block position, block width, block height, and / or block area of ​​the current block. In another example, the mode type prediction is generated in extrapolation filter-based intra prediction using any predefined mode, such as EIP merge mode, with the EIP merge index equal to a predefined value, where the predefined value is fixed to 0 (i.e., the EIP merge mode at the front of the reordered or non-reordered EIP merge list) or is adaptive based on the block position, block width, block height, and / or block area of ​​the current block. In another embodiment, the mode type prediction in combined prediction is generated using one or more prediction mode information suggested by the TIMD process. Several aspects are presented below. Aspect 1 (as presented in Section I) is related to the candidate list suggested by the TIMD process. Aspect 2 (as presented in Section II) is related to how to generate a mode type prediction based on one or more suggested prediction mode information. Aspect 3 (as presented in Section III) is related to how to generate an inter-frame prediction. Aspect 4 (as presented in Section IV) is related to combining predictions and signals. In another embodiment, the inter-frame prediction here can be replaced by any predefined block vector prediction if the proposed method is applied to a target mode that refers to IBC or intra-frame TMP. I. Suggested candidate list The candidate list of the TIMD is generated based on the prediction mode information of the previously coded block and / or the preset prediction mode information. The prediction mode information includes or only includes the mode type, the prediction mode, and / or any subset of the above. In one embodiment, the candidate list here is aligned with the MPM list or any merge list for conventional intra modes. For example -1, the prediction mode information refers to the mode type equal to intra and the prediction mode equal to DC, PL, any directional mode, any intra prediction mode in the related art (i.e., WAIP or MIP), any intra prediction scheme in the standard (i.e., EIP including EIP normal mode, EIP merge mode, any subset / extension of the above modes), or a combination thereof. For example -2, the prediction mode information refers to the mode type equal to intra TMP and the prediction mode indicating a block vector obtained by using a template matching search within a predefined area. For example -3, the prediction mode information refers to the mode type equal to IBC and the prediction mode indicating a block vector. When (1) a previously coded block is available and (2) the mode type and / or prediction mode and / or any predefined prediction mode information of the previously coded block is supported by the mode using the proposed combined prediction, the prediction mode information of the previously coded block is valid and can be inserted as a candidate into the candidate list herein. In one embodiment, Example-1, Example-2, and Example-3 are all supported by the mode using the proposed combined prediction. In another embodiment, any subset of Example-1, Example-2, and Example-3 is supported by the mode using the proposed combined prediction. In another embodiment, the candidate list can be any subset of the MPM list or any merge list of conventional intra mode. This means that previously coded blocks that were examined when constructing the MPM list or any merge list of conventional intra mode will be examined when constructing the candidate list. In another embodiment, the candidate list can be a merge candidate list containing candidates with prediction mode information, established for the current block. As with conventional inter merge mode, the merge candidate list can include spatially neighboring candidates, non-neighboring candidates, historical candidates, temporal candidates, preset candidates, or any subset of the above. Spatially neighboring candidates are selected from the current block's adjacent neighboring blocks, where the adjacent neighboring blocks can be the same as the five spatially neighboring blocks of conventional inter merge mode, or any subset of the current block's adjacent neighboring blocks. Non-neighboring candidates are selected from a search range surrounding (but not adjacent to) the current block. The search range can be the same as the search range for non-neighboring candidates in conventional inter merge mode, or a different search range for the current block. The historical candidates are selected from a history-based buffer array. In the history-based buffer array, prediction mode information for each valid previously coded block is stored, where a valid previously coded block is any block containing information about a supported prediction mode (e.g., (1) information related to the mode type, such as intra-frame TMP, IBC, or a combination thereof, and / or information related to the prediction mode, such as a block vector and / or (2) information related to the mode type being intra-frame, and / or the prediction mode being EIP normal mode, EIP merge mode, or a combination thereof). Just like the historical candidates in the merge list for the regular inter merge mode, if the buffer array is full, the first stored information may be removed to include information related to the latest valid coded block. The buffer array is cleared (or flushed) at the beginning or end of a predefined unit. The predefined unit can be a CTU, a CTU row, a slice, a tile, a picture, or any predefined region. In a sub-embodiment, the merge candidate list refers only to the history buffer array. That is, only historical candidates are included and / or candidates from distant neighboring regions are not used. Temporal candidates are obtained from prediction mode information stored in one or more predefined previously coded pictures, if the stored information is valid. In one sub-embodiment, temporal candidates are only available for inter slices that use the predefined previously coded pictures as the same position picture as in the conventional inter merging process. In one embodiment, the preset candidates are candidates that contain preset (valid) prediction mode information and / or are derived based on candidates that have been placed in the merge candidate list. In another embodiment, the merge candidate list here is aligned with the merge candidate list of the conventional inter-frame merge mode, or can be any subset of the merge candidate list. In another embodiment, full or partial pruning is used to avoid duplication of prediction mode information in the list. Before adding a candidate to the list, all or a subset of the prediction mode information of the candidate to be added is checked with the corresponding prediction mode information of all or any subset of candidates already in the list. All prediction mode information refers to all stored prediction mode information (such as mode type and / or prediction mode). The subset of prediction mode information can be only mode type or only prediction mode or any subset predefined from all. II. Generate pattern type predictions The selection is based on implicitly or explicitly selecting one or more (K) candidates from the candidate list of Section I. After selection, the selected one or more candidates are used to generate a mode type prediction for the current block. If the mode types in Examples 1, 2, and / or 3 are all supported and K > 1, the mode type prediction can be Intra (i.e., EIP) + Intra (i.e., EIP), and / or Intra (i.e., EIP) + Intra TMP, and / or Intra (i.e., EIP) + IBC. If the mode types in Examples 1, 2, and / or 3 are all supported and K = 1, the mode type prediction can be Intra (i.e., EIP), and / or Intra TMP, and / or IBC. In one embodiment, the selection is based on a template cost, such as TIMD. This means that each candidate in the list generates a prediction based on a template, such as a prediction template, to obtain a template prediction. The template cost of each candidate is measured based on the distortion between the predicted template and the reconstructed template. The candidate list is reordered based on the cost, and / or promising candidates with the lowest cost are ranked. In another embodiment, the selection uses the top K candidates in the candidate list. When K is 1, a single selected candidate is used to generate a pattern type prediction for the current block. When K is greater than 1, multiple hypothetical predictions generated by each selected candidate are used to form a pattern type prediction using predefined weights. In one embodiment, the predefined weights follow the cost. Predictions generated by candidates with higher costs are given smaller weights. Generate inter-frame prediction The inter prediction of CIIP may be generated by using one or more inter motion candidates. In one embodiment, inter-frame motion candidates may refer to sub-block-based inter-frame motion candidates, and inter-frame prediction may refer to sub-block-based prediction. In particular, one or more sub-block-based merge candidates may be used to generate the inter portion of CIIP. This is referred to as sub-block-based CIIP. An explicit signal (i.e., an additional flag, such as a sub-block-based CIIP flag) may be used to indicate whether sub-block-based CIIP is used for the current block. In one sub-embodiment, sub-block-based motion candidates are selected from a sub-block merge candidate list that is unified with the original sub-block merge candidates for affine and SbTMVP. One (or more) indices indicate that one or more specific candidates are signaled in the sub-block-based merge candidate list, and / or the top N (i.e., 1 or any predefined positive integer less than or equal to the list size) reordered candidates in the sub-block-based merge candidate list are selected. In one sub-embodiment, sub-block-based CIIP may support CIIP-TM. That is, the sub-block-based merge candidates are refined through TM. In one sub-embodiment, sub-block-based CIIP may support CIIP-PDPC. In this case, when using subblock-based CIIP, the intra portion can be CIIP-PDPC. In one sub-embodiment, a prohibition setting is predefined for subblock-based CIIP. The prohibition setting is that under predefined circumstances, the inter portion of CIIP cannot be generated using any subblock-based inter motion candidates, and / or the signaling of subblock-based CIIP can be bypassed and inferred to be disabled. In one case, CIIP-TM and subblock-based CIIP cannot be used simultaneously. If the CIIP-TM flag is true, subblock-based CIIP is prohibited. (That is, when the CIIP flag is true and the CIIP-TM flag is false, subblock-based CIIP can be allowed.) In another case, subblock-based CIIP and CIIP-PDPC cannot be used simultaneously. If the subblock-based CIIP flag is true, TIMD is used by default to generate intra signals, without signaling the CIIP-PDPC flag. In another embodiment, inter-frame prediction may refer to GPM prediction, generated as GPM or any GPM variant, such as GPM-TM, GPM-MMVD, GPM-Intra, and / or any combination or extension of the aforementioned variants. In particular, a GPM merge candidate pair may be used to generate the inter portion of the CIIP. This scenario is referred to as GPM-based CIIP. An explicit signal (i.e., an additional flag, such as a GPM-based CIIP flag) may be used to indicate whether GPM-based CIIP is to be used for the current block. In one sub-embodiment, the GPM-based motion candidate pair is selected from a GPM merge candidate list that is unified with the original GPM merge candidates of the GPM. Two indices are signaled, respectively, to indicate two specific candidates in the GPM merge candidate list, and / or the first candidate pair reordered in the GPM merge candidate list is selected. In one sub-embodiment, the GPM-based CIIP may support CIIP-TM. In this case, at least one or both GPM-based merge candidates undergo TM refinement. In one sub-embodiment, the GPM-based CIIP may support CIIP-PDPC. In this case, when GPM-based CIIP is used, the intra-frame portion can be CIIP-PDPC. In one sub-embodiment, a prohibition setting is predefined for GPM-based CIIP. The prohibition setting is that under predefined circumstances, the inter-frame portion of CIIP cannot be generated using GPM, and / or the signaling of GPM-based CIIP can be bypassed and inferred to be disabled. In one case, CIIP-TM and GPM-based CIIP cannot be used at the same time. If the CIIP-TM flag is true, the use of GPM-based CIIP is prohibited. (That is, when the CIIP flag is true and the CIIP-TM flag is false, GPM-based CIIP can be allowed.) In another case, GPM-based CIIP and CIIP-PDPC cannot be used at the same time. If the GPM-based CIIP flag is true, TIMD is used by default to generate intra-frame signals without signaling the CIIP-PDPC flag. In another embodiment, the inter prediction herein can be replaced by any predefined block vector prediction if the proposed method is applied to a target mode that refers to IBC or intra TMP, and / or the inter prediction herein can be a combination of more than one hypothetical inter prediction and / or block vector prediction. In addition, the inter motion candidate herein can refer to {motion vector, reference index} and / or {block vector}. IV. Combining predictions and signals After determining the mode type prediction in Section II and the inter prediction in Section III, a combined prediction is formed by using a weighted average. In one embodiment, if the mode using combined prediction is CIIP, the weighted average follows the weighting used for combining with inter-frame prediction. For example, if the intra-frame prediction mode is angular mode, the weights (wIntra, wInter) are derived as shown in Table 4. For nearly horizontal modes (e.g., 2 <= angular mode index < 34), the current block is divided into sub-blocks vertically; for nearly vertical modes (e.g., 34 <= angular mode index <= 66), the current block is divided into sub-blocks horizontally. Table 4. Weights for combined prediction In another embodiment, the weighted average follows the template cost, such as TIMD. The prediction hypothesis (whether mode type prediction or inter prediction) with a smaller template cost has a larger weight. In another embodiment, the weights in the weighted average are derived using a regression-based derivation method. The proposed weight setting is to determine the weights, that is, to estimate the relationship between the combined result (e.g., combined prediction) and the reconstructed samples on the reference area through a predefined regression method (e.g., minimizing distortion). A weight (which may refer to a model parameter) is then generated based on the regression method, and then the weight is applied to derive the target (prediction) sample in the current block. In one embodiment, the predefined regression method can be a linear minimum mean square error (LMMSE) method for a cross-component linear model (CCLM), or any method that is consistent with the regression method used for CCLM. In another embodiment, the predefined regression method can be an LDL decomposition method for CCCM, or any method that is consistent with the regression method used for CCCM. In another embodiment, the predefined regression method can be a Gaussian elimination method. In one sub-embodiment, the reference region of the current block is the spatially neighboring region of the current block, which may include only the spatially neighboring regions of the current block, only the spatially non-adjacent neighboring regions of the current block, both the spatially adjacent and non-adjacent neighboring regions of the current block, and / or any predefined coding region. The reference region of the current block may vary with the block width, block height, block area, signal mode information of the current block, signal mode information of any neighboring blocks and / or any coding blocks, and / or syntax elements at the block, CTU, SPS, PPS, picture, slice, tile and / or sequence level. The spatially neighboring region 2310 of the current block includes an upper reference region 2320, a left reference region 2330, an upper left reference region 2340, and / or any subset of the above, as shown in FIG. 23 . The size of the above reference region is A W xA H , the size of the left reference area is L W xL H , the size of the upper left reference area is AL W xD H , where – A W = block width (W) of the current block, k*W, W + block height (H) of the current block, any predefined value, or any adaptive value based on the block position, block width, block height and / or block area of ​​the current block. – A H or AL H = H, any predefined value (e.g. 1, 2, 4, ...), or any adaptive value based on the block position, block width, block height and / or block area of ​​the current block. – L W or AL W = W, any predefined value (e.g. 1, 2, 4, ...), or any adaptive value based on the block position, block width, block height and / or block area of ​​the current block. – L H = H, k*H, H + W, any predefined value, or any adaptive value based on the block position, block width, block height and / or block area of ​​the current block. In this example, the reference region is spatially adjacent to the current block. In other cases, the reference region may be spatially adjacent to the co-located block of the current block. In one embodiment, the proposed combined prediction replaces the combined prediction in the current CIIP design. This means that when the CIIP enable flag indicates that CIIP should be applied to the current block, the proposed method is inferred to generate the final prediction for CIIP. In another embodiment, an additional flag is signaled to indicate whether the proposed combined prediction should be used after the target mode for encoding the current block has been determined. For example, the target mode is CIIP and / or the CIIP enable flag indicates that CIIP should be applied to the current block. In another embodiment, the inter-frame prediction used in the proposed combined prediction can be merge prediction, AMVP (Advanced MVP) prediction, or merge prediction plus AMVP prediction. As another example, the target mode can be any mode mentioned in the related art, such as CIIP, GPM, any GPM variant, IBC, and / or intra TMP. In another embodiment, after deciding to use the proposed combined prediction for the current block, different weighting methods can be considered as different selectable modes for the combined prediction mode. An implicit rule and / or an explicit mode index indicates the selectable mode for selecting the combined prediction mode for the current block. For example, option 1 uses fixed weight combining and / or option 2 uses regression weight combining. In one embodiment, the method proposed in this disclosure can be enabled and / or disabled based on implicit rules (e.g., block width, height, or area) or explicit rules (e.g., syntax at the block, tile, slice, picture, SPS, or PPS level). For example, the method is applied when the block area is less than / greater than a threshold. In another embodiment, the method uses a DIMD process to derive the intra prediction mode for the current block based on the used prediction samples, such as all or any subset of combined prediction samples and / or intermediate prediction samples (i.e., any prediction hypothesis that will be used to form the combined prediction) before combining. In this case, horizontal and vertical gradients are calculated for each used prediction sample to create a Histogram of Gradients (HoG). The intra prediction mode corresponding to the maximum histogram count is then used to determine the transform set for the current block and / or the encoding and decoding process for subsequent coded blocks. In one example, the transform process for the current block can be LFNST, NSPT, and / or MTS. Note that in the present invention, LFNST / NSPT / MTS can be used only for the transform process of intra blocks, only for the transform process of inter blocks, or only for the transform process of the third type (non-intra and non-inter) blocks, and / or for any subset or combination of the aforementioned transform processes. In another example, the encoding and decoding process of a subsequent codec block can refer to the MPM (or merge list) construction and / or any inheritance scheme of the subsequent codec block. When a subsequent codec block references (or inherits) the intra prediction mode of the current block, it can refer to the derived intra prediction mode of the current block. In another example, if the current block is a co-located chroma block with a subsequent codec chroma block, then if the current block is a luminance block and the subsequent codec block is a chroma block, the chroma DM of the subsequent codec block can use the derived intra prediction mode of the current block. The term "block" in the present invention may refer to a TU / TB, a CU / CB, a PU / PB, a predefined area, or a CTU / CTB. Any combination of methods proposed in the present invention can be applied. Any of the aforementioned methods proposed in the present invention, wherein the inter-frame prediction portion of the combined prediction is extended to reference sub-block-based inter-frame prediction or prediction associated with the Geometric Partitioning Mode (GPM), can be implemented in an encoder and / or decoder. For example, any of the proposed methods can be implemented in an inter-frame / intra-frame / IBC / prediction / transform module of an encoder and / or in an inter-frame / intra-frame / IBC / prediction / transform module of a decoder. Alternatively, any of the proposed methods can be implemented as circuitry connected to the inter-frame / intra-frame / IBC / prediction / transform module of an encoder and / or the inter-frame / intra-frame / IBC / prediction / transform module of a decoder to provide information required by the inter-frame / intra-frame / IBC / prediction / transform module. FIG24 illustrates a flowchart of an exemplary video encoding and decoding system, according to one embodiment of the present invention, that derives a combined prediction using a target inter-frame prediction directed to a sub-block-based inter-frame prediction or a prediction associated with a Geometric Partitioning Mode (GPM). The steps shown in the flowchart may also be implemented in hardware, such as by one or more electronic devices or processors configured to perform the steps in the flowchart. According to the method, in step 2410, input data associated with a current block is received, where the input data includes pixel data to be encoded at an encoder or coded data associated with the current block to be decoded at a decoder. In step 2420, a combined prediction is determined, including a target inter-frame prediction and at least one target intra-frame prediction or neighboring reconstructed samples, where the target inter-frame prediction is a sub-block-based inter-frame prediction or a prediction associated with a Geometric Partitioning Mode (GPM). In step 2430, the current block is encoded or decoded using the prediction data including the combined prediction. The flowchart shown is intended to illustrate an example of video encoding and decoding according to the present invention. A skilled person may modify each step, rearrange steps, split steps, or combine steps to practice the present invention without departing from the spirit of the present invention. In this disclosure, specific syntax and semantics are used to illustrate examples of implementing embodiments of the present invention. A skilled person may practice the present invention by replacing syntax and semantics with equivalent syntax and semantics without departing from the spirit of the present invention. The foregoing description is intended to enable a person of ordinary skill in the art to practice the present invention in the context of a particular application and its requirements. It will be apparent to those skilled in the art that various modifications will be apparent to those skilled in the art with respect to the described embodiments, and the general principles defined herein may be applied to other embodiments. Therefore, the present invention is not intended to be limited to the specific embodiments shown and described, but rather to be given the widest scope consistent with the principles and novel features disclosed herein. In the foregoing detailed description, various specific details have been presented in order to provide a thorough understanding of the present invention. However, it will be understood by those skilled in the art that the present invention may be practiced. The embodiments of the present invention described above can be implemented in various hardware, software code, or a combination of both. For example, an embodiment of the present invention may be implemented as one or more circuits integrated into a video compression chip, or as program code integrated into video compression software, to perform the processes described herein. An embodiment of the present invention may also be implemented as program code to be executed on a digital signal processor (DSP) to perform the processes described herein. The present invention may also involve multiple functions being performed by a computer processor, digital signal processor, microprocessor, or field programmable gate array (FPGA). These processors can be configured to perform specific tasks according to the invention by executing machine-readable software code or firmware code that defines the specific methods embodied in the invention. The software code or firmware code can be developed in different programming languages ​​and in different formats or styles. The software code can also be compiled for different target platforms. However, different code formats, styles, and languages ​​for the software code, as well as other means of configuring the code to perform tasks according to the invention, do not deviate from the spirit and scope of the invention. The invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described examples are to be considered in all respects only as illustrative and not restrictive. The scope of the invention is, therefore, to be indicated by the appended claims rather than by the foregoing description. All changes coming within the meaning and scope are intended to be embraced therein. 110: Intra-frame prediction 112: Inter-frame prediction 114: Switch 116: Adder 118: Transform (T) 120: Quantization (Q) 122: Entropy encoder 130: Loop filter 124: Inverse quantization (IQ) 126: Inverse transform (IT) 128: Reconstruction (REC) 136: Prediction data 134: Reference picture buffer 140: Entropy decoder 150: Intra prediction 152: Motion compensation 310, 510, 610, 1810: Current block 320: Template 330: Reconstructed area 340: Unavailable area 350: Window 352, 360, 362, 352: Pixels 420, 422: Reference samples 412, 414: Prediction samples 520: Prediction block 530: Search range 540: Current frame 550: Reconstructed portion 512: Current template 522: Most similar template 610: Square 620: Horizontal strip 630: Vertical strip 910, 1010, 1610: Current CU 1020: Co-located CU 1030, 1040: motion vector 1110: L0 reference block 1120: L1 reference block 1112, 1122: starting point 1410, 1210, 1510, 2012: current block 1420: CU 1910: current picture 1912: current CU 1914: Upper neighboring block 1916: Left neighboring block 1920, 2020, 2030, 2120: Reference image 1924, 1926, 2112: Block 1930: Initial motion 1928: Position 1922: Search range 2012, 2022, 2032: Block 2010, 2110: Current image 2014, 2016, 2024, 2026, 2034, 2036: Template 2022, 2032: Reference block 2040, 2042, 2044, 2060, 2062, 2064: Motion vector 2322: Co-located block 2320: Upper reference area 2330: Left reference area 2340: Upper left reference area 2410-2430: Steps Figure 1A shows an exemplary adaptive Inter / Intra video codec system including loop processing. Figure 1B shows the decoder corresponding to the encoder in Figure 1A. Figures 2A-2B show that to support wide angular prediction directions, blocks with much greater width than height (Figure 2A) and blocks with much greater height than width (Figure 2B) have a top reference length of 2W+1 and a left reference length of 2H+1. Figure 3A shows an example of a selected template for the current block, where the template includes T rows above the current block and T columns to the left of the current block. Figure 3B shows an example where T=3, with the HoG (Histogram of Gradients) calculated for pixels in the middle row and middle column. Figure 3C shows an example of the amplitude (ampl) of the angular intra prediction mode. Figure 4 shows an example of the Template-Based Intra Mode Derivation (TIMD) mode, where TIMD implicitly uses neighboring templates to derive the intra prediction mode for a CU at both the encoder and decoder. Figure 5 shows an example of Template Matching Prediction (TMP). Figure 6 shows examples of three filter shapes in the EIP process, which has fifteen inputs and produces one output. Figures 7A-7C show three types of reconstruction regions used to derive filter coefficients for the EIP (Figure 7A: top left region, Figure 7B: top region, and Figure 7C: left region). Figure 8 shows an example of the scan order used to generate predictions for different positions of the current block in diagonal order. Figure 9 shows neighboring blocks used to derive spatial merge candidates. Figure 10 shows an example of temporal candidate derivation, where a scaled motion vector is derived based on the POC (Picture Order Count) distance. Figure 11 shows the horizontal and vertical offsets from the starting MV of the MVD (MMVD) depending on the merge mode. Figure 12A shows an example of describing the affine motion field of a block, described by the motion information (4 parameters) of two control point motion vectors. Figure 12B shows an example of describing the affine motion field of a block, described by the motion information (6 parameters) of three control point motion vectors. Figure 13 shows an example of block-based affine transformation prediction, where the motion vector of each 4×4 luma subblock is derived from the control point MV. Figure 14 shows an example of deriving inherited affine candidates based on the control point MVs of neighboring blocks. Figure 15 shows an example of constructing affine candidates by combining the translational motion information of each control point from both spatial and temporal neighbors. Figure 16 shows an example of deriving weight values ​​for combined inter- and intra-frame prediction (CIIP) based on the codec modes of the top and left neighboring blocks.Figure 17-C shows examples of available IPM candidates: parallel angle mode to GPM block boundaries (parallel mode, Figure 17A), vertical angle mode to GPM block boundaries (vertical mode, Figure 17B), and planar mode (Figure 17C), as well as intra and intra prediction (Figure 17D). Figure 18 shows exemplary patterns for spatial merge candidates. Figure 19 shows an example of refining template matching of an initial MV by searching a range around the initial MV. Figure 20 shows examples of templates for a current block and corresponding reference blocks to measure the matching cost associated with merge candidates. Figure 21 shows an example of templates and reference samples for a block with sub-block motion, using motion information of the current block's sub-blocks. Figure 22 shows an example of CIIP-PDPC according to an embodiment of the present invention. Figure 23 shows examples of spatially neighboring regions of a current block or a corresponding block, including an upper reference region, a left reference region, and an upper-left reference region, used to derive blending weights for combined prediction. FIG. 24 illustrates a flow chart of an exemplary video coding system for deriving a combined prediction using target inter prediction, where the target inter prediction refers to sub-block based inter prediction or prediction associated with a geometric partitioning mode (GPM), according to an embodiment of the present invention. 2410-2430: Steps

Claims

1. A prediction method for color images, the method comprising: Receive input data associated with the current block, wherein the input data includes pixel data to be encoded at the encoder end or encoded data associated with the current block to be decoded at the decoder end; determine a combined prediction including target inter-frame prediction and at least one target intra-frame prediction or neighbor reconstruction sample, wherein the target inter-frame prediction refers to sub-block-based inter-frame prediction or prediction associated with geometric segmentation mode (GPM); and encode or decode the current block using the prediction data including the combined prediction.

2. The method as described in claim 1, wherein an explicit signal is used to indicate whether to use sub-block-based combined inter-frame and intra-frame prediction (CIIP) for the current block or to indicate whether to use GPM-based CIIP for the current block.

3. The method as described in claim 1, wherein the sub-block-based inter-frame prediction is generated using one or more sub-block-based motion candidates from a sub-block merging candidate list.

4. The method as described in claim 3, wherein the sub-block merging candidate list is unified with the sub-block merging candidates of the affine and sub-block-based temporal motion vector prediction (SbTMVP) modes.

5. The method as described in request item 1, wherein sub-block-based CIIP is supported for CIIP-TM (template matching), CIIP-PDPC (position-dependent intra-prediction combination), or both.

6. The method as described in claim 1, wherein the GPM-related prediction is generated using GPM-TM, GPM-MMVD (Merged Mode and Motion Vector Difference), GPM-Intraframe, or any combination or extension thereof.

7. The method as described in claim 1, wherein the GPM-based CIIP supports CIIP-TM, CIIP-PDPC, or both.

8. The method of claim 1, wherein the combined prediction is formed by mixing the neighboring reconstructed samples described herein with the target inter-frame prediction.

9. The method of claim 1, wherein the target inter-frame prediction is generated using one or more template-matched refined inter-frame motion candidates.

10. The method of claim 9, wherein the one or more inter-frame motion candidates correspond to one or more merged candidates after template matching.

11. The method of claim 1, wherein the target intra-frame prediction and the target inter-frame prediction are mixed using hybrid weights, and multiple mixing methods with different weights are considered as multiple optional modes of combined prediction modes.

12. The method as described in request item 1, wherein one or more intra-frame modes are determined using a histogram analysis process.

13. The method as described in claim 12, wherein the one or more intra-frame modes correspond to the maximum histogram count for determining the transform set of the current block and / or the encoding / decoding process of subsequent codec blocks.

14. The method as described in claim 13, wherein the transformation process of the current block corresponds to LFNST (low-frequency non-separated transform), NSPT (non-separated master transform), MTS (multiple transform selection), or a combination thereof.

15. An apparatus for video encoding / decoding, the apparatus comprising one or more electronic devices or processors configured to: receive input data associated with a current block, wherein the input data includes pixel data to be encoded at the encoder side or encoded data associated with the current block to be decoded at the decoder side; determine a combined prediction including a target inter-frame prediction and at least one target intra-frame prediction and a neighboring reconstructed sample, wherein the target inter-frame prediction refers to a sub-block-based inter-frame prediction or a prediction associated with GPM (Geometric Partition Mode); and encode or decode the current block using the prediction data including the combined prediction.

Images