Method and apparatus for combining prediction mode and extrapolated intra prediction for video coding

Abstract

A method and apparatus for video coding are disclosed. According to the method, input data related to a current block is received, where the input data includes pixel data to be encoded at an encoder side or encoded data related to the current block to be decoded at a decoder side. A mode type prediction is determined, where the mode type prediction is generated by using target prediction mode information from candidate prediction mode information, where the candidate prediction mode information includes filtering-based intra prediction information. A second prediction generated by an interaction mode or an IBC (Intra Block Copy) mode is determined. A combined prediction is determined using the mode type prediction and the second prediction and a hybrid weight. And encoding or decoding the current block by using the combined prediction.

Description

[0001] Cross-references This invention is a non-provisional application and claims priority to U.S. Provisional Patent Application No. 63 / 591,792, filed October 20, 2023. This U.S. Provisional Patent Application is incorporated herein by reference in its entirety. Technical Field

[0002] This invention relates to video encoding and decoding systems. In particular, this invention relates to deriving and using a combined prediction by mixing a mode type prediction with inter-frame / intra-frame block copy (IBC) prediction with mixed weights in a video encoding and decoding system. Background Technology

[0003] Versatile Video Coding (VVC) is the latest international video coding standard developed by the Joint Video Experts Group (JVET) of the International Telecommunication Union-Telecommunication Standardization Sector (ITU-T) Video Coding Experts Group (VCEG) and the International Organization for Standardization / International Electrotechnical Commission Moving Picture Experts Group (ISO / IEC MPEG). This standard has been published as an international standard: ISO / IEC 23090-3:2021, Information technology – Encoded immersive media representation – Part 3: Versatile Video Coding, published in February 2021. VVC was developed based on its predecessor, High-Efficiency Video Coding (HEVC), by adding more coding and decoding tools to improve coding and decoding efficiency and handle various types of video sources, including three-dimensional (3D) video signals.

[0004] Figure 1A An exemplary adaptive inter-frame / intra-frame video coding system is described, which includes loop processing. For intra-frame prediction 110, the prediction data is derived from previously encoded video data in the current frame. For inter-frame prediction 112, motion estimation (ME) is performed on the encoder side, and motion compensation (MC) is performed based on the results of ME to provide prediction data derived from other frames and motion data. Switch 114 selects either intra-frame prediction 110 or inter-frame prediction 112 and provides the selected prediction data to adder 116 to form a prediction error, also known as a residual. The prediction error is then processed by transform (T) 118, followed by quantization (Q) 120. The residuals of the transform and quantization are then encoded by entropy encoder 122 to be included in the video bitstream corresponding to the compressed video data. The bitstream associated with the transform coefficients is then packaged together with side information such as motion and encoding / decoding modes associated with intra-frame and inter-frame prediction, and parameters associated with loop filters applied to the underlying image regions. Side information associated with intra-frame prediction 110, inter-frame prediction 112, and loop filters 130, such as Figure 1A As shown, the data is provided to the entropy encoder 122. When using the cross-prediction mode, the reference image or picture must also be reconstructed at the encoder. Therefore, the residuals from the transform and quantization are processed by inverse quantization (IQ) 124 and inverse transform (IT) 126 to recover the residuals. The residuals are then added back to the prediction data 136 to reconstruct the video data at reconstruction (REC) 128. The reconstructed video data can be stored in the reference image buffer 134 and used to predict other frames.

[0005] like Figure 1A As shown, the incoming video data undergoes a series of processes in the encoding system. The reconstructed video data from REC 128 may be subject to various degradations due to these processes. Therefore, a loop filter 130 is typically applied to the reconstructed video data to improve video quality before storing it in the reference image buffer 134. For example, a deblocking filter (DF), sample adaptive offset (SAO), and adaptive loop filter (ALF) may be used. It may be necessary to incorporate loop filter information into the bitstream so that the decoder can correctly recover the required information. Therefore, loop filter information is also provided to the entropy encoder 122 for inclusion in the bitstream. Figure 1A In the process, the loop filter 130 is applied to the reconstructed video, and then the reconstructed samples are stored in the reference image buffer 134. Figure 1A The system described herein is intended to illustrate an exemplary architecture of a typical video encoder. It may correspond to a High Efficiency Video Codec (HEVC) system, VP8, VP9, ​​H.264, or VVC.

[0006] like Figure 1B The decoder shown can use the same or partially the same function blocks as the encoder, except for transform 118 and quantization 120, since the decoder only needs inverse quantization 124 and inverse transform 126. Instead of using entropy encoder 122, the decoder uses entropy decoder 140 to decode the video bitstream, converting it into quantized transform coefficients and the required encoding / decoding information (e.g., ILPF information, intra-frame prediction information, and inter-frame prediction information). Intra-frame prediction 150 on the decoder side does not require mode search. Instead, the decoder only needs to generate intra-frame predictions based on the intra-frame prediction information received from entropy decoder 140. Furthermore, for inter-frame prediction, the decoder only needs to perform motion compensation (MC 152) based on the inter-frame prediction information received from entropy decoder 140, without performing motion estimation.

[0007] Intra-mode encoding and decoding with 67 intra-prediction modes In order to capture arbitrary edge directions presented in natural video, the number of directional intra-frame modes in VVC has been expanded from 33 used in HEVC to 65.

[0008] In VVC, several conventional angular intra-prediction modes are adaptively replaced with non-square block wide-angle intra-prediction modes.

[0009] Intra-frame mode encoding and decoding To keep the generation of the Most Probable Mode (MPM) list low in complexity, an intra-mode encoding / decoding method using six MPMs (or primary MPMs) is employed, taking into account two available neighboring intra-modes. The following three aspects are considered when constructing the MPM list: Default intra-frame mode Nearest intra-frame mode Derivative Intra-Frame Mode Wide-angle intra-frame prediction (WAIP) for non-rectangular blocks Traditional angular intra-prediction directions are defined from 45 degrees clockwise to -135 degrees. In VVC, several conventional angular intra-prediction modes are adaptively replaced with non-square block wide-angle intra-prediction modes.

[0010] To support these predicted directions, such as Figure 2A and Figure 2B As shown, a top reference with a length of 2W+1 and a left reference with a length of 2H+1 are defined.

[0011] Decoder-side Intra-Modal Derivation (DIMD) When DIMD is applied, two intra-frame modes are derived from the reconstructed neighboring samples (templates). These two predictors are combined with the planar mode predictor, with weights derived from gradients. The DIMD modes are used as alternative prediction modes and are always checked in high-complexity RDO modes.

[0012] To implicitly derive the intra-prediction mode of the block, texture gradient analysis is performed on both the encoder and decoder sides. This process begins with an empty Histogram of Gradients (HoG) containing 65 entries, corresponding to 65 angular modes. The magnitudes of these entries are determined during texture gradient analysis.

[0013] In the first step, DIMD selects T=3 columns and rows from the left and top of the current block, respectively, as templates. This region is used as a reference for gradient-based intra-prediction mode derivation.

[0014] In the second step, horizontal and vertical Sobel filters are applied to all 3×3 window locations, centered on the pixels of the middle row of the template. At each window location, the Sobel filter calculates the intensity in the purely horizontal and vertical directions, respectively. and Then, the texture angle of the window is calculated as follows: (1) This can be converted into one of 65 intra-prediction modes with different angles. Once the intra-prediction mode index idx for the current window is derived, its magnitude is updated by adding to the entry in HoG[idx]: (2) Figure 3 shows an example of a HoG, which is calculated after applying the above operations to all pixel positions in the template. Figure 3A This illustrates an example of a selected template 320 for a current block 310. Template 320 contains T rows above the current block and T columns to the left of the current block. For intra-frame prediction of the current block, a reconstructed region 330 corresponds to a reconstructed region above and to the left of the current block, while an unavailable region 340 corresponds to an unavailable region below and to the right of the block. Figure 3B The example with T=3 is illustrated, and HoGs are calculated for pixel 360 in the middle row and pixel 362 in the middle column. For example, for pixel 352, a 3x3 window of 350 is used. Figure 3C An example of the amplitude (ampl) calculated based on equation (2) is illustrated for intra-frame prediction of the angle determined by equation (1).

[0015] Once the HoG is computed, the indices with the two highest histogram bars are selected as the two implicitly derived intra-prediction modes (IPMs) of the block, and further combined with the planar mode as the prediction for the DIMD mode. Prediction fusion is applied as a weighted average of the three predictors mentioned above. For this purpose, the planar IPM's weight is fixed at 21 / 64 (approximately 1 / 3). The remaining weight, 43 / 64 (approximately 2 / 3), is then shared between the two HoG IPMs according to the amplitude ratio of their HoG bars. Figure 4 The DIMD process is explained.

[0016] Template-based Intra Mode Derivation (TIMD) Template-based intra-mode derivation (TIMD) implicitly uses the intra-prediction mode of the neighboring template-derived CU at both ends of the encoder and decoder, instead of transmitting the intra-prediction mode signal to the decoder. For example... Figure 5As shown, the predicted samples for the templates (512 and 514) of the current block (current CU) 510 are generated for each candidate mode using the reference samples (520 and 522) of the template. The SATD (Sum of Absolute Transformed Differences) between the predicted samples and the reconstructed samples of the template is calculated as the cost. The intra-prediction mode with the lowest cost is selected as the TIMD mode (similar to a derivative of the DIMD mode) and used for intra-prediction of the CU. The candidate modes may be one of the 67 intra-prediction modes in the VVC, or expanded to 131 intra-prediction modes. Generally, MPMs can provide clues indicating the orientation information of the CU. Therefore, to reduce the intra-prediction mode search space and utilize the characteristics of the CU, intra-prediction modes can be implicitly derived from the list of MPMs.

[0017] For each intra-prediction mode in the MPMs, the SATD between the predicted and reconstructed samples of the template is calculated. The top two intra-prediction modes with the minimum SATD are selected as TIMD modes. These two TIMD modes are weighted and fused after applying the PDPC process, and this weighted intra-prediction is used to encode the current CU. Position-dependent intra-prediction combination (PDPC) is included in the derivation of the TIMD modes.

[0018] The costs of the two selected modes are compared with a threshold. In the test, a cost factor of 2 is applied as follows: costMode2 < 2 * costMode1.

[0019] If this condition is true, fusion is applied; otherwise, only mode 1 is used. The weights of the modes are calculated based on their SATD costs as follows: weight1 = costMode2 / (costMode1+ costMode2), weight2 = 1 - weight1.

[0020] Intra Sub-Partitions (ISP) Intra-framing segmentation (ISP) divides the luminance intra-prediction block vertically or horizontally into 2 or 4 sub-segments based on the block size. For each sub-segment, reconstructed samples are obtained by adding residual signals to the prediction signal. Here, residual signals are generated through processes such as entropy decoding, inverse quantization, and inverse transform. Therefore, the reconstructed sample values ​​of each sub-segment can be used to generate the prediction for the next sub-segment, and each sub-segment is processed repeatedly. Furthermore, the first sub-segment to be processed is the one containing the sample at the top left corner of the CU, and then it continues downwards (horizontal segmentation) or to the right (vertical segmentation). Therefore, the reference samples used to generate the sub-segment prediction signal are only located to the left and above the line. All sub-segments share the same intra-framing mode.

[0021] Template Matching Prediction (TMP) In JVET-V0130 and JVET-U0048, Template Matching Prediction (TMP) was disclosed. TMP is a special intra-frame prediction mode that copies the best-predicted block from the reconstructed portion of the current frame, whose L-shaped template matches the current template. This is in Figure 6 The description states that block 610 is a current block and block 620 is a prediction block. For a predefined search range 630, the encoder searches for the template 622 most similar to the current template 612 in the reconstruction portion 650 of the current frame 640, and uses the corresponding block 620 as the prediction block (as a reference block). The encoder then signals the use of this pattern and performs the inverse operation on the decoder side.

[0022] Chroma intra-frame mode encoding and decoding Chroma mode encoding and decoding directly rely on the intra-prediction mode of the corresponding luma block. Because a separate block partitioning structure for luma and chroma components is enabled in the I-slice, one chroma block may correspond to multiple luma blocks. Therefore, for chroma DM mode, the intra-prediction mode of the corresponding luma block covering the center position of the current chroma block is directly inherited.

[0023] An extrapolation filter is based on the Extrapolation Filter-Based IntraPrediction (EIP) mode. In JVET-AF0080 (Luhang Xu et al., “EE2-2.7: An Extrapolation Filter Based on Intra-Prediction Mode,” Joint Video Experts Group (JVET) ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29, 32nd Meeting, Hannover, Germany, October 13–20, 2023, Archive: JVET-AF0080), an extrapolation filter based on intra-prediction is disclosed, in which EIP prediction is performed in three steps. First, the extrapolation filter coefficients are inherited from the neighboring reconstructed region of the current block or from the previous EIP block. Second, the extrapolation process generates the prediction signal from the top left corner to the bottom right corner within the current block. Third, the intra-prediction angle is derived by analyzing the gradient of the prediction block, and then the transformation is performed using the corresponding intra-mode selection cores: MTS (Multiple Transform Selection), NSPT (Non-Separable Primary Transform), and LFNST (Low Frequency Non-Separable Transform).

[0024] The application of EIP is limited to blocks and luminance components no larger than 32x32.

[0025] Obtaining EIP filters Three EIP filter shapes are shown Figure 7 Among them, the three filter shapes correspond to square 710, horizontal bar 720 and vertical bar 730.

[0026] According to JVET-AF0080, there are two methods to obtain the filter coefficients of the current CU. First, the coefficients can be derived from neighboring reconstructed pixels; second, they can also be inherited from previously decoded blocks.

[0027] Derivative EIP coefficients The decoder decodes the relevant syntax elements to determine the type of the selected reconstructed region and the filter shape of the current block. The selected filter moves horizontally or vertically within the selected reconstructed region, moving one pixel at a time to construct the autocorrelation matrix and cross-correlation vector. The method for calculating the coefficients from the autocorrelation matrix and cross-correlation vector is the same as that in the convolutional cross-component model (CCCM).

[0028] Three reconstruction region types are defined, as shown in Figure 8, where the three reconstruction regions correspond to the upper left region ( Figure 8A ), the area above ( Figure 8B ) and the left area ( Figure 8CThe size of the reconstructed region depends on min(block width, block height) and the selected filter shape. For example, when the current block is an 8x16 block and the selected filter shape is 4x4, the upper size (aboveSize) of the reconstructed region is equal to min(8, 16) + 4 – 1 = 11, and the left size (leftSize) of the reconstructed region is equal to min(8, 16) + 4 – 1 = 11.

[0029] EIP filter inheritance The EIP merging pattern is also disclosed in JVET-AF0080. Filter shapes and filter coefficients can be inherited from previously decoded blocks with EIPs or EIP merging patterns. When the current block uses an EIP pattern, the decoder decodes an EIP merging flag to determine whether to use the proposed merging pattern. When the EIP merging flag is true, a merging index is further decoded. The EIP merging list includes spatially adjacent and non-adjacent candidates, temporal candidates, and historical candidates. The constructed EIP merging list can include up to 12 candidates, and is reduced to a maximum of 6 candidates through a reordering process based on SAD cost, which is measured on an L-shaped template with a column width and row height of 1. In the SAD computation, EIP filter predictions for the template region are generated only from reconstructed (neighboring and template) samples, allowing EIP filters to be applied in parallel rather than sequentially.

[0030] Prediction of the current block In EIP mode, predicted values ​​are generated for the current block in diagonal prediction order from the top left to the bottom right. Figure 9 As shown.

[0031] The calculation of the predicted value in JVET-AF0080 is shown below. ; in It is the predicted value at (x, y) in the current block. It is the selected EIP filter Coefficients, with indices ranging from 0 to 14. It is the reconstructed or predicted value used for current location prediction. and These are the position offsets from the current position along the x and y directions, respectively.

[0032] Mapping to LFNST / NSPT / MTS sets In JVET-AF0080, a method was proposed to derive the intra-prediction mode of the current block from EIP prediction samples using the DIMD process. Specifically, the horizontal and vertical gradients are calculated for each prediction sample to construct a histogram of gradients (HoG). The intra-prediction mode corresponding to the maximum histogram count is then used to determine the LFNST, NSPT, or MTS transform set.

[0033] The proposed CU-level syntax EIP-related syntax performs signal passing at the CU level. The following table shows an example of EIP-related syntax.

[0034] Table 1. EIP-related syntax at the CU level Enhanced MTS for intra-frame coding and decoding In the current VVC design, only the DST7 and DCT8 transform cores are used for MTS, which are used for intra-frame and inter-frame encoding and decoding.

[0035] The additional main transforms, including DCT5, DST4, DST1, and the identity transform (IDT), are employed. Furthermore, the MTS set is determined based on the TU size and intra-frame mode information. For blocks predicted via IntraTMP (Intra-Template Matching Prediction), the predicted blocks are processed using the DIMD procedure to derive intra-frame modes for transform selection. Specifically, the horizontal and vertical gradients are calculated for each predicted sample to establish the HoG. The intra-frame prediction mode with the largest histogram amplitude value is then used to select the MTS transform set.

[0036] Secondary Transformation: LFNST Extension and Large Core The LFNST design in VVC is extended as follows: The number of LFNST sets (S) and candidates (C) is expanded to S=35 and C=3, and the LFNST set (lfnstTrSetIdx) for a given intra-mode (predModeIntra) is derived according to the following formula: For predModeIntra<2, lfnstTrSetIdx equals 2; lfnstTrSetIdx = predModeIntra, where predModeIntra is in the range [0,34]. lfnstTrSetIdx = 68 – predModeIntra, where predModeIntra is in the range [35, 66]. Three different cores, LFNST4, LFNST8, and LFNST16, are defined for application in 4xN / Nx4 (N 4) 8xN / Nx8 (N 8) and MxN(M, N) 16).

[0037] The mappings of these sets to intra-prediction modes are shown in Table 2.

[0038] Table 2. Mapping of Intra-Prediction Modes to LFNST Set Indices For blocks predicted using MIP (Matrix-based IntraPrediction) or IntraTMP, the LFNST set index is derived as follows. DIMD is used to derive the intra-mode of the current block based on MIP or IntraTMP prediction samples.

[0039] Non-Separable Primary Transform (NSPT) is used for intra-frame encoding and decoding. The separable DCT-II plus LFNST transform combination is replaced by NSPT for block shapes 4x4, 4x8, 8x4 and 8x8, 4x16, 16x4, 8x16 and 16x8.

[0040] All NSPTs consist of 35 sets and 3 candidates (similar to the current LFNST). The core of the NSPT has the following shape: NSPT4x4: 16x16; NSPT4x8 / NSPT8x4: 32x20; NSPT8x8: 64x32; NSPT4x16 / NSPT16x4: 64x24; NSPT8x16 / NSPT16x8: 128x40; NSPT4x32 / NSPT32x4: 128x20; NSPT8x32 / NSPT32x8: 256x24; Therefore, NSPT4x8 / NSPT8x4, NSPT8x8, NSPT4x16 / NSPT16x4, and NSPT8x16 / NSPT16x8 are used to zero out 12, 32, 40, and 88 coefficients, respectively. For NSPT4x32 / NSPT32x4 and NSPT8x32 / NSPT32x8, the remaining 108 and 232 positions in each transform block are zeroed out, respectively.

[0041] Inter-frame prediction overview According to Section 3.4 of JVET-T2002 (Jianle Chen et al., “Algorithm description of Multifunctional Video Coding and Test Model 11 (VTM 11)”, Joint Video Experts Group (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29, 20th Meeting, via teleconference, 7-16 October 2020, document: JVET-T2002), for each codec unit (CU) for inter-frame prediction, motion parameters include motion vectors, reference picture indexes, and reference picture list usage indexes, as well as additional information required for using new codec features of VVC for inter-frame prediction sample generation. Motion parameters can be labeled in an explicit or implicit manner. When a CU is coded in skip mode, the CU is associated with a prediction unit (PU) and has no significant residual coefficients, no encoded motion vector difference, or reference picture index. A merge mode is specified where the motion parameters of the current CU are obtained from neighboring CUs, including spatial and temporal candidates, as well as additional plans introduced in VVC. The merge mode can be applied to any inter-frame predicted CU, not just the skip mode. An alternative to the merge mode is the explicit transmission of motion parameters, where each CU is explicitly labeled with a motion vector, a reference image index corresponding to each reference image list, a reference image list usage flag, and other necessary information.

[0042] In addition to the inter-frame encoding and decoding capabilities in HEVC, VVC also includes many new and improved inter-frame prediction encoding and decoding tools, as listed below: Extended merge forecast; Merge pattern with MVD (MMVD); Symmetrical MVD (SMVD) signal; Affine motion compensation prediction; Sub-block-based temporal motion vector prediction (SbTMVP); Adaptive Motion Vector Resolution (AMVR); Sports field storage: 1 / 16 th Luminance sample MV storage and 8x8 motion field compression; Double prediction (BCW) with CU-level weights; Bidirectional optical flow (BDOF); Decoder-side motion vector refinement (DMVR); Geometric Partitioning (GPM); Combine inter-frame and intra-frame prediction (CIIP); The following description provides detailed information about the inter-frame prediction method specified in VVC.

[0043] Extended merge forecast In VVC, the merge candidate list is constructed by including the following five types of candidates in order: MVP from the adjacent CU; The MVP of the same CU; A history-based MVP derived from a FIFO table; Pair average MVP; Zero MV.

[0044] The size of the merge list is indicated in the Sequence Parameter Set (SPS) header, and the maximum allowed size of the merge list is 6. For each CU encoded in merge mode, the index of the best merge candidate is encoded using truncated unary binary (TU) encoding. The first bit of the merge index is encoded using context encoding, and the remaining bits are encoded using bypass encoding / decoding.

[0045] The derivation process for each merge candidate category is provided. As in HEVC, VVC also supports parallel derivation of the merge candidate list (or merge candidate list) for all CUs within a given area.

[0046] Space candidate derivation The derivation of spatial merge candidates in VVC is the same as in HEVC, except that the positions of the first two merge candidates are swapped. Figure 10 At the positions shown, a maximum of four merge candidates (B0, A0, B1, and A1) are selected for the current CU 1010. The derived order is B0, A0, B1, A1, and B2. Position B2 is considered only if one or more neighboring CU positions B0, A0, B1, and A1 are unavailable (e.g., belonging to another slice or tile) or if it is intra-frame encoded. After adding the candidate for position A1, the process of adding the remaining candidates is constrained by a redundancy check, which ensures that candidates with the same motion information are excluded from the list, thereby improving encoding and decoding efficiency. To reduce computational complexity, not all possible candidate pairs are considered in the aforementioned redundancy check. Instead, only... Figure 11 Pairs connected by arrows are considered, and a candidate is added to the list only if the corresponding candidate used for redundancy checking does not have the same motion information.

[0047] Time candidate derivatives In this step, only one candidate is added to the list. Specifically, when deriving this time-merging candidate for the current CU 1210, based on... Figure 12 The corresponding CU 1220 in the shown corresponding reference image derives a scaled motion vector. The list of reference images and reference indices used to derive the corresponding CU are explicitly marked in the slice header. The scaled motion vector 1230, a candidate for time merging, is shown below. Figure 12As shown by the dashed lines, the motion vector 1240 from the co-located CU is scaled using the POC (Picture Order Count) distances tb and td, where tb is defined as the POC difference between the current image and the reference image, and td is defined as the POC difference between the reference image and the co-located image. The reference image index for the temporal merging candidate is set to zero.

[0048] The position of the time candidate is Figure 13 The candidate is chosen between C0 and C1 as shown. If the CU at position C0 is unavailable, is intra-coded, or is outside the current CTU line, then position C1 is used. Otherwise, position C0 is used when merging candidates at derived time.

[0049] Historical Merger Candidate Derivatives Following the spatial MVP and TMVP, a history-based MVP (HMVP) merge candidate is added to the merge list. In this method, motion information from previous coded blocks is stored in a table and used as the MVP for the current CU. A table containing multiple HMVP candidates is maintained during encoding and decoding. The table is reset (cleared) when a new CTU row is encountered. Whenever a non-sub-block inter-coded CU is encountered, the associated motion information is added to the last entry of the table as a new HMVP candidate.

[0050] The size S of the HMVP table is set to 6, indicating that a maximum of 5 history-based MVP (HMVP) candidates can be added to the table. When inserting a new motion candidate into the table, a restricted first-in-first-out (FIFO) rule is used. First, a redundancy check is applied to find if there is a duplicate HMVP in the table. If found, the duplicate HMVP is removed from the table, all subsequent HMVP candidates are shifted forward, and the duplicate HMVP is inserted as the last entry in the table.

[0051] HMVP candidates can be used during the candidate list construction process. The latest HMVP candidates in the table are checked sequentially and inserted into the candidate list after the TMVP candidates. Redundancy checks are applied to HMVP candidates to determine spatial or temporal merge candidates.

[0052] To reduce the number of redundant check operations, the following simplification measures were introduced: The last two entries in the table perform redundancy checks on the candidates in space A1 and B1, respectively.

[0053] Once the total number of available merge candidates reaches the maximum allowed merge candidates minus 1, the merge candidate list building process terminates from HMVP.

[0054] Combining inter-frame and intra-frame prediction (CIIP) In VVC, when a codec unit (CU) is encoded in merged mode, if the CU contains at least 64 luma samples (i.e., the CU width multiplied by the CU height is equal to or greater than 64), and if both the CU width and height are less than 128 luma samples, an additional flag is sent to indicate whether Combined Inter-Frame / Intra-Frame Prediction (CIIP) mode is applied to the current CU. As its name suggests, CIIP prediction combines an inter-frame prediction signal with an intra-frame prediction signal. The inter-frame prediction signal in CIIP mode... It is derived using the same inter-frame prediction process applied to the regular merging mode; while the intra-frame prediction signal... It follows the standard intra-frame prediction process and uses a planar mode derived from it. Then, a weighted average is used to combine the intra-frame and inter-frame prediction signals, where the weight value wt is calculated based on the encoding / decoding modes of the current CU 1410's upper and left neighboring blocks (e.g., ...). Figure 14 As shown below: If the upper neighbor is available and is intra-coded, set isIntraTop to 1; otherwise, set isIntraTop to 0. If the left neighbor is available and is intra-coded, set isIntraLeft to 1; otherwise, set isIntraLeft to 0. If (isIntraLeft + isIntraTop) equals 2, then set wt to 3; Otherwise, if (isIntraLeft + isIntraTop) equals 1, then set wt to 2; Otherwise, set wt to 1.

[0055] The CIIP predictions are as follows: .

[0056] Non-adjacent spatial candidates As described in JVET-L0399 (Yu Han et al., “CE4.4.6: Improvement of Merge / Skip Modes,” ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 of the Joint Video Exploration Group (JVET), 12th Meeting: Macau, China, CN, October 3-12, 2018, Document: JVET-L0399), non-adjacent spatial merge candidates are inserted after the TMVP in the regular merge candidate list. The modes of spatial merge candidates are shown in… Figure 15 The distance between non-adjacent spatial candidates and the current codec block is based on the width and height of the current codec block. Line buffer constraints do not apply.

[0057] Intra-block copy (IBC) Intra-Block Copy (IBC) is a tool employed in the HEVC extension of Screen Content Coding (SCC). It is well-known for significantly improving the coding and decoding efficiency of screen content material. Since IBC mode is implemented as a block-level coding mode, the encoder performs block matching (BM) to find the optimal block vector (or motion vector) for each CU. Here, the block vector indicates the displacement from the current block to a reference block that has been reconstructed within the current frame. IBC-coded CUs are treated as a third prediction mode, in addition to intra-frame or inter-frame prediction modes. IBC mode is suitable for CUs with a width and height of 64 luma samples or less.

[0058] Intra-frame template matching Intra-Template Matching Prediction (IntraTMP, similar to or the same as the previously mentioned TMP) is a special intra-prediction mode that copies the best prediction block from the reconstructed portion of the current frame, whose L-shaped template matches the current template. For a predetermined search range, the encoder searches the reconstructed portion of the current frame for the template most similar to the current template and uses the corresponding block as the prediction block. The encoder then sends this mode for use, and the same prediction operation is performed on the decoder side.

[0059] This invention discloses a method and apparatus for deriving combined prediction by mixing mode type prediction with inter-frame / IBC prediction with mixed weights, for use in video encoding and decoding. Summary of the Invention

[0060] A method and apparatus for video encoding and decoding are disclosed. According to the method, input data associated with a current block is received, wherein the input data includes pixel data to be encoded at an encoder end or encoded data associated with the current block to be decoded at a decoder end; a mode type prediction is determined, wherein the mode type prediction is generated using target prediction mode information from candidate prediction mode information, wherein the candidate prediction mode information includes filter-based intra-frame prediction information; a second prediction is determined by an inter-frame mode or intra-frame copy (IBC) mode; a combined prediction is determined using the mode type prediction and the second prediction, along with mixing weights; and the current block is encoded or decoded using the combined prediction.

[0061] In one embodiment, candidate prediction mode information includes mode type, prediction mode, or a combination thereof. In one embodiment, the mode type corresponds to an intra-frame mode, and the prediction mode corresponds to DC, planar, directional, or intra-frame correlation modes. In one embodiment, intra-frame correlation modes include WAIP (Wide Angle Intra Prediction), MIP (Matrix-Based Intra Prediction), Extrapolation Filter-Based Intra Prediction (EIP) regular mode, EIP merging mode, EIP extension, or a combination thereof. In one embodiment, if a previously coded block is available and supports mode type, prediction mode, predefined prediction mode information, or a combination thereof, the prediction mode information of that previously coded block is used as a candidate.

[0062] In one embodiment, candidate prediction mode information is located in a candidate list. In one embodiment, the candidate list corresponds to a merged candidate list. In one embodiment, the candidate list includes one or more spatially adjacent candidates, one or more spatially non-adjacent candidates, one or more historical candidates, one or more temporal candidates, one or more default candidates, or combinations thereof. In one embodiment, one or more preset candidates correspond to candidates containing default prediction mode information and / or derived from one or more existing candidates already placed in the merged candidate list. In one embodiment, when a candidate is inserted into the candidate list, all or partial pruning is used. In one embodiment, before adding a candidate to the candidate list, the candidate prediction mode information of all or a subset of the candidate is checked against the corresponding prediction mode information of all or any subset of existing candidates already in the candidate list.

[0063] In one embodiment, one or more selected candidates are used to generate a pattern type prediction for the current block. In one embodiment, the one or more selected candidates are chosen based on a template cost associated with available candidates. In one embodiment, the template cost is calculated based on the distortion between the reconstruction on the template and each candidate prediction on the template.

[0064] In one embodiment, the combined prediction corresponds to CIIP (Combined Inter-Frame and Intra-Frame Prediction). In one embodiment, a regression-based derivation is used to determine the blending weights. In one embodiment, the regression-based derivation estimates the relationship between the combined prediction of the reference region of the current block and the reconstructed samples of the current block of the reference region to generate blending weights based on the regression-based derivation. In one embodiment, the current block of the reference region varies with block width, block height, block area, current block flag pattern information, neighboring blocks or coded blocks, one or more syntax elements of the flags in the block, CTU, SPS, PPS, picture, slice, tile or sequence level, or a combination thereof.

[0065] Simple Explanation of the Diagram Figure 1AAn exemplary adaptive inter-frame / intra-frame video encoding and decoding system is described, which includes loop processing.

[0066] Figure 1B Explanation Figure 1A The corresponding decoder for the encoder.

[0067] Figure 2A-2B This explains that, in order to support wide-angle prediction directions, blocks with a width much larger than their height ( Figure 2A ) and blocks whose height is much greater than their width ( Figure 2B The length of the top reference is 2W+1 and the length of the left reference is 2H+1.

[0068] Figure 3A This describes an example of the selected template for the current block, where the template includes T rows above the current block and T columns to the left of the current block.

[0069] Figure 3B An example with T=3 is given, and the HoG (gradient histogram) is calculated for the pixels in the middle row and middle column.

[0070] Figure 3C An example of amplitude (ampl) for the angular intra-frame prediction mode is illustrated.

[0071] Figure 4 An example of the blending process is illustrated, in which two intra-frame modes (M1 and M2) are selected based on the indices of the two highest bars in the histogram.

[0072] Figure 5 An example of template-based intra-frame mode derivation (TIMD) mode is illustrated, where TIMD implicitly uses the intra-frame prediction mode of the neighbor template-derived CU in both the encoder and decoder.

[0073] Figure 6 An example of Template Matching Prediction (TMP) is illustrated.

[0074] Figure 7 This describes three filter shapes that use fifteen inputs and generate one output during the EIP process.

[0075] Figures 8A-8C This describes three types of reconstructed regions used for the derived EIP filter coefficients. Figure 8A Top left area Figure 8B The upper area, and Figure 8C (Left side area).

[0076] Figure 9 This example illustrates the scan order in which predictions are generated for different positions of the current block in diagonal order.

[0077] Figure 10This describes the neighboring blocks used to derive VVC space merge candidates.

[0078] Figure 11 This illustrates the possible candidate pairs for considering redundancy checks in VVC.

[0079] Figure 12 An example of temporal candidate derivation is illustrated, where a scaled motion vector is derived from the POC (picture order count) distance.

[0080] Figure 13 This explains the time candidate positions selected between candidates C0 and C1.

[0081] Figure 14 This illustrates an example of deriving weight values ​​for combined inter-frame and intra-frame prediction (CIIP) based on the encoding / decoding modes of the top and left neighboring blocks.

[0082] Figure 15 An exemplary pattern for spatial merging candidates is illustrated.

[0083] Figure 16 This indicates that the spatial proximity region of the current block includes the upper reference region, the left reference region, and the upper left reference region, which are used to derive weight settings.

[0084] Figure 17 A flowchart of an exemplary video codec system is illustrated, which, according to an embodiment of the invention, derives a combined predictor by mixing mode type prediction and inter-frame / IBC prediction, as well as mixing weights.

[0085] Implementation The components of the present invention, as generally described and illustrated in the figures, can be arranged and designed in various different configurations. Therefore, the following more detailed description of embodiments of the systems and methods of the present invention, as illustrated in the figures, is not intended to limit the scope of the invention, as requested, but merely represents selected embodiments of the invention. References to “one embodiment,” “an embodiment,” or similar language throughout this specification mean that a particular feature, structure, or characteristic described in connection with that embodiment may be included in at least one embodiment of the invention. Therefore, the phrases “in one embodiment” or “in one embodiment” appearing throughout this specification do not necessarily refer to the same embodiment.

[0086] Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. However, those skilled in the art will recognize that the invention can be practiced without using one or more specific details, or using other methods, components, etc. In other instances, well-known structures or operations are not shown or described in detail to avoid obscuring aspects of the invention. Embodiments of the invention can be best understood by referring to the drawings, where like parts are designated by like numbers. The following description is merely illustrative and simply illustrates embodiments of certain selected apparatuses and methods consistent with the invention declared herein.

[0087] To improve encoding and decoding performance, this invention discloses a combined prediction method formed by hybrid mode type prediction, inter-frame / IBC prediction, and hybrid weights.

[0088] In this invention, the following aspects are proposed to improve combined prediction. In one embodiment, combined prediction is formed by using mode type (e.g., intra-frame) prediction and inter-frame prediction, along with mixed weights. For example, combined prediction originates from a CIIP mode. In another embodiment, mode type prediction is generated using target prediction mode information, such as filter-based intra-frame prediction information, like information from an extrapolation filter based on intra-frame prediction (EIP). For example, mode type prediction can be generated using an extrapolation filter based on any predefined mode in intra-frame prediction, such as a regular EIP mode (i.e., a non-merged EIP mode), where the EIP mode index is equal to a predefined value, which is fixed at 0 or adapted based on the current block's block position, block width, block height, and / or block area. Another example is that mode type prediction is generated using an extrapolation filter based on any predefined mode in intra-frame prediction, such as an EIP merge mode, where the EIP merge index is equal to a predefined value, where the predefined value is fixed at 0 (i.e., the EIP merge mode preceding a reordered or unreordered EIP merge list) or adapted based on the current block's block position, block width, block height, and / or block area.

[0089] In another embodiment, mode type prediction in combined prediction is generated by one or more sets of predicted mode information proposed according to a template-based mode derivation process, such as the TIMD process. In another embodiment, inter-frame prediction here can be replaced by any predefined block vector prediction if the proposed method is applied to the target mode of IBC or intra-frame TMP. Several aspects are presented below. The first section relates to the proposed candidate list. The second section relates to how mode type prediction is generated based on one or more sets of proposed predicted mode information. The third section relates to combined prediction and signal.

[0090] I. Suggested Candidate List The suggested candidate list may resemble TIMD, generated based on prediction mode information from previously coded blocks and / or default prediction mode information. Prediction mode information includes or only includes mode type, prediction mode, and / or any subset thereof. In one embodiment, the candidate list here is aligned with an MPM list or a merge list of any regular intra-frame modes. For example 1, prediction mode information refers to a mode type equal to intra and a prediction mode equal to DC, PL, any directional mode, any intra-frame prediction mode of related technologies (i.e., WAIP or MIP), any intra-frame prediction scheme of the standard (i.e., including EIP regular mode, EIP merge mode, any subset / extension of the above modes), or a combination thereof. For example 2, prediction mode information refers to a mode type equal to intra TMP and a prediction mode indicating a block vector obtained by template matching search within a predefined region. For example 3, prediction mode information refers to a mode type equal to IBC and a prediction mode indicating a block vector. When (1) a previously encoded block is available, and (2) the mode type and / or prediction mode and / or any predefined prediction mode information of the previously encoded block is supported by a mode using the proposed combined prediction, and the prediction mode information of the previously encoded block is valid, it can be inserted as a candidate into the candidate list here. In one embodiment, Examples 1, 2, and 3 are all supported by a mode using the proposed combined prediction. In another embodiment, any subset of Examples 1, 2, and 3 is supported by a mode using the proposed combined prediction.

[0091] In another embodiment, the candidate list here can be all or any subset of the MPM list or any merge list of a regular intra-frame mode. This means that all or any subset of previously coded blocks examined when constructing the MPM list or any merge list of a regular intra-frame mode will be examined when constructing the candidate list here. In another embodiment, the candidate list here refers to a candidate list containing prediction mode information, established for the current block. As done in a regular inter-frame merge mode, the merge candidate list includes candidates of spatially adjacent, non-adjacent, historical, temporal, preset, or any subset thereof. Spatially adjacent candidates are from the neighboring blocks of the current block, where the neighboring blocks can be the same as the five spatially adjacent blocks of a regular inter-frame merge mode, or any subset of the neighboring blocks of the current block. Non-adjacent candidates are from the search range surrounding (but not adjacent to) the current block. The search range can be the same as the search range of non-adjacent candidates in a regular inter-frame merge mode, or a different search range for the current block.

[0092] Historical candidates are selected from a history-based buffer array. In the history-based buffer array, prediction mode information for each valid previously encoded block is stored, where a valid previously encoded block refers to any block containing information about supported prediction modes (e.g., (1) information related to mode types such as intra-frame TMP, IBC, or combinations thereof, and / or information related to prediction modes such as block vectors, and / or (2) information related to mode types such as intra-frame and / or prediction modes such as EIP regular mode, EIP merge mode, or combinations thereof). Similar to historical candidates in the merge list for regular inter-frame merge mode, if the buffer array is full, the first stored information may be removed to include information related to the latest valid encoded block. The buffer array is cleaned (or emptied) at the beginning or end of a predefined unit. A predefined unit can be a CTU, CTU line, slice, tile, picture, or any predefined region. In one sub-implementation, the merge candidate list refers only to the history buffer array. That is, only historical candidates are included and / or candidates from far-from-adjacent regions are not used. Temporal candidates are obtained from prediction mode information stored in one or more predefined previously encoded pictures, if the stored information is valid. In one sub-implementation, the temporal candidate is only available for inter-frame slices, which may have a predefined previously encoded image as a co-image in a regular inter-frame merging process.

[0093] In one embodiment, the preset candidate is a candidate that includes default (valid) prediction mode information and / or candidates derived from candidates already placed in the merge candidate list. In another embodiment, the merge candidate list here is aligned with the merge candidate list of a regular inter-frame merging mode or can be any subset of that merge candidate list. In yet another embodiment, full or partial pruning is used to avoid duplicate prediction mode information in the list. Before adding a candidate to the list, the prediction mode information of all or a subset of the candidates to be added is checked against the corresponding prediction mode information of all or any subset of candidates already in the list. All prediction mode information refers to all stored prediction mode information (e.g., mode type and / or prediction mode). A subset of prediction mode information can be only mode type, only prediction mode, or any predefined subset of all.

[0094] II. Generation of Pattern Type Prediction Selection depends on implicitly or explicitly choosing one or more (K) candidates from the candidate list in Section I. After selection, the selected one or more candidates are used to generate a mode type prediction for the current block. If the mode types in Examples-1,-2, and / or-3 are all supported and K > 1, the mode type prediction can be intra-frame (i.e., EIP) + intra-frame (i.e., EIP), and / or intra-frame (i.e., EIP) + intra-frame TMP and / or intra-frame (i.e., EIP) + IBC. If the mode types in Examples-1,-2, and / or-3 are all supported and K = 1, the mode type prediction can be intra-frame (i.e., EIP), and / or intra-frame TMP and / or IBC. In one embodiment, selection depends on template cost, such as TIMD. This means that each candidate in the list generates a prediction on a template to obtain a template prediction, such as a predicted template, and the template cost of each candidate is measured based on the distortion between the predicted template and the reconstructed template. The candidate list is reordered according to cost and / or promising candidates with lower costs are recorded. In another embodiment, the top K candidates in the candidate list are selected. When K is 1, the single selected candidate is used to generate a pattern type prediction for the current block. When K is greater than 1, multiple hypothetical predictions generated from a single selected candidate are used to form a pattern type prediction through a predefined weighting. In one embodiment, the predefined weighting follows cost. For predictions generated from candidates with higher costs, the weight of that prediction decreases.

[0095] III. Combining prediction and signals After determining the pattern type forecast in Section II, a combined forecast is formed by using a weighted average.

[0096] In one embodiment, if the combined prediction mode is CIIP, the weighted average follows the weighting used for combining with inter-frame predictions in CIIP.

[0097] In another embodiment, the weighted average follows template cost, such as TIMD. Prediction assumptions with lower template costs (whether mode type prediction or inter-frame prediction) have larger weights.

[0098] In another embodiment, the weights in the weighted average are derived using regression-based derivation. The recommended settings for determining the weights are estimated using a predefined regression method to correlate the combined result (e.g., combined prediction) with reconstructed samples on a reference region of the current block (e.g., minimizing distortion). Weights (potentially model parameters) are then generated based on the regression method, and these weights are applied to derive the target (predicted) samples in the current block. In one embodiment, the predefined regression method could be a linear minimum mean squared error (LMMSE) method for a cross-component linear model (CCLM), or it could be any unified method with the regression method used for CCLM. In another embodiment, the predefined regression method could be an LDL decomposition method for CCLM, or it could be any unified method with the regression method used for CCLM. In yet another embodiment, the predefined regression method could be Gaussian elimination.

[0099] In one sub-implementation, the reference region of the current block is the spatial proximity region of the current block, which may include only the spatially adjacent proximity region of the current block, only the spatially non-adjacent proximity region of the current block, the spatially adjacent and non-adjacent proximity regions of the current block, and / or any predefined coding region. The reference region of the current block can vary with block width, block height, block area, flag pattern information of the current block, flag pattern information of any neighboring blocks and / or any coding block, and / or syntax elements at the block, CTU, SPS, PPS, picture, slice, tile, and / or sequence level. The spatial proximity region 1610 of the current block includes the aforementioned reference region 1620, left reference region 1630, upper-left reference region 1640, and / or any subset thereof, such as... Figure 16 As shown. The size of the reference region mentioned above is A. W xA H The size of the left reference region is L W xL H The size of the upper left reference region is AL. W xAL H ,in: A W = Current block width (W), k*W, W + current block height (H), any predefined value, or any adaptive value depending on the current block's block position, block width, block height, and / or block area.

[0100] A H or AL H = H, any predefined value (e.g., 1, 2, 4, ...), or any adaptive value that depends on the current block's position, width, height, and / or area.

[0101] LW or AL W = W, any predefined value (e.g., 1, 2, 4, ...), or any adaptive value that depends on the current block's position, width, height, and / or area.

[0102] L H = H, k*H, H + W, any predefined value, or any adaptive value depending on the current block's block position, block width, block height, and / or block area.

[0103] In this example, the reference region is adjacent to the current block space. In other cases, the reference region may be adjacent to the co-located block space of the current block.

[0104] In one embodiment, the proposed combined prediction is used to replace the combined prediction in the current CIIP design. This means that the inferred method is used to generate the final prediction for CIIP when the CIIP enable flag indicates that CIIP is applied to the current block. In another embodiment, an additional flag is used to indicate whether the proposed combined prediction is used after the current block has been determined to be encoded according to the target mode. For example, the target mode is CIIP and / or an existing CIIP enable flag indicating that CIIP is applied to the current block. In another embodiment, the inter-frame prediction used in the proposed combined prediction should be merge prediction, AMVP (Advanced MVP) prediction, or merge prediction plus AMVP prediction. For another example, the target mode can be any mode mentioned in the related art, such as CIIP, GPM, any GPM variant, IBC, and / or intra-frame TMP. In another embodiment, after deciding to use the proposed combined prediction for the current block, different weighting methods can be considered as different alternative modes of the combined prediction mode. The alternative modes of the combined prediction for the current block are selected using implicit rules and / or explicit mode indexing indicators. For example, option 1 uses fixed-weighted combined prediction and / or option 2 uses regressive-weighted combined prediction.

[0105] In one embodiment, the proposed method of the present invention can be enabled and / or disabled based on implicit rules (e.g., block width, height, or area) or explicit rules (e.g., syntax at the block, tile, slice, picture, SPS, or PPS level). For example, the proposed method is applied when the block area is less than / greater than a threshold. In another embodiment, the proposed method uses a DIMD process to derive the intra-prediction mode of the current block based on the prediction samples used, such as a combination of prediction samples from all or any subset and / or intermediate prediction samples (i.e., any prediction hypothesis that will be used to form the combination prediction). In this case, horizontal and vertical gradients are computed for each prediction sample used to construct a gradient histogram (HoG). The intra-prediction mode corresponding to the maximum histogram count is then used to determine the transform set of the current block and / or the transform set in the encoding / decoding process of subsequent codec blocks.

[0106] In one example, the conversion process for the current block can be LFNST, NSPT, and / or MTS. Note that in this invention, LFNST / NSPT / MTS can be used only for intra-block conversion processes, only for inter-block conversion processes, or only for third-type (neither intra-nor inter-block) block conversion processes, and / or for any subset or combination of the above. Another example is that the encoding / decoding process for the next codec block can refer to the construction of the MPM (or merge list) and / or any inheritance scheme for the next codec block. When the next codec block references (or inherits) the intra-prediction mode of the current block, it can refer to the derived intra-prediction mode of the current block. Another example is that if the current block is a luma block and the next codec block is a chroma block, then if the current block is a co-lluminum block of the next codec chroma block, the chroma DM of the next codec block can use the derived intra-prediction mode of the current block.

[0107] In this invention, the term "block" can refer to TU / TB, CU / CB, PU / PB, a predefined region, or CTU / CTB.

[0108] Any combination of methods proposed in this invention can be applied.

[0109] Any of the aforementioned methods proposed in this invention, which derive combined predictions using mode type prediction, a second prediction, and mixed weights, can be implemented in the encoder and / or decoder. For example, any proposed method can be implemented in the inter-frame / intra-frame / IBC / prediction / conversion module of the encoder, and / or in the inter-frame / intra-frame / IBC / prediction / conversion module of the decoder. Alternatively, any proposed method can be implemented as circuitry connected to the inter-frame / intra-frame / IBC / prediction / conversion module of the encoder and / or the inter-frame / intra-frame / IBC / prediction / conversion module of the decoder, in order to provide the necessary information to the inter-frame / intra-frame / IBC / prediction / conversion module.

[0110] Figure 17 A flowchart illustrating a video codec system according to an embodiment of the present invention, which derives a combined predictor by mixing mode type prediction, inter-frame / IBC prediction, and mixing weights, is presented. The steps shown in the flowchart can also be implemented in hardware, such as one or more electronic devices or processors configured to perform the steps in the flowchart. According to the method, in step 1710, input data associated with the current block is received, wherein the input data includes pixel data to be encoded at the encoder end or encoded data associated with the current block to be decoded at the decoder end. In step 1720, a mode type prediction is determined, wherein the mode type prediction is generated using target prediction mode information from candidate prediction mode information, wherein the candidate prediction mode information includes filter-based intra-frame prediction information. In step 1730, a second prediction generated by an inter-frame mode or an IBC (intra-frame copy) mode is determined. In step 1740, a combined prediction is determined using the mode type prediction, the second prediction, and mixing weights. In step 1750, the current block is encoded or decoded using the combined prediction.

[0111] The flowchart shown is intended to illustrate video encoding and decoding according to the present invention. Those skilled in the art can modify each step, rearrange the steps, split the steps, or combine the steps to practice the invention without departing from its spirit. Specific syntax and semantics are used in this disclosure to illustrate embodiments of the invention. Those skilled in the art can practice the invention by substituting equivalent syntax and semantics without departing from its spirit.

[0112] The foregoing description is intended to enable those with ordinary technical skills to practice the invention in the context of specific applications and their requirements. Various modifications to the described embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments. Therefore, the object of the invention is not to be limited to the specific embodiments shown and described, but to accord the broadest scope consistent with the principles and novel features disclosed herein. In the foregoing detailed description, various specific details have been shown to provide a thorough understanding of the invention. However, those skilled in the art will understand that the invention can be practiced.

[0113] The embodiments of the present invention described above can be implemented in various hardware, software code, or a combination of both. For example, one embodiment of the invention may be one or more circuits integrated into a video compression chip, or program code integrated into video compression software to perform the processes described herein. Another embodiment of the invention may also be program code to be executed on a digital signal processor (DSP) to perform the processes described herein. The invention may also relate to multiple functions performed by a computer processor, digital signal processor, microprocessor, or field-programmable gate array (FPGA). These processors can be configured to perform specific tasks according to the invention, defining the specific methods embodied in the invention by executing machine-readable software code or firmware code. The software code or firmware code can be developed in different programming languages ​​and different formats or styles. The software code can also be compiled for different target platforms. However, different code formats, styles, and languages ​​of the software code, as well as other means of configuring the code to perform tasks according to the invention, do not depart from the spirit and scope of the invention.

[0114] This invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The examples described are to be regarded in all respects as illustrative rather than restrictive. Therefore, the scope of the invention should be indicated by the appended claims rather than the foregoing description. All variations within the meaning and equivalence of the claims should be included within its scope.

Claims

1. A video encoding / decoding method, the method comprising: Receive input data related to the current block, wherein the input data includes pixel data to be encoded at the encoder end or encoded data related to the current block to be decoded at the decoder end; Determine the mode type prediction, wherein the mode type prediction is generated using target prediction mode information from candidate prediction mode information, wherein the candidate prediction mode information includes filter-based intra-frame prediction information; Determine the second prediction generated by the inter-frame mode or intra-frame copy (IBC) mode; The combined prediction is determined using the pattern type prediction, the second prediction, and the mixed weights. Use combined predictions to encode or decode the current block.

2. The method according to claim 1, characterized in that, The candidate prediction model information includes the model type, prediction model, or a combination thereof.

3. The method according to claim 2, characterized in that, This mode type corresponds to the intra-frame mode, and the prediction mode corresponds to DC, planar, directional mode, or intra-frame correlation mode.

4. The method according to claim 3, characterized in that, The intra-frame correlation modes include Wide Angle Intra-Frame Prediction (WAIP), Matrix-Based Intra-Frame Prediction (MIP), Extrapolation Filter-Based Intra-Frame Prediction (EIP) regular mode, EIP merging mode, EIP extensions, or combinations thereof.

5. The method according to claim 2, characterized in that, If a previously encoded block is available and supports mode type, prediction mode, predefined prediction mode information, or a combination thereof, then the prediction mode information of that previously encoded block is used as a candidate.

6. The method according to claim 1, characterized in that, The candidate prediction pattern information is located in the candidate list.

7. The method according to claim 6, characterized in that, The candidate list corresponds to the merged candidate list.

8. The method according to claim 7, characterized in that, The candidate list includes one or more spatially adjacent candidates, one or more spatially non-adjacent candidates, one or more historical candidates, one or more time candidates, one or more default candidates, or a combination thereof.

9. The method according to claim 8, characterized in that, One or more preset candidates correspond to candidates that include default prediction mode information and / or are derived from one or more existing candidates that have been included in the merged candidate list.

10. The method according to claim 6, characterized in that, When inserting a candidate into the candidate list, use full or partial pruning.

11. The method according to claim 10, characterized in that, Before adding a candidate to the candidate list, check the candidate prediction pattern information of all or a subset of the candidate against the corresponding prediction pattern information of all or any subset of the existing candidates already in the candidate list.

12. The method according to claim 1, characterized in that, Use one or more selected candidates to generate a pattern type prediction for the current block.

13. The method according to claim 12, characterized in that, The one or more selected candidates are chosen based on the template cost associated with the available candidates.

14. The method according to claim 13, characterized in that, The template cost is calculated based on the distortion between the reconstruction on the template and each candidate prediction on the template.

15. The method according to claim 1, characterized in that, Combined prediction corresponds to combined inter-frame and intra-frame prediction (CIIP).

16. The method according to claim 15, characterized in that, Use regression-based derivation to determine the mixed weights.

17. The method according to claim 16, characterized in that, The relationship between the combined prediction of the reference region of the current block and the reconstructed sample of the current block of the reference region is estimated based on regression-based derivation to generate mixed weights.

18. The method according to claim 17, characterized in that, The current block of the reference region varies with block width, block height, block area, current block flag pattern information, neighboring blocks or coded blocks, one or more syntax elements of the flags in the block, CTU, SPS, PPS, picture, slice, tile or sequence level or a combination thereof.

19. The method according to claim 17, characterized in that, After the current block has been determined to be encoded according to the target mode, a flag is used to indicate whether to use the combined prediction of the mode type prediction, the second prediction, and the mixed weights, and to use the combined prediction to encode or decode the current block.

20. The method according to claim 1, characterized in that, The target combination prediction generated by using different mixed weights is considered an optional mode of the combination prediction mode.

21. An apparatus for video encoding and decoding, the apparatus comprising one or more electronic devices or processors configured to: Receive input data related to the current block, wherein the input data includes pixel data to be encoded on the encoder side or encoded data related to the current block to be decoded on the decoder side; Determine the mode type prediction, wherein the mode type prediction is generated by using target prediction mode information from candidate prediction mode information, wherein the candidate prediction mode information includes filter-based intra-frame prediction information; Determine the second prediction generated by the interactive mode or the intra-block copy (IBC) mode; The combined prediction is determined using the pattern type prediction, the second prediction, and the mixed weights; and Use this combination of predictions to encode or decode the current block.

Images