Region-based implicit intra mode derivation and prediction
By adopting the template-based intra mode export method in video encoding and decoding, the intra prediction mode is calculated using reference samples from the upper and left template areas, the problem of low intra mode prediction efficiency in the prior art is solved, and the more Efficient video encoding.
Patent Information
- Application Number
- CN202380027919.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-04-14
- Filing Date
- 2023-04-12
- Publication Date
- 2024-11-08
Smart Images

Figure CN118923115A_ABST
Abstract
Description
[0001]
Cross reference
[0002] The present invention is part of a non-provisional application claiming priority to U.S. Provisional Patent Application No. 63 / 330,825 filed on April 14, 2022. The contents of the above U.S. Provisional Patent Application are incorporated into this specification by reference. [Technical field]
[0003] The present invention relates to video coding and decoding. More specifically, the present invention relates to intra mode prediction. [Background technology]
[0004] Unless otherwise indicated herein, the approaches described in this section are not prior art to the claims listed later and are not admitted to be prior art by inclusion in this section.
[0005] High-Efficiency Video Coding (HEVC) is an international video coding standard developed by the Joint Collaboration on Video Coding (JCT-VC). HEVC is based on a hybrid block-based motion compensation DCT-like transform codec architecture. The basic unit of compression is called a codec unit (CU), which is a 2Nx2N square block of pixels. Each CU can be recursively split into four smaller CUs until a predefined minimum size is reached. Each CU contains one or more prediction units (PUs).
[0006] Versatile video coding (VVC) is the latest international video codec standard developed by the Joint Video Experts Group (JVET) of ITU-T SG16WP3 and ISO / IEC JTC1 / SC29 / WG11. The input video signal is predicted from the reconstructed signal, which is derived from the coded image region. The prediction residual signal is processed by block transform. The transform coefficients are quantized and entropy coded together with other side information in the bitstream. The reconstructed signal is generated from the prediction signal and the reconstructed residual signal after inverse transforming the dequantized transform coefficients. The reconstructed signal is further processed by loop filtering to remove codec artifacts. The decoded pictures are stored in a frame buffer and used to predict future pictures in the input video signal.
[0007] In VVC, a coded picture is divided into non-overlapping square block areas represented by associated codec tree units (CTUs). The leaf nodes of the codec tree correspond to codec units (CUs). A coded picture can be represented by a set of slices, each containing an integer number of CTUs. Each CTU in a slice is processed in raster scan order. Bi-predictive (abbreviated as B) slices can be decoded using intra prediction or inter prediction with up to two motion vectors and reference indices to predict the sample values of each block. Predictive (abbreviated as P) slices are decoded using intra prediction or inter prediction with up to one motion vector and reference index to predict the sample values of each block. Intra (abbreviated as I) slices are decoded using only intra prediction.
[0008] A CTU can be partitioned into one or more non-overlapping codec units (CUs) using a quadtree (QT) with a nested multi-type tree (MTT) structure to accommodate various local motion and texture characteristics. A CU can be further partitioned into smaller CUs using one of five partition types: quadtree partitioning, vertical binary tree partitioning, horizontal binary tree partitioning, vertical center-side ternary tree partitioning, and horizontal center-side ternary tree partitioning.
[0009] Each CU contains one or more prediction units (PUs). The prediction unit, together with the associated CU syntax, is used as a basic unit for signaling prediction sub-information. A specified prediction process is used to predict the values of relevant pixel samples within the PU. Each CU may contain one or more transform units (TUs) for representing prediction residual blocks. A transform unit (TU) consists of a transform block (TB) of luminance samples and two corresponding transform blocks of chrominance samples, and each TB corresponds to a residual block of samples from one color component. An integer transform is applied to the transform block. The level value of the quantization coefficient is entropy encoded and decoded in the bitstream together with other auxiliary information. The terms codec tree block (CTB), codec block (CB), prediction block (PB), and transform block (TB) are defined as 2-D sample arrays specifying one color component associated with a CTU, CU, PU, and TU, respectively. Therefore, a CTU consists of a luminance CTB, two chrominance CTBs, and associated syntax elements. Similar relationships are also valid for CUs, PUs, and TUs.
[0010] For each inter-predicted CU, motion parameters consisting of motion vector, reference picture index and reference picture list usage index, as well as additional information, are used for inter-predicted sample generation. Motion parameters can be communicated explicitly or implicitly. When a CU is encoded and decoded using skip mode, the CU is associated with one PU and has no significant residual coefficient, no encoded and decoded motion vector increment or reference picture index. A merge mode is specified, whereby the motion parameters of the current CU are obtained from neighboring CUs, including spatial and temporal candidates and additional scheduling introduced in VVC. Merge mode can be applied to any inter-predicted CU. An alternative to merge mode is the explicit transmission of motion parameters, where motion vectors, corresponding reference picture indexes for each reference picture list and reference picture list usage flags, as well as other required information, are explicitly communicated for each CU. [Summary of the invention]
[0011] The following summary is illustrative only and is not intended to be limiting in any way. That is, the following summary is provided to introduce the concepts, highlights, benefits, and advantages of the novel and non-obvious technology described herein. Selected, but not all, implementations are further described below in the detailed description. Therefore, the following summary is not intended to identify essential features of the claimed subject matter, nor is it intended to be used to determine the scope of the claimed subject matter.
[0012] Some embodiments of the present disclosure provide a method for implicitly deriving region-based intra prediction. A video codec receives data of a pixel block of a current block of a current picture to be encoded or decoded as a video. The video codec identifies an upper template region and a left template region of the current block among reconstructed pixels of the current picture. The video codec derives a first intra prediction mode based on the upper template region, and derives a second intra prediction mode based on the left template region. The video codec generates a first and a second predictor of the current block based on the first and the second intra prediction modes. The video codec encodes or decodes the current block by using the first predictor and the second predictor to reconstruct the current block.
[0013] In some embodiments, first and second intra-frame prediction modes are identified based on the cost of candidate intra-frame prediction modes by a template-based intra-frame mode derivation (TIMD) process. The cost of the candidate of the first intra-frame prediction mode is calculated based on the reconstructed samples of the upper template area and the predicted samples of the upper template area, wherein the predicted samples of the upper template area are generated by using the reference samples identified by the candidate of the first intra-frame prediction mode. The cost of the candidate of the second intra-frame prediction mode is calculated based on the reconstructed samples of the left template area and the predicted samples of the left template area, wherein the predicted samples of the left template area are generated by using the reference samples identified by the candidate of the second intra-frame prediction mode. The reference sample is identified from a reference area including an area above the upper template area, an area to the left of the left template area, or an area above and to the left of the upper and left template areas.
[0014] In some embodiments, the first and second intra prediction modes are identified by a decoder-side intra mode derivation (DIMD) process based on histograms of gradients (HoGs) at different intra prediction angles. Specifically, the first intra prediction mode is identified based on a first HoG of gradient magnitudes at different pixel positions along the upper template region, and the second intra prediction mode is identified based on a second HoG of gradient magnitudes at different pixel positions along the left template region.
[0015] In some embodiments, the decoder generates a combined intra prediction of the current block by mixing the first predictor and the second predictor, and reconstructs the current block using the combined intra prediction. In some embodiments, the combined prediction is a weighted sum of the first and second predictors, wherein the weight values assigned to the first and second predictors are determined based on the distance from the upper template region and the distance from the left template region.
[0016] In some embodiments, a geometrically positioned straight line derived from the angle and offset parameters divides the current block into a first partition and a second partition. The first predictor is used to reconstruct the first partition, and the second predictor is used to reconstruct the second partition, wherein samples along a boundary between the first partition and the second partition are reconstructed using combined intra prediction.
[0017] In some embodiments, the current block is a first sub-block of a plurality of sub-blocks of a larger block, and the upper template area is a sub-template of a plurality of sub-templates above the larger block, and the left template area is a sub-template of a plurality of sub-templates to the left of the larger block. In some embodiments, samples along a boundary between a first sub-block and a second sub-block are reconstructed using a hybrid prediction that is a weighted sum of (i) a combined intra-prediction of the current block and (ii) an intra-prediction of the current block or an intra-prediction of a second sub-block generated using an intra-prediction mode of the second sub-block, wherein the second sub-block is adjacent to the first sub-block (the current block), wherein the intra-prediction of the second sub-block is derived from third and fourth intra-prediction modes that are different from the first and second intra-prediction modes.
Brief Description of the Drawings
[0018] The accompanying drawings are included to provide a further understanding of the present disclosure and are incorporated into and constitute a part of the present disclosure. The accompanying drawings illustrate embodiments of the present disclosure and, together with the description, are used to explain the principles of the present disclosure. It should be understood that the drawings are not necessarily drawn to scale, as some components may be shown to be out of scale with the actual implementation in order to clearly illustrate the concepts of the present disclosure.
[0019] Figure 1 Different directional intra prediction modes are shown.
[0020] Figure 2 The use of decoder-side intra mode derivation (DIMD) to implicitly derive the intra prediction mode of the current block is shown.
[0021] Figure 3 It is shown that the intra prediction mode of the current block is implicitly derived using template-based intra mode derivation (TIMD).
[0022] Figure 4 It is shown that the current block is divided into multiple block areas based on angles to apply the DIMD / TIMD derivation process.
[0023] Figure 5A -B conceptually illustrates the derivation of two different intra prediction modes from two different template regions.
[0024] Figure 6 Conceptually shown is a blend of two intra prediction predictors from two different intra modes derived from the top and left template regions.
[0025] Figure 7 Conceptually illustrated are blocks divided into grids and different intra prediction modes derived for different grids.
[0026] Figure 8The mixing of different intra predictions along the grid boundaries is shown.
[0027] Fig. 9 The partitioning of the template and / or the current block by irregular partitioning is shown.
[0028] Fig.10 The application of DIMD / TIMD to sub-blocks of a large block is shown.
[0029] Fig.11 A current block is shown whose intra prediction mode is determined based on the intra prediction modes of sub-block templates within a predefined range.
[0030] Fig.12 The coding and decoding of large blocks through multiple intra prediction modes and merged transform blocks are presented.
[0031] Fig.13 DIMD / TIMD is shown applied to sub-blocks of a block in reverse order.
[0032] Fig.14 An exemplary video encoder that can implement region-based implicit intra prediction is shown.
[0033] Fig.15 Portions of a video encoder implementing region-based implicit intra prediction are shown.
[0034] Fig.16 The process of encoding a block of pixels using region-based implicitly derived intra prediction is conceptually illustrated.
[0035] Fig.17 An example video decoder 1700 is shown that can implement region-based implicit intra prediction.
[0036] Fig.18 A portion of a video decoder 1700 that implements region-based implicit intra prediction is shown.
[0037] Fig.19 A process 1900 for decoding a block of pixels using region-based implicitly derived intra prediction is conceptually illustrated.
[0038] Fig. 20 An electronic system for implementing some embodiments of the present disclosure is conceptually illustrated. [Specific implementation method]
[0039] In the detailed description below, many specific details are set forth by way of example to provide a thorough understanding of the relevant teachings. Any variation, derivation and / or extension based on the teachings described herein are within the scope of protection of the present disclosure. In some cases, well-known methods, processes, components and / or circuits associated with one or more example implementations disclosed herein may be described at a relatively high level without details to avoid unnecessarily obscuring the various aspects of the teachings of the present disclosure.
[0040] Intra-frame prediction
[0041] The intra prediction method uses a reference tier and an intra prediction mode adjacent to the current prediction unit (PU) to generate a prediction sub-unit for the current PU. The intra prediction direction can be selected from a mode set containing multiple prediction directions (angles) and / or multiple non-angle prediction modes (such as DC mode and planar mode). For each PU decoded by intra prediction, an index will be used and encoded to select one of the intra prediction modes. The corresponding prediction will be generated, and the residual can then be derived and transformed.
[0042] Figure 1 Intra-frame prediction modes in different directions are shown. These intra-frame prediction modes are called directional modes, excluding DC mode or planar mode. As shown in the figure, there are 33 directional modes (V: vertical direction; H: horizontal direction), so H, H+1~H+8, H-1~H-7, V, V+1~V+8, V-1~V-8 are used. In general, the directional mode can be expressed as H+k or V+k mode, where k=±1, ±2,..., ±8. Each of such intra-frame prediction modes can also be called an intra-frame prediction angle. In order to capture arbitrary edge directions presented in natural videos, the number of directional intra-frame modes can be extended from the 33 directional modes used in HEVC to 65 directional modes, so that k ranges from ±1 to ±16. These denser directional intra-frame prediction modes are applicable to all block sizes and luminance and chrominance intra-frame predictions. By including DC and planar modes, the number of intra-frame prediction modes is 35 (or 67).
[0043] Of the 35 (or 67) intra prediction modes, three modes are considered to be the most probable modes (MPMs) for predicting the intra prediction mode in the current prediction block. These three modes are selected as the MPM set. For example, the intra prediction mode used in the left prediction block and the intra prediction mode used in the above prediction block are used as MPMs. When the intra prediction modes of two neighboring blocks use the same intra prediction mode, the intra prediction mode can be used as an MPM. When only one of the two neighboring blocks is available and encoded in a directional mode, the two adjacent directions adjacent to the directional mode can be used as MPMs. DC mode and planar mode are also considered as MPMs to fill in the available points in the MPM set, especially if the above or top neighboring block is not available or does not use intra prediction encoding or decoding, or if the intra prediction mode in the neighboring block is not a directional mode. If the intra prediction mode of the current prediction block is one of the modes in the MPM set, 1 or 2 bits are used to signal which one it is. Otherwise, the intra prediction mode of the current block is different from any entry in the MPM set, and the current block will be encoded as non-MPM mode. There are 32 such non-MPM modes in total, and a (5-bit) fixed-length codec method is used to signal the mode.
[0044] Decoder-SideIntraModeDerivation (DIMD)
[0045] Decoder-side intra mode derivation (DIMD) is a technique that derives two intra prediction modes / angles / directions from the reconstructed neighboring samples (template) of a block, and these two predictors are combined with a planar mode predictor with weights derived from gradients. DIMD mode is used as an alternative prediction mode and is always checked in high-complexity RDO mode. In order to implicitly derive the intra prediction mode of a block, texture gradient analysis is performed on both the encoder and decoder sides. The process starts with an empty gradient histogram (Histogram of Gradient, abbreviated as HoG) with 65 entries, corresponding to the 65 angle / direction intra prediction modes. The magnitudes of these entries are determined during texture gradient analysis.
[0046] A video codec that implements DIMD performs the following steps: In the first step, the video codec picks a template of T = 3 columns and rows from the left and above the current block, respectively. This region is used as a reference for gradient-based intra prediction mode derivation. In the second step, horizontal and vertical Sobel filters are applied to all 3×3 window locations, centered on the pixels in the template midline. At each window location, the Sobel filter computes the pure horizontal and vertical intensities as G x and Gy . The texture angle of the window is then calculated as follows:
[0047] angle=arctan(G x / G y ),
[0048] It can be converted into one of 65 angular intra prediction modes. Once the intra prediction mode index of the current window is derived as idx, the magnitude of its entry in HoG[idx] is updated by addition:
[0049] ampl=|G x |+|G y |
[0050] Figure 2 The use of decoder-side intra mode derivation (DIMD) to implicitly derive the intra prediction mode of the current block is shown. The figure shows an example histogram of gradients (HoG) 210, which is calculated after applying the above operations to all pixel positions in the template 215 around the current block 200. Once the HoG is calculated, the indices of the two highest histogram bars (M1 and M2) are selected as the two implicitly derived intra prediction modes (IPMs) for the block. The predictions of the two IPMs are further combined with the plane mode as the prediction of the DIMD mode. The prediction fusion is applied as a weighted average of the above three predictors (M1 prediction, M2 prediction, and plane mode prediction). To this end, the weight of the plane can be set to 21 / 64 (~1 / 3). Then, the remaining weight 43 / 64 (~2 / 3) is distributed between the two HoG IPMs, proportional to the amplitude of their HoG bars. The prediction fusion or combined prediction of DIMD can be:
[0051] Pred DIMD =(43*(w1*pred M1 +w2*pred M2 )+21*pred planar )>>6
[0052] w1=amp M1 / (amp M1 +amp M2 )
[0053] w2=amp M2 / (amp M1 +amp M2 )
[0054] Additionally, the two implicitly derived intra prediction modes are added to the most probable mode (MPM) list, so the DIMD process is performed before building the MPM list. The primary derived intra mode of a DIMD block is stored with the block and used for MPM list construction for neighboring blocks.
[0055] Template-based IntraMode Derivation (TIMD)
[0056] Template-based intra mode derivation (TIMD) is a coding approach in which the intra prediction mode of a CU is implicitly derived by using neighboring templates at the encoder and decoder, instead of the encoder signaling the exact intra prediction mode to the decoder.
[0057] Figure 3 The intra prediction mode of the current block 300 is implicitly derived using template-based intra mode derivation (TIMD). As shown, the neighborhood pixels of the current block 300 are used as template 310. For each candidate mode, a reference sample 320 in a reference area located above and to the left of the template 310 is used to generate a prediction sample of the template 310. The cost is calculated based on the difference between the prediction sample and the reconstructed sample of the template (e.g., SATD). The intra prediction mode with the lowest cost (as the intra prediction mode with the largest histogram in the DIMD mode) is selected for intra prediction of the CU. The candidate mode may include 67 intra prediction modes (as in VVC) or be extended to 131 intra prediction modes. MPM can be used to indicate the directional information of the CU. Therefore, in order to reduce the intra mode search space and utilize the characteristics of the CU, the intra prediction mode is implicitly derived from the MPM list. That is, the candidate mode includes all or any subset of the MPM list.
[0058] For each intra prediction mode in the MPM, the SATD between the predicted sample and the reconstructed sample of the template is calculated. The first two intra prediction modes with the smallest SATD are selected as TIMD modes. These two TIMD modes are fused with weights after applying the PDPC process, and this weighted intra prediction is used to encode and decode the current CU. Position dependent intra prediction combination (PDPC for short) is included in the derivation of TIMD mode. When generating predictions for templates of candidate modes, the prediction generation process can be simplified. For example, the reference samples used in the prediction generation process are not filtered by a reference sample filtering process such as [1,2,1] filtering. For another example, the interpolation filter used to generate prediction samples from non-integer positions is predefined as only one interpolation filter, such as cubic interpolation filtering. For another example, PDPC is applied in the prediction generation process only when the current block has a block size (block width and / or height) greater than a predefined threshold.
[0059] The costs of the two selected modes (mode 1 and mode 2) are compared with the threshold value. In the test, a cost factor of 2 is applied as follows:
[0060] costMode2<2*costMode1
[0061] If this condition is true, prediction fusion is applied, otherwise only mode 1 is used. The weight of the mode is calculated based on its SATD cost as follows:
[0062] weight1=costMode2 / (costMode1+costMode2)
[0063] weight2=1-weight1
[0064] Improve DIMD / TIMD prediction performance
[0065] Some embodiments of the present disclosure provide a method for improving TIMD / DIMD prediction accuracy or encoding and decoding performance. When TIMD / DIMD is used to derive one or more intra-frame prediction modes of the current block, the candidate intra-frame prediction mode may include all, any subset, or any extension of the intra-frame prediction mode specified in Part I (Intra-frame Prediction). For example, the candidate intra-frame prediction mode includes only or at least includes MPM or any subset of MPM. For another example, the candidate intra-frame prediction mode includes only or at least includes DC mode, plane mode, horizontal mode, vertical mode, diagonal mode and / or any subset of the above. For another example, the candidate intra-frame prediction mode includes only or at least includes WAIP mode allowed by non-square blocks (for example, (block width divided by block height) is equal to 2, (block width divided by block height) is equal to 4, (block width divided by block height) is equal to 1 / 2, or (block width divided by block height) is equal to 1 / 4). In one case, when the current block is a non-square block, the WAIP mode is added to the candidate intra-frame prediction mode. In another case, the WAIP mode is added to the candidate intra prediction mode according to the check of the availability of the upper right and / or lower left reference samples of the current block and / or the template of the current block. If the check of the upper right reference sample is qualified, the WAIP mode of the block with (block width divided by block height) equal to K1 is added to the TIMD search, where K1 is a predefined positive integer greater than 1. When the intra prediction mode in VVC is 67 intra prediction modes, the mode number of the newly added WAIP mode is greater than the maximum angle mode number 66 in the 67 intra prediction modes. If the check of the lower left reference sample is qualified, the WAIP mode of the block with (block width divided by block height) equal to 1 / K2 is added to the TIMD search, where K2 is a predefined positive integer greater than 1. When the intra prediction mode in VVC is 67 intra prediction modes, the mode number of the added WAIP mode is less than the minimum angle mode number 2 or mode number 0 in the 67 intra prediction modes. K1 and K2 are predefined according to the availability of the upper right reference sample and the lower left reference sample, respectively.
[0066] A. Sub-block DIMD / TIMD
[0067] In some embodiments, vertical or horizontal partitioning is applied to divide the block into sub-blocks, and DIMD / TIMD is applied to derive the intra prediction angle or mode for each sub-block. In some embodiments, when a block is divided into sub-blocks for TIMD and / or DIMD, a partitioning method of intra sub-partitions (ISP) can be used (depending on the block size, the ISP mode divides the luma intra prediction block vertically or horizontally into 2 or 4 sub-partitions).
[0068] In some embodiments, when TIMD and / or DIMD are used to derive the intra prediction mode of a subblock, a spatially adjacent reference L shape (the neighboring reconstructed samples above and to the left) of the current subblock is used as a template for TIMD / DIMD. In some embodiments, the intra prediction mode of each subblock may be different according to the TIMD / DIMD derivation result of each subblock. In some embodiments, the intra prediction mode of each subblock is collected, and the intra prediction mode used (e.g., voted) by the most subblocks in a specific area may be the intra prediction mode of the entire block.
[0069] B. Multi-area DIMD / TIMD
[0070] According to DIMD and TIMD, a predefined template (neighboring area) of the current block is used to determine the intra prediction mode. In some embodiments, the predefined template is divided into a plurality of template areas. For each template area, a DIMD / TIMD derivation operation is applied to determine a recommended intra prediction mode. In some embodiments, the current block is divided into a plurality of block areas. Different intra prediction modes can be derived for different block areas by applying the DIMD / TIMD derivation process. The derived different intra prediction modes can be the intra prediction modes with the minimum TIMD cost or with the highest DIMD histogram bar.
[0071] In some embodiments, in order to segment or divide a template into a plurality of template regions (template parts) or to segment or divide a current block into a plurality of block regions, angle-based segmentation is applied. Figure 4 The current block is shown to be divided into multiple block areas based on angles to apply the DIMD / TIMD derivation process. The figure shows that the current block 400 is divided into multiple block areas 421-423 by dividing line 1 and dividing line 2. The figure also shows that the template 410 of the current block 400 is divided into multiple template parts 431-434 along the same dividing lines 1 and 2. Template part 432 can be used for block area 421 to obtain intra-frame prediction mode by using TIMD or DIMD. Template parts 431 and 433 can be used for block area 422 to obtain intra-frame prediction mode by using TIMD / DIMD. Template part 434 can be used for block area 423 to obtain intra-frame prediction mode by using TIMD / DIMD. In some embodiments, the angle used in the angle-based segmentation is set to an angle with a smaller TIMD cost (or with a higher DIMD histogram bar).
[0072] In some embodiments, the prediction of the current block is a combined prediction by mixing predictions from two different intra-frame prediction modes (e.g., two different angles or two different intra-frame prediction modes from DC mode, planar mode and / or angle), where the two different intra-frame prediction modes are derived by applying the TIMD / DIMD derivation process on two different template regions. Figure 5A -B conceptually illustrates deriving two different intra prediction modes from two different template regions (e.g., two different angles or two different intra prediction modes from DC mode, planar mode and / or angle). In some cases, the combined prediction is not used as the prediction for the current block. In one case, the two intra prediction modes are the same. In another case, the template region on the left or the template region at the top is not available. In this case, the prediction for the current block comes from the available template region on the left or the top.
[0073] As shown, the current block 500 has a template 510 divided into a top template region 511 and a left template region 512. A first intra prediction angle or mode (denoted as angle 1 or ModeA) is derived from the top template region 511 through TIMD / DIMD, and a second intra prediction angle or mode is (denoted as angle 2 or ModeL) is derived from the left template region 512 through the TIMD / DIMD derivation process. Then, the prediction of the current block by using ModeA and the prediction of the current block by using ModeL are weighted mixed to generate a final combined prediction of the current block.
[0074] Figure 5AConceptually illustrated is the use of the TIMD derivation process to derive two intra prediction modes. Both intra prediction modes are determined based on reference samples 520 located at the top and left of the template 510. The ModeA intra prediction mode is determined based on the top template region 511 and all or any subset of the reference samples 520, while the ModeL intra prediction mode is determined based on the left template region 512 and all or any subset of the reference samples 520. In order to determine ModeA, for each candidate intra prediction mode, the cost is calculated based on the difference (e.g., SATD) between the prediction of the template 511 (by using the candidate intra prediction mode and all or any subset of the reference samples 520) and the reconstructed samples of the template 511. The candidate intra prediction modes may include only angles, only non-angle modes (DC mode and / or planar mode), or all or any subset of the above modes. The candidate intra prediction mode / angle with the minimum (lowest) cost is selected as ModeA. To determine ModeL, for each candidate intra prediction mode, a cost is calculated based on the difference (e.g., SATD) between the prediction of template 511 (by using the candidate intra prediction mode and all or any subset of reference samples 520) and the reconstructed samples of template 512. The candidate intra prediction modes may include only angles, only non-angle modes (DC mode and / or planar mode), or all or any subset of the above modes. The candidate intra prediction mode / angle with the smallest (lowest) cost is selected as ModeL. The reference samples used to generate predictions for templates 511 and / or 512 may be referred to as reference samples 520. In another way, the reference samples used to generate predictions for template 511 may be reference samples that are spatially adjacent to the corresponding template 511, and / or the reference samples used to generate predictions for template 512 may be reference samples that are spatially adjacent to the corresponding template 512.
[0075] Figure 5B Conceptually illustrated is the use of the DIMD derivation process to derive two intra prediction modes. Both intra prediction modes are determined by identifying the highest bar in the gradient histogram (HoG) for different intra prediction angles. Specifically, the ModeA intra prediction angle is identified by using the HoG 531 of the gradient magnitude calculated along the pixel positions of the top template region 511, while the ModeL intra prediction angle is identified by using the HoG 532 of the gradient magnitude calculated along the pixel positions of the left template region 512.
[0076] Figure 6Conceptually illustrates a blend of two intra-prediction predictors from two different intra-modes (ModeA and ModeL) derived from the top template region and the left template region. The figure shows a blend of two intra-prediction predictions for the current block 500. As shown, the current block 500 is divided into a ModeA prediction region 541 and a ModeL prediction region 542. Pixels across the boundary or edge between the ModeA prediction region 541 and the ModeL prediction region 542 can be blended by a weighting scheme. The division and blending of the two intra-prediction regions can be similar to a geometric partitioning mode (GPM), a combined inter / intra prediction (CIIP) mode, a bidirectional prediction (BCW) mode with CU-level weights, or another type of partitioning / or blending scheme.
[0077] In some embodiments, the current block 500 can be divided into two partitions in a GPM-like manner by geometrically positioned straight lines derived mathematically from angle and offset parameters. One geometric partition is predicted by ModeA intra-frame prediction mode, and the other geometric partition is predicted by ModeL intra-frame prediction mode. The hybrid weight of each position of the CU is derived based on the distance between each sample position and the partition boundary.
[0078] In some embodiments, the current block 500 may not be split into two partitions. Instead, both ModeA and ModeL are used to generate two intra-frame prediction P for the entire block 500. ModeA and P ModeL In some embodiments, the two intra-frame prediction signals P ModeA and P ModeL The combined prediction P of the whole block can be combined or mixed according to the following formula:
[0079] P(x,y)=(w modeA (x,y)*P modeA (x,y)+w modeL (x,y)*P modeL (x,y)+32)>>6
[0080] The prediction at each position (x, y) in the current block (x from 0 to block width - 1 and y from 0 to block height - 1) is assigned a weight value w based on its distance to the template region above 511 and the template region to the left 512. modeA (x, y) and w modeL (x, y). In some embodiments, when the sample (x, y) is close to the upper template area 511, w modeA (x, y) is assigned a larger value; when the sample (x, y) is close to the left template area 512, w modeL(x, y) is assigned the larger value. The offset value 32 and the right shift value 6 depend on the weight value. The offset value is half of the sum of the weight values for each prediction. The right shift value is the log2 number of the sum of the weight values for each prediction. Example values of the offset values 32 and 6 and example values of the right shift value when the sum of the weight values is equal to 64. The present invention is not limited to this example. An example specification of this position-based weighting scheme: (W and H refer to the width and height of the block in pixels / sample)
[0081] w modeA (x,y)+w modeL (x,y)=64
[0082]
[0083] In some embodiments, the two intra-frame prediction signals P ModeA and P ModeL It can be combined in a similar way to CIIP to generate the intra prediction P using weighted averaging according to the following formula:
[0084] P=((4-wt)*P ModeA +wt*P ModeL +2)>>2
[0085] The weight value wt is calculated according to the encoding and decoding modes of the top and left neighboring blocks. For example, if only the left neighboring block is intra-coded, wt can be 3; if both the left and top neighboring blocks are intra-coded, wt can be 2; if only the top neighboring block is intra-coded, wt can be 1.
[0086] In some embodiments, the two intra-frame prediction signals P ModeA and P ModeL They can be combined in a similar way to BCW using weighted averaging according to the following formula:
[0087] P=((8-w)*P ModeA +w*P ModeL +4)>>3
[0088] The weight factor w can be selected from a set of allowed numbers, such as {-2, 3, 4, 5, 10} or {3, 4, 5}. A weight index can be used to represent this selection. The weight index can be inferred from neighboring blocks based on the merge candidate index.
[0089] In some embodiments, sample-based or region-based segmentation is applied to segment (partition) the template or current block into multiple template regions or multiple block regions. For example, the intra prediction mode / angle derived from a specific template region is applied to a specific block region.
[0090] In some embodiments, the block is partitioned into multiple grids (regions) and for each grid, the intra prediction mode of the grid is derived using the corresponding L-shaped template (neighboring reconstructed samples) by using the DIMD and / or TIMD derivation process.
[0091] Figure 7 The block 700 is conceptually divided into a grid and different intra prediction modes derived for different grids. 11 、grid 21 、grid 12 、grid 22 . The L-shaped area above and to the left of block 700 is divided into template areas A1, A2, L1, L2. The TIMD / DIMD process described with reference to FIGS. 5A-B can be used to derive the ModeA and ModeL intra prediction modes / angles for each grid. The reference samples used to generate predictions on each template area can be referred to as reference samples 520. That is, different template areas share the reference samples 520, and the costs of different template areas are calculated on the template areas separately. In other words, the reference samples used to generate predictions for each template area can be reference samples that are spatially adjacent to each corresponding template area. ModeA i It is in A i The mode with the smallest SATD (e.g., ModeA1, ModeA2), while ModeL i It is in L i The mode with the smallest SATD (e.g., ModeL1, ModeL2) is selected. Therefore, template area A1 is used to derive ModeA1, template area A2 is used to derive ModeA2, template area L1 is used to derive ModeL1, and template area L2 is used to derive ModeL2. This is an example of dividing the current block into 4 grids. The present invention is not limited to this example. For another example, only the upper template area is divided. For another example, only the left template area is divided. For another example, when the block width of the current block is greater than a predefined threshold (e.g., 2, 4, 16, or any positive integer greater than 1 specified in the standard or signaled in the bitstream), the upper template area is divided into N1 sub-areas. N1 is fixed to a predefined number (e.g., 2, 4, or any positive integer greater than 1 specified in the standard or signaled in the bitstream). When the block height of the current block is greater than a predefined threshold (e.g., 2, 4, or any positive integer greater than 1 specified in the standard or signaled in the bitstream), the left template area is divided into N2 sub-areas. N2 is fixed to a predefined number (e.g., 2, 4, or any positive integer greater than 1 specified in the standard or signaled in the bitstream). N1 and N2 can be the same or different. The current block is divided according to the division on the template. Figure 7 In the example, N1 and N2 are equal to 2.
[0092] For each grid ij , the corresponding ModeAi and ModeLj intra predictions are combined / mixed at position (x,y) in the current block (x from 0 to block width - 1 and y from 0 to block height - 1) or position (x,y) in the current grid (x from 0 to grid width - 1, y from 0 to grid height - 1) according to the following formula:
[0093] P(x,y)=(w modeAi (x,y)*P modeAi (x,y)+w modeLj (x,y)*P modeLj (x,y)+32)>>6
[0094] Therefore, for the grid 11 , combining the intra prediction of ModeA1 and ModeL1; for grid 21 , combining the intra prediction of ModeA2 and ModeL1; for grid 12 , Intra-frame prediction for ModeA1 and ModeL2, for grid 22 ; Combine intra-frame prediction of ModeA2 and ModeL2.
[0095] Since different intra prediction modes may be applied to different grids within the current block, some embodiments apply blending along grid boundaries. Figure 8 The grid of block 700 is shown as mixing different intra predictions along the grid boundary. 11 It can be divided into four components 811, 812, 821 and 822. Component 811 is far from the boundary / edge with other grids and therefore does not mix. Component 812 is located in the grid 12 At the boundary of 12 The intra prediction mode is the current grid. 11 ) generated intra prediction or (2) grid 12 An example of (1) is shown. The blended prediction of component 812 is:
[0096] P(x,y)=(48*(mixed prediction(x,y) from ModeA1 and ModeL1))+
[0097] 16*(mixed prediction(x,y) from ModeA1 and ModeL2)+32)>>6
[0098] Component 821 is located in grid 21At the boundary of 21 The intra prediction mode is the current grid. 11 ) generated intra prediction or (2) grid 21 An example of (1) is shown. The mixed prediction of component 821 is:
[0099] P(x,y)=(48*(mixed prediction(x,y) from ModeA1 and ModeL1))+
[0100] 16*(mixed prediction(x,y) from ModeA2 and ModeL1)+32)>>6
[0101] Component 822 is located in grid 21 and grid 12 At the boundary of 21 and grid 12 The intra prediction mode of both is the intra prediction generated by the current grid or (2)grid 21 and grid 12 The intra prediction of both is mixed. An example of (1) is shown. The mixed prediction of component 822 is as follows:
[0102] P(x,y) = (32*(mixed prediction(x,y) from ModeA1 and ModeL1)
[0103] +16*(mixed prediction(x,y) from ModeA2 and ModeL1)
[0104] +16*(mixed prediction(x,y) from ModeA1 and ModeL2)+32)>>6
[0105] In some embodiments, grids that are not located along the top and / or left border of the current block may inherit their intra prediction modes from their neighboring grids. Thus, for example, grid 21 From the grid 11 Inherit the intra prediction mode, and grid 22 From the grid 12 and grid 21 Inherit intra prediction mode.
[0106] Ck is used for DIMD / TIMD of irregularly partitioned templates and blocks
[0107] In some embodiments, after irregular partitioning such as GPM partitioning, the template or current block may be partitioned into multiple template regions or multiple block regions. Fig. 9 The segmentation of template and / or current block by irregular partition is shown. As shown in the figure, the template 905 of current block 900 is segmented into multiple template areas (or template parts) 911-914 by using GPM segmentation. The same GPM segmentation also divides the current block 900 into corresponding multiple block areas 921-923. The intra-frame prediction angle from template part 911 (or 913), the intra-frame prediction angle from template part 912 and the intra-frame prediction angle from part 914 are applied to corresponding block areas 921, 922 and 923 respectively to obtain their own predictions. The final prediction of the current whole block 900 is formed by mixing multiple predictions of multiple block areas. In certain embodiments, mixing is performed according to the weight of GPM.
[0108] In some embodiments, large blocks can be implicitly partitioned with QT, and then each QT leaf block can have its own intra prediction mode. In some embodiments, when the current block contains multiple block regions, each block region can have its own intra prediction mode and its own transform mode (union transform may have distribution problems because splitting multiple block regions results in sub-block based distribution.). If each sub-block has its own transform mode, each sub-block can be regarded as a separate transform block (TB). In some embodiments, in order to avoid the sub-TB overhead for transformation, a merged transform block can be used. The merged transform block will be further described in Section IV-F below.
[0109] D. Neighbor-based DIMD / TIMD
[0110] In some embodiments, for the purpose of applying DIMD / TIMD, large blocks or blocks whose long sides are much larger than their short sides can be divided into multiple sub-blocks. For each such sub-block, the default mode is initialized to the adjacent intra-frame prediction mode. Then TIMD / DIMD is used to refine the default mode.
[0111] Fig.10The application of DIMD / TIMD to sub-blocks of a large block is shown. As shown, a large block 1000 having an aspect ratio of 4:1 as a current block is divided into two sub-blocks 1010 and 1020 by vertical partitioning. For the left sub-block 1010, the intra-frame prediction angle A is inherited from one or more neighbors 1011-1013 close to the left and top boundaries of the left sub-block 1010. For the right sub-block 1020, the intra-frame prediction angle B is inherited from one or more neighbors 1021 close to the top boundary. TIMD / DIMD can be used to refine the intra-frame prediction angle A. For example, the candidate modes tried in the TIMD derivation process include angle A or adjacent modes of angle A (+n to -n modes of angle A, where n can be any positive integer). TIMD / DIMD can also be used to refine the intra-frame prediction angle B. For example, the candidate modes tried in the TIMD derivation process include angle B or adjacent modes of angle B (+n to -n modes of angle B, where n can be any positive integer).
[0112] E. Sub-block-based TIMD with search range
[0113] In some embodiments, an intra prediction mode is selected for the current block based on the intra prediction modes selected for subblocks in a predefined search range. Fig.11 A current block 1100 is shown, whose intra prediction mode is determined based on the intra prediction modes of the sub-block templates 1121-1123 in the predefined range 1110. In some embodiments, the intra prediction mode with the smallest TIMD cost or with the highest DIMD histogram bin is identified and selected as the intra prediction mode for a certain (or each) sub-block template in the predefined search range 1110. In some embodiments, one or more selected intra prediction modes are used for the current block, wherein the selected intra prediction modes are those selected or identified by the most sub-block templates in the predefined search range 1110.
[0114] In some embodiments, if the sub-block templates 1121-1123 in the predefined search range 1110 recommend very different intra-prediction modes (which may mean that the texture around the current block is complex), the planar prediction is mixed with the TIMD / DIMD prediction. The TIMD / DIMD prediction of the current block can be a hybrid prediction of one or more selected intra-prediction modes from each sub-block template in the predefined search range. In some embodiments, the hybrid weight depends on the number of sub-blocks in the predefined search range that select the intra-prediction mode. In some embodiments, the hybrid weight can depend on the SATD cost of the mode.
[0115] F. Multiple intra prediction modes for large blocks
[0116] In some embodiments, multiple intra prediction modes are used for large blocks to improve the accuracy of intra prediction, thereby bringing codec gain. In some embodiments, the large block is divided into multiple sub-blocks, and then for each sub-block, the intra prediction mode and / or transform mode is signaled or parsed. In some embodiments, each sub-block has its own transform mode, and each sub-block can be regarded as a separate transform block (TB).
[0117] Some embodiments of the present disclosure provide a merged transform block method to avoid TB overhead for transform. The merged transform includes using multiple transform blocks, and each transform block in the merged transform block uses a unified transform mode and / or shares the same transform syntax and / or shares the same transform implicit rule.
[0118] In some embodiments, in order to avoid the sub-block intra prediction angle syntax overhead, the DIMD / TIMD derivation process is used to reduce the syntax overhead by reordering the index of the intra prediction mode. The DIMD / TIMD derivation process can be used to select or recommend the priority order of candidate modes. In some embodiments, for DIMD, candidate modes with higher histogram bars obtain higher priority orders, while for TIMD, candidate modes with smaller SATD costs obtain higher priority orders. Candidate modes based on priority order can be communicated or parsed. In some embodiments, the candidate mode with the highest priority is communicated or parsed with the shortest codeword. In some embodiments, the candidate mode with the highest priority is inferred as the selected mode of the current sub-block.
[0119] Fig.12 The encoding and decoding of a large block by multiple intra prediction modes and a merged transform block is shown. As shown, the current block 1200 is a large block divided into four sub-blocks 1211-1214. On the encoder side, the intra prediction mode / angle is determined for each sub-block (using DIMD or TIMD). The residuals 1220 of the four sub-blocks are calculated based on the intra prediction of the four sub-blocks. The residuals of the four sub-blocks are transformed to obtain the transform coefficients of the four sub-blocks. The transform coefficients of the four sub-blocks are merged to form a merged transform block 1230.
[0120] At the decoder side, the merged transform block 1230 is inversely transformed to obtain a residual 1220, which is divided into four sub-blocks 1211-1214. The encoder also signals an intra-frame prediction mode to the decoder using the reordered index to minimize the codeword length. The one intra-frame prediction mode is used to reconstruct the first of the four sub-blocks (the first sub-block 1211) based on the residual of the sub-block. In the case where the first sub-block is reconstructed and can be used as a neighboring template for the second sub-block 1212, DIMD is performed to determine the intra-frame prediction angle of the second sub-block. The intra-frame prediction angle is then used to reconstruct the second sub-block based on its residual, and so on until all four sub-blocks are reconstructed.
[0121] G. Iterative DIMD
[0122] Since DIMD uses a template (neighboring samples) of the current block to suggest an intra-frame prediction mode for the current block, a more accurate template can suggest a more appropriate intra-frame prediction mode for the current block. Some embodiments of the present disclosure provide an iterative method for improving the DIMD template. To perform the iterative method, the decoder (1) uses DIMD to derive a first intra-frame prediction mode or angle; (2) generates a first prediction of the template (neighboring L-shape of the current block) using the first intra-frame prediction mode; (3) (may or may not) add the residual to the first prediction; (4) uses DIMD to derive a second intra-frame prediction mode or angle by using the template and the first prediction (for example, by using a weighted average of the template and the first prediction).
[0123] At the encoder, the source data is used to obtain the most accurate intra-frame prediction angle ("angle_best"). The encoder then calculates the final predictor through angle_best and uses the final predictor to obtain the final residual. The encoder then performs steps (1)-(4) of the iterative method to derive a second intra-frame prediction mode. If the second intra-frame prediction mode (from step 4) matches angle_best, then the derived second intra-frame prediction mode is valid. If the second intra-frame prediction mode does not match angle_best, the encoder will not select the derived second intra-frame prediction mode.
[0124] H. Reversed Subblock Scan in DIMD Mode
[0125] In some embodiments, when a block is divided into several sub-blocks, the DIMD / TIMD process can be applied to each sub-block in reverse order. The template area of the first sub-block is larger than the original and can be accurate for the first sub-block. In some embodiments, the reverse order is from right to left. Fig.13The DIMD / TIMD is shown applied to the sub-blocks of the block in reverse order. The figure also shows the template used for each sub-block when applying the DIMD / TIMD process.
[0126] In some embodiments, any of the above methods or any combination of the proposed methods can be applied to other intra modes (not limited to TIMD / DIMD), such as normal intra mode, WAIP (wide-angle intra prediction mode), intra angle mode, ISP, MIP, or any intra mode specified in VVC / HEVC. The above methods can be enabled or disabled according to implicit rules (e.g., based on block width, height, or area) or according to explicit rules (e.g., based on block, tile, slice, picture, SPS, or PPS level). For example, according to an explicit CU level or CB level flag, the above multi-region DIMD / TIMD is supported as an optional mode of DIMD / TIMD. If the explicit flag indicates enablement, the above multi-region DIMD / TIMD will be applied to the current block. The signaling (e.g., the enabling conditions or context selection of the signaling) may depend on the codec information, block width, block height, block area and / or the block position, codec information, block width, block height, block area of the current block, and / or the codec information, block width, block height, block area of the neighboring blocks and / or the block position, codec information, block width, block height, block area of the current block. The explicit flag is signaled / parsed in the bitstream only when all enabling conditions of the explicit flag are met. The enabling conditions may include that the block position (cbX, cbY) is not in a boundary case. The boundary case refers to (1) the current block is located at the leftmost and topmost position (0,0) of the current picture, CTU, slice or tile or (2) the current block is located at the leftmost or topmost position of the current picture, CTU, slice or tile. cbX and cbY refer to the block position in the current picture, CTU, slice or tile. The enabling conditions may include that the current block has been selected for TIMD / DIMD encoding and decoding. For another example, according to the implicit rule, the above-mentioned multi-region DIMD / TIMD is supported as a replacement mode of DIMD / TIMD. If the implicit rule is met, the above-mentioned multi-region DIMD / TIMD is applied to the current block (if the current block is coded and decoded with TIMD / DIMD). The implicit rule may depend on the coding and decoding information, block width, block height, block area and / or block position of the current block, the coding and decoding information, block width, block height, block area and / or block position of the adjacent block. For an example of an implicit rule, when the block width, height and / or area is greater than a threshold (e.g., 2, 4, ..., 512, 1024, the maximum transform block size, or any positive integer specified in the standard or indicated in the bitstream), the above-mentioned multi-region DIMD / TIMD is applied. For another example of an implicit rule, when the block position is not in a boundary situation, the above-mentioned multi-region DIMD / TIMD is applied. The term "block" in the present disclosure may refer to TU / TB, CU / CB, PU / PB, a predefined area or CTU / CTB.
[0127] Any combination of the methods proposed in the present invention may be applied. Any of the aforementioned proposed methods may be implemented in an encoder and / or a decoder. For example, any of the proposed methods may be implemented in an inter-frame / intra-frame / prediction module of an encoder and / or an inter-frame / intra-frame / prediction module of a decoder. Alternatively, any of the proposed methods may be implemented as a circuit coupled to an inter-frame / intra-frame / prediction module of an encoder and / or an inter-frame / intra-frame / prediction module of a decoder to provide the information required by the inter-frame / intra-frame / prediction module.
[0128] Example video encoder
[0129] Fig.14 An example video encoder 1400 that can implement region-based implicit intra prediction is shown. As shown, the video encoder 1400 receives an input video signal from a video source 1405 and encodes the signal into a bitstream 1495. The video encoder 1400 has several components or modules for encoding the signal from the video source 1405, including at least some components selected from the following: a transform module 1410, a quantization module 1411, an inverse quantization module 1414, an inverse transform module 1415, an intra-picture estimation module 1420, an intra-frame prediction module 1425, a motion compensation module 1430, a motion estimation module 1435, a loop filter 1445, a reconstructed picture buffer 1450, an MV buffer 1465, an MV prediction module 1475, and an entropy encoder 1490. The motion compensation module 1430 and the motion estimation module 1435 are part of the inter-frame prediction module 1440.
[0130] In some embodiments, modules 1410-1490 are software instruction modules executed by one or more processing units (e.g., processors) of a computing device or electronic device. In some embodiments, modules 1410-1490 are hardware circuit modules implemented by one or more integrated circuits (ICs) of an electronic device. Although modules 1410-1490 are shown as separate modules, some modules can be combined into a single module.
[0131] The video source 1405 provides a raw video signal that presents pixel data for each video frame without compression. A subtractor 1408 calculates the difference between the raw video pixel data of the video source 1405 and the predicted pixel data 1413 from the motion compensation module 1430 or the intra-frame prediction module 1425. The transform module 1410 converts the difference (or residual pixel data or residual signal 1408) into transform coefficients (e.g., by performing a discrete cosine transform, or DCT). The quantization module 1411 quantizes the transform coefficients into quantized data (or quantized coefficients) 1412, which are encoded into a bitstream 1495 by an entropy encoder 1490.
[0132] The inverse quantization module 1414 inversely quantizes the quantized data (or quantized coefficients) 1412 to obtain transform coefficients, and the inverse transform module 1415 inversely transforms the transform coefficients to generate a reconstructed residual 1419. The reconstructed residual 1419 is added to the predicted pixel data 1413 to generate reconstructed pixel data 1417. In some embodiments, the reconstructed pixel data 1417 is temporarily stored in a line buffer (not shown) for intra-frame prediction and spatial MV prediction. The reconstructed pixels are filtered by the loop filter 1445 and stored in the reconstructed picture buffer 1450. In some embodiments, the reconstructed picture buffer 1450 is a memory external to the video encoder 1400. In some embodiments, the reconstructed picture buffer 1450 is an internal memory of the video encoder 1400.
[0133] The intra-picture estimation module 1420 performs intra-prediction based on the reconstructed pixel data 1417 to generate intra-prediction data. The intra-prediction data is provided to the entropy encoder 1490 to be encoded into a bitstream 1495. The intra-prediction data is also used by the intra-prediction module 1425 to generate predicted pixel data 1413.
[0134] The motion estimation module 1435 performs inter-frame prediction by generating MVs to refer to pixel data of a previously decoded frame stored in the reconstructed picture buffer 1450. These MVs are provided to the motion compensation module 1430 to generate predicted pixel data.
[0135] Instead of encoding the complete actual MV in the bitstream, the video encoder 1400 uses MV prediction to generate a predicted MV, and the difference between the MV used for motion compensation and the predicted MV is encoded as residual motion data and stored in the bitstream 1495.
[0136] The MV prediction module 1475 generates a predicted MV based on a reference MV generated for encoding a previous video frame, i.e., a motion compensated MV for performing motion compensation. The MV prediction module 1475 retrieves a reference MV from a previous video frame from the MV buffer 1465. The video encoder 1400 stores the MV generated for the current video frame in the MV buffer 1465 as a reference MV for generating the predicted MV.
[0137] The MV prediction module 1475 creates a predicted MV using a reference MV. The predicted MV can be calculated by spatial MV prediction or temporal MV prediction. The entropy encoder 1490 encodes the difference (residual motion data) between the predicted MV and the motion compensated MV (MC MV) of the current frame into a bitstream 1495.
[0138] The entropy encoder 1490 encodes various parameters and data into a bitstream 1495 by using an entropy coding technique such as context adaptive binary arithmetic coding and decoding (CABAC) or Huffman coding. The entropy encoder 1490 encodes various header elements, flags along with quantized transform coefficients 1412 and residual motion data as syntax elements into the bitstream 1495. The bitstream 1495 is in turn stored in a storage device or transmitted to a decoder through a communication medium such as a network.
[0139] The loop filter 1445 performs a filtering or smoothing operation on the reconstructed pixel data 1417 to reduce encoding and decoding artifacts, particularly at the boundaries of pixel blocks. In some embodiments, the filtering operation performed includes sample adaptive offset (SAO). In some embodiments, the filtering operation includes adaptive loop filtering (ALF).
[0140] Fig.15 1 shows a portion of a video encoder 1400 that implements region-based implicit intra prediction. Specifically, the figure shows components of an intra prediction module 1425 of the video encoder 1400. As shown, the intra prediction module 1425 fetches content from a reconstructed picture buffer 1450, which provides reconstructed pixel data from a region near the current block as a template.
[0141] As shown, the intra prediction module 1425 includes an upper intra prediction module 1510 and a left intra prediction module 1520. The upper intra prediction module 1510 uses samples in the upper template area (e.g., 511) and / or the reference area (e.g., 520) of the current block to calculate the cost of different intra prediction modes or accumulate HoGs of different intra prediction modes. Based on the cost or HoG stored in the ModeA cost / Hog memory 1515, the upper intra prediction module 1510 identifies the ModeA intra prediction mode in the TIMD or DIMD process. Similarly, the left intra prediction module 1520 uses samples in the left template area (e.g., 512) and / or the reference area (e.g., 520) of the current block to calculate the cost or accumulated HoG. Based on the cost or HoG stored in the ModelL cost / HoG memory 1525, the left intra prediction module 1520 identifies the ModelL intra prediction mode in the TIMD or DIMD process.
[0142] The intra prediction mixing module 1530 receives the identified ModeA and ModeL intra prediction modes and generates corresponding predictors based on the contents of the reconstructed picture buffer 1450. The intra prediction mixing module 1530 mixes the two predictors into a weighted sum as a combined prediction. The result of the intra prediction mixing can be used as the predicted pixel data 1413.
[0143] The current block may be a sub-block or grid of a larger block divided into sub-blocks or grids. The intra-prediction blending module 1530 may store the prediction samples along the boundary of the current block in the sub-block prediction memory 1535 for later use in blending with other sub-blocks or grids. In another way, the intra-prediction blending module 1530 may generate a prediction larger in size than the current block (current sub-block or current grid), and may store the prediction samples along the boundary of the current block and outside the current block to the sub-block prediction memory 1535 for later use in blending with other sub-blocks or grids.
[0144] Fig.16 A process 1600 for encoding a pixel block using region-based implicitly derived intra prediction is conceptually illustrated. In some embodiments, one or more processing units (e.g., processors) of a computing device implementing the encoder 1400 perform the process 1600 by executing instructions stored in a computer-readable medium. In some embodiments, an electronic device implementing the encoder 1400 performs the process 1600.
[0145] The encoder receives (at step 1610) data for a current block of pixels in a current picture to be encoded as a video.
[0146] The encoder identifies (at step 1620) an upper template region and a left template region of the current block among already reconstructed pixels of the current picture.
[0147] The encoder (at step 1630) derives a first intra-frame prediction mode based on the upper template region. The encoder (at step 1640) derives a second intra-frame prediction mode based on the left template region. In some embodiments, the first and second intra-frame prediction modes are identified by the TIMD process based on the cost of the candidate intra-frame prediction mode. The cost of the candidate for the first intra-frame prediction mode is calculated based on the reconstructed samples of the upper template region and the predicted samples of the upper template region, wherein the predicted samples of the upper template region are generated by using the reference samples identified by the candidate for the first intra-frame prediction mode. The cost of the candidate for the second intra-frame prediction mode is calculated based on the reconstructed samples of the left template region and the predicted samples of the left template region, wherein the predicted samples of the left template region are generated by using the reference samples identified by the candidate for the second intra-frame prediction mode. A reference sample is identified from a reference region including an area above the upper template region, an area to the left of the left template region, or an area above and to the left of the upper and left template regions.
[0148] In some embodiments, the first and second intra prediction modes are identified by a DIMD process based on gradient histograms (HoGs) at different intra prediction angles. Specifically, the first intra prediction mode is identified based on a first HoG of gradient magnitudes at different pixel positions along the upper template region, and the second intra prediction mode is identified based on a second HoG of gradient magnitudes at different pixel positions along the left template region.
[0149] The encoder generates (at step 1650) first and second predictors of the current block based on the first and second intra prediction modes. The encoder then encodes (at step 1660) the current block using the first and second predictors to generate a prediction residual and reconstruct the current block.
[0150] In some embodiments, the encoder generates a combined intra prediction of the current block by mixing the first predictor and the second predictor, and uses the combined intra prediction to produce a prediction residual for the current block. In some embodiments, the combined prediction is a weighted sum of the first and second predictors, wherein weight values assigned to samples in the current block of the first and second predictors are determined based on distances from an upper template region and from a left template region.
[0151] In some embodiments, a geometrically positioned straight line derived from the angle and offset parameters divides the current block into a first partition and a second partition. The first predictor is used to reconstruct the first partition, the second predictor is used to reconstruct the second partition, and samples along a boundary between the first partition and the second partition are reconstructed by using combined intra prediction.
[0152] In some embodiments, the current block is a first sub-block of a plurality of sub-blocks of a larger block, and the upper template region is a sub-template of a plurality of sub-templates above the larger block, and the left template region is a sub-template of a plurality of sub-templates to the left of the larger block. In some embodiments, samples along a boundary between the first sub-block and the second sub-block are reconstructed using a hybrid prediction that is a weighted sum of (i) a combined intra-prediction of the current block and (ii) an intra-prediction of the current block or an intra-prediction of the second sub-block generated using an intra-prediction mode of the second sub-block, wherein the second sub-block is adjacent to the first sub-block (the current block). The intra-prediction of the second sub-block is derived from the third and fourth intra-prediction modes.
[0153] VI. Example Video Decoder
[0154] In some embodiments, an encoder may signal (or generate) one or more syntax elements in a bitstream such that a decoder may parse the one or more syntax elements from the bitstream.
[0155] Fig.17An example video decoder 1700 that can implement region-based implicit intra prediction is illustrated. As shown, the video decoder 1700 is an image decoding or video decoding circuit that receives a bitstream 1795 and decodes the content of the bitstream into pixel data of a video frame for display. The video decoder 1700 has several components or modules for decoding the bitstream 1795, including some components selected from an inverse quantization module 1711, an inverse transform module 1710, an intra prediction module 1725, a motion compensation module 1730, a loop filter 1745, a decoded picture buffer 1750, an MV buffer 1765, an MV prediction module 1775, and a parser 1790. The motion compensation module 1730 is part of the inter prediction module 1740.
[0156] In some embodiments, modules 1710-1790 are software instruction modules executed by one or more processing units (e.g., processors) of a computing device. In some embodiments, modules 1710-1790 are hardware circuit modules implemented by one or more ICs of an electronic device. Although modules 1710-1790 are illustrated as separate modules, some modules may be combined into a single module.
[0157] The parser 1790 (or entropy decoder) receives the bitstream 1795 and performs initial parsing according to the syntax defined by the video codec or image codec standard. The parsed syntax elements include various header elements, flags, and quantized data (or quantized coefficients) 1712. The parser 1790 parses out the various syntax elements by using entropy coding techniques such as context adaptive binary algorithm (CABAC) or Huffman coding.
[0158] The inverse quantization module 1711 dequantizes the quantized data (or quantized coefficients) 1712 to obtain transform coefficients, and the inverse transform module 1710 performs inverse transform on the transform coefficients 1716 to generate a reconstructed residual signal 1719. The reconstructed residual signal 1719 is added to the predicted pixel data 1713 from the intra prediction module 1725 or the motion compensation module 1730 to generate decoded pixel data 1717. The decoded pixel data is filtered by the loop filter 1745 and stored in the decoded picture buffer 1750. In some embodiments, the decoded picture buffer 1750 is a storage component external to the video decoder 1700. In some embodiments, the decoded picture buffer 1750 is a component internal to the video decoder 1700.
[0159] The intra prediction module 1725 receives intra prediction data from the bitstream 1795 and generates predicted pixel data 1713 from the decoded pixel data 1717 stored in the decoded picture buffer 1750. In some embodiments, the decoded pixel data 1717 is also stored in a line buffer (not shown) for intra picture prediction and spatial MV prediction.
[0160] In some embodiments, the contents of decoded picture buffer 1750 are used for display. Display device 1755 either fetches the contents of decoded picture buffer 1750 for direct display or fetches the contents of decoded picture buffer to a display buffer. In some embodiments, the display device receives pixel values from decoded picture buffer 1750 via pixel transfer.
[0161] The motion compensation module 1730 generates predicted pixel data 1713 from the decoded pixel data 1717 stored in the decoded picture buffer 1750 according to the motion compensation MV (MC MV). These motion compensation MVs are decoded by adding the residual motion data received from the bitstream 1795 and the predicted MV received from the MV prediction module 1775.
[0162] The MV prediction module 1775 generates a predicted MV based on a reference MV generated for decoding a previous video frame, for example, a motion compensated MV for performing motion compensation. The MV prediction module 1775 retrieves the reference MV of the previous video frame from the MV buffer 1765. The video decoder 1700 stores the motion compensated MV generated for decoding the current video frame in the MV buffer 1765 as a reference MV for generating the predicted MV.
[0163] The loop filter 1745 performs filtering or smoothing operations on the decoded pixel data 1717 to reduce encoding and decoding artifacts, particularly at the boundaries of pixel blocks. In some embodiments, the filtering operations performed include sample adaptive offset (SAO). In some embodiments, the filtering operations include adaptive loop filtering (ALF).
[0164] Fig.18 17 shows a portion of a video decoder 1700 that implements region-based implicit intra prediction. Specifically, the figure shows components of an intra prediction module 1725 of the video decoder 1700. As shown, the intra prediction module 1725 fetches content from a decoded picture buffer 1750, which provides reconstructed pixel data from a region near the current block as a template.
[0165] As shown, the intra prediction module 1725 includes an upper intra prediction module 1810 and a left intra prediction module 1820. The upper intra prediction module 1810 uses samples in the upper template area (e.g., 511) and / or the reference area (e.g., 520) of the current block to calculate the cost of different intra prediction modes or accumulate HoGs of different intra prediction modes. Based on the cost or HoG stored in the ModeA cost / Hog memory 1815, the upper intra prediction module 1810 identifies the ModeA intra prediction mode in the TIMD or DIMD process. Similarly, the left intra prediction module 1820 uses samples in the left template area (e.g., 512) and / or the reference area (e.g., 520) of the current block to calculate the cost or accumulated HoG. Based on the cost or HoG stored in the ModelL cost / HoG memory 1825, the left intra prediction module 1820 identifies the ModelL intra prediction mode in the TIMD or DIMD process.
[0166] The intra prediction mixing module 1830 receives the identified ModeA and ModeL intra prediction modes and generates corresponding predictors based on the contents of the reconstructed picture buffer 1750. The intra prediction mixing module 1830 mixes the two predictors into a weighted sum as a combined prediction. The result of the intra prediction mixing can be used as the predicted pixel data 1713.
[0167] The current block may be a sub-block or grid of a larger block divided into sub-blocks or grids. The intra-prediction blending module 1830 may store the prediction samples along the boundary of the current block in the sub-block prediction memory 1835 for later use in blending with other sub-blocks or grids. In another way, the intra-prediction blending module 1830 may generate a prediction larger in size than the current block (current sub-block or current grid), and may store the prediction samples along the boundary of the current block and outside the current block to the sub-block prediction memory 1835 for later use in blending with other sub-blocks or grids.
[0168] Fig.19 A process 1900 for decoding a pixel block using region-based implicitly derived intra prediction is conceptually illustrated. In some embodiments, one or more processing units (e.g., processors) of a computing device implementing the decoder 1700 perform the process 1900 by executing instructions stored in a computer-readable medium. In some embodiments, an electronic device implementing the decoder 1700 performs the process 1900.
[0169] The decoder receives (at step 1910) data for a current block of pixels in a current picture to be decoded as a video.
[0170] The decoder identifies (at step 1920) an upper template region and a left template region of the current block among already reconstructed pixels of the current picture.
[0171] The decoder (at step 1930) derives a first intra-frame prediction mode based on the upper template region. The decoder (at step 1940) derives a second intra-frame prediction mode based on the left template region. In some embodiments, the first and second intra-frame prediction modes are identified by the TIMD process based on the cost of the candidate intra-frame prediction mode. The cost of the candidate of the first intra-frame prediction mode is calculated based on the reconstructed samples of the upper template region and the predicted samples of the upper template region, wherein the predicted samples of the upper template region are generated by using the reference samples identified by the candidate of the first intra-frame prediction mode. The cost of the candidate of the second intra-frame prediction mode is calculated based on the reconstructed samples of the left template region and the predicted samples of the left template region, wherein the predicted samples of the left template region are generated by using the reference samples identified by the candidate of the second intra-frame prediction mode. A reference sample is identified from a reference region including an area above the upper template region, an area to the left of the left template region, or an area above and to the left of the upper and left template regions.
[0172] In some embodiments, the first and second intra prediction modes are identified by a DIMD process based on gradient histograms (HoGs) at different intra prediction angles. Specifically, the first intra prediction mode is identified based on a first HoG of gradient magnitudes at different pixel positions along the upper template region, and the second intra prediction mode is identified based on a second HoG of gradient magnitudes at different pixel positions along the left template region.
[0173] The decoder generates (at step 1950) first and second predictors for the current block based on the first and second intra prediction modes.
[0174] The decoder then reconstructs the current block (at step 1960) by using the first and second predictors. The decoder may then provide the reconstructed current block for display as part of a reconstructed current picture.
[0175] In some embodiments, the encoder generates a combined intra prediction of the current block by mixing the first predictor and the second predictor, and reconstructs the current block using the combined intra prediction. In some embodiments, the combined prediction is a weighted sum of the first and second predictors, wherein weight values assigned to samples in the current block of the first and second predictors are determined based on distances from an upper template region and from a left template region.
[0176] In some embodiments, a geometrically positioned straight line derived from the angle and offset parameters divides the current block into a first partition and a second partition. The first predictor is used to reconstruct the first partition, the second predictor is used to reconstruct the second partition, and samples along a boundary between the first partition and the second partition are reconstructed by using combined intra prediction.
[0177] In some embodiments, the current block is a first sub-block of a plurality of sub-blocks of a larger block, and the upper template region is a sub-template of a plurality of sub-templates above the larger block, and the left template region is a sub-template of a plurality of sub-templates to the left of the larger block. In some embodiments, samples along a boundary between the first sub-block and the second sub-block are reconstructed using a hybrid prediction that is a weighted sum of (i) a combined intra-prediction of the current block and (ii) an intra-prediction of the current block or an intra-prediction of the second sub-block generated using an intra-prediction mode of the second sub-block, wherein the second sub-block is adjacent to the first sub-block (the current block). The intra-prediction of the second sub-block is derived from the third and fourth intra-prediction modes.
[0178] VII. Example Electronic Systems
[0179] Many of the above features and applications are implemented as software processes, which are specified as a set of instructions recorded on a computer-readable storage medium (also referred to as a computer-readable medium). When these instructions are executed by one or more computing or processing units (e.g., one or more processors, processor cores, or other processing units), they cause the processing units to perform the actions indicated in the instructions. Examples of computer-readable media include, but are not limited to, CD-ROMs, flash drives, random access memory (RAM) chips, hard drives, erasable programmable read-only memories (EPROMs), electrically erasable programmable read-only memories (EEPROMs), etc. Computer-readable media do not include carrier waves and electronic signals transmitted wirelessly or via wired connections.
[0180] In this specification, the term "software" is meant to include firmware residing in a read-only memory or an application stored in a magnetic storage device that can be read into the memory for processing by the processor. In addition, in some embodiments, multiple software inventions can be implemented as sub-parts of a larger program while retaining different software inventions. In some embodiments, multiple software inventions can also be implemented as separate programs. Finally, any combination of separate programs that implement the software inventions described herein together are within the scope of this disclosure. In some embodiments, when the software program is installed to run on one or more electronic systems, one or more specific machine implementations for executing and performing the operations of the software program are defined.
[0181] Fig. 20An electronic system 2000 for implementing some embodiments of the present disclosure is conceptually illustrated. The electronic system 2000 may be a computer (e.g., a desktop computer, a personal computer, a tablet computer, etc.), a phone, a PDA, or any other type of electronic device. Such an electronic system includes various types of computer-readable media and interfaces for various other types of computer-readable media. The electronic system 2000 includes a bus 2005, a processing unit 2010, a graphics processing unit (GPU) 2015, a system memory 2020, a network 2025, a read-only memory 2030, a permanent storage device 2035, an input device 2040, and an output device 2045.
[0182] Bus 2005 collectively represents all system, peripheral, and chipset buses that communicatively connect numerous internal devices of electronic system 2000. For example, bus 2005 communicatively connects processing unit 2010 with GPU 2015, read-only memory 2030, system memory 2020, and permanent storage device 2035.
[0183] From these various memory units, the processing unit 2010 retrieves instructions to be executed and data to be processed in order to perform the processes of the present disclosure. In different embodiments, the processing unit may be a single processor or a multi-core processor. Some instructions are passed to and executed by the GPU 2015. The GPU 2015 may offload various calculations or supplement the image processing provided by the processing unit 2010.
[0184] The read-only memory (ROM) 2030 stores static data and instructions used by the processing unit 2010 and other modules of the electronic system. On the other hand, the permanent storage device 2035 is a read-write storage device. This device is a non-volatile storage unit that stores instructions and data even when the electronic system 2000 is turned off. Some embodiments of the present disclosure use a mass storage device (such as a magnetic disk or optical disc and its corresponding disk drive) as the permanent storage device 2035.
[0185] Other embodiments use removable storage devices (e.g., floppy disks, flash memory devices, etc., and their corresponding disk drives) as permanent storage devices. Like the permanent storage device 2035, the system memory 2020 is a read-write storage device. However, unlike the storage device 2035, the system memory 2020 is a volatile read-write memory, such as a random access memory. The system memory 2020 stores some instructions and data used by the processor at runtime. In some embodiments, the processes according to the present disclosure are stored in the system memory 2020, the permanent storage device 2035 and / or the read-only memory 2030. For example, various memory units include instructions for processing multimedia clips and some embodiments. From these different memory units, the processing unit 2010 retrieves instructions to be executed and data to be processed in order to execute the processes of some embodiments.
[0186] The bus 2005 is also connected to input and output devices 2040 and 2045. The input device 2040 enables a user to transmit information and select commands to the electronic system. The input device 2040 includes an alphanumeric keyboard and a pointing device (also referred to as a "mouse control device"), a camera (e.g., a webcam), a microphone or a similar device for receiving voice commands, etc. The output device 2045 displays images generated by the electronic system or otherwise outputs data. The output device 2045 includes a printer and a display device, such as a cathode ray tube (CRT) or a liquid crystal display (LCD), and a speaker or similar audio output device. Some embodiments include devices that are used as both input and output devices, such as a touch screen.
[0187] Finally, if Fig. 20 As shown, bus 2005 also couples electronic system 2000 to a network 2025 via a network adapter (not shown). In this manner, the computer may be part of a computer network (e.g., a local area network ("LAN"), a wide area network ("WAN"), or an intranet, or a network of networks. For example, any or all components of electronic system 2000 may be used in conjunction with the present disclosure.
[0188] Some embodiments include electronic components, such as microprocessors, storage devices, and memories, which store computer program instructions in machine-readable or computer-readable media (or alternatively referred to as computer-readable storage media, machine-readable media, or machine-readable storage media). Some examples of such computer-readable media include RAM, ROM, compact disc-read only (CD-ROM), compact disc-recordable (CD-R), compact disc-rewritable (CD-RW), read-only versatile discs (e.g., DVD-ROM, dual-layer DVD-ROM), various recordable / rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (e.g., SD card, mini-SD card, micro SD card, etc.), magnetic and / or solid-state hard drives, read-only and recordable Blu-ray discs, and the like. CDs, ultra-density CDs, any other optical or magnetic media, and floppy disks. Computer-readable media can store a computer program that is executable by at least one processing unit and includes a set of instructions for performing various operations. Examples of computer programs or computer code include machine code, such as that generated by a compiler, and files including high-level code that is executed by a computer, electronic component, or microprocessor using an interpreter.
[0189] Although the above discussion primarily involves microprocessors or multi-core processors executing software, many of the above features and applications are performed by one or more integrated circuits, such as application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions stored on the circuits themselves. In addition, some embodiments execute software stored in programmable logic devices (PLDs), ROM, or RAM devices.
[0190] As used in this specification and any claims of this application, the terms "computer", "server", "processor" and "memory" refer to electronic or other technical devices. These terms do not include people or groups of people. For the purposes of this specification, the terms display or display means display on an electronic device. As used in this specification and any claims of this application, the terms "computer-readable medium", "computer-readable storage medium" and "machine-readable medium" are entirely limited to tangible physical objects that store information in a readable form. These terms do not include any wireless signals, wired download signals, and any other temporary signals.
[0191] Although the present disclosure has been described with reference to many specific details, those skilled in the art will recognize that the present disclosure may be implemented in other specific forms without departing from the spirit of the present disclosure. Fig.11 and Fig.14) conceptually illustrate the processes. The specific operations of these processes may not be performed in the exact order shown and described. Specific operations may not be performed in a continuous series of operations, and different specific operations may be performed in different embodiments. In addition, the process can be implemented using multiple sub-processes or as part of a larger macro process. Therefore, those skilled in the art will understand that the present disclosure is not limited by the foregoing illustrative details, but is defined by the appended claims.
[0192] The subject matter described herein sometimes illustrates that different components are contained within or connected to different other components. It should be understood that the architectures depicted in this way are merely examples, and that many other architectures that implement the same functionality can actually be implemented. Conceptually, any arrangement of components that implement the same functionality is effectively "associated" so that the desired functionality is implemented. Therefore, any two components combined here to implement a specific functionality can be considered to be "associated" with each other so that the desired functionality is implemented, regardless of the architecture or intermediate components. Similarly, any two components so associated can also be considered to be "operably connected" or "operably coupled" to each other to achieve the desired functionality, and any two components that can be so associated can also be considered to be "operably connected" or "coupled" to each other to achieve the desired functionality. Specific examples of operably coupled include, but are not limited to, physically matable and / or physically interactive components and / or wirelessly interactive and / or wirelessly interactive components and / or logically interactive and / or logically interactive components.
[0193] In addition, with respect to the use of substantially any plural and / or singular terms herein, those skilled in the art can translate from the plural to the singular and / or from the singular to the plural and / or apply according to the context. For clarity, various singular / plural arrangements may be explicitly set forth herein.
[0194] In addition, those skilled in the art will understand that, in general, the terms used herein, especially in the appended claims, such as the bodies of the appended claims, are generally intended as "open" terms, for example, the word "including" should be interpreted as "including but not limited to", the word "having" should be interpreted as "at least having", the word "including" should be interpreted as "including but not limited to", etc. Those skilled in the art will further understand that if a specific number of claim recitations is intended to be introduced, such intent will be explicitly stated in the claim, and in the absence of such statement, such intent is absent. For example, to aid understanding, the following appended claims may contain the use of the introductory phrases "at least one" and "one or more" to introduce claim recitations. However, the use of such phrases should not be interpreted as implying that a claim introduced by the indefinite article "a" or "an" limits any particular claim containing such introduced claim to implementations containing only one such statement, even when the same claim includes the introductory phrases "one or more" or "at least one" and an indefinite article such as "a" or "an", for example, "a" and / or "an" should be interpreted as "at least" one or "one or more"; the same applies to the use of definite articles to introduce claim statements. In addition, even if a specific number of an introduced claim recitation is explicitly cited, a person skilled in the art will recognize that such recitation should be interpreted as indicating at least the number of citations, for example, "two recitations", without other modifiers, means at least two citations, or two or more citations. In addition, in those cases where the convention is similar to "at least one of A, B, and C, etc.", generally, such construction is intended to be understood by those skilled in the art in the sense of the convention, for example, "a system having at least one of A, B, and C" will include but is not limited to such systems having A alone, B alone, C alone, A and B together, A and C together, B and C together, and / or A, B, and C together, etc. In those cases where the convention is similar to "at least one" is used, generally, such construction is intended to be understood by those skilled in the art in the sense of the convention, for example, "a system having at least one of A, B, or C" will include but is not limited to such systems having A alone, B alone, C alone, A and B together, A and C together, B and C together, and / or A, B, and C together, etc. Those skilled in the art will further understand that in fact, whether in the specification, claims, or drawings, any disjunctive words and / or phrases that appear with two or more alternative terms should be understood to contemplate the possibility of including one term, one term, or both terms. For example, the phrase "A or B" will be understood to include the possibility of "A" or "B" or "A and B".
[0195] It can be understood from the above that various embodiments of the present disclosure have been described herein for illustrative purposes, and various modifications can be made without departing from the scope and spirit of the present disclosure. Therefore, the various embodiments disclosed herein are not intended to be limiting, and the true scope and spirit are indicated by the appended claims.
Claims
1. A video encoding and decoding method, comprising: Receiving data of a pixel block to be encoded or decoded as a current block of a current picture of a video; Identify an upper template region and a left template region of the current block in reconstructed pixels of the current picture; deriving a first intra-frame prediction mode based on the upper template area; deriving a second intra-frame prediction mode based on the left template area; generating first and second predictors of the current block based on the first intra prediction mode and the second intra prediction mode; as well as The current block is reconstructed by encoding or decoding the current block using the first predictor and the second predictor.
2. The video encoding and decoding method according to claim 1, characterized in that: identifying the first intra prediction mode and the second intra prediction mode based on costs of candidate intra prediction modes, calculating the cost of the candidate for the first intra prediction mode based on the reconstructed samples of the upper template region and the predicted samples of the upper template region, wherein the predicted samples of the upper template region are generated by using the reference samples identified by the candidate for the first intra prediction mode, The cost of the candidate for the second intra prediction mode is calculated based on the reconstructed samples of the left template region and the predicted samples of the left template region, wherein the predicted samples of the left template region are generated by using the reference samples identified by the candidate for the second intra prediction mode.
3. The video encoding and decoding method according to claim 1, characterized in that: The reference sample is identified from a reference region including a region above the upper template region, a region to the left of the left template region, or regions above and to the left of the upper template region and the left template region.
4. The video encoding and decoding method according to claim 1, characterized in that: identifying the first intra prediction mode based on first gradient histograms for different intra prediction angles, the first gradient histogram being based on gradient magnitudes at different pixel positions along the upper template region, The second intra prediction mode is identified based on a second gradient histogram for a different intra prediction angle, the second gradient histogram being based on gradient magnitudes at different pixel locations along the left template region.
5. The video encoding and decoding method according to claim 1, characterized in that: Also includes: generating a combined intra prediction of the current block by mixing the first predictor and the second predictor; as well as The current block is reconstructed using the combined intra prediction.
6. The video encoding and decoding method according to claim 5, characterized in that: A geometrically positioned straight line derived from the angle and offset parameters divides the current block into a first partition and a second partition, wherein the first predictor is used to reconstruct the first partition and the second predictor is used to reconstruct the second partition, wherein samples along a boundary between the first partition and the second partition are reconstructed by using the combined intra-frame prediction.
7. The video encoding and decoding method according to claim 5, characterized in that: The current block is the first sub-block of a plurality of sub-blocks of a larger block, The upper template area is a sub-template among the multiple sub-templates above the larger block. The left template area is a sub-template among the multiple sub-templates on the left side of the larger block.
8. The video encoding and decoding method according to claim 7, characterized in that: Reconstructing samples along a boundary between the first subblock and the second subblock using a hybrid prediction, the hybrid prediction being a weighted sum of (i) the combined intra prediction of the current block and (ii) an intra prediction of the current block or an intra prediction of the second subblock generated using an intra prediction mode of the second subblock, wherein the second subblock is adjacent to the current block, wherein the intra prediction of the second subblock is derived from third and fourth intra prediction modes, the third and fourth intra prediction modes being different from the first and second intra prediction modes.
9. The video encoding and decoding method according to claim 5, characterized in that: The combined intra prediction is a weighted sum of the first predictor and the second predictor, wherein weight values assigned to the first predictor and the second predictor are determined based on distances from the upper template region and from the left template region.
10. An electronic device comprising: Receiving data of a pixel block to be encoded or decoded as a current block of a current picture of a video; Identify an upper template region and a left template region of the current block in reconstructed pixels of the current picture; deriving a first intra-frame prediction mode based on the upper template area; deriving a second intra-frame prediction mode based on the left template area; generating first and second predictors of the current block based on the first intra prediction mode and the second intra prediction mode; as well as The current block is reconstructed by encoding or decoding the current block using the first predictor and the second predictor.
11. A video decoding method, comprising: Receiving data of a pixel block to be decoded as a current block of a current picture of a video; Identify an upper template region and a left template region of the current block in reconstructed pixels of the current picture; deriving a first intra-frame prediction mode based on the upper template area; deriving a second intra-frame prediction mode based on the left template area; generating first and second predictors of the current block based on the first intra prediction mode and the second intra prediction mode; as well as The current block is reconstructed by using the first predictor and the second predictor.
12. A video encoding method, comprising: Receiving data of a pixel block to be encoded as a current block of a current picture of a video; Identify an upper template region and a left template region of the current block in reconstructed pixels of the current picture; deriving a first intra-frame prediction mode based on the upper template area; deriving a second intra-frame prediction mode based on the left template area; Generate a first predictor and a second predictor of the current block based on the first intra prediction mode and the second intra prediction mode; as well as The current block is reconstructed by encoding the current block using the first predictor and the second predictor to generate a residual.
Citation Information
Cited By
Encoding method, decoding method, bitstream, encoder, decoder, and storage medium
WO2026143376A1
