Electronic equipment and video coding and decoding method

Abstract

A video coding and decoding method using decoder-side intra mode derivation (DIMD) based on a plurality of histogram of gradient (HoG) is provided. The video codec receives data to be encoded or decoded as a current block of a current picture of a video. The video codec derives a plurality of gradient histograms for different intra prediction angles, wherein the different gradient histograms are derived based on different sets of reconstructed pixels adjacent to the current block. The video codec selects a gradient histogram from a plurality of gradient histograms. The video codec identifies one or more intra prediction modes based on the selected histogram of gradients. The video codec generates an intra prediction of the current block based on the identified one or more intra prediction modes. The video codec encodes or decodes the current block using the generated intra prediction.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This article claims priority to U.S. Provisional Application No. 63 / 351,505 filed on June 13, 2022 and U.S. Provisional Application No. 63 / 369,526 filed on July 27, 2022, the subject matter of which is incorporated herein by reference. Technical Field

[0003] The present disclosure relates to video coding, and more particularly to a method for coding pixel blocks via intra-frame mode. Background Art

[0004] Unless otherwise indicated in this disclosure, the approaches described in this section are not prior art to the claims listed later and are not admitted to be prior art by inclusion in this section.

[0005] High-efficiency video coding (HEVC) is an international video coding standard developed by the Joint Collaborative Team on Video Coding (JCT-VC). HEVC is a block-based discrete cosine transform (DCT)-like hybrid motion compensation coding and decoding architecture. The basic unit of compression is called a coding unit (CU), which is a square block of 2Nx2N pixels (pixel block), and each CU can be continuously divided into smaller CU units until a predefined minimum size is reached. Each CU includes one or more prediction units (PU).

[0006] Versatile video coding (VVC) is the latest video coding standard developed by the Joint Video Expert Team (JVET) of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11. The input video signal is predicted by a reconstruction signal from the coded picture area. The prediction residual signal is processed by block transform. The transform coefficients are quantized and entropy coded in the bitstream along with other information. The reconstructed signal is generated by inverse transformation of the prediction signal and the reconstructed residual signal by de-quantizing the transform coefficients. The reconstructed signal is further processed by in-loop filtering to remove coding artifacts. The decoded picture is stored in a frame buffer for prediction of subsequent pictures of the input video signal.

[0007] In VVC, a coded picture is divided into non-overlapping square block areas, which are represented by related coding tree units (CTUs). A plurality of leaf nodes on a coding tree correspond to the coding units (CUs). A coded picture can be represented as a set of slices, each slice corresponding to a positive integer number of CTUs. Independent CTUs in a slice are processed in raster-scan order. A bidirectional prediction (B) slice can be decoded using intra prediction or inter prediction and up to two motion vectors and reference indices to predict the sample values ​​of each block. A prediction (P) slice is decoded using intra prediction or inter prediction and up to one motion vector and reference indices to predict the sample values ​​of each block. An intra (I) slice is decoded only using intra prediction.

[0008] A CTU can be allocated into one or more non-overlapping coding units (CUs) using a nested multi-type-tree (MTT) structured quadtree (QT) to accommodate a variety of local dynamic and texture characteristics. A CU can be split into smaller CUs in five ways: quadtree partitioning, vertical binary tree partitioning, horizontal binary tree partitioning, vertical center-side ternary tree partitioning, and horizontal center-side ternary tree partitioning.

[0009] Each CU includes one or more prediction units (PUs). The above prediction units and the associated CU syntax form a basic unit for signaling the predictor information. A specified prediction procedure is used to predict the values ​​of the relevant pixel samples within the PU. Each CU may include one or more transform units (TUs) to represent prediction residual blocks. A transform unit (TU) includes a transform block (TB) of luma samples and two transform blocks of corresponding chroma samples, and each TB corresponds to a residual block of samples of a single color component. A positive integer transform is applied to a transform block. The level values ​​of the quantization coefficients are entropy encoded and decoded in the bitstream together with other information. Among them, coding tree block (CTB), coding block (CB), prediction block (PB) and transform block (TB) are defined to specify a two-dimensional sample array of a single color component associated with CTU, CU, PU and TU respectively. Therefore, a CTU is composed of a luma CTB, two chroma CTBs, and related syntax elements. The same can be said for CU, PU, ​​and TU.

[0010] For each inter-prediction CU, dynamic parameters consisting of a motion vector, a reference picture index, and a reference picture list usage index, and other additional information are used to generate inter-prediction samples. The above dynamic parameters can be signaled in an explicit or implicit way. When a CU is encoded in skip mode, the above CU is associated with a PU and has no significant residual coefficients, coded motion vector delta or reference picture index. If the above dynamic parameters of the current CU come from neighboring CUs (including temporal and spatial neighbors) and the additional schedule brought by VVC, a merge mode will be specified. The above merge mode can be applied to any inter-prediction CU. An alternative to the above merge mode is the explicit transmission of dynamic parameters, where the motion vector, the corresponding reference picture index of each reference picture list, the reference picture list usage flag and other necessary information are explicitly passed to each CU. Summary of the invention

[0011] The following summary of the invention is illustrative only and is not intended to be limiting in any way. That is, the following summary of the invention is only intended to introduce the concepts, highlights, benefits and advantages of the novel and non-obvious technologies described herein. Selected and not all embodiments are further described in the following detailed description. Therefore, the following summary of the invention is not intended to indicate the essential features of the claimed subject matter, nor is it intended to be used to determine the claimed subject matter.

[0012] The present disclosure provides a video encoding and decoding method, comprising: receiving data to be encoded or decoded as a current block of a current picture of a video; deriving a plurality of gradient histograms for different intra-frame prediction angles, wherein different gradient histograms are derived based on different groups of reconstructed pixels adjacent to the current block; selecting a gradient histogram from the plurality of gradient histograms; identifying one or more intra-frame prediction modes based on the selected gradient histogram; generating an intra-frame prediction of the current block based on the identified one or more intra-frame prediction modes; and encoding or decoding the current block using the generated intra-frame prediction.

[0013] In some embodiments, a first histogram of gradient (HoG) is derived based on reconstructed pixels above the current block, a second HoG is derived based on reconstructed pixels on the left side of the current block, and a third HoG is derived based on reconstructed pixels on the left and above the current block. In some embodiments, the multiple HoGs include HoGs derived using different numbers of adjacent lines of the current block.

[0014] In some embodiments, the selection of the HoG is based on a syntax element signaled in the bitstream. In some embodiments, the encoder identifies one or more intra prediction modes from the HoG by identifying the N candidate intra prediction modes with the highest amplitude in the HoG and then selecting M intra prediction modes from the N candidate intra prediction modes based on cost. In some embodiments, the reconstructed pixel at the upper left corner of the current block is not used to derive the third HoG.

[0015] In some embodiments, the encoder calculates a cost of an intra prediction mode identified based on a plurality of HoGs, and selects a HoG from the plurality of HoGs based on the calculated cost. In some embodiments, the cost of the intra prediction mode is a TM cost calculated by comparing a difference between a template reconstructed sample and a template prediction sample adjacent to the current block, the template prediction sample generated by the intra prediction mode using an L-shaped reference region near the template.

[0016] In some embodiments, the plurality of HoGs include HoGs derived based on different neighboring lines of the current block. Different neighboring lines are associated with different HoG generation indexes. In some embodiments, if a first intra prediction mode is derived using a HoG derived based on a neighboring line associated with a first HoG generation index, intra prediction of the current block based on the first intra prediction mode is based on an L-shaped reference region identified according to the first HoG generation index.

[0017] The present disclosure also provides an electronic device, including: a video codec circuit, configured to perform the following operations: receiving data to be encoded or decoded as a current block of a current picture of a video; deriving a plurality of gradient histograms for different intra-frame prediction angles, wherein different gradient histograms are derived based on different groups of reconstructed pixels adjacent to the current block; selecting a gradient histogram from the plurality of gradient histograms; identifying one or more intra-frame prediction modes based on the selected gradient histogram; generating an intra-frame prediction of the current block based on the identified one or more intra-frame prediction modes; and encoding or decoding the current block using the generated intra-frame prediction.

[0018] The present disclosure also provides a video decoding method, comprising: receiving data to be decoded as a current block of a current picture of a video; deriving a plurality of gradient histograms for different intra-frame prediction angles, wherein different gradient histograms are derived based on different groups of reconstructed pixels adjacent to the current block; selecting a gradient histogram from the plurality of gradient histograms; identifying one or more intra-frame prediction modes based on the selected gradient histogram; generating an intra-frame prediction of the current block based on the identified one or more intra-frame prediction modes; and reconstructing the current block using the generated intra-frame prediction.

[0019] The present disclosure also provides a video encoding method, comprising: receiving data to be encoded into a bit stream as a current block of a current picture of a video; deriving a plurality of gradient histograms for different intra-frame prediction angles, wherein different gradient histograms are derived based on different groups of reconstructed pixels adjacent to the current block; selecting a gradient histogram from the plurality of gradient histograms; identifying one or more intra-frame prediction modes based on the selected gradient histogram; generating an intra-frame prediction of the current block based on the identified one or more intra-frame prediction modes; and encoding the current block using the generated intra-frame prediction to generate a prediction residual. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The following drawings are included to provide a further understanding of the present disclosure and are incorporated into and constitute a part of the present disclosure. The drawings illustrate embodiments of the present disclosure and are used together with the description thereof to explain the principles of the present disclosure. It is noted that the drawings are not necessarily drawn to scale, as some components may be shown as being not proportional to the size in the actual embodiment in order to clearly illustrate the concepts of the present disclosure.

[0021] Figure 1 Different directional intra prediction modes are shown.

[0022] Figure 2A-2B Conceptual illustration of top and left reference templates with extended lengths to support wide-angle directional mode for non-square blocks of different aspect ratios.

[0023] Figure 3 It is shown that the intra prediction mode of the current block 300 is indirectly derived using template-based intra mode derivation (TIMD).

[0024] Figure 4 It is indicated that decoder-side intra mode derivation (DIMD) is used to indirectly derive the intra prediction mode of the current block.

[0025] Figures 5A-5C Conceptually illustrated are DIMD intra modes derived from a plurality of different HoGs for the current block.

[0026] Figures 6A-6C The DIMD process of some embodiments is shown when additional neighboring lines are present.

[0027] Figure 7A-7B It shows how to identify the reference area for generating intra prediction based on the neighboring lines used to derive the DIMD intra mode of the current block.

[0028] Figures 8A-8B The position of the upper left corner is shown eliminated from the HoG accumulation.

[0029] Fig. 9 An example video encoder that may implement DIMD intra prediction is shown.

[0030] Fig.10 A portion of a video encoder implementing DIMD based on multiple HoGs is shown.

[0031] Fig.11 The process of using multiple HoGs to perform intra prediction on the current block is conceptually illustrated.

[0032] Fig.12 An example video decoder that can implement DIMD intra prediction is shown.

[0033] Fig.13 A portion of a video decoder implementing DIMD based on multiple HoGs is shown.

[0034] Fig.14 The process of using multiple HoGs to perform intra prediction on the current block is conceptually illustrated.

[0035] Fig.15 An electronic system for implementing some embodiments of the present disclosure is conceptually illustrated. DETAILED DESCRIPTION

[0036] In the following detailed description, many specific details are set forth by way of example in order to provide a thorough understanding of the relevant teachings. Any variations, derivations, and / or extensions based on the teachings described herein are within the scope of protection of the present disclosure. In some cases, well-known methods, procedures, components, and / or circuits associated with one or more implementation examples disclosed herein are described at a relatively high level without details to avoid unnecessarily obscuring the embodiments of the teachings of the present disclosure.

[0037] 1. Intra-frame prediction mode

[0038] The intra prediction method uses a reference tier adjacent to the current prediction unit (PU) and one of a plurality of intra prediction modes to generate a predictor for the current prediction unit. The direction of the intra prediction can be selected from a set of modes, which include a plurality of prediction directions. For each PU encoded using intra prediction, an index is used and encoded to select one of the intra prediction modes, thereby generating the corresponding prediction, and then the residual can be derived and transformed.

[0039] Figure 1 Different directional intra prediction modes are shown. These intra prediction modes are called directional modes and are not included in DC mode or planar mode. Figure 1There are 33 directional modes (V: vertical direction; H: horizontal direction), so H, H+1~H+8, H-1~H-7, V, V+1~V+8, V-1~V-8 are used for labeling. General directional modes can be represented as H+k mode or V+k mode, where k=±1, ±2,...±8. Each of the above intra-frame prediction modes can also be called an intra-frame prediction angle. In order to capture the random edge directions shown in naturally generated movies, the number of directional intra-frame modes can be expanded from 33 (as used in HEVC) to 65, so that the range of k becomes ±1 to ±16. These more dense directional intra-frame prediction modes are applied to blocks of all sizes, and also to luma and chroma intra predictions. Together with DC mode and planar mode, the number of the above intra-frame prediction modes is 35 (or 67).

[0040] Among the 35 (or 67) intra prediction modes, some (e.g., 3 or 5) modes are identified as a set of most probable modes (MPMs) for intra prediction in the current prediction block. Instead of reducing the bit rate by signaling an index to select one of the 35 (or 67) intra prediction modes, the encoder may reduce the bit rate by signaling an index to select one of the MPMs. For example, the intra prediction mode for the left prediction block and the intra prediction mode for the upper prediction block may be used as MPMs. When the intra prediction modes of the two adjacent blocks use the same intra prediction mode, the intra prediction mode can be used as the MPM. When only one of the two adjacent blocks is available and is encoded as a directional mode, the two directional modes adjacent to the directional mode may be used as MPMs. DC mode and planar mode are also identified as MPMs to fill in the available fields of the MPM set, especially when the upper or upper neighboring block is not available or is not coded as intra-frame prediction, or when the intra-frame prediction mode in the neighboring block is not a directional mode. If the intra-frame prediction mode of the current prediction block is one of the modes in the MPM set, one or two bits will be used to indicate which mode it is. Otherwise, the intra-frame prediction mode of the current block is different from any mode in the MPM set, and the current block will be encoded as a non-MPM mode. There are a total of 32 non-MPM modes, and a fixed-length (five-bit) encoding method is used to indicate this mode.

[0041] The MPM list is constructed based on the intra modes of the left and upper neighboring blocks. Assuming that the mode of the left neighboring block is marked as Left and the mode of the upper neighboring block is marked as Above, the unified MPM list can be constructed as follows:

[0042] – When a neighboring block is not available, its intra mode is set to planar by default.

[0043] – If both “Left” and “Above” modes are non-angle modes:

[0044] ●MPM list→{plane,DC,V,H,V-4,V+4}.

[0045] – If one of the “Left” and “Above” modes is an angle mode and the other is a non-angle mode:

[0046] ●Set Mode Max to the larger of “Left” and “Above”.

[0047] ●MPM list →{Plane,Max,Max-1,Max+1,Max–2,Max+2}.

[0048] – If Left and Above are both angular modes and they are different:

[0049] ●Set Mode Max to the larger of “Left” and “Above”.

[0050] Set the mode "Min" to the smaller of "Left" and "Above".

[0051] If Max-Min is equal to 1:

[0052] – MPM list → {Plane,Left,Above,Min-1,Max+1,Min-2}

[0053] ● Otherwise, if Max-Min is greater than or equal to 62:

[0054] – MPM list → {Plane,Left,Above,Min+1,Max-1,Min+2}

[0055] Otherwise, if Max-Min is equal to 2:

[0056] – MPM list → {Plane,Left,Above,Min+1,Min-1,Max+1}

[0057] ·otherwise:

[0058] – MPM list → {Plane,Left,Above,Min-1,Min+1,Max-1}

[0059] – If Left and Above are both angled and they are the same:

[0060] MPM list → {plane, Left, Left-1, Left+1, Left-2, Left+2}

[0061] The traditional angular intra prediction direction is defined as 45 degrees to -135 degrees clockwise. In VVC, some traditional angular intra prediction modes are adaptively replaced with wide-angle intra prediction modes for non-square blocks. The replaced modes can be indicated by the original mode index, which is remapped to the index of the wide-angle mode after parsing.

[0062] For some embodiments, the total number of intra prediction modes does not change (i.e., 67), and the intra mode coding method does not change. To support these prediction directions, an upper reference template of length 2W+1 and a left reference template of length 2H+1 are defined. FIGS. 2A and 2B conceptually illustrate the upper reference template and the left reference template being extended in length to support wide-angle directional modes for non-square blocks of different aspect ratios.

[0063] The number of wide-angle directional modes replaced depends on the aspect ratio of a block. The intra-frame prediction modes replaced for different blocks with different aspect ratios are shown in Table 1 below.

[0064] [Table 1] Intra-frame prediction modes replaced by wide-angle mode

[0065]

[0066] 2. Template-based Intra Mode Derivation (TIMD)

[0067] For mode selection, a template matching method can be applied by calculating the cost between the reconstructed samples and the predicted samples. One example is Template-based Intra Mode Derivation (TIMD). TIMD is a coding method in which the intra prediction mode of a CU is indirectly derived by using neighboring templates at both the encoder and the decoder, instead of the encoder sending the exact intra prediction mode to the decoder.

[0068] Figure 3It is shown that an intra prediction mode is indirectly derived for the current block 300 using template-based intra mode derivation (TIMD). As shown, the neighboring pixels of the current block 300 are used as templates 310. For each candidate intra mode, reference samples in an L-shaped reference area 320 located on the upper and left sides of the template 310 are used to generate prediction samples of the template 310. These reference samples are located in the L-shaped reference area 320 on the upper and left sides of the template 310. The template matching (TM) cost of the candidate intra mode cost is calculated based on the difference between the reconstructed samples of the template and the prediction samples of the template generated by the candidate intra mode (e.g., the sum of absolute transformed differences (SATD)). The candidate intra prediction mode with the lowest cost (such as in DIMD mode) is selected and used for intra prediction of the CU. The candidate mode may include 67 intra prediction modes (such as in VVC) or be extended to 131 intra prediction modes. MPM may be used to indicate the directional information of the CU. Therefore, in order to reduce the intra mode search space and exploit the characteristics of the CU, the intra prediction mode is indirectly derived from the MPM list.

[0069] In some embodiments, for each intra prediction mode in the MPM list, the SATD between the prediction sample and the reconstructed sample of the template is calculated as the TM cost of the intra mode. The first two intra prediction modes with the smallest SATD are selected as TIMD modes. The two TIMD modes are fused with weights after applying the Position dependent intraprediction combination (PDPC) process, and this weighted intra prediction is used to encode the current CU. The Position dependent intra prediction combination (PDPC) is included in the derivation of the TIMD mode.

[0070] The cost of the two selected modes (mode 1 and mode 2) is compared with the threshold, applying a cost factor of 2 in the test as follows:

[0071] Mode 2 cost < 2*Mode 1 cost

[0072] If this condition is true, prediction fusion is applied, otherwise only mode 1 is used. The weight of the mode is calculated based on its SATD cost as follows:

[0073] Weight 1 = Mode 2 cost / (Mode 1 cost + Mode 2 cost)

[0074] Weight 2 = 1 - Weight 1

[0075] 3. Decoder-side Intra-mode Derivation (DIMD)

[0076] Decoder side intra mode derivation (DIMD) refers to a technique where two intra prediction modes / angles / directions are derived from neighboring reconstructed samples (or templates) of a block and the two predictors are combined with the planar mode predictor with weights coming from the gradient. The DIMD mode is used as an alternative prediction mode and is always checked in high complexity rate distortion optimization (RDO) mode. To implicitly derive the intra prediction mode for a block, a texture gradient analysis is performed on both the encoder and decoder sides. The process starts with a blank histogram of gradient (HoG) with 65 entries, corresponding to the 65 angular / directional intra prediction modes. The amplitudes of these entries are determined during texture gradient analysis.

[0077] A video codec that implements DIMD performs the following steps: In the first step, the video codec selects T=3 columns and lines of the template from the left and top of the current block, respectively. This area is used as a reference for gradient-based intra prediction mode derivation. In the second step, horizontal and vertical Sobel filters are applied to all 3×3 window positions, centered on the pixels of the middle line of the template. At each window position, the Sobel filter calculates the pure horizontal and vertical intensities as Gx and Gy, respectively. Then, the material angle of the window is calculated as follows:

[0078] Angle = arctan(Gx / Gy)

[0079] It can be converted to one of 65 angular intra prediction modes. Once the intra prediction mode index of the current window is derived as idx, the amplitude of its entry in HoG[idx] is updated by the following addition:

[0080] Amplitude (ampl) = |Gx| + |Gy|

[0081] Figure 4The use of decoder-side intra mode derivation (DIMD) to indirectly derive the intra prediction mode of the current block is indicated. The figure shows an example gradient histogram (HoG) 410 calculated after applying the above operations to all pixel positions in a template 415, which includes neighboring lines of pixel samples around the current block 400. Once the HoG is calculated, the two highest histogram bins (M1 and M2) are selected as the two indirectly derived intra prediction modes (IPMs) for the block. The predictions of the two IPMs are further combined with the planar mode as the prediction of the DIMD mode. Prediction fusion is applied as a weighted average of the above three predictions (M1 prediction, M2 prediction, and planar mode prediction). To this end, the weight of the planar mode can be set to 21 / 64 (~1 / 3). Then, the remaining weight 43 / 64 (~2 / 3) is distributed between the two HoG IPMs, proportional to the amplitude of their HoG histogram bins. The prediction fusion or combined prediction of DIMD can be written as:

[0082] Pred DIMD =(43*(w1*pred M1 +w2*pred M2 )+21*pred planar )>>6

[0083] w1=amp M1 / (amp M1 +amp M2 )

[0084] w2=amp M2 / (amp M1 +amp M2 )

[0085] In addition, the two indirectly derived intra prediction modes are added to the most probable mode (MPM) list, so the DIMD process is performed before constructing the MPM list. The primary derived intra modes of a DIMD block are stored in a block and are used to construct the MPM lists of neighboring blocks.

[0086] DIMD with multiple HoGs

[0087] Some embodiments of the present disclosure provide a method for improving DIMD prediction accuracy or coding performance. In some embodiments, in order to reduce calculations, the HoG calculation is based on partially selected adjacent window positions. For some embodiments, the video codec can select the upper middle, upper right, middle left, and lower left adjacent window positions to apply the Sobel filter to construct the HoG. The video encoder can also select even or odd adjacent window positions to apply the Sobel filter to construct the HoG.

[0088] In some embodiments, when encoding a video, three HoGs are constructed, including: a left HoG, an upper HoG, and an original upper-left HoG. Two or more DIMD intra modes (e.g., Figure 4 M1 and M2). To decide the final DIMD mode, a cost evaluation process (e.g., a high complexity RDO process) is applied to each of the two or more DIMD modes derived from each of the three HoGs. In some embodiments, an additional syntax element is added to the bitstream to indicate which side (left / top / top-left) is used for HoG accumulation to derive the DIMD intra mode.

[0089] Figures 5A-5C DIMD intra modes derived from a plurality of different HoGs for a current block 500 are conceptually shown. Figure 5A HoG 510A accumulated based on template region 515A is indicated. Pixel samples of template region 515A include neighboring lines of the current block above and to the left thereof. (Thus, HoG 510A is the top left HoG.) Intra modes MA1 and MA2 are identified based on the two highest accumulated amplitudes of HoG 510A.

[0090] Figure 5B 5 shows a HoG 510B accumulated based on a template region 515B. The pixel samples of the template region 515B consist of neighboring lines to the left of the current block. (Thus, HoG 510B is a left HoG.) Two intra modes MB1 and MB2 are identified based on the two highest accumulated amplitudes of HoG 510B.

[0091] Figure 5C 5 shows a HoG 510C accumulated based on a template region 515C. The pixel samples of the template region 515C are composed of neighboring lines of the current block on its upper side. (Thus, HoG 510C is an upper HoG.) Two intra modes MC1 and MC2 are identified based on the two highest accumulated amplitudes of HoG 510C.

[0092] In some embodiments, to determine the final DIMD intra-frame mode, the template matching (TM) cost of the DIMD intra-frame mode derived from the three HoGs can be used to select one of the three (left / top / left-top) HoGs to derive the final DIMD intra-frame mode. (The TM cost of the intra-frame mode has been described in Section 2 above.) Figures 5A-5CAs shown, the video codec can select one of the three HoGs 510A-510C based on the TM cost calculated for the DIMD intra modes MA1, MA2, MB1, MB2, MC1, MC2. (The TM cost is calculated based on the L-shaped reference regions 520A, 520B, and 520C adjacent to the template regions 515A, 515B, and 515C, respectively).

[0093] For example, in some embodiments, three HoGs are evaluated and compared (e.g., the intra-frame directions with the highest accumulation, namely MA1, MB1, and MC1 of HoGs 510A to 510C, respectively). The HoG with the lowest TM cost among the three HoGs is selected as the first DIMD intra-frame mode to provide the final DIMD intra-frame mode for the current block. (Therefore, if MB1 has the lowest cost, HoG510B is selected). Other cost evaluations can also be used for each HoG to select one of the three HoGs, such as using the TM cost of the first DIMD intra-frame mode plus the TM cost of the second DIMD intra-frame mode, or a weighted sum of the two TM costs.

[0094] In some embodiments, through the above explicit HoG side or indirect HoG side selection, HoGs on different sides are constrained to different selectable intra mode sets. Table 2 below shows a possible implementation.

[0095] [Table 2]

[0096]

[0097]

[0098] With the mode selection constraint, the DIMD Intra mode derived from the left HoG is more likely to be different from the DIMD Intra mode derived from the upper HoG. In Table 2, there are some overlaps between the optional modes on the left and the optional modes on the upper side (modes 33, 34, 35). Therefore, there is still a chance that the left HoG and the upper HoG can derive the same DIMD Intra mode.

[0099] In some embodiments, Table 3 is used and the intra modes derived from the left side and from the top side are different (no overlap).

[0100] [Table 3]

[0101] DIMD HoG side Selectable intra mode Left + Up 2,3,4,5,……,64,65,66 Left 2,3,4,5,……,32,33,34 superior 35,36,37,……,64,65,66

[0102] To decide which table to use (Table 2 or Table 3), in some embodiments the video codec can compress the video database to check the coding gain and select a predefined table for encoding and decoding. In some embodiments, a set of possible tables is defined and the video codec can use a notification syntax element to select the best table for encoding and decoding.

[0103] In some embodiments, using the explicit or indirect HoG side selection described above, the selectable Intra / DIMD modes associated with the three DIMD HoG sides are completely different (no overlap). Table 4 shows an implementation for a field of 131 angle modes, where the Intra modes of different HoGs do not overlap:

[0104] [Table 4]

[0105]

[0106]

[0107] In Table 4, the DIMD process is applied to 131 intra angle mode fields, the left HoG provides two intra modes from (4××i+2) intra modes, where i=0...32, the right HoG provides two intra modes from (4×j+4) intra modes, where j=0...32, and the upper left HoG provides two intra modes from (2×k+3) intra modes, where k=0...64. Table 5 shows the implementation of 67 angle mode fields, where the intra / DIMD modes of different HoGs do not overlap:

[0108] [Table 5]

[0109]

[0110] In Table 5, assuming that the DIMD process is applied to the field of 67 intra-frame angle modes, the HoG on the left provides two intra-frame modes from the (4×i+2) intra-frame modes, where i=0…16, the HoG on the right provides two intra-frame modes from the (4×j+4) intra-frame modes, where j=0…16, and the HoG on the upper left provides two intra-frame modes from the (2×k+3) intra-frame modes, where k=0…63.

[0111] In some embodiments, the number of neighboring lines used to calculate the HoG in the DIMD process can be signaled in the PPS, SPS, picture header, slice header, CTU level syntax, or indirectly derived from other syntaxes. For example, when the current block size is less than or greater than a threshold, the DIMD process may use more neighboring lines (i.e., more than 3 lines) to calculate the HoG. Figures 6A-6C The DIMD process in some embodiments is shown when additional adjacent lines are present.

[0112] As shown, the original 3x3 Sobel filter is applied to additional neighboring lines to generate additional HoGs, so there are multiple left HoGs 610A-612A, multiple top HoGs 610B-612B, and multiple top left HoGs 610C-612C (so there are nine HoGs in total). Each HoG corresponds to a scan of the Sobel window at a different distance from the current block. In some embodiments, each of the plurality of neighboring lines of the current block is assigned a HoG generation index, for example, the neighboring line directly adjacent to the current block is assigned a HoG generation index of 0, and the next neighboring line closest to the current block is assigned a HoG generation index of 1, etc. In this example, HoG 610A, 610B, 610C correspond to a Sober scan centered on adjacent line 1 (HoG generates index 1), HoG 611A, 611B, 611C correspond to a Sober scan centered on adjacent line 2 (HoG generates index 2), and HoG 612A, 612B, 612C correspond to a Sober scan centered on adjacent line 3 (HoG generates index 3).

[0113] Each HoG is used to identify two (or more) DIMD intra modes. The TM cost process described in Section 2 is used to determine the costs of these DIMD intra modes. The video codec can select one of the nine HoGs as the basis for generating the final DIMD intra mode. For example, if HoG 611B has the lowest cost, the final DIMD intra mode is selected from HoG 611B (HoG generation index 2) selected from the current block.

[0114] In some embodiments, a larger window (and a Sobel filter with a larger kernel) may be used to accumulate the HoGs so that the selection of the HoG providing the final (e.g., two) DIMD Intra modes is still from one of the three HoGs (left / top / top-left).

[0115] For some embodiments, the DIMD HoG accumulation and intra prediction generation processes are orthogonal. DIMD HoG accumulation can be performed based on the nearest L-shaped reference region or neighboring line of the current block, while intra prediction generation can refer to multiple L-shaped reference regions or neighboring lines. In some embodiments, multiple L-shaped regions can be applied to both DIMD HoG accumulation and intra prediction generation, so that an index (or multiple indices) of the L-shaped HoG generation can be carried from the DIMD HoG accumulation process to the intra prediction generation process. In addition, the generated prediction corresponding to the index (or multiple indices) of the selected L-shaped HoG generation can be reused in the intra prediction generation process. Shared calculations between the two processes (DIMD HoG and intra prediction) can be reused to reduce computational load. More generally, the neighboring lines or reference regions used to generate intra prediction pixels for the current block (for the TM cost of each candidate intra mode or for the final intra prediction of the current block) can be identified based on the DIMD HoG accumulated by the neighboring lines and the HoG generation index that identifies the DIMD intra mode.

[0116] Figure 7A-7B Identification of a reference region for generating intra prediction based on neighboring lines used to derive the DIMD intra mode of the current block 600 is shown. Fig. 7A A HoG 611A generated by Sober scanning with a HoG generation index of 2 is shown. HoG 611A is used to identify two DIMD intra modes MA3 and MA4. Intra-prediction pixels based on intra modes MA3 and MA4 (for TM cost and / or for final intra prediction) are generated based on an L-shaped region 712, which is identified based on HoG generation index 2.

[0117] Figure 7B A HoG 612A generated by Sobel scanning with a HoG generation index of 3 is shown. The HoG 612A is used to identify two DIMD intra modes MA5 and MA6. Intra-prediction pixels based on the intra modes MA5 and MA6 (for TM cost and / or for final intra prediction) are generated based on an L-shaped region 713, which is identified based on the HoG generation index 3 and is farther away from the current block 600 than the L-shaped region 712 identified based on the HoG generation index 2.

[0118] In some embodiments, to reduce the amount of HoG computation, the gradient from the top left corner position is eliminated from the HoG accumulation. Figures 8A-8B The position of the upper left corner is shown to be eliminated from the HoG accumulation. For the current block 800, Fig. 8A Shown are the positions used for the top left HoG when no positions are eliminated. Figure 8BThe positions for the top left HoG that are eliminated when the top left corner position is eliminated from the HoG accumulation are shown. In some embodiments, these positions are always eliminated from the HoG accumulation. In some embodiments, the elimination of these positions from the HoG accumulation is based on some determination based on the block size. For example, if the current block width plus the current block height or the current block area is greater than a predefined threshold, then the gradients for these corner positions are discarded, i.e., only the gradients from the top and left sides are included in the HoG calculation.

[0119] In some embodiments, when DIMD is used for the current block, two intra modes with the highest gradient values ​​are determined from the reconstructed neighboring samples, and the predictions of the two intra modes are further combined with the planar mode predictor (as a weighted sum) to generate a final intra predictor. When deciding the first two intra modes, the video codec may compare the gradient of each intra mode (as the current candidate intra mode) with the current best and second best intra modes (in terms of having the highest gradient value). If the gradient of the current candidate intra mode is the same as the current best and / or second best intra mode, the video codec may also compare the TIMD cost of the current intra prediction mode with the TIMD cost of the current best and / or second best intra mode. Specifically, if the current candidate intra mode has the same gradient amplitude as the current best and / or second best intra mode, the template cost of the candidate intra mode is calculated as the SATD between the prediction sample and the reconstructed sample of the template. If the current candidate intra mode has a lower template cost than the current best and / or second best intra mode, the current candidate intra mode is selected as the current best or second best intra mode.

[0120] In some embodiments, after constructing the DIMD HoG, K candidate intra modes with the highest gradient values ​​are selected, and the TM process is applied to these K candidate intra modes to determine the final intra mode as the DIMD intra mode. Specifically, the TM cost (as described in Section 2) is calculated for these K candidate intra modes with the highest gradient values. If the number of candidate intra modes with non-zero amplitude is less than K, the video encoder will only calculate the TM cost of the candidate intra modes with non-zero amplitude.

[0121] In some embodiments, the DIMD process as described above can be used as a first pass selection to generate two angle mode candidates with the highest and second highest gradient values. The TM process is then used as a second pass selection to improve the intra-frame mode selected in the first pass. In some embodiments, if the two intra-frame modes selected by the DIMD process in the first pass are M and N, TM is applied to the intra-frame modes {M-1, M, M+1} and {N-1, N, N+1} to improve the two intra-frame modes DIMD. However, the two improved DIMD intra-frame modes can be the same. In this case, in order to have two different DIMD intra-frame modes, a predefined rule can be applied to select the second intra-frame mode. For example, the video codec can select a second mode different from the refined single DIMD intra-frame mode from the list {M, N, M-1, M+1, N-1, N+1}.

[0122] 5. An Example of Video Encoder

[0123] Fig. 9 An example video encoder 900 that can implement decoder-side intra mode derivation (DIMD) is shown. As shown, the video encoder 900 receives an input video signal from a video source 905 and encodes the signal into a bitstream 995. The video encoder 900 has several components or modules for encoding the signal from the video source 905, including at least some components selected from the following: a transform module 910, a quantization module 911, an inverse quantization module 914, an inverse transform module 915, an intra picture estimation module 920, an intra prediction module 925, a motion compensation module 930, a motion estimation module 935, an in-loop filter 945, a reconstructed picture buffer 950, a motion vector (MV) buffer 965, an MV prediction module 975, and an entropy encoder 990. The motion compensation module 930 and the motion estimation module 935 are part of the inter prediction module 940.

[0124] In some embodiments, these modules 910-990 are software instruction modules executed by a computing device or electronic device of one or more processing units (e.g., processors). In some embodiments, these modules 910-990 are hardware circuit modules implemented by one or more integrated circuits (ICs) of an electronic device. Although modules 910-990 are depicted as separate modules, some modules may be combined into a single module.

[0125] The video source 905 provides a raw video signal that presents pixel data of each video frame without compression. The subtractor 908 calculates the difference between the raw video pixel data of the video source 905 and the predicted pixel data from the motion compensation module 930 or the intra-frame prediction module 925 as a prediction residual 909. The transform module 910 converts the difference (or residual pixel data or residual signal) into a transform coefficient 916 (for example, by performing a discrete cosine transform, also known as DCT). The quantization module 911 quantizes the transform coefficient 916 into quantized data (or quantized coefficients) 912, which is encoded into a bit stream 995 by the entropy encoder 990.

[0126] The inverse quantization module 914 inversely quantizes the quantized data (or quantized coefficients) 912 to obtain transform coefficients, and the inverse transform module 915 inversely transforms the transform coefficients to generate a reconstructed residual 919. The reconstructed residual 919 is added to the predicted pixel data 913 to generate reconstructed pixel data 917. In some embodiments, the reconstructed pixel data 917 is temporarily stored in a line buffer (not depicted) for intra-frame picture prediction and spatial MV prediction. The reconstructed pixels are filtered by the in-loop filter 945 and stored in the reconstructed picture buffer 950. In some embodiments, the reconstructed picture buffer 950 is a memory external to the video encoder 900. In some embodiments, the reconstructed picture buffer 950 is a memory internal to the video encoder 900.

[0127] The intra picture estimation module 920 performs intra prediction based on the reconstructed pixel data 917 to generate intra prediction data. The intra prediction data is provided to the entropy encoder 990 to be encoded into a bitstream 995. The intra prediction data is also used by the intra prediction module 925 to generate predicted pixel data 913.

[0128] The motion estimation module 935 performs inter-frame prediction by generating MVs for reference pixel data of previously decoded frames stored in the reconstructed picture buffer 950. These MVs are provided to the motion compensation module 930 to generate predicted pixel data.

[0129] Instead of encoding the complete actual MV in the bitstream, the video encoder 900 uses MV prediction to generate a predicted MV, and encodes the difference between the MV used for motion compensation and the predicted MV as residual motion data and stores it in the bitstream 995.

[0130] The MV prediction module 975 generates a predicted MV based on a reference MV generated for decoding a previous video frame (e.g., a motion compensation MV for performing motion compensation). The MV prediction module 975 retrieves the reference MV of the previous video frame from the MV buffer 965. The video encoder 900 stores the MV generated for the current video frame in the MV buffer 965 as a reference MV for generating the predicted MV.

[0131] The MV prediction module 975 uses the reference MV to create a predicted MV. The predicted MV can be calculated by spatial MV prediction or temporal MV prediction. The difference between the predicted MV of the current frame (residual dynamic data) and the motion compensation MV (MC MV) is encoded into the bitstream 995 by the entropy encoder 990.

[0132] The entropy encoder 990 encodes various parameters and data into a bitstream 995 by using an entropy coding technique such as context-adaptive binary arithmetic coding (CABAC) or Huffman encoding. The entropy encoder 990 encodes various header elements, flags, and quantized coefficients 912 and residual dynamic data as syntax elements into the bitstream 995. The bitstream 995 is then stored in a storage device or transmitted to a decoder via a communication medium such as a network.

[0133] The in-loop filter 945 performs filtering or smoothing operations on the reconstructed pixel data 917 to reduce coding artifacts, particularly at the boundaries of pixel blocks. In some embodiments, the filtering or smoothing operations performed by the in-loop filter 945 include a deblocking filter (DBF), a sample adaptive offset (SAO), and / or an adaptive loop filter (ALF).

[0134] Fig.10 A portion of a video encoder 900 implementing DIMD based on multiple HoGs is shown. Specifically, the figure illustrates components of an intra prediction module 925 of the video encoder 900. As shown, the intra prediction module 925 includes a gradient accumulation module 1030, a candidate HoG storage 1020, an intra mode selection module 1010, and an intra prediction generation module 1040. The intra prediction module 925 can use these modules to perform DIMD intra prediction on both luma and chroma components.

[0135] The gradient accumulation module 1030 receives neighboring samples of the current block from the reconstructed picture buffer 950 and calculates the gradient amplitudes for different intra-mode directions. Gradient amplitudes are generated and accumulated for different HoGs in the candidate HoG storage 1020. Different HoGs are based on different subsets of neighboring samples and / or different numbers of neighboring lines. In some embodiments, each HoG is associated with a HoG generation index that identifies the neighboring lines used to derive the HoG.

[0136] The intra mode selection module 1010 checks candidate intra modes 1025 from different HoGs stored in the candidate HoG storage 1020. The intra mode selection module 1010 can identify a set of intra modes with the highest cumulative amplitude from each HoG. The intra mode selection module 1010 then calculates the template matching (TM) cost of those identified intra modes. The TM cost is calculated based on the reference samples (in the L-shaped reference area) provided by the reconstructed picture buffer 950 and the neighboring samples of the current block. In some embodiments, the reference samples for the TM cost calculation of the candidate intra mode are identified based on the HoG generation index of the HoG that provides the candidate intra mode. Based on the calculated TM cost, the intra mode selection module 1010 selects a HoG and a set of DIMD intra modes from the selected HoG as the final intra prediction mode 1015.

[0137] The intra prediction generation module 1040 generates an intra prediction 1045 of the current block using the final intra prediction mode 1015. The final prediction mode 1015 may include two or more DIMD intra modes, and the intra prediction generation module 1040 may fetch a plurality of predictions / predictors from the reconstructed picture buffer 950 based on the plurality of DIMD intra modes. The fetched plurality of predictors are mixed to generate an intra prediction 1045 to be used as the predicted pixel data 913.

[0138] Fig.11 The process 1100 of performing intra prediction on a current block using multiple HoGs is conceptually illustrated. In some embodiments, one or more processing units (e.g., processors) of a computing device implementing the encoder 900 perform the process 1100 by executing instructions stored in a computer-readable medium. In some embodiments, an electronic device implementing the encoder 900 performs the process 1100.

[0139] At step 1110 , an encoder receives data to be encoded as a current block of pixels in a current picture of a video.

[0140] At step 1120, the encoder derives a plurality of histograms of gradients (HoGs) for different intra prediction angles. Different HoGs are derived based on different groups of reconstructed pixels adjacent to the current block. In some embodiments, a first HoG is derived based on reconstructed pixels on the upper side of the current block, a second HoG is derived based on reconstructed pixels on the left side of the current block, and a third HoG is derived based on reconstructed pixels on the left and upper sides of the current block. In some embodiments, the plurality of HoGs include HoGs derived using different numbers of adjacent lines of the current block.

[0141] At step 1130, the encoder selects a HoG from the plurality of HoGs. At step 1135, the encoder identifies one or more intra prediction modes based on the selected HoG. In some embodiments, the selection of the HoG is based on syntax elements signaled in the bitstream. In some embodiments, the encoder identifies one or more intra prediction modes from the HoG by identifying the N candidate intra prediction modes with the highest amplitude in the HoG and then selecting M intra prediction modes from the N candidate intra prediction modes based on cost. In some embodiments, the reconstructed pixel located at the upper left corner of the current block is not used to derive the third HoG.

[0142] In some embodiments, the encoder calculates a cost of an intra prediction mode identified based on a plurality of HoGs, and selects a HoG from the plurality of HoGs based on the calculated cost. In some embodiments, the cost of the intra prediction mode is calculated by comparing a difference between a template reconstructed sample and a template prediction sample, wherein the template reconstructed sample is adjacent to the current block and the template prediction sample is generated by the intra prediction mode using an L-shaped reference region near the template.

[0143] At step 1140, the encoder generates an intra prediction for the current block based on one or more intra prediction modes identified by the selected HoG. In some embodiments, the plurality of HoGs include HoGs derived based on different neighboring lines of the current block, and different neighboring lines are associated with different HoG generation indexes. In some embodiments, if a HoG derived based on neighboring lines associated with a first HoG generation index is used to derive a first intra prediction mode, then the intra prediction for the current block based on the first intra prediction mode is generated based on an L-shaped reference region identified by the first HoG generation index.

[0144] In step 1150 , the encoder encodes the current block using the generated intra prediction to generate a prediction residual.

[0145] 6. An Example of a Video Decoder

[0146] In some embodiments, the encoder may signal (or generate) one or more syntax elements in the bitstream so that the decoder may parse the one or more syntax elements from the bitstream.

[0147] Fig.12 An example video decoder 1200 is shown that can implement decoder-side intra mode derivation (DIMD). As shown, the video decoder 1200 is a picture decoding or video decoding circuit that receives a bitstream 1295 and decodes the content of the bitstream into pixel data of a video frame for display. The video decoder 1200 has several components or modules for decoding the bitstream 1295, including some components selected from the following: an inverse quantization module 1211, an inverse transform module 1210, an intra prediction module 1225, a motion compensation module 1230, an in-loop filter 1245, a decoded picture buffer 1250, an MV buffer 1265, an MV prediction module 1275, and a parser 1290. The motion compensation module 1230 is a part of the inter prediction module 1240.

[0148] In some embodiments, modules 1210-1290 are software instruction modules executed by one or more processing units (e.g., processors) of a computing device. In some embodiments, modules 1210-1290 are hardware circuit modules implemented by one or more integrated circuits of an electronic device. Although modules 1210-1290 are depicted as separate modules, some modules may be combined into a single module.

[0149] The parser (or entropy decoder) 1290 receives the bitstream 1295 and performs initial parsing according to the syntax defined by the video coding or picture coding standard. The parsed syntax elements include various header elements, flags, and quantization data (or quantization coefficients) 1212. The parser 1290 uses entropy coding techniques such as context adaptive binary arithmetic coding (CABAC) or Huffman coding to parse various syntax elements.

[0150] The inverse quantization module 1211 inversely quantizes the quantized data (or quantized coefficients) 1212 to obtain transform coefficients, and the inverse transform module 1210 inversely transforms the transform coefficients 1216 to generate a reconstructed residual signal 1219. The reconstructed residual signal 1219 is added to the predicted pixel data 1213 from the intra-frame prediction module 1225 or the motion compensation module 1230 to generate decoded pixel data 1217. The decoded pixel data is filtered by the in-loop filter 1245 and stored in the decoded picture buffer 1250. In some embodiments, the decoded picture buffer 1250 is a memory external to the video decoder 1200. In some embodiments, the decoded picture buffer 1250 is a memory internal to the video decoder 1200.

[0151] The intra prediction module 1225 receives the intra prediction data from the bitstream 1295 and generates the predicted pixel data 1213 from the decoded pixel data 1217 stored in the decoded picture buffer 1250 according to the intra prediction data. In some embodiments, the decoded pixel data 1217 is also stored in a line buffer (not shown) for intra picture prediction and spatial MV prediction.

[0152] In some embodiments, the contents of the decoded picture buffer 1250 are used for display. The display device 1255 can retrieve the contents of the decoded picture buffer 1250 for direct display, or retrieve the contents of the decoded picture buffer to a display buffer. In some embodiments, the display device receives pixel values ​​from the decoded picture buffer 1250 via pixel transfer.

[0153] The motion compensation module 1230 generates predicted pixel data 1213 from the decoded pixel data 1217 stored in the decoded picture buffer 1250 according to the motion compensation MV (MC MV). These motion compensation MVs are decoded by adding the residual motion data received from the bitstream 1295 and the predicted MV received from the MV prediction module 1275.

[0154] The MV prediction module 1275 generates a predicted MV based on a reference MV generated for decoding a previous video frame (e.g., a motion compensated MV for performing motion compensation). The MV prediction module 1275 retrieves the reference MV of the previous video frame from the MV buffer 1265. The video decoder 1200 stores the motion compensated MV generated for decoding the current video frame in the MV buffer 1265 as a reference MV for generating the predicted MV.

[0155] The in-loop filter 1245 performs filtering or smoothing operations on the decoded pixel data 1217 to reduce coding artifacts, particularly at the boundaries of pixel blocks. In some embodiments, the filtering or smoothing operations performed by the in-loop filter 1245 include a deblocking filter (DBF), a sample adaptive offset (SAO), and / or an adaptive in-loop filter (ALF).

[0156] Fig.13 A portion of a video decoder 1200 implementing DIMD based on multiple HoGs is shown. Specifically, the figure illustrates components of an intra prediction module 1225 of the video decoder 1200. As shown, the intra prediction module 1225 includes a gradient accumulation module 1330, a candidate HoG storage 1320, an intra mode selection module 1310, and an intra prediction generation module 1340. The intra prediction module 1225 can use these modules to perform DIMD intra prediction on both luma and chroma components.

[0157] The gradient accumulation module 1330 receives the reconstructed neighboring samples of the current block from the decoded picture buffer 1250 and calculates the gradient amplitudes for different intra mode directions. Gradient amplitudes are generated and accumulated for different HoGs in the candidate HoG storage 1320. Different HoGs are based on different subsets of neighboring samples and / or different numbers of neighboring lines. In some embodiments, each HoG is associated with a HoG generation index, which identifies the neighboring lines used to derive the HoG.

[0158] The intra mode selection module 1310 checks candidate intra modes 1325 from different HoGs stored in the candidate HoG storage 1320. The intra mode selection module 1310 can identify a set of intra modes with the highest cumulative amplitude from each HoG. The intra mode selection module 1310 then calculates the template matching (TM) cost of those identified intra modes. The TM cost is calculated based on the reference samples (in the L-shaped reference area) provided by the decoded picture buffer 1250 and the neighboring samples of the current block. In some embodiments, the reference samples for the TM cost calculation of the candidate intra mode are identified based on the HoG generation index of the HoG that provides the candidate intra mode. Based on the calculated TM cost, the intra mode selection module 1310 selects a HoG and a set of DIMD intra modes from the selected HoG as the final intra prediction mode 1315.

[0159] The intra prediction generation module 1340 generates an intra prediction 1345 of the current block using the final intra prediction mode 1315. The final intra prediction mode 1315 may include two or more DIMD intra modes, and the intra prediction generation module 1340 may fetch a plurality of predictions / predictors from the decoded picture buffer 1250 based on the plurality of DIMD intra modes. The fetched plurality of predictors are mixed to generate the intra prediction 1345 to be used as the predicted pixel data 1213.

[0160] Fig.14 The process 1400 of performing intra prediction on a current block using multiple HoGs is conceptually illustrated. In some embodiments, one or more processing units (e.g., processors) of a computing device implementing the decoder 900 perform the process 1400 by executing instructions stored in a computer-readable medium. In some embodiments, an electronic device implementing the decoder 900 performs the process 1400.

[0161] In step 1410, a decoder receives data to be decoded as a current pixel block in a current picture of a video.

[0162] At step 1420, the decoder derives a plurality of histograms of gradients (HoGs) for different intra prediction angles. Different HoGs are derived based on different groups of reconstructed pixels adjacent to the current block. In some embodiments, a first HoG is derived based on reconstructed pixels on the upper side of the current block, a second HoG is derived based on reconstructed pixels on the left side of the current block, and a third HoG is derived based on reconstructed pixels on the left and upper sides of the current block. In some embodiments, the plurality of HoGs include HoGs derived using different numbers of adjacent lines of the current block.

[0163] In step 1430, the decoder selects a HoG from the plurality of HoGs. In step 1435, the decoder identifies one or more intra prediction modes based on the selected HoG. In some embodiments, the selection of the HoG is based on syntax elements signaled in the bitstream. In some embodiments, the decoder identifies one or more intra prediction modes from the HoG by identifying the N candidate intra prediction modes with the highest amplitude in the HoG and then selecting M intra prediction modes from the N candidate intra prediction modes based on cost. In some embodiments, the reconstructed pixel located at the upper left corner of the current block is not used to derive the third HoG.

[0164] In some embodiments, the decoder calculates a cost of an intra prediction mode identified based on a plurality of HoGs, and selects a HoG from the plurality of HoGs based on the calculated cost. In some embodiments, the cost of the intra prediction mode is calculated by comparing a difference between a template reconstructed sample and a template prediction sample, wherein the template reconstructed sample is adjacent to the current block and the template prediction sample is generated by the intra prediction mode using an L-shaped reference area near the template.

[0165] At step 1440, the decoder generates an intra prediction for the current block based on one or more intra prediction modes identified by the selected HoG. In some embodiments, the plurality of HoGs include HoGs derived based on different neighboring lines of the current block, and different neighboring lines are associated with different HoG generation indexes. In some embodiments, if a HoG derived based on neighboring lines associated with a first HoG generation index is used to derive a first intra prediction mode, then the intra prediction for the current block based on the first intra prediction mode is generated based on an L-shaped reference region identified by the first HoG generation index.

[0166] In step 1450, the decoder reconstructs the current block by using the generated intra prediction. The decoder may then provide the reconstructed current block for display as part of the reconstructed current picture.

[0167] 7. An Example of Electronic System

[0168] Many of the above features and applications are implemented as software processes that are specified as a set of instructions recorded on a computer-readable storage medium (also referred to as a computer-readable medium). When these instructions are executed by one or more computing or processing units (e.g., one or more processors, processor cores, or other processing units), they cause the processing units to perform the actions represented by the instructions. An example of a computer-readable medium includes, but is not limited to, a read-only compact disc (CD-ROM), a flash drive, a random-access memory (RAM) chip, a hard disk, an erasable programmable read only memory (EPROM), an electrically erasable programmable read only memory (EEPROM), etc. Computer-readable media does not include carrier waves and electronic signals transmitted wirelessly or via wired connections.

[0169] In this specification, the term "software" is meant to include firmware residing in a read-only memory or an application stored in a magnetic storage device, which can be read into memory for processing by a processor. In addition, in some embodiments, a plurality of software inventions can be implemented as sub-parts of a larger program while retaining different software inventions. In some embodiments, a plurality of software inventions can also be implemented as independent programs. Finally, any combination of independent programs that implement the software inventions described herein together is within the scope of this disclosure. In some embodiments, when the software program is installed to run on one or more electronic systems, one or more specific machine implementations are defined that implement and execute the operations of the software program.

[0170] Fig.15 An electronic system 1500 is conceptually depicted for implementing some embodiments of the present disclosure. The electronic system 1500 may be a computer (e.g., a desktop computer, a personal computer, a tablet computer, etc.), a phone, a PDA, or any other kind of electronic device. Such an electronic system includes various types of computer-readable media and interfaces for various other types of computer-readable media. The electronic system 1500 includes a bus 1505, a processing unit 1510, a graphics-processing unit (GPU) 1515, a system memory 1520, a network 1525, a read-only memory (ROM) 1530, a permanent storage device 1535, an input device 1540, and an output device 1545.

[0171] The bus 1505 collectively represents all system, peripheral, and chipset buses that communicatively couple the numerous internal components of the electronic system 1500. For example, the bus 1505 communicatively couples: the processing unit 1510, the GPU 1515, the read-only memory 1530, the system memory 1520, and the permanent storage device 1535.

[0172] From these various memory units, processing unit 1510 retrieves instructions to be executed and data to be processed in order to perform the processes of the present disclosure. In various embodiments, the processing unit may be a single processor or a multi-core processor. Some instructions are passed to and executed by GPU 1515. GPU 1515 may offload various computations or supplement the image processing provided by processing unit 1510.

[0173] The read-only memory (ROM) 1530 stores static data and instructions used by the processing unit 1510 and other modules of the electronic system. On the other hand, the permanent storage device 1535 is an erasable memory device. This device is a non-volatile memory unit that stores instructions and data even when the electronic system 1500 is turned off. Some embodiments of the present disclosure use a mass storage device (such as a magnetic disk or optical disk and its corresponding magnetic disk drive) as the permanent storage device 1535.

[0174] Other embodiments use removable storage devices (e.g., floppy disks, flash memory devices, etc. and their corresponding disk drives) as permanent storage devices. Like permanent storage device 1535, system memory 1520 is an erasable memory device. However, unlike storage device 1535, system memory 1520 is a volatile read-write memory, such as random access memory. System memory 1520 stores some instructions and data used by the processor at runtime. In some embodiments, processes according to the present disclosure are stored in system memory 1520, permanent storage device 1535 and / or read-only memory 1530. For example, in some embodiments, various memory units include instructions for processing multimedia clips. From these various memory units, processing unit 1510 retrieves instructions to be executed and data to be processed in order to perform the processes of some embodiments.

[0175] The bus 1505 is also connected to an input device 1540 and an output device 1545. The input device 1540 enables a user to transmit information and select commands to the electronic system. The input device 1540 includes an alphanumeric keyboard and a pointing device (also called a "cursor control device"), a camera (e.g., a webcam), a microphone, or a similar device for receiving voice commands, etc. The output device 1545 displays pictures or output data generated by the electronic system. The output device 1545 includes a copier and a display device, such as a cathode ray tube (CRT) display or a liquid crystal display (LCD), and a speaker or similar audio output device. Some embodiments include devices that are used as both input and output devices, such as a touch screen.

[0176] Finally, if Fig.15 As shown, bus 1505 also couples electronic system 1500 to network 1525 via a network adapter (not shown). In this manner, the computer can be part of a network of computers (e.g., a local area network (LAN), a wide area network (WAN), or an intranet), or one of a plurality of networks such as the Internet. Any or all components of electronic system 1500 may be used in conjunction with the present disclosure.

[0177] Some embodiments include electronic components, such as microprocessors, storage and memory, which store computer program instructions in machine-readable or computer-readable media (or computer-readable storage media, machine-readable media, or machine-readable storage media). Some examples of such computer-readable media include RAM, ROM, read-only compact discs (CD-ROM), recordable compact discs (CD-R), rewritable compact discs (CD-RW), read-only digital versatile discs (such as DVD-ROM, double-layer DVD-ROM), various recordable / rewritable DVDs (such as DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (such as SD card, mini-SD card, micro SD card, etc.), magnetic and / or solid-state hard drives, read-only and recordable CD-ROMs, ultra-density CD-ROMs, any other optical or magnetic media, and floppy disks. Computer readable media may store computer programs that are executable by at least one processing unit and include sets of instructions for performing various operations. Examples of computer programs or computer code include machine code, such as that produced by a compiler, and archives including high-level code that is executed by a computer, electronic component, or microprocessor using an interpreter.

[0178] Although the above discussion primarily involves microprocessors or multi-core processors executing software, many of the above functions and applications are performed by one or more integrated circuits, such as application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions stored on the circuits themselves. In addition, some embodiments execute software stored in programmable logic devices (PLDs), ROM, or RAM devices.

[0179] The terms "computer", "server", "processor" and "memory" used in this specification and any claims of this application refer to electronic or other technical devices. These terms do not include people or groups of people. For the purpose of this specification, the term "display" means display on an electronic device. The terms "computer readable medium", "computer readable medium" and "machine readable medium" used in this specification and any claims of this application are entirely limited to tangible physical objects that store information in a computer readable form. These terms do not include any wireless signals, wired download signals and any other temporary signals.

[0180] Although the present disclosure has been described with reference to many specific details, it will be appreciated by those skilled in the art that the present disclosure may be implemented in other specific forms without departing from the spirit of the present disclosure. Fig.11 and Fig.14 ) conceptually illustrate the processes. The specific operations of these processes may not be performed in the exact order shown and described. Specific operations may not be performed in a continuous series of operations, and different specific operations may be performed in different embodiments. In addition, the process can be implemented using multiple sub-processes or as part of a larger macro-process. Therefore, it will be understood by those skilled in the art that the present disclosure is not limited by the foregoing illustrative details, but is limited by the scope of the appended claims.

[0181] [Additional notes]

[0182] The subject matter described herein sometimes depicts different components contained in or connected to different other components. It should be understood that the architecture depicted in this way is only an example, and in fact, many other architectures that realize the same function can be implemented. Conceptually, any arrangement of components that realize the same function is effectively "associated", thereby realizing the desired function. Therefore, any two components combined here to realize a specific function can be regarded as "associated" with each other so that the desired function is realized, regardless of how the architecture or intermediate components are. Similarly, any two components so associated can also be regarded as "operably connected" or "operably coupled" to each other to realize the desired function, and any two components that can be so associated can also be regarded as "operably coupled" to realize the desired function. Specific examples of operably coupled include but are not limited to physically matched and / or physically interactive components and / or wirelessly interactive and / or wirelessly interactive components and / or logically interactive and / or logically interactive components.

[0183] In addition, with respect to the use of substantially any plural and / or singular terms in the present disclosure, those skilled in the art can translate from the plural to the singular and / or from the singular to the plural, depending on the context and / or application. For clarity, various singular / plural arrangements may be explicitly set forth in the present disclosure.

[0184] In addition, those skilled in the art will understand that, in general, the terms used in the present disclosure, and in particular the terms used in the appended claims, such as the body of the appended claims, are generally intended as "open" terms, for example, the term "including" should be interpreted as "including but not limited to", "having" should be interpreted as "at least", "comprising" should be interpreted as "including but not limited to", etc. Those skilled in the art will further understand that if a specific number is intended to be introduced in the claims, such intention will be explicitly stated in the claims, and in the absence of such a statement, such intention is not present. For example, to aid understanding, the following appended claims may contain narrations that use the introductory phrases "at least one" and "one or more" to introduce the claims. However, the use of such phrases should not be construed as implying that a claim introduced by the indefinite article "a" or "an" limits any particular claim that includes such introduced claim to implementations containing only one such statement, even when the same claim includes the introductory phrases "one or more" or "at least one" and an indefinite article, such as "a" or "an", for example, "a" and / or "an" should be construed as "at least one" or "one or more"; the same applies to the use of definite articles to introduce claim statements. In addition, even if a specific number of the claims introduced is explicitly cited, those skilled in the art will recognize that such a statement should be interpreted as referring to at least the number cited, for example, a statement of "repeatedly cited twice" without other modifiers means at least two repetitions, or two or more repetitions. In addition, in those cases where the definition is similar to "at least one of A, B, C, etc. is used", generally speaking, such a structure is intended to make those skilled in the art understand the specific meaning, for example, "a system having at least one of A, B, and C" will include but is not limited to such a system: A independently, B independently, C independently, A and B at the same time, A and C at the same time, B and C at the same time, and / or A, B, C at the same time, etc. In these cases , using a definition similar to "at least one of A, B or C, etc.", generally speaking, such a structure is intended to make those skilled in the art understand the specific meaning, for example, "a system having at least one of A, B or C" will include but is not limited to such a system having: only A, only B, only C, both A and B, both A and C, both B and C, and / or both A, B, C, etc. Those skilled in the art will further understand that in fact, whether in the specification, the scope of the patent application or the drawings, any disjunctive words and / or phrases that appear with two or more alternative terms should be understood to include the possibility of one of the terms, any term, or both terms. For example, the phrase "A or B" will be understood to include the possibility of "A" or "B" or "A and B".

[0185] It can be understood from the above that for the purpose of illustration, the present disclosure has described various embodiments of the present disclosure, and various modifications can be made without departing from the scope and spirit of the present disclosure. Therefore, the various embodiments disclosed by the present disclosure are not intended to be limiting, and the true scope and spirit are represented by the attached patent scope.

Claims

1. A video encoding and decoding method, comprising: Receiving data to be encoded or decoded as a current block of a current picture of a video; deriving a plurality of gradient histograms for different intra prediction angles, wherein different gradient histograms are derived based on different groups of reconstructed pixels adjacent to the current block; Selecting a gradient histogram from the plurality of gradient histograms; identifying one or more intra prediction modes based on the selected gradient histogram; generating an intra prediction of the current block based on the identified one or more intra prediction modes; as well as The current block is encoded or decoded using the generated intra prediction.

2. The video encoding and decoding method as claimed in claim 1, wherein: Each of the gradient histograms is used to identify a different set of one or more intra-prediction modes.

3. The video encoding and decoding method as claimed in claim 2, further comprising: Costs of the plurality of intra prediction modes identified based on the plurality of gradient histograms are calculated, wherein the selected gradient histogram is selected from the plurality of gradient histograms based on the calculated costs.

4. The video encoding and decoding method as claimed in claim 3, wherein: The cost for each of the plurality of intra prediction modes is calculated by comparing a difference between a template reconstructed sample adjacent to the current block and a template prediction sample generated by the intra prediction mode using an L-shaped reference area near the template.

5. The video encoding and decoding method of claim 2, further comprising: Identifying N candidate intra prediction modes from the gradient histogram having the highest amplitude; Calculating a cost for each of the N candidate intra prediction modes; Based on the calculated costs, M intra-frame prediction modes are selected from the N candidate intra-frame prediction modes as the plurality of intra-frame prediction modes identified based on the gradient histogram.

6. The video encoding and decoding method of claim 1, wherein the plurality of gradient histograms further comprises: A first gradient histogram derived based on reconstructed pixels on an upper side of the current block; a second gradient histogram derived based on reconstructed pixels to the left of the current block; as well as The third gradient histogram is derived based on the reconstructed pixels on the left and above the current block. 7 . The video coding method of claim 6 , wherein the reconstructed pixels at the upper left corner of the current block are not used to derive the third gradient histogram. 8 . The video coding method of claim 1 , wherein the plurality of gradient histograms include gradient histograms derived using different numbers of adjacent lines of the current block.

9. The video coding method of claim 1 , wherein the plurality of gradient histograms include gradient histograms derived based on different neighboring lines of the current block, the different neighboring lines are associated with different gradient histogram generation indexes, wherein the first gradient histogram derived based on the neighboring line associated with the first gradient histogram generation index is used to derive a first intra-frame prediction mode, wherein the intra-frame prediction of the current block based on the first intra-frame prediction mode is generated based on an L-shaped reference area identified by the first gradient histogram generation index.

10. The video coding method of claim 1, wherein the gradient histogram is selected based on a syntax element signaled in a bitstream.

11. An electronic device, comprising: The video codec circuit is configured to perform: receiving data to be encoded or decoded as a current block of a current picture of a video; deriving a plurality of gradient histograms for different intra prediction angles, wherein different gradient histograms are derived based on different groups of reconstructed pixels adjacent to the current block; Selecting a gradient histogram from the plurality of gradient histograms; identifying one or more intra prediction modes based on the selected gradient histogram; generating an intra prediction of the current block based on the identified one or more intra prediction modes; as well as The current block is encoded or decoded using the generated intra prediction.

12. A video decoding method, comprising: receiving data to be decoded as a current block of a current picture of a video; deriving a plurality of gradient histograms for different intra prediction angles, wherein different gradient histograms are derived based on different groups of reconstructed pixels adjacent to the current block; Selecting a gradient histogram from the plurality of gradient histograms; identifying one or more intra prediction modes based on the selected gradient histogram; generating an intra prediction of the current block based on the identified one or more intra prediction modes; as well as The current block is reconstructed using the generated intra prediction.

13. A video encoding method, comprising: Receiving data to be encoded into a bitstream as a current block of a current picture of a video; deriving a plurality of gradient histograms for different intra prediction angles, wherein different gradient histograms are derived based on different groups of reconstructed pixels adjacent to the current block; Selecting a gradient histogram from the plurality of gradient histograms; identifying one or more intra prediction modes based on the selected gradient histogram; generating an intra prediction of the current block based on the identified one or more intra prediction modes; as well as The current block is encoded using the generated intra prediction to generate a prediction residual.

Images