Encoding / decoding video picture data

Abstract

The present disclosure relates to a method of template-based intra prediction of a block of a video picture depending on template-based intra prediction mode information derived depending on at least one reconstructed sample of a template, the template comprising at least one reconstructed sample of a current block neighboring the template-based intra predicted block, wherein the template-based intra prediction mode information is derived at least partially when reconstructing at least one sample of the current block.

Description

Technical Field

[0001] This application generally relates to video frame encoding and decoding. In particular, but not exclusively, the technical field of this application relates to template-based intra-frame prediction of video frame blocks. Background Technology

[0002] This section is intended to introduce the reader to various aspects of the art that may relate to aspects of at least one exemplary embodiment of the present application described below and / or claimed. This detailed description is intended to provide background information to facilitate a better understanding of the various aspects of the present application. Therefore, it should be understood that these statements should be interpreted in this light and not as an admission of prior art. At least one exemplary embodiment is designed to embody the foregoing.

[0003] ECM (Algorithm description of Enhanced Compression Model 10 (ECM 10), JVET-AE2025, Geneva, July 11-19, 2023) defines template-based intra-prediction modes, such as the so-called Decoder-side Intra Mode Derivation (DIMD) mode and the so-called Template-based Intra Mode Derivation (TIMD) mode, for determining the predicted sample of the current block for both luma and chroma samples.

[0004] Based on at least one reconstructed sample of a template defined around the current block, template-based intra-prediction mode information is derived for the current block. Deriving this template-based intra-prediction mode information during decoding the current block introduces a delay for decoding / reconstructing the current block.

[0005] One of the issues is reducing latency for decoding the current block when decoding requires template-based intra-prediction mode information. Summary of the Invention

[0006] The following sections provide a brief overview of at least one exemplary embodiment to provide a basic understanding of some aspects of this application. This overview is not an exhaustive summary of the exemplary embodiments. Its purpose is not to identify key or core elements of the exemplary embodiments. The following overview only gives some aspects of at least one exemplary embodiment in a simplified form, serving as a prelude to a more detailed description provided elsewhere in this document.

[0007] According to a first aspect of this application, a method for determining a block of a video frame based on template-based intra-prediction mode information is provided, wherein the template-based intra-prediction mode information is derived based on at least one reconstructed sample of a template, the template including at least one reconstructed sample of a current block adjacent to the template-based intra-prediction block, wherein the template-based intra-prediction mode information is derived at least in part when reconstructing at least one sample of the current block.

[0008] In some embodiments, template-based intra-frame prediction mode information is stored at the block level.

[0009] In some embodiments, template-based intra-prediction mode information is derived from reconstructed samples of at least one sub-block of the template, and the template-based intra-prediction mode information is stored at the sub-block level, where the sub-block size is smaller than the size of neighboring blocks.

[0010] In some embodiments, the size of the sub-block is fixed, or the size of the sub-block is determined based on the template size or the current block size.

[0011] In some embodiments, at least one reconstructed sample of the template is used to derive more than one template-based intra-prediction mode information.

[0012] In some embodiments, template-based intra-frame prediction mode information is derived from all reconstructed samples of the template.

[0013] In some embodiments, the template-based intra-frame prediction information excludes samples outside the boundaries of the current block, or The template-based intra-frame prediction information is derived based on reconstructed samples outside the boundary of the current block, or The template-based intra-frame prediction information is derived based on the padding samples outside the current block boundary, which are based on the already reconstructed samples inside the current block.

[0014] In some embodiments, template-based intra-frame prediction mode information is derived only when all samples of the template are reconstructed.

[0015] In some embodiments, template-based intra-prediction mode information is derived once at least one sample within a reference block is reconstructed, the reference block referring to the block in which the template for the template-based intra-prediction mode is located.

[0016] In some embodiments, template-based intra-frame prediction mode information is derived when reconstructing each block of a video frame.

[0017] In some embodiments, a flag or index signal is transmitted to the bitstream to indicate that template-based intra-frame prediction mode information is derived from the reconstructed samples of the block during block reconstruction.

[0018] In some embodiments, the video frame is divided into multiple adjacent codec tree units, and when reconstructing samples of blocks located at the boundary between the current codec tree unit and the next codec tree unit, template-based intra-frame prediction mode information is forced to be derived when reconstructing blocks located at the boundary of the next codec tree unit.

[0019] In some embodiments, the video frame is divided into multiple adjacent codec tree units, and template-based intra-prediction mode information must not be included when reconstructing blocks located at the boundaries of codec tree units.

[0020] In some embodiments, a flag or index is signaled to a sequence parameter set, a picture parameter set, or a picture header, or the flag or index is signaled at the slice, tile, codec tree unit, block, prediction unit, or transform unit level.

[0021] In some embodiments, for blocks with a size less than a threshold, template-based prediction mode information is not enabled when reconstructing blocks of a video frame.

[0022] According to a second aspect of this application, an apparatus is provided, comprising components for performing one of the methods described in the first aspect of this disclosure.

[0023] According to a third aspect of this application, a computer program product is provided, including instructions that, when executed by one or more processors, cause the one or more processors to perform the method according to a first aspect of this application.

[0024] According to a fourth aspect of this application, a non-transient storage medium is provided, the storage medium carrying program code instructions for performing the method according to the first aspect of this application.

[0025] The specific properties of at least one of the exemplary embodiments, as well as other objects, advantages, features, and uses of the at least one exemplary embodiment, will become apparent from the following description of the examples taken in conjunction with the accompanying drawings. Attached Figure Description

[0026] Reference will now be made to the accompanying drawings, which illustrate exemplary embodiments of this application, wherein: Figure 1 An example of a codec tree unit based on HEVC is shown; Figure 2 An example of partitioning codec units into prediction units according to HEVC is shown; Figure 3 An example of CTU partitioning based on VVC is shown; Figure 4 An example of the splitting patterns supported in VVC's multi-type tree partitioning is shown; Figure 5 A schematic block diagram of a method 100 for encoding a video frame VP according to the prior art is shown; Figure 6 A schematic block diagram of a method 200 for decoding a video frame VP according to the prior art is shown; Figure 7 An example of a template for calculating gradient histograms according to existing techniques is shown; Figure 8 An example is shown of an intra-predictor that calculates the current block of a video frame based on a weighted linear combination of three intra-predictors, according to existing technology. Figure 9 An example of a co-located luminance block corresponding to a chrominance block is shown, according to the prior art; Figure 10 An example of the definition of non-adjacent spatial neighbor blocks according to existing technology is shown; Figure 11 An example of a first template for template prediction and an example of a second template for calculating prediction costs are shown; Figure 12 An example of a reference line is shown; Figure 13 An example template is shown that is used to obtain a template-based intra-frame prediction pattern to predict the current block of the current block; Figure 14 A block diagram is shown of a method for reconstructing the current block based on template-based intra-frame prediction information patterns, according to the prior art. Figure 15 An example of a template according to at least one embodiment of the present disclosure is illustrated; Figure 16 An example of a HoG for deriving DIMD information according to at least one embodiment of the present disclosure is illustrated; Figure 17 An example of a DIMD-based predicted block of a template of a current block according to at least one exemplary embodiment of the present disclosure is illustrated; Figure 18 The illustration shows an example of deriving multiple template-based intra-frame prediction information according to at least one exemplary embodiment of the present disclosure; Figure 19 An example is shown of deriving template-based intra-frame prediction mode information at the sub-block level according to at least one exemplary embodiment of the present disclosure; Figure 20 An example of a 3x3 Sobel filter according to at least one exemplary embodiment of the present disclosure is shown, which is applied to reconstructed samples of a block of a template to derive template-based intra-frame prediction mode information; Figure 21 The illustration shows an example of obtaining template-based intra-frame prediction mode information once at least one required sample of the template is reconstructed, according to at least one exemplary embodiment of the present disclosure. Figure 22 The illustration shows an example of decoding the current block located at the current CTU boundary according to at least one exemplary embodiment of the present disclosure; Figure 23 The illustration shows an example of transmitting template-based intra-frame prediction mode derived information signaled in a sequence parameter set (SPS) according to at least one exemplary embodiment of the present disclosure; Figure 24 The illustration shows an example of transmitting template-based intra-frame prediction mode derived information signaled in a sequence parameter set (SPS) according to at least one exemplary embodiment of the present disclosure; Figure 25 A block diagram of a system example is shown, in which various aspects and exemplary embodiments are implemented.

[0027] Similar or identical elements are indicated by the same reference numerals. Detailed Implementation

[0028] At least one of the exemplary embodiments will be described more fully below with reference to the accompanying drawings, which depict examples of at least one of the exemplary embodiments. However, the exemplary embodiments may be implemented in various alternative forms and should not be construed as limited to the examples set forth herein. Accordingly, it should be understood that the present invention is not intended to limit the exemplary embodiments to the specific forms disclosed. Rather, this application is intended to cover all modifications, equivalents, and alternatives falling within the spirit and scope of this application.

[0029] At least one of these aspects generally relates to video image encoding and decoding, another generally relates to the transmission of a bit stream provided or encoded, and yet another relates to the reception / access of a decoded bit stream.

[0030] At least one of the exemplary embodiments is described in relation to encoding / decoding a single video frame, but is extended to encoding / decoding multiple video frames (a sequence of frames), since each video frame is encoded / decoded sequentially as described below.

[0031] Moreover, for example, at least one exemplary embodiment is not limited to MPEG standards, such as AVC (ISO / IEC 14496-10 Advanced Video Coding for generic audio-visual services, ITU-T Recommendation H.264, https: / / www.itu.int / rec / T-REC-H.264-202108-P / en), EVC (ISO / IEC 23094-1 Essential video coding), HEVC (ISO / IEC 23008-2 High Efficiency Video Coding, ITU-T Recommendation H.265, https: / / www.itu.int / rec / T-REC-H.265-202108-P / en), and VVC (ISO / IEC 23090-3 Versatile Video Coding, ITU-T...). Recommendation H.266 (https: / / www.itu.int / rec / T-REC-H.266-202008-I / en) can be applied, but may also be applied to other standards and recommendations, such as AV1 (AOMedia Video 1, http: / / aomedia.org / av1 / specification / ). At least one exemplary embodiment may be adapted to existing or future developments and extensions of any such standard and recommendation. Unless otherwise stated or technically excluded, the aspects described in this application may be used individually or in combination.

[0032] A pixel corresponds to the smallest display unit on the screen, and it can consist of one or more light sources (one for a monochrome screen and three or more for a color screen).

[0033] A video frame, also known as a frame or video frame, includes at least one component (also known as a frame component or channel) defined by a specific video / video format that specifies all information related to pixel values ​​and all information that can be used by a display unit and / or any other device to display and / or decode video frame data associated with the video frame.

[0034] The video frame includes at least one component, which is typically represented as an array of samples.

[0035] Monochrome video footage consists of a single component, while color video footage can consist of three components.

[0036] For example, when the picture / video format is a well-known (Y,Cb,Cr) format, a color video picture can include one luminance (or brightness) component and two chrominance components, while when the picture / video format is a well-known (R,G,B) format, a color video picture can include three color components (one for red, one for green, and one for blue).

[0037] Each component of a video frame can include a number of samples relative to the number of pixels on the screen on which the video frame is to be displayed. In variations, the number of samples included in a component can be a multiple (or fraction) of the number of samples included in another component of the same video frame.

[0038] For example, in a video format that includes one luminance component and two chrominance components (such as the (Y,Cb,Cr) format), depending on the color format under consideration, the chrominance component may contain half the number of samples in width and / or height relative to the luminance component.

[0039] A sample is the smallest unit of visual information that makes up a video frame. Sample values ​​can be, for example, luminance or chrominance values, or color values ​​in (R, G, B) format.

[0040] A pixel value is the value of a pixel on the screen. For monochrome video, a pixel value can be represented by a single sample, while for color video, a pixel value can be represented by multiple co-localized samples. A co-localized sample associated with a pixel refers to the sample corresponding to the pixel's position on the screen.

[0041] Video frames are typically viewed as a set of pixel values, with each pixel represented by at least one sample.

[0042] A video frame block is a set of samples of one component of a video frame. When the frame / video format is a well-known (Y,Cb,Cr) format, a block of at least one luminance sample or at least one chrominance sample can be considered; or when the frame / video format is a well-known (R,G,B) format, a block of at least one color sample can be considered.

[0043] At least one exemplary embodiment is not limited to a specific image / video format.

[0044] In state-of-the-art video compression systems, such as HEVC or VVC, low-level and high-level picture partitioning is provided to divide the video frame into picture blocks, called codec tree units (CTUs). The size of these units is typically between 16x16 and 64x64 pixels for HEVC, and between 32x32, 64x64, or 128x128 pixels for VVC.

[0045] The CTU (Central Unit) division of a video frame forms a grid composed of CTUs of a fixed size; that is, a CTU grid whose top and left edges coincide spatially with the top and left edges of the video frame. The CTU grid represents the spatial partitioning of the video frame.

[0046] In VVC and HEVC, the CTU size (CTU width and CTU height) of all CTUs in the CTU mesh is equal to the same default CTU size (default CTU width CTU DW and default CTU height CTU DH). For example, the default CTU size (default CTU height, default CTU width) can be equal to 128 (CTU DW = CTU DH = 128). The default CTU size (height, width) is encoded into the bitstream, for example, at the sequence level in the Sequence Parameter Set (SPS).

[0047] The spatial position of a CTU within the CTU mesh is determined by its address ctuAddr, which defines the spatial position of the CTU's top-left corner relative to the origin. For example... Figure 1 As shown, the CTU address can define a spatial location starting from the top left corner of the higher-level spatial structure S containing the CTU.

[0048] Each CTU is associated with a codec tree to determine the tree partitioning of the CTU.

[0049] like Figure 1 As shown, in HEVC, the codec tree is a quadtree partition of CTUs, where each node is called a codec unit (CU). The spatial location of a CU in a video frame is defined by the CU index cuIdx, which indicates the spatial location starting from the top left corner of the CTU. CUs are spatially partitioned into one or more prediction units (PUs). The spatial location of a PU in a video frame (VP) is defined by the PU index puIdx, which defines the spatial location starting from the top left corner of the CTU, and the spatial location of the elements of a partitioned PU is defined by the PU partition index puPartIdx, which defines the spatial location starting from the top left corner of the PU. Each PU is assigned some intra-frame or inter-frame prediction data.

[0050] The intra-frame or inter-frame encoding / decoding mode is assigned at the CU level. This means that although the prediction parameters vary from PU to CU, each PU is assigned the same intra-frame / inter-frame encoding / decoding mode.

[0051] According to a quadtree called a transform tree, a CU can also be spatially partitioned into one or more transform units (TUs). A transform unit is a leaf of the transform tree. The spatial location of a TU in a video frame is defined by a TU index tuIdx, which defines the spatial location starting from the top-left corner of the CU. Each TU is assigned a number of transform parameters. The transform type is assigned at the TU level, and a separate 2D transform is performed at the TU level during the encoding / decoding or decoding of a frame block.

[0052] Figure 2 The diagram above illustrates the existing PU partition types in HEVC. These include square partitions (2Nx2N and NxN), which are the only partitions used in both intra-frame and inter-frame prediction CUs; ​​symmetric non-square partitions (2NxN, Nx2N, used only in inter-frame prediction CUs); and asymmetric partitions (used only in inter-frame prediction CUs). For example, PU type 2NxnU represents an asymmetric horizontal partition of the PU, where the smaller partition is located at the top of the PU. According to another example, PU type 2NxnL represents an asymmetric horizontal partition of the PU, where the smaller partition is located at the top of the PU.

[0053] like Figure 3 As shown, in VVC, the codec tree starts from the root node (i.e., CTU). Next, a quadtree (or quaternion tree) split divides the root node into four nodes, corresponding to four equal-sized sub-blocks (solid lines). Then, the quadtree (or quaternion tree) leaves can be further partitioned using a so-called multi-type tree, which involves... Figure 4 One of the four splitting patterns shown is used for binary or ternary splits. These splitting types are vertical and horizontal binary splitting patterns (denoted as SBTV and SBTH) and vertical and horizontal ternary splitting patterns SPTTV and STTH.

[0054] In the case of a joint codec tree where luminance and chrominance components share a common codec tree, the leaf of the CTU's codec tree is the CU.

[0055] In contrast to HEVC, in VVC, CU, PU, ​​and TU are of equal size in most cases, which means that, except in some specific codec modes, codec units are generally not partitioned into PU or TU.

[0056] Figure 5 and Figure 6 It provides an overview of video encoding / decoding methods used in current video standard compression systems such as HEVC or VVC.

[0057] Figure 5 A schematic block diagram illustrating the steps of a method 100 for encoding a video frame VP according to the prior art is shown.

[0058] In step 110, the video frame VP is partitioned into sample blocks, and the partition information data is signaled and transmitted to the bitstream. Each block includes a sample of one component of the video frame VP. Therefore, these blocks include samples that define each component of the video frame VP.

[0059] For example, in HEVC, the frame is divided into codec tree units (CTUs). Each CTU can be further subdivided using a quadtree partition, where each leaf of the quadtree represents a codec unit (CU). The partition information data can then include data describing the CTU and the quadtree subdivision of each CTU.

[0060] Therefore, each sample block (or block for short) can be a CU (if the CU includes a single PU) or a PU of the CU.

[0061] Using intra-frame or inter-frame prediction modes, each block is encoded along the coding cycle (also known as "in the cycle").

[0062] Intra-frame prediction (step 120) uses intra-frame prediction data. Intra-frame prediction involves predicting the current block using blocks of intra-frame prediction based on samples that have been encoded, decoded, and reconstructed, located around the current block, typically at the top and left of the current block. Intra-frame prediction is performed in the spatial domain.

[0063] In inter-frame prediction mode, motion estimation (step 130) and motion compensation (135) are performed. Motion estimation searches for candidate reference blocks in one or more reference video frames used for predictively encoding the current video frame as good predictors for the current block. For example, a good predictor for the current block is a predictor similar to the current block. The output of motion estimation step 130 is inter-frame prediction data, which includes motion information associated with the current block (typically one or more motion vectors and one or more reference video frame indices) and other information for obtaining the same prediction block on the encoding / decoding side. Next, motion compensation (step 135) obtains the prediction block using the motion vector(s) and reference video frame(s) determined by motion estimation step 130. Essentially, a block belonging to a selected reference video frame and pointed to by a motion vector can be used as the prediction block for the current block. Furthermore, since motion vectors are expressed as fractions of integer pixel positions (this is called sub-pixel accuracy motion vector representation), motion compensation generally involves spatial interpolation of some reconstructed samples of the reference video frames to compute the prediction block.

[0064] The prediction information data is signaled and transmitted into the bitstream. The prediction information may include the prediction mode (intra-frame, inter-frame, or skipped), intra-frame / inter-frame prediction data, and any other information used to obtain the same prediction CU on the decoding side.

[0065] Taking into account the encoding of the calculated prediction residual block (e.g., by subtracting the candidate prediction block from the current block) and the signaling of the prediction information data required to determine the candidate prediction block on the decoding side, method 100 selects a prediction mode (intra-frame or inter-frame prediction mode) by optimizing the rate-distortion tradeoff.

[0066] Typically, intra-frame and inter-frame prediction modes are set in a "most likely candidate mode list (MPM list)," and the best prediction mode is given by the following formula for the current block. p Given: , in P It is the list of MPMs for the current block. p This indicates the candidate codec modes in the MPM list. RD cost（p） It is a candidate prediction model p The rate-distortion cost is usually expressed as: , D ( p () is the current block and the candidate prediction pattern p Distortion between reconstructed blocks obtained after encoding / decoding the current block R ( p (This is based on the prediction model) p The rate cost associated with encoding and decoding the current block, and λ It is a Lagrangian parameter representing the rate constraint for encoding and decoding the current block, and is usually calculated based on the quantization parameter Qp used to encode the current block.

[0067] The current block is typically encoded from the prediction residual block PR. More precisely, for example, the prediction residual block PR is calculated by subtracting the best prediction block from the current block. The prediction residual block PR is then transformed using a transformation of type DCT (Discrete Cosine Transform) or DST (Discrete Sine Transform) or any other suitable transformation (step 140), and the resulting transformed coefficient block is quantized (step 150).

[0068] In a variant, method 100 can also skip transform step 140 according to a so-called transform-skip encoding / decoding mode and directly apply quantization to the predicted residual block PR (step 150).

[0069] Encode the quantized transform coefficient block (or quantized prediction residual block) into the bit stream (step 160).

[0070] Next, as part of the encoding loop, the quantized transform coefficient block (or quantized residual block) is dequantized (step 170) and inverse transformed (180) (or not) to obtain the decoded prediction residual block. Then, the decoded prediction residual block and the prediction block are combined (usually summed) to provide the reconstructed block.

[0071] In step 160, other information data can also be entropy encoded to encode the current block of the video frame VP.

[0072] A loop filter (step 190) can be applied to the reconstructed frame (including reconstructed blocks) to reduce compression artifacts. Loop filters can be applied after all frame blocks have been reconstructed. Examples include deblocking filters, Sample Adaptive Offset (SAO) filters, or adaptive loop filters.

[0073] The reconstructed block or the filtered reconstructed block forms a reference frame, which can be stored in the decoded frame buffer (DPB) so that it can be used as a reference frame for encoding the next current block of the video frame VP or as a reference frame for encoding the next video frame to be encoded.

[0074] Figure 6 A schematic block diagram of the steps of a method 200 for decoding a video frame VP according to the prior art is shown.

[0075] In step 210, partition information data, prediction information data, and quantized transform coefficient blocks (or quantized residual blocks) are obtained by entropy decoding of the bitstream of encoded video frame data. For example, this bitstream has been generated according to method 100.

[0076] It can also perform entropy decoding on other information data to decode the current block of the video frame VP from the bitstream.

[0077] In step 220, the reconstructed image is divided into current blocks based on the partitioning information. Each current block is entropy-decoded from the bitstream along the decoding loop (also known as "in loop"). Each decoded current block is either a quantized transform coefficient block or a quantized prediction residual block.

[0078] In step 230, the current block is dequantized and possibly inverse transformed (step 240) to obtain the decoded prediction residual block.

[0079] On the other hand, prediction information data is used to predict the current block. The predicted block is obtained through its intra-frame prediction (step 250) or its motion-compensated temporal prediction (step 260). The prediction process performed on the decoding side is exactly the same as the prediction process on the encoding side.

[0080] Next, the decoded prediction residual blocks and prediction blocks are combined (usually by summation), which provides the reconstructed blocks.

[0081] In step 270, the loop filter can be applied to the reconstructed frame (including the reconstructed block), and the reconstructed block or the filtered reconstructed block forms a reference frame, which can be stored in the decoded frame buffer (DPB), as discussed above. Figure 5 ).

[0082] exist Figure 5 Step 130 / 135 or Figure 6 In step 260, an inter-frame prediction block is defined based on inter-frame prediction data associated with the current block (CU or PU in the CU) of the video frame. This inter-frame prediction data contains motion information, which can be represented (encoded / decoded) according to either a so-called block-based Adaptive Motion Vector Prediction (AMVP) mode or a so-called block-based merging mode.

[0083] ECM defines a DIMD mode for determining the predicted samples of both the luma and chroma samples of the current block. DIMD mode is not based on a linear-model (LM) mode; in short, it is a non-LM mode, i.e., an intra-prediction mode that does not reference a linear model. Furthermore, DIMD mode does not require explicit parameter signaling, such as planar prediction mode or direct mode (DM).

[0084] For luminance sample prediction, the use of the DIMD luminance pattern is signaled in the bitstream via a single flag, and the intra-predictor is not explicitly signaled in the bitstream. Instead, the intra-predictor is derived based on gradient analysis of neighboring reconstructed luminance samples; that is, the DIMD luminance pattern is implicit. Therefore, the intra-predictor is derived in the same way during the reconstruction process on both the encoder and decoder sides based on the DIMD luminance pattern.

[0085] In DIMD brightness mode, such as Figure 7The illustration defines an L-shaped template surrounding the current luma block. For example, an L-shaped template is defined as a template 3-sample wide (in width or height) consisting of reconstructed luma samples from the left, top, and upper left of the reconstructed region R, e.g., an L-shaped template. Edge detection filters (3x3 horizontal and vertical Sobel filters) adjacent to the current block B (current CU) are applied to the samples in the L-shaped template to determine the magnitude and angle of the luma orientation for each middle row sample in the (available) template. A gradient histogram (HoG) is computed, where each entry corresponds to a regular intra-frame angle pattern, and the cumulative intensity (magnitude) is stored. ; , Among them G hor and G ver These are the pure horizontal and vertical intensities calculated by the Sobel filter. Regular intra-angle modes refer to the intra-angle modes existing in specifications such as HEVC and VVC, where each angle mode is specifically designed to capture the direction or angle of the texture. Note that the term non-angle mode refers to an intra-angle mode that does not capture the direction or angle of the texture. For example, Direct Coding (DC) mode and planar intra-angle modes can be considered non-angle modes.

[0086] In DIMD luma mode, luma blending (mixing, fusion) is applied as a weighted average of multiple luma predictors for the current block: two HoG-based predictors and one planar predictor (M. Abdoli et al., “Non-CE3: Decoder-side Intra Mode Derivation with Prediction Fusion Using Planar”, JVET-O0449, Gothenburg, July 2019).

[0087] The planar predictor is derived from the planar pattern, where the predicted sample values ​​are obtained by a weighted average of four reference samples used as an L-shaped template. Here, the reference samples are located in the same row or column as the current sample of the current block, and reference samples located relative to the lower left and upper right positions of the current block are used. In VVC, depending on the block size, a reference sample smoothing filter is also applied to the planar pattern in the luminance component.

[0088] like Figure 8As illustrated, the predicted block for the current block is derived from a weighted linear combination of, for example, the following six predictors: five HoG-based predictors are derived using the five most representative angular intra-frame modes Mi (i=1 to 5), while one predictor is derived using a planar mode. The number of predictors may be fewer or more than six. Figure 8 The illustration shows a variant where the predicted block is derived only as a weighted linear combination of three HoG-based predictors, but the principle can be easily extended to six predictors: two HoG-based predictors M1 and M2 selected from HoG (with the largest histogram magnitude values) and a third predictor derived using a planar mode. The weights w1, w2, and w3 used in the weighted linear combination of these three predictors are derived, for example, as follows: the planar weight is fixed at 21 / 64 (approximately 1 / 3). The remaining 43 / 64 (approximately 2 / 3) weights are proportionally distributed among the other two weights to the magnitudes of their HoG histogram bars. As another example, for a block of size W×H, the three weights are derived as follows: weights are derived for each of the three predictors as discussed below regarding the DIMD merging mode. wDimd i Next, based on the following weights... wDimd i The weights of each of the three resulting patterns are determined. w i If the HoG amplitude above or to the left is twice that of the other, then the weights are... w i ( x , y Location-related and calculated as follows: If the upper HoG is twice the left HoG, then: , If the HoG on the left is twice the HoG on the top, then: , in It is predefined and can be set to 10.

[0089] For chroma sample prediction, the use of the DIMD chroma mode is signaled in the bitstream via a single flag, and the intra-predictor is not explicitly signaled in the bitstream. Instead, the intra-predictor is derived using gradient analysis of adjacent reconstructed luma and chroma samples; that is, the DIMD chroma mode is implicit. Therefore, the intra-predictor is derived in the same manner during the reconstruction process on both the encoder and decoder sides based on the DIMD chroma mode.

[0090] In DIMD chroma mode, an intra-frame predictor for a chroma block can be derived based on an L-shaped template or a portion thereof. For example, the L-shaped template may include co-bit reconstructed luminance samples of the luminance block associated with the chroma block to be predicted. Specifically, for each co-bit reconstructed luminance sample of the template ( Figure 9 The gray circles in the image are taken from JVET-Y0092 (HyeongmunJang et al., “CE5-2.5: De-block filtering with 4x4 chroma sample grid”, JVET-Y0092, Gothenburg, July 2019). The horizontal and vertical gradients are calculated to construct the HoG. Then, the intra-prediction mode with the largest histogram magnitude value is selected from the HoG.

[0091] In a variant, in DIMD chroma mode, an intra-frame predictor for the chroma block is obtained based on a template comprising neighboring reconstructed chroma samples of an L-shaped template surrounding the chroma block.

[0092] In a variant, an intra-frame predictor for the chroma block is obtained based on a template and neighboring reconstructed chroma samples, the template including co-located reconstructed luminance samples of the luminance block associated with the chroma block to be predicted.

[0093] Typically, when the intra-prediction mode derived from the DIMD chroma mode is the same as the intra-prediction mode derived from the DM (Direct Mode) mode, the intra-prediction mode with the second largest histogram amplitude value is selected as the DIMD chroma mode.

[0094] JVET-AF0120 (Saverio Blasi et al., “EE2-2.1 DIMD merge”, JVET-AF0120, Hanover, October 2023) also proposes a DIMD merging mode. When using the DIMD merging mode, the predicted block for the current block is derived from DIMD information extracted from neighboring blocks. First, the merged histogram of gradients (HoG) of the current block is calculated based on the histogram of gradients (HoG) of at least one DIMD neighboring block (i.e., the neighboring blocks to which the predicted block is encoded / decoded, and the predicted block is derived based on DIMD or the DIMD merging mode). When a single DIMD neighboring block is available, its gradient histogram (HoG) is used to form the MHoG of the current block. If more than one DIMD neighboring block is available, the corresponding HoGs are combined using magnitude averaging to obtain the MHoG. Up to 13 DIMD neighboring blocks are considered for DIMD information extraction. Next, as discussed above regarding DIMD modes, MHoG is used to define the DIMD merging mode and weights. Regular intra-frame angle modes and their weights corresponding to the five highest amplitudes in the MHoG are selected, and the corresponding predictors are mixed as in DIMD merging mode.

[0095] The DIMD information used to derive the predictor for the current block depends on the DIMD neighboring blocks. In JVET-AF0106 (Junyan Huo et al., “EE2-related: Non-adjacent spatial candidates for DIMDmerge”, Hanover, October 2023), the DIMD neighboring block may be a non-adjacent spatial neighboring block, that is, a neighboring block of the current block that does not share corners or edges with the current block.

[0096] For example, Figure 10 An example of defining non-adjacent spatial neighbor blocks is shown. The distance between the current block and its non-adjacent spatial neighbor blocks can be defined based on the width and height of the current block.

[0097] DIMD modes or DIMD merging modes (for luminance or chrominance) are associated with related information such as weights discussed above. The information associated with DIMD modes or DIMD merging modes can be stored in memory for blocks of video frames (VPs) that are encoded / decoded based on prediction blocks derived from DIMD modes or DIMD merging modes.

[0098] The ECM further defines Template-based Intra Mode Derivation (TIMD) modes. TIMD modes are used to predict blocks and result in the intra predictor for the block being derived from previously reconstructed samples of an L-shaped template or from a portion of an L-shaped template defined around the block. The TIMD mode associated with the block is also associated with the prediction cost evaluated based on samples of the L-shaped template or a portion of the L-shaped template.

[0099] For example, a TIMD mode can be derived for each intra-prediction candidate mode in the MPM list and for each wide-angle mode (if the upper right and / or lower left samples of the first template T1 are available). Wide-angle modes are described in JVET-Q2002 (Algorithm description for Versatile Video Coding and Test Model 8 (VTM 8), JVET-Q2002, Brussels, January 7-17, 2020). In short, the regular angle intra-prediction direction is defined as clockwise from 45 degrees to -135 degrees. In VVC, for non-square blocks, several regular angle intra-prediction modes are adaptively replaced by wide-angle intra-prediction modes. The replaced modes are signaled using the original mode indices, which are remapped to the wide-angle mode indices after resolution. The number of modes replaced in the wide-angle direction mode depends on the block's aspect ratio. In the case of wide-angle intra-prediction, two vertically adjacent prediction samples can use two non-adjacent reference samples. Therefore, low-pass reference sample filtering and side smoothing are applied to wide-angle prediction to reduce gaps. The negative impact of increased pα. If the wide-angle mode represents a non-fractional offset, then there are 8 modes in the wide-angle mode that satisfy this condition, namely [-14, -12, -10, -6, 72, 76, 78, 80]. When predicting blocks using these modes, samples from the reference buffer are directly copied without applying any interpolation. This modification reduces the number of samples that need to be smoothed. Furthermore, it aligns the design of the regular prediction mode with that of the non-fractional modes in the wide-angle mode.

[0100] The prediction cost associated with a TIMD mode can be evaluated based on the sum of absolute transformed differences (SATD), sum of absolute differences (SAD), or sum of squared errors (SSE) between samples of the second template T2 of the block and samples of the reconstructed template of the block. Samples of the reconstructed template of the block are samples located in the second template T2 of the block that are encoded and decoded according to intra-prediction candidate modes or wide-angle modes (if present). A list of TIMD candidate modes associated with the prediction cost can then be constructed. Next, two intra-prediction modes, TIMD1 and TIMD2, with the minimum prediction cost are selected from the list of TIMD candidate modes. The condition between the prediction costs of the two selected TIMD modes is then checked to determine whether a mixture of the two selected TIMD modes is applied.

[0101] For example, the condition is met when the prediction cost of the second selected TIMD pattern, TIMD2, is less than twice the prediction cost of the first selected TIMD pattern, TIMD1 (corresponding to the minimum prediction cost). cost(TIMD1). If the condition is met, fusion is applied; otherwise, the TIMD pattern added to the MPM list is the first TIMD pattern, TIMD1.

[0102] Figure 11 An example of a first template T1 for the block (current CU) used for template prediction and an example of a second template used to calculate the prediction cost are shown. PDPC weights are obtained by applying PDPC (Position-Dependent Intra-Prediction Combination) filtering to the predictions derived based on these two selected TIMD modes. These PDPC weights are similar to those used in the PDPC process of VVC, where the weights depend on the block size. Thus, the predictions derived based on these two selected TIMD modes can be mixed with the PDPC weights, and this weighted intra-prediction is used to encode and decode the current block.

[0103] The weights can be calculated based on the predicted costs, as follows: weight1 = costMode2 / (costMode1+ costMode2) weight2 = 1 - weight1 Therefore, TIMD modes can be associated with information indicating the two selected TIMD modes and a blending (mixing) flag indicating whether blending of the two selected TIMD modes is enabled. This association information may also indicate weights weight1 and weight2 and / or prediction cost. The information associated with the TIMD modes can be stored in memory for blocks of video frames (VPs) encoded / decoded based on prediction blocks derived from the TIMD modes.

[0104] The ECM further defines an intra-prediction fusion method that can be used to derive the predicted block of the current block as a weighted combination of multiple intra-prediction blocks generated according to different reference lines. A reference line refers to samples from specific rows and / or columns of the reconstructed block used for intra-prediction. For example, samples from the immediate row adjacent to the upper reference side of the current block are called reference line 0. Figure 12 An example of a reference line is illustrated. Basically, in intra-prediction fusion methods, multiple intra-prediction blocks are first generated, and then fused using a weighted average. Intra-prediction fusion methods can be summarized as follows: For angular intra-prediction modes including TIMD and DIMD, the intra-prediction fusion method derives an intra-prediction block by weighting the intra-prediction blocks obtained from multiple reference lines, represented as... ,in The first intra-prediction block is from reference line 0 (the immediate reference line from the top and / or left block edge), and the second intra-prediction block is from the reference line above the default reference line. Weights are set to w0=3 / 4 and w1=1 / 4. The angular intra-prediction mode can be obtained in various ways. For example, the same intra-prediction mode derived from TIMD and DIMD modes can be used. Alternatively, the intra-prediction mode can be determined based on rate-distortion optimization on the encoder side and signaled to the decoder in the bitstream.

[0105] For TIMD modes with hybridity, For the first selected TIMD mode TIMD1 ( w 0=1 w 1=0); will For the second selected TIMD mode TIMD2 ( w 0=0、 w 1=1).

[0106] For DIMD modes with hybrid features, the number of intra-prediction blocks selected for weighted averaging is increased from 3 to 6.

[0107] When the intra-frame mode has a non-integer slope (requiring reference sample interpolation) and the block size is greater than 16, the intra-frame prediction fusion method is applied to the luma block. This method is used in conjunction with Multiple Reference Lines (MRLs), such as... Figure 12 The diagram shown is not applicable to ISP codec blocks. ISP is an intra-frame sub-partition as defined in JVET-Q2002.

[0108] Derivation of template-based intra-prediction modes (such as DIMD and TIMD) for the current block requires memory access to reconstructed samples (called templates) in the neighborhood of the current block. Memory access to samples of the template occurs at the start of the reconstruction process for the current block, thus introducing an additional step (delay) into the decoder's reconstruction pipeline.

[0109] Figure 13 An example template is shown that is used to obtain a template-based intra-frame prediction mode to predict the current block of the current block.

[0110] like Figure 13 As shown in the illustration, a template is typically defined as an L-shaped template (or a portion of an L-shaped template) surrounding the current block (the shaded block). Only the left or top portion of the L-shaped template can be used.

[0111] The template can include samples from at least one of the current block's neighboring blocks. For example, all samples from neighboring blocks could be like... Figure 13 The sample is a template illustrated in block 131. For example, only a portion of the samples from neighboring blocks may be as follows: Figure 13 The sample of the template illustrated in block 132.

[0112] The template-based intra-prediction of the current block to be predicted is determined based on the template-based intra-prediction mode information.

[0113] Determining template-based intra-frame prediction mode information may include the following steps: retrieving rows of reconstructed samples in a template surrounding the current block, possibly applying filtering, calculating weights, and / or calculating gradient histograms.

[0114] The steps described above for obtaining the template-based intra-prediction mode information for the current block introduce additional latency when reconstructing the current block. This is because reconstructing the current block based on the template-based intra-prediction mode requires waiting for the reconstruction of all the required samples of the template before obtaining the template-based intra-prediction mode information for the current block.

[0115] Figure 14 A block diagram is shown of a method for reconstructing the current block based on template-based intra-frame prediction mode information, according to the prior art.

[0116] The gray shaded blocks indicate steps for reconstructing the current block that introduce latency and memory access problems. Dashed boxes can be optional. For example, DIMD and / or TIMD may not require filtering of template samples, or if, for example, the method is only intended to derive a template-based intra-prediction mode for the current block, then the mixing weights may not need to be computed. On the other hand, solid boxes are mandatory and are considered to always be performed for reconstructing the current block.

[0117] The challenge lies in reducing the latency and / or memory accesses used to derive template-based intra-prediction mode information for the current block.

[0118] At least one exemplary embodiment of this application is designed with the above circumstances in mind.

[0119] Embodiments of this disclosure relate to a template-based intra-prediction method for determining blocks of a video frame based on template-based intra-prediction mode information derived from at least one reconstructed sample of a template. The template includes at least one reconstructed sample of a current block adjacent to the template-based intra-prediction block, and the template-based intra-prediction mode information is derived at least partially during the reconstruction of at least one sample of the current block.

[0120] Since template-based intra-prediction mode information is derived at least partially during the reconstruction of at least one sample of the current block, the template-based intra-prediction mode information of the template-based intra-prediction block has already (at least partially) been determined when reconstructing the samples of the template-based intra-prediction block. Therefore, one advantage of embodiments of this disclosure is reduced latency. Furthermore, embodiments of this disclosure avoid retrieving reconstructed samples from previous blocks. Instead, the already computed template-based intra-prediction mode information is retrieved. This means another advantage of embodiments of this disclosure is reduced memory access during the reconstruction of template-based intra-prediction blocks.

[0121] In the following description, embodiments of this disclosure are presented in DIMD and TIMD modes. However, this disclosure extends to any other template-based intra-prediction mode.

[0122] In the following text, the term "template" refers to the reconstructed samples in the neighborhood of a template-based intra-prediction block, which are used to derive template-based intra-prediction mode information. The template may contain all reconstructed samples of an L-shape, or it may contain only at least one reconstructed sample from the left portion of the L-shaped template, or it may contain only at least one reconstructed sample from the upper portion of the L-shaped template.

[0123] Figure 15The illustration shows an example template according to at least one embodiment of the present disclosure. The sizes of the upper and left portions of the template may be different or the same. For example, the size of each portion of the template may depend on the size (height and width) of the current block.

[0124] In the following text, the term "reconstruction" refers to the process of building a block of samples by adding the predictions of the samples to the residuals. Reconstructing a block means reconstructing the samples of that block.

[0125] In some embodiments, template-based intra-frame mode information may include at least one of the following: - At least one intra-frame prediction direction or pattern; - At least one gradient value; - At least one gradient histogram; - At least one mixed or fused weight; - At least one template cost.

[0126] When reconstructing samples of the current block adjacent to the template-based intra-prediction block to be reconstructed, template-based intra-mode information may be at least partially stored in memory. When the stored template-based intra-mode information is required to reconstruct the template-based intra-prediction block, at least a portion of the template-based intra-mode information may be retrieved from memory.

[0127] For example, if DIMD information is required to reconstruct DIMD-based prediction blocks adjacent to the current block, then the DIMD information can be obtained when reconstructing samples of the current block. The DIMD information may include HoG information calculated based on reconstructed samples of a template (including at least one reconstructed sample of the current block). For example, the HoG information may be calculated based on reconstructed samples located in the upper and left portions of the template. The calculated HoG information can be stored. When reconstructing a DIMD-based prediction block, the HoG information can be obtained, then the obtained HoG information is accumulated, and the accumulated HoG information can be used to determine the DIMD-based prediction of the DIMD-based prediction block, such as... Figure 16 As shown in the diagram.

[0128] For example, if DIMD information is required to reconstruct a DIMD-based prediction block adjacent to the current block, then when reconstructing the current block, obtaining the DIMD information may include: selecting at least one mode with the highest HoG magnitude based on reconstructed samples of the template (including at least one reconstructed sample of the current block); and when reconstructing the DIMD-based prediction block, at least one of the selected modes may be used as an intra-prediction mode for the DIMD-based prediction block.

[0129] For example, if TIMD information is required to reconstruct TIMD-based prediction blocks adjacent to the current block, then deriving the TIMD information when reconstructing samples of the template (including at least one sample of the current block) may include: calculating the intra-mode and the prediction cost of the intra-mode based on at least one reconstructed sample of the template. When reconstructing the TIMD-based prediction block, the intra-mode and prediction cost can be obtained to derive the TIMD information based on the aggregation of the intra-mode and prediction cost.

[0130] In some embodiments, template-based intra-frame prediction mode information is stored in memory at the block level.

[0131] In some embodiments, the blocks of the template are reconstructed according to the decoding order.

[0132] Figure 15 An example of the decoding order of blocks of a template according to at least one exemplary embodiment of the present disclosure is shown.

[0133] exist Figure 15 In the example, block 7 refers to the template-based intra-prediction block adjacent to the current block (one of blocks 1 through 6). Block 7 will be reconstructed based on template-based intra-prediction mode information. Assume block 1 is reconstructed, followed by block 2, then block 3, then block 4, then block 5, then block 6, and finally block 7. The template-based intra-prediction mode information for block 7 is derived and stored during the reconstruction of blocks 1 through 6, and during the reconstruction of block 7, the stored template-based intra-prediction mode information is retrieved from memory.

[0134] For example, if block 7 is to be reconstructed using DIMD mode, then when reconstructing at least one of blocks 1 through 6, HoG information is calculated (possibly partially) for each of blocks 1 through 6 based on at least one reconstructed sample from blocks 1 through 6. The HoG information calculated for each of blocks 1 through 6 is then stored at the block level. When reconstructing block 7, the stored HoG information associated with blocks 1 through 6 can be retrieved and used to determine the template-based intra-prediction mode information for block 7. Therefore, when reconstructing block 7, the reconstructed samples from blocks 1 through 6 (which are required to derive the template-based intra-prediction mode information for blocks 1 through 6) do not need to be revisited.

[0135] Similarly, if block 7 is to be reconstructed using TIMD mode, the template-based intra-prediction mode information is calculated and stored when reconstructing blocks 1 through 6, and the template-based intra-prediction mode information is retrieved from memory when reconstructing block 7, instead of obtaining the template-based intra-prediction mode information by revisiting the reconstructed samples of blocks 1 through 6 when reconstructing block 7.

[0136] In some embodiments, template-based intra-prediction mode information may include only selected information from a list of information related to the template-based intra-prediction mode.

[0137] This is advantageous because storage requirements are reduced.

[0138] For example, when the HoG in a DIMD pattern is computed based on reconstructed samples of a template, only M gradient values ​​(with the highest amplitude) out of N candidate gradient values ​​can be stored. Accordingly, only the M patterns associated with the selected HoG used for storage are stored.

[0139] For example, for TIMD, among N candidate prediction models, only the M best prediction models with the lowest prediction cost values ​​can be stored.

[0140] In some embodiments, the TIMD cost can be normalized before storage to take into account the number of samples, and the cost can be calculated based on the number of samples.

[0141] Specifically, if the cost metric is based on averages, then the actual TIMD block can be normalized to the different costs of blocks from templates of different sizes.

[0142] In some embodiments, the HoG values ​​in the DIMD pattern may be normalized before being stored in memory to account for the number of samples for which HoG is calculated.

[0143] In some embodiments, template-based intra-frame mode information may be added to a history memory that stores template-based intra-frame mode information for a set of blocks at a specific granularity (e.g., CTU, slice, tile).

[0144] The historical memory can then be used as a potential prediction candidate for one or more future blocks to be reconstructed.

[0145] In some embodiments, at least one block of the template is reconstructed based on template-based intra-frame prediction.

[0146] Figure 17 An example of a DIMD-based prediction block is illustrated, which is a template of the current block according to at least one exemplary embodiment of the present disclosure.

[0147] The illustration shows two DIMD-based prediction blocks, B1 and B2, where blocks B1 and B2 are adjacent neighbors according to the decoding order. A template comprising five shadow blocks is defined around block B1. At least one reconstructed sample from this shadow block is used to derive template-based intra-prediction information for block B1. The template for block B2 may include three shadow blocks adjacent to both block B2 and block B1. The template-based intra-prediction information for block B2 is then derived during the reconstruction of block B1, i.e., once at least one sample from block B1 (required to derive template-based intra-prediction information for block B2) has been reconstructed.

[0148] In some embodiments, first template-based intra-prediction mode information may be derived from at least one reconstructed sample of a third block of a template defined around a first block for use in the first block; and second template-based intra-prediction mode information may be derived from at least one reconstructed sample of a third block of a template defined around a second block for use in the second block.

[0149] Figure 18 An example of deriving multiple template-based intra-frame prediction information according to at least one exemplary embodiment of the present disclosure is illustrated.

[0150] Blocks B1 and B2 are reconstructed based on template-based intra-prediction mode information. The template-based intra-prediction mode information for block B1 is derived from reconstructed samples of block B3, which belong to the template of block B1. For example, reconstructed samples S31 and S33 of block B3, located to the right of block B3, are used. The template-based intra-prediction information for block B2 is derived from reconstructed samples of block B3, which belong to the template of block B2. For example, reconstructed samples S32 and S33 of block B3, located at the bottom of block B3, are used. When reconstructing block B3, the template-based intra-prediction information for block B1 is derived and stored when the requested samples to the right of block B3 are reconstructed. When reconstructing block B3, the template-based intra-prediction information for block B2 is derived and stored when the requested samples to the bottom of block B3 are reconstructed.

[0151] In some embodiments, at least one reconstructed sample of the template can be used to derive more than one template-based intra-prediction mode information.

[0152] For example, such as Figure 18 As shown in the figure, the reconstructed sample S33 of block B3 is used to obtain template-based intra-prediction mode information of block B1 and template-based intra-prediction mode information of block B2.

[0153] In some embodiments, template-based intra-frame prediction mode information is derived from all reconstructed samples based on a template defined around the current block.

[0154] In some embodiments, template-based intra-prediction mode information can be derived from reconstructed samples of at least one sub-block of the template, and the template-based intra-prediction mode information can be stored at the sub-block level. The sub-block size is smaller than the neighboring block size.

[0155] For example, an M x N block of a template (or the entire template) can be divided into multiple sub-blocks, such as m×n sub-blocks where m <= M and n <= N, where template-based intra-prediction mode information is derived for each sub-block within the M×N block. The derived information is then stored in the memory of the M×N block for use in each m×n sub-block.

[0156] For example, a template can be divided into 8x8 or 4x4 sub-blocks, and HoG can be calculated for each 8x8 or 4x4 sub-block of the template. HoG can be stored at the sub-block level, that is, storing each HoG calculated based on the samples of the sub-block, with each HoG associated with that sub-block.

[0157] In some embodiments, the size of the sub-block can be fixed, or the size of the sub-block can be determined based on the template size or the current block size.

[0158] Derivation of template-based intra-prediction mode information at the sub-block level improves the quality of template-based intra-prediction while maintaining low latency and memory access requirements.

[0159] Figure 19 An example of deriving template-based intra-frame prediction mode information at the sub-block level according to at least one exemplary embodiment of the present disclosure is shown.

[0160] During the reconstruction of blocks B1, B2, and B3, template-based intra-prediction mode information is required for each block. The template-based intra-prediction mode information for each block B1, B2, and B3 is derived based on at least one reconstructed sample located at the bottom of block B4, where at least one reconstructed sample at the bottom of block B4 belongs to the template of blocks B1, B2, and B3. If a single template-based intra-prediction mode information is derived for blocks B1, B2, and B3, that single template-based intra-prediction mode information will not be optimal because the correlation between samples from a block (e.g., block B1) and samples in the template of block B4 (samples closer to block B1) is higher than the correlation with samples in the templates of other blocks B2 and B3.

[0161] exist Figure 19In this model, the bottom of block B4 is divided into 10 sub-blocks SB0 to SB9. Template-based intra-prediction mode information Ti (i = 0 to 9) is derived based on at least one reconstructed sample from each of the sub-blocks SB0 to SB9. For example, template-based intra-prediction mode information for B1 is derived based on template-based intra-prediction mode information T0 to T4 (i.e., template-based intra-prediction information derived for sub-blocks SB0 to SB4), template-based intra-prediction mode information for B2 is derived based on template-based intra-prediction mode information T4 to T7 (i.e., template-based intra-prediction information derived for sub-blocks SB4 to SB7), and template-based intra-prediction mode information for B3 is derived based on template-based intra-prediction mode information T7 to T9 (i.e., template-based intra-prediction information derived for sub-blocks SB7 to SB9).

[0162] Derived template-based intra-prediction information from the reconstructed samples of a block may require access to samples outside the block boundaries. However, these samples outside the block boundaries may not have been reconstructed yet, because the template-based intra-prediction information is derived simultaneously with the block reconstruction.

[0163] In some embodiments, template-based intra-frame prediction information of a block can exclude samples outside the block boundary.

[0164] This embodiment can affect the accuracy of template-based intra-frame prediction of blocks, especially for blocks with small sizes, because there are relatively many boundary samples compared to the total number of samples in the block.

[0165] In some embodiments, template-based intra-frame prediction information for a block can be derived based on reconstructed samples outside the block boundary.

[0166] This implementation may suffer from latency issues because the template-based intra-frame prediction information for the block has to wait for the reconstruction of samples outside the boundary.

[0167] In some embodiments, samples outside the block boundary may be filled based on samples that have already been reconstructed within the block.

[0168] For example, replication padding can be applied from samples within a block.

[0169] Figure 20 An example of a 3x3 Sobel filter according to at least one exemplary embodiment of the present disclosure is shown, which is applied to reconstructed samples of a block of a template to derive template-based intra-prediction mode information.

[0170] In this example, a 3x3 Sobel filter must be applied to a reconstructed sample of a block with a template-based intra-prediction mode, the reconstructed sample being located at the right boundary of the template. The three samples of the template can be padded to apply the Sobel filter to that sample.

[0171] In some embodiments, template-based intra-frame prediction mode information can only be derived when all samples of the template have been reconstructed.

[0172] This implementation is advantageous because it allows the use of reconstructed samples from templates without checking whether the requested sample in the block has been reconstructed.

[0173] In some embodiments, template-based intra-prediction mode information can be derived once at least one sample within a reference block is reconstructed. The reference block refers to the block in which the template for template-based intra-prediction modes (e.g., DIMD / TIMD) is located.

[0174] Figure 21 The illustration shows an example of obtaining template-based intra-frame prediction mode information once at least one required sample of the template is reconstructed, according to at least one exemplary embodiment of the present disclosure.

[0175] Sample S1 is a reconstructed sample from the template, while sample S2 of the template has not yet been reconstructed. The derivation is permitted to begin once sample S1 is reconstructed, but before sample S2 of the template is reconstructed. This allows the reconstruction of block samples and the template-based intra-frame mode derivation process to proceed in parallel. Once some or all of sample S2 has been reconstructed, sample S2 will also be used in the derivation of template-based intra-frame mode information.

[0176] Compared to the case where all samples in a block are reconstructed, this embodiment further reduces the latency of deriving template-based intra-prediction mode information because the reconstruction of block samples and the derivation of template-based intra-prediction mode information are performed jointly. Furthermore, memory accesses for deriving template-based intra-prediction mode information are also significantly reduced because reconstructed samples do not need to be retrieved from memory for this purpose.

[0177] In some embodiments, template-based intra-prediction mode information is derived when reconstructing at least one sample of the template only if the decoder knows that template-based intra-prediction mode information derived from at least one reconstructed sample of the template can be used to predict at least one other block to be reconstructed.

[0178] In some embodiments, template-based intra-frame prediction mode information is derived when reconstructing each block of a video frame.

[0179] Then, each time a block of video frame is reconstructed, template-based intra-frame prediction information for at least one future block to be reconstructed must be derived. This can be the default behavior of the decoder.

[0180] In some embodiments, template-based intra-prediction mode information (e.g., flags or indexes) may be signaled in the bitstream to indicate that the template-based intra-prediction mode information is derived based on the reconstructed samples of the block when the block is reconstructed.

[0181] In some embodiments, template-based intra-prediction mode derivation information is carried by at least one syntax element associated with the prediction mode of a set of CUs (blocks).

[0182] In some embodiments, template-based intra-predictive mode information is derived when encoding each block of a video frame. This can be the default behavior of the encoder. Therefore, the template-based intra-predictive mode information is calculated and stored regardless of whether it will be used in a future block.

[0183] In some embodiments, on the encoder side, template-based intra-prediction mode information is derived according to existing techniques, wherein if the coding mode of a block is a template-based intra-mode, template-based intra-mode information is derived for that block. This is accomplished by obtaining template samples from the neighborhood and calculating the template-based intra-mode information.

[0184] The encoder and decoder generate the same prediction results in the two scenarios described above, but the decoder has lower latency and memory access requirements than the encoder.

[0185] In some CTU boundaries, blocks may not know the prediction information for the next CTU block during the sample reconstruction stage.

[0186] In some embodiments, when reconstructing samples of a block located at the boundary between the current CTU and the next CTU, or during reconstruction, template-based intra-frame prediction mode information is forced to be derived when reconstructing the block of the next CTU located at that boundary.

[0187] Figure 22 An example of reconstructing the current block located at the boundary of the current CTU according to at least one exemplary embodiment of the present disclosure is illustrated.

[0188] When reconstructing the current block B1, which is located to the left of the boundary B separating the current CTU from the next CTU, the decoder is unaware that the next CTU, B2, located to the right of boundary B, requires template-based intra-prediction mode information when reconstructing the next CTU's block B2. Therefore, the decoder can force template-based intra-prediction information when reconstructing the current CTU's block B1.

[0189] In some embodiments, template-based intra-prediction mode information is not derived when reconstructing blocks located at the boundaries of the CTU.

[0190] Then, if the block requires prediction based on template-based intra-prediction mode information, the template-based intra-prediction mode information is derived when reconstructing the block located at the boundary of the CTU.

[0191] It can complete the signal transmission at a certain granularity by obtaining information from the template-based intra-frame prediction mode for blocks.

[0192] In some embodiments, template-based intra-frame prediction mode information may be signaled to a sequence parameter set, image parameter set, or image header, or signaled at the slice, tile, CTU, CU, PU, ​​or TU level.

[0193] Figure 23 An example of transmitting template-based intra-frame prediction mode derived information signaling in a sequence parameter set (SPS) according to at least one exemplary embodiment of the present disclosure is illustrated.

[0194] For example, the flag sps_proposed_enabled_flag can be added to the well-known SPS to indicate that if DIMD prediction mode is enabled, DIMD information can be derived when reconstructing blocks of a video (sequence).

[0195] In some embodiments, the resulting template-based intra-mode derivation information can be used for template-based intra-prediction modes in different channels. For example, if the chroma block is encoded and decoded in DIMD chroma mode, it requires deriving the HoG from both chroma and co-bit luma samples (an example is shown in...). Figure 9 (In the middle). To reduce latency and memory access, this information can be calculated and stored while the corresponding luma sample is being reconstructed, rather than acquiring co-bit luma samples for HoG calculations. When reconstructing a luma block, if the decoder knows that some samples of the luma block will be used for the template-based intra-frame mode of the chroma block, then the corresponding information has already been derived and stored in the luma block CU information.

[0196] Figure 24 An example of template-based intra-frame prediction mode derived information signal transmission in a sequence parameter set (SPS) according to at least one exemplary embodiment of the present disclosure is illustrated.

[0197] For example, the flag sps_proposed_enabled_flag can be added to the well-known SPS to indicate that if TIMD prediction mode is enabled, TIMD information can be derived when reconstructing blocks of a video (sequence).

[0198] In some embodiments, for blocks with a size less than a threshold, template-based prediction mode information is not enabled when reconstructing blocks of a video frame.

[0199] Typically, the threshold is 4 (4x4 blocks) because most samples are located on the block boundaries, so they need to be either padded or excluded.

[0200] This implementation is advantageous because it avoids performance loss in template-based prediction pattern information derived from these samples.

[0201] In some embodiments, for small blocks of size less than a threshold, the neighboring blocks of the small block that should have been obtained during the reconstruction of the small block may be examined for threshold conditions and it may be determined to obtain template-based intra-prediction mode information during the reconstruction of the neighboring block, as in the prior art.

[0202] Figure 25 A schematic block diagram illustrating an example of a system 600 in which various aspects and exemplary embodiments are implemented is shown.

[0203] System 600 can be embedded as one or more devices, including the various components described below. In various exemplary embodiments, system 600 can be configured to implement one or more aspects described in this application.

[0204] Examples of equipment that may constitute all or part of System 600 include personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, connected vehicles and their associated processing systems, head-mounted display devices (HMDs, see-through glasses), projectors, "caves" (systems including multiple displays), servers, video encoders, video decoders, post-processors that process the output from the video decoder, pre-processors that provide input to the video encoder, web servers, video servers (e.g., broadcast servers, video-on-demand servers, or web servers), still or video cameras, encoding or decoding chips, or any other communication devices. The elements of System 600 may be implemented individually or in combination in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one exemplary embodiment, the processing and encoder / decoder elements of System 600 may be distributed across multiple ICs and / or discrete components. In various exemplary embodiments, System 600 may be communicatively connected to other similar systems or other electronic devices via, for example, a communication bus or through dedicated input and / or output ports.

[0205] System 600 may include at least one processor 610 configured to execute instructions loaded therein for implementing various aspects, such as those described in this application. Processor 610 may include embedded memory, input / output interfaces, and various other circuitry known in the art. System 600 may include at least one memory 620 (e.g., a volatile memory device and / or a non-volatile memory device). System 600 may include a storage device 640, which may include non-volatile memory and / or volatile memory, including but not limited to electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, disk drives, and / or optical disk drives. As a non-limiting example, storage device 640 may include internal storage devices, attached storage devices, and / or network-accessible storage devices.

[0206] System 600 may include an encoder / decoder module 630 configured, for example, to process data to provide encoded / decoded video image data, and the encoder / decoder module 630 may include its own processor and memory. The encoder / decoder module 630 may represent one or more modules that can be included in a device to perform encoding and / or decoding functions. As is known, a device may include one or both of the encoding and decoding modules. Furthermore, the encoder / decoder module 630 may be implemented as a separate element of system 600, or may be incorporated into processor 610 as a combination of hardware and software known to those skilled in the art.

[0207] Program code to be loaded into processor 610 or encoder / decoder 630 to execute the various aspects described in this application may be stored in storage device 640 and subsequently loaded into memory 620 for execution by processor 610. According to various exemplary embodiments, during the execution of the processes described in this application, one or more of processor 610, memory 620, storage device 640, and encoder / decoder module 630 may store one or more of various items. Such stored items may include, but are not limited to, video frame data, information data for encoding video frame data, bitstreams, matrices, variables, and intermediate or final results of equations, formulas, operations, and arithmetic logic processing.

[0208] In several exemplary embodiments, the memory within the processor 610 and / or encoder / decoder module 630 may be used to store instructions and provide working memory for processes that can be performed during encoding or decoding.

[0209] However, in other exemplary embodiments, external memory (e.g., the processing device may be processor 610 or encoder / decoder module 630) is used for one or more of these functions. External memory may be memory 620 and / or storage device 640, such as dynamic volatile memory and / or non-volatile flash memory. In several exemplary embodiments, external non-volatile flash memory is used to store the television's operating system. In at least one exemplary embodiment, fast external dynamic volatile memory, such as RAM, may be used as working memory for video encoding / decoding operations, for example, for MPEG-2 Part 2 (also known as ITU-T Recommendation H.262 and ISO / IEC 13818-2, also known as MPEG-2 video), AVC, HEVC, EVC, VVC, AVI, etc.

[0210] As indicated in box 690, input to the components of system 600 can be provided through various input devices. Such input devices include, but are not limited to, (i) an RF section capable of receiving, for example, RF signals transmitted over the air by a broadcasting device, (ii) a composite input terminal, (iii) a USB input terminal, (iv) an HDMI input terminal, and (v) a bus, such as CAN (Controller Area Network), CAN FD (Controller Area Network Flexible Data Rate), FlexRay (ISO 17458), or Ethernet (ISO / IEC 802-3) bus, when the invention is implemented in the automotive field.

[0211] In various exemplary embodiments, the input device of block 690 has associated corresponding input processing elements, as known in the art. For example, the RF section may be associated with elements necessary for: (i) selecting a desired frequency (also known as selecting a signal, or limiting the signal band to a band), (ii) down-converting the selected signal, (iii) further limiting the band to a narrower band to select, for example, the signal band that may be referred to as a channel in some exemplary embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select the desired data packet stream. The RF section of various exemplary embodiments may include one or more elements performing these functions, such as a frequency selector, signal selector, band limiter, channel selector, filter, downconverter, demodulator, error corrector, and demultiplexer. The RF section may include tuners performing various functions among these functions, including, for example, down-converting a received signal to a lower frequency (e.g., intermediate frequency or near-baseband frequency) or baseband.

[0212] In one set-top box embodiment, the RF section and its associated input processing elements can receive RF signals transmitted over a wired (e.g., cable) medium. The RF section can then perform frequency selection by filtering, down-converting, and re-filtering to the desired frequency band.

[0213] Various exemplary embodiments may rearrange the order of the above (and other) elements, remove some of these elements, and / or add other elements that perform similar or different functions.

[0214] Adding components may include inserting components between existing components, such as, for example, inserting amplifiers and analog-to-digital converters. In various exemplary embodiments, the RF portion may include an antenna.

[0215] Furthermore, the USB and / or HDMI terminals may include corresponding interface processors for connecting system 600 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Reed-Solomon error correction) may be implemented, for example, within a separate input processing IC or within processor 610, as needed. Similarly, various aspects of USB or HDMI interface processing may be implemented within a separate interface IC or within processor 610, as needed. The demodulated, error-corrected, and demultiplexed streams may be provided to various processing elements, including, for example, processor 610 and encoder / decoder 630, which operate in conjunction with memory and storage elements to process the data streams for presentation on an output device as needed.

[0216] Various components of system 600 can be provided within an integrated housing. Within the integrated housing, suitable connection arrangements 690, such as internal buses (including I2C buses), wiring, and printed circuit boards known in the art, can be used to interconnect various components and transfer data between them.

[0217] System 600 may include a communication interface 650 that enables communication with other devices via a communication channel 651. The communication interface 650 may include, but is not limited to, a transceiver configured to send and receive data on the communication channel 651. The communication interface 650 may include, but is not limited to, a modem or network interface card (NIC), and the communication channel 651 may be implemented, for example, within a wired and / or wireless medium.

[0218] In various exemplary embodiments, a Wi-Fi network such as IEEE 802.11 can be used to stream data to system 600. The Wi-Fi signals of these exemplary embodiments can be received via a communication channel 651 and a communication interface 650 suitable for Wi-Fi communication. The communication channel 651 of these exemplary embodiments can typically be connected to an access point or router that provides access to external networks, including the Internet, to allow streaming applications and other over-the-top cloud communications.

[0219] Other exemplary embodiments may use a set-top box to provide streaming data to system 600, the set-top box delivering data via an HDMI connection in input block 690.

[0220] Other exemplary embodiments may use the RF connection of input block 690 to provide streaming data to system 600.

[0221] Streamed data can be used as a means of transmitting signaling information by System 600. This signaling information may include bitstream B and / or information such as the number of pixels in a video frame and / or any encoding / decoding settings.

[0222] It should be recognized that signal transmission can be implemented in a variety of ways. For example, in various exemplary embodiments, one or more syntax elements, flags, etc., can be used to transmit signaling information to the corresponding decoder.

[0223] System 600 can provide output signals to various output devices, including a display 661, a speaker 671, and other peripheral devices 681. In various examples of exemplary embodiments, other peripheral devices 681 may include one or more of a standalone DVR, a disc player, a stereo system, a lighting system, and other devices that provide functionality based on the output of system 600.

[0224] In various exemplary embodiments, control signals may be communicated between system 600 and display 661, speaker 671 or other peripheral devices 681 using signaling such as AV.Link (audio / video link), CEC (consumer electronics control), or other communication protocols that enable device-to-device control with or without user intervention.

[0225] Output devices can be connected to system 600 via dedicated connections through the corresponding interfaces 660, 670 and 680.

[0226] Alternatively, the output device can be connected to the system 600 via communication interface 650 using communication channel 651. The display 661 and speaker 671 can be integrated with other components of the system 600 into a single unit in an electronic device, such as a television set.

[0227] In various exemplary embodiments, the display interface 660 may include a display driver, such as, for example, a timing controller (T Con) chip.

[0228] For example, if the RF portion of input 690 is part of a separate set-top box, then display 661 and speaker 671 may optionally be separate from one or more other components. In various exemplary embodiments where display 661 and speaker 671 can be external components, output signals may be provided via dedicated output connections, including, for example, an HDMI port, a USB port, or a COMP output.

[0229] exist Figure 1-24 This document describes various methods, each comprising one or more steps or actions to implement the described method. Unless the correct operation of the method requires a specific order of steps or actions, the order and / or use of specific steps and / or actions may be modified or combined.

[0230] Examples are described regarding the block diagrams and / or operation flowcharts. Each block represents a portion of circuitry, modules, or code, which includes one or more executable instructions for implementing one or more specified logical functions. It should also be noted that in other implementations, the functions (one or more) marked in the blocks may occur out of order. For example, depending on the functions involved, two blocks shown sequentially may actually execute substantially concurrently, or sometimes these blocks may be executed in reverse order.

[0231] The embodiments and aspects described herein may be implemented in, for example, methods or processes, apparatus, computer programs, data streams, bit streams, or signals. Even if discussed only in the context of a single form of implementation (e.g., discussed only as a method), implementations of the discussed features may be implemented in other forms (e.g., apparatus or computer programs).

[0232] The method can be implemented in, for example, a processor, which generally refers to a processing device, including, for example, a computer, microprocessor, integrated circuit, or programmable logic device. Processors also include communication devices.

[0233] Furthermore, the method can be implemented by instructions executed by a processor, and such instructions (and / or data values ​​generated by the implementation) can be stored on a computer-readable storage medium. The computer-readable storage medium can take the form of a computer-readable program product implemented in one or more computer-readable media and having computer-executable computer-readable program code implemented thereon. Considering the inherent ability to store information therein and the inherent ability to retrieve information provided therefrom, the computer-readable storage medium as used herein can be considered a non-transitory storage medium. The computer-readable storage medium can be, for example, but not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any suitable combination of the foregoing. It should be understood that while more specific examples of computer-readable storage media to which this exemplary embodiment can be applied are provided below, they are merely illustrative and not exhaustive, as will be readily apparent to those skilled in the art: portable computer floppy disks; hard disks; read-only memory (ROM); erasable programmable read-only memory (EPROM or flash memory); portable optical disc read-only memory (CD-ROM); optical storage devices; magnetic storage devices; or any suitable combination of the foregoing.

[0234] Instructions can form applications that are tangibly implemented on processor-readable media.

[0235] For example, instructions can be found in hardware, firmware, software, or a combination thereof. Instructions can be found, for example, in an operating system, a standalone application, or a combination of both. Therefore, a processor can be characterized as, for example, a device configured to execute a process and a device including a processor-readable medium (such as a storage device) having instructions for executing the process. Additionally, in addition to or instead of instructions, the processor-readable medium can store data values ​​generated by the implementation.

[0236] The device can be implemented, for example, in appropriate hardware, software, and firmware. Examples of such devices include personal computers, laptops, smartphones, tablets, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, head-mounted display devices (HMDs, see-through glasses), projectors, "caves" (systems comprising multiple displays), servers, video encoders, video decoders, post-processors that process the output from the video decoder, pre-processors that provide input to the video encoder, web servers, set-top boxes, and any other devices used to process video images, or other communication devices. It should be clear that the equipment can be mobile and even mounted in mobile vehicles.

[0237] The computer software may be implemented by the processor 610 or by hardware, or by a combination of hardware and software. As a non-limiting example, exemplary embodiments may also be implemented by one or more integrated circuits. The memory 620 may be of any type suitable for the technical environment and may be implemented using any suitable data storage technology, such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory, as non-limiting examples. The processor 610 may be of any type suitable for the technical environment and may encompass one or more of microprocessors, general-purpose computers, special-purpose computers, and processors based on multi-core architectures, as non-limiting examples.

[0238] As will be apparent to those skilled in the art, implementations can generate various signals formatted to carry, for example, information that can be stored or transmitted. The information may include, for example, instructions for performing a method or data generated by one of the described implementations. For example, the signal may be formatted to carry a bit stream of the described exemplary embodiments. Such a signal may be formatted as, for example, electromagnetic waves (e.g., using the radio frequency portion of a spectrum) or baseband signals. Formatting may include, for example, encoding the data stream and modulating a carrier wave with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, the signal can be transmitted via various wired or wireless links. The signal may be stored on a processor-readable medium.

[0239] The terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. As used herein, the singular forms “an,” “a,” and “the” may also be intended to include the plural forms unless the context clearly indicates otherwise. It will be further understood that, when used in this specification, the terms “include / comprise” and / or “including / comprising” may specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. Furthermore, when an element is referred to as “in response to,” “connected to,” or “associated with,” another element, it may be directly responsive to, connected to, or associated with another element, or there may be intermediate elements. In contrast, when an element is referred to as “directly responsive to,” “directly connected to,” or “directly associated with,” another element, there are no intermediate elements.

[0240] It should be recognized that, for example, in the cases of “A / B,” “A and / or B,” and “at least one of A and B,” the use of any of the symbols / terms “ / ,” “and / or,” and “at least one of” can be intended to cover the selection of only the first listed option (A), or only the second listed option (B), or the selection of both options (A and B). As a further example, in the cases of “A, B, and / or C” and “at least one of A, B, and C,” such wording is intended to cover the selection of only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or the selection of all three options (A, B, and C). As will be clear to those skilled in the art and related fields, this can be extended to as many items as are listed.

[0241] Various numerical values ​​may be used in this application. Specific values ​​may be used for illustrative purposes and the aspects described are not limited to these specific values.

[0242] It will be understood that while the terms first, second, etc., may be used herein to describe various elements, these elements are not limited by these terms. These terms are used only to distinguish one element from another. For example, without departing from the teachings of this application, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element. There is no implied order between the first element and the second element.

[0243] References to “an exemplary embodiment” or “an exemplary embodiment” or “an implementation” or “implementation” and other variations thereof are frequently used to convey that a particular feature, structure, characteristic, etc. (described in conjunction with an exemplary embodiment / implementation) is included in at least one exemplary embodiment / implementation. Therefore, the phrases “in an exemplary embodiment” or “in an exemplary embodiment” or “in one implementation” or “in one implementation” appearing throughout this application, as well as any other variations, do not necessarily refer to the same exemplary embodiment.

[0244] Similarly, the references to "according to an exemplary embodiment / example / implementation" or "in an exemplary embodiment / example / implementation" and their variations are frequently used to convey that a particular feature, structure, or characteristic (described in conjunction with an exemplary embodiment / example / implementation) may be included in at least one exemplary embodiment / example / implementation. Therefore, the expressions "according to an exemplary embodiment / example / implementation" or "in an exemplary embodiment / example / implementation" appearing throughout this application do not necessarily refer to the same exemplary embodiment / example / implementation, nor are individual or alternative exemplary embodiments / examples / implementations necessarily mutually exclusive with other exemplary embodiments / examples / implementations.

[0245] The reference numerals appearing in the claims are for illustrative purposes only and do not limit the scope of the claims. Although not explicitly described, these exemplary embodiments / examples and variations may be employed in any combination or sub-combination.

[0246] When a diagram is presented as a flowchart, it should be understood that it also provides a block diagram of the corresponding apparatus. Similarly, when a diagram is presented as a block diagram, it should be understood that it also provides a flowchart of the corresponding method / process.

[0247] While some diagrams include arrows along the communication path to indicate the main direction of communication, it should be understood that communication can occur in the opposite direction to the arrows depicted.

[0248] Various implementations involve decoding. As used herein, "decoding" can encompass all or part of a process performed, for example, on a received video frame (which may include a received bitstream encoded with one or more video frames) to produce a final output suitable for display or further processing in a reconstructed video domain. In various exemplary embodiments, such a process includes one or more processes typically performed by a decoder. In various exemplary embodiments, such a process, for example, also includes, or optionally includes, a process performed by a decoder of the various embodiments described herein.

[0249] As a further example, in one exemplary embodiment, "decoding" may refer only to dequantization; in another exemplary embodiment, "decoding" may refer to entropy decoding; in yet another exemplary embodiment, "decoding" may refer only to differential decoding; and in yet another exemplary embodiment, "decoding" may refer to a combination of dequantization, entropy decoding, and differential decoding. It will be clear, and believed to be well understood by those skilled in the art, whether the phrase "decoding process" is intended to specifically refer to a subset of operations or generally to a broader decoding process, depending on the context of the specific description.

[0250] Various implementations involve encoding. In a manner similar to the above discussion of "decoding," the term "encoding" as used herein can encompass all or part of a process performed on an input video frame to generate an output bitstream. In various exemplary embodiments, such a process includes one or more processes typically performed by an encoder. In various exemplary embodiments, such a process also includes, or optionally includes, a process performed by an encoder of the various embodiments described herein.

[0251] As a further example, in one exemplary embodiment, "encoding" may refer only to quantization; in another exemplary embodiment, "encoding" may refer only to entropy encoding; in yet another exemplary embodiment, "encoding" may refer only to differential encoding; and in still another exemplary embodiment, "encoding" may refer to a combination of quantization, differential encoding, and entropy encoding. It will be clear, and believed to be well understood, by those skilled in the art, whether the phrase "encoding process" is intended to specifically refer to a subset of operations or generally to a broader encoding process, depending on the context of the particular description.

[0252] Furthermore, this application may refer to "obtaining" various types of information. Obtaining information may include one or more of the following: estimated information, calculated information, predicted information, or information retrieved from memory, processed information, moved information, copied information, erased information, calculated information, determined information, predicted information, or estimated information.

[0253] Furthermore, this application may refer to "receiving" various messages. Receiving messages may include one or more of the following, such as access information or receiving information from a communication network.

[0254] Moreover, as used herein, the word "signal" specifically refers to instructing the corresponding decoder to do something. For example, in some exemplary embodiments, the encoder signals specific information, such as encoding / decoding parameters or encoded video frame data. In this way, in exemplary embodiments, the same parameter can be used on both the encoder and decoder sides. Thus, for example, the encoder can transmit (explicitly signal) a specific parameter to the decoder so that the decoder can use the same specific parameter. Conversely, if the decoder already has the specific parameter as well as other parameters, then signaling can be used without transmission (implicitly signal) to simply allow the decoder to know and select the specific parameter. Bit savings are achieved in various exemplary embodiments by avoiding the transmission of any actual functionality. It should be recognized that signaling can be accomplished in a variety of ways. For example, in various exemplary embodiments, one or more syntax elements, flags, etc., are used to signal information to the corresponding decoder. Although the verb form of the word "signal" has been referred to above, the word "signal" can also be used as a noun herein.

[0255] Several implementations have been described. However, it should be understood that various modifications can be made. For example, elements of different implementations can be combined, supplemented, modified, or removed to produce other implementations. Furthermore, those skilled in the art will understand that other structures and processes can replace the disclosed structures and processes, and the resulting implementations will perform at least substantially the same functions in at least substantially the same manner to achieve at least substantially the same results as the disclosed implementations. Therefore, these and other implementations are contemplated in this application.

Claims

1. A method for determining a block of a video frame based on template-based intra-prediction mode information, wherein the template-based intra-prediction mode information is derived from at least one reconstructed sample of a template, the template including at least one reconstructed sample of the current block adjacent to the template-based intra-prediction block, wherein, Template-based intra-frame prediction mode information is derived at least in part when reconstructing at least one sample of the current block.

2. The method according to claim 1, wherein, Template-based intra-prediction mode information is stored at the block level.

3. The method according to claim 1, wherein, Template-based intra-prediction mode information is derived from reconstructed samples of at least one sub-block of the template, and the template-based intra-prediction mode information is stored at the sub-block level, where the sub-block size is smaller than the neighboring block size.

4. The method according to claim 3, wherein, The size of the sub-block is fixed, or the size of the sub-block is determined based on the template size or the current block size.

5. The method according to any one of claims 1 to 4, wherein, At least one reconstructed sample of the template is used to derive more than one template-based intra-prediction mode information.

6. The method according to any one of claims 1 to 5, wherein, Template-based intra-frame prediction mode information is derived from all reconstructed samples of the template.

7. The method according to any one of claims 1 to 6, wherein, The template-based intra-frame prediction information excludes samples outside the boundaries of the current block, or The template-based intra-frame prediction information is derived based on reconstructed samples outside the boundary of the current block, or The template-based intra-frame prediction information is derived based on the padding samples outside the current block boundary, which are based on the already reconstructed samples inside the current block.

8. The method according to any one of claims 1 to 7, wherein, Template-based intra-frame prediction mode information is only obtained when all samples of the template are reconstructed.

9. The method according to any one of claims 1 to 7, wherein, Template-based intra-prediction mode information is obtained once at least one sample in a reference block is reconstructed, the reference block being the block in which the template for the template-based intra-prediction mode is located.

10. The method according to any one of claims 1 to 9, wherein, Template-based intra-frame prediction mode information is derived when reconstructing each block of the video frame.

11. The method according to any one of claims 1 to 10, wherein, A flag or index signal is transmitted into the bitstream to indicate that template-based intra-frame prediction mode information is derived from the reconstructed samples of the block during block reconstruction.

12. The method according to any one of claims 1 to 11, wherein, The video frame is divided into multiple adjacent codec tree units. When reconstructing samples of blocks located at the boundary between the current codec tree unit and the next codec tree unit, template-based intra-frame prediction mode information is forced to be derived when reconstructing blocks located at the boundary of the next codec tree unit.

13. The method according to any one of claims 1 to 12, wherein, The video frame is divided into multiple adjacent codec tree units. When reconstructing the blocks located at the boundaries of the codec tree units, template-based intra-prediction mode information must not be included.

14. The method according to any one of claims 12 to 13, wherein the flag or index is signaled to a sequence parameter set, a picture parameter set, or a picture header, or the flag or index is signaled at the level of a slice, tile, codec tree unit, block, prediction unit, or transform unit.

15. The method according to any one of claims 1 to 14, wherein, For blocks smaller than a threshold, this feature is disabled: template-based prediction mode information is derived when reconstructing blocks of a video frame.

16. An apparatus comprising components for performing one of the methods according to any one of claims 1 to 15.

17. A computer program product comprising instructions that, when executed by one or more processors, cause the one or more processors to perform the method according to any one of claims 1 to 15.

18. A non-transient storage medium carrying instructions for performing program code according to any one of claims 1 to 15.

Images