Encoding / decoding video picture data

Abstract

The present disclosure relates to a TIMD merge mode including at least one intra mode derivation candidate, i.e. at least one DIMD and / or TIMD mode, and associated information for intra prediction of a current block of a video picture VP. The at least one DIMD and / or TIMD mode candidate is collected from at least one previously coded (decoded) block predicted using a TIMD mode, a DIMD mode, a TIMD merge mode or a DIMD merge mode. A merge list of TIMD merge mode candidates can be obtained (1510). Each TIMD merge mode candidate can contain an intra prediction mode of at least one DIMD or TIMD mode and its associated information. Each TIMD merge mode candidate is used for intra prediction. A TIMD merge mode for intra prediction of the current block is determined (1520) by selecting one TIMD merge mode candidate from the merge list based on a prediction cost obtained based on the TIMD merge mode candidates in the merge list.

Description

Cross-references to related applications

[0001] This application is based on and claims priority to European Patent Application No. 23307055.6, filed on 24 November 2023, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application generally relates to video frame encoding and decoding. In particular, but not limited thereto, the technical field of this application relates to intra-frame prediction of video frame blocks. Background Technology

[0003] This section is intended to introduce the reader to various aspects of the art that may relate to aspects of at least one exemplary embodiment of the present application described below and / or claimed. This discussion is intended to help provide the reader with background information to facilitate a better understanding of the various aspects of the present application. Therefore, these statements should be understood to be read in this light and not as an admission of prior art.

[0004] At least one exemplary embodiment of this application was designed based on the above considerations.

[0005] ECM (Algorithm description of Enhanced Compression Model 10 (ECM 10), JVET-AE2025, July 11-19, 2023, Geneva) defines the decoder-side intra-mode derivation (DIMD) mode and template-based intra-mode derivation (TIMD) mode for determining the predicted samples of both the luma and chroma samples of the current block.

[0006] The TIMD pattern is derived based on template samples that may contain noise, such as quantization noise (which is typically higher in block boundary samples). Consequently, the derived TIMD pattern may not be optimal for predicting the current block of a video frame VP.

[0007] One of the challenges is to improve the derivation of TIMD patterns to predict samples of the current block of a video frame VP. Summary of the Invention

[0008] The following sections provide a brief overview of at least one exemplary embodiment to provide a basic understanding of some aspects of this application. This overview is not an exhaustive summary of the exemplary embodiments. Its purpose is not to identify key or core elements of the exemplary embodiments. The following overview only gives some aspects of at least one exemplary embodiment in a simplified form, serving as a prelude to a more detailed description provided elsewhere in this document.

[0009] According to a first aspect of this application, a method for determining an intra-frame prediction mode is provided, the intra-frame prediction mode being denoted as TIMD combining mode, the method comprising: - Obtain a merging list of at least one TIMD merging mode candidate, each TIMD merging mode candidate including at least one intra-prediction mode and associated information for intra-prediction; and - Determine the TIMD merge mode as a candidate for intra-frame prediction of the current block from the merge list of TIMD merge mode candidates.

[0010] In some embodiments, the prediction cost of the TIMD merge pattern is lower than that of other TIMD merge pattern candidates in the merge list.

[0011] In some embodiments, the TIMD merge mode is determined by the following: - Obtain the prediction cost on the template for each TIMD merge pattern candidate pattern in the merge list; and - Select the TIMD merge mode based on the predicted cost obtained for the TIMD merge mode candidates.

[0012] In some embodiments, the selection of TIMD merging is based on rate-distortion optimization, in which the rate-distortion cost of the intra-prediction for the current block is evaluated based on the obtained prediction cost, and each intra-prediction is derived based on a TIMD merging mode candidate in the merging list.

[0013] In some embodiments, the index of the selected TIMD merge mode candidate is signaled in the bitstream.

[0014] In some embodiments, the method further includes encoding information in the bitstream indicating whether a TIMD merging mode is enabled as a candidate for intra-frame prediction of the current block of a video frame.

[0015] In some embodiments, at least one TIMD merge mode candidate in the merge list has previously been used to perform intra-frame prediction of adjacent and / or non-adjacent spatial neighbor blocks of the current block.

[0016] In some embodiments, at least one TIMD mode candidate in the merge list has previously been used for intra-frame prediction of blocks in a reference frame.

[0017] In some embodiments, a block in the reference frame is a co-located block of the current block or an extended block of a co-located block in the reference frame, or a block in the reference frame is a block indicated by motion vectors from adjacent and / or non-adjacent spatial neighboring blocks of the current block.

[0018] In some embodiments, at least one TIMD merge mode candidate in the merge list includes at least one template-based prediction mode and associated information for intra-frame prediction of previously encoded or decoded blocks.

[0019] In some embodiments, at least one TIMD merge mode candidate in the merge list includes at least one decoder-side intra-mode derivation mode (DIMD) and associated information for intra-frame prediction of previously encoded or decoded blocks, and / or at least one TIMD merge mode candidate in the merge list includes at least one TIMD merge mode and associated information for intra-frame prediction of previously encoded or decoded blocks.

[0020] In some embodiments, the index of the selected TIMD merging mode is signaled in the bitstream depending on the availability of at least one block previously encoded or decoded based on the following modes: template-based prediction mode, decoder-side intra-frame mode derivation mode, or TIMD merging mode.

[0021] In some embodiments, a merge list is obtained for each channel of the video frame, and the TIMD merge mode candidates in the merge list obtained for the first channel belong to the merge list obtained for the second channel of the video frame.

[0022] In some embodiments, the TIMD merging mode determined as a candidate for intra-frame prediction of the current block of the first channel of the video frame is inherited by the co-bit block in the second channel of the video frame.

[0023] In some embodiments, the information associated with TIMD merge mode candidates is at least one of the following: - TIMD mode is preferred; - TIMD fusion mode for mixing with preferred TIMD modes; - The fusion weight of preferred mode and fusion mode; - Template cost for hybrid flag indicators, preferred mode, and / or fusion mode; - Transformation type; - Wide-angle conditions for preferred mode and fusion mode.

[0024] According to a second aspect of this application, a method for encoding video frame data is provided, including the method described in the first aspect.

[0025] According to a third aspect of this application, a method for decoding video image data is provided, including the method described in the first aspect.

[0026] According to four aspects of this application, an apparatus is provided, including components for performing one of the methods described in the first, second and / or third aspects of this application.

[0027] According to a fifth aspect of this application, a computer program product including instructions is provided, which, when executed by one or more processors, cause the one or more processors to perform the methods according to the first, second, and / or third aspects of this application.

[0028] According to a sixth aspect of this application, a non-transitory storage medium is provided, carrying instructions for performing program code for executing the methods described in the first, second, and / or third aspects of this application.

[0029] The specific properties of at least one of the exemplary embodiments, as well as other objects, advantages, features, and uses of the at least one exemplary embodiment, will become apparent from the following description of the examples taken in conjunction with the accompanying drawings. Attached Figure Description

[0030] Reference will now be made to the accompanying drawings, which illustrate exemplary embodiments of this application, wherein: Figure 1 An example of a codec tree unit based on HEVC is shown; Figure 2 An example of partitioning codec units into prediction units according to HEVC is shown; Figure 3 An example of CTU partitioning based on VVC is shown; Figure 4 An example of the splitting patterns supported in VVC's multi-type tree partitioning is shown; Figure 5 A schematic block diagram of a method 100 for encoding a video frame VP according to the prior art is shown; Figure 6 A schematic block diagram of a method 200 for decoding video frames VP according to the prior art is shown; Figure 7 An example of a template for calculating a gradient histogram according to the prior art is shown; Figure 8 An example of an intra-frame predictor for calculating the current block of a video frame based on a weighted linear combination of three intra-frame predictors, according to the prior art, is shown. Figure 9 An example of a co-bit luminance block corresponding to a chrominance block is shown according to the prior art, with the selection of PSAO parameters; Figure 10 An example of the definition of a non-adjacent spatial neighbor block according to the prior art is shown; Figure 11An example of a first template for template prediction and an example of a second template for calculating prediction costs are shown; Figure 12 An example of a reference line according to the prior art is shown; Figure 13 An example of vertical and horizontal splitting of a block according to existing technology is shown; Figure 14 The illustration shows an example of a merge list according to some embodiments and an example of information associated with TIMD merge mode candidates in the merge list; Figure 15 A flowchart of a method 1500 for determining a TIMD merging mode as a candidate for intra-frame prediction of the current block of a video frame VP, according to some embodiments, is shown. Figure 16 An example of the syntax of SPS that can be added to the underlying codec according to an embodiment is shown for signaling information indicating whether TIMD merge mode is enabled as a candidate for intra-frame prediction of the current block of a video frame. Figure 17 Examples of syntax for CU syntax that can be added to the underlying codec according to some embodiments are shown for signaling information indicating whether TIMD merge mode is enabled as a candidate for intra-frame prediction of the current block of a video frame. Figure 18 An example of adjacent spatial neighboring blocks of the current block is shown according to some embodiments; Figure 19 An example of a motion vector inherited from a neighboring spatial block of the current block, according to some embodiments, is shown; Figure 20 Examples of different block partitions of chroma blocks and co-bit luminance blocks according to some embodiments are shown; Figure 21 A block diagram illustrating an example of a system in which various aspects and exemplary embodiments are implemented.

[0031] Similar or identical elements are referenced using the same figure labels. Detailed Implementation

[0032] At least one of the exemplary embodiments will be described more fully below with reference to the accompanying drawings, which depict examples of at least one of the exemplary embodiments. However, the exemplary embodiments may be implemented in various alternative forms and should not be construed as limited to the examples set forth herein. Accordingly, it should be understood that the present invention is not intended to limit the exemplary embodiments to the specific forms disclosed. Rather, this application is intended to cover all modifications, equivalents, and alternatives falling within the spirit and scope of this application.

[0033] At least one of these aspects generally relates to video image encoding and decoding, another generally relates to the transmission of a bit stream provided or encoded, and yet another relates to the reception / access of a decoded bit stream.

[0034] At least one of the exemplary embodiments is described in relation to encoding / decoding a single video frame, but is extended to encoding / decoding multiple video frames (a sequence of frames), since each video frame is encoded / decoded sequentially as described below.

[0035] Moreover, for example, at least one exemplary embodiment is not limited to MPEG standards, such as AVC (ISO / IEC 14496-10 Advanced Video Coding for generic audio-visual services, ITU-T Recommendation H.264, https: / / www.itu.int / rec / T-REC-H.264-202108-P / en), EVC (ISO / IEC 23094-1 Essential video coding), HEVC (ISO / IEC 23008-2 High Efficiency Video Coding, ITU-T Recommendation H.265, https: / / www.itu.int / rec / T-REC-H.265-202108-P / en), VVC (ISO / IEC 23090-3 Versatile Video Coding, ITU-T...). Recommendation H.266 (https: / / www.itu.int / rec / T-REC-H.266-202008-I / en) can be applied to other standards and recommendations, such as AV1 (AOMedia Video 1, http: / / aomedia.org / av1 / specification / ). At least one exemplary embodiment can be adapted to existing or future developments and extensions of any such standard and recommendation. Unless otherwise stated or technically excluded, the aspects described in this application may be used individually or in combination.

[0036] A pixel corresponds to the smallest display unit on the screen, and it can consist of one or more light sources (one for a monochrome screen and three or more for a color screen).

[0037] A video frame, also known as a frame or video frame, includes at least one component (also known as a frame component or channel) defined by a specific video / video format that specifies all information related to pixel values ​​and all information that can be used by a display unit and / or any other device to display and / or decode video frame data associated with the video frame.

[0038] The video frame includes at least one component, which is typically represented as an array of samples.

[0039] Monochrome video footage consists of a single component, while color video footage can consist of three components.

[0040] For example, when the picture / video format is the well-known (Y,Cb,Cr) format, a color video picture can include one luminance (or brightness) component and two chrominance components, while when the picture / video format is the well-known (R,G,B) format, a color video picture can include three color components (one for red, one for green, and one for blue).

[0041] Each component of a video frame can include a number of samples relative to the number of pixels on the screen on which the video frame is to be displayed. In variations, the number of samples included in a component can be a multiple (or fraction) of the number of samples included in another component of the same video frame.

[0042] For example, in a video format that includes one luminance component and two chrominance components (such as the (Y,Cb,Cr) format), depending on the color format under consideration, the chrominance component may contain half the number of samples in width and / or height relative to the luminance component.

[0043] A sample is the smallest unit of visual information that makes up a video frame. Sample values ​​can be, for example, luminance or chrominance values, or color values ​​in (R,G,B) format.

[0044] A pixel value is the value of a pixel on the screen. For monochrome video, a pixel value can be represented by a single sample, while for color video, a pixel value can be represented by multiple co-localized samples. A co-localized sample associated with a pixel refers to the sample corresponding to the pixel's position on the screen.

[0045] Video frames are typically viewed as a set of pixel values, with each pixel represented by at least one sample.

[0046] A video frame block is a set of samples of one component of a video frame. When the frame / video format is a well-known (Y,Cb,Cr) format, a block of at least one luminance sample or at least one chrominance sample can be considered; or when the frame / video format is a well-known (R,G,B) format, a block of at least one color sample can be considered.

[0047] At least one exemplary embodiment is not limited to a specific image / video format.

[0048] In state-of-the-art video compression systems, such as HEVC or VVC, low-level and high-level picture partitioning is provided to divide the video frame into picture blocks, called codec tree units (CTUs). The size of these units is typically between 16x16 and 64x64 pixels for HEVC, and between 32x32, 64x64, or 128x128 pixels for VVC.

[0049] The CTU (Central Unit) division of a video frame forms a grid composed of CTUs of fixed size; that is, a CTU grid whose top and left edges spatially coincide with the top and left edges of the video frame. The CTU grid represents the spatial partitioning of the video frame.

[0050] In VVC and HEVC, the CTU size (CTU width and CTU height) of all CTUs in the CTU mesh is equal to the same default CTU size (default CTU width CTU DW and default CTU height CTU DH). For example, the default CTU size (default CTU height, default CTU width) can be equal to 128 (CTU DW = CTU DH = 128). The default CTU size (height, width) is encoded into the bitstream, for example, at the sequence level in the Sequence Parameter Set (SPS).

[0051] The spatial position of a CTU within the CTU mesh is determined by its address ctuAddr, which defines the spatial position of the CTU's top-left corner relative to the origin. For example... Figure 1 As shown, the CTU address can define a spatial location starting from the top left corner of the higher-level spatial structure S containing the CTU.

[0052] Each CTU is associated with a codec tree to determine the tree partitioning of the CTU.

[0053] like Figure 1 As shown, in HEVC, the codec tree is a quadtree partition of CTUs, where each node is called a codec unit (CU). The spatial location of a CU in a video frame is defined by the CU index cuIdx, which indicates the spatial location starting from the top left corner of the CTU. CUs are spatially partitioned into one or more prediction units (PUs). The spatial location of a PU in a video frame (VP) is defined by the PU index puIdx, which defines the spatial location starting from the top left corner of the CTU, and the spatial location of the elements of a partitioned PU is defined by the PU partition index puPartIdx, which defines the spatial location starting from the top left corner of the PU. Each PU is assigned some intra-frame or inter-frame prediction data.

[0054] The intra-frame or inter-frame encoding / decoding mode is assigned at the CU level. This means that although the prediction parameters vary from PU to CU, each PU is assigned the same intra-frame / inter-frame encoding / decoding mode.

[0055] According to a quadtree called a transform tree, a CU can also be spatially partitioned into one or more transform units (TUs). A transform unit is a leaf of the transform tree. The spatial location of a TU in a video frame is defined by a TU index tuIdx, which defines the spatial location starting from the top-left corner of the CU. Each TU is assigned a number of transform parameters. The transform type is assigned at the TU level, and individual 2D transforms are performed at the TU level during the encoding / decoding or decoding of a frame block.

[0056] Figure 2 The diagram above illustrates the existing PU partition types in HEVC. These include square partitions (2Nx2N and NxN), which are the only partitions used in both intra-frame and inter-frame prediction CUs; ​​symmetric non-square partitions (2NxN, Nx2N, used only in inter-frame prediction CUs); and asymmetric partitions (used only in inter-frame prediction CUs). For example, PU type 2NxnU represents an asymmetric horizontal partition of the PU, where the smaller partition is located at the top of the PU. According to another example, PU type 2NxnL represents an asymmetric horizontal partition of the PU, where the smaller partition is located at the top of the PU.

[0057] like Figure 3 As shown, in VVC, the codec tree starts from the root node (i.e., CTU). Next, a quadtree (or quaternion tree) split divides the root node into four nodes, corresponding to four child blocks of equal size (solid lines). Then, the quadtree (or quaternion tree) leaves can be further partitioned using a so-called multi-type tree, which involves... Figure 4 One of the four splitting patterns shown is used for binary or ternary splits. These splitting types are vertical and horizontal binary splitting patterns (denoted as SBTV and SBTH) and vertical and horizontal ternary splitting patterns SPTTV and STTH.

[0058] In the case of a joint codec tree where luminance and chrominance components share a common codec tree, the leaf of the CTU's codec tree is the CU.

[0059] In contrast to HEVC, in VVC, CU, PU, ​​and TU have the same size in most cases, which means that, except in some specific codec modes, codec units are generally not partitioned into PU or TU.

[0060] Figure 5 and Figure 6It provides an overview of video encoding / decoding methods used in current video standard compression systems such as HEVC or VVC.

[0061] Figure 5 A schematic block diagram illustrating the steps of a method 100 for encoding a video frame VP according to the prior art is shown.

[0062] In step 110, the video frame VP is partitioned into sample blocks, and the partition information data is sent to the bitstream via a signal. Each block includes a sample of one component of the video frame VP. Therefore, these blocks include samples that define each component of the video frame VP.

[0063] For example, in HEVC, the frame is divided into codec tree units (CTUs). Each CTU can be further subdivided using a quadtree partition, where each leaf of the quadtree represents a codec unit (CU). The partition information data can then include data describing the CTU and the quadtree subdivision of each CTU.

[0064] Therefore, each sample block (or block for short) can be a CU (if the CU includes a single PU) or a PU of the CU.

[0065] Using intra-frame or inter-frame prediction modes, each block is encoded along the coding cycle (also known as "in the cycle").

[0066] Intra-frame prediction (step 120) uses intra-frame prediction data. Intra-frame prediction involves predicting the current block using blocks of intra-frame prediction based on samples that have been encoded, decoded, and reconstructed, located around the current block, typically at the top and left of the current block. Intra-frame prediction is performed in the spatial domain.

[0067] In inter-frame prediction mode, motion estimation (step 130) and motion compensation (135) are performed. Motion estimation searches for candidate reference blocks in one or more reference video frames used for predictively encoding the current video frame as good predictors for the current block. For example, a good predictor for the current block is a predictor similar to the current block. The output of motion estimation step 130 is inter-frame prediction data, which includes motion information associated with the current block (typically one or more motion vectors and one or more reference video frame indices) and other information for obtaining the same prediction block on the encoding / decoding side. Next, motion compensation (step 135) obtains the prediction block using the motion vector(s) and reference video frame(s) determined by motion estimation step 130. Essentially, a block belonging to a selected reference video frame and pointed to by a motion vector can be used as the prediction block for the current block. Furthermore, since motion vectors are expressed as fractions of integer pixel positions (this is referred to as sub-pixel accuracy motion vector representation), motion compensation generally involves spatial interpolation of some reconstructed samples of the reference video frames to compute the prediction block.

[0068] The prediction information data is signaled to the bitstream. The prediction information may include the prediction mode (intra-frame, inter-frame, or skip), intra-frame / inter-frame prediction data, and any other information used to obtain the same prediction CU on the decoding side.

[0069] Taking into account the encoding of the calculated prediction residual block (e.g., by subtracting the candidate prediction block from the current block) and the signaling of the prediction information data required to determine the candidate prediction block on the decoding side, method 100 selects a prediction mode (intra-frame or inter-frame prediction mode) by optimizing the rate-distortion tradeoff.

[0070] Typically, intra-frame and inter-frame prediction modes are set in a "most likely candidate mode list (MPM list)," and the best prediction mode is given by the following formula for the current block. p Given: , in P It is the list of MPMs for the current block. p This indicates the candidate codec modes in the MPM list. RD cost(p) It is a candidate prediction model p The rate-distortion cost is usually expressed as: , D ( p ) is the current block and the candidate prediction pattern p Distortion between reconstructed blocks obtained after encoding / decoding the current blockR ( p ) is related to prediction patterns p The rate cost associated with encoding and decoding the current block, and λ It is a Lagrangian parameter representing the rate constraint for encoding and decoding the current block, and is usually calculated based on the quantization parameter Qp used to encode the current block.

[0071] The current block is typically encoded from the prediction residual block PR. More precisely, for example, the prediction residual block PR is calculated by subtracting the best prediction block from the current block. The prediction residual block PR is then transformed using a transform of type DCT (Discrete Cosine Transform) or DST (Discrete Sine Transform) or any other suitable transform (step 140), and the resulting transformed coefficient block is quantized (step 150).

[0072] In a variant, method 100 can also skip transform step 140 according to a so-called transform-skip encoding / decoding mode and directly apply quantization to the predicted residual block PR (step 150).

[0073] Encode the quantized transform coefficient block (or quantized prediction residual block) into the bit stream (step 160).

[0074] Next, as part of the encoding loop, the quantized transform coefficient block (or quantized residual block) is dequantized (step 170) and inverse transformed (180) (or not) to obtain the decoded prediction residual block. Then, the decoded prediction residual block and the prediction block are combined (usually summed) to provide the reconstructed block.

[0075] In step 160, other information data can also be entropy encoded to encode the current block of the video frame VP.

[0076] A loop filter (step 190) can be applied to the reconstructed frame (including reconstructed blocks) to reduce compression artifacts. Loop filters can be applied after all frame blocks have been reconstructed. Examples include deblocking filters, Sample Adaptive Offset (SAO) filters, or adaptive loop filters.

[0077] The reconstructed block or the filtered reconstructed block forms a reference frame, which can be stored in the decoded frame buffer (DPB) so that it can be used as a reference frame for encoding the next current block of the video frame VP or as a reference frame for encoding the next video frame to be encoded.

[0078] Figure 6 A schematic block diagram of the steps of a method 200 for decoding a video frame VP according to the prior art is shown.

[0079] In step 210, partition information data, prediction information data, and quantized transform coefficient blocks (or quantized residual blocks) are obtained by entropy decoding of the bitstream of encoded video frame data. For example, this bitstream has been generated according to method 100.

[0080] It can also perform entropy decoding on other information data to decode the current block of the video frame VP from the bitstream.

[0081] In step 220, the reconstructed frame is divided into current blocks based on the partitioning information. Each current block is entropy-decoded from the bitstream along a decoding loop (also known as "in loop"). Each decoded current block is either a quantized transform coefficient block or a quantized prediction residual block.

[0082] In step 230, the current block is dequantized and possibly inverse transformed (step 240) to obtain the decoded prediction residual block.

[0083] On the other hand, prediction information data is used to predict the current block. The predicted block is obtained through its intra-frame prediction (step 250) or its motion-compensated temporal prediction (step 260). The prediction process performed on the decoding side is exactly the same as the prediction process on the encoding side.

[0084] Next, the decoded prediction residual blocks and prediction blocks are combined (usually by summation), which provides the reconstructed blocks.

[0085] In step 270, the loop filter can be applied to the reconstructed frame (including the reconstructed block), and the reconstructed block or the filtered reconstructed block forms a reference frame, which can be stored in the decoded frame buffer (DPB), as discussed above. Figure 5 ).

[0086] exist Figure 5 Step 130 / 135 or Figure 6 In step 260, an inter-frame prediction block is defined based on inter-frame prediction data associated with the current block (CU or PU in the CU) of the video frame. This inter-frame prediction data contains motion information, which can be represented (encoded / decoded) according to either a so-called block-based Adaptive Motion Vector Prediction (AMVP) mode or a so-called block-based merging mode.

[0087] ECM defines a DIMD mode for determining the predicted samples of both the luma and chroma samples of the current block. DIMD mode is not based on a linear-model (LM) mode; in short, it is a non-LM mode, i.e., an intra-prediction mode that does not reference a linear model. Furthermore, DIMD mode does not require explicit parameter signaling, such as planar prediction mode or Direct Mode (DM).

[0088] For luminance sample prediction, the use of the DIMD luminance mode is signaled in the bitstream via a single flag. Instead of explicitly signaling the intra-predictor in the bitstream, the intra-predictor is derived based on gradient analysis of neighboring reconstructed luminance samples; that is, the DIMD luminance mode is implicit. Therefore, the intra-predictor is derived in the same manner during the reconstruction process of the DIMD luminance mode on both the encoder and decoder sides.

[0089] In DIMD brightness mode, such as Figure 7 The diagram illustrates a template block T with a 3-sample width (width or height) (composed of the left, top, and top-left reconstructed luminance samples of the reconstructed block R). Within this block T, edge detection filters (3x3 horizontal and vertical Sobel filters) adjacent to the current block B (current CU) are applied to determine the magnitude and angle of the luminance direction (orientation) of each midline sample in the (available) template block T. A histogram of gradients (HoG) is calculated, where each entry corresponds to a regular intra-frame angle pattern, and the cumulative intensity (magnitude) is stored. ; , Among them G hor and G ver These are the pure horizontal and vertical intensities calculated by the Sobel filter. Regular intra-angle modes refer to the intra-angle modes existing in specifications such as HEVC and VVC, where each angle mode is specifically designed to capture the direction or angle of the texture. Note that the term non-angle mode refers to an intra-angle mode that does not capture the direction or angle of the texture. For example, Direct Coding (DC) mode and planar intra-angle modes can be considered non-angle modes.

[0090] In DIMD luma mode, luma blending is applied as a weighted average of the three luma predictors for the current block: two HoG-based predictors and one planar predictor (M. Abdoli et al., “Non-CE3: Decoder-side Intra Mode Derivation with Prediction Fusion Using Planar”, JVET-O0449, Gothenburg, July 2019).

[0091] The planar predictor is derived from a planar pattern in which the predicted sample value is obtained as a weighted average of four reference sample values. Here, reference samples located in the same row or column as the current sample and those located in the lower left and upper right positions relative to the current block are used. In VVC, depending on the block size, a reference sample smoothing filter is also applied to the luminance component for the planar pattern.

[0092] like Figure 8 As illustrated, the predicted block for the current block is derived from a weighted linear combination of six predictors: five HoG-based predictors are derived using the five most representative angular intra-frame patterns Mi (i=1 to 5), while one predictor is derived using a planar pattern. Figure 8 A variant is illustrated where the prediction block is derived only as a weighted linear combination of three HoG-based predictors, but the principle can be easily extended to six predictors: two HoG-based predictors M1 and M2 selected from HoG (with the largest histogram amplitude values) and a third predictor derived using a planar pattern. The weights w1, w2, and w3 used in the weighted linear combination of these three predictors are derived, for example, as follows: the planar weight is fixed at 21 / 64 (approximately 1 / 3). The remaining 43 / 64 (approximately 2 / 3) weights are proportionally distributed among the other two weights to the amplitude of their HoG histogram bars. As another example, for a size of... W×H The three weights of the block are derived as follows: the weights are derived for each of the three predictors according to the following discussion on the DIMD merging mode. wDimd i Next, proceed according to the following weight-based... wDimd i The weights of each of the three derived patterns are derived. w i If the HoG amplitude above or to the left is twice that of the other, then the weights are... w i ( x , yLocation-related and calculated as follows: If the upper HoG is twice the left HoG, then: , If the HoG on the left is twice the HoG on the top, then: , in i It is predefined and can be set to 10.

[0093] For chroma sample prediction, the use of the DIMD chroma mode is signaled in the bitstream via a single flag. Instead of explicitly signaling the intra-predictor in the bitstream, the intra-predictor is derived using gradient analysis of neighboring reconstructed luma and chroma samples; that is, the DIMD chroma mode is implicit. Therefore, the intra-predictor is derived in the same manner during the reconstruction process on both the encoder and decoder sides based on the DIMD chroma mode.

[0094] In DIMD chroma mode, the intra-predictor of the chroma block can be derived based on the co-position reconstructed luminance samples of the luminance block associated with the chroma block to be predicted. Specifically, for each co-position reconstructed luminance sample ( Figure 9 The gray circles on the graph, taken from JVET-Y0092 (Hyeongmun Jang et al., “CE5-2.5: De-block filtering with 4x4chroma sample grid”, JVET-Y0092, Gothenburg, July 2019), calculate the horizontal and vertical gradients to establish the HoG. Then, the intra-prediction mode with the largest histogram magnitude value is selected from the HoG.

[0095] In the variant, under DIMD chroma mode, an intra-predictor for the chroma block can be derived based on neighboring reconstructed chroma samples.

[0096] In the variant, an intra-frame predictor for the chroma block can be derived by reconstructing luminance samples from co-locations of the luminance block associated with the chroma block to be predicted and by reconstructing chroma samples from neighboring blocks.

[0097] Typically, when the intra-prediction mode derived from the DIMD chroma mode is the same as the intra-prediction mode derived from the DM (Direct Mode) mode, the intra-prediction mode with the second largest histogram amplitude value is selected as the DIMD chroma mode.

[0098] JVET-AF0120 (Saverio Blasi et al., “EE2-2.1 DIMD merge”, JVET-AF0120, Hanover, October 2023) also proposes a DIMD merging mode. When using the DIMD merging mode, the predicted block for the current block is derived from DIMD information extracted from its neighboring blocks. First, the Merged Histogram of Gradients (MHoG) of the current block is calculated based on the gradient histograms (HoGs) of at least one DIMD neighboring block (i.e., based on neighboring blocks encoded / decoded by the predicted block, which are derived from DIMD or the DIMD merging mode). When a single DIMD neighboring block is available, its gradient histogram (HoG) is used to form the MHoG of the current block. If more than one DIMD neighboring block is available, the corresponding HoGs are combined using magnitude averaging to derive the MHoG. Up to 13 DIMD neighboring blocks are considered for DIMD information extraction. Next, as discussed above regarding DIMD modes, MHoG is used to define the DIMD merging mode and weights. Regular intra-frame angle modes and their weights corresponding to the five highest amplitudes in the MHoG are selected, and the corresponding predictors are mixed as in DIMD merging mode.

[0099] The DIMD information used to derive the predictor for the current block depends on the DIMD neighboring blocks. In JVET-AF0106 (Junyan Huo et al., “EE2-related: Non-adjacent spatial candidates for DIMDmerge”, Hanover, October 2023), the DIMD neighboring block may be a non-adjacent spatial neighboring block, that is, a neighboring block of the current block that does not share corners or edges with the current block.

[0100] For example, Figure 10 An example of defining non-adjacent spatial neighbor blocks is shown. The distance between the current block and its non-adjacent spatial neighbor blocks can be defined based on the width and height of the current block.

[0101] DIMD modes or DIMD merging modes (for luminance or chrominance) are associated with related information such as weights discussed above. The information associated with DIMD modes or DIMD merging modes can be stored in memory for blocks of video frames (VPs) that are encoded / decoded based on prediction blocks derived from DIMD modes or DIMD merging modes.

[0102] EMC further defines Template-based Intra Mode Derivation (TIMD) modes. TIMD modes are used to predict blocks and derive the intra predictor for that block based on previously reconstructed samples in the video. The TIMD mode associated with a block is also associated with the prediction cost evaluated based on samples defined around the template for that block.

[0103] For example, the TIMD mode can be derived for each intra-prediction candidate mode in the MPM list and for each wide-angle mode (if the upper-right and / or lower-left samples of the first template T1 are available). Wide-angle modes are described in JVET-Q2002 (Algorithm description for Versatile Video Coding and Test Model 8 (VTM 8), JVET-Q2002, Brussels, January 7-17, 2020). In short, the regular angular intra-prediction direction is defined as clockwise from 45 degrees to -135 degrees. In VVC, for non-square blocks, several regular angular intra-prediction modes are adaptively replaced by wide-angle intra-prediction modes. The original mode indices are used to signal the replaced modes; these original mode indices are remapped to the wide-angle mode indices after resolution. The number of modes replaced in the wide-angle direction mode depends on the block's aspect ratio. In the case of wide-angle intra-prediction, two vertically adjacent prediction samples can use two non-adjacent reference samples. Therefore, low-pass reference sample filtering and side smoothing are applied to wide-angle prediction to reduce gaps. The negative impact of increased pα. If the wide-angle mode represents a non-fractional offset, then there are 8 modes in the wide-angle mode that satisfy this condition, namely [-14, -12, -10, -6, 72, 76, 78, 80]. When predicting blocks using these modes, samples from the reference buffer are directly copied without applying any interpolation. This modification reduces the number of samples that need to be smoothed. Furthermore, it aligns the design of the regular prediction mode with that of the non-fractional modes in the wide-angle mode.

[0104] The sum of absolute transformed differences (SATD), sum of absolute differences (SAD), or sum of squared errors (SSE) between samples from the second template T2 of the block and samples from the reconstructed template of the block can be used to evaluate the prediction cost associated with a TIMD mode. The samples from the reconstructed template of the block are samples located in the second template T2 of the block that are encoded and decoded based on intra-prediction candidate modes or based on wide-angle modes (if present). Thus, a list of DIMD candidate modes associated with the prediction cost can be constructed. Next, two intra-prediction modes, DIMD1 and DIMD2, with the lowest prediction costs are selected from the list of DIMD candidate modes. These two selected DIMD modes, DIMD1 and DIMD2, are considered as two TIMD modes, TIMD1 and TIMD2. Next, the condition between the prediction costs of these two selected TIMD modes is examined to determine whether a mixture of the two selected TIMD modes should be applied.

[0105] For example, the condition is met when the prediction cost of the second selected TIMD pattern, TIMD2, is less than twice the prediction cost of the first selected TIMD pattern, TIMD1 (corresponding to the minimum prediction cost). cost(TIMD1). If the condition is met, fusion is applied; otherwise, the TIMD pattern added to the MPM list is the first TIMD pattern, TIMD1.

[0106] Figure 11 An example of a first template T1 for the block (current CU) used for template prediction and an example of a second template used to calculate the prediction cost are shown. PDPC weights are obtained by applying PDPC (Position-Dependent Intra-Prediction Combination) filtering to the predictions derived based on these two selected TIMD modes. These PDPC weights are similar to those used in the PDPC process of VVC, where the weights depend on the block size. Thus, the predictions derived based on these two selected TIMD modes can be mixed with the PDPC weights, and this weighted intra-prediction is used to encode and decode the current block.

[0107] The weights can be calculated based on the predicted costs, as follows: weight1 = costMode2 / (costMode1+ costMode2) weight2 = 1 - weight1 Therefore, TIMD modes can be associated with information indicating the two selected TIMD modes and a blending (mixing) flag indicating whether blending of the two selected TIMD modes is enabled. This association information may also indicate weights weight1 and weight2 and / or prediction costs. The information associated with the TIMD modes can be stored in memory for blocks of video frames (VPs) encoded / decoded based on prediction blocks derived from the TIMD modes.

[0108] EMC further defines an intra-prediction fusion method that can be used to derive the predicted block of the current block as a weighted combination of multiple intra-prediction blocks generated according to different reference lines. A reference line refers to samples from specific rows and / or columns of the reconstructed block used for intra-prediction. For example, samples from the immediate row adjacent to the upper reference side of the current block are called reference line 0. Figure 12 An example of a reference line is illustrated. Basically, in the intra-prediction fusion method, multiple intra-prediction blocks are first generated, and then fused using a weighted average. The intra-prediction fusion method can be summarized as follows: For the angle intra-prediction mode in a single-mode case including TIMD and DIMD, the intra-prediction fusion method derives the intra-prediction block by weighting the intra-prediction blocks obtained from multiple reference lines, represented as... p fusion =w 0 p line +w 1 p line+1 ,in p line It is an intra-predicted block from reference line 0 (from the immediate reference line above and / or the left block edge). p line+1 This is an intra-predicted block from a reference line above the default reference line. The weights are set to... w 0 = 3 / 4 and w 1 = 1 / 4. The intra-frame prediction mode can be obtained in various ways. For example, the same intra-frame mode derived from TIMD and DIMD modes can be used. Alternatively, the intra-frame mode can be determined based on rate-distortion optimization on the encoder side and signaled to the decoder in the bitstream.

[0109] For TIMD modes with hybridity, p line For the first selected TIMD mode TIMD1 ( w 0=1 w 1=0); p line+1For the second selected TIMD mode TIMD2 ( w 0=0、 w 1=1).

[0110] For DIMD modes with hybrid features, the number of intra-prediction blocks selected for weighted averaging is increased from 3 to 6.

[0111] When the intra-frame mode has a non-integer slope (requiring reference sample interpolation) and the block size is greater than 16, the intra-frame prediction fusion method is applied to the luma block. This method is used in conjunction with Multiple Reference Lines (MRLs), such as... Figure 12 The diagram shown is not applicable to ISP codec blocks. ISP is an intra-frame sub-partition as defined in JVET-Q2002.

[0112] VVC further defines a Combined Intra / Interpretation (CIIP) mode, which essentially involves deriving the prediction block of the current block as a weighted linear combination of an intra-prediction block and an inter-prediction block. A first weight, wIntra, is applied to the intra-prediction block, and a second weight, wInter, is applied to the inter-prediction block.

[0113] For example, inter-frame prediction blocks are derived based on CIIP-TM merging candidates (referring to inter-frame prediction with template matching mechanism), and intra-frame prediction blocks are derived based on the selected TIMD mode.

[0114] For near-horizontal mode (2 <= angle mode index < 34), the current block is vertically split into 4 sub-blocks, such as... Figure 13 As shown in (a); for near-vertical mode (34 <= angle mode index <= 66), the current block is horizontally split into 4 sub-blocks, as follows: Figure 13 As shown in (b). For example, the weights wIntra and wInter for different sub-blocks are shown in Table 1.

[0115] Table 1

[0116] According to one aspect, this disclosure relates to a TIMD merging mode, including at least one intra-frame mode derivation candidate, namely at least one DIMD and / or TIMD mode, and associated information for intra-frame prediction of the current block of a video frame VP. The at least one DIMD and / or TIMD mode candidate is collected from at least one previously encoded (decoded) block that was predicted using a TIMD mode, a DIMD mode, a TIMD merging mode, or a DIMD merging mode. A merging list of TIMD merging mode candidates is available. Each TIMD merging mode candidate may contain an internal prediction mode of at least one DIMD or TIMD mode and its associated information. Each TIMD merging mode candidate is used for intra-frame prediction of a previously encoded (decoded) block. The TIMD merging mode for intra-frame prediction of the current block is determined by selecting a TIMD merging mode candidate from the merging list based on the prediction cost obtained from the TIMD merging mode candidates in the merging list.

[0117] Figure 14 The illustration shows an example of a merge list according to some embodiments and an example of information associated with each TIMD merge mode candidate in the merge list.

[0118] In this example, the merge list contains N TIMD merge pattern candidates associated with indices "candidate index 1" to "candidate index N".

[0119] Each TIMD merge mode candidate contains at least one internal prediction mode and associated information for intra-frame prediction of previously encoded or decoded blocks. For example, the TIMD merge mode candidate with index "Candidate Index 3" contains a list of internal prediction modes with associated information.

[0120] Preferred TIMD mode: The primary intra-frame prediction mode (can be any angle mode or non-angle mode), which typically has the highest fusion weight among multiple TIMD modes; TIMD Fusion Mode: The fusion weight of the remaining intra-frame modes (which can be any angle mode or non-angle mode) is lower than that of the preferred mode. Mixing flag: A flag indicating whether the preferred mode and blending mode are mixed; Template cost: For example, the prediction cost of each preferred pattern and fusion pattern on a defined template; Transform type: The transform type used for TIMD codec blocks; The wide-angle condition indicates whether each candidate (preferred mode and / or blend mode) is in wide-angle mode.

[0121] This type of related information can be one or more of the following: • Preferred TIMD mode: The primary intra-frame prediction mode (can be any angle mode or non-angle mode), which typically has the highest fusion weight among multiple TIMD modes; • TIMD fusion mode, used to blend with the preferred TIMD mode: the remaining intra-frame modes (which can be any angle mode or non-angle mode) have a lower fusion weight than the preferred mode; • The fusion weight of the preferred mode and the fusion mode; • Mixing flag indicator: A flag indicating whether the preferred mode and blending mode are mixed; • Template costs for preferred and / or hybrid modes; • Transformation type: The transformation type used for TIMD codec blocks; • The wide-angle condition for preferred mode and blended mode indicates whether each candidate (preferred mode and / or blended mode) is in wide-angle mode.

[0122] The TIMD merge mode is selected from the TIMD merge mode candidates in the merge list as a candidate for intra-frame prediction of the current block of the video frame VP. This avoids deriving noisy TIMD or DIMD modes, because these TIMD or DIMD modes have been previously selected for prediction of other blocks of the video frame VP (containing the current block to be predicted) or previously encoded / decoded video frames.

[0123] Figure 15 A flowchart of a method 1500, according to some embodiments, for determining a TIMD merging mode as a candidate for intra-frame prediction of the current block of a video frame VP, is shown. For example, method 1500 may be performed by an encoder or a decoder. In some examples, the steps of method 1500 may be... Figure 5 Method 100, step 120, or sub-step 120 Figure 6 Method 200 includes steps 250 and sub-steps.

[0124] In step 1510, a merging list of at least one TIMD merging mode candidate is obtained. Each TIMD merging mode candidate contains at least one internal prediction mode and association information for intra-frame prediction.

[0125] In some embodiments, each TIMD merge mode candidate includes at least one internal prediction mode and associated information for intra-frame prediction of at least one previously encoded or decoded block. For example, the TIMD merge mode may be a TIMD or DIMD mode for intra-frame prediction of at least one previously encoded or decoded block.

[0126] In step 1520, the TIMD merge mode is determined as a candidate for intra-frame prediction of the current block in the merge list of TIMD merge mode candidates.

[0127] In some embodiments, determining the (1520) TIMD merge mode includes steps 1521 and 1522.

[0128] In step 1521, the prediction cost of each TIMD merge pattern candidate pattern in the merge list on the defined template is obtained.

[0129] The defined template can be a block from a reconstruction sample of the block's neighbors. For example, the template around the current block could be... Figure 11 Templates. For example, the size and shape of the template can be determined based on the block size. When the TIMD merge mode is DIMD mode, the template definition for TIMD mode and DIMD mode can be the same, or the template definition for TIMD mode and DIMD mode can be different.

[0130] In step 1522, a TIMD merge pattern is selected based on the predicted costs obtained for the TIMD merge pattern candidates. In other words, the predicted cost of the TIMD merge pattern is lower than the predicted costs of other TIMD merge pattern candidates in the merge list.

[0131] In some embodiments of method 1500, in step 1530, information indicating whether TIMD merging mode is enabled as a candidate for intra-frame prediction of the current block of a video frame is signaled in the bitstream, i.e., encoded in the bitstream by the encoder and decoded from the bitstream by the decoder.

[0132] This embodiment allows enabling / disabling the use of TIMD merging mode as a candidate for intra-frame prediction of the current block.

[0133] For example, this information is the TIMD merge flag for the current block encoded in the bitstream.

[0134] In some embodiments, information indicating whether TIMD merging mode is enabled as a candidate for intra-frame prediction of the current block is signaled at the sequence level (such as in the Sequence Parameter Set (SPS)), at the picture level (such as in the Picture Parameter Set (PPS) and the Picture Header (PH)), or at the block level (CU level).

[0135] Figure 16An example of the syntax of SPS that can be added to the underlying codec according to an embodiment is shown for signaling information indicating whether TIMD merge mode is enabled as a candidate for intra-frame prediction of the current block of a video frame.

[0136] The syntax element `sps_timd_enabled_flag` indicates whether TIMD mode is enabled as a candidate for intra-frame prediction of blocks of video frames.

[0137] The syntax element `sps_timd_merge_enabled_flag` indicates whether the TIMD merging mode is enabled as a candidate for intra-frame prediction of blocks of video frames.

[0138] Figure 17 An example of a syntax that can be added to the underlying codec according to an embodiment is shown for signaling information indicating whether TIMD merge mode is enabled as a candidate for intra-frame prediction of the current block of a video frame.

[0139] The syntax element intra_timd_flag indicates whether the traditional TIMD mode is enabled as a candidate for intra-frame prediction of the current CU.

[0140] The syntax element intra_timd_merge_flag indicates whether the TIMD merging mode is enabled as a candidate for intra-frame prediction of the current CU.

[0141] The syntax element intra_timd_merge_index[ x0 ][ y0 ] indicates the candidate index of the TIMD merge pattern in the merge list.

[0142] exist Figure 16 and Figure 17 In the example, the TIMD merge mode is defined as a sub-mode of the traditional TIMD mode. Therefore, if the traditional TIMD mode is disabled, the TIMD merge mode is also disabled. It is important to understand that the TIMD merge mode can also be used independently of the traditional TIMD mode.

[0143] In some embodiments, a TIMD merging mode is enabled for certain frames or slices. For example, the TIMD merging mode may be enabled only in intra-frame prediction-based codec frames or slices, and disabled in inter-frame prediction-based codec frames or slices.

[0144] In some embodiments, at least one TIMD merging pattern candidate previously used for intra-frame prediction of block B of the video in the merging list can be a TIMD pattern with associated information. This TIMD pattern uses a first template defined around block B of the video to determine the intra-frame predictor for block B, and uses a second template defined around block B to evaluate the prediction cost. Figure 11 ).

[0145] In some embodiments, at least one TIMD merge mode candidate previously used for intra-frame prediction of block B of the video in the merge list may be a DIMD mode or a DIMD merge mode and associated information.

[0146] In some embodiments, the prediction cost between block B and the reconstructed block can be evaluated based on an intra-frame predictor (derived from DIMD or DIMD merging mode).

[0147] In some embodiments, when a TIMD merge pattern candidate includes multiple internal prediction patterns, the template prediction cost of the TIMD merge pattern candidate is calculated by summing or averaging the prediction costs of the internal prediction patterns of the TIMD merge pattern candidate on the template (step 1521).

[0148] In some embodiments, the template prediction cost of the TIMD merge pattern candidate is calculated by summing or averaging the prediction costs of the internal prediction patterns of the TIMD merge pattern candidate at the block level (step 1521).

[0149] In some embodiments, the costs of the two internal prediction modes can be weighted at the pixel level. This means that each internal prediction mode is applied as a predictor to the template sample, and the pixel-wise distortion of each prediction mode is calculated. Then, the prediction cost of the TIMD merged mode candidate is calculated pixel-wise, such that different pixel positions on the template have different pixel-wise weights in the averaging.

[0150] In some embodiments, the prediction cost associated with the DIMD mode can be evaluated as the sum of absolute differences (SAD), the sum of absolute transform differences (SATD), or the sum of squared errors (SSE) between the samples of the block to be predicted and the samples of the reconstructed block. The samples of the reconstructed block are the samples located in the block encoded and decoded based on the intra-frame predictor (derived from the DIMD mode).

[0151] In some embodiments, at least one TIMD merge mode candidate mode previously used for intra-frame prediction of the current block of the video in the merge list is a TIMD merge mode.

[0152] In some embodiments of method 1500, when step 1520 is a sub-step of step 120 of method 100, a TIMD merging mode is selected from at least one TIMD merging mode candidate in the merging list, which may be based on rate-distortion optimization (RDO) similar to equation (1). The encoder can then evaluate the rate-distortion cost for intra-frame prediction of the current block of the video frame VP, each intra-frame prediction being derived based on a TIMD merging mode candidate in the merging list and its associated prediction cost.

[0153] In some embodiments, the index of the TIMD merge mode candidate determined in the merge list can then be signaled in the bitstream, i.e., the encoder encodes the index into the bitstream, and the decoder decodes the index from the bitstream.

[0154] In one embodiment of step 1522, at least one TIMD merge mode candidate in the merge list may be reordered or rearranged to reduce the bit rate of the index used to signal the indication of the TIMD merge mode.

[0155] For example, TIMD merge pattern candidates in the merge list can be ordered based on their associated prediction costs. Before using these prediction costs to order the TIMD merge pattern candidates in the merge list, they can be normalized. Specifically, normalization is necessary because the stored prediction cost values ​​may have already been averaged over different numbers of pixels based on their block size. Therefore, normalization can be performed based on the block size, the number of samples in the block, or the associated template.

[0156] In one embodiment of method 1500, when step 15210 is a sub-step of step 250 of method 200, and when the TIMD merge mode candidates in the merge list contain only TIMD modes, the decoder can evaluate the prediction cost of each TIMD mode on the defined template, and can select the TIMD merge mode candidate in the merge list that minimizes the template-based prediction cost.

[0157] This embodiment is advantageous because it avoids signaling the index of a TIMD merge mode candidate in the merge list.

[0158] In some embodiments, the information associated with TIMD and / or DIMD patterns calculated as discussed above may be modified before being considered as information associated with TIMD merge pattern candidates in the merge list.

[0159] In some embodiments, TIMD and / or DIMD patterns may be associated with fusion weights, which may be modified and then treated as associated information of TIMD and / or DIMD patterns of TIMD merging pattern candidates in the merging list.

[0160] For example, the fusion weights can be modified or refined based on the prediction costs associated with TIMD and / or DIMD candidate modes. For instance, a refinement range can be defined, where the weights iteratively modified based on the refinement value and the best-performing refinement weights can be determined based on the post-fusion prediction costs. The code in Table 1 below illustrates an example of such a process where TIMD fusion mode candidates include two internal prediction modes. Using two corresponding fusion weights ( Weight 0 and Weight 1) To mix the two predictions derived from these two internal prediction models ( Pred 0 and Pred 1) Based on considering the cost of SAD after hybridization refinementRange The values ​​are then refined for these two weights. Based on this process, the optimal refined weights are determined and used as the combined weights in the final prediction.

[0161] Table 1

[0162] In some embodiments, information associated with at least one internal prediction pattern in TIMD and / or DIMD patterns may be modified and then treated as associated information for TIMD merge pattern candidates in the merge list.

[0163] For example, information associated with TIMD and / or DIMD patterns can be refined using a refined incremental angle (± increment).

[0164] For example, the refinement increment angle used to modify information associated with TIMD and / or DIMD patterns can be determined to be a value within a certain range that minimizes the prediction cost, which is calculated for predicting the template of the current block based on the TIMD and / or DIMD patterns (after modification by a value within that range).

[0165] More precisely, the range of refinement increment angles is determined. The prediction cost is calculated for predicting the template of the current block based on the TIMD and / or DIMD patterns (modified by a value within this range). The refinement increment angle is the value within this range associated with the minimum prediction cost.

[0166] In some embodiments, the refinement increment angle can be calculated based on the template of the current block.

[0167] In this scenario, a TIMD merging mode is first selected for each block. Then, its intra-prediction mode is modified using different increment angles within the value range. For each increment angle, the modified intra-prediction mode is applied to the reconstructed samples on the template, and its cost is calculated. Finally, the increment angle whose modified intra-prediction mode minimizes the cost is selected. Because this refinement process depends on the reconstructed samples, the decoder can perform the same process, thus eliminating the need to signal the selected increment.

[0168] This embodiment is advantageous because it avoids signaling the refinement increment angle in the bitstream, thus not increasing the overall bit rate.

[0169] Table 2 provides information for use refinementRange Internal prediction patterns for TIMD merge pattern candidates in the merge list ( Mode Example of program code that refines a related piece of information (0). The optimal refinement mode is stored based on the SAD-based cost on the template and used as the final intra-frame prediction mode.

[0170] Table 2

[0171] In some embodiments, when a TIMD merge mode candidate is associated with a refinement increment angle, information indicating the refinement increment angle can be signaled in the bitstream. The refinement increment angle is determined by decoding the information from the bitstream by a decoder, and the TIMD and / or DIMD modes of the TIMD merge mode candidate are modified accordingly.

[0172] In some embodiments, at least one TIMD merge mode candidate in the merge list has previously been used to perform intra-frame prediction of adjacent spatial neighbor blocks of the current block.

[0173] Figure 18 An example of adjacent spatial neighbor blocks of the current block is shown according to some embodiments. The adjacent spatial neighbor blocks (nearby CU1-CU5) of the current block (current CU) can be defined as causal neighbor blocks that share at least one pixel on an edge with the current block. The causality of the neighbor blocks is defined based on the encoding / decoding order of the blocks in the video frame VP.

[0174] In some embodiments, at least one TIMD merge mode candidate in the merge list has previously been used to perform intra-frame prediction of non-adjacent spatial neighbor blocks of the current block.

[0175] For example, Figure 10 The non-adjacent spatial neighboring blocks of the current block are shown.

[0176] In some embodiments, the merge list may contain at least one TIMD merge candidate previously used for intra-frame prediction of non-adjacent neighboring blocks of the current block and at least one TIMD merge mode candidate previously used for intra-frame prediction of adjacent spatial neighboring blocks of the current block.

[0177] In some embodiments, the reference block for non-adjacent spatial neighboring blocks in the previous embodiments may be defined in a standard specification.

[0178] For example, in practice, considering each block in the search area, and if the intra-prediction mode is TIMD, TIMD merge, or DIMD or DIMD merge (i.e., if the block's intra-predictor can be derived from reconstructed samples of the video), then the intra-prediction mode previously used for intra-prediction of the block is a TIMD merge candidate in the merge list. In this case, the block is labeled as a TIMD / DIMD block.

[0179] In some embodiments, the search block may be fixed.

[0180] In some embodiments, the search block can be defined based on a previous reconstructed block of the video frame VP.

[0181] In some embodiments, the search block may depend on the size (height and width) of the current block.

[0182] In some embodiments, the current block belongs to the current CTU, and the search block can be limited to reconstructed blocks within the current CTU or multiple CTUs.

[0183] In some embodiments, the current block belongs to a tile and / or slice and / or sub-picture edge, and the search block may be limited to previously reconstructed blocks within that tile and / or slice and / or sub-picture edge.

[0184] In some embodiments, TIMD merge pattern candidates in the merge list can be reordered based on the spatial distance (relative to, for example, the horizontal and / or vertical distance from the top-left corner of the current block) between the current block and the blocks from which it inherits or collects TIMD merge pattern candidates. For example, TIMD merge pattern candidates collected from the current block's immediate neighboring blocks can be placed at the beginning of the merge list, while TIMD merge pattern candidates collected from the current block's non-neighboring blocks can be placed later.

[0185] In some embodiments, at least one TIMD mode candidate in the merge list may have been previously used for intra-frame prediction of blocks in a reference frame in a reference frame buffer.

[0186] In some embodiments, the block in the reference frame is the co-located block of the current block in the horizontal and vertical coordinates relative to the top-left corner of the video frame.

[0187] In some embodiments, the index indicating the reference frame can be fixed or signaled in the bitstream, such as in High Level Syntax (HLS), such as at the frame level, subframe level, slice level, tile level, or block level.

[0188] In some embodiments, the block in the reference frame is a co-location block within the extended block of the co-location block in the reference frame where the current block is located.

[0189] For example, the size of each extended block can be fixed or signaled (e.g. at the frame level) in the bitstream, or the size of each extended block can be determined based on block characteristics (such as block size) or video frame characteristics (such as frame number) or search block characteristics (such as the number of reference frames).

[0190] In some embodiments, a block in a reference frame may be a block in a reference frame indicated by motion vectors from adjacent and / or non-adjacent blocks of the current block.

[0191] For example, such as Figure 19 As illustrated, co-occurrence block 1900 in the search block may belong to reference block 1910 in reference frame RP of video frame VP, and reference block 1910 may be indicated by motion vector 1920 pointing to reference block 1910 in reference frame RP. Reference block 1910 may therefore contain co-occurrence block 1900 of the current block and / or other blocks in reference frame RP.

[0192] In some embodiments, motion vector 1920 may be inherited from one or more motion information of adjacent and / or non-adjacent spatial neighboring blocks of the current block. Figure 19 An example of motion vector 1920 inherited from a neighboring spatial block 1930 of the current block 1940 is shown.

[0193] In some embodiments, the information indicating whether the TIMD merging mode is enabled as a candidate for intra-frame prediction of the current block of a video frame may depend on the availability of TIMD / DIMD blocks.

[0194] In some embodiments, if no TIMD / DIMD block is available in the search block, instead of signaling in the bitstream whether TIMD merging mode is enabled as a candidate for intra-frame prediction of the current block of the video frame, the information is implicitly deduced as false on the decoder side.

[0195] In some embodiments, the signaling indicating the index of the TIMD merge mode candidate identified in the merge list may depend on the availability of TIMD / DIMD blocks in the search block.

[0196] In some embodiments, if the number of available TIMD / DIMD blocks in the search block is less than the maximum list size, the signaling indicating the index of the TIMD merge mode candidate identified in the merge list can be modified to save bits.

[0197] For example, if the maximum list size is five, but there are only three available candidates, the signaling can be changed to disregard the index.

[0198] For example, suppose the maximum list size is five. In this case, the following unary code signaling can be used for the five possible indices: 0: Index 0 10: Index 1 110: Index 2 1110: Index 3 1111: Index 4 However, if only three candidates are available in the search block during parsing, the unary code is modified as follows: 0: Index 0 10: Index 1 110: Index 2 111: Index 3 This saves one bit for index 3, since index 4 of the initial unary code is impossible for the current block.

[0199] In another example, if there is only one available TIMD / DIMD block in the search block, the index of the TIMD merge mode candidate identified in the merge list is not signaled, but is implicitly known by the decoder.

[0200] In some embodiments, TIMD or DIMD or TIMD merge patterns and associated information, as well as the predicted costs of possible TIMD / DIMD blocks, can be stored in a history memory to serve as potential TIMD merge pattern candidates for future blocks in the merge list.

[0201] In some embodiments, the number of TIMD, DIMD and TIMD merged patterns in the historical memory can be limited to a certain number of TIMD, DIMD and TIMD merged patterns, and when a new TIMD, DIMD and TIMD merged pattern is inserted into the historical memory, the historical memory can be updated in a first-in-first-out manner, that is, if the historical memory has reached a certain number of TIMD, DIMD and TIMD merged patterns, the oldest pattern can be discarded from the historical memory.

[0202] In some embodiments, a trimming check may be performed when a new TIMD, DIMD, and TIMD merge pattern is inserted into the history memory to avoid adding duplicate patterns to the history memory.

[0203] In some embodiments, the history memory may be reset at the start of each CTU, tile, slice, or sub-picture. The history memory reset process may be indicated in the bitstream, such as in the HLS, for example at the slice level, picture level, or sequence level.

[0204] In some embodiments, TIMD merge mode candidates may also inherit transform types from blocks in the video corresponding to candidate modes in the merge list, such as Low-Frequency Non-Separable Transform and / or Multiple Transform Selection as defined in VVC.

[0205] In some embodiments, when a TIMD merge mode candidate contains two or more angular prediction modes (internal prediction modes), or when a TIMD merge mode candidate is a single angular mode (internal prediction mode) and not fused, intra-frame prediction for the current block can be obtained by mixing (one or more) angular prediction modes with non-angular modes (such as planar or DC prediction). The weights of the non-angular modes can be fixed or recalculated based on, for example, the prediction cost of the template.

[0206] In some embodiments, when a TIMD merge mode candidate contains two or more angle prediction modes (internal prediction modes) or when a TIMD merge mode candidate is a single angle mode (internal prediction mode) and is not mixed, the encoder can test the RDO performance of the TIMD merge mode candidate in the case of mixing and not mixing non-angle modes. In some embodiments, a signaling flag can be sent in the bitstream to indicate whether two or more angle prediction modes have been mixed.

[0207] For example, suppose we consider the current block. B In the neighborhood of the current block B, a merge list containing a single TIMD merge mode candidate D was obtained. TIMD merge mode candidates D It can contain two angle modes: 18 (pure horizontal) and 20 (diagonal). Based on the flags signaled, the predicted TIMD merged mode candidates in mixed scenarios indicate one of two predictions: When the flag is equal to 0, the current block B is predicted without DC / plane and using only mode 18 and 20 fusion. No fusion with DC / plane is used.

[0208] When the flag is equal to 1, the current block is predicted using either DC or planar mode as the first mode, and modes 18 and 20 as the second and third modes, respectively. B To determine whether DC mode or planar mode should be used as the first mode, prediction is performed on the template of the current block and the cost on the template sample is calculated. The DC mode or planar mode is selected based on the cost. Clearly, the decoder must also perform the same cost calculation when determining the first mode.

[0209] In some embodiments, the decoder can determine the use of non-angled modes (DC or plane) in the intra predictor of the current block based on the performance of the non-angled modes in the template. For example, if the performance of the non-angled modes is worse than that of the angled modes in the TIMD merge mode candidates, the decoder may decide not to use the non-angled modes in the final prediction intra predictor of the current block.

[0210] In some embodiments, a merge list can be obtained for each channel of the video frame, and the TIMD merge mode candidates in the merge list obtained for the first channel may also belong to the merge list obtained for the second channel.

[0211] For example, the TIMD merge mode candidates (and their associated information) in the merge list obtained for the luma block (the block of the first channel) can also be the TIMD merge mode candidates in the merge list obtained for the co-occurrence chroma block (the block of the second channel).

[0212] In some embodiments, the TIMD merging mode determined for the current block (e.g., the current luma block) of the first channel of the video frame VP can be inherited by the co-bit block (e.g., the co-bit chroma block) in the second channel of the video frame VP.

[0213] In some embodiments, when a TIMD merging mode and its associated information are determined for a block (e.g., a luma block) of a first channel of a video frame VP and inherited by a co-occurrence block (e.g., a chroma block) of the second channel of the video frame VP, some information (such as weights) associated with the TIMD merging mode determined for the first channel block can be recalculated or modified based on the prediction cost (evaluated based on samples of a template defined around the second channel block). Specifically, one piece of information that can be modified is the mode weights. This modification can be performed by applying predictions to template samples using a given prediction mode. Then, a distortion-based cost for each prediction mode is calculated. Finally, the modified weights for each prediction mode are determined using the costs just calculated.

[0214] This embodiment can be useful when the second channel is a chroma channel, because the texture of the chroma channel sample is usually smoother than the texture of the luminance sample (first channel).

[0215] When the luminance and chroma channels of a video frame VP may have different block partitions, the block covered by the co-bit luminance block may contain multiple blocks associated with different TIMD merging modes.

[0216] Figure 20 Examples of different block partitions for chroma blocks and co-occurring luma blocks according to some embodiments are shown. In this example, the luma block covers an area containing 10 blocks, and more than one of these 10 blocks may be associated with the TIMD merge mode.

[0217] In some embodiments, the TIMD merge block inherited by a co-occurrence block (e.g., a chroma block) of the second channel can be selected from TIMD merge modes associated with more than one co-occurrence block (e.g., a luma block) of the first channel. This is because, when using dual-tree block partitioning, the size of the luma block is typically smaller than the size of the co-occurrence chroma block. Therefore, a given chroma region can contain several co-occurrence luma blocks.

[0218] In some embodiments, the luma channel and chroma channel may have different block partitions. In this case, for the chroma block, in order to inherit TIMD merge mode information from the luma block, a search range in the co-location region of the chroma block in the luma channel can be defined. The search range can be fixed (e.g., defined in a standard specification) or it can depend on the block size. The search process scans the search block defined in the co-location luma to find the TIMD merge mode information to be used for the chroma block. The search process can stop when the first TIMD merge mode is found.

[0219] In some embodiments, the search range depends on the size of the co-occurrence block of the second channel.

[0220] In some embodiments, scanning of a sub-block can be stopped when the first sub-block associated with the TIMD merge pattern is found.

[0221] In some embodiments, information indicating whether the TIMD merge mode is enabled for the intra-predictor of the current block and / or the index indicating the candidate mode selected in the merge list can be entropy encoded and decoded.

[0222] In some embodiments, information indicating whether a TIMD merging mode is enabled as a candidate for intra-frame prediction of the current block and / or an index indicating the candidate mode selected in the merging list can be binarized, and these bits can be encoded / decoded by a binary encoder / decoder.

[0223] In some embodiments, the binary encoder / decoder is a context-adaptive binary arithmetic codec (CABAC).

[0224] In some embodiments, the characteristics of the current block or neighboring blocks or both can be used to select from the set of contexts a CABAC context (or equivalent probabilistic model) that encodes and decodes bits representing the following information and / or index: whether the TIMD merge mode is enabled as a candidate for intra-frame prediction of the current block, and the index indicating the candidate mode selected in the merge list.

[0225] For example, a specific context can be selected when at least one TIMD and / or DIMD block is available in the search block.

[0226] In another example, a specific context can be selected based on the size of the current block.

[0227] Figure 21 A schematic block diagram illustrating an example of a system 600 in which various aspects and exemplary embodiments are implemented is shown.

[0228] System 600 can be embedded as one or more devices, including the various components described below. In various exemplary embodiments, system 600 can be configured to implement one or more aspects described in this application.

[0229] Examples of equipment that may constitute all or part of System 600 include personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, connected vehicles and their associated processing systems, head-mounted display devices (HMDs, see-through glasses), projectors, "cave" (systems including multiple displays), servers, video encoders, video decoders, post-processors that process the output from the video decoder, pre-processors that provide input to the video encoder, web servers, video servers (e.g., broadcast servers, video-on-demand servers, or web servers), still or video cameras, encoding or decoding chips, or any other communication devices. The elements of System 600 may be implemented individually or in combination in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one exemplary embodiment, the processing and encoder / decoder elements of System 600 may be distributed across multiple ICs and / or discrete components. In various exemplary embodiments, System 600 may be communicatively connected to other similar systems or other electronic devices via, for example, a communication bus or through dedicated input and / or output ports.

[0230] System 600 may include at least one processor 610 configured to execute instructions loaded therein for implementing various aspects, such as those described in this application. Processor 610 may include embedded memory, input / output interfaces, and various other circuitry known in the art. System 600 may include at least one memory 620 (e.g., a volatile memory device and / or a non-volatile memory device). System 600 may include a storage device 640, which may include non-volatile memory and / or volatile memory, including but not limited to electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, disk drives, and / or optical disk drives. As a non-limiting example, storage device 640 may include internal storage devices, attached storage devices, and / or network-accessible storage devices.

[0231] System 600 may include an encoder / decoder module 630 configured to, for example, process data to provide encoded / decoded video image data, and the encoder / decoder module 630 may include its own processor and memory. The encoder / decoder module 630 may represent one or more modules that can be included in a device to perform encoding and / or decoding functions. As is known, a device may include one or both of the encoding and decoding modules. Furthermore, the encoder / decoder module 630 may be implemented as a separate element of system 600, or may be incorporated into processor 610 as a combination of hardware and software known to those skilled in the art.

[0232] Program code to be loaded into processor 610 or encoder / decoder 630 to execute the various aspects described in this application may be stored in storage device 640 and subsequently loaded into memory 620 for execution by processor 610. According to various exemplary embodiments, during the execution of the processes described in this application, one or more of processor 610, memory 620, storage device 640, and encoder / decoder module 630 may store one or more of various items. Such stored items may include, but are not limited to, video frame data, information data for encoding video frame data, bitstreams, matrices, variables, and intermediate or final results of equations, formulas, operations, and arithmetic logic processing.

[0233] In several exemplary embodiments, the memory within the processor 610 and / or encoder / decoder module 630 may be used to store instructions and provide working memory for processes that can be performed during encoding or decoding.

[0234] However, in other exemplary embodiments, external memory (e.g., the processing device may be processor 610 or encoder / decoder module 630) is used for one or more of these functions. External memory may be memory 620 and / or storage device 640, such as dynamic volatile memory and / or non-volatile flash memory. In several exemplary embodiments, external non-volatile flash memory is used to store the television's operating system. In at least one exemplary embodiment, fast external dynamic volatile memory, such as RAM, may be used as working memory for video encoding / decoding operations, for example, for MPEG-2 Part 2 (also known as ITU-T Recommendation H.262 and ISO / IEC 13818-2, also known as MPEG-2 video), AVC, HEVC, EVC, VVC, AVI, etc.

[0235] As indicated in box 690, input to the components of system 600 can be provided through various input devices. Such input devices include, but are not limited to, (i) an RF section capable of receiving, for example, RF signals transmitted over the air by a broadcasting device, (ii) a composite input terminal, (iii) a USB input terminal, (iv) an HDMI input terminal, and (v) a bus, such as CAN (Controller Area Network), CAN FD (Controller Area Network Flexible Data Rate), FlexRay (ISO 17458), or Ethernet (ISO / IEC 802-3) bus, when the invention is implemented in the automotive field.

[0236] In various exemplary embodiments, the input device of block 690 has associated corresponding input processing elements, as known in the art. For example, the RF section may be associated with elements necessary for: (i) selecting a desired frequency (also known as selecting a signal, or limiting the signal band to a specific band), (ii) down-converting the selected signal, (iii) further limiting the band to a narrower band to select, for example, the signal band that may be referred to as a channel in some exemplary embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select the desired data packet stream. The RF section of various exemplary embodiments may include one or more elements performing these functions, such as a frequency selector, signal selector, band limiter, channel selector, filter, downconverter, demodulator, error corrector, and demultiplexer. The RF section may include tuners performing various functions among these functions, including, for example, down-converting a received signal to a lower frequency (e.g., intermediate frequency or near-baseband frequency) or baseband.

[0237] In one set-top box embodiment, the RF section and its associated input processing elements can receive RF signals transmitted over a wired (e.g., cable) medium. The RF section can then perform frequency selection by filtering, down-converting, and re-filtering to the desired frequency band.

[0238] Various exemplary embodiments may rearrange the order of the above (and other) elements, remove some of these elements, and / or add other elements that perform similar or different functions.

[0239] Adding components may include inserting components between existing components, such as, for example, inserting amplifiers and analog-to-digital converters. In various exemplary embodiments, the RF portion may include an antenna.

[0240] Furthermore, the USB and / or HDMI terminals may include corresponding interface processors for connecting system 600 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Reed-Solomon error correction) may be implemented, for example, within a separate input processing IC or within processor 610, as needed. Similarly, various aspects of USB or HDMI interface processing may be implemented within a separate interface IC or within processor 610, as needed. The demodulated, error-corrected, and demultiplexed streams may be provided to various processing elements, including, for example, processor 610 and encoder / decoder 630, which operate in conjunction with memory and storage elements to process the data streams for presentation on an output device as needed.

[0241] Various components of system 600 can be provided within an integrated housing. Within the integrated housing, suitable connection arrangements 690, such as internal buses (including I2C buses), wiring, and printed circuit boards known in the art, can be used to interconnect various components and transfer data between them.

[0242] System 600 may include a communication interface 650 that enables communication with other devices via a communication channel 651. The communication interface 650 may include, but is not limited to, a transceiver configured to send and receive data on the communication channel 651. The communication interface 650 may include, but is not limited to, a modem or network interface card (NIC), and the communication channel 651 may be implemented, for example, within a wired and / or wireless medium.

[0243] In various exemplary embodiments, a Wi-Fi network such as IEEE 802.11 can be used to stream data to system 600. The Wi-Fi signals of these exemplary embodiments can be received via a communication channel 651 and a communication interface 650 suitable for Wi-Fi communication. The communication channel 651 of these exemplary embodiments can typically be connected to an access point or router that provides access to external networks, including the Internet, to allow streaming applications and other over-the-top cloud communications.

[0244] Other exemplary embodiments may use a set-top box to provide streaming data to system 600, the set-top box delivering data via an HDMI connection in input block 690.

[0245] Other exemplary embodiments may use the RF connection of input block 690 to provide streaming data to system 600.

[0246] Streamed data can be used as a means of signaling notification information by System 600. Signaling notification information may include bitstream B and / or information such as the number of video frame pixels and / or any encoding / decoding settings parameters.

[0247] It should be recognized that signaling notification can be implemented in a variety of ways. For example, in various exemplary embodiments, one or more syntax elements, flags, etc., can be used to send signaling notification information to the corresponding decoder.

[0248] System 600 can provide output signals to various output devices, including a display 661, a speaker 671, and other peripheral devices 681. In various examples of exemplary embodiments, other peripheral devices 681 may include one or more of a standalone DVR, a disc player, a stereo system, a lighting system, and other devices that provide functionality based on the output of system 600.

[0249] In various exemplary embodiments, control signals may be communicated between system 600 and display 661, speaker 671 or other peripheral devices 681 using signaling such as AV.Link (audio / video link), CEC (consumer electronics control), or other communication protocols that enable device-to-device control with or without user intervention.

[0250] Output devices can be connected to system 600 via dedicated connections through the corresponding interfaces 660, 670 and 680.

[0251] Alternatively, the output device can be connected to the system 600 via communication interface 650 using communication channel 651. The display 661 and speaker 671 can be integrated with other components of the system 600 into a single unit in an electronic device, such as a television set.

[0252] In various exemplary embodiments, the display interface 660 may include a display driver, such as, for example, a timing controller (T Con) chip.

[0253] For example, if the RF portion of input 690 is part of a separate set-top box, then display 661 and speaker 671 may optionally be separate from one or more other components. In various exemplary embodiments where display 661 and speaker 671 can be external components, output signals may be provided via dedicated output connections, including, for example, an HDMI port, a USB port, or a COMP output.

[0254] exist Figure 1-20 This document describes various methods, each comprising one or more steps or actions to implement the described method. Unless the correct operation of the method requires a specific order of steps or actions, the order and / or use of specific steps and / or actions may be modified or combined.

[0255] Examples of block diagrams and / or operation flowcharts are described. Each block represents a portion of circuitry, a module, or code, which includes one or more executable instructions for implementing one or more specified logical functions. It should also be noted that in other implementations, the functions (one or more) marked in the blocks may occur out of order. For example, depending on the functions involved, two blocks shown sequentially may actually execute substantially concurrently, or sometimes these blocks may be executed in reverse order.

[0256] The embodiments and aspects described herein may be implemented in, for example, methods or processes, apparatus, computer programs, data streams, bit streams, or signals. Even if discussed only in the context of a single form of implementation (e.g., discussed only as a method), implementations of the discussed features may be implemented in other forms (e.g., apparatus or computer programs).

[0257] The method can be implemented in, for example, a processor, which generally refers to a processing device, including, for example, a computer, microprocessor, integrated circuit, or programmable logic device. Processors also include communication devices.

[0258] Furthermore, the method can be implemented by instructions executed by a processor, and such instructions (and / or data values ​​generated by the implementation) can be stored on a computer-readable storage medium. The computer-readable storage medium can take the form of a computer-readable program product implemented in one or more computer-readable media and having computer-executable computer-readable program code implemented thereon. Considering the inherent ability to store information therein and the inherent ability to retrieve information provided therefrom, the computer-readable storage medium as used herein can be considered a non-transitory storage medium. The computer-readable storage medium can be, for example, but not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any suitable combination of the foregoing. It should be understood that while more specific examples of computer-readable storage media to which this exemplary embodiment can be applied are provided below, they are merely illustrative and not exhaustive, as will be readily recognized by those skilled in the art: portable computer floppy disks; hard disks; read-only memory (ROM); erasable programmable read-only memory (EPROM or flash memory); portable optical disc read-only memory (CD-ROM); optical storage devices; magnetic storage devices; or any suitable combination of the foregoing.

[0259] Instructions can form applications that are tangibly implemented on processor-readable media.

[0260] For example, instructions can be found in hardware, firmware, software, or a combination thereof. Instructions can be found, for example, in an operating system, a standalone application, or a combination of both. Therefore, a processor can be characterized as, for example, a device configured to execute a process and a device including a processor-readable medium (such as a storage device) having instructions for executing the process. Additionally, in addition to or instead of instructions, the processor-readable medium can store data values ​​generated by the implementation.

[0261] The device can be implemented, for example, in appropriate hardware, software, and firmware. Examples of such devices include personal computers, laptops, smartphones, tablets, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, head-mounted display devices (HMDs, see-through glasses), projectors, "caves" (systems comprising multiple displays), servers, video encoders, video decoders, post-processors that process the output from the video decoder, pre-processors that provide input to the video encoder, web servers, set-top boxes, and any other devices or communication equipment used to process video images. It should be clear that the equipment can be mobile and even mounted in mobile vehicles.

[0262] The computer software may be implemented by the processor 610, by hardware, or by a combination of hardware and software. As a non-limiting example, exemplary embodiments may also be implemented by one or more integrated circuits. The memory 620 may be of any type suitable for the technical environment and may be implemented using any suitable data storage technology, such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory, as non-limiting examples. The processor 610 may be of any type suitable for the technical environment and may encompass one or more of microprocessors, general-purpose computers, special-purpose computers, and processors based on multi-core architectures, as non-limiting examples.

[0263] As will be apparent to those skilled in the art, implementations can generate various signals formatted to carry information, such as information that can be stored or transmitted. The information may include, for example, instructions for performing a method or data generated by one of the described implementations. For example, the signal may be formatted to carry a bit stream of the described exemplary embodiments. Such a signal may be formatted as, for example, electromagnetic waves (e.g., using the radio frequency portion of the spectrum) or baseband signals. Formatting may include, for example, encoding the data stream and modulating a carrier wave with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, the signal can be transmitted via various wired or wireless links. The signal may be stored on a processor-readable medium.

[0264] The terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. As used herein, the singular forms “an,” “a,” and “the” may also be intended to include the plural forms unless the context clearly indicates otherwise. It will be further understood that, when used in this specification, the terms “include / comprise” and / or “including / comprising” may specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. Furthermore, when an element is referred to as “in response to,” “connected to,” or “associated with,” another element, it may be directly responsive to, connected to, or associated with another element, or there may be intermediate elements. In contrast, when an element is referred to as “directly responsive to,” “directly connected to,” or “directly associated with,” another element, there are no intermediate elements.

[0265] It should be recognized that, for example, in the cases of “A / B,” “A and / or B,” and “at least one of A and B,” the use of any of the symbols / terms “ / ,” “and / or,” and “at least one of” can be intended to cover the selection of only the first listed option (A), or only the second listed option (B), or the selection of both options (A and B). As a further example, in the cases of “A, B, and / or C” and “at least one of A, B, and C,” such wording is intended to cover the selection of only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or the selection of all three options (A, B, and C). As will be apparent to those skilled in the art and related fields, this can be extended to as many items as are listed.

[0266] Various numerical values ​​may be used in this application. Specific values ​​may be used for illustrative purposes and the aspects described are not limited to these specific values.

[0267] It will be understood that while the terms first, second, etc., may be used herein to describe various elements, these elements are not limited by these terms. These terms are used only to distinguish one element from another. For example, without departing from the teachings of this application, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element. There is no implied order between the first element and the second element.

[0268] References to “an exemplary embodiment” or “an exemplary embodiment” or “an implementation” or “implementation” and other variations thereof are frequently used to convey that a particular feature, structure, characteristic, etc. (described in conjunction with an exemplary embodiment / implementation) is included in at least one exemplary embodiment / implementation. Therefore, the phrases “in an exemplary embodiment” or “in an exemplary embodiment” or “in one implementation” or “in one implementation” appearing throughout this application, as well as any other variations, do not necessarily refer to the same exemplary embodiment.

[0269] Similarly, the references to "according to an exemplary embodiment / example / implementation" or "in an exemplary embodiment / example / implementation" and their variations are frequently used to convey that a particular feature, structure, or characteristic (described in conjunction with an exemplary embodiment / example / implementation) may be included in at least one exemplary embodiment / example / implementation. Therefore, the expressions "according to an exemplary embodiment / example / implementation" or "in an exemplary embodiment / example / implementation" appearing throughout this application do not necessarily refer to the same exemplary embodiment / example / implementation, nor are individual or alternative exemplary embodiments / examples / implementations necessarily mutually exclusive with other exemplary embodiments / examples / implementations.

[0270] The reference numerals appearing in the claims are for illustrative purposes only and do not limit the scope of the claims. Although not explicitly described, these exemplary embodiments / examples and variations may be employed in any combination or subcombination.

[0271] When a diagram is presented as a flowchart, it should be understood that it also provides a block diagram of the corresponding apparatus. Similarly, when a diagram is presented as a block diagram, it should be understood that it also provides a flowchart of the corresponding method / process.

[0272] While some diagrams include arrows along the communication path to indicate the main direction of communication, it should be understood that communication can occur in the opposite direction to the arrows depicted.

[0273] Various implementations involve decoding. As used herein, "decoding" can encompass all or part of a process performed, for example, on a received video frame (which may include a received bitstream encoded with one or more video frames) to produce a final output suitable for display or further processing in a reconstructed video domain. In various exemplary embodiments, such a process includes one or more processes typically performed by a decoder. In various exemplary embodiments, such a process, for example, also includes, or optionally includes, a process performed by a decoder of the various embodiments described herein.

[0274] As a further example, in one exemplary embodiment, "decoding" may refer only to dequantization; in another exemplary embodiment, "decoding" may refer to entropy decoding; in yet another exemplary embodiment, "decoding" may refer only to differential decoding; and in yet another exemplary embodiment, "decoding" may refer to a combination of dequantization, entropy decoding, and differential decoding. It will be clear, and believed to be well understood by those skilled in the art, whether the phrase "decoding process" is intended to specifically refer to a subset of operations or generally to a broader decoding process, depending on the context of the specific description.

[0275] Various implementations involve encoding. In a manner similar to the above discussion of "decoding," the term "encoding" as used herein can encompass all or part of a process performed on an input video frame to generate an output bitstream. In various exemplary embodiments, such a process includes one or more processes typically performed by an encoder. In various exemplary embodiments, such a process also includes, or optionally includes, a process performed by an encoder of the various embodiments described herein.

[0276] As a further example, in one exemplary embodiment, "encoding" may refer only to quantization; in another exemplary embodiment, "encoding" may refer only to entropy encoding; in yet another exemplary embodiment, "encoding" may refer only to differential encoding; and in still another exemplary embodiment, "encoding" may refer to a combination of quantization, differential encoding, and entropy encoding. It will be clear, and believed to be well understood, by those skilled in the art, whether the phrase "encoding process" is intended to specifically refer to a subset of operations or generally to a broader encoding process, depending on the context of the particular description.

[0277] Furthermore, this application may refer to "obtaining" various types of information. Obtaining information may include one or more of the following: estimated information, calculated information, predicted information, or information retrieved from memory, processed information, moved information, copied information, erased information, calculated information, determined information, predicted information, or estimated information.

[0278] Furthermore, this application may refer to "receiving" various messages. Receiving messages may include one or more of the following, such as access information or receiving information from a communication network.

[0279] Moreover, as used herein, the word "signal" specifically refers to instructing the corresponding decoder to do something. For example, in some exemplary embodiments, the encoder signals specific information, such as encoding / decoding parameters or encoded video frame data. In this way, in exemplary embodiments, the same parameter can be used on both the encoder and decoder sides. Thus, for example, the encoder can transmit (explicitly signal) a specific parameter to the decoder so that the decoder can use the same specific parameter. Conversely, if the decoder already has the specific parameter as well as other parameters, signaling can be used without transmission (implicitly signal) to simply allow the decoder to know and select the specific parameter. Bit savings are achieved in various exemplary embodiments by avoiding the transmission of any actual functionality. It should be recognized that signaling can be accomplished in various ways. For example, in various exemplary embodiments, one or more syntax elements, flags, etc., are used to signal information to the corresponding decoder. Although the verb form of the word "signal" has been referred to above, the word "signal" can also be used as a noun herein.

[0280] Several implementations have been described. However, it should be understood that various modifications can be made. For example, elements of different implementations can be combined, supplemented, modified, or removed to produce other implementations. Furthermore, those skilled in the art will understand that other structures and processes can replace the disclosed structures and processes, and the resulting implementations will perform at least substantially the same functions in at least substantially the same manner to achieve at least substantially the same results as the disclosed implementations. Therefore, these and other implementations are contemplated in this application.

Claims

1. A method (1500) for determining an intra-frame prediction mode, the intra-frame prediction mode being denoted as a TIMD merging mode, the method comprising: - Obtain (1510) a merging list of at least one TIMD merging mode candidate, each TIMD merging mode candidate including at least one internal prediction mode and association information for intra-frame prediction; and - Select (1520) TIMD merge mode from the merge list of TIMD merge mode candidates as a candidate for intra-frame prediction of the current block.

2. The method according to claim 1, wherein the prediction cost of the TIMD merge pattern is less than the prediction cost of other TIMD merge pattern candidates in the merge list.

3. The method of claim 1, wherein the TIMD merge mode is determined by: - Obtain the prediction cost on the template for each TIMD merge pattern candidate pattern in the merge list described in (1521); and - Select the (1522) TIMD merge mode based on the predicted cost obtained for the TIMD merge mode candidates.

4. The method of claim 3, wherein the selection of TIMD merging is based on rate-distortion optimization, in which the rate-distortion cost of intra-prediction for the current block is evaluated based on the obtained prediction cost, and each intra-prediction is derived based on a TIMD merging mode candidate in the merging list.

5. The method of claim 3 or 4, wherein a signal is sent in the bitstream to indicate the index of the selected TIMD merging mode candidate.

6. The method according to any one of claims 1 to 5, further comprising encoding (1530) information in the bitstream indicating whether a TIMD merging mode is enabled as a candidate for intra-frame prediction of the current block of the video frame.

7. The method according to any one of claims 1 to 6, wherein the at least one TIMD merge mode candidate in the merge list has been previously used for intra-frame prediction of adjacent and / or non-adjacent spatial neighbor blocks of the current block.

8. The method according to any one of claims 1 to 7, wherein the at least one TIMD mode candidate in the merge list has previously been used for intra-frame prediction of blocks in a reference frame.

9. The method of claim 8, wherein the block in the reference frame is a co-located block of the current block or an extended block of the co-located block in the reference frame, or the block in the reference frame is a block in the reference frame indicated by motion vectors from adjacent and / or non-adjacent spatial neighboring blocks of the current block.

10. The method according to any one of claims 1 to 9, wherein the at least one TIMD merge mode candidate in the merge list includes at least one template-based prediction mode and associated information for intra-frame prediction of previously encoded or decoded blocks.

11. The method according to any one of claims 1 to 10, wherein the at least one TIMD merging mode candidate in the merging list includes at least one decoder-side intra-mode derivation mode (DIMD) and associated information for intra-frame prediction of previously encoded or decoded blocks, and / or the at least one TIMD merging mode candidate in the merging list includes at least one TIMD merging mode and associated information for intra-frame prediction of previously encoded or decoded blocks.

12. The method according to any one of claims 5 to 11, wherein the index indicating the selected TIMD merging mode is signaled in the bitstream depends on the availability of at least one block previously encoded or decoded based on the following mode: Template-based prediction mode, decoder-side intra-frame mode derivation mode, or TIMD merging mode.

13. The method according to any one of claims 1 to 12, wherein a merge list is obtained for each channel of the video frame, and the TIMD merge mode candidates in the merge list obtained for the first channel belong to the merge list obtained for the second channel of the video frame.

14. The method of claim 13, wherein the TIMD merging mode determined as a candidate for intra-frame prediction of the current block of the first channel of the video frame is inherited by the co-bit block in the second channel of the video frame.

15. The method according to any one of claims 1 to 14, wherein the information associated with the TIMD merge mode candidate is at least one of the following: - TIMD mode is preferred; - TIMD fusion mode for mixing with preferred TIMD modes; - The fusion weight of preferred mode and fusion mode; - Template cost for hybrid flag indicators, preferred mode, and / or fusion mode; - Transformation type; - Wide-angle conditions for preferred mode and fusion mode.

16. A method for encoding video image data, comprising the method according to any one of claims 1 to 15.

17. A method for decoding video image data, comprising the method according to any one of claims 1 to 15.

18. An apparatus comprising components for performing one of the methods as claimed in any one of claims 1 to 17.

19. A computer program product comprising instructions which, when executed by one or more processors, cause the one or more processors to perform the method as described in any one of claims 1 to 17.

20. A non-transitory storage medium carrying program code instructions for performing the method as described in any one of claims 1 to 17.

Images