Method and apparatus for region-adaptive mixing of DIMD, and encoder / decoder including the thereof.

Abstract

A method of describing a respective sample-derived decoder-side intra mode derivation (DIMD) predictor for a coding unit (CU) of a picture is described. The method includes selecting one or more intra prediction modes (IPMs) in a template block neighboring the CU, determining blending weights for blending at least the one or more selected IPMs, and generating the DIMD predictor by blending the one or more selected IPMs using the blending weights determined for the CU such that the blending varies over the CU.

Description

[0001] Cross-reference to related applications

[0002] This application is based on and claims priority to European Patent Application No. 22167216.5, filed on 7 April 2022, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This invention generally relates to the field of encoding / decoding pictures, images, or videos, and embodiments of the invention relate to improvements regarding intra-frame prediction, and more specifically, to improvements regarding decoder-side intra-frame mode derivation (DIMD) processing. More specific embodiments of the invention relate to region-adaptive DIMD mixing. Background Technology

[0004] The encoding and decoding of pictures, images, or videos are performed according to certain standards, such as the Advanced Video Codec (AVC) standard (see reference [1]), the High Efficiency Video Codec (HEVC) standard (see reference [2]), or the Universal Video Codec (VVC) standard (see reference [3]).

[0005] Figure 1 The diagram illustrates a block diagram of a standard video compression system 100 operating according to the VVC standard. The standard video codec 100 compresses and encodes images 102 of a video sequence. The images 102 to be encoded are partitioned into blocks 104, also referred to as codec units (CUs). The encoder 100 includes a precoding filter 106 and a prediction residual signal former 108, which generates a prediction residual signal 110 to measure the deviation between the prediction signal 112 and the signal 114 output by the filter 106. The encoder also includes a transformer 116, a quantizer 118, and uses an entropy codec 122 to provide an output bitstream or data stream 120. Additionally, encoder 100 includes a prediction stage 124 for determining prediction signal 112. Prediction stage 124 includes a dequantizer or inverse quantizer 126, an inverse transformer 128, a combiner 130, an in-loop filter 134, a motion estimator 138, an intra-frame / inter-frame mode selector 140, an inter-frame predictor 142, and an intra-frame predictor 144.

[0006] For reference Figure 1The video codec 100 described compresses and encodes images 102 of a video sequence as follows. The images 102 to be encoded are partitioned into blocks or CUs 104. Each codec unit 104 is encoded using an intra-frame or inter-frame codec mode. When a CU is encoded in intra-frame mode, intra-frame prediction is performed by an intra-frame predictor 144. Intra-frame prediction involves predicting the current CU 114 being encoded using encoded, decoded, and reconstructed image samples located around the current CU (e.g., above and to the left of the current CU). Intra-frame prediction is performed in the spatial domain. In inter-frame mode, motion compensation and motion estimation are performed by a motion estimator 138, which searches for a CU that is a good predictor of the current CU among one or more reference images provided by an image buffer 136 and used for predictively encoding and decoding the current image. For example, a good predictor of the current CU is a predictor similar to the current CU, i.e., the distortion between the two CUs is low or below a certain threshold. Motion estimation may also consider signaling the predictor to optimize the rate-cost rate-distortion tradeoff. The output of the motion estimation step is one or more motion vectors and reference indices associated with the current CU. Motion compensation then predicts the current CU using one or more motion vectors and reference picture indices determined by the motion estimator 138. Essentially, the block or CU contained in the selected reference picture and pointed to by the determined motion vector is used as the prediction block for the current CU. Encoder 100 selects one of the intra-frame or inter-frame coding modes for encoding the CU using selector 140 and indicates the intra / inter-frame decision, for example, using a prediction mode flag. The prediction residual 110 is then transformed and quantized by blocks 116 and 118, and the quantized transform coefficients, along with the motion vectors and other syntax elements, are entropy-encoded and written to the output bitstream 120. Encoder 100 may skip the transform stage 116 and directly apply quantization to the untransformed residual signal 110 in a so-called transform-skip coding mode. After encoding a block or CU, the encoder decodes and reconstructs the CU to obtain a reconstructed signal 132, which can be used as reference data for predicting future CUs or blocks to be encoded. The quantized transform coefficients 110' are dequantized and inversely transformed, resulting in a decoded predicted CU or block 110', and the decoded prediction residual and the predicted block are then combined (typically added) at 130 to provide the reconstructed block or CU 132. An in-loop filter 134 is applied to the reconstructed picture to reduce compensation artifacts. For example, a deblocking filter, a sample adaptive offset (SAO) filter, and an adaptive loop filter (ALF) can be applied to reduce coding artifacts. The filtered picture is stored in a buffer 136 (also called a decoded picture buffer (DPB)) so that it can be used as a reference picture for encoding and decoding subsequent pictures.

[0007] Figure 2This is a block diagram of a video decoder 150 used for predictively decoding images or video provided at the output 154 of decoder 150 from data stream 152. Decoder 150 includes an entropy decoder 156, a partition block 158, an inverse quantizer 160, an inverse transformer 162, a combiner 164, an in-loop filter 166, an optional post-decoding processor 168, and a prediction module 170. Prediction module 170 includes a decoded image buffer 180, a motion compensator 182, and an intra-frame predictor 184.

[0008] The decoder 150 decompresses and decodes the encoded images of the video sequence as follows. The decoder 156 performs entropy decoding on the input bitstream 152, providing information such as block partitioning, the encoding / decoding mode for each codec unit, the transform coefficients contained in each transform block, prediction information (e.g., intra-frame prediction mode), motion vectors, a reference image index, and other encoding / decoding information. The block partitioning information indicates how the image is partitioned, and the decoder 150 can divide the input image into codec tree units (CTUs) (typically 64×64 or 128×128 pixels in size) and divide each CTU into rectangular or square codec units (CUs) based on the decoded partitioning information. The quantized coefficients 172 of the entropy decoding are dequantized 160 and inverse transformed 162 to obtain the decoded residual image or CU 174. The decoded prediction parameters are used to predict the current block or CU, i.e., whether the predicted block is obtained through its intra-frame prediction or its motion-compensated temporal prediction. The prediction processing performed on the decoder side is the same as the prediction processing performed on the encoder side. The decoded residual block 174 is added to the predicted block 176, resulting in the reconstructed current image block 164. An in-loop filter 166 is applied to the reconstructed picture or image, which is also stored in the decoded picture buffer 180 for use with a reference picture in future picture decoding. As mentioned above, the decoded picture can also undergo post-decoding processing, such as performing inverse color transformations, for example, from YCbCr4:2:0 to RGB 4:4:4.

[0009] As mentioned above, the intra-prediction mode to be used for decoding can be carried in the bitstream provided by the encoder and received by the decoder. However, according to other methods, instead of introducing the actual intra-prediction mode into the bitstream, it can be derived by using gradient analysis of the reconstructed pixels of neighboring currently processed CUs. In other words, the intra-prediction mode is not explicitly indicated in the bitstream, but is implicit. This method is called decoder-side intra-mode derivation (DIMD), which can be signaled using a simple flag and then the actual intra-prediction mode is derived during the reconstruction process, such as the reconstruction process performed by the prediction blocks 124 / 170 of encoder 100 or decoder 150. Encoder 100 can encode information about whether the DIMD is used for the current CU into bitstream 120, and in the absence of DIMD, the actual intra-prediction mode is signaled by the encoder and parsed from the bitstream by the decoder.

[0010] Taken from reference [5] Figure 3 An example of an intra-block general reconstruction process is illustrated, which can be applied at encoder 100 or decoder 150. During parsing process 200, intra-block decoding is determined, as indicated at block 202, and pixel residual parsing 204 and DIMD flag parsing 206 are performed. If the DIMD flag 206 is zero, then intra-prediction mode parsing 208 is performed to obtain intra-mode 210 to be used when predicting and reconstructing pixels during reconstruction process 212, as indicated at 216. If the DIMD flag 206 is set to one, then intra-prediction derivation 218 is performed during reconstruction process 212, which produces a derived intra-mode 220, which is used together with the pixel residual 214 at 216 for reconstruction processing.

[0011] Further details of the DIMD method are described in references [5] through [8]. Generally, DIMD processing is based on a reconstructed template area adjacent to the currently processed CU, the template area being three sample widths in both width and height. The template area comprises a left-side area, an upper-side area, and a top-left-side area. These areas are also referred to below as the left-side template area region, the upper-side template area region, and the top-left-side template area region. Within the template area, appropriate edge detection filters (such as 3×3 horizontal and vertical Sobel filters) are applied to determine the amplitude and angle of the brightness direction or orientation of each midline sample in the template area. A histogram of gradients (HoG) is calculated, and each entry corresponds to a regular intra-angle pattern, and the accumulated intensity is stored.

[0012] angle = arctan(G) hor / G ver )

[0013] amplitude=|Ghor |+|G ver |

[0014] Among them G hor and G ver These are the intensities in the pure horizontal and vertical directions calculated by the Sobel filter. (Taken from reference [8]) Figure 4 The diagram illustrates the DIMD template block or region used to calculate HoG. Figure 4 (a) The image 250 is partially reconstructed, as indicated by reconstructed block 252 and unusable block 254. In other words, reconstructed block 252 includes the already reconstructed codec units or blocks of the image to be reconstructed, while the corresponding codec units or blocks in unusable block 254 still need to be reconstructed. The currently processed CU or block 256 and the aforementioned template block 258 are also illustrated. Figure 4 (b) The illustration shows that template block 258 is 3 samples or pixels wide and high, and as described above, HoG 260 is calculated using 3×3 horizontal and vertical Sobel filters 259 to obtain the intra-angle prediction mode (also known as intra-frame prediction mode, IPM) based on the amplitude and angle of the brightness direction of each midline sample 262 of template block 258, and to obtain the associated cumulative intensity or amplitude for each IPM in the template block. Two of the most representative IPMs are selected from HoG 260 (in... Figure 4 (b) indicates M1 and M2), and the two most representative IPMs are combined with the Planar mode using weights for M1 and M2 derived from the ratio of their cumulative amplitudes and a fixed weight for the Planar mode (e.g., 1 / 3, i.e., 21 / 64 with 6-bit integer precision).

[0015] Taken from reference [8] Figure 5 The illustration shows prediction fusion through weighted averaging of M1, M2, and Planar modes. Assuming a square CU 256, the corresponding pixels of CU 256 are predicted by the two most representative IPMs M1 and M2 taken from HoG 260, resulting in predictor blocks Pred1, Pred2, and Pred3. The corresponding predictor blocks are weighted by weights w1, w2, and w3, and fused or combined into a final predictor block comprising DMID predictors for the corresponding samples or pixels in CU 256.

[0016] Figure 6The illustration shows an example of the typical DIMD predictor derivation process. At S100, using input CU 256, the neighborhood of input CU 256 is analyzed to determine the number of available upper and left inner samples. At S102, HoG 260 is constructed for template block 258; that is, based on the available samples, the angular prediction mode IPM is determined by its cumulative amplitude in HoG 260 (step S104). At S106, it is determined how many IPMs are selected, i.e., whether no IPMs are selected, whether one IPM is selected, or whether two IPMs are selected. If no IPMs are selected, then only the Planar mode (which can be the default mode) is applied, as indicated at S108, to obtain the DIMD predictor. If only one IPM is selected, then the selected IPM is used to obtain the DIMD predictor, as indicated at S110. If two IPMs are selected, then at S112, the mixing weight of each selected mode is determined according to its cumulative amplitude ratio, and the weight used for the Planar mode is set to 1 / 3. At S114, predictor blocks Pred1 to Pred3 are calculated for each selected IPM and Planar mode, and at S116, the DIMD predictor is obtained by mixing the two selected IPM and Planar modes using determined mixing weights.

[0017] Figure 7 The illustration shows an example of the currently processed CU 256 and an associated template block 258 with a width / height of T = 3 samples or pixels. For each median sample 262, the corresponding orientation is illustrated by the direction of the corresponding arrow, and the corresponding gradient intensity is indicated by the length of the arrow. HoG is generated by accumulating the intensity of each orientation / IPM entry, and the two most important modes in the HoG (if they exist) are selected as the modes to be mixed with the Planar mode or the planar intra-frame prediction mode. Figure 7 In the example, IPMs 5 and 28 have the first and second highest cumulative intensity / amplitude. As mentioned above, these modes are weighted, and the mixing weights are generally fixed and / or determined according to the ratio of the cumulative intensity / amplitude of the selected IPMs. Although different mixing modes have been proposed in the prior art, all mixing modes are global with respect to the currently processed block or CU. More specifically, when considering the conventional approach and when two IPMs (such as M1 and M2) are selected, the mixing weights can be calculated as follows.

[0018]

[0019] w1 is associated with IPM M1, w2 with IPM M2, and w3 with the Planar mode. Different modifications to the mixing process are described in the prior art. For example, reference

[10] suggests selecting multiple mixing modes based on rate distortion optimization, reference

[11] suggests implicitly deriving two of the three mixing modes using HoG, reference

[13] suggests implicitly deriving the mixing modes using template matching, and reference

[20] suggests selecting different mixing modes based on the determined multiple DIMD modes. When applying DIMD, one or two IPMs are derived from the reconstructed neighboring samples (i.e., from the template block), and if two intra-frame modes are derived, they are combined (i.e., mixed) with the Planar mode predictor. Conventional solutions always achieve a global or CU-level weighted mixing of the two IPMs with the Planar mode.

[0020] In general, in the reconstruction portion 252 of image 250, samples or pixels of the three columns and rows are used for DIMD processing, i.e., HoG 260 is computed using a filter with a fixed filter window of 3×3 samples located at the same position in template block 258. In reference [6], when computing gradients on pixels located in the left column, top row, or top left sample that are directly adjacent to the current CU, a 3×2 filter can be used instead of the 3×3 Sobel filter 259. In reference [8], instead of using the 3×3 Sobel filter 259 on all pixels or samples in the middle line of template block 258, the 3×3 Sobel filter 259 can be applied more sparsely, for example, only at one middle line sample in the left template block and one middle line sample in the upper template block. Further aspects of DIMD processing are described, for example, in the following references:

[0021] References

[10] ,

[11] ,

[13] and

[20] discuss the mixing of patterns.

[0022] References

[16] and

[18] discuss behavior correction (error repair) related to the original intention.

[0023] -References [9],

[12] and

[14] discuss combinations with other modes,

[0024] - References

[15] ,

[17] ,

[19] or

[21] discuss the syntax or simplifications used for DIMD processing.

[0025] However, the above reference Figure 6 The basic DIMD process described in reference [8] remains the same.

[0026] The conventional hybrid processing used to obtain the final intra-prediction block has drawbacks because it uses weights that are globally applicable to the entire block or CU being processed. Therefore, the weights do not consider local characteristics of the IPM, such as the orientation and intensity of the IPM in the reconstructed template block, leading to a less faithful intra-predictor. Another drawback is that if only one IPM is selected, the predictor is not combined with the Planar or DC mode predictor; that is, it is the same as the conventional intra-prediction mode. However, this mode is encoded as DIMD mode, thus generating additional computational overhead compared to the conventional intra-prediction mode because it requires computation of the HoG, which may ultimately result in only a single IPM to be used.

[0027] Therefore, further improvements to DIMD processing are needed. Summary of the Invention

[0028] This invention provides a method for deriving a decoder-side intra-frame mode derivation (DIMD) predictor from corresponding samples of a codec unit (CU) for an image, the method comprising:

[0029] Select one or more intra-prediction modes (IPMs) in the template block adjacent to the CU.

[0030] Determine the mixing weights for mixing at least one or more of the selected IPMs, and

[0031] A DIMD predictor is generated by mixing one or more selected IPMs using mixing weights determined for the CU, such that the mixing varies on the CU.

[0032] Optionally, the method includes:

[0033] Split the CU into two or more CU regions.

[0034] Wherein, for each CU region in at least a subset of two or more CU regions, the blending weight is determined based on the presence of one or more selected IPMs in the portion of the template block adjacent to the corresponding CU region, and

[0035] The DIMD predictor is generated using a hybrid weight determined for each CU region.

[0036] Optionally, one or more IPMs can be selected using globally determined IPM statistics (such as a histogram of gradients, HoG) over the entire template block, which comprises multiple template block regions, where

[0037] - Perform a global selection on the entire CU, or

[0038] - This selection involves a global selection of the entire set of IPMs for the entire CU, followed by a further region-based selection of the set of IPMs for each CU region.

[0039] Optionally, one or more IPMs can be selected using IPM statistics (such as histograms of gradients, HoG), which are determined individually in the following sections.

[0040] - Each of multiple template block areas within a template block, including the left template block area and the top template block area, or

[0041] - Each of the multiple partial template block regions, where a partial template block region is a template block region that is adjacent to only one of the CU regions, and

[0042] The selection involves making a region-based selection from the IPMs that are determined individually for each CU region.

[0043] Optionally, determining the mixed weights includes

[0044] - For a CU region located adjacent to a template block, the blending weight is determined based on the presence of one or more selected IPMs in one or more template block regions or portions of the template block region adjacent to the CU region, and

[0045] -For CU regions that are positioned not adjacent to template blocks

[0046] ○ Select only the Planar mode as the DIMD predictor, or

[0047] ○ Select only DC mode as the DIMD predictor, or

[0048] ○ Determine the blending weights using one or more of the selected IPMs from one or more or all template block regions, or

[0049] ○ The mixing weight is determined by weighting the mixing weights of the CU regions adjacent to the template block region.

[0050] Optionally,

[0051] - Select one or more IPMs in a manner that recognizes the subdivision of the template block into multiple template block regions, including the left template block region and the top template block region.

[0052] - The template block also includes a top-left template block area, which is allocated

[0053] ○ Go to the template block area above, or

[0054] ○ Go to the left template block area, or

[0055] ○ Go to either the upper template block area or the left template block area, or

[0056] ○ As an additional template block area, and / or

[0057] - Select one or more IPMs in a way that recognizes the subdivision of template blocks into multiple partial template block regions.

[0058] - When defining two or more CU regions, define a partial template block region by splitting the template block with the same vertical and / or horizontal lines as the CU.

[0059] Optionally, one or more first IPMs are selected in the template block area of ​​the template block, and one or more second IPMs are selected in a portion of the template block area of ​​the template block, wherein...

[0060] - If the CU has a rectangular shape, then select the first IPM in the template block area adjacent to the longer dimension of the CU, and select the second IPM in the template block area adjacent to the shorter dimension of the CU, or

[0061] - The selection of the first IPM and the second IPM within the template block region or from a portion of the template block region adjacent to the CU depends on the orientation of the peak IPM within the template block region or throughout the entire template block, for example,

[0062] ○ If the principal direction of the peak IPM calculated in the template block area or throughout the entire template block is more vertical than the horizontal, then a first IPM is selected in the left or upper template block area, and optionally in the upper left template block area adjacent to the CU area, and a second IPM is selected in the template block area above or to the left of the portion adjacent to the CU area.

[0063] ○ If the main direction of the peak IPM calculated in the template block area or the entire template is more horizontal than vertical, then select the first IPM in the upper or left template block area and optionally in the upper left template block area adjacent to the CU area, and select the second IPM in the left or upper template block area adjacent to the CU area.

[0064] Optionally,

[0065] If two or more IPMs are selected, then at least some, but not all, of the IPMs are selected from the entire template block, while the remaining IPMs are selected either from within the template block area or from a portion of the template block area adjacent to the CU area.

[0066] If two IPMs are selected, then the first IPM in the entire template block is selected and

[0067] -If there is no second IPM from a template block region or a portion of a template block region adjacent to the CU region, then

[0068] ○ The first IPM is used as a DIMD predictor, or

[0069] ○ Use predefined weights to mix the first IPM with the Planar or DC pattern to obtain the DIMD predictor.

[0070] -If a second IPM exists in the template block region or in a portion of the template block region adjacent to the CU region, then

[0071] If the second IPM is different from the first IPM, then the first IPM and the second IPM are mixed to obtain the DIMD predictor.

[0072] If the second IPM is equal to the first IPM, then select a further IPM in the template block region adjacent to the CU region, and mix the first IPM with the further IPM to obtain the DIMD predictor.

[0073] Optionally, for CU regions that are positioned not adjacent to any template block regions,

[0074] - Use only the first IPM as the DIMD predictor, or

[0075] - Use only Planar mode as the DIMD predictor, or

[0076] - Use only DC mode as a DIMD predictor, or

[0077] - Use predefined weights to mix the first IPM with the Planar or DC pattern to obtain the DIMD predictor.

[0078] Optionally,

[0079] If more than one IPM is selected, then

[0080] - If the IPM exists in a template block region adjacent to the CU region, then use the blending weight associated with the IPM.

[0081] - If only one of the IPMs exists in the template block region adjacent to the CU region, then use the blending weight associated with that IPM or the weighted blending weight associated with that IPM.

[0082] And set the blending weights associated with any other IPM to a predefined value, such as 0.

[0083] - Optionally, if an IPM does not exist in an adjacent template block region of the CU region, then the blending weight associated with the IPM is set to a predefined value, such as 0, and

[0084] - Optionally, the blending weights associated with the Planar or DC mode can be set to predefined values, or

[0085] If you choose an IPM, then

[0086] - If the IPM exists in a template block region adjacent to the CU region, then use the blending weight associated with that IPM or the weighted blending weight associated with that IPM, and set the blending weight associated with any other IPM to a predefined value, such as 0.

[0087] - Optionally, if the IPM does not exist in a template block region adjacent to the CU region, then the blending weight associated with the IPM and the blending weight associated with any other IPM are set to predefined values, such as 0.

[0088] - Optionally, the blending weights associated with the Planar or DC mode can be set to predefined values, or

[0089] If no IPM is selected, the blending weights associated with any other IPM are set to predefined values, such as 0, and optionally, the blending weights associated with Planar or DC modes are set to predefined values.

[0090] Optionally,

[0091] - For a CU region located adjacent to a first template block region for which no IPM has been selected and far from a second template block region for which one or more IPMs have been selected, the blending weight is the weighted blending weight of the CU regions adjacent to the second template block region, or

[0092] - For a CU region that is located adjacent to the first template block region for which the first IPM is selected and far from the second template block region for which the second IPM is selected, the mixing weight for the second IPM is the weighted mixing weight for the second IPM of the CU region adjacent to the second template block region.

[0093] The present invention provides a non-transitory computer-readable storage medium including instructions that, when executed by a computer, cause the computer to perform the method of the present invention.

[0094] This invention provides an apparatus for a corresponding sample-derived decoder-side intra-frame mode derivation (DIMD) predictor for a codec unit (CU) of an image, the apparatus comprising:

[0095] The intra-prediction mode (IPM) selection module is configured to select one or more IPMs from the template blocks adjacent to the CU.

[0096] The blending weight determination module is configured to determine blending weights for blending at least one or more selected IPMs, and

[0097] The DIMD predictor generation module is configured to generate a DIMD predictor by mixing one or more selected IPMs using mixing weights determined for the CU such that the mixing varies on the CU.

[0098] This invention provides an apparatus for decoding an encoded data stream, the encoded data stream including data representing an image, the apparatus comprising:

[0099] The decoder module is configured to decode images from the encoded data stream, and

[0100] The prediction module includes means of the present invention for deriving a decoder-side intra-frame mode derivation (DIMD) predictor for a corresponding sample of a codec unit (CU) for a picture.

[0101] This invention provides an apparatus for encoding an image into an encoded data stream, the encoded data stream including data representing the image, the apparatus comprising:

[0102] The encoder module is configured to receive the raw image and encode it into an encoded data stream.

[0103] The prediction module includes means of the present invention for deriving a decoder-side intra-frame mode derivation (DIMD) predictor for a corresponding sample of a codec unit (CU) for a picture.

[0104] The technical solutions provided by the embodiments of the present invention have the following beneficial effects.

[0105] In existing methods, the mixing weights are selected for the entire CU being processed or predicted; that is, conventional methods focus on a global or CU-level approach when determining the mixing weights for a selected IPM. However, the signal characteristics of an IPM can vary across template blocks, so the selected weights may be more suitable for obtaining a DIMD predictor for samples in one CU region than for obtaining a DIMD predictor for samples in another CU region. In other words, the weights may not consider the local characteristics of the IPM, such as the orientation and intensity of the IPM in the reconstructed template block, leading to a less faithful intra-predictor.

[0106] This invention addresses the aforementioned drawbacks by instead using a fixed blending method on the CU, and instead providing a method for blending one or more selected IPMs with blending weights determined for the CU, in a manner that varies across the CU. Consequently, the blending process generates an intra-predictor that more faithfully fits the local orientation of the CU to be predicted, thereby improving prediction accuracy.

[0107] According to embodiments of the invention, region-based mixing weights are selected based on the adjacency of the region of the currently processed CU with template blocks (such as corresponding template blocks or template block regions). The overall template block is used to determine the IPM from IPM statistics (such as HoG), and a region-based weight map can be generated for the CU to be predicted as a result of the processing according to embodiments of the invention. This map can be used during mixing processing to generate an intra-frame predictor that more faithfully fits the local orientation of the CU to be predicted, thereby improving prediction accuracy. The embodiments provide improved prediction accuracy by fully utilizing existing HoG techniques, and therefore do not require substantial changes to existing DIMD processing except for a few additional computational operations used only to implement the selection of region-based mixing weights. Therefore, the method of the invention can be readily incorporated into existing DIMD processing.

[0108] A further drawback of conventional methods is that if only one IPM is selected, the predictor is not combined with the Planar or DC mode predictor; that is, it is the same as the conventional intra-prediction mode. However, this mode is encoded and decoded as a DIMD mode, thus generating additional overhead compared to the conventional intra-prediction mode because HoG computation is required, which may ultimately result in only a single IPM to be used. The present invention is advantageous because, according to embodiments, when a single IPM is derived by combining it with the Planar or DC mode using region-specific mixing weights determined for a selected IPM, the region-specific mixing weights are also used, thereby also utilizing the DIMD mode for this scenario.

[0109] It should be understood that the description in this section is not intended to identify key or essential features of embodiments of the invention, nor is it intended to limit the scope of the invention. Other features of the invention will become apparent from the following description. Attached Figure Description

[0110] The accompanying drawings are illustrative and intended to explain the invention, and should not be construed as limiting the invention to the illustrated embodiments.

[0111] Figure 1 This is a block diagram of a conventional video encoder;

[0112] Figure 2 This is a block diagram of a conventional video decoder;

[0113] Figure 3 The diagram illustrates what can be applied to Figure 1 encoder or Figure 2 An example of intra-block generic refactoring at the decoder;

[0114] Figure 4 The illustration shows a DIMD template block or region used to calculate the histogram (HoG) of gradients in DIMD processing;

[0115] Figure 5 The illustration shows the prediction fusion through a weighted average of the selected IPM and Planar patterns;

[0116] Figure 6 The illustration shows an example of a conventional DIMD predictor derivation process;

[0117] Figure 7 The illustration shows an example of the currently processed 256 and the associated template block with the width / height of a sample or pixel;

[0118] Figure 8 The figure illustrates a flowchart of a method for deriving a DIMD predictor for a corresponding sample of a CU according to an embodiment of the present invention;

[0119] Figure 9 The illustration shows an embodiment of the invention where the current CU is split into multiple CU regions associated with different mixing weights;

[0120] Figure 10 The illustration shows the DIMD predictor derivation process according to an embodiment of the present invention;

[0121] Figure 11 The illustration shows a schematic diagram of selecting a hybrid weight based on the adjacency of a certain CU region relative to a certain template block region according to an embodiment of the present invention;

[0122] Figure 12 The illustration shows a hybrid weighting distribution according to an embodiment of the present invention;

[0123] Figure 13 The illustration shows a flowchart of determining the general region mixing weight according to an embodiment of the present invention;

[0124] Figure 14 The illustration shows a hybrid weighting distribution according to an embodiment of the present invention;

[0125] Figure 15 The diagram illustrates the use of a hybrid weighting based on the distance between a CU region and a template block region that includes the selected IPM;

[0126] Figure 16 The diagram illustrates the calculation of each template block area in HoG;

[0127] Figure 17 The illustration shows an embodiment of selecting a first IPM based on the entire HoG and selecting a second IPM based on the HoG used for adjacent template blocks;

[0128] Figure 18 A flowchart illustrating an embodiment of the present invention is shown;

[0129] Figure 19 The illustration shows an example of using template blocks to compute a portion of the template block HoG to be used in deriving local blending weights;

[0130] Figure 20 The illustration shows an example of using a mixture of partial and full / entire template block regions to calculate HoG to select one or more IPMs;

[0131] Figure 21 The figure illustrates a block diagram of an apparatus for deriving a DIMD predictor according to an embodiment of the present invention;

[0132] Figure 22 A block diagram of a decoder according to an embodiment of the present invention is shown;

[0133] Figure 23 A block diagram of an encoder according to an embodiment of the present invention is illustrated; and

[0134] Figure 24 A block diagram of an electronic device 900 according to an embodiment of the present invention is shown. Detailed Implementation

[0135] The illustrative embodiments of the present invention will now be described with reference to the accompanying drawings, including various details of the embodiments of the invention to facilitate understanding and should be considered illustrative only. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of the invention. Furthermore, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0136] In this invention, the term "and / or" is intended to cover all possible combinations and sub-combinations of the listed elements, including any one, any sub-combination, or all of the elements listed individually, without necessarily excluding additional elements.

[0137] In this invention, the phrase “...or at least one of ...” is intended to cover any one or more of the listed elements, including any one of the individually listed elements, any sub-combination of elements, or all elements, without necessarily excluding any additional elements, and without necessarily requiring all elements.

[0138] In this invention, the term "encoder / decoder" refers to "encoding" or "decoding," as is apparent from the context of the described embodiments. Similarly, the term "encoder / decoder" refers to "encoder" or "decoder."

[0139] Figure 8 The illustration shows a flowchart of a method for deriving a decoder-side intra-frame mode derivation (DIMD) predictor for a corresponding sample of a codec unit (CU) of an image according to an embodiment of the present invention. The method includes the following steps:

[0140] S200: Select one or more intra-prediction modes (IPMs) in the template block adjacent to the CU.

[0141] S202: Determine the mixing weights for mixing at least one or more of the selected IPMs.

[0142] S204: A DIMD predictor is generated by mixing one or more selected IPMs using mixing weights determined for CU such that the mixing varies on CU.

[0143] According to embodiments, the mixing of one or more selected IPMs is achieved by selecting mixing weights per region (i.e., per individual region of a CU), such that the mixing varies across the CU. The CU is split into two or more CU regions, and mixing weights are determined for each CU region from at least a subset of the two or more CU regions based on the presence of one or more selected IPMs in a portion of a template block adjacent to the corresponding CU region. A DIMD predictor is then generated using the mixing weights determined for each CU region. In other words, embodiments of the invention provide a method for deriving a DIMD predictor for samples of the currently processed CU by mixing (one or more) IPM / Planar or DC patterns using mixing weights determined per CU region, rather than using weights determined only for the entire CU.

[0144] According to embodiments of the invention, the current processing control unit (CU) predicted using DIMD is split into rectangular or square CU regions. CU regions can have the same size and can be split horizontally and / or vertically. Therefore, according to embodiments, the corresponding CU regions can have rectangular or square shapes. It is important to note that a blending weight is selected and applied for each region. Regarding the splitting operation, it is important to note that the splitting is not related to CU partitioning, but only defines two or more regions within a CU, and assigns appropriate blending weights to such CU regions. The actual CU splitting operation can be performed before or after HoG construction.

[0145] Figure 9 The illustration shows an embodiment of the invention in which the current CU 256 is split into multiple CU regions associated with different mixing weights. Figure 9 (a) The illustration shows the currently processed CU 256, template block 258, and corresponding samples or pixels within template block 258, which are filtered using a filter with a 3×3 filter window 280 centered on the middle line sample 262 of template block 258, as described in the conventional method above. Template block 258 includes a left template block or left template block region 258a, an upper template block or upper template block region 258b, and a upper left template block or upper left template block region 258c. The boundaries between template block regions are indicated by dashed lines in template block 258. The currently processed CU 256 is split into four CU regions or mixed weight regions, as indicated by the dashed lines in CU 256, resulting in a first or upper left (TL) CU region 256a, a second or upper right (TR) CU region 256b, a third or lower left (BL) CU region 256c, and a fourth or lower right (BR) CU region 256d. Figure 9 (b) Another embodiment of the invention is illustrated, according to which the currently processed CU 256 is split into two CU regions, thereby producing a first or left (L) CU region 256a and a second or right (R) CU region 256b. Figure 9 In (a), CU 256 is divided into four rectangular regions, while Figure 9 In (b), CU 256 is divided into two square-shaped CU regions.

[0146] As in conventional DIMD processing, a HoG is constructed for template block 258, and according to embodiments of the invention, IPMs associated with each filter window location are stored. It should be noted that the invention is not limited to determining IPM statistics based on the HoG and selecting one or more IPMs accordingly. According to other embodiments, other methods can be applied to determine IPM statistics; for example, a Hough transform or gradient-based Hough transform can be used on the filter window size to determine the orientation of each center pixel (of the filter window), and the histogram of the found orientations can then allow selection of peak orientations (and matching IPMs). As in prior art methods, two or more IPMs are selected from the HoG, and finally, a blending weight is determined for each CU region. Figure 10 The illustration depicts the DIMD predictor derivation process according to an embodiment of the present invention. At S220, the neighborhood of the currently processed CU256 is analyzed to determine the number of available upper and left inner samples or pixels. Based on this, a HoG is constructed at S222. Furthermore, the IPM and associated amplitude for each filter window location are stored. At S224, two or more of the most representative IPMs in terms of amplitude in the HoG are determined. (Refer to...) Figure 10 In the described embodiment, two of the most representative IMPs are identified, and at S226, it is determined whether to select 0, 1, or 2 IMPs. If no IMP is selected, then the Planar or DC intra-prediction mode (which may be the default mode) is used as the DIMD predictor, as indicated at S228. If an IMP is selected, then, according to the embodiment, the selected IMP is used as the DIMD predictor, as indicated at S230.

[0147] If two IPMs are selected at S226, then the currently processed CU is split into multiple CU regions or CU mixed weight regions, as indicated at S232. For example, the CU can be configured as described above. Figure 9 The process is as described above. At S234, the cumulative amplitudes of those IPMs present in the template block regions adjacent to the CU region are used to determine the mixing weights for each region. At S236, a prediction is calculated for each selected IPM, and at S238, a DIMD predictor is obtained by mixing the two selected IPMs with the Planar or DC mode by region using the determined regional mixing weights. According to other embodiments, as mentioned above, step S232 can be performed before constructing the HoG at step S222.

[0148] According to other embodiments, when only one IPM is selected, the corresponding regional mixing weights can be determined, and instead of using one selected IPM as the DIMD predictor, the DIMD predictor is obtained by mixing one selected IPM with the Planar mode or DC mode using the determined regional mixing weights.

[0149] First aspect: Using the weights calculated for the entire template block to derive the mixed weights of the region.

[0150] According to an embodiment of the first aspect of the invention, after calculating the HoG, each CU region is attached to or associated with at least one adjacent template block (e.g., the upper, left, and / or upper left template block). The mixing weight of a particular CU region is selected based on the presence of the IPM associated with the globally calculated mixing weight in the template block regions adjacent to the CU region. Figure 11 The illustration illustrates an embodiment of the invention whereby a blending weight is selected based on the adjacency of a specific CU region relative to a specific template block region. As shown, the current CU 256 is divided into four square regions: TL region, TR region, BL region, and BR region. Of course, the currently processed CU 256 can also be divided into rectangular regions, and instead of dividing CU 256 into four regions, it can be divided into even fewer regions, such as... Figure 9 As shown in (b), or divided into more rectangular or square areas.

[0151] According to an embodiment, the splitting of CU 256 can depend on the size of the CU currently being processed. Therefore, for example, a CU with a size of 16×8 samples can be split into two square-shaped regions, each with a size of 8×8 samples. According to a further embodiment, the splitting of CU 256 can only be applied when the size of the CU currently being processed is higher than a predefined threshold, for example, for CUs with a size of at least 8×8, 16×8, or 8×16.

[0152] According to the reference Figure 11In the described embodiment, the corresponding CU region uses global weights associated with IPMs present in adjacent template block regions. Statistics (such as cumulative amplitude) of the IPMs present in the corresponding template block region are calculated over the entire HoG. In other words, the CU region does not modify the weights calculated based on global statistics, but rather selects weights based on the presence of matching IPMs in adjacent template block regions. For example, for the upper left CU region, mixed weights associated with IPMs present in the upper and left template block regions 258a, 258b, and optionally also the upper left template block 258c are selected. For the upper right CU region, only mixed weights associated with IPMs present in the upper template block region 258b are selected. Similarly, for the lower left CU region, only mixed weights associated with IPMs present in the left template block region 258a are selected. For the lower right CU region not adjacent to template block 258, the weights associated with any mode other than the Planar or DC mode are set to zero; that is, the Planar / DC mode is used as the predictor for the lower right CU region.

[0153] According to the embodiment, the region-based mixing weights can be selected as follows:

[0154] Initially, the IPM to be blended is selected using, for example, HoG. If more than one IPM is selected, the blending weights per region can be set according to one or more of the following:

[0155] (R1.1) If the IPM exists in a template block region adjacent to the CU region, then use the blending weight associated with the IPM.

[0156] (R1.2) If only one of the IPMs exists in the template block region adjacent to the CU region, then the blending weight associated with that IPM or the weighted blending weight associated with that IPM shall be used.

[0157] And set the blending weights associated with any other IPM to a predefined value, such as 0.

[0158] (R1.3) If no IPM exists in the adjacent template block region of the CU region, then the blending weight associated with the IPM is set to a predefined value, such as 0.

[0159] (R1.4) The hybrid weights associated with the Planar or DC mode are set to predefined values.

[0160] If you select an IPM, you can set the region-based blending weights based on one or more of the following:

[0161] (R2.1) If an IPM exists in a template block region adjacent to the CU region, then the blending weight associated with that IPM or the weighted blending weight associated with that IPM is used, and the blending weight associated with any other IPM is set to a predefined value, such as 0.

[0162] (R2.2) If an IPM does not exist in a template block region adjacent to the CU region, then the blending weight associated with that IPM and the blending weight associated with any other IPM are set to predefined values, such as 0.

[0163] (R2.3) The hybrid weights associated with the Planar or DC mode are set to predefined values, or

[0164] If no IPM is selected, the blend weights associated with any other IPM are set to predefined values, such as 0, and / or the blend weights associated with Planar or DC modes are set to predefined values.

[0165] Furthermore, more detailed embodiments for determining or setting region-based mixing weights will now be described, but the invention is not limited to these embodiments. According to the embodiments, when considering a maximum selection of two IPMs M1 and M2, the mixing weights w1, w2, w3 can initially be calculated using conventional processing, as referenced above. Figure 6 As described below:

[0166]

[0167] When modes M1 and M2 are selected as the two most representative in-angle prediction modes in the HoG for the entire template block 258, the mixing weights for each CU region are selected based on the presence of modes M1 and / or M2 in adjacent template block regions (i.e., the upper, upper left, and / or left template block regions). Let w represent the i-th mixing weight for region r. i,r The mixing weights for each CU region are selected as follows:

[0168] (R1.1) If M1 and M2 exist in adjacent template blocks of region r, then the blending weights of region r are allocated as follows:

[0169] w 1,r =w1w 2,r =w2w 3,r =w3

[0170] (R1.2) If only Mx exists in the adjacent template blocks of region r (where x = 1 or 2 and y equals another value), then the mixed weights of region r are assigned as follows:

[0171] w y,r =0

[0172] (R1.3) If only Mx exists in the adjacent template blocks of region r (where x = 1 or 2 and y equals another value), then the mixing weights of region r are assigned as follows (choosing the current intra-prediction mode, since DIMD mode has no mixing):

[0173] w x,r =1 w y,r =0 w 3,r =0

[0174] (R1.4) If M1 and M2 do not exist in the adjacent template blocks of region r, then the blending weights of region r are allocated as follows:

[0175] w 1,r =0 w 2,r =0 w 3,r =1

[0176] Figure 12 The illustration shows a mixed weight allocation according to an embodiment of the present invention, assuming that IPMs M1 and M2 are selected, and... Figure 12 In (a), it is assumed that IPM M1 exists in the left template block region 258a and the upper left template block region 258c, while IPMM2 exists in the upper template block region 258b. On the other hand, in Figure 12 In (b), it is assumed that IPMs M1 and M2 exist in the upper template block region 258b and the upper left template block region 258c, but not in the left template block region 258a. Therefore, when considering Figure 12 In the upper left CU region (a), the mixing weights associated with the IPMs are determined according to (R.1) above, since IPMs M1 and M2 exist in all template block regions adjacent to the upper left CU region. On the other hand, for the upper right and lower left CU regions, only IPMs M2 and M1 exist in adjacent template block regions, respectively, and local or regional mixing weights are calculated according to (R.3) above, thereby generating DIMD predictors as IPMs M2 and M1, respectively. For the lower right CU region, since this region is not adjacent to any template block region, the Planar or DC intra-prediction mode is selected as the DIMD predictor according to (R.4) above.

[0177] According to a further embodiment, for Figure 12In the lower left CU region of (a), instead of applying (R.3) above, (R.2) can also be applied to obtain a DIMD predictor by mixing IPM M1 with Planar or DC modes. Similarly, a DIMD predictor for the upper right CU region can be obtained by mixing IPM M2 with Planar or DC according to (R.2) above.

[0178] exist Figure 12 In (b), since the two selected IPMs M1 and M2 are adjacent to both the upper left and upper right CU regions, the local or regional blending weights are calculated according to (R.1) above. Since no IPM comes from the left template block region adjacent to the lower left CU region, the weights are calculated according to (R.4) above, thereby generating the DIMD predictor as either Planar intra-prediction mode or DC intra-prediction mode.

[0179] Figure 13 The diagram illustrates a flowchart summarizing the determination of the region blending weights according to an embodiment of the present invention. First, as indicated at S250, the current CU 256 is divided into two or more regions, and as indicated at S252, subsequent steps are performed for each of the plurality of CU regions, i.e., for each region r. At S254, for the currently processed region r, those blocks / regions adjacent to region r in the left and upper left template block regions 258a, 258c are determined. At S256, it is determined whether M1 exists in one or more of the template block regions adjacent to the currently processed region r. If so, then at S258, it is determined whether M2 exists in one or more of the template block regions adjacent to the currently processed CU region r. If so, then at S260, a blending weight is selected according to (R1.1) above. If at S258 it is determined that M2 does not exist in any adjacent block, then at S262, a blending weight for CU region r is calculated according to (R1.2) above. According to other embodiments, in S262, the mixed weights for region r can be calculated according to (R1.3) above.

[0180] If it is determined at S256 that M1 does not exist in any template block region adjacent to region r, then at S264 it is determined that M2 exists in one or more template block regions adjacent to region CU, and if so, then at S266 the local blending weight for region r is determined according to (R1.2) or (R1.3) above. If it is determined at S254 that M2 does not exist in any adjacent block, then the local blending weight for region r is determined according to (R1.4) above.

[0181] If only one IPM M1 is selected from the HoG used for the entire template block 258, then the blending weights for each region r are selected as follows based on the presence of IPM M1 in the template block regions adjacent to the corresponding CU regions:

[0182] (R2.1) If M1 exists in a template block region adjacent to CU region r, then the mixing weights of region r are allocated as follows (since DIMD mode band mixing is used, the current intra-prediction mode is selected):

[0183] (R2.2) If M1 exists in a template block region adjacent to CU region r, then the mixing weights of region r are assigned as follows (since DIMD mode does not involve any mixing, the current intra-prediction mode is selected):

[0184] w 1,r =1w 2,r =0w 3,r =0

[0185] (R2.3) If M1 does not exist in the template block region adjacent to CU region r, then the blending weights of region r are allocated as follows:

[0186] w 1,r =0w 2,r =0w 3,r =1

[0187] Figure 14 The illustration shows a mixed weight allocation according to an embodiment of the present invention when only one IPM M1 is selected. Figure 14 In (a), it is assumed that IPM M1 exists in the left template block region 258a and the upper left template block region 258c, but not in the upper template block region 258b. On the other hand, in Figure 14 In (b), IPM M1 exists only in the upper left template block area 258c and the upper template block area 258b, but not in the left template block area 258a. For example, from... Figure 14 (a) As can be seen, the region weights for the upper left CU region are determined according to (R2.1) above, while the region weights for the lower left CU region are determined according to (R2.2) above. That is, for the upper left CU region, a mixture of IPM M1 and Planar mode or DC mode is performed, while for the lower left CU region, this mixture is not performed. For the upper left CU region, since IPM M1 does not exist in the adjacent upper template block region 258b, Planar mode or DC mode is selected as the DIMD predictor (R2.3). Figure 14In (b), the local or regional blending weights for the upper left and upper right CU regions are determined according to (R2.1), while for the lower left region, they are determined according to (R2.3) because IPM M1 does not exist in the left template block region 258a.

[0188] If an IPM is not selected from the HoG from the entire template block 258, then the blending weights of the corresponding region r are positioned as follows:

[0189] w 1,r =0,w 2,r =0 and w 3,r =1

[0190] Therefore, in this case, Planar or DC intra-prediction mode is used as the DIMD predictor for all CU regions.

[0191] According to the embodiment, if at least one of its M is in the template block region x If the amplitude of a sample is greater than a predefined threshold (e.g., amplitude greater than 50), then the selected IPM M can be considered as such. x It exists in the template block area. According to other embodiments, if the template block area is related to IPM M... x The number of associated sample locations exceeds a predefined number, for example, if there are more than two known patterns M. x If the filtered sample positions are known, then the selected IPM M can be considered as... x It exists in the template block area.

[0192] According to a further embodiment, for a region adjacent to one or more template block regions from which an IPM is selected, the blending weights associated with M1 and M2 can be applied to other CU regions adjacent to template block regions for which no IPM has been selected. In such further CU regions, the blending weights from the first CU region can be reduced, while w 1,r +w 2,r +w 3,r =1 Applicable. Figure 15 The illustration shows the use of a hybrid weighting based on the distance between a given region and a template block region that includes the selected IPM. Figure 15 In the middle, assuming similar to Figure 12 (b) and according to Figure 15 For the upper left and upper right CU regions, the blending weights are calculated according to (R1.1) above. Besides... Figure 12(b) Outside the lower left CU region, instead of selecting the Planar mode as the DIMD predictor, a DIMD predictor is obtained by mixing weighted blending weights used for the upper right CU region. The blending weights w1 and w2 are weighted by 1 / 2, and the blending weight w3 is weighted by a factor of 2. Therefore, w 1,BL +w 2,BL +w 3,BL =1. Optionally, this can also be applied to the lower right CU region. According to a further embodiment, this can also be used by using a weighted blending weight w2 from the upper left CU region. Figure 12 The lower left CU region in (a).

[0193] The second aspect: using the weights calculated for each template block region to derive the region hybrid weights.

[0194] In the embodiment of the first aspect of the invention described above, the cumulative amplitude of the selected IPM stored in the HoG is used to calculate the blending weight for the entire template block. According to an embodiment of the second aspect of the invention, the blending weight is calculated for each adjacent template block. According to this embodiment, instead of calculating the HoG for the entire template block, the HoG is calculated only for portions of the template blocks adjacent to the CU region (e.g., calculating the HoG for the left template block region 258a and / or the upper template block region 258b). According to an embodiment, the upper left block region 258c can be handled as follows:

[0195] - It is assigned to the template block area above, or

[0196] - It is assigned to the left template block area, or

[0197] - It is assigned to the top and left template block areas, or

[0198] - It is treated as a separate template block area.

[0199] The currently processed CU 256 is divided into multiple separate CU regions in the manner described above with reference to the embodiments of the first aspect.

[0200] When the current CU region is adjacent to several template block regions, HoG is calculated on the adjacent template blocks, and one or more IPMs are selected based on the cumulative amplitude peaks in the adjacent template blocks for which HoG is calculated. For example, for the upper left CU region, which is adjacent to the left template block region 258a, the upper template block region 258b, and the upper left template block region 258c, the mixing weights are calculated based on the statistics of the entire template block (i.e., based on the HoG calculated on the entire template block 258). Therefore, the determination of the mixing weights for the upper left CU region is the same as described above with reference to the first aspect. On the other hand, the upper right CU region is only adjacent to the upper template block region 258b, and HoG is calculated only on this upper template block region 258b. At least one IPM is selected based on the cumulative amplitude peaks, and the weights to be applied to the upper right CU region are selected based on the weights determined for the HoG calculated for the upper template block region. If only one IPM is selected, then the mixing weights for the upper right CU region can be selected according to (R2.1) to (R2.3) described above with reference to the first aspect. If two IPMs are selected from the adjacent template block regions in the upper right region, then the blending weights for the upper right region are selected according to (R1.1) to (R1.4) described above with reference to the first aspect.

[0201] Figure 16 The diagram illustrates the calculation of HoG for each template block region. For the upper right region of the currently processed CU, which is only adjacent to the upper template block region 258b, the associated IPM is shown for each midline sample in this region, and its orientation is illustrated by the direction of the corresponding arrow, while its amplitude is indicated by the length of the arrow. To determine the regional or local blending weights of the upper right region of the currently processed CU 256, the HoG is determined. Above In the depicted embodiments, IPMs 4, 5, and 10 are generated. Only one of IPMs 4 and 5, or both IPMs 5 and 4, is selected and used to determine the blending weights for the upper right region of CU 256. According to the embodiments, different HoGs (see above) for the left, top, and upper left template block regions can be calculated in the manner described above with reference to the first aspect. Figure 10 Step S222). For each template block region, the intra-frame mode and amplitude for each filter window position in the corresponding template block region are stored, and one or more of the most representative IPMs can be determined in the HoG for the considered template block region (see step S222). Figure 10 (Step S224). In other words, the mixed weight calculation is similar to that in the first aspect of the invention, except that HoG is not performed on the entire template block, but only on one or more template block regions adjacent to the CU region for which the mixed weight is calculated.

[0202] exist Figure 16 In the embodiment, when two IPMs (i.e., mode 4 and mode 5) are selected, w is used to determine 1,TR and w 2,TR The amplitude ratio is calculated based on the amplitude of the selected IPM relative to the HoG calculated for the upper template block region. In other words, the amplitude ratio is calculated based on the cumulative amplitude from the HoG used for one or more adjacent template block regions. For the upper left region, the HoG is calculated for the upper, upper left, and left template block regions. For the lower left region, the HoG is calculated only based on the left template block region 258a. For the lower right region, either the Planar or DC mode can be selected as the predictor. According to other embodiments, for the lower right region, a corresponding IPM can be selected, and the HoG can be calculated based on one or more IPMs from one or a combination of the upper, upper left, and left template block regions.

[0203] Regarding the IPMs selected in different template block areas, it should be noted that the same or different IPMs can be selected in different template block areas, and the number of IPMs selected in each template block area can vary; that is, the number of IPMs selected for different template block areas can be the same or different. According to the embodiment, the maximum number of IPMs selected in each template block area can be one or two.

[0204] The second aspect of the implementation is superior to methods that use only one or more IPMs selected from the entire template block because they allow for localized adjustments to content characteristics. This is because using corresponding IPMs obtained from different template block regions better matches different regions of the currently processed CU. When using IPMs selected from the HoG used for the entire template block for all regions, the selected IPMs, which may exist in different template block regions, can have different content characteristics between the corresponding template block regions, such as slightly different orientations or angles. In fact, the number of possible IPMs (e.g., 67, without counting wide-angle intra-patterns) will produce broad peaks in the HoG for which the selected patterns match most regions but may not be suitable for matching the local characteristics of a specific CU region.

[0205] According to other embodiments of the second aspect, if more than one IPM is selected, the first IPM can be calculated based on the entire template block 258 (i.e., the upper, left, and upper left template block regions), while a second IPM is selected from the template block regions adjacent to the CU region for which the mixing weights are to be calculated. According to an embodiment, if the second IPM differs from the first IPM, then the second IPM is selected. If no IPM is selected as the second IPM, then the first selected IPM can be used as the DIMD predictor. On the other hand, if the second IPM is equal to the first IPM, then the second best peak in the adjacent template block region HoG is assigned as the second IPM. Figure 17 An embodiment is illustrated, according to which a first IPM is selected based on the entire HoG and a second IPM is selected based on the HoG used for adjacent template blocks. Figure 17 In the embodiment, when considering the upper right region of the currently processed CU 258, as shown in the left part of the figure, the HoG calculated over the entire template block 258 is... entire Determine the first IPM, and calculate the HoG only for the template block region above. above A second IPM is determined. In the depicted embodiment, IPM 28 is selected as the first IPM and IPM 5 is selected as the second IPM. Based on the selected IPMs, the mixing weights are calculated as described above with reference to the first aspect of the invention.

[0206] Figure 18The illustration shows a flowchart of one embodiment of the invention, according to which a first angle IPM is selected from the entire template block HoG and a second IPM is selected from the HoG of the template block adjacent to the CU region. At S280, the neighborhood of the CU 256 to be processed is analyzed to determine the number of available upper and left inner samples. At S282, a histogram of gradients is constructed, more specifically, determining the HoG for the left, upper, and upper left template block regions 258a to 258c. Additionally, the corresponding intra-frame mode and amplitude for each filter window position are stored. At S284, a first IPM is determined, for example, the most representative IPM in the entire HoG, and at S286, the number of IPMs selected is determined. In the absence of IPM 1, i.e., no first IPM is selected, for example, because there is no IPM with an amplitude higher than a certain threshold, then the Planar or DC intra-frame prediction mode is used as the DIMD predictor, as indicated at S288. Otherwise, if the first IPM is selected, then at S290, the current CU 256 is split into multiple CUs or mixed weight regions. At S292, for each region, a second IPM is determined in the HoG established for the template block region adjacent to a certain CU region. At S294, it is determined whether the second IPM determined at S294 is equal to the first IPM. If the two IPMs are the same, then at S296, the first IPM is used as the DIMD predictor. Otherwise, if the selected IPMs are different, then the mixed weights for a certain CU region are determined using the IPMs in the template block region adjacent to the CU region for which local or regional mixed weights are calculated, as indicated at S298. At S300, a prediction is calculated for each selected IPM, and at S302, the DIMD predictor is obtained by regionally mixing the two selected IPMs with the Planar pattern using the determined regional or local mixed weights. The actual mixed weights can be calculated in the manner described above with reference to the embodiments of the first aspect of the invention (e.g., according to (R1.1) to (R1.4)).

[0207] Regarding the lower right CU region where there are no adjacent template blocks, according to an embodiment, the second IPM can also be selected from the entire template block HoG. According to other embodiments, only the first IPM is selected and used as the DIMD predictor for the lower right CU region, or only the Planar or DC pattern is used as the DIMD predictor for the lower right region. According to yet another embodiment, the DIMD predictor can be obtained by mixing the Planar or DC pattern with the first IPM with a weight of 0.5.

[0208] According to the embodiment, the mixing weight can be calculated based on the cumulative amplitude ratio of the entire template block HoG, and the weight of the second IPM can be greater than the weight of the first IPM. For example, for the first IPM, its cumulative amplitude ratio with the second IPM is 2 / 9, and for the second IPM, its cumulative amplitude ratio is 4 / 9, and for Planar, its cumulative amplitude ratio is 1 / 3.

[0209] According to a further embodiment, if the first IPM exists in an adjacent template block region and the second IPM also exists in the adjacent template block region, then the cumulative amplitude ratio for determining the mixing weight can be calculated based on the cumulative amplitude in the adjacent template regions HoG.

[0210] According to a further embodiment, if no in-angle mode is selected as the second IPM, then the first IPM can be mixed with the Planar or DC mode using a weight of 0.5. If more than two IPMs are selected, such as three or more, then the first two IPMs can be calculated from the entire template block HoG, and the third IPM can be calculated from the adjacent template block region HoG. According to other embodiments, only the first IPM can be calculated from the entire template block HoG, while further IPMs (such as the second and third) can be calculated from the template block region HoG.

[0211] Thirdly: Using the weight derivation region hybrid weights calculated for partial template blocks.

[0212] According to a third aspect of the invention, the blending weights, more specifically local or regional blending weights for the corresponding CU regions, are calculated for each portion of the adjacent sample sets in the template block, i.e., using only the position of the filter window adjacent to the CU region of the currently processed CU projected from the center sample. Conceptually, the template block 258 is divided using the same vertical and horizontal lines as the currently processed CU, such that each CU region is attached to a portion of the template block, and one or more IPMs are selected from the HoG generated for this portion of the template block, hereinafter referred to as a partial template block or a partial template block region.

[0213] Figure 19 The illustration shows an example of using template blocks to compute a partial template block HoG for deriving local blending weights. Figure 19 (a) An embodiment is illustrated in which the IPM for determining the blending weights for the upper right region of CU 256 is determined by a partial template block 258', which is implemented by splitting the upper template block region 258b along the vertical dashed line that splits CU 256. For the partial template block 258', HoG is determined, referred to as HoG in this figure. AboveRightIn the depicted embodiment, HoG generates IPM 4 and IPM 5, and for the corresponding centerline sample, the orientation is indicated by an arrow and the amplitude by the length of the arrow. HoG shows the cumulative amplitude of the IPMs, i.e., three cumulative amplitudes for IPM 5 and two cumulative amplitudes for IPM 4. Figure 19 (b) It is illustrated that CU 256 is divided into a left region and a right region, and for the left region, a portion of template block 258' adjacent to the left CU region is illustrated, which includes a portion of the left template block region 258a, the upper left template block region 258c, and the upper template block region 258b. Figure 19 (c) The diagram illustrates that CU 256 is divided into four regions, as follows: Figure 19 As shown in (a), and for the upper left region illustrated, a portion of template block 258' is shown, which includes a portion of the left template block region 258a, a portion of the upper template block region 258b, and the upper left template block region 258c. (See from...) Figure 19 As can be seen, the width / height of the partial template block defined in the upper or left template block region is defined by the vertical / horizontal lines defining the corresponding CU region. The concepts of the first and second aspects apply to the partial template block region, except that only strictly partial or portion-sized template blocks adjacent to CU regions are considered. Therefore, HoG calculation and IPM selection only occur on samples present in the corresponding partial template block 258', i.e., samples in the template block that are adjacent to a certain CU region for which the mixing weight is to be determined.

[0214] The third aspect of the invention is advantageous because the adaptation and matching to the local characteristics of a certain CU region is a better match. According to an embodiment, if the direction in the corresponding template block region is far from the horizontal line relative to the left template block region or far from the vertical line relative to the upper template block region, then additional information collected from adjacent template block portions can be combined so that, according to an embodiment, the second and third aspects can be combined. Figure 20 The illustration shows an example of using a mixture of partial and full / entire template block regions to calculate the HoG for selecting one or more IPMs. (See also...) Figure 20As shown, according to an embodiment, CU 256 is divided into four CU regions, and for the upper left region, only a portion of the upper template block, which is part of the left template block region 258, is used, while the entire left template block region is used. That is, the template block region for which the HoG is calculated to select the IPM includes the left template block region 258a, the upper left template block region 258c, and a portion of the upper template block region adjacent to the upper left CU region. According to a further embodiment, when multiple IPMs (such as two or more IPMs) can be selected, one or some, but not all, of the IPMs can be determined from the HoG obtained for the entire left template block region, while the remaining IPMs are obtained from the HoG obtained for the portion of the template block. According to a further embodiment, the first IPM can be calculated based on the entire template block, while the second IPM can be determined based only on the HoG used for the portion of the template block.

[0215] The blending weights are determined for each CU region; therefore, the number of blending weights can depend on the current CU region for which local blending weights are currently being calculated.

[0216] According to the embodiment, the mixing weight can be calculated based on the cumulative amplitude ratio of the entire template block HoG, and the weight of the second IPM can be greater than the weight of the first IPM. For example, the cumulative amplitude ratio of the first IPM to the second IPM is 2 / 9, the cumulative amplitude ratio of the second IPM is 4 / 9, and the cumulative amplitude ratio of Planar is 1 / 3.

[0217] According to a further embodiment, if the first IPM exists in an adjacent template block region and the second IPM also exists in the adjacent template block region, then the cumulative amplitude ratio for determining the mixing weight can be calculated based on the cumulative amplitude in the adjacent template regions HoG.

[0218] According to a further embodiment, if the angle-to-angle pattern is not selected as the second IPM, then the first IPM can be mixed with the Planar or DC pattern using a weight of 0.5. If more than two IPMs are selected, such as three or more, then the first two IPMs can be calculated from the entire template block HoG, and the third IPM can be calculated from the adjacent template block region HoG. According to other embodiments, only the first IPM can be calculated from the entire template block HoG, while further IPMs (such as the second and third) can be calculated from the template block region HoG.

[0219] According to a further embodiment of the third aspect of the invention, the direction of the IPM with the highest peak (cumulative amplitude) in the global HoG calculated for the entire template block can determine which template block region should be used entirely and which should be used partially. For example, if the principal direction in the HoG calculated for the entire template block or for a template block region is more vertical than horizontal, for example, if the IPM is between 34 and 66 and between 53 and 66 (including the endpoints), then the upper left CU region can use part of the upper template block 258' (see...). Figure 20 ) and the entire left template block area. Otherwise, if the main direction in the HoG is more horizontal than vertical, such as between 2 and 34 and IPM greater than 66 (including endpoints), then the upper left CU area (see Figure 20 You can use the entire top template block area, the entire top left template block area, and only a portion of the left template block area.

[0220] Further embodiments

[0221] The concept of the invention has been described so far with reference to aspects and embodiments of methods for determining a DIMD predictor. According to a further embodiment, the invention also provides an apparatus for deriving a DIMD predictor and an encoder / decoder including such an apparatus.

[0222] Figure 21 The illustration shows a block diagram of an apparatus for a corresponding sample-derived decoder-side intra-frame mode derivation (DIMD) predictor for a codec unit (CU) of a picture, according to an embodiment of the present invention. The apparatus 300 includes the following modules:

[0223] 302: Intra-Prediction Mode (IPM) selection module, configured to select one or more IPMs in a template block adjacent to the CU.

[0224] 304: The blending weight determination module is configured to determine blending weights for blending at least one or more selected IPMs.

[0225] 306: DIMD predictor generation module, configured to generate a DIMD predictor by mixing one or more selected IPMs using mixing weights determined for the CU such that the mixing varies on the CU.

[0226] Figure 22 The diagram illustrates a block diagram of a decoder for decoding an encoded data stream according to an embodiment of the present invention. The encoded data stream includes data representing an image. The decoder 400 includes the following modules:

[0227] 402: Decoder module, configured to decode images from an encoded data stream.

[0228] 404: Prediction module, including means for deriving a decoder-side intra-frame mode derivation (DIMD) predictor for a corresponding sample of a codec unit (CU) for a picture according to an embodiment of the present invention.

[0229] Figure 23 The diagram illustrates a block diagram of an encoder for encoding an image into an encoded data stream according to an embodiment of the present invention. The encoded data stream includes data representing the image. The encoder 500 includes the following modules:

[0230] 502: Encoder module, configured to receive the raw image and encode it into an encoded data stream.

[0231] 504: Prediction module, the prediction module including means for deriving a decoder-side intra-frame mode derivation (DIMD) predictor for a corresponding sample of a codec unit (CU) according to an embodiment of the invention.

[0232] Although some aspects of the disclosed concepts have been described in the context of the apparatus, it is clear that these aspects also represent descriptions of the corresponding methods, where boxes or devices correspond to method steps or features of method steps. Similarly, aspects described in the context of method steps also represent descriptions of corresponding boxes, items, or features of the corresponding apparatus.

[0233] Figure 24 This is a block diagram illustrating an electronic device 900 according to an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are described by way of example only and are not intended to limit the embodiments of the invention described herein and / or claimed. Device 900 includes a computing unit 901 to perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) 902 or loaded from storage unit 908 into random access memory (RAM) 903. Various programs and data for storing the operation of device 900 may also be stored in RAM 903. The computing unit 901, ROM 902, and RAM 903 are connected to each other via bus 904. Input / output (I / O) interface 905 is also connected to bus 904.

[0234] Components in device 900 are connected to I / O interface 905, including: input unit 906, such as a keyboard and mouse; output unit 907, such as various types of displays and speakers; storage unit 908, such as a disk and optical disk; and communication unit 909, such as a network card, modem, wireless communication transceiver, etc. Communication unit 909 allows device 900 to exchange information / data with other devices via computer networks (such as the Internet and / or various telecommunications networks). Computing unit 901 may consist of various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Computing unit 901 performs the various methods and processes described above, such as image processing methods. For example, in some embodiments, image processing methods may be implemented as computer software programs tangibly implemented on a machine-readable medium (such as storage unit 908). In some embodiments, part or all of the computer program may be loaded and / or installed on the device 900 via ROM 902 and / or communication unit 909. When the computer program is loaded into RAM 903 and executed by computing unit 901, one or more steps of the image processing method described above may be performed. In some embodiments, computing unit 901 may be configured to execute the image processing method in any other suitable manner (e.g., by means of firmware).

[0235] Various implementations of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may be implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, and the programmable processor may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and capable of transferring data and instructions to the storage system, at least one input device, and at least one output device.

[0236] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when the processor or controller executes the program code, it performs the functions and / or operations specified in the flowcharts and / or block diagrams. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0237] In the context of this invention, a machine-readable medium can be a tangible medium that can contain or store a program used by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, optical disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0238] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having a display device (e.g., a cathode ray tube (CRT) or liquid crystal display (LCD)) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0239] The systems and technologies described herein can be implemented on a computing system including back-end components (e.g., as data servers), or on a computing system including middleware components (e.g., application servers), or on a computing system including front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or on a computer system including such back-end components, middleware components, front-end components, or any combination thereof. The components of the system can be interconnected via digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (LANs), wide area networks (WANs), the Internet, and blockchain networks.

[0240] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is generated by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in a cloud computing service system and addresses the management difficulties and weak business scalability of traditional physical hosts and Virtual Private Servers (VPS). A server can also be a server in a distributed system or a server combined with blockchain technology.

[0241] It should be understood that steps can be reordered, added, or deleted using the various forms of processes shown above. For example, the steps described in this invention can be executed in parallel, sequentially, or in a different order, as long as the desired result of the technical solution in this invention can be achieved, and no limitation is imposed herein.

[0242] The specific embodiments mentioned above do not limit the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this invention should be included within the scope of protection of this invention.

[0243] References

[0244] [1]H.264:Advanced video coding for generic audio-visual services,https: / / www.itu.int / rec / T-REC-H.264-202108-P / en

[0245] [2]H.265: High efficiency video coding, https: / / www.itu.int / rec / T-REC-H.265-202108-P / en

[0246] [3]H.266:Versatile video coding,https: / / www.itu.int / rec / T-REC-H.266-202008-I / en

[0247] [4]AV1 Bitstream&Decoding Process Specification, http: / / aomedia.org / av1 / specification /

[0248] [5] E. Mora et al., “CE3-related: Decoder-side Intra Mode Derivation”, JVET-L0164, Macau, China, October 2018

[0249] [6]CE3:Decoder-side Intra Mode Derivation(tests 3.1.1,3.1.2,3.1.3and3.1.4),JVET-M0094,Marrakech,January 2019

[0250] [7] M. Abdoli et al., E. Mora, T. Guionnet, M. Raulet, Non-CE3: Decoder-side IntraMode Derivation with Prediction Fusion, JVET-N0342, Geneva, March 2019

[0251] [8] M. Abdoli et al., “Non-CE3: Decoder-side Intra Mode Derivation with Prediction Fusion Using Planar”, JVET-O0449, Gothenburg, July 2019

[0252] [9] J. Zhao et al., “EE2-Related: Improvements of Decoder-Side Intra ModeDerivation”, JVET-V0087, online, April 2021.

[0253]

[10] J. Zhao et al., “EE2 Test 2.2: DIMD with multiple blending modes”, JVET-W0054, online, July 2021.

[0254]

[11] X.Li et al., “EE2-related: Implicit derivation of DIMD blend modes”, JVET-W0067, online, July 2021.

[0255]

[12] X.Li et al., “EE2-related: A combination of CIIP and DIMD / TIMD”, JVET-W0068, online, July 2021.

[0256]

[13] J. Zhao et al., “EE2 Related-DIMD with implicitly derived multipleblending modes”, JVET-W0126, online, July 2021.

[0257]

[14] X.Li et al., “EE2-3.1: Combination of CIIP and DIMD / TIMD”, JVET-X0098, online, October 2021.

[0258]

[15] C. Zhou et al., “EE2-related: Optimization on the second modederivation of DIMD blending mode”, JVET-X0115, online, October 2021.

[0259]

[16] T. Dumas et al., “AHG12:removing a discontinuity in the discrete angle comparison in DIMD”, JVET-X0139, online, October 2021.

[0260]

[17] X. Xiu et al., “AHG12: Removal of floating operations in DIMD and TIMD”, JVET-X0149, online, October 2021.

[0261]

[18] K. Cao et al., “Non-EE2: Fix for histogram of gradients derivation in DIMD mode”, JVET-X0156, online, October 2021.

[0262]

[19] K. Naser et al., “AhG12:Removed DIMD from MPM list of TIMD”, JVET-Y0097, online, January 2022.

[0263]

[20] S. Yoo et al., “EE2-related: Clean-up on DIMD”, JVET-Y0131, online, January 2022.

[0264]

[21] J. Zhao et al., “Non-EE2:DIMD Flag Signalling Clean-up”, JVET-Y0144, online, January 2022.

[0265]

[22] J. Pfaff et al., “Intra Prediction and Mode Coding in VVC”, IEEE Transactions on Circuits and Systems for Video Technology, Vol. 31, No. 10, October 2021.

Claims

1. A method for deriving a decoder-side intra-frame mode derivation (DIMD) predictor for a corresponding sample of a codec unit (CU) of an image, the method comprising: Select one or more intra-prediction modes (IPMs) in the template block adjacent to the CU. Determine the mixing weights for mixing at least one or more of the selected IPMs, and The DIMD predictor is generated by mixing one or more selected IPMs using mixing weights determined for the CU, such that the mixing varies across the CU. The method further includes: The CU is split into two or more CU regions. Wherein, determining the blending weight for blending at least one or more of the selected IPMs includes, for each CU region in at least a subset of the two or more CU regions, determining the blending weight based on the presence of one or more selected IPMs in the portion of the template block adjacent to the corresponding CU region, and The step of generating the DIMD predictor by mixing the one or more selected IPMs using mixing weights determined for the CU so that the mixing varies on the CU includes generating the DIMD predictor using mixing weights determined for each CU region.

2. The method of claim 1, wherein the one or more IPMs are selected using globally determined IPM statistics across the entire template block, the template block comprising a plurality of template block regions, wherein Perform the selection globally on the entire CU, or The selection involves a global selection of the set of IPMs for the entire CU, followed by a further region-based selection of the set of IPMs for each CU region.

3. The method of claim 1, wherein IPM statistics are used to select the one or more IPMs, the statistics being determined individually on the following portions. Each of the multiple template block regions of the template block includes a left template block region and an upper template block region, or Each of the multiple partial template block regions of the template block is a template block region that is adjacent to only one of the CU regions. The selection mentioned therein involves making a region-based selection from an IPM that is determined individually for each CU region.

4. The method of claim 2 or 3, wherein the IPM statistics are a histogram of gradients (HoG).

5. The method of any one of claims 1 to 3, wherein determining the mixing weights includes For a CU region positioned adjacent to the template block, the blending weight is determined based on the presence of one or more selected IPMs in one or more template block regions or portions of the template block regions adjacent to the CU region. For a CU region that is not adjacent to the template block, perform one of the following operations: Select only the Planar mode as the DIMD predictor, or Select only the DC mode as the DIMD predictor, or The blending weight is determined using one or more of the selected IPMs in one or all template block regions, or The mixed weights are determined by weighting the mixed weights of the CU regions adjacent to the template block region.

6. The method as described in any one of claims 1 to 3, in The one or more IPMs are selected in a manner that recognizes the subdivision of the template block into multiple template block regions, the multiple template block regions including a left template block region and an upper template block region. The template block also includes a top-left template block area, which is assigned to one of the following areas: To the above template block area, or To the left template block area, or To both the upper template block area and the left template block area, or As an additional template block area, and / or in Select the one or more IPMs in a manner that recognizes the subdivision of the template block into multiple partial template block regions. When defining the two or more CU regions, the partial template block region is defined by splitting the template block with the same vertical and / or horizontal lines as the CU.

7. The method of claim 6, wherein one or more first IPMs are selected in a template block region of the template block, and one or more second IPMs are selected in a portion of the template block region of the template block, wherein... If the CU has a rectangular shape, then the first IPM is selected in the template block region adjacent to the longer dimension of the CU, and the second IPM is selected in the partial template block region adjacent to the shorter dimension of the CU, or... The selection of the first IPM and the second IPM in the template block region or from the portion of the template block region adjacent to the CU depends on the orientation of the peak IPM in the template block region or throughout the entire template block.

8. The method of claim 7, wherein If the principal direction of the peak IPM calculated in the template block region or throughout the entire template block is more vertical than the horizontal, then the first IPM is selected in the left template block region or the upper template block region, and in the upper left template block region adjacent to the CU region, and the second IPM is selected in the portion of the upper template block region or the left template block region adjacent to the CU region. If the main direction of the peak IPM calculated in the template block area or the entire template block is more horizontal than vertical, then the first IPM is selected in the upper template block area or the left template block area and in the upper left template block area adjacent to the CU area, and the second IPM is selected in the portion of the left template block area or the upper template block area adjacent to the CU area.

9. The method of claim 6, in, If more than two IPMs are selected, then at least some, but not all, of the IPMs are selected throughout the template block, while the remaining IPMs are selected either within the template block area or from a portion of the template block area adjacent to the CU area. If two IPMs are selected, then the first IPM in the entire template block is selected, and If there is no second IPM from the template block region or the portion of the template block region adjacent to the CU region, then one of the following operations is performed: The first IPM is used as the DIMD predictor, or The first IPM is mixed with a Planar or DC mode using predefined weights to obtain the DIMD predictor. If a second IPM exists in the template block region or in the portion of the template block region adjacent to the CU region, then one of the following operations is performed: If the second IPM is different from the first IPM, then the first IPM and the second IPM are mixed to obtain the DIMD predictor. If the second IPM is equal to the first IPM, then other IPMs in the template block region adjacent to the CU region are selected, and the first IPM is mixed with the other IPMs to obtain the DIMD predictor.

10. The method of claim 9, wherein for CU regions positioned not adjacent to any of the template block regions, Using only the first IPM as the DIMD predictor, or Use only the Planar pattern as the DIMD predictor, or Use only the DC mode as the DIMD predictor, or The first IPM is mixed with a Planar or DC pattern using predefined weights to obtain the DIMD predictor.

11. The method as described in any one of claims 1 to 3, in, If more than one IPM is selected, then If more than one IPM exists in a template block region adjacent to the CU region, then the blending weight associated with the more than one IPM is used. If only one of the more than one IPMs exists in the template block region adjacent to the CU region, then the blending weight associated with that IPM or the weighted blending weight associated with that IPM is used, and the blending weight associated with any other IPM is set to a predefined value. If no IPM exists in an adjacent template block region of the CU region, then the blending weights associated with more than one IPM are set to predefined values, and Set the hybrid weights associated with the Planar or DC mode to predefined values, or If an IPM is selected, then If the IPM exists in a template block region adjacent to the CU region, then the blending weight associated with the IPM or the weighted blending weight associated with the IPM is used, and the blending weight associated with any other IPM is set to a predefined value. If the IPM does not exist in a template block region adjacent to the CU region, then the blending weight associated with the IPM and the blending weight associated with any other IPM are set to predefined values, and Set the hybrid weights associated with the Planar or DC mode to predefined values, or If no more than one IPM or one IPM is selected, then the blending weight associated with any other IPM is set to a predefined value, and the blending weight associated with the Planar or DC mode is set to a predefined value.

12. The method of any one of claims 1 to 3, wherein For a CU region located adjacent to a first template block region for which no IPM has been selected and far from a second template block region for which one or more IPMs have been selected, the blending weight is the weighted blending weight of the CU regions adjacent to the second template block region, or For a CU region that is located adjacent to the first template block region for which the first IPM is selected and far from the second template block region for which the second IPM is selected, the mixing weight for the second IPM is the weighted mixing weight for the second IPM of the CU region adjacent to the second template block region.

13. A non-transitory computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to perform a method for a corresponding sample-derived decoder-side intra-frame pattern derivation (DIMD) predictor for a codec unit (CU) for a picture according to any one of claims 1 to 12.

14. An apparatus for a corresponding sample-derived decoder-side intra-frame mode derivation (DIMD) predictor for a codec unit (CU) of a picture, the apparatus comprising: The intra-prediction mode (IPM) selection module is configured to select one or more IPMs from template blocks adjacent to the CU. The blending weight determination module is configured to determine blending weights for blending at least one or more of the selected IPMs, and The DIMD predictor generation module is configured to generate the DIMD predictor by mixing one or more selected IPMs using mixing weights determined for the CU such that the mixing varies over the CU. The device is further configured to split the CU into two or more CU regions. Wherein, determining the blending weight for blending at least one or more of the selected IPMs includes, for each CU region in at least a subset of the two or more CU regions, determining the blending weight based on the presence of one or more selected IPMs in the portion of the template block adjacent to the corresponding CU region, and The step of generating the DIMD predictor by mixing the one or more selected IPMs using mixing weights determined for the CU so that the mixing varies on the CU includes generating the DIMD predictor using mixing weights determined for each CU region.

15. An apparatus for decoding an encoded data stream, the encoded data stream including data representing an image, the apparatus comprising: A decoder module is configured to decode the image from the encoded data stream, and A prediction module, the prediction module including means for a corresponding sample-derived decoder-side intra-frame mode derivation (DIMD) predictor for a codec unit (CU) of the image as described in claim 14.

16. An apparatus for encoding an image into an encoded data stream, the encoded data stream comprising data representing the image, the apparatus comprising: An encoder module is configured to receive the image and encode the image into the encoded data stream. A prediction module, the prediction module including means for a corresponding sample-derived decoder-side intra-frame mode derivation (DIMD) predictor for a codec unit (CU) of the image as described in claim 14.

Images