Method and apparatus for region-by-region adaptive mixing of DIMD and encoder / decoder including same

Abstract

A method of deriving a decoder-side intra mode derivation (DIMD) predictor for respective samples of a codec unit (CU) of a picture is described. The method includes selecting one or more intra prediction modes (IPMs) in a template block adjacent to the CU, determining a mixing weight for mixing at least the one or more selected IPMs, and generating a DIMD predictor by mixing the one or more selected IPMs using the mixing weight determined for the CU such that the mixing varies over the CU.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is based on and claims priority from European patent application No. “22167216.5” filed on April 7, 2022, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present invention generally relates to the field of encoding / decoding pictures, images or videos, and embodiments of the present invention relate to improvements regarding intra prediction, and more specifically, to improvements regarding decoder-side intra mode derivation (DIMD) processing. More specific embodiments of the present invention relate to DIMD per-region adaptive mixing. Background Art

[0004] Encoding and decoding of pictures, images or videos are performed according to certain standards, for example, according to the Advanced Video Codec (AVC) standard (see reference [1]), the High Efficiency Video Codec (HEVC) standard (see reference [2]) or the Versatile Video Codec (VVC) standard (see reference [3]).

[0005] Figure 1 A block diagram of a standard video compression system 100 operating according to the VVC standard is illustrated in FIG. The standard video codec 100 compresses and encodes pictures 102 of a video sequence. The pictures 102 to be encoded are partitioned into blocks 104, also referred to as codec units (CUs). The encoder 100 includes a pre-coding filter 106 and a prediction residual signal former 108, which generates a prediction residual signal 110 to measure the deviation between the prediction signal 112 and a signal 114 output by the filter 106. The encoder also includes a transformer 116, a quantizer 118, and provides an output bitstream or data stream 120 using an entropy codec 122. In addition, the encoder 100 includes a prediction stage 124 for determining the prediction signal 112, and the prediction stage 124 includes an inverse quantizer or inverse quantizer 126, an inverse transformer 128, a combiner 130, an in-loop filter 134, a motion estimator 138, an intra-frame / inter-frame mode selector 140, an inter-frame predictor 142 and an intra-frame predictor 144.

[0006] As reference Figure 1The described video codec 100 compresses and encodes a picture 102 of a video sequence as follows. The picture 102 to be encoded is partitioned into blocks or CUs 104. Each codec unit 104 is encoded using an intra or inter codec mode. When a CU is encoded in intra mode, intra prediction is performed by an intra predictor 144. Intra prediction includes predicting the current CU 114 being encoded with the help of coded, decoded and reconstructed picture samples located around the current CU (e.g., the top and left of the current CU). Intra prediction is performed in the spatial domain. In inter mode, motion compensation and motion estimation are performed by a motion estimator 138, which searches for a CU that is a good predictor of the current CU in one or more reference pictures provided by a picture buffer 136 and used to predictively encode and decode the current picture. For example, a good predictor of the current CU is a predictor similar to the current CU, that is, the distortion between the two CUs is low or below a certain threshold. Motion estimation can also consider signaling to the predictor to optimize the rate cost of the rate-distortion trade-off. The output of the motion estimation step is one or more motion vectors and reference indices associated with the current CU. Motion compensation is then used to predict the current CU with the aid of one or more motion vectors and reference picture indices determined by the motion estimator 138. Basically, the block or CU contained in the selected reference picture and pointed to by the determined motion vector is used as the prediction block of the current CU. The encoder 100 selects one of the intra-frame coding mode or inter-frame coding mode for encoding the CU with the aid of the selector 140 and indicates the intra / inter decision, for example, with the aid of a prediction mode flag. The prediction residual 110 is then transformed and quantized by blocks 116 and 118, and the quantized transform coefficients as well as the motion vectors and other syntax elements are entropy encoded and written to the output bitstream 120. The encoder 100 can skip the transform stage 116 and directly apply quantization to the untransformed residual signal 110 in a so-called transform-skip coding mode. After encoding a block or CU, the encoder decodes the CU and reconstructs it to obtain a reconstructed signal 132, which can be used as reference data for predicting future CUs or blocks to be encoded. The quantized transform coefficients 110' are dequantized and inverse transformed, resulting in a decoded predicted CU or block 110", and the decoded prediction residual and the predicted block are then combined (usually added) at 130 to provide a reconstructed block or CU 132. An in-loop filter 134 is applied to the reconstructed picture to reduce compensation artifacts. For example, a deblocking filter, a sample adaptive offset (SAO) filter, and an adaptive loop filter (ALF) can be applied to reduce coding artifacts. The filtered picture is stored in a buffer 136 (also called a decoded picture buffer (DPB)) so that it can be used as a reference picture for encoding and decoding subsequent pictures.

[0007] Figure 21 is a block diagram of a video decoder 150 for predictively decoding a picture or video provided at an output 154 of the decoder 150 from a data stream 152. The decoder 150 includes an entropy decoder 156, a partition block 158, an inverse quantizer 160, an inverse transformer 162, a combiner 164, an in-loop filter 166, an optional post-decoding processor 168, and a prediction module 170. The prediction module 170 includes a decoded picture buffer 180, a motion compensator 182, and an intra-frame predictor 184.

[0008] The encoded pictures of the video sequence are decompressed and decoded by the decoder 150 as follows. The input bitstream 152 is entropy decoded by the decoder 156, which provides, for example, block partition information, the codec mode for each codec unit, the transform coefficients contained in each transform block, prediction information (such as intra-frame prediction mode), motion vectors, reference picture indexes, and other codec information. The block partition information indicates how the picture is partitioned, and the decoder 150 can divide the input picture into codec tree units (CTUs) (usually 64×64 or 128×128 pixels in size) and divide each CTU into rectangular or square codec units (CUs) according to the decoded partition information. The entropy decoded quantized coefficients 172 are dequantized 160 and inverse transformed 162 to obtain a decoded residual picture or CU 174. The decoded prediction parameters are used to predict the current block or CU, that is, whether the predicted block is obtained by its intra-frame prediction or by its motion compensated temporal prediction. The prediction process performed on the decoder side is the same as the prediction process performed on the encoder side. The decoded residual block 174 is added to the predicted block 176, thereby producing a reconstructed current image block 164. The in-loop filter 166 is applied to the reconstructed picture or image, which is also stored in the decoded picture buffer 180 for use with reference pictures for future picture decoding. As mentioned above, the decoded picture may also be subjected to post-decoding processing, such as for performing an inverse color transform, such as a conversion from YCbCr4:2:0 to RGB 4:4:4.

[0009] As mentioned above, the intra-frame prediction mode to be used for decoding can be carried in the bitstream provided by the encoder and received by the decoder, but, according to other methods, instead of introducing the actual intra-frame prediction mode into the bitstream, the intra-frame prediction mode can be derived by gradient analysis using reconstructed pixels adjacent to the currently processed CU. In other words, the intra-frame prediction mode is not explicitly indicated in the bitstream, but is implicit. This method is called decoder-side intra-frame mode derivation (DIMD), which can be signaled using a simple flag and then the actual intra-frame mode is derived during the reconstruction process, such as the reconstruction process performed by the prediction block 124 / 170 of the encoder 100 or decoder 150. The encoder 100 can encode information whether DIMD is used for the current CU into the bitstream 120, and in the case where DIMD is not used, the actual intra-frame mode is signaled by the encoder and parsed from the bitstream by the decoder.

[0010] Taken from reference [5] Figure 3 An example of an intra-block general reconstruction process is illustrated, which may be applied at the encoder 100 or at the decoder 150. During the parsing process 200, intra-block decoding is determined, as indicated at block 202, and pixel residual parsing 204 is performed, as well as DIMD flag parsing 206. If the DMID flag 206 is zero, then intra-prediction mode parsing 208 is performed to obtain an intra-mode 210 to be used when predicting and reconstructing pixels during a reconstruction process 212, as indicated at 216. If the DIMD flag 206 is set to one, then intra-prediction derivation 218 is performed during the reconstruction process 212, which produces a derived intra-mode 220, which is used in the reconstruction process together with the pixel residual 214 at 216.

[0011] Further details of the DIMD method are described in references [5] to [8]. Generally speaking, DIMD processing is based on a reconstructed template area adjacent to the currently processed CU, the width and height of which are three samples wide. The template area includes a left area, an upper area, and an upper left area. These areas are also referred to as the left template area, the upper template area, and the upper left template area in the following text. In the template area, a corresponding edge detection filter (such as a 3×3 horizontal and vertical Sobel filter) is applied to determine the amplitude and angle of the brightness direction or orientation of each midline sample of the template area. A histogram of gradients (HoG) is calculated, and each entry corresponds to a conventional intra-angle pattern and the accumulated intensity is stored:

[0012] angle=arctan(G hor / G ver )

[0013] amplitude=|Ghor |+|G ver |

[0014] Among them G hor and G ver is the pure horizontal and vertical intensity calculated by the Sobel filter. Figure 4 The DIMD template blocks or regions used to calculate the HoG are illustrated. Figure 4 (a) illustrates a partially reconstructed picture 250, as indicated by a reconstructed block 252 and an unavailable block 254. In other words, the reconstructed block 252 includes an already reconstructed codec unit or block of the picture to be reconstructed, while the corresponding codec unit or block in the unavailable block 254 remains to be reconstructed. A currently processed CU or block 256 and the above-mentioned template block 258 are illustrated. Figure 4 (b) illustrates that the template block 258 is 3 samples or pixels wide in width and height, and as described above, a 3×3 horizontal and vertical Sobel filter 259 is used to calculate the HoG 260 to obtain an angular intra prediction mode (also called intra prediction mode, IPM) based on the amplitude and angle of the luma direction of each middle line sample 262 of the template block 258, and to obtain the associated cumulative strength or amplitude for each IPM in the template block. The two most representative IPMs (in Figure 4 (b) indicates M 1 and M 2 ), and use the ratio of their cumulative amplitudes for M 1 and M 2 The two most representative IPMs are combined with the Planar mode using a weight of and a fixed weight for the Planar mode (e.g., 1 / 3, i.e., 21 / 64 with 6-bit integer precision).

[0015] Taken from reference [8] Figure 5 The diagram shows the 1 、M 2 Assuming a square CU 256, the corresponding pixels of CU 256 are represented by the two most representative IPM M taken from HoG 260. 1 、M 2 Make a prediction to obtain the predictor block Pred 1 、Pred 2 and Pred 3 The corresponding predictor block is represented by weights w 1 、w 2 、w 3The DMID predictors are weighted and fused or combined into a final predictor block comprising the DMID predictor for corresponding samples or pixels in the CU 256 .

[0016] Figure 6 An example of a conventional DIMD predictor derivation process is illustrated. At S100, using the input CU 256, the neighborhood of the input CU 256 is analyzed to determine the number of available upper and left inner samples. At S102, the HoG 260 is constructed for the template block 258, that is, based on the available samples, the angle intra prediction mode IPM is determined by its cumulative amplitude in the HoG 260 (step S104). At S106, it is determined how many IPMs are selected, that is, whether no IPM is selected, whether one IPM is selected, or whether two IPMs are selected. If no IPM is selected, then only the Planar mode (which can be the default mode) is applied, as indicated at S108, to obtain the DIMD predictor. If only one IPM is selected, then the selected IPM is used to obtain the DIMD predictor, as indicated at S110. If two IPMs are selected, then at S112, the mixing weight of each selected mode is determined according to its cumulative amplitude ratio, and the weight for the Planar mode is set to 1 / 3. At S114, a predictor block Pred is calculated for each selected IPM and Planar mode. 1 To Pred 3 , and at S116 a DIMD predictor is obtained by mixing the two selected IPM and Planar modes using the determined mixing weight.

[0017] Figure 7 An example of a currently processed CU 256 and an associated template block 258 of width / height T=3 samples or pixels is illustrated. For each intermediate line sample 262, the corresponding orientation is illustrated by the direction of the corresponding arrow, and the corresponding strength of the gradient is indicated by the length of the arrow. The HoG is generated by accumulating the strength of each orientation / IPM entry, and the two most significant modes in the HoG (if they exist) are selected as the modes to be mixed with the Planar mode or the planar intra prediction mode. Figure 7 In the example of , IPMs 5 and 28 have the first and second highest cumulative strengths / amplitudes. As described above, these modes are weighted, and the blending weights are conventionally fixed and / or determined according to the ratio of the cumulative strengths / amplitudes of the selected IPMs. Although different blending modes are proposed in the prior art, all blending modes are global with respect to the currently processed block or CU. More specifically, when considering conventional methods and when selecting two IPMs (such as M 1 and M 2 ), the mixing weight can be calculated as follows

[0018]

[0019] where w 1 With IPM M 1 Related, w 2 With IPM M 2 is associated, and w 3 Associated with Planar mode. Different modifications of the blending process are described in the prior art, for example, reference

[10] proposes selecting multiple blending modes based on rate-distortion optimization, reference

[11] proposes using HoG to implicitly derive two of the three blending modes, reference

[13] proposes using template matching to implicitly derive the blending mode, and reference

[20] proposes selecting different blending modes based on multiple DIMD modes determined. When DIMD is applied, one or two IPMs are derived from reconstructed neighboring samples (i.e., from the template block), and if two intra modes are derived, they are combined (i.e., blended) with the Planar mode predictor. Conventional solutions always implement a global or CU-level weighted blending of the two IPMs with the Planar mode.

[0020] Conventionally, in the reconstructed portion 252 of the picture 250, three columns and rows of reconstructed samples or pixels are used for DIMD processing, i.e., the HoG 260 is calculated using a filter having a fixed filter window of 3×3 samples and located at the same position in the template block 258. In reference [6], when calculating the gradient on the pixel located in the left column, top row or upper left sample and directly adjacent to the current CU, the 3×3 Sobel filter 259 can be replaced by a 3×2 filter. In reference [8], instead of using the 3×3 Sobel filter 259 on all pixels or samples in the middle line of the template block 258, the 3×3 Sobel filter 259 can be applied more sparsely, for example, only at one middle line sample in the left template block and one middle line sample in the upper template block. Further aspects of the DIMD processing are described, for example, in the following references:

[0021] - References

[10] ,

[11] ,

[13] and

[20] discuss the mixing of modes,

[0022] - References

[16] and

[18] discuss behavior correction (error repair) relative to the original intent,

[0023] - References [9],

[12] and

[14] discuss combinations with other modes,

[0024] - References

[15] ,

[17] ,

[19] or

[21] discuss syntax or simplifications for DIMD processing.

[0025] However, the above reference Figure 6 The basic DIMD processing remains the same as described in reference [8].

[0026] The conventional hybrid process for obtaining the final intra-prediction block has disadvantages because it uses weights that are globally applicable to the entire block or CU currently being processed, so the weights do not take into account local characteristics of the IPM, such as the orientation and strength of the IPM in the reconstructed template block, which results in a less faithful intra-predictor. Another disadvantage is that if only one IPM is selected, the predictor is not combined with the Planar mode or DC mode predictor, i.e., it is the same as the conventional intra-prediction mode. However, this mode is encoded as a DIMD mode, so an additional computational overhead is generated when compared to the conventional intra-prediction mode, because the HoG is required to be calculated, which may ultimately result in only a single IPM to be used.

[0027] Therefore, there is a need to provide further improvements in DIMD processing. Summary of the invention

[0028] The present invention provides a method for deriving a decoder-side intra mode derivation (DIMD) predictor for corresponding samples of a coding unit (CU) of a picture, the method comprising:

[0029] Select one or more intra prediction modes (IPM) in the template blocks adjacent to the CU.

[0030] determining a blending weight for blending at least one or more selected IPMs, and

[0031] The DIMD predictor is generated by blending one or more selected IPMs using a blending weight determined for the CU such that the blending varies across the CU.

[0032] Optionally, the method comprises:

[0033] Split a CU into two or more CU regions,

[0034] wherein, for each CU region of at least a subset of the two or more CU regions, a blending weight is determined based on the presence of one or more selected IPMs in a portion of the template block adjacent to the corresponding CU region, and

[0035] The DIMD predictor is generated using hybrid weights determined for each CU region.

[0036] Optionally, one or more IPMs are selected using IPM statistics (such as histogram of gradients, HoG) determined globally over the entire template patch, the template patch comprising a plurality of template patch regions, wherein

[0037] - perform the selection globally for the entire CU, or

[0038] - The selection involves a global selection of the set of IPMs for the entire CU, followed by a further per-region selection of the set of IPMs for each CU region.

[0039] Optionally, one or more IPMs are selected using IPM statistics (e.g., histogram of gradients, HoG), which are determined separately on

[0040] - each of a plurality of template block regions in the template block, the plurality of template block regions comprising a left template block region and an upper template block region, or

[0041] - each of a plurality of partial template block regions of the template block, the partial template block region being a template block region adjacent to only one of the CU regions, and

[0042] The selection involves a per-region selection from an IPM determined separately for each CU region.

[0043] Optionally, determining the blending weights includes

[0044] - for a CU region positioned adjacent to a template block, determining a blending weight according to the presence of one or more selected IPMs in one or more template block regions or parts of template block regions adjacent to the CU region, and

[0045] - for CU regions that are not located adjacent to the template block,

[0046] ○ Select only Planar mode as DIMD predictor, or

[0047] ○ Select only DC mode as DIMD predictor, or

[0048] o Determine blending weights using one or more selected IPMs in one or more or all template block regions, or

[0049] ○ The blending weight is determined by weighting the blending weights of the CU regions adjacent to the template block region.

[0050] Optionally,

[0051] - selecting one or more IPMs in a manner conscious of subdividing the template block into a plurality of template block regions, the plurality of template block regions comprising a left template block region and an upper template block region,

[0052] - The template block also includes an upper left template block area, which is allocated

[0053] ○ to the upper template block area, or

[0054] ○ to the left template block area, or

[0055] ○ to both the upper template block area and the left template block area, or

[0056] ○ as additional template block areas, and / or

[0057] - selecting one or more IPMs in a manner that is aware of the subdivision of the template block into a plurality of partial template block areas,

[0058] - When defining two or more CU regions, a partial template block region is defined by splitting the template block with the same vertical and / or horizontal lines as the CUs.

[0059] Optionally, one or more first IPMs are selected in a template block area of ​​the template block, and one or more second IPMs are selected in a partial template block area of ​​the template block, and wherein

[0060] - if the CU has a rectangular shape, then the first IPM is selected in the template block area adjacent to the longer dimension of the CU, and the second IPM is selected in the partial template block area adjacent to the shorter dimension of the CU, or

[0061] - Selecting the first IPM and the second IPM in the template block area or from a portion of the template block area adjacent to the CU depends on the direction of the peak IPM in the template block area or in the entire template block, for example,

[0062] o If the main direction of the peak IPM calculated in the template block area or in the entire template block is more vertical than horizontal, then select the first IPM in the left or upper template block area and optionally in the upper left template block area adjacent to the CU area, and select the second IPM in the partially upper or left template block area adjacent to the CU area, and

[0063] ○ If the main direction of the peak IPM calculated in the template block area or in the entire template is more horizontal than vertical, then the first IPM is selected in the upper or left template block area and optionally in the upper left template block area adjacent to the CU area, and the second IPM is selected in the partial left or upper template block area adjacent to the CU area.

[0064] Optionally,

[0065] If two or more IPMs are selected, at least some but not all of the IPMs are selected in the entire template block, and the remaining IPMs are selected in the template block area or from a portion of the template block area adjacent to the CU area, or

[0066] If two IPMs are selected, the first IPM in the entire template block is selected and

[0067] If there is no second IPM from the template block area or part of the template block area adjacent to the CU area, then

[0068] ○ The first IPM is used as a DIMD predictor, or

[0069] ○ Mix the first IPM with Planar or DC mode using predefined weights to obtain a DIMD predictor,

[0070] - If there is a second IPM selected in the template block area or a portion of the template block area adjacent to the CU area, then

[0071] o If the second IPM is different from the first IPM, then mix the first IPM with the second IPM to obtain a DIMD predictor,

[0072] o If the second IPM is equal to the first IPM, then a further IPM in a template block area adjacent to the CU area is selected and the first IPM is blended with the further IPM to obtain a DIMD predictor.

[0073] Optionally, for a CU region that is not located adjacent to any template block region,

[0074] - use only the first IPM as a DIMD predictor, or

[0075] - Use only Planar mode as DIMD predictor, or

[0076] - Use only DC mode as a DIMD predictor, or

[0077] - Mix the first IPM with Planar or DC mode using predefined weights to obtain a DIMD predictor.

[0078] Optionally,

[0079] If more than one IPM is selected, then

[0080] - if an IPM exists in a template block area adjacent to the CU area, then the blending weights associated with the IPM are used,

[0081] - if only one of the IPMs is present in the template block area adjacent to the CU area, then using the blending weight associated with the one IPM or the weighted blending weight associated with the one IPM,

[0082] And set the blending weight associated with any other IPM to a predefined value, such as 0,

[0083] - optionally, if no IPM exists in the adjacent template block area of ​​the CU area, then setting the blending weight associated with the IPM to a predefined value, such as 0, and

[0084] - Optionally, set the blending weights associated with Planar or DC mode to predefined values, or

[0085] If you choose an IPM, then

[0086] - if the one IPM exists in a template block region adjacent to the CU region, then use the blending weight associated with the one IPM or the weighted blending weight associated with the one IPM, and set the blending weight associated with any other IPM to a predefined value, such as 0,

[0087] - optionally, if the one IPM does not exist in the template block area adjacent to the CU area, then setting the blending weight associated with the one IPM and the blending weights associated with any other IPM to a predefined value, such as 0, and

[0088] - Optionally, set the blending weights associated with Planar or DC mode to predefined values, or

[0089] If no IPM is selected, then the blending weights associated with any other IPM are set to a predefined value, such as 0, and optionally, the blending weights associated with Planar or DC mode are set to a predefined value.

[0090] Optionally,

[0091] - for a CU region located adjacent to a first template block region for which no IPM is selected and away from a second template block region for which one or more IPMs are selected, the blending weight is a weighted blending weight of CU regions adjacent to the second template block region, or

[0092] For a CU region located adjacent to the first template block region for which the first IPM is selected and away from the second template block region for which the second IPM is selected, the mixing weight for the second IPM is the weighted mixing weight of the second IPM for the CU region adjacent to the second template block region.

[0093] The present invention provides a non-transitory computer-readable storage medium including instructions, which, when executed by a computer, cause the computer to perform the method of the present invention.

[0094] The present invention provides an apparatus for deriving a decoder-side intra mode derivation (DIMD) predictor for corresponding samples of a coding unit (CU) of a picture, the apparatus comprising:

[0095] An intra prediction mode (IPM) selection module is configured to select one or more IPMs in a template block adjacent to the CU,

[0096] a mixing weight determination module configured to determine a mixing weight for mixing at least one or more selected IPMs, and

[0097] A DIMD predictor generation module is configured to generate a DIMD predictor by blending one or more selected IPMs using a blending weight determined for the CU such that the blending varies across the CU.

[0098] The present invention provides a device for decoding a coded data stream, wherein the coded data stream includes data representing a picture, and the device includes:

[0099] a decoder module configured to decode pictures from the encoded data stream, and

[0100] A prediction module comprises means for deriving a decoder-side intra mode derivation (DIMD) predictor for corresponding samples of a coding unit (CU) of a picture according to the present invention.

[0101] The present invention provides an apparatus for encoding a picture into a coded data stream, the coded data stream comprising data representing the picture, the apparatus comprising:

[0102] an encoder module configured to receive an original picture and encode the picture into an encoded data stream, and

[0103] A prediction module comprises means for deriving a decoder-side intra mode derivation (DIMD) predictor for corresponding samples of a coding unit (CU) of a picture according to the present invention.

[0104] The technical solution provided according to the embodiment of the present invention has the following beneficial effects.

[0105] In prior art methods, blending weights are selected for the entire CU currently being processed or predicted, i.e., conventional methods focus on a global or CU level approach when determining blending weights for a selected IPM. However, the signal characteristics of an IPM may vary across template blocks, and thus the selected weights may be more suitable for obtaining a DIMD predictor for samples in one CU region than for obtaining a DIMD predictor for samples in another CU region. In other words, the weights may not take into account local characteristics of the IPM, such as the orientation and strength of the IPM in the reconstructed template block, which results in a less faithful intra predictor.

[0106] The present invention solves the above disadvantages by no longer using a fixed blend for a CU, but providing blending of one or more selected IPMs using blending weights determined for the CU in a manner that varies across the CU. Thus, the blending process generates an intra predictor that more faithfully fits the local orientation of the CU to be predicted, thereby improving prediction accuracy.

[0107] According to an embodiment of the present invention, hybrid weights are selected by region based on the proximity of the region of the currently processed CU to a template block (such as a corresponding template block or a template block region). The overall template block is used to determine the IPM from IPM statistics (such as HoG), and a weight map by region can be generated for the CU to be predicted as a result of processing according to an embodiment of the present invention. This map can be used during hybrid processing to generate an intra-frame predictor that more faithfully fits the local orientation of the CU to be predicted, thereby improving prediction accuracy. The embodiment provides improved prediction accuracy by making full use of the HoG of the prior art, so that except for a few additional computing operations used only to implement the selection of hybrid weights by region, no substantial changes are required to the existing DIMD processing, so the method of the present invention can be easily incorporated into the existing DIMD processing.

[0108] A further disadvantage of the conventional approach is that if only one IPM is selected, the predictor is not combined with the Planar mode or DC mode predictor, i.e. it is identical to the conventional intra prediction mode. However, this mode is coded as DIMD mode and therefore generates additional overhead compared to the conventional intra prediction mode, since the calculation of the HoG is required, which may ultimately result in only a single IPM to be used. The present invention is advantageous because, according to an embodiment, when deriving a single IPM by combining a single IPM with Planar or DC mode using the region-by-region mixing weights determined for a selected IPM, the region-by-region mixing weights are also used, thereby also utilizing the DIMD mode for this scenario.

[0109] It should be understood that the content described in this section is not intended to identify key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become apparent from the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0110] The drawings are illustrative and serve to explain the invention and are not to be construed as limiting the invention to the illustrated embodiments.

[0111] Figure 1 is a block diagram of a conventional video encoder;

[0112] Figure 2 is a block diagram of a conventional video decoder;

[0113] Figure 3 The diagram shows the application Figure 1 The encoder or Figure 2 An example of a generic intra-block reconstruction process at a decoder of;

[0114] Figure 4 illustrates a DIMD template block or region used to calculate a histogram of gradients (HoG) in a DIMD process;

[0115] Figure 5 The diagram shows the prediction fusion by weighted averaging of the selected IPM and Planar modes;

[0116] Figure 6 An example of a conventional DIMD predictor derivation process is illustrated;

[0117] Figure 7 An example of a currently processed 256 and associated template block with a width / height of samples or pixels is illustrated;

[0118] Figure 8 A flowchart of a method for deriving a DIMD predictor for corresponding samples of a CU according to an embodiment of the present invention is illustrated;

[0119] Fig. 9 The diagram illustrates splitting a current CU into a plurality of CU regions associated with different mixing weights according to an embodiment of the present invention;

[0120] Fig.10 illustrates a DIMD predictor derivation process according to an embodiment of the present invention;

[0121] Fig.11 A schematic diagram illustrating selecting a mixing weight according to the proximity of a certain CU region to a certain template block region according to an embodiment of the present invention is illustrated;

[0122] Fig.12 illustrates hybrid weight allocation according to an embodiment of the present invention;

[0123] Fig.13 illustrates a flow chart outlining the determination of region blending weights according to an embodiment of the present invention;

[0124] Fig.14 illustrates hybrid weight allocation according to an embodiment of the present invention;

[0125] Fig.15 The use of a blending weight weighted according to the distance between a certain CU region and a template block region including a selected IPM is illustrated;

[0126] Fig.16 The diagram shows the per-template block area calculation of HoG;

[0127] Fig.17 An embodiment is illustrated in which a first IPM is selected based on the entire HoG and a second IPM is selected based on the HoG for an adjacent template block;

[0128] Fig.18 illustrates a flow chart according to an embodiment of the present invention;

[0129] Fig.19 An embodiment of partially using a template block to calculate a partial template block HoG to be used in deriving a local mixing weight is illustrated;

[0130] Fig. 20 An embodiment is illustrated in which a mixture of partial and full / entire template block regions is used to calculate the HoG to select one or more IPMs;

[0131] Fig.21 A block diagram of an apparatus for deriving a DIMD predictor according to an embodiment of the present invention is illustrated;

[0132] Fig. 22 A block diagram of a decoder according to an embodiment of the present invention is illustrated;

[0133] Fig.23 A block diagram of an encoder according to an embodiment of the present invention is illustrated; and

[0134] Fig.24 A block diagram of an electronic device 900 according to an embodiment of the present invention is illustrated. DETAILED DESCRIPTION

[0135] Illustrative embodiments of the present invention will be described below with reference to the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding and should be regarded as illustrative only. Thus, those of ordinary skill in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present invention. Moreover, for the sake of clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0136] In the present invention, the term “and / or” is intended to cover all possible combinations and sub-combinations of the listed elements, including any one, any sub-combination or all of the elements listed individually, and does not necessarily exclude additional elements.

[0137] In the present invention, the phrase "at least one of... or..." is intended to cover any one or more of the listed elements, including any one, any subcombination or all of the elements listed individually, without necessarily excluding any additional elements, and without necessarily requiring all elements.

[0138] In the present invention, the term "codec" refers to "encoding" or "decoding", as is apparent from the context of the described embodiments. Likewise, the term "codec" refers to "encoder" or "decoder".

[0139] Figure 8 A flowchart of a method for deriving a decoder-side intra mode derivation (DIMD) predictor for corresponding samples of a coding unit (CU) of a picture according to an embodiment of the present invention is illustrated. The method comprises the following steps:

[0140] S200: Select one or more intra prediction modes (IPMs) in a template block adjacent to the CU.

[0141] S202: Determine a mixing weight for mixing at least one or more selected IPMs.

[0142] S204: Generate a DIMD predictor by blending one or more selected IPMs using a blending weight determined for the CU such that the blending varies across the CU.

[0143] According to an embodiment, mixing of one or more selected IPMs is achieved by region-by-region (i.e., by separate regions of a CU) selection of mixing weights such that the mixing varies across the CU. The CU is split into two or more CU regions, and a mixing weight is determined for each CU region in at least a subset of the two or more CU regions based on the presence of one or more selected IPMs in a portion of a template block adjacent to the corresponding CU region. A DIMD predictor is then generated using the mixing weights determined for each CU region. In other words, an embodiment of the present invention provides a method for deriving a DIMD predictor for samples of a currently processed CU using mixing weights determined by CU region to mix (one or more) IPM / Planar or DC modes, rather than using weights determined only for the entire CU.

[0144] According to an embodiment of the present invention, the currently processed CU predicted using DIMD processing is split into rectangular or square CU regions. The CU regions may have the same size and may be split horizontally and / or vertically. Therefore, according to an embodiment, the corresponding CU region may have a rectangular or square shape. It should be noted that a hybrid weight is selected and applied for each region. Regarding the splitting operation, it should be noted that the segmentation is not related to the CU partition, but only defines two or more regions within the CU and assigns corresponding hybrid weights to such CU regions. The actual CU splitting operation may be performed before or after the HoG construction.

[0145] Fig. 9 It illustrates splitting the current CU 256 into multiple CU regions associated with different blending weights according to an embodiment of the present invention. Fig. 9 (a) illustrates the currently processed CU 256, the template block 258, and the corresponding samples or pixels in the template block 258, which are filtered using a filter with a 3×3 filter window 280, which is centered on the middle line sample 262 of the template block 258, as described in the conventional method above. The template block 258 includes a left template block or left template block area 258a, an upper template block or upper template block area 258b, and an upper left template block or upper left template block area 258c. The boundaries between the template block areas are indicated by the dotted lines in the template block 258. The currently processed CU 256 is split into four CU areas or mixed weight areas, as indicated by the dotted lines in the CU 256, resulting in a first or upper left (TL) CU area 256a, a second or upper right (TR) CU area 256b, a third or lower left (BL) CU area 256c, and a fourth or lower right (BR) CU area 256d. Fig. 9 (b) illustrates another embodiment of the present invention, according to which the currently processed CU 256 is split into two CU regions, resulting in a first or left (L) CU region 256a and a second or right (R) CU region 256b. Fig. 9 In (a), CU 256 is split into four rectangular regions, while Fig. 9 In (b), CU 256 is split into two square-shaped CU regions.

[0146] As in conventional DIMD processing, a HoG is constructed for the template block 258, and according to an embodiment of the present invention, an IPM associated with each filter window position is stored. It should be noted that the present invention is not limited to determining IPM statistics based on the HoG, and selecting one or more IPMs on this basis. According to other embodiments, other methods can be applied to determine the IPM statistics, for example, a Hough transform or a gradient-based Hough transform can be used on the filter window size to determine the direction of each center pixel (of the filter window), and then the histogram of the found directions can allow the selection of the peak direction (and the matching IPM). As in the prior art method, two or more IPMs are selected from the HoG, and finally the mixing weights are determined for each CU area. Fig.10 The DIMD predictor derivation process according to an embodiment of the present invention is illustrated. At S220, the neighborhood of the currently processed CU256 is analyzed to determine the number of available upper inner and left inner samples or pixels, based on which a HoG is constructed at S222. In addition, the IPM and associated amplitude for each filter window position are stored. At S224, two or more of the most representative IPMs in the HoG in terms of amplitude are determined. In reference Fig.10 In the described embodiment, two of the most representative IPMs are determined, and it is determined at S226 whether 0, 1, or 2 IPMs are selected. If no IPM is selected, then Planar or DC intra prediction mode (which may be the default mode) is used as the DIMD predictor, as indicated at S228. If one IPM is selected, then according to an embodiment, the selected IPM is used as the DIMD predictor, as indicated at S230.

[0147] If it is determined at S226 that two IPMs are selected, the currently processed CU is split into a plurality of CU regions or CU mixed weight regions, as indicated at S232. For example, the CU may be divided into a plurality of CU regions or CU mixed weight regions as indicated at S232. Fig. 9 The CU region is split in the manner described. At S234, the mixing weight for each region is determined using the cumulative amplitude of those IPMs present in the template block region adjacent to the CU region. At S236, a prediction for each selected IPM is calculated, and at S238, a DIMD predictor is obtained by mixing the two selected IPMs with the Planar or DC mode by region using the determined regional mixing weight. According to other embodiments, as mentioned above, step S232 can be performed before constructing the HoG at step S222.

[0148] According to other embodiments, when only one IPM is selected, a corresponding regional mixing weight can be determined, and, instead of using the selected IPM as a DIMD predictor, a DIMD predictor is obtained by mixing the selected IPM with a Planar mode or a DC mode using the determined regional mixing weight.

[0149] First aspect: deriving regional blending weights using weights calculated for the entire template patch

[0150] According to an embodiment of the first aspect of the present invention, after calculating the HoG, each CU region is attached or associated with at least one adjacent template block (e.g., above, left and / or above-left template blocks). The mixing weight of a specific CU region is selected based on whether an IPM associated with the globally calculated mixing weight exists in a template block region adjacent to the CU region. Fig.11 The diagram illustrates selecting a blending weight according to an embodiment of the present invention based on the proximity of a specific CU region relative to a specific template block region. As shown in the figure, the current CU 256 is split into four square regions, namely, a TL region, a TR region, a BL region, and a BR region. Of course, the currently processed CU 256 may also be split into rectangular regions, and instead of splitting the CU 256 into four regions, it may be split into fewer regions, such as Fig. 9 As shown in (b), or split into more rectangular or square regions.

[0151] According to an embodiment, splitting the CU 256 may depend on the size of the currently processed CU, so, for example, a CU with a size of 16×8 samples may be split into two square-shaped regions, each with a size of 8×8 samples. According to a further embodiment, the splitting of the CU 256 may only be applied if the size of the currently processed CU is above a predefined threshold, for example, for a CU with a size of at least 8×8 or 16×8 or 8×16.

[0152] According to reference Fig.11In the described embodiment, the corresponding CU region uses the global weights associated with the IPMs present in the adjacent template block region. The statistics (such as cumulative amplitude) of the IPMs present in the corresponding template block region are calculated over the entire HoG. In other words, the CU region does not modify the weights calculated based on the global statistics, but selects the weights based on the presence of matching IPMs in the adjacent template block region. For example, for the upper left CU region, the mixed weights associated with the IPMs present in the upper and left template block regions 258a, 258b, and optionally in the upper left template 258c block are selected. For the upper right CU region, only the mixed weights associated with the IPMs present in the upper template block region 258b are selected. Similarly, for the lower left CU region, only the mixed weights associated with the IPMs present in the left template block region 258a are selected. For the lower right CU region that is not adjacent to the template block 258, the weights associated with any mode other than Planar or DC mode are set to zero, that is, Planar mode / DC mode is used as the predictor for the lower right CU region.

[0153] According to an embodiment, the per-region blending weights may be selected as follows:

[0154] Initially, the IPM to be blended is selected using, for example, HoG, and if more than one IPM is selected, the per-region blending weights may be set according to one or more of the following:

[0155] (R1.1) If an IPM exists in a template block region adjacent to the CU region, then use the blending weights associated with the IPM,

[0156] (R1.2) if there is only one of the IPMs in the template block area adjacent to the CU area, then using the blending weight associated with the one IPM or the weighted blending weight associated with the one IPM,

[0157] And set the blending weight associated with any other IPM to a predefined value, such as 0,

[0158] (R1.3) If there is no IPM in the adjacent template block area of ​​the CU area, then the blending weight associated with the IPM is set to a predefined value, such as 0.

[0159] (R1.4) The blending weights associated with Planar or DC mode are set to predefined values.

[0160] If you select an IPM, you can set blending weights by region based on one or more of the following:

[0161] (R2.1) If the one IPM exists in a template block region adjacent to the CU region, then the blending weight associated with the one IPM or the weighted blending weight associated with the one IPM is used, and the blending weights associated with any other IPM are set to a predefined value, such as 0,

[0162] (R2.2) If the one IPM does not exist in the template block area adjacent to the CU area, then the blending weight associated with the one IPM and the blending weight associated with any other IPM are set to a predefined value, such as 0.

[0163] (R2.3) The blending weights associated with the Planar or DC modes are set to predefined values, or

[0164] If no IPM is selected, then the blending weights associated with any other IPM are set to a predefined value, such as 0, and / or the blending weights associated with Planar or DC mode are set to a predefined value.

[0165] In addition, more detailed embodiments for determining or setting the mixing weights by region will now be described, but the present invention is not limited to these embodiments. According to the embodiment, when considering selecting at most two IPM M 1 and M 2 The mixing weights w can initially be calculated by conventional processing 1 ,w 2 ,w 3 , as referenced above Figure 6 Described as follows:

[0166]

[0167] When modes M1 and M2 are selected as the two most representative angular intra prediction modes in the HoG for the entire template block 258, the hybrid weight for each CU region is selected according to the presence of modes M1 or / and M2 in the adjacent template block regions (i.e., the above, upper left, and / or left template block regions). The i-th hybrid weight of region r is denoted as w i,r , the blending weight for each CU region is chosen as follows:

[0168] (R1.1) If M1 and M2 exist in adjacent template blocks of region r, then the mixing weights of region r are distributed as follows:

[0169] w 1,r =w 1 w 2,r =w 2 w 3,r =w 3

[0170] (R1.2) If there are only Mx (where x=1 or 2 and y equals another value) in the neighboring template blocks of region r, then the mixing weight of region r is distributed as follows:

[0171] w y,r =0

[0172] (R1.3) If there are only Mx (where x=1 or 2 and y equals another value) in the neighboring template blocks of region r, then the blending weights of region r are assigned as follows (the current intra prediction mode is selected because DIMD mode does not have any blending):

[0173] w x,r =1w y,r =0 w 3,r =0

[0174] (R1.4) If M1 and M2 do not exist in the adjacent template blocks of region r, the mixing weight of region r is allocated as follows:

[0175] w 1,r =0 w 2,r =0 w 3,r =1

[0176] Fig.12 The hybrid weight allocation according to an embodiment of the present invention is illustrated, assuming that IPM M is selected. 1 and M 2 , and in Fig.12 In (a), assuming that IPM M 1 exists in the left template block area 258a and the upper left template block area 258c, and IPMM 2 exists in the upper template block area 258b. On the other hand, Fig.12 In (b), assuming that IPM M 1 and M 2 exists in the upper template block area 258b and the upper left template block area 258c, but does not exist in the left template block area 258a. Therefore, when considering Fig.12 For the upper left CU region in (a), the mixing weight associated with the IPM is determined according to (R.1) above, because the IPM M 1 and M 2 exists in all template block areas adjacent to the upper left CU area. On the other hand, for the upper right and lower left CU areas, only IPM M exists in the adjacent template block areas, respectively. 2 and IPM M 1, and calculate the local or regional mixing weights according to (R.3) above, thereby generating IPM M 2 and IPM M 1 For the lower right CU area, since this area is not adjacent to any template block area, the Planar or DC intra prediction mode is selected as the DIMD predictor according to (R.4) above.

[0177] According to a further embodiment, for Fig.12 In the lower left CU region in (a), instead of applying the above (R.3), (R.2) can also be applied so that the mixed IPM M 1 Similarly, the DIMD predictor for the top right CU region can be obtained by mixing IPM M according to (R.2) above. 2 Get it with Planar or DC.

[0178] exist Fig.12 In (b), due to the two selected IPM M 1 and M 2 Adjacent to both the upper left and upper right CU regions, the local or regional hybrid weights are calculated according to (R.1) above. Since there is no IPM from the left template block region adjacent to the lower left CU region, the weights are calculated according to (R.4) above, resulting in a DIMD predictor as a Planar intra prediction mode or a DC intra prediction mode.

[0179] Fig.13 The flowchart summarizing the determination of the above-mentioned regional blending weights according to an embodiment of the present invention is illustrated. First, as indicated at S250, the current CU 256 is split into two or more regions, and as indicated at S252, the subsequent steps are performed for each of the multiple CU regions, that is, for each region r. At S254, for the currently processed region r, those blocks / regions in the template block adjacent to the region r are determined in the left and upper left template block regions 258a, 258c. At S256, M is determined. 1 Whether it exists in one or more of the template block regions adjacent to the currently processed region r. If so, then determine M at S258 2 Does it exist in one or more of the template block regions adjacent to the currently processed CU region r? If yes, then at S260, a blending weight is selected according to (R1.1) above. If it is determined at S258 that M 2If it does not exist in any neighboring block, then at S262, the hybrid weight for CU region r is calculated according to (R1.2) above. According to other embodiments, at S262, the hybrid weight for region r may be calculated according to (R1.3) above.

[0180] If M is determined at S256 1 does not exist in any template block region adjacent to region r, then it is determined at S264 that M 2 Is it present in one or more of the template block regions adjacent to the CU region and, if so, then at S266, a local blending weight for region r is determined according to (R1.2) above or according to (R1.3) above. If M is determined at S254 2 does not exist in any neighboring block, then the local mixing weight for region r is determined according to (R1.4) above.

[0181] If only one IPM M is selected from the HoG for the entire template block 258 1 , then according to the IPM M in the template block area adjacent to the corresponding CU area 1 The existence of is used to select the blending weights for each region r:

[0182] (R2.1) If M 1 If it exists in the template block area adjacent to the CU area r, the hybrid weight of area r is allocated as follows (since the DIMD mode has hybrid, the current intra prediction mode is selected):

[0183] (R2.2) If M 1 exists in the template block area adjacent to the CU area r, then the blending weight of area r is assigned as follows (since the DIMD mode does not have any blending, the current intra prediction mode is selected):

[0184] w 1,r =1w 2,r =0w 3,r =0

[0185] (R2.3) If there is no M in the template block area adjacent to the CU area r 1 , then the mixing weight of region r is assigned as follows:

[0186] w 1,r =0w 2,r =0w 3,r =1

[0187] Fig.14 The diagram shows when only one IPM M is selected 1 The hybrid weight allocation according to the embodiment of the present invention is performed. Fig.14In (a), assuming that IPM M 1 exists in the left template block area 258a and the upper left template block area 258c, but does not exist in the upper template block area 258b. Fig.14 (b) IPM M 1 It only exists in the upper left template block area 258c and the upper template block area 258b, but not in the left template block area 258a. Fig.14 (a) As can be seen, the regional weight for the upper left CU region is determined according to the above (R2.1), and the regional weight for the lower left CU region is determined according to the above (R2.2), that is, for the upper left CU region, IPM M is performed. 1 With Planar mode or DC mode, the mixing is not performed for the lower left CU area. For the upper left CU area, due to IPM M 1 does not exist in the adjacent upper template block area 258b, so the Planar mode or DC mode is selected as the DIMD predictor (R2.3). Fig.14 In (b), the local or regional blending weights for the top left and top right CU regions are determined according to (R2.1), while for the bottom left region, they are determined according to (R2.3) because IPM M 1 Not present in the left template block area 258a.

[0188] If an IPM is not selected from the HoG from the entire template block 258, the blending weight for the corresponding region r is located as follows:

[0189] w 1,r =0,w 2,r = 0 and w 3,r =1

[0190] Therefore, in this case, for all CU regions, Planar or DC intra prediction mode is used as the DIMD predictor.

[0191] According to an embodiment, if at least one of the template block regions has x If the amplitude of the sample position is greater than a predefined threshold (such as an amplitude greater than 50), then it can be considered that the selected IPM M x According to other embodiments, if the template block area is in the same region as the IPM M x The number of associated sample positions exceeds a predefined number, for example if there are more than two patterns M for which x The filtered sample position of x Present in the template block area.

[0192] According to a further embodiment, for a region adjacent to one or more template block regions from which an IPM is selected, 1 and M 2 The associated blending weights may be used for other CU regions adjacent to the template block region for which the IPM is not selected. In such further CU regions, the blending weights from the first CU region may be reduced while w 1,r +w 2,r +w 3,r =1Applicable. Fig.15 The use of blending weights weighted according to the distance between a region and a template block region including a selected IPM is illustrated. Fig.15 In the assumption that Fig.12 (b) and in accordance with Fig.15 , for the upper left and upper right CU regions, the blending weights are calculated according to (R1.1) above. Fig.12 Outside the lower left CU region in (b), instead of selecting the Planar mode as the DIMD predictor, the DIMD predictor is obtained by mixing the weighted mixing weights used for the upper right CU region, where the mixing weight w 1 and w 2 Use 1 / 2 as weight and mix weight w 3 With factor 2 as weight, w 1,BL +w 2,BL +w 3,BL =1. Optionally, this can also be applied to the lower right CU region. According to a further embodiment, by using the weighted mixing weight w from the upper left CU region 2 , which can also be used for Fig.12 The lower left CU region in (a).

[0193] Second aspect: deriving regional blending weights using the weights calculated for each template block region

[0194] In the embodiments of the first aspect of the present invention described above, the cumulative amplitude of the selected IPM stored in the HoG is used to calculate the mixing weight for the entire template block. According to an embodiment of the second aspect of the present invention, the mixing weight of each adjacent template block is calculated. According to such embodiments, instead of calculating the HoG for the entire template block, the HoG is calculated only for a portion of the template block adjacent to the CU region (e.g., the HoG is calculated for the left template block region 258a and / or for the upper template block region 258b). According to an embodiment, the upper left block region 258c can be handled as follows:

[0195] - it is assigned to the upper template block area, or

[0196] - it is assigned to the left template block area, or

[0197] - it is assigned to the upper and left template block areas, or

[0198] -It is treated as a separate template block area.

[0199] The currently processed CU 256 is split into a plurality of separate CU regions in the manner described above with reference to the embodiment of the first aspect.

[0200] When the current CU region is adjacent to several template block regions, HoG is calculated on the adjacent template blocks, and one or more IPMs are selected based on the cumulative amplitude peaks in the adjacent template blocks for which HoG is calculated. For example, for the upper left CU region adjacent to the left template block region 258a, the upper template block region 258b, and the upper left template block region 258c, the mixing weight is calculated based on the entire template block statistics (i.e., based on the HoG calculated on the entire template block 258), and therefore, the determination of the mixing weight for the upper left CU region is the same as described above with reference to the first aspect. On the other hand, the upper right CU region is adjacent only to the upper template block region 258b, and the HoG is calculated only on this upper template block region 258b. At least one IPM is selected based on the accumulated amplitude peaks, and the weight to be applied in the upper right CU region is selected based on the weight determined for the HoG calculated for the upper template block region. If only one IPM is selected, the mixing weight for the upper right CU region can be selected according to (R2.1) to (R2.3) described above with reference to the first aspect. If two IPMs are selected from the template block region adjacent to the upper right region, then the mixing weights for the upper right region are selected according to (R1.1) to (R1.4) described above with reference to the first aspect.

[0201] Fig.16 258b, for each midline sample in the region, the associated IPM is shown, and its orientation is illustrated by the direction of the corresponding arrow, while its amplitude is indicated by the length of the arrow. To determine the regional or local blending weight for the upper right region of the currently processed CU 256, the HoG is determined. Above , and in the depicted embodiment, IPMs 4, 5, and 10 are generated. Only one of IPMs 4 and 5, or both IPM5 and IPM4, is selected and used to determine the blending weight for the upper right region of CU 256. According to an embodiment, different HoGs for the left, upper, and upper left template block regions may be calculated in the manner described above with reference to the first aspect (see Fig.10For each template block region, the intra mode and amplitude of each filter window position in the corresponding template block region are stored, and one or more of the most representative IPMs can be determined in the HoG for the template block region under consideration (see Fig.10 In other words, the hybrid weight calculation is similar to that in the first aspect of the present invention, except that HoG is not performed on the entire template block, but only on one or more template block areas adjacent to the CU area for which the hybrid weight is calculated.

[0202] exist Fig.16 In the embodiment of the present invention, when two IPMs (ie, mode 4 and mode 5) are selected, the 1,TR and w 2,TR The amplitude ratio of is calculated based on the amplitude of the selected IPM relative to the HoG calculated for the upper template block area. In other words, the amplitude ratio is calculated based on the cumulative amplitude of the HoG from one or more adjacent template block areas. For the upper left area, the HoG is calculated for the upper, upper left and left template block areas. For the lower left area, the HoG is calculated only based on the left template block area 258a. For the lower right area, Planar or DC mode can be selected as the predictor. According to other embodiments, for the lower right area, a corresponding IPM can be selected, and the HoG can be calculated based on one or more IPMs from one or a combination of the upper, upper left and left template block areas.

[0203] Regarding the IPMs selected in different template block areas, it should be noted that the same or different IPMs may be selected in different template block areas, and the number of IPMs selected for each template block area may vary, i.e., the number of IPMs selected for different template block areas may be the same or different. According to an embodiment, the maximum number of IPMs selected for each template block area may be one or two.

[0204] Embodiments of the second aspect are superior to methods that use only one or more IPMs selected from the entire template block because they allow local adjustment of content characteristics because different regions of the currently processed CU are better matched using corresponding IPMs obtained from different block template regions. When using IPMs for all regions selected from the HoG for the entire template block, the selected IPMs that may exist in different template block regions may have content characteristics that are different between the corresponding template block regions, such as slightly different directions or angles. In fact, the number of possible IPMs (e.g., 67, without counting wide angle intra-modes) will produce broad peaks in the HoG, for which the selected mode matches most regions, but may not be suitable for matching the local characteristics of a specific CU region.

[0205] According to other embodiments of the second aspect, if more than one IPM is selected, a first IPM may be calculated based on the entire template block 258 (i.e., the upper, left, and upper-left template block regions), and a second IPM may be selected from a template block region adjacent to the CU region for which the hybrid weight is to be calculated. According to an embodiment, if the second IPM is different from the first IPM, the second IPM is selected. If no IPM is selected as the second IPM, the first selected IPM may be used as the DIMD predictor. On the other hand, if the second IPM is equal to the first IPM, the second best peak in the adjacent template block region HoG is assigned as the second IPM. Fig.17 An embodiment is illustrated according to which a first IPM is selected based on the entire HoG and a second IPM is selected based on the HoG for a neighboring template block. Fig.17 In the embodiment of FIG. 1 , when considering the upper right area of ​​the currently processed CU 258, as shown in the left part of the figure, according to the HoG calculated on the entire template block 258 entire Determine the first IPM and calculate the HoG based on the upper template block area only. above The second IPM is determined. In the depicted embodiment, IPM 28 is selected as the first IPM and IPM 5 is selected as the second IPM. Based on the selected IPMs, the blending weights are calculated as described above with reference to the first aspect of the invention.

[0206] Fig.18A flow chart of one embodiment of the present invention is illustrated, according to which a first angle IPM is selected from the entire template block HoG and a second IPM is selected from the HoG of the template block adjacent to the CU area. At S280, the neighborhood of the CU 256 to be processed is analyzed to determine the number of available upper and left inner samples. At S282, a histogram of gradients is constructed, more specifically, the HoG for the left, upper and upper left template block areas 258a to 258c is determined. In addition, the corresponding intra-frame mode and amplitude for each filter window position are stored. At S284, the first IPM is determined, for example, the most representative IPM in the entire HoG, and at S286, it is determined how many IPMs are selected. In the absence of IPM 1, that is, the first IPM is not selected, for example, because there is no IPM with an amplitude above a certain threshold, then the Planar or DC intra-frame prediction mode is used as the DIMD predictor, as indicated at S288. Otherwise, if the first IPM is selected, then at S290 the current CU 256 is split into multiple CUs or blending weight regions. At S292, for each region, a second IPM is determined in the HoG established for the template block region adjacent to a CU region. At S294, it is determined whether the second IPM determined at S294 is equal to the first IPM. If the two IPMs are the same, then at S296, the first IPM is used as the DIMD predictor. Otherwise, if the selected IPMs are different, the blending weights for a CU region are determined using the IPM in the template block region adjacent to the CU region for which the local or regional blending weights are calculated, as indicated at S298. At S300, predictions are calculated for each selected IPM, and at S302, a DIMD predictor is obtained by blending the two selected IPMs with the Planar mode by region using the determined regional or local blending weights. The actual mixing weights may be calculated in the manner as described above with reference to the embodiments of the first aspect of the invention (eg, according to (R1.1) to (R1.4)).

[0207] Regarding the lower right CU area without adjacent template block areas, according to an embodiment, the second IPM may also be selected from the entire template block HoG. According to other embodiments, only the first IPM is selected and used as the DIMD predictor for the lower right CU area, or only the Planar or DC mode is used as the DIMD predictor for the lower right area. According to yet other embodiments, the DIMD predictor may be obtained by mixing the Planar or DC mode with the first IPM with a weight of 0.5.

[0208] According to an embodiment, the mixing weight can be calculated based on the cumulative amplitude ratio of the entire template block HoG, and the weight of the second IPM can be greater than the weight of the first IPM. For example, for the first IPM, its cumulative amplitude ratio with the second IPM is 2 / 9, and for the second IPM, its cumulative amplitude ratio is 4 / 9, and for Planar, its cumulative amplitude ratio is 1 / 3.

[0209] According to a further embodiment, if the first IPM exists in an adjacent template block region and the second IPM also exists in the adjacent template block region, the cumulative amplitude ratio for determining the mixing weight may be calculated based on the cumulative amplitudes in the adjacent template region HoG.

[0210] According to further embodiments, if the intra-angle mode is not selected as the second IPM, the first IPM may be mixed with the Planar or DC mode using a weight of 0.5. If more than two IPMs are selected, for example three or more, the first two IPMs may be calculated from the entire template block HoG, and the third IPM may be calculated from an adjacent template block region HoG. According to other embodiments, only the first IPM may be calculated from the entire template block HoG, while further (such as second and third) IPMs may be calculated from the template block region HoG.

[0211] Aspect 3: Using weights calculated for some template blocks to derive regional blending weights

[0212] According to the third aspect of the invention, blending weights, more specifically local or regional blending weights for the corresponding CU region, are calculated for each portion of the set of neighboring samples in the template block, i.e., positions of the filter windows whose center samples are projected adjacent to the CU region of the currently processed CU using only. Conceptually, the template block 258 is split using the same vertical and horizontal lines as the currently processed CU, so that each CU region is attached to a portion of the template block, and one or more IPMs are selected from the HoG generated for this portion of the template block, hereinafter referred to as a partial template block or a partial template block region.

[0213] Fig.19 An embodiment of partially using a template block to calculate a partial template block HoG for deriving a local mixing weight is illustrated. Fig.19 (a) illustrates an embodiment according to which the IPM used to determine the blending weights for the upper right region of CU 256 is determined by a partial template block 258′, which is implemented by splitting the upper template block region 258b along the vertical dashed line that splits CU 256. For the partial template block 258′, the HoG is determined, referred to as HoG in the figure. AboveRightIn the depicted embodiment, the HoG generates IPM 4 and IPM 5, and for the corresponding midline samples, the direction is indicated by the arrow and the amplitude is indicated by the length of the arrow. The HoG shows the cumulative amplitudes of the IPMs, i.e., three cumulative amplitudes for IPM 5 and two cumulative amplitudes for IPM 4. Fig.19 (b) illustrates that the CU 256 is split into a left region and a right region, and for the left region, illustrates a partial template block 258' adjacent to the left CU region, which includes a left template block region 258a, an upper left template block region 258c, and a portion of an upper template block region 258b. Fig.19 (c) illustrates that CU 256 is split into four regions, such as Fig.19 As shown in (a), a partial template block 258' is illustrated for the upper left area, which includes a portion of the left template block area 258a, a portion of the upper template block area 258b and the upper left template block area 258c. Fig.19 As can be seen, the width / height of the partial template block defined in the upper or left template block area is defined by the vertical / horizontal lines defining the corresponding CU area. The concepts according to the first and second aspects apply to partial template block areas, except that only a strict part or portion of the template block adjacent to the CU area is considered. Therefore, the HoG calculation and IPM selection only occur on samples present in the corresponding partial template block 258', i.e., blocks in the template block whose samples are adjacent to a CU area for which the mixing weights are to be determined.

[0214] The third aspect of the present invention is advantageous because the adaptation and matching to the local characteristics of a certain CU region is a better match. According to an embodiment, if the direction in the corresponding template block region is away from the horizontal line for the left template block region or away from the vertical line for the upper template block region, then additional information collected from the adjacent template block portion can be combined so that according to an embodiment, the second aspect and the third aspect can be combined. Fig. 20 An embodiment is illustrated in which a mixture of partial and full / entire template block regions is used to calculate the HoG for selecting one or more IPMs. Fig. 20As shown in , according to an embodiment, CU 256 is split into four CU regions, and for the upper left region, only a portion of the upper template block that is a part of the left template block region 258 is used, and the entire left template block region is used, that is, the template block region for which HoG is calculated to select the IPM includes the left template block region 258a, the upper left template block region 258c, and a portion of the upper template block region adjacent to the upper left CU region. According to a further embodiment, when multiple IPMs (such as two or more IPMs) can be selected, one or some but not all IPMs can be determined from the HoG obtained for the entire left template block region, and the remaining IPMs are obtained from the HoG obtained for the partial template block. According to a further embodiment, the first IPM can be calculated based on the entire template block, and the second IPM can be determined based only on the HoG for the partial template block.

[0215] The blending weights are determined per CU region, and thus the number of blending weights may depend on the current CU region for which the local blending weights are currently being calculated.

[0216] According to an embodiment, the mixing weight can be calculated based on the cumulative amplitude ratio of the entire template block HoG, and the weight of the second IPM can be greater than the weight of the first IPM. For example, for the first IPM, the cumulative amplitude ratio with the second IPM is 2 / 9, for the second IPM, its cumulative amplitude ratio is 4 / 9, and for Planar, its cumulative amplitude ratio is 1 / 3.

[0217] According to a further embodiment, if the first IPM exists in an adjacent template block region and the second IPM also exists in the adjacent template block region, the cumulative amplitude ratio for determining the mixing weight may be calculated based on the cumulative amplitude in the adjacent template region HoG.

[0218] According to further embodiments, if the inter-angle mode is not selected as the second IPM, the first IPM may be mixed with the Planar or DC mode using a weight of 0.5. If more than two IPMs are selected, e.g. three or more, the first two IPMs may be calculated from the entire template block HoG, and the third IPM may be calculated from an adjacent template block region HoG. According to other embodiments, only the first IPM may be calculated from the entire template block HoG, while further (e.g. second and third) IPMs may be calculated from the template block region HoG.

[0219] According to a further embodiment of the third aspect of the present invention, the direction of the IPM with the highest peak (cumulative amplitude) in the global HoG calculated for the entire template block can determine which template block area is to be used completely and which template block area is to be used partially. For example, if the main direction in the HoG calculated for the entire template block or for the template block area is more vertical than horizontal, for example, the IPM is between 34 and 66 and between 53 and 66 (including the endpoints), then the upper left CU area can use part of the upper template block 258' (see Fig. 20 ) and the entire left template block area. Otherwise, if the main direction in HoG is more horizontal than vertical, such as IPM between 2 and 34 and greater than 66 (inclusive), then the upper left CU area (see Fig. 20 ) can use the entire upper template block area, the entire upper left template block area, and only part of the left template block area.

[0220] Further Embodiments

[0221] So far, the inventive concept has been described with reference to aspects and embodiments of a method for determining a DIMD predictor. According to further embodiments, the invention also provides an apparatus for deriving a DIMD predictor and an encoder / decoder comprising such an apparatus.

[0222] Fig.21 A block diagram of an apparatus for deriving a decoder-side intra mode derivation (DIMD) predictor for corresponding samples of a coding unit (CU) of a picture according to an embodiment of the present invention is illustrated. The apparatus 300 includes the following modules:

[0223] 302: An intra prediction mode (IPM) selection module, configured to select one or more IPMs in a template block adjacent to the CU.

[0224] 304: A mixing weight determination module is configured to determine a mixing weight for mixing at least one or more selected IPMs.

[0225] 306: A DIMD predictor generation module configured to generate a DIMD predictor by blending one or more selected IPMs using a blending weight determined for a CU such that the blending varies across the CU.

[0226] Fig. 22 The block diagram of a decoder for decoding a coded data stream including data representing a picture according to an embodiment of the present invention is illustrated. The decoder 400 includes the following modules:

[0227] 402: A decoder module configured to decode a picture from an encoded data stream.

[0228] 404: A prediction module, comprising means for deriving a decoder-side intra mode derivation (DIMD) predictor for corresponding samples of a coding unit (CU) of a picture according to an embodiment of the present invention.

[0229] Fig.23 A block diagram of an encoder for encoding a picture into a coded data stream according to an embodiment of the present invention is illustrated, the coded data stream including data representing the picture. The encoder 500 includes the following modules:

[0230] 502: An encoder module is configured to receive an original picture and encode the picture into a coded data stream.

[0231] 504: A prediction module, the prediction module comprising means for deriving a decoder-side intra mode derivation (DIMD) predictor for corresponding samples of a coding unit (CU) according to an embodiment of the present invention.

[0232] Although some aspects of the disclosed concepts have been described in the context of an apparatus, it is clear that these aspects also represent a description of a corresponding method, where a block or device corresponds to a method step or a feature of a method step. Similarly, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus.

[0233] Fig.24 900 is a block diagram illustrating an electronic device 900 according to an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are described only as examples and are not intended to limit the embodiments of the present invention described and / or claimed herein. The device 900 includes a computing unit 901 to perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 902 or loaded from a storage unit 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data for the operation of the storage device 900 can also be stored. The computing unit 901, ROM 902, and RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0234] The components in the device 900 are connected to the I / O interface 905, including: an input unit 906, such as a keyboard, a mouse; an output unit 907, such as various types of displays, speakers; a storage unit 908, such as a disk, an optical disk; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices through a computer network (such as the Internet and / or various telecommunications networks). The computing unit 901 can be composed of various general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any suitable processors, controllers, microcontrollers, etc. The computing unit 901 performs the various methods and processes described above, such as image processing methods. For example, in some embodiments, the image processing method can be implemented as a computer software program tangibly implemented on a machine-readable medium (such as a storage unit 908). In some embodiments, part or all of the computer program may be loaded and / or installed on the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the above-described image processing method may be performed. In some embodiments, the computing unit 901 may be configured to perform the image processing method in any other suitable manner (e.g., by means of firmware).

[0235] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementation in one or more computer programs that can be executed and / or interpreted on a programmable system that includes at least one programmable processor, and the programmable processor may be a special purpose or general purpose programmable processor, and may receive data and instructions from a storage system, at least one input device, and at least one output device, and may transmit data and instructions to a storage system, at least one input device, and at least one output device.

[0236] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that when the processor or controller executes the program code, the functions and / or operations specified in the flow chart and / or block diagram are executed. The program code can be executed completely on the machine, partially on the machine, partially on the machine as an independent software package and partially on a remote machine or completely on a remote machine or server.

[0237] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program that is used by or in connection with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0238] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having a display device (e.g., a cathode ray tube (CRT) or a liquid crystal display (LCD)) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user, for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and any form (including acoustic input, voice input, or tactile input) can be used to receive input from the user.

[0239] The systems and techniques described herein may be implemented on a computing system that includes a back-end component (e.g., as a data server), or on a computing system that includes a middleware component (e.g., an application server), or on a computing system that includes a front-end component (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or on a computer system that includes such back-end components, middleware components, front-end components, or any combination thereof. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0240] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in a cloud computing service system and solves the defects of difficult management and weak business scalability in traditional physical hosts and virtual private servers (referred to as "VPS"). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0241] It should be understood that steps can be reordered, added or deleted by using the various forms of processes shown above. For example, the steps described in the present invention can be performed in parallel, sequentially or in different orders, as long as the expected results of the technical solutions in the present invention can be achieved, and are not limited herein.

[0242] The specific embodiments mentioned above do not limit the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and replacements can be made according to design requirements and other factors. Any modifications, equivalent replacements and improvements made within the principles of the present invention should be included in the protection scope of the present invention.

[0243] References

[0244] [1]H.264:Advanced video coding for generic audio-visual services,https: / / www.itu.int / rec / T-REC-H.264-202108-P / en

[0245] [2]H.265: High efficiency video coding, https: / / www.itu.int / rec / T-REC-H.265-202108-P / en

[0246] [3]H.266:Versatile video coding,https: / / www.itu.int / rec / T-REC-H.266-202008-I / en

[0247] [4]AV1 Bitstream&Decoding Process Specification, http: / / aomedia.org / av1 / specification /

[0248] [5] E.Mora et al., “CE3-related: Decoder-side Intra Mode Derivation”, JVET-L0164, Macau, China, October 2018

[0249] [6]CE3:Decoder-side Intra Mode Derivation(tests 3.1.1,3.1.2,3.1.3and3.1.4),JVET-M0094,Marrakech,January 2019

[0250] [7] M. Abdoli et al., E. Mora, T. Guionnet, M. Raulet, Non-CE3: Decoder-side IntraMode Derivation with Prediction Fusion, JVET-N0342, Geneva, March 2019

[0251] [8] M. Abdoli et al., “Non-CE3: Decoder-side Intra Mode Derivation with Prediction Fusion Using Planar”, JVET-O0449, Gothenburg, July 2019

[0252] [9] J. Zhao et al., “EE2-Related: Improvements of Decoder-Side Intra Mode Derivation”, JVET-V0087, online, April 2021.

[0253]

[10] J. Zhao et al., “EE2 Test 2.2: DIMD with multiple blending modes”, JVET-W0054, online, July 2021.

[0254]

[11] X. Li et al., “EE2-related: Implicit derivation of DIMD blend modes”, JVET-W0067, online, July 2021.

[0255]

[12] X. Li et al., “EE2-related: A combination of CIIP and DIMD / TIMD”, JVET-W0068, online, July 2021.

[0256]

[13] J. Zhao et al., “EE2 Related-DIMD with implicitly derived multiple blending modes”, JVET-W0126, online, July 2021.

[0257]

[14] X. Li et al., “EE2-3.1: Combination of CIIP and DIMD / TIMD”, JVET-X0098, online, October 2021.

[0258]

[15] C. Zhou et al., “EE2-related: Optimization on the second mode derivation of DIMD blending mode”, JVET-X0115, online, October 2021.

[0259]

[16] T. Dumas et al., “AHG12: removing a discontinuity in the discrete angle comparison in DIMD,” JVET-X0139, online, October 2021.

[0260]

[17] X. Xiu et al., “AHG12: Removal of floating operations in DIMD and TIMD,” JVET-X0149, online, October 2021.

[0261]

[18] K. Cao et al., “Non-EE2: Fix for histogram of gradients derivation in DIMD mode”, JVET-X0156, online, October 2021.

[0262]

[19] K. Naser et al., “AhG12: Removed DIMD from MPM list of TIMD”, JVET-Y0097, online, January 2022.

[0263]

[20] S. Yoo et al., “EE2-related: Clean-up on DIMD,” JVET-Y0131, online, January 2022.

[0264]

[21] J. Zhao et al., “Non-EE2: DIMD Flag Signalling Clean-up”, JVET-Y0144, online, January 2022.

[0265]

[22] J. Pfaff et al., “Intra Prediction and Mode Coding in VVC”, IEEE Transactions on Circuits and Systems for Video Technology, Vol. 31, No. 10, October 2021.

Claims

1. A method for deriving a decoder-side intra mode derivation (DIMD) predictor for corresponding samples of a codec unit (CU) (256) of a picture (102), the method comprising: selecting (S200) one or more intra prediction modes (IPMs) in a template block (258) adjacent to the CU (256), determining (S202) a mixing weight for mixing at least the one or more selected IPMs, and The DIMD predictor is generated (S204) by blending the one or more selected IPMs using blending weights determined for the CU (256) such that the blending varies across the CU (256).

2. The method of claim 1, comprising: Splitting (S232) the CU (256) into two or more CU regions (256a-256d, TL, TR, BL, BR), wherein, for each CU region of at least a subset of the two or more CU regions (256a-256d, TL, TR, BL, BR), the blending weight is determined based on the presence of the one or more selected IPMs in a portion of the template block (258) adjacent to the corresponding CU region (256a-256d, TL, TR, BL, BR), and The DIMD predictor is generated using hybrid weights determined for each CU region (256a-256d, TL, TR, BL, BR).

3. The method of claim 2, wherein the one or more IPMs are selected using IPM statistics such as a histogram of gradients (HoG) determined globally over the entire template patch (258), the template patch (258) comprising a plurality of template patch regions, wherein - performing said selection globally for said entire CU (256), or - The selection involves a global selection of a set of IPMs for the entire CU (256) and then a further per-region selection of the set of IPMs for each CU region (256a-256d, TL, TR, BL, BR).

4. The method of claim 2, wherein the one or more IPMs are selected using IPM statistics, such as a histogram of gradients (HoG), the statistics being determined separately on - each of a plurality of template block regions of the template block (258), the plurality of template block regions comprising a left template block region (258a) and an upper template block region (258b), or - each partial template block region of the plurality of partial template block regions of the template block (258), the partial template block region (258') being a template block region adjacent to only one of the CU regions (256a-256d, TL, TR, BL, BR), and The selection involves a per-region selection from an IPM determined separately for each CU region (256a-256d, TL, TR, BL, BR).

5. The method of any one of claims 2 to 4, wherein determining the blending weights comprises - for a CU region positioned adjacent to the template block, determining the mixing weight according to the presence of the one or more selected IPMs in one or more template block regions or partial template block regions adjacent to the CU region, and - for a CU region that is not located adjacent to the template block (258), ○ Select only Planar mode as the DIMD predictor, or ○ Select only DC mode as the DIMD predictor, or o determining the blending weights using the one or more selected IPMs in one or more or all template block regions, or ○ The mixing weight is determined by weighting the mixing weights of the CU regions adjacent to the template block region.

6. A method as claimed in any one of the preceding claims, in - selecting the one or more IPMs in a manner conscious of subdividing the template block (258) into a plurality of template block regions, the plurality of template block regions comprising a left template block region (258a) and an upper template block region (258b), - The template block (258) further includes an upper left template block area (258c), and the upper left template block area (258c) is allocated o to the upper template block area (258b), or o to the left template block area (258a), or o to both the upper template block area (258b) and the left template block area (258a), or ○ as additional template block areas, and / or in - selecting the one or more IPMs in a manner conscious of the subdivision of the template block (258) into a plurality of partial template block areas (258'), - when defining the two or more CU regions, defining the partial template block region (258') by splitting the template block (258) by the same vertical and / or horizontal lines as the CUs (256).

7. The method of claim 6, wherein one or more first IPMs are selected in a template block area of ​​the template block (258), and one or more second IPMs are selected in a partial template block area of ​​the template block (258), and wherein - if the CU (256) has a rectangular shape, selecting the first IPM in the template block area adjacent to the longer dimension of the CU (256) and selecting the second IPM in the portion of the template block area adjacent to the shorter dimension of the CU (256), or - selecting the first IPM and the second IPM in the template block area or from the portion of the template block area adjacent to the CU (256) depends on the direction of the peak IPM in the template block area or in the entire template block, for example, If the main direction of the peak IPM calculated in the template block area or in the entire template block is more vertical than horizontal, then the first IPM is selected in the left template block area or the upper template block area and optionally in the upper left template block area adjacent to the CU area, and the second IPM is selected in the partial upper template block area or the left template block area adjacent to the CU area, and ○ If the main direction of the peak IPM calculated in the template block area or in the entire template is more horizontal than vertical, then the first IPM is selected in the upper template block area or the left template block area and optionally in the upper left template block area adjacent to the CU area, and the second IPM is selected in the partial left template block area or the upper template block area adjacent to the CU area.

8. The method according to claim 6 or 7, in, If two or more IPMs are selected, at least some but not all of the IPMs are selected in the entire template block (258), and the remaining IPMs are selected in a template block region or from a portion of a template block region adjacent to the CU region, or wherein, if two IPMs are selected, then the first IPM in the entire template block (258) is selected, and If there is no second IPM from the template block area or the portion of the template block area adjacent to the CU area, then o The first IPM is used as the DIMD predictor, or ○ Mixing the first IPM with Planar or DC mode using predefined weights to obtain the DIMD predictor, - if there is a second IPM selected in the template block area or the portion of the template block area adjacent to the CU area, then o If the second IPM is different from the first IPM, then blending the first IPM with the second IPM to obtain the DIMD predictor, o If the second IPM is equal to the first IPM, selecting a further IPM in the template block region adjacent to the CU region, and blending the first IPM with the further IPM to obtain the DIMD predictor.

9. The method of claim 8, wherein for a CU region that is not located adjacent to any of the template block regions, - using only the first IPM as the DIMD predictor, or - use only the Planar mode as the DIMD predictor, or - use only the DC mode as the DIMD predictor, or - Mixing the first IPM with Planar or DC mode using predefined weights to obtain the DIMD predictor.

10. The method according to any one of the preceding claims, in, If more than one IPM is selected, then - if the IPM exists in a template block area adjacent to the CU area, then use the blending weight associated with the IPM, - if only one of the IPMs is present in the template block area adjacent to the CU area, then the blending weight associated with the one IPM or the weighted blending weight associated with the one IPM is used, and the blending weights associated with any other IPM are set to a predefined value, such as 0, - optionally, if no IPM exists in the adjacent template block area of ​​the CU area, setting the blending weight associated with the IPM to a predefined value, such as 0, and - Optionally, set the blending weights associated with Planar or DC mode to predefined values, or If you select an IPM, then - if the one IPM is present in a template block region adjacent to the CU region, then using the blending weight associated with the one IPM or the weighted blending weight associated with the one IPM, and setting the blending weight associated with any other IPM to a predefined value, such as 0, - optionally, if the one IPM does not exist in a template block area adjacent to the CU area, setting the blending weight associated with the one IPM and the blending weights associated with any other IPM to a predefined value, such as 0, and - Optionally, set the blending weights associated with Planar or DC mode to predefined values, or Therein, if no IPM is selected, then the blending weight associated with any other IPM is set to a predefined value, such as 0, and optionally, the blending weight associated with Planar or DC mode is set to a predefined value.

11. A method as claimed in any one of the preceding claims, wherein - for a CU region located adjacent to a first template block region for which no IPM is selected and away from a second template block region for which one or more IPMs are selected, the blending weight is a weighted blending weight of CU regions adjacent to the second template block region, or For a CU region located adjacent to the first template block region for which the first IPM is selected and away from the second template block region for which the second IPM is selected, the mixing weight for the second IPM is the weighted mixing weight of the second IPM for the CU region adjacent to the second template block region.

12. A non-transitory computer-readable storage medium comprising instructions, which, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 11.

13. An apparatus (300) for deriving a decoder-side intra mode derivation (DIMD) predictor for corresponding samples of a codec unit (CU) (256) of a picture (102), the apparatus (300) comprising: An intra prediction mode (IPM) selection module (302) is configured to select one or more IPMs in a template block (258) adjacent to the CU (256), a mixing weight determination module (304) configured to determine a mixing weight for mixing at least the one or more selected IPMs, and A DIMD predictor generation module (306) is configured to generate the DIMD predictor by blending the one or more selected IPMs using a blending weight determined for the CU (256) such that the blending varies across the CU (256).

14. An apparatus (400) for decoding an encoded data stream (152), the encoded data stream (152) comprising data representing a picture (102), the apparatus (400) comprising: a decoder module (402) configured to decode the picture (102) from the encoded data stream (152), and A prediction module (402) comprising means (300) as claimed in claim 13 for deriving a decoder-side intra mode derivation (DIMD) predictor for corresponding samples of a codec unit (CU) (256) of the picture (102).

15. An apparatus (500) for encoding a picture (102) into a coded data stream (120), the coded data stream (120) comprising data representing the picture (102), the apparatus (500) comprising: an encoder module (502) configured to receive the original picture (102) and encode the picture (102) into the encoded data stream (120), and A prediction module (504) comprising means (300) for deriving a decoder-side intra mode derivation (DIMD) predictor for corresponding samples of a codec unit (CU) (256) of the picture (102) as described in claim 13.

Images