Method and apparatus for DIMD intra prediction mode selection in template block and encoder / decoder including same

By splitting the template block into multiple regions and calculating local HoG, selecting and mixing intra prediction modes, the problem of intra pattern derivation in the prior art is solved, and the accuracy and diversity of intra prediction are improved. Improved codec quality.

CN118947111APending Publication Date: 2024-11-12BEIJING XIAOMI MOBILE SOFTWARE CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202380029705.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-04-07
Filing Date
2023-03-02
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

In the prior art, intra-mode derivation (DIMD) processing has non-optimal predictor generation, resulting in intra-mode prediction not being fine enough to effectively match the different directions and angle characteristics in the template block, affecting the quality of encoding and decoding.

Method used

Split the template block into multiple regions, calculate the local HoG separately, select and mix the intra prediction mode (IPM) to generate a more refined DIMD predictor, using the peak and cumulative amplitude ratios in the local HoG to determine the mixed weight.

Benefits of technology

Through local HoG calculation and hybrid weight optimization, IPMs are generated closer to the CU part to be predicted, which improves the fine granularity and diversity of intra prediction and improves the prediction effect of the codec.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118947111A_ABST
    Figure CN118947111A_ABST
Patent Text Reader

Abstract

A method of deriving a decoder-side intra mode derivation (DIMD) predictor for respective samples of a coding unit (CU) of a picture is described. The method comprises the following steps: splitting a template block adjacent to a CU into a plurality of template block areas; independently selecting zero, one or more intra prediction modes (IPMs) in each template block region, wherein the selection involves selecting from the IPMs independently determined for the entire CU or for one or more regions of the CU; determining a hybrid weight using the selected IPM; and generating a DIMD predictor by blending at least the selected IPM using the determined blending weight.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is based on and claims priority from European patent application No. “22167220.7” filed on April 7, 2022, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present invention generally relates to the field of encoding / decoding pictures, images or videos, and embodiments of the present invention relate to improvements regarding intra prediction, and more specifically, to improvements regarding decoder-side intra mode derivation (DIMD) processing. More specific embodiments of the present invention relate to DIMD intra prediction mode selection in template blocks. Background Art

[0004] Encoding and decoding of pictures, images or videos are performed according to certain standards, for example, according to the Advanced Video Codec (AVC) standard (see reference [1]), the High Efficiency Video Codec (HEVC) standard (see reference [2]) or the Versatile Video Codec (VVC) standard (see reference [3]).

[0005] Figure 1 A block diagram of a standard video compression system 100 operating according to the VVC standard is illustrated in FIG. The standard video codec 100 compresses and encodes pictures 102 of a video sequence. The pictures 102 to be encoded are partitioned into blocks 104, which are also referred to as codec units (CUs). The encoder 100 includes a pre-coding filter 106 and a prediction residual signal former 108, which generates a prediction residual signal 110 to measure the deviation between the prediction signal 112 and the signal 114 output by the filter 106. The encoder also includes a transformer 116, a quantizer 118, and uses an entropy codec 122 to provide an output bitstream or data stream 120. In addition, the encoder 100 includes a prediction section 124 for determining the prediction signal 112, the prediction section 124 including an inverse quantizer or inverse quantizer 126, an inverse transformer 128, a combiner 130, an in-loop filter 134, a motion estimator 138, an intra / inter mode selector 140, an inter predictor 142, and an intra predictor 144.

[0006] As reference Figure 1The described video codec 100 compresses and encodes a picture 102 of a video sequence as follows. The picture 102 to be encoded is partitioned into blocks or CUs 104. Each codec unit 104 is encoded using either intra or inter coding mode. When a CU is encoded in intra mode, intra prediction is performed by an intra predictor 144. Intra prediction includes predicting the current CU 114 being encoded with the help of coded, decoded and reconstructed picture samples located around the current CU (e.g., the top and left of the current CU). Intra prediction is performed in the spatial domain. In inter mode, motion compensation and motion estimation are performed by a motion estimator 138, which searches for a CU that is a good predictor of the current CU in one or more reference pictures provided by a picture buffer 136 and used to predictively encode and decode the current picture. For example, a good predictor of the current CU is a predictor similar to the current CU, that is, the distortion between the two CUs is low or below a certain threshold. Motion estimation can also take into account the rate cost of signaling to the predictor to optimize the rate-distortion trade-off. The output of the motion estimation step is one or more motion vectors and reference indices associated with the current CU. Motion compensation then predicts the current CU with the aid of the one or more motion vectors and reference picture indices determined by the motion estimator 138. Basically, the block or CU contained in the selected reference picture and pointed to by the determined motion vector is used as the prediction block of the current CU. The encoder 100 selects one of the intra-frame coding mode or the inter-frame coding mode with the aid of the selector 140 for encoding the CU and indicates the intra / inter decision, for example, with the aid of a prediction mode flag. The prediction residual 110 is then transformed and quantized by blocks 116 and 118, and the quantized transform coefficients as well as the motion vectors and other syntax elements are entropy encoded and written to the output bitstream 120. The encoder 100 can skip the transform stage 116 and directly apply quantization to the untransformed residual signal 110 in the so-called transform-skip coding mode. After encoding the block or CU, the encoder decodes the CU and reconstructs it to obtain a reconstructed signal 132, which can be used as reference data for predicting future CUs or blocks to be encoded. The quantized transform coefficients 110' are dequantized and inverse transformed to obtain a decoded predicted CU or block 110", and the decoded prediction residual and the predicted block are then combined (usually added) at 130 to provide a reconstructed block or CU 132. An in-loop filter 134 is applied to the reconstructed picture to reduce compensation artifacts. For example, a deblocking filter, a sample adaptive offset (SAO), a filter, and an adaptive loop filter (ALF) can be applied to reduce coding artifacts. The filtered picture is stored in a buffer 136 (also called a decoded picture buffer (DPB)) so that it can be used as a reference picture for encoding and decoding subsequent pictures.

[0007] Figure 2 1 is a block diagram of a video decoder 150 for predictively decoding a picture or video provided at an output 154 of the decoder 150 from a data stream 152. The decoder 150 includes an entropy decoder 156, a partition block 158, an inverse quantizer 160, an inverse transformer 162, a combiner 164, an in-loop filter 166, an optional post-decoding processor 168, and a prediction module 170. The prediction module 170 includes a decoded picture buffer 180, a motion compensator 182, and an intra-frame predictor 184.

[0008] The coded pictures of the video sequence are decompressed and decoded by the decoder 150 as follows. The input bitstream 152 is entropy decoded by the decoder 156, which provides, for example, block partition information, the codec mode for each codec unit, the transform coefficients contained in each transform block, prediction information (such as intra-frame prediction mode), motion vectors, reference picture indexes, and other codec information. The block partition information indicates how the picture is partitioned, and the decoder 150 can divide the input picture into codec tree units (CTUs) (usually 64x64 or 128x128 pixels in size) and divide each CTU into rectangular or square codec units (CUs) according to the decoded partition information. The entropy decoded quantized coefficients 172 are dequantized 160 and inverse transformed 162 to obtain a decoded residual picture or CU 174. The decoded prediction parameters are used to predict the current block or CU, that is, whether the predicted block will be obtained by its intra-frame prediction or by its motion compensated temporal prediction. The prediction process performed on the decoder side is the same as the prediction process performed on the encoder side. The decoded residual block 174 is added to the predicted block 176, thereby producing a reconstructed current image block 164. The in-loop filter 166 is applied to the reconstructed picture or image, which is also stored in the decoded picture buffer 180 to be provided with reference pictures for future picture decoding. As mentioned above, the decoded picture may also be subjected to post-decoding processing, such as for performing an inverse color transform, such as from YCbCr4:2:0 to RGB4:4:4.

[0009] As mentioned above, the intra-frame prediction mode to be used for decoding can be carried in the bitstream provided by the encoder and received by the decoder, but, according to other methods, in addition to introducing the actual intra-frame prediction mode into the bitstream, the intra-frame prediction mode can also be derived by using the gradient analysis of the reconstructed pixels adjacent to the currently processed CU. In other words, the intra-frame prediction mode is not explicitly indicated in the bitstream, but is implicit. This method is called decoder-side intra-frame mode derivation (DIMD), which can be signaled using a simple flag and then derive the actual intra-frame mode during the reconstruction process, such as the reconstruction process performed by the prediction block 124 / 170 of the encoder 100 or decoder 150. The encoder 100 can encode information whether DIMD is used for the current CU into the bitstream 120, and in the case where DIMD is not used, the actual intra-frame mode is signaled by the encoder and parsed from the bitstream by the decoder.

[0010] Taken from reference [5] Figure 3 An example of an intra-block generic reconstruction process is illustrated, which may be applied at the encoder 100 or at the decoder 150. During the parsing process 200, intra-block decoding is determined (as indicated at block 202), and pixel residual parsing 204 is performed as well as DIMD flag parsing 206. In the event that the DMID flag 206 is zero, intra-prediction mode parsing 208 is performed to obtain the intra-mode 210 to be used during the reconstruction process 212 when predicting and reconstructing the pixels (as indicated at 216). If the DIMD flag 206 is set to "one", intra-prediction derivation 218 is performed during the reconstruction process 212, which produces a derived intra-mode 220, which is used in the reconstruction process together with the pixel residual 214 at 216.

[0011] Further details of the DIMD method are described in references [5] to [8]. In general, DIMD processing is based on a reconstructed template block adjacent to the currently processed CU, the width and height of which are three samples wide. The template block includes a left block, an upper block, and an upper left block. These blocks are also referred to as the left template block area, the upper template block area, and the upper left template block area in the following. In the template block, corresponding edge detection filters (such as 3x3 horizontal and vertical Sobel filters) are applied to determine the amplitude and angle of the brightness direction or orientation of each intermediate line sample of the template block. A histogram of gradients (HoG) is calculated, and each entry corresponds to a conventional intra-frame angle pattern, and the accumulated intensity is stored:

[0012] angle=arctan(G hor / G ver )

[0013] amplitude=|Ghor |+|G ver |

[0014] Among them G hor and G ver are the pure horizontal and vertical intensities calculated by the Sobel filter. FIG. 4 taken from reference [8] illustrates a DIMD template block or region for calculating HoG. FIG. 4( a) illustrates a partially reconstructed picture 250, as indicated by a reconstructed block 252 and an unavailable block 254. In other words, the reconstructed block 252 includes the already reconstructed codec units or blocks of the picture to be reconstructed, while the corresponding codec units or blocks in the unavailable block 254 are still to be reconstructed. The currently processed CU or block 256 and the template block 258 mentioned above are illustrated. FIG4(b) illustrates that the template block 258 is 3 samples or pixels wide and high, and as described above, the HoG 260 is calculated using 3x3 horizontal and vertical Sobel filters 259 to obtain the intra-frame angle prediction mode (also called intra-frame prediction mode, IPM) based on the amplitude and angle of the luma direction of each middle line sample 262 of the template block 258, and to obtain the cumulative intensity or amplitude associated with each IPM in the template block. The two most representative IPMs (indicated as M1 and M2 in FIG4(b)) are selected from the HoG 260 and combined with the Planar mode using weights for M1 and M2 derived from the ratio of their cumulative amplitudes and a fixed weight for the Planar mode (e.g., 1 / 3, i.e., 21 / 64 with 6-bit integer precision).

[0015] Taken from reference [8] Figure 5 The diagram illustrates prediction fusion by weighted averaging of M1, M2 and Planar modes. Assuming a square CU 256, corresponding pixels of CU 256 are predicted by the two most representative IPMs M1, M2 taken from HoG 260, thereby obtaining predictor blocks Pred1, Pred2 and Pred3. The corresponding predictor blocks are weighted by weights w1, w2, w3 and fused or combined into a final predictor block including DMID predictors for corresponding samples or pixels in CU 256.

[0016] Figure 6An example of a conventional DIMD predictor derivation process is illustrated. At S100, using an input CU 256, a neighborhood of the input CU 256 is analyzed to determine a plurality of available intra-top and intra-left samples. At S102, a HoG 260 is constructed for a template block 258, i.e., based on the available samples, an intra-angle prediction mode IPM is determined by its cumulative amplitude in the HoG 260 (step S104). At S106, it is determined how many IPMs are selected, i.e., whether no IPM is selected, whether one IPM is selected, or whether two IPMs are selected. In the case where no IPM is selected, only the Planar mode (which may be the default mode) is applied, as indicated at S108, to obtain a DIMD predictor. In the case where only one IPM is selected, the selected IPM is used to obtain a DIMD predictor, as indicated at S110. In the case where two IPMs are selected, a mixing weight of each selected mode is determined according to their cumulative amplitude ratio at S112, and the weight for the Planar mode is set to 1 / 3. Predictor blocks Pred1 to Pred3 for each selected IPM and Planar mode are calculated at S114, and a DIMD predictor is obtained by mixing the two selected IPMs and Planar mode using the determined mixing weight at S116.

[0017] Figure 7 An example of a currently processed CU 256 and an associated template block 258 of width / height T=3 samples or pixels is illustrated. For each intermediate line sample 262, the corresponding orientation is illustrated by the direction of the corresponding arrow, and the corresponding strength of the gradient is indicated by the length of the arrow. The HoG is generated by accumulating the strength of each orientation / IPM entry, and the two most significant modes in the HoG (if they exist) are selected as the modes to be mixed with the Planar mode or the Planar intra prediction mode. Figure 7 In the example of , IPMs 5 and 28 have the first and second highest cumulative strengths / amplitudes. As described above, these modes are weighted, and the blending weights are conventionally fixed and / or determined according to the ratio of the cumulative strengths / amplitudes of the selected IPMs. Although different blending modes are proposed in the prior art, all blending modes are global with respect to the currently processed block or CU. More specifically, when considering the conventional approach and when two IPMs (such as M1 and M2) are selected, the blending weights can be calculated as follows

[0018]

[0019]

[0020]

[0021] Where w1 is associated with IPM M1, w2 is associated with IPM M2, and w3 is associated with Planar mode. Different modifications of the mixing process are described in the prior art, for example, reference

[10] proposes selecting multiple mixing modes based on rate-distortion optimization, reference

[11] proposes using HoG to implicitly derive two of the three mixing modes, reference

[13] proposes using template matching to implicitly derive the mixing mode, and reference

[20] proposes selecting different mixing modes based on the determination of multiple DIMD modes. When DIMD is applied, one or two IPMs are derived from reconstructed neighboring samples (i.e., from the template block), and in the case where two intra-frame modes are derived, they are combined (i.e., mixed) with the Planar mode predictor. Conventional solutions always implement a global or CU-level weighted mixing of two IPMs with the Planar mode.

[0022] Conventionally, in the reconstructed portion 252 of the picture 250, three columns and three rows of reconstructed samples or pixels are used for DIMD processing, i.e., the HoG 260 is calculated using a filter having a fixed filter window of 3x3 samples and located at the same position in the template block 258. In reference [6], when calculating the gradient on the pixels located in the left column, top row or upper left sample and directly adjacent to the current CU, the 3x3 Sobel filter 259 can be replaced by a 3x2 filter. In reference [8], the 3x3 Sobel filter 259 can be applied more sparsely, for example, only at one middle line sample in the left template block and one middle line sample in the upper template block, instead of using the 3x3 Sobel filter 259 on all pixels or samples in the middle line of the template block 258. Further aspects of the DIMD processing are described, for example, in the following references:

[0023] - References

[10] ,

[11] ,

[13] and

[20] discuss the mixing of modes,

[0024] - References

[16] and

[18] discuss behavior correction (error repair) relative to the original intent,

[0025] - References [9],

[12] and

[14] discuss combinations with other modes,

[0026] - References

[15] ,

[17] ,

[19] or

[21] discuss syntax or simplifications for DIMD processing.

[0027] However, the above reference Figure 6 The basic DIMD processing described and described in reference [8] remains the same.

[0028] As described above, the HoG is calculated based on the entire template block adjacent to the currently processed CU, and at most two peaks are selected as angular intra prediction modes to be mixed using weights calculated according to the ratio of the amplitudes accumulated in the HoG, thereby generating a DIMD predictor for the corresponding samples in the CU, as described above with reference to FIGS. 4 and Figure 5 This can lead to non-optimal DIMD predictor generation.

[0029] Therefore, there is a need to provide further improvements in DIMD processing. Summary of the invention

[0030] The present invention provides a method for deriving a decoder-side intra mode derivation (DIMD) predictor for corresponding samples of a coding unit (CU) of a picture, the method comprising:

[0031] Split the template block adjacent to the CU into multiple template block areas.

[0032] independently selecting zero, one or more intra prediction modes (IPMs) in each template block region, wherein the selection involves selecting from IPMs independently determined for the entire CU or for one or more regions of the CU,

[0033] Determine the blending weights using the chosen IPM, and

[0034] The DIMD predictor is generated by mixing at least the selected IPM using the determined mixing weight.

[0035] Optionally, the number of IPMs selected for each template block region depends on the size of the CU, where - when the CU is square in shape,

[0036] ○ Select the same number of IPMs in all template block areas, or

[0037] ○ Select the same number of IPMs in all template block regions adjacent to the edges of the CU, and select no IPMs in template block regions adjacent only to the corners of the CU, or

[0038] o Select one or more IPMs in template block regions adjacent only to the corners of the CU, and select no IPMs in all template block regions adjacent to the edges of the CU, or

[0039] -When the CU is rectangular in shape,

[0040] ○ Selecting a first number of IPMs in the template block area adjacent to the long side of the CU, selecting a second number of IPMs in the template block area adjacent to the short side, and selecting a third number of IPMs in the upper left template block area, wherein the first number is higher than the second number and the third number, and the second number is the same as or different from the third number, or

[0041] o Select the same number of IPMs in the template block regions adjacent to the short sides of the CU and in the template block regions adjacent to the long sides of the CU, and select no IPMs in the top left template block region, or

[0042] ○ Select the same number of IPMs in the template block regions adjacent to the short side of the CU and for the upper left template block region, and select no IPMs in the template block regions adjacent to the long side of the CU, or

[0043] o The same number of IPMs are selected in the template block regions adjacent to the long sides of the CU and for the upper left template block region, and no IPMs are selected in the template block regions adjacent to the short sides of the CU.

[0044] Optionally, the number of IPMs selected for each template block area depends on the size of the CU, and when the CU has a first size, a first number of IPMs is selected, and when the CU has a second size, a second number of IPMs is selected, wherein the second size is larger than the first size.

[0045] Optionally, a predefined number of IPMs is selected for each template block area, wherein the predefined number is the same or different for corresponding template block areas.

[0046] Optionally, a first number of IPMs is selected in a template block region having a maximum cumulative amplitude among all template block regions, and a second number of IPMs is selected in other template block regions, wherein the first number is higher than the second number.

[0047] Alternatively, if two or more selected IPMs in different template block regions are identical, the IPM for the first or last processed template block region is selected and retained, and other identical IPMs are discarded.

[0048] Optionally, the total number of selectable IPMs is fixed.

[0049] Optionally, a mixing weight for the selected IPM is determined by weighting a first value obtained from one or more predefined characteristics associated with one or more selected IPMs based on the number of selected IPMs, and optionally, in addition to the mixing weight of the IPM, a mixing weight for the Planar or DC mode is determined based solely on the number of selected IPMs.

[0050] Optionally,

[0051] - in case an IPM is selected in all template patch regions, then the blending weights are derived from one or more predefined features associated with one or more selected IPMs and obtained from the entire template patch, or

[0052] - in the case where an IPM is selected in one or more but not all template block regions, then the blending weights are derived from one or more predefined characteristics associated with the one or more selected IPMs and obtained from the template block regions from which the one or more IPMs were selected, or

[0053] - In case an IPM is selected in only one template block region, then the blending weights are derived from one or more predefined features associated with one or more selected IPMs and obtained from the one template block region. Optionally, the plurality of template block regions of the template block include

[0054] - the left template block area, the upper template block area and the upper left template block area, or

[0055] - In case a CU is split into multiple CU regions, the corresponding template block regions associated with the CU regions adjacent to the template block and an optional upper left template block region.

[0056] The present invention provides a non-transitory computer-readable storage medium including instructions, which, when executed by a computer, cause the computer to perform the method of the present invention.

[0057] The present invention provides an apparatus for deriving a decoder-side intra mode derivation (DIMD) predictor for corresponding samples of a coding unit (CU) of a picture, the apparatus comprising:

[0058] The template block splitting module is configured to split the template block adjacent to the CU into multiple template block areas,

[0059] an intra prediction mode (IPM) selection module configured to select zero, one or more intra prediction modes (IPMs) in each template block region, wherein the selection involves selecting from individually determined IPMs,

[0060] a blending weight determination module configured to determine a blending weight using the selected IPM, and

[0061] The DIMD predictor generation module is configured to generate a DIMD predictor by blending at least the selected IPM using the determined blending weight.

[0062] The present invention provides a device for decoding a coded data stream, wherein the coded data stream includes data representing a picture, the device comprising:

[0063] a decoder module configured to decode pictures from the encoded data stream, and

[0064] A prediction module comprising means of the invention for deriving a decoder-side intra mode derivation (DIMD) predictor for corresponding samples of a coding unit (CU) of a picture.

[0065] The present invention provides a device for encoding a picture into a coded data stream, wherein the coded data stream includes data representing the picture, and the device includes:

[0066] an encoder module configured to receive an original picture and encode the picture into a data stream, and

[0067] A prediction module comprising means of the invention for deriving a decoder-side intra mode derivation (DIMD) predictor for corresponding samples of a coding unit (CU) of a picture.

[0068] The technical solution provided according to the embodiment of the present invention has the following beneficial effects.

[0069] As described above with reference to the conventional method, the HoG is calculated based on the entire template block. However, it is found that the conventional method has disadvantages, more specifically, the calculation of the HoG based on the entire available template block makes the selection of one or more IPMs irrespective of their position in the template block. Due to the fine granularity of the angular intra prediction mode, two consecutive modes with the same direction may be selected in the same part of the template block, while other strong directions may exist in other parts of the template block, which may be closer to the part of the currently predicted CU in terms of sample distance / similarity. This results in the generation of a suboptimal DIMD predictor.

[0070] The present invention solves the above-mentioned shortcomings in conventional methods and provides a method according to an embodiment, which splits the template block into corresponding areas and calculates the HoG of each template block area to determine and select IPM from the peaks in the local HoG of these areas. The method of the present invention is advantageous because it improves prediction by selecting IPMs that may be located in different template block areas and are therefore closer to a portion of the CU to be predicted and / or by selecting IPMs that are more diverse and representative of the texture of the CU to be predicted. The embodiments of the present invention make full use of existing IPM calculation principles (such as calculation of HoG, selection of IPM from HoG peaks, determination of mixing weights from the ratio of the cumulative amplitudes of IPMs, and generation of predictors) while requiring only a few additional calculation operations.

[0071] It should be understood that the content described in this section is not intended to identify key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become apparent from the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] The drawings are illustrative and serve to explain the invention and are not to be construed as limiting the invention to the embodiments shown.

[0073] Figure 1 is a block diagram of a conventional video encoder;

[0074] Figure 2 is a block diagram of a conventional video decoder;

[0075] Figure 3 The diagram shows the application Figure 1 The encoder or Figure 2 An example of a generic reconstruction process for intra blocks of a decoder of FIG.

[0076] FIG4 illustrates a DIMD template block or region used to calculate a histogram of gradients (HoG) in a DIMD process;

[0077] Figure 5 The diagram shows the prediction fusion by weighted averaging of the selected IPM and Planar modes;

[0078] Figure 6 An example of a conventional DIMD predictor derivation process is illustrated;

[0079] Figure 7 illustrates an example of a currently processed and associated template block with a width / height of samples or pixels;

[0080] Figure 8 illustrates a flow chart of a method for deriving a decoder-side intra-mode derivation predictor for corresponding samples of a coding unit of a picture according to an embodiment of the present invention;

[0081] Fig. 9 The diagram shows splitting the template block of the reconstructed part of the image into three regions;

[0082] FIG10 illustrates selection of an IPM based on local HoG calculation according to an embodiment of the present invention;

[0083] Fig.11 A flowchart of selecting an IPM based on a local HoG according to an embodiment of the present invention is illustrated;

[0084] Fig.12 An example of splitting a CU into four CU regions is illustrated, namely, an upper left region TL, an upper right region TR, a lower left region BL, and a lower right CU region BR;

[0085] Fig.13 A block diagram of an apparatus for deriving a DIMD predictor according to an embodiment of the present invention is illustrated;

[0086] Fig.14 A block diagram of a decoder according to an embodiment of the present invention is illustrated;

[0087] Fig.15 A block diagram of an encoder according to an embodiment of the present invention is illustrated; and

[0088] Fig.16 A block diagram of an electronic device according to an embodiment of the present invention is illustrated. DETAILED DESCRIPTION

[0089] Illustrative embodiments of the present invention will be described below with reference to the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding and should be regarded as illustrative only. Thus, those of ordinary skill in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present invention. Moreover, for the sake of clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0090] In the present invention, the term “and / or” is intended to cover all possible combinations and sub-combinations of the listed elements, including any one, any sub-combination or all of the elements listed individually, and does not necessarily exclude additional elements.

[0091] In the present invention, the phrase "at least one of... or..." is intended to cover any one or more of the listed elements, including any one, any subcombination or all of the elements listed individually, without necessarily excluding any additional elements, and without necessarily requiring all elements.

[0092] In the present invention, the term "codec" refers to "encoding" or "decoding", as is apparent from the context of the described embodiments. Likewise, the term "codec" refers to "encoder" or "decoder".

[0093] Figure 8 A flowchart of a method for deriving a decoder-side intra mode derivation predictor for corresponding samples of a coding unit of a picture according to an embodiment of the present invention is illustrated, the method comprising the following steps:

[0094] S200: Split a template block adjacent to the CU into multiple template block areas.

[0095] S202: Select individually in each template block area: "none" intra prediction mode (IPM), that is, zero IPM; one IPM or more than one IPM, where the selection involves selecting from IPMs determined individually for the entire CU or for one or more regions of the CU. For example, 1 or more IPMs can be selected for each template block area, but 0, 1 or more than one IPM (i.e., IPM / region) can also be selected for each template block area. According to an embodiment, a maximum number of IPMs can be allocated for the entire template block.

[0096] S204: Determine a mixing weight using the selected IPM.

[0097] S206: Generate a DIMD predictor by mixing at least the selected IPM using the determined mixing weight.

[0098] In other words, an embodiment of the present invention provides a method that splits a template block into several regions, calculates the HoG of each region, determines and selects (one or more) IPMs from the peaks in the HoGs of these regions, and finally calculates a mixing weight from the HoG calculated for a block (e.g., the entire template block) including at least a plurality of template block regions. In other words, the HoG of each region is used to select one or more IPMs on which prediction is to be performed, but, in order to determine the mixing weight, a more global HoG created on some or all regions (e.g., on the entire template block) is used, and more specifically, the cumulative amplitude of the selected IPM included in the global HoG is used.

[0099] It should be noted that the present invention is not limited to determining IPM statistics based on HoG (based on which one or more IPMs are selected). According to other embodiments, other methods can be applied to determine IPM statistics, for example, a Hough transform or a gradient-based Hough transform can be used on the filter window size to determine the direction for each center pixel (of the filter window), and then the histogram of the directions established can allow the selection of peak directions (and matching IPMs).

[0100] According to an embodiment of the present invention, IPM selection is performed in the corresponding template block area. Fig. 9 The diagram shows the splitting of the template block of the reconstructed part of the picture into three regions. Fig. 9 A CU 256 that is currently processed or predicted and is adjacent to a template block 258 is illustrated. The template block 258 is split into a left template block or left template block area 258a, an upper template block or upper template block area 258b, and an upper left template block or upper left template block area 258c by the boundaries or edges 256x and 256y of the CU 256. The template block 258 includes samples or pixels of corresponding rows and columns that can be used for filtering processing, and as described above, the filter can have a fixed 3x3 filtering window, and among the samples represented by circles in the template block 258, a corresponding amplitude and angle are determined for each middle line sample 262, which is a center sample for the filtering window 280. Fig. 9 Also illustrated is a current center sample 282 of the window 280 .

[0101] According to an embodiment of the present invention, constraints are provided for the selection of IPMs for each template block region so that a portion of the current CU 256 can be better predicted. The advantages of embodiments of the present invention are seen in the fact that content characteristics can be locally adjusted because different IPMs better match different regions of the currently processed or predicted CU 256. In contrast to the method of the present invention, conventional methods only select the two best IPMs from the entire template block, but a certain template block region may have content characteristics that exhibit slightly different directions or angles, so the number of possible IPMs (for example, 67 IPMs in the case of wide-angle intra-frame modes) may create a wide peak in the HoG, for which the selected mode may match in most template block regions, but may not be suitable for the best local characteristics of a specific template block region. Therefore, unlike the conventional method (according to which the HoG is calculated over the entire template block and the two highest peaks in the HoG are selected as two IPMs), according to an embodiment of the present invention, one HoG is calculated for each template block area, that is, there is one HoG for the upper template block area 258b, one HoG for the left template block area 258a, and one HoG for the upper left template block area 258c. In addition, a predetermined or maximum number of IPMs is allocated to each of the mentioned template block areas, and the IPM in each template block area is selected as the peak of the cumulative amplitude in the HoG of the corresponding template block area.

[0102] FIG10 illustrates the selection of an IPM based on local HoG calculation according to an embodiment of the present invention. FIG10 assumes that an IPM is selected for each template block region, and FIG10( a) illustrates the selection of an IPM based on the HoG generated only for the upper template block region. Above Select an IPM. In Figure 10, Figure 7 As in FIG. 10( b ), the orientation of the IPM is indicated by the corresponding arrow, and the amplitude is represented by the length of the arrow. In the depicted embodiment, the cumulative amplitude of IPM5 in the upper template region is the highest or above a certain threshold, so IPM5 is selected. FIG. 10( b ) illustrates the calculation of HoG for the IPM5. Left , which illustrates the cumulative amplitudes of the corresponding IPMs determined in the left template block region. As shown, the highest peak, ie, the IPM with the highest cumulative amplitude, is selected, which is IPM28 in the depicted embodiment.

[0103] It should be noted that the present invention is not limited to selecting only one IPM from the corresponding template block area, but more than one IPM or no IPM may be selected for each template block.

[0104] According to an embodiment, the number of IPMs for each template tile region may depend on the size / shape of the currently processed CU 256. For example, if the height of the current CU 256 is greater than the width, two IPMs may be selected for the left template tile region, one IPM may be selected for the upper template tile region, and one IPM may be selected for the upper left template tile region. According to other embodiments, one IPM may be selected for the left and upper left template tile regions, while no IPM may be selected for the upper template tile region. According to other embodiments, one IPM may be selected for the left and upper template tile regions, while no IPM may be selected for the upper left template tile region.

[0105] For example, when considering the size of the current CU, the total number of IPMs (for the entire template block) is higher for large CU sizes (e.g., 16x16 and above), and lower for smaller CU sizes (e.g., 4x4, 8x8 and below). The total number of IPMs is selected for the entire template block, and then those IPMs are spread over each region, with larger regions having the most IPMs and smaller regions having fewer IPMs. For example, if 4 IPMs can be selected for a 16x16 CU, there can be 2 IPMs in the template block regions along the long sides, 1 (or 2) IPMs in the template block regions along the short sides, and 0 (or 1) IPMs in the top left template block region.

[0106] According to a further embodiment, in the case where two or more angular IPMs can be selected, such that, for example, in the case where three IPMs are selected, two IPMs can be selected for the larger template block area (i.e., the template block area with the most reconstructed samples / pixels), and the remaining IPMs can be selected from the second largest template block area. In the case where two IPMs are to be selected, for example, the third mode is Planar or DC mode, then two IPMs can be selected in the largest template block area.

[0107] According to further embodiments, different IPMs or the same IPM may be selected for each template block region. above Select the two best IPMs for the upper template block area, which can be obtained from HoG above-left Select an optimal IPM for the upper left template block and HoG left The two best IPMs are selected for the left template block region, resulting in a total of five different IPMs, some or all of which can be the same IPM.

[0108] According to other embodiments, in the case where the number of IPMs to be selected is greater than two, then two IPMs may be selected in the template block area presenting the maximum cumulative amplitude, wherein the maximum cumulative amplitude may be determined by the peak value of the cumulative amplitude provided by the global HoG calculated over the entire template block 258. The remaining IPMs may be selected in the one or more template blocks for which the maximum cumulative amplitude derivable from the corresponding local HoG is the highest. For example, in the case where three IPMs are to be selected, if the maximum amplitude in the global HoG is presented in the upper template block area, then the two IPMs with the highest cumulative amplitude are selected in the upper template block, then the highest amplitudes in the local HoGs in the upper left and left template block areas are compared, and the last IPM is selected from the area where the local HoG indicates the highest cumulative amplitude. This principle may be repeated in the case of selecting more IPMs.

[0109] According to an embodiment, only one IPM may be selected from the template block region having the largest cumulative amplitude in the global HoG, and another IPM may be selected from another template region, for example, one IPM may be selected from each template block region, and / or the remaining IPMs may be selected from the template block region having the most samples or the template block adjacent to the long side of the CU 256.

[0110] According to other embodiments, the selected IPMs in different template block regions may be different. However, if the two selected IPMs are exactly the same in different template regions, the second peak in the local HoG calculated on the corresponding template block region may be selected. According to other embodiments, in the case where the two selected IPMs are exactly the same in different template block regions, the selected IPM calculated based on the local template block region HoG may be discarded, and only the IPM associated with the first template block region may be retained.

[0111] According to other embodiments, the total number of IPMs to be selected may be fixed. For example, if the total number of possible IPMs is two, one IPM may be selected in the upper template block area, another IPM may be selected in the left template block area, and no IPM may be selected in the upper left template block area.

[0112] Fig.11 The flowchart of selecting an IPM based on a local HoG according to an embodiment of the present invention is illustrated. First, at S210, the neighborhood of the currently processed or predicted CU 256 is analyzed to determine the number of available intra-frame top and intra-frame left samples, that is, for determining the template block 258. At S212, a global HoG is constructed for the entire template block 258, and a corresponding local HoG is constructed for each template block area, that is, a HoG is constructed for the available left, above, and upper left template block areas. At S214, for each template block area, the number of represented IPMs is determined, where the number may depend on, for example, the size of the currently predicted CU 256. When the cumulative amplitude of the IPM in the HoG exceeds a certain threshold (e.g., 50), or when the number of samples in the template block associated with the same IPM exceeds a predefined value, it can be considered that an IPM is represented in the template area. At S216, one or more IPMs are selected for each template area based on the template area HoG. In Fig.11In the embodiment, it is assumed that at most two IPMs are selected in total, so it is determined at S218 whether no IPM, one IPM, or two IPMs are selected. In the case where no IPM is selected, the Planar or DC mode is used as the DIMD predictor, as indicated at S220. In the case where only one IPM is selected, the selected one IPM is used as the DIMD predictor, as indicated at S222. On the other hand, in the case where it is determined that two IPMs are selected, the mixing weight of each selected IPM is determined based on the cumulative amplitude ratio obtained from the global or overall HoG, and the Planar mode can be mixed with a weight of 1 / 3, as indicated at S224. At S226, the prediction of each selected IPM is calculated, and at S228, the DIMD predictor is obtained by mixing the two selected modes with the Planar mode using the determined weights.

[0113] According to an embodiment, the mixing weights may be calculated by first selecting an IPM using, for example, HoG, and then determining the mixing weights for the selected IPMs by weighting a first value obtained from one or more predefined characteristics (such as cumulative amplitude or any other suitable characteristic) associated with one or more selected IPMs based on the number of selected IPMs. Optionally, in addition to the mixing weights of the IPMs, the mixing weights for the Planar or DC modes may be determined based solely on the number of selected IPMs.

[0114] Further, more detailed embodiments for determining or setting the blending weights will now be described, but the present invention is not limited to these embodiments. As described above, the IPM selection can be performed on each template block region HoG, but the actual IPM blending weight to be used is calculated based on the cumulative amplitude obtained from the HoG for the entire template block. The blending weights can be adjusted according to the number of different IPMs selected, so for example, where N angular IPMs are selected and merged (optionally, also merged with Planar or DC mode), the weights for each IPM can be calculated as follows:

[0115]

[0116] where w i is the mixing weight of the ith IPM, M i is the i-th selected IPM, i belongs to [1, N], ampl(X) is the cumulative amplitude of IPM X in the global HoG, and N is the number of selected angular IPMs. Planar is the mixing weight of the Planar mode.

[0117] In the presently described embodiment, the blending weights are derived using cumulative amplitudes obtained from a global HoG obtained over the entire template patch 258. According to other embodiments, the blending weights may be derived from the ratio of cumulative amplitudes extracted from the HoGs calculated over those template patch regions for which an IPM is selected.

[0118] According to other embodiments, in any of the described embodiments, the Planar mode may be replaced by a DC mode.

[0119] According to other embodiments, only IPM may be used, and the blending weights may be adjusted as follows:

[0120]

[0121] According to other embodiments, the mixing weight can be calculated by the ratio of the selected IPM amplitude extracted from the local HoG and the other selected IPM cumulative amplitudes extracted from the global HoG or the local HoG (i.e., the numerator of the above equation depends on the amplitude collected on the local HoG, while the denominator depends on the amplitude collected on either the local or global HoG).

[0122] According to the above-described embodiment of the present invention, HoGs are created for the left, top, and top-left template block regions. However, the present invention is not limited to dividing the entire template block 258 in this manner. More specifically, the template block may be separated into other template block regions. For example, the CU 256 may be split into a plurality of regions along one or more vertical lines and one or more horizontal lines, and when the corresponding vertical / horizontal lines that create the corresponding CU regions in the CU 256 are extended into the template block 258, the template block 258 is separated into the corresponding template block regions adjacent to the corresponding CU regions, and the template block regions may also be referred to as partial template block regions. Fig.12 The diagram illustrates an example where CU 256 is split into four CU regions, namely, an upper left region TL, an upper right region TR, a lower left region BL, and a lower right CU region BR. The line that divides CU 256 into the corresponding regions is extended into the template block 258, thereby generating partial template block regions 258a, 258b, and 258c. When compared with the initial embodiment, the partial template block region 258a is a portion of the above-mentioned left template block region, which is actually a portion thereof that is adjacent only to the lower left CU region. The partial template block region 258b is a portion of the above-mentioned upper template block region, which is actually a portion thereof that is adjacent only to the upper right CU region. The third partial template block region 258 includes the above-mentioned upper left template block region and those portions of the upper and left template block regions that are adjacent only to the upper left CU region.

[0123] According to a further embodiment, the above principle of generating local HoG may also be applied to generate corresponding local HoG for the above partial template block areas 258a-258c to select corresponding IPM to be used for DIMD processing.

[0124] Further embodiments

[0125] So far, the inventive concept of the present invention has been described with reference to aspects and embodiments of a method for determining a DIMD predictor. According to further embodiments, the present invention also provides an apparatus for deriving a DIMD predictor and an encoder / decoder comprising such an apparatus.

[0126] Fig.13 A block diagram of an apparatus for deriving a decoder-side intra mode derivation (DIMD) predictor for corresponding samples of a coding unit (CU) of a picture is shown. The apparatus 300 includes the following modules:

[0127] 302: A template block splitting module is configured to split a template block adjacent to a CU into multiple template block areas.

[0128] 304: An intra prediction mode (IPM) selection module, configured to select zero, one or more intra prediction modes (IPMs) in each template block region, wherein the selection involves selecting from individually determined IPMs.

[0129] 306: A mixing weight determination module is configured to determine a mixing weight using the selected IPM.

[0130] 308: A DIMD predictor generation module configured to generate a DIMD predictor by mixing at least the selected IPM using the determined mixing weight.

[0131] Fig.14 A block diagram of a decoder for decoding a coded data stream according to an embodiment of the present invention is illustrated, wherein the coded data stream includes data representing a picture. The decoder 400 includes the following modules:

[0132] 402: A decoder module configured to decode a picture from an encoded data stream.

[0133] 404: A prediction module comprising means for deriving a decoder-side intra mode derivation (DIMD) predictor for corresponding samples of a coding unit (CU) of a picture according to an embodiment of the present invention.

[0134] Fig.15 The block diagram of an encoder for encoding a picture into a coded data stream according to an embodiment of the present invention is illustrated, wherein the coded data stream includes data representing the picture. The encoder 500 includes the following modules:

[0135] 502: An encoder module is configured to receive an original picture and encode the picture into a coded data stream.

[0136] 504: A prediction module, the prediction module comprising means for deriving a decoder-side intra mode derivation (DIMD) predictor for corresponding samples of a coding unit (CU) according to an embodiment of the present invention.

[0137] The present invention provides a device for encoding an image into a coded data stream, the coded data stream comprising data representing the picture, the device comprising:

[0138] an encoder module configured to receive an original picture and encode the picture into a data stream, and

[0139] A prediction module comprising means of the invention for deriving a decoder-side intra mode derivation (DIMD) predictor for corresponding samples of a coding unit (CU) of a picture.

[0140] Although some aspects of the disclosed concepts have been described in the context of an apparatus, it is clear that these aspects also represent a description of a corresponding method, where a block or device corresponds to a method step or a feature of a method step. Similarly, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus.

[0141] Fig.16 900 is a block diagram illustrating an electronic device 900 according to an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are described only as examples and are not intended to limit the implementation of the present invention described and / or claimed herein. The device 900 includes a computing unit 901 to perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 902 or loaded from a storage unit 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data for the operation of the storage device 900 can also be stored. The computing unit 901, ROM 902, and RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0142] The components in the device 900 are connected to the I / O interface 905, including: an input unit 906, such as a keyboard, a mouse; an output unit 907, such as various types of displays, speakers; a storage unit 908, such as a disk, an optical disk; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks. The computing unit 901 can be composed of various general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any suitable processors, controllers, microcontrollers, etc. The computing unit 901 performs the various methods and processes described above, such as image processing methods. For example, in some embodiments, the image processing method can be implemented as a computer software program tangibly implemented on a machine-readable medium (such as a storage unit 908). In some embodiments, part or all of the computer program may be loaded and / or installed on the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the above-described image processing method may be performed. In some embodiments, the computing unit 901 may be configured to perform the image processing method in any other suitable manner (e.g., by means of firmware).

[0143] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0144] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0145] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0146] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0147] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0148] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0149] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0150] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the principles of the present invention should be included in the protection scope of the present invention.

[0151] References

[0152] [1]H.264:Advanced video coding for generic audiovisual services,https: / / www.itu.int / rec / T-REC-H.264-202108-P / en

[0153] [2]H.265: High efficiency video coding, https: / / www.itu.int / rec / T-REC-H.265-202108-P / en

[0154] [3]H.266:Versatile video coding,https: / / www.itu.int / rec / T-REC-H.266-202008-I / en

[0155] [4]AV1 Bitstream&Decoding Process Specification, http: / / aomedia.org / av1 / specification /

[0156] [5] E.Mora et al., “CE3-related: Decoder-side Intra Mode Derivation”, JVET-L0164, Macao, October 2018

[0157] [6]CE3:Decoder-side Intra Mode Derivation(tests 3.1.1,3.1.2,3.1.3and3.1.4),JVET-M0094,Marrakech,January 2019

[0158] [7] M. Abdoli et al., E. Mora, T. Guionnet, M. Raulet, Non-CE3: Decoder-side IntraMode Derivation with Prediction Fusion, JVET-N0342, Geneva, March 2019

[0159] [8] M. Abdoli et al., “Non-CE3: Decoder-side Intra Mode Derivation with Prediction Fusion Using Planar”, JVET-O0449, Gothenburg, July 2019

[0160] [9] J. Zhao et al., “EE2-Related: Improvements of Decoder-Side Intra Mode Derivation”, JVET-V0087, online, April 2021.

[0161]

[10] J. Zhao et al., “EE2 Test 2.2: DIMD with multiple blending modes”, JVET-W0054, online, July 2021.

[0162]

[11] X. Li et al., “EE2-related: Implicit derivation of DIMD blend modes”, JVET-W0067, online, July 2021.

[0163]

[12] X. Li et al., “EE2-related: A combination of CIIP and DIMD / TIMD”, JVET-W0068, online, July 2021.

[0164]

[13] J. Zhao et al., “EE2 Related–DIMD with implicitly derived multiple blending modes”, JVET-W0126, online, July 2021.

[0165]

[14] X. Li et al., “EE2-3.1: Combination of CIIP and DIMD / TIMD”, JVET-X0098, online, October 2021.

[0166]

[15] C. Zhou et al., “EE2-related: Optimization on the second mode derivation of DIMD blending mode”, JVET-X0115, online, October 2021.

[0167]

[16] T. Dumas et al., “AHG12: removing a discontinuity in the discrete angle comparison in DIMD,” JVET-X0139, online, October 2021.

[0168]

[17] X. Xiu et al., “AHG12: Removal of floating operations in DIMD and TIMD,” JVET-X0149, online, October 2021.

[0169]

[18] K. Cao et al., “Non-EE2: Fix for histogram of gradients derivation in DIMD mode”, JVET-X0156, online, October 2021.

[0170]

[19] K. Naser et al., “AhG12: Removed DIMD from MPM list of TIMD”, JVET-Y0097, online, January 2022.

[0171]

[20] S. Yoo et al., “EE2-related: Clean-up on DIMD,” JVET-Y0131, online, January 2022.

[0172]

[21] J. Zhao et al., “Non-EE2: DIMD Flag Signalling Clean-up”, JVET-Y0144, online, January 2022.

[0173]

[22] J. Pfaff et al., “Intra Prediction and Mode Coding in VVC”, IEEE Transactions on Circuits and Systems for Video Technology, Vol. 31, No. 10, October 2021.

Claims

1. A method for deriving a decoder-side intra mode derivation (DIMD) predictor for corresponding samples of a codec unit (CU) (256) of a picture (102), the method comprising: Splitting (S200) a template block (258) adjacent to the CU (256) into a plurality of template block regions, Individually selecting (S202) zero, one or more intra prediction modes (IPMs) in each template block region, wherein the selecting involves selecting from IPMs individually determined for the entire CU (256) or for one or more regions of the CU (256), The blending weights are determined (S204) using the selected IPM, and The DIMD predictor is generated (S206) by mixing at least the selected IPM using the determined mixing weight.

2. The method of claim 1, wherein the number of IPMs selected for each template block region depends on the size of the CU (256), and wherein - when the CU (256) is of square shape, ○ Select the same number of IPMs in all template block areas, or o Selecting the same number of IPMs in all template block regions adjacent to the edges of the CU (256), and selecting no IPMs in the template block regions adjacent only to the corners of the CU (256), or o selecting one or more IPMs in the template block regions adjacent only to the corners of the CU (256), No IPM is selected in any template block area adjacent to the edge of the CU (256), or - when the CU (256) is rectangular in shape, ○ Selecting a first number of IPMs in a template block area adjacent to a long side of the CU (256), selecting a second number of IPMs in a template block area adjacent to a short side, and selecting a third number of IPMs in an upper left template block area, wherein the first number is higher than the second number and the third number, and the second number is the same as or different from the third number, or o Select the same number of IPMs in the template block regions adjacent to the short sides of the CU (256) and in the template block regions adjacent to the long sides of the CU (256), and select no IPMs in the upper left template block region, or ○ Select the same number of IPMs in the template block areas adjacent to the short side of the CU (256) and for the upper left template block area, and select no IPMs in the template block areas adjacent to the long side of the CU (256), or ○ The same number of IPMs are selected in the template block areas adjacent to the long sides of the CU (256) and for the upper left template block area, and no IPMs are selected in the template block areas adjacent to the short sides of the CU (256).

3. The method of claim 1 or 2, wherein the number of IPMs selected for each template block region depends on the size of the CU (256), and Wherein, when the CU (256) has a first size, a first number of IPMs is selected, and when the CU (256) has a second size, a second number of IPMs is selected, and the second size is larger than the first size.

4. A method as claimed in any one of the preceding claims, wherein a predefined number of IPMs are selected for each template block area, wherein: For the corresponding template block areas, the predefined numbers are the same or different.

5. The method of any one of the preceding claims, wherein a first number of IPMs is selected in a template block region having a maximum cumulative amplitude among all template block regions, and a second number of IPMs is selected in other template block regions, wherein the first number is higher than the second number.

6. A method as claimed in any one of the preceding claims, wherein: If two or more selected IPMs in different template tile regions are identical, the IPM for the first or last processed template tile region is selected and retained, and the other identical IPMs are discarded.

7. A method as claimed in any preceding claim, wherein the total number of selectable IPMs is fixed.

8. The method of any one of the preceding claims, wherein the blending weights for the selected IPMs are determined by weighting first values ​​obtained from one or more predefined characteristics associated with the one or more selected IPMs based on the number of selected IPMs, and in, Optionally, in addition to the mixing weight of the IPM, the mixing weight of the Planar or DC mode is determined based only on the number of selected IPMs.

9. A method as claimed in any one of the preceding claims, wherein - in case an IPM is selected in all template patch regions, then said blending weights are derived from one or more predefined characteristics associated with said one or more selected IPMs and obtained from said entire template patch, or - in case an IPM is selected in one or more but not all of said template block regions, then said blending weights are derived from one or more predefined characteristics associated with said one or more selected IPMs and obtained from the template block regions from which the one or more IPMs are selected, or - In case an IPM is selected in only one template patch region, then said blending weights are derived from one or more predefined characteristics associated with said one or more selected IPMs and obtained from said one template patch region.

10. The method of any one of the preceding claims, wherein the plurality of template block regions of the template block (258) comprises - a left template block area (258a), an upper template block area (258b) and an upper left template block area (258c), or - In case the CU (256) is split into multiple CU regions, the corresponding template block regions associated with the CU regions adjacent to the template block (258) and an optional upper left template block region (258c).

11. A non-transitory computer-readable storage medium comprising instructions, which, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 10.

12. An apparatus (300) for deriving a decoder-side intra mode derivation (DIMD) predictor for corresponding samples of a codec unit (CU) (256) of a picture (102), the apparatus (300) comprising: A template block splitting module (302) is configured to split a template block (258) adjacent to the CU (256) into a plurality of template block regions, an intra prediction mode (IPM) selection module (304) configured to select zero, one or more intra prediction modes (IPMs) in each template block region, wherein the selection involves selecting from individually determined IPMs, a blending weight determination module (306) configured to determine a blending weight using the selected IPM, and A DIMD predictor generation module (308) is configured to generate the DIMD predictor by blending at least the selected IPM using the determined blending weight.

13. An apparatus (400) for decoding an encoded data stream (152), the encoded data stream (152) comprising data representing a picture (102), the apparatus (400) comprising: a decoder module (402) configured to decode the picture (102) from the encoded data stream (152), and A prediction module (404) comprising means (300) for deriving a decoder-side intra mode derivation (DIMD) predictor for corresponding samples of a codec unit (CU) (256) of the picture (102) according to claim 12.

14. An apparatus (500) for encoding a picture (102) into a coded data stream (120), the coded data stream (120) comprising data representing the picture (102), the apparatus (500) comprising: an encoder module (502) configured to receive an original picture (102) and encode the picture (102) into the data stream (120), and A prediction module (504) comprising means (300) for deriving a decoder-side intra mode derivation (DIMD) predictor for corresponding samples of a codec unit (CU) (256) of the picture (102) according to claim 12.