Method and apparatus for DIMD edge detection adjustment and encoder / decoder including the same

Abstract

A method of determining filter characteristics of a filter for filtering respective samples of a template block adjacent to a codec unit (CU) of a picture to obtain a respective intra prediction mode (IPM) from the template block for deriving a decoder-side intra mode derivation (DIMD) predictor for the respective samples of the CU is described. The method includes determining a size and / or position of a filtering window of the filter according to one or more characteristics of the CU and / or the template block or according to information data associated with the picture.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is based on and claims priority from European patent application No. “22167219.9” filed on April 7, 2022, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present invention generally relates to the field of encoding / decoding pictures, images or videos, and embodiments of the present invention relate to improvements regarding intra prediction, and more specifically, to improvements regarding decoder-side intra mode derivation (DIMD) processing. More specific embodiments of the present invention relate to DIMD edge detection adjustments. Background Art

[0004] Encoding and decoding of pictures, images or videos are performed according to certain standards, for example, according to the Advanced Video Codec (AVC) standard (see reference [1]), the High Efficiency Video Codec (HEVC) standard (see reference [2]) or the Versatile Video Codec (VVC) standard (see reference [3]).

[0005] Figure 1 A block diagram of a standard video compression system 100 operating according to the VVC standard is illustrated in FIG. The standard video codec 100 compresses and encodes pictures 102 of a video sequence. The pictures 102 to be encoded are partitioned into blocks 104, also referred to as codec units (CUs). The encoder 100 includes a pre-coding filter 106 and a prediction residual signal former 108, which generates a prediction residual signal 110 to measure the deviation between the prediction signal 112 and a signal 114 output by the filter 106. The encoder also includes a transformer 116, a quantizer 118, and provides an output bitstream or data stream 120 using an entropy codec 122. In addition, the encoder 100 includes a prediction stage 124 for determining the prediction signal 112, and the prediction stage 124 includes an inverse quantizer or inverse quantizer 126, an inverse transformer 128, a combiner 130, an in-loop filter 134, a motion estimator 138, an intra-frame / inter-frame mode selector 140, an inter-frame predictor 142 and an intra-frame predictor 144.

[0006] As reference Figure 1The described video codec 100 compresses and encodes a picture 102 of a video sequence as follows. The picture 102 to be encoded is partitioned into blocks or CUs 104. Each codec unit 104 is encoded using an intra or inter codec mode. When a CU is encoded in intra mode, intra prediction is performed by an intra predictor 144. Intra prediction includes predicting the current CU 114 being encoded with the help of coded, decoded and reconstructed picture samples located around the current CU (e.g., the top and left of the current CU). Intra prediction is performed in the spatial domain. In inter mode, motion compensation and motion estimation are performed by a motion estimator 138, which searches for a CU that is a good predictor of the current CU in one or more reference pictures provided by a picture buffer 136 and used to predictively encode and decode the current picture. For example, a good predictor of the current CU is a predictor similar to the current CU, that is, the distortion between the two CUs is low or below a certain threshold. Motion estimation can also consider signaling to the predictor to optimize the rate cost of the rate-distortion trade-off. The output of the motion estimation step is one or more motion vectors and reference indices associated with the current CU. Motion compensation is then used to predict the current CU with the aid of one or more motion vectors and reference picture indices determined by the motion estimator 138. Basically, the block or CU contained in the selected reference picture and pointed to by the determined motion vector is used as the prediction block of the current CU. The encoder 100 selects one of the intra-frame coding mode or inter-frame coding mode for encoding the CU with the aid of the selector 140 and indicates the intra / inter decision, for example, with the aid of a prediction mode flag. The prediction residual 110 is then transformed and quantized by blocks 116 and 118, and the quantized transform coefficients as well as the motion vectors and other syntax elements are entropy encoded and written to the output bitstream 120. The encoder 100 can skip the transform stage 116 and directly apply quantization to the untransformed residual signal 110 in a so-called transform-skip coding mode. After encoding a block or CU, the encoder decodes the CU and reconstructs it to obtain a reconstructed signal 132, which can be used as reference data for predicting future CUs or blocks to be encoded. The quantized transform coefficients 110' are dequantized and inverse transformed, resulting in a decoded predicted CU or block 110", and the decoded prediction residual and the predicted block are then combined (usually added) at 130 to provide a reconstructed block or CU 132. An in-loop filter 134 is applied to the reconstructed picture to reduce compensation artifacts. For example, a deblocking filter, a sample adaptive offset (SAO) filter, and an adaptive loop filter (ALF) can be applied to reduce coding artifacts. The filtered picture is stored in a buffer 136 (also called a decoded picture buffer (DPB)) so that it can be used as a reference picture for encoding and decoding subsequent pictures.

[0007] Figure 21 is a block diagram of a video decoder 150 for predictively decoding a picture or video provided at an output 154 of the decoder 150 from a data stream 152. The decoder 150 includes an entropy decoder 156, a partition block 158, an inverse quantizer 160, an inverse transformer 162, a combiner 164, an in-loop filter 166, an optional post-decoding processor 168, and a prediction module 170. The prediction module 170 includes a decoded picture buffer 180, a motion compensator 182, and an intra-frame predictor 184.

[0008] The encoded pictures of the video sequence are decompressed and decoded by the decoder 150 as follows. The input bitstream 152 is entropy decoded by the decoder 156, which provides, for example, block partition information, the codec mode for each codec unit, the transform coefficients contained in each transform block, prediction information (such as intra-frame prediction mode), motion vectors, reference picture indexes, and other codec information. The block partition information indicates how the picture is partitioned, and the decoder 150 can divide the input picture into codec tree units (CTUs) (usually 64×64 or 128×128 pixels in size) and divide each CTU into rectangular or square codec units (CUs) according to the decoded partition information. The entropy decoded quantized coefficients 172 are dequantized 160 and inverse transformed 162 to obtain a decoded residual picture or CU 174. The decoded prediction parameters are used to predict the current block or CU, that is, whether the predicted block is obtained by its intra-frame prediction or by its motion compensated temporal prediction. The prediction process performed on the decoder side is the same as the prediction process performed on the encoder side. The decoded residual block 174 is added to the predicted block 176, thereby producing a reconstructed current image block 164. The in-loop filter 166 is applied to the reconstructed picture or image, which is also stored in the decoded picture buffer 180 for use with reference pictures for future picture decoding. As mentioned above, the decoded picture may also be subjected to post-decoding processing, such as for performing an inverse color transform, such as a conversion from YCbCr4:2:0 to RGB 4:4:4.

[0009] As mentioned above, the intra-frame prediction mode to be used for decoding can be carried in the bitstream provided by the encoder and received by the decoder, but, according to other methods, instead of introducing the actual intra-frame prediction mode into the bitstream, the intra-frame prediction mode can be derived by gradient analysis using reconstructed pixels adjacent to the currently processed CU. In other words, the intra-frame prediction mode is not explicitly indicated in the bitstream, but is implicit. This method is called decoder-side intra-frame mode derivation (DIMD), which can be signaled using a simple flag and then the actual intra-frame mode is derived during the reconstruction process, such as the reconstruction process performed by the prediction block 124 / 170 of the encoder 100 or decoder 150. The encoder 100 can encode information whether DIMD is used for the current CU into the bitstream 120, and in the case where DIMD is not used, the actual intra-frame mode is signaled by the encoder and parsed from the bitstream by the decoder.

[0010] Taken from reference [5] Figure 3 An example of an intra-block general reconstruction process is illustrated, which may be applied at the encoder 100 or at the decoder 150. During the parsing process 200, intra-block decoding is determined, as indicated at block 202, and pixel residual parsing 204 is performed, as well as DIMD flag parsing 206. If the DMID flag 206 is zero, then intra-prediction mode parsing 208 is performed to obtain an intra-mode 210 to be used when predicting and reconstructing pixels during a reconstruction process 212, as indicated at 216. If the DIMD flag 206 is set to one, then intra-prediction derivation 218 is performed during the reconstruction process 212, which produces a derived intra-mode 220, which is used in the reconstruction process together with the pixel residual 214 at 216.

[0011] Further details of the DIMD method are described in references [5] to [8]. Generally speaking, DIMD processing is based on a reconstructed template area adjacent to the currently processed CU, the width and height of which are three samples wide. The template area includes a left area, an upper area, and an upper left area. These areas are also referred to as the left template area, the upper template area, and the upper left template area in the following text. In the template area, a corresponding edge detection filter (such as a 3×3 horizontal and vertical Sobel filter) is applied to determine the amplitude and angle of the brightness direction or orientation of each midline sample of the template area. A histogram of gradients (HoG) is calculated, and each entry corresponds to a conventional intra-angle pattern and the accumulated intensity is stored:

[0012] angle=arctan(G hor / G ver )

[0013] amplitude=|Ghor |+|G ver |

[0014] Among them G hor and G ver is the pure horizontal and vertical intensity calculated by the Sobel filter. FIG. 4 taken from reference [8] illustrates a DIMD template block or region used to calculate HoG. FIG. 4( a) illustrates a partially reconstructed picture 250, as indicated by a reconstructed block 252 and an unavailable block 254. In other words, the reconstructed block 252 includes the already reconstructed codec units or blocks of the picture to be reconstructed, while the corresponding codec units or blocks in the unavailable block 254 are still to be reconstructed. The currently processed CU or block 256 and the template block 258 mentioned above are illustrated. FIG4( b) illustrates that the template block 258 is 3 samples or pixels wide in width and height, and as described above, the HoG 260 is calculated using 3×3 horizontal and vertical Sobel filters 259 to obtain an angular intra-prediction mode (also called intra-prediction mode, IPM) based on the amplitude and angle of the luma direction of each intermediate line sample 262 of the template block 258, and obtain the associated cumulative intensity or amplitude for each IPM in the template block. The two most representative IPMs (indicated as M1 and M2 in FIG4( b)) are selected from the HoG 260 and combined with the Planar mode using weights for M1 and M2 derived from the ratio of their cumulative amplitudes and a fixed weight for the Planar mode (e.g., 1 / 3, i.e., 21 / 64 with 6-bit integer precision).

[0015] Taken from reference [8] Figure 5 The diagram illustrates prediction fusion by weighted averaging of M1, M2 and Planar modes. Assuming a square CU 256, corresponding pixels of CU 256 are predicted by the two most representative IPMs M1, M2 taken from HoG 260, thereby obtaining predictor blocks Pred1, Pred2 and Pred3. The corresponding predictor blocks are weighted by weights w1, w2, w3 and fused or combined into a final predictor block including DMID predictors for corresponding samples or pixels in CU 256.

[0016] Figure 6An example of a conventional DIMD predictor derivation process is illustrated. At S100, using the input CU 256, the neighborhood of the input CU 256 is analyzed to determine the number of available upper and left inner samples. At S102, the HoG 260 is constructed for the template block 258, that is, based on the available samples, the angle intra prediction mode IPM is determined by its cumulative amplitude in the HoG 260 (step S104). At S106, it is determined how many IPMs are selected, that is, whether no IPM is selected, whether one IPM is selected, or whether two IPMs are selected. If no IPM is selected, then only the Planar mode (which can be the default mode) is applied, as indicated at S108, to obtain the DIMD predictor. If only one IPM is selected, then the selected IPM is used to obtain the DIMD predictor, as indicated at S110. If two IPMs are selected, then at S112, the mixing weight of each selected mode is determined according to its cumulative amplitude ratio, and the weight for the Planar mode is set to 1 / 3. Predictor blocks Pred1 to Pred3 for each selected IPM and Planar mode are calculated at S114 , and a DIMD predictor is obtained by mixing the two selected IPM and Planar modes using the determined mixing weight at S116 .

[0017] Figure 7 An example of a currently processed CU 256 and an associated template block 258 of width / height T=3 samples or pixels is illustrated. For each intermediate line sample 262, the corresponding orientation is illustrated by the direction of the corresponding arrow, and the corresponding strength of the gradient is indicated by the length of the arrow. The HoG is generated by accumulating the strength of each orientation / IPM entry, and the two most significant modes in the HoG (if they exist) are selected as the modes to be mixed with the Planar mode or the planar intra prediction mode. Figure 7 In the example of , IPMs 5 and 28 have the first and second highest cumulative strengths / amplitudes. As described above, these modes are weighted, and the blending weights are conventionally fixed and / or determined according to the ratio of the cumulative strengths / amplitudes of the selected IPMs. Although different blending modes are proposed in the prior art, all blending modes are global with respect to the currently processed block or CU. More specifically, when considering the conventional approach and when two IPMs (such as M1 and M2) are selected, the blending weights can be calculated as follows

[0018]

[0019] Where w1 is associated with IPM M1, w2 is associated with IPM M2, and w3 is associated with Planar mode. Different modifications of the blending process are described in the prior art, for example, reference

[10] proposes selecting multiple blending modes based on rate-distortion optimization, reference

[11] proposes using HoG to implicitly derive two of the three blending modes, reference

[13] proposes using template matching to implicitly derive the blending mode, and reference

[20] proposes selecting different blending modes based on multiple DIMD modes determined. When DIMD is applied, one or two IPMs are derived from reconstructed neighboring samples (i.e., from the template block), and if two intra modes are derived, they are combined (i.e., blended) with the Planar mode predictor. Conventional solutions always implement a global or CU-level weighted blending of two IPMs with the Planar mode.

[0020] Another codec tool present in H.266 / VVC is the so-called Multiple Reference Line MRL, which is described, for example, in reference

[22] . This codec tool uses multiple reference lines, such as the second or third line to the left and / or above the block or CU to be decoded, instead of using the adjacent reconstructed line as the reference line with reference line index 0. Figure 8 The concept of using multiple reference lines to predict a CU or target block 256 is illustrated. In the depicted example, three reference lines with reference line indices 0, 1, and 2 are illustrated, and the best prediction line index can be signaled using the bitstream. Figure 8 As shown in , padding processing can be applied to the corresponding reference lines.

[0021] Conventionally, in the reconstructed portion 252 of the picture 250, three columns and rows of reconstructed samples or pixels are used for DIMD processing, i.e., the HoG 260 is calculated using a filter having a fixed filter window of 3×3 samples and located at the same position in the template block 258. In reference [6], when calculating the gradient on the pixel located in the left column, top row or upper left sample and directly adjacent to the current CU, the 3×3 Sobel filter 259 can be replaced by a 3×2 filter. In reference [8], instead of using the 3×3 Sobel filter 259 on all pixels or samples in the middle line of the template block 258, the 3×3 Sobel filter 259 can be applied more sparsely, for example, only at one middle line sample in the left template block and one middle line sample in the upper template block. Further aspects of the DIMD processing are described, for example, in the following references:

[0022] - References

[10] ,

[11] ,

[13] and

[20] discuss the mixing of modes,

[0023] - References

[16] and

[18] discuss behavior correction (error repair) relative to the original intent,

[0024] - References [9],

[12] and

[14] discuss combinations with other modes,

[0025] - References

[15] ,

[17] ,

[19] or

[21] discuss syntax or simplifications for DIMD processing.

[0026] However, the above reference Figure 6 The basic DIMD processing remains the same as described in reference [8].

[0027] The above-described prior art method for determining the DIMD predictor for corresponding samples of the currently processed CU is disadvantageous in that the filter used to determine the HoG (on the basis of which the IPM for determining the DIMD predictor is derived), such as the Sobel filter, is fixed to a size of 3×3 samples and is systematically applied only to the middle line samples of the template block, as described above with reference to FIG. 4 , which, however, does not allow for adjustments to certain content characteristics as they may be desirable to improve codec efficiency.

[0028] Therefore, there is a need to provide further improvements in DIMD processing. Summary of the invention

[0029] The present invention provides a method for determining filter characteristics of a filter for filtering corresponding samples of a template block adjacent to a coding unit (CU) of a picture so as to obtain a corresponding intra prediction mode (IPM) from the template block for deriving a decoder-side intra mode derivation (DIMD) predictor for the corresponding samples of the CU, the method comprising:

[0030] - based on one or more characteristics of the CU and / or template block, or

[0031] - Based on the information data associated with the image

[0032] Determine the size and / or position of the filter window of the filter.

[0033] Optionally, in order to determine the size and / or position according to one or more characteristics of the CU and / or the template block,

[0034] - The size of the filter window is derived from one or more of the following

[0035] ○ the number of available samples in the template block,

[0036] ○ CU dimensions,

[0037] ○ The multiple reference line (MRL) index associated with the template block, and / or

[0038] - The position of the filter window is derived from one or more of the following

[0039] ○ the number of available samples in the template block,

[0040] ○ CU dimensions,

[0041] ○ the multiple reference line (MRL) index associated with the template block,

[0042] ○ Distance metric to samples at the edge of a CU.

[0043] Optionally, in order to determine the size and / or position according to one or more characteristics of the CU and / or the template block, the template block includes a plurality of template block regions, the plurality of template block regions include an upper template block region, a left template block region, and an upper-left template block region, and the size of the filtering window for the corresponding template block region is derived according to available rows of samples in the upper template block region, available columns of samples in the left template block region, and available columns and rows of samples in the upper-left template block region,

[0044] The method comprises: for each template block region, selecting a filter whose filter window size corresponds to available columns and / or rows of samples in the corresponding template block region from a plurality of predefined filters having filter windows of different sizes, and

[0045] Wherein, optionally, for a block where the filter window overlaps with an adjacent template block area, the filter is selected according to the following items:

[0046] - the template block area to which the center sample of the filter window belongs, or

[0047] - The template block area with the lowest or highest number of samples contained in the filter window.

[0048] Optionally,

[0049] In order to determine the size and / or position according to one or more characteristics of the CU and / or the template block, when the CU is larger than a certain size, the samples in the template block are filtered by a first filter having a first filtering window size, otherwise, the samples in the template block are filtered by a second filter having a second filtering window size different from the first filtering window size, and / or

[0050] To determine the size and / or position according to one or more characteristics of the CU and / or the template block, the template block includes a plurality of template block areas, the plurality of template block areas include an upper template block area, a left template block area, and an upper-left template block area, and if samples available in the template block area allow use of a plurality of filters with different filter window sizes, then

[0051] - when the CU has a square shape, samples in all template block areas are filtered by the same filter, wherein, optionally, the filter is a filter having a maximum or minimum filtering window size among the multiple filters,

[0052] - when the CU has a rectangular shape, samples in a template block region adjacent to a long side of the CU are filtered by a first filter, and samples in a template block region adjacent to a short side of the CU are filtered by a second filter, the first filter and the second filter have different filter window sizes, and wherein in order to filter samples in the upper left template block region

[0053] o Select one of the first filter and the second filter based on the availability of samples in the top left template block region, or

[0054] One of the first filter and the second filter is always used.

[0055] Optionally, in order to determine the size and / or position according to one or more characteristics of the CU and / or the template block, the template block includes a plurality of template block areas, the plurality of template block areas include an upper template block area, a left template block area, and an upper left template block area, and the size of the filtering window for the corresponding template block area is derived according to an MRL index (such as intra_luma_ref_idx) associated with the corresponding template block area,

[0056] in

[0057] - when the MRL index is zero or when the MRL index is not indicated, selecting a first filter having a first filter window size for the template block region,

[0058] - When the MRL index is different from zero, for example has a certain value or a value exceeding a predefined threshold, then a second filter having a second filter window size is selected for the template block region, the second filter window being larger than the first filter window size.

[0059] Optionally, in order to determine the size and / or position according to one or more characteristics of the CU and / or the template block, the template block includes a plurality of template block areas, the plurality of template block areas include an upper template block area, a left template block area, and an upper left template block area, and a filter with a fixed filtering window size is used for the corresponding template block area,

[0060] Wherein, if the available rows / columns of samples in the template block area exceed the rows / columns of samples covered by the fixed filter window size, then the position of the center sample of the filter window is selected so that the filter window is not adjacent to the CU, thereby offsetting the edge of the CU from the boundary of the filter window by a predefined number of samples,

[0061] Wherein, optionally, if the available rows / columns of samples in the template block area are equal to the rows / columns of samples covered by the fixed filter window size, then the size of the filter window is reduced so that the rows and / or columns of samples in the template block area are covered by the filter window.

[0062] Optionally,

[0063] In order to determine the size and / or position according to one or more characteristics of the CU and / or the template block, when the CU is larger than a certain size, the samples in the template block are filtered by a filter positioned such that the filter window is offset from the edge of the CU by a first number of samples, otherwise the samples in the template block are filtered by a filter positioned such that the filter window is adjacent to the CU or offset from the edge of the CU by a second number of samples, wherein the second number of samples is different from the first number of samples, and / or

[0064] In order to determine the size and / or position according to one or more characteristics of the CU and / or the template block, the template block includes a plurality of template block areas, the plurality of template block areas include an upper template block area, a left template block area, and an upper left template block area, and a filter with a fixed filter window size is used for the corresponding template block area,

[0065] in,

[0066] - When the CU has a square shape, the filter window

[0067] ○ Not adjacent to the CU, so that the edge of the CU is offset from the boundary of the filter window by a predefined number of samples, or

[0068] ○ Adjacent to the CU, so that the edge of the CU is not offset from the boundary of the filter window,

[0069] -When the CU has a rectangular shape,

[0070] ○ In the template block area adjacent to the long side of the CU, the filter window is not adjacent to the CU, so that the edge of the CU is offset from the boundary of the filter window by a predefined number of samples, and in the template block area adjacent to the short side of the CU, the filter window is adjacent to the CU, so that the edge of the CU is not offset from the boundary of the filter window, or

[0071] ○ In the template block area adjacent to the short side of the CU, the filter window is not adjacent to the CU, so that the edge of the CU is offset from the boundary of the filter window by a predefined number of samples, and in the template block area adjacent to the long side of the CU, the filter window is adjacent to the CU, so that the edge of the CU is not offset from the boundary of the filter window,

[0072] as well as

[0073] Optionally, the offset is calculated based on the ratio between the width and height of the CU.

[0074] Optionally, in order to determine the size and / or position according to one or more characteristics of the CU and / or the template block, the template block includes a plurality of template block areas, the plurality of template block areas include an upper template block area, a left template block area, and an upper left template block area, and a filter with a fixed filtering window size is used for the corresponding template block area,

[0075] The position of the filter window for the corresponding template block region is derived according to the MRL index (such as intra_luma_ref_idx) associated with the corresponding template block region.

[0076] in

[0077] - when the MRL index is zero or when no MRL index is indicated, the filtering window is adjacent to the CU such that the edge of the CU is not offset from the boundary of the filtering window,

[0078] - when the MRL index is different from zero, e.g. has a certain value or a value exceeding a predefined threshold, the filtering window is not adjacent to the CU such that the edge of the CU is offset from the boundary of the filtering window by a predefined number of samples,

[0079] Optionally, in order to determine the size and / or position according to one or more characteristics of the CU and / or the template block, the template block includes a plurality of template block areas, the plurality of template block areas include an upper template block area, a left template block area, and an upper left template block area, and a filter with a fixed filtering window size is used for the corresponding template block area,

[0080] wherein, for different offsets between the edge of the CU and the boundary of the filtering window, one or more or all samples contained in the filtering window are compared with samples projected at the edge of the CU using a distance metric (such as the sum of absolute differences, SAD) and the positions of the filtering windows for the upper and left template block areas are derived by selecting the offset that produces the minimum distance,

[0081] wherein, optionally, the offset selection is performed for the entire template block area or for each center sample of the filter window,

[0082] Wherein, optionally, the position of the filter window for the upper left template block area is derived as follows:

[0083] - after determining the filter window positions in the upper and left template block regions, determining the center sample of the filter window of the upper left template block region as the intersection of the row of the center sample of the filter window of the upper template block region and the column of the center sample of the filter window of the left template block region, or

[0084] -For different positions of the filter window in the upper left template block area, one or more or all samples contained in the filter window are compared with multiple samples located in the upper left corner of the CU using a distance metric (such as the sum of absolute differences, SAD), and the position that produces the minimum distance is selected.

[0085] Optionally, when the method of the present invention is implemented at a decoder and when a distance metric is used to determine the position or offset of the filtering window, additional signaling of the distance or offset is required, for example, using information data associated with the picture, because the decoder ignores samples at the edges or inside the CU that have not yet been reconstructed.

[0086] Optionally, when the size and / or position is determined according to information data associated with the picture, the information data associated with the picture uses a high-level syntax structure (such as a picture parameter set, a sequence parameter set or a picture header) to signal the filter size and / or filter position, and

[0087] Wherein, optionally, the high-level grammatical structure includes

[0088] - a syntax element, such as sps_dimd_filter_size, indicating which filter and / or which filter size to use during DIMD processing, and / or

[0089] - An index, such as dimd_filter_shift_minus1, indicating the position or position shift of the center sample of the filter window relative to the edge of the CU in an orthogonal direction.

[0090] The present invention provides a non-transitory computer-readable storage medium including instructions, which, when executed by a computer, cause the computer to perform the method of the present invention.

[0091] The present invention provides a device for decoding a coded data stream, wherein the coded data stream includes data representing a picture, and the device includes:

[0092] A first module is configured to decode a picture or decode a picture and information data associated with the picture from a coded data stream, and

[0093] A second module is configured to derive a decoder-side intra mode derivation (DIMD) predictor for corresponding samples of a coding unit (CU) of the picture, the second module comprising:

[0094] The first submodule is configured as

[0095] - based on one or more characteristics of the CU and / or template block, or

[0096] - Based on the information data associated with the image,

[0097] determining the size and / or position of the filter window of the filter, and

[0098] The second submodule is configured to filter corresponding samples of the template block adjacent to the CU so as to obtain a corresponding intra prediction mode (IPM) for deriving the DIMD predictor from the template block.

[0099] The present invention provides an apparatus for encoding a picture into a coded data stream, wherein the coded data stream includes data representing the picture, the apparatus comprising:

[0100] A first module is configured to receive an original picture and encode the picture into a data stream, and

[0101] A second module is configured to derive a decoder-side intra mode derivation (DIMD) predictor for corresponding samples of a coding unit (CU) of the picture, the second module comprising:

[0102] A first submodule is configured to determine the size and / or position of a filter window of the filter according to one or more characteristics of the CU and / or the template block, and

[0103] The second submodule is configured to filter corresponding samples of the template block adjacent to the CU so as to obtain a corresponding intra prediction mode (IPM) for deriving the DIMD predictor from the template block.

[0104] Optionally, the first module is configured to encode information data associated with the picture into the data stream, the information data signaling a size and / or position of a filter window of a filter used for DMID processing.

[0105] The present invention provides a data stream having a picture encoded into the data stream and information data associated with the picture,

[0106] Therein the information data signals the size and / or position of the filter window of the filter used for DMID processing.

[0107] The technical solution provided according to the embodiments of the present invention has the following beneficial effects.

[0108] In prior art methods, the filter used to determine the HoG (on the basis of which the IPM used to determine the DIMD predictor is derived), such as a Sobel filter, is fixed to a size of 3×3 samples and is systematically applied only to the middle line samples of the template block, which, however, does not allow adjustment for certain content characteristics as they may be desirable to improve codec efficiency. The present invention solves the problems associated with the prior art and is based on the following discovery: introducing more flexibility in filter size / position, i.e., in the selection of reconstructed samples used as reference samples in angular, planar and DC prediction modes, improves the compression gain, especially in cases where the sample values ​​are more representative of the signal direction and in cases where such samples are not located in the direct neighborhood of the currently processed block or CU during the encoding or decoding process.

[0109] It should be understood that the content described in this section is not intended to identify key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become apparent from the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0110] The drawings are illustrative and serve to explain the invention and are not to be construed as limiting the invention to the illustrated embodiments.

[0111] Figure 1 is a block diagram of a conventional video encoder;

[0112] Figure 2 is a block diagram of a conventional video decoder;

[0113] Figure 3 The diagram shows the application Figure 1 The encoder or Figure 2 An example of a generic intra-block reconstruction process at a decoder of;

[0114] FIG4 illustrates a DIMD template block or region used to calculate a histogram of gradients (HoG) in a DIMD process;

[0115] Figure 5 The diagram shows the prediction fusion by weighted averaging of the selected IPM and Planar modes;

[0116] Figure 6 An example of a conventional DIMD predictor derivation process is illustrated;

[0117] Figure 7 illustrates an example of a currently processed and associated template block with a width / height of samples or pixels;

[0118] Figure 8 The concept of using multiple reference lines to predict a CU or target block is illustrated;

[0119] Fig. 9 A flow chart of a method for determining filter characteristics of a filter for filtering corresponding samples of a template adjacent to a coding unit of a picture according to an embodiment of the present invention is illustrated;

[0120] Fig.10 A conventional 5×5 Sobel horizontal or vertical filter is shown;

[0121] Fig.11 A conventional 5×5 Prewitt horizontal or vertical filter is shown;

[0122] Fig.12 A conventional 5×5 Kirsch horizontal or vertical filter is shown;

[0123] Fig.13 A conventional Gaussian blur filter is illustrated;

[0124] FIG14 illustrates the currently processed CU and template block;

[0125] Fig.15 An embodiment is shown in which the template block includes 5×5 samples only in one of the template block regions;

[0126] Fig.16 A flowchart of filter size selection by availability of samples in a template block according to an embodiment of the first aspect of the present invention is illustrated;

[0127] FIG. 17 illustrates an embodiment of deriving filter size based on the dimension or shape of the currently used CU;

[0128] Fig.18 A flowchart of selecting a filter size according to the shape or size of a currently processed CU according to an embodiment of the first aspect of the present invention is illustrated;

[0129] Fig.19 A flowchart of selecting a filter size by using the MRL index state in each template block region according to an embodiment of the first aspect of the present invention is illustrated;

[0130] Fig. 20 illustrates the use of so-called non-adjacent filter windows according to an embodiment of the second aspect of the invention;

[0131] FIG21 illustrates a HoG filter window based on an MRL index and not placed adjacent to a current CU according to an embodiment of the second aspect of the present invention;

[0132] Fig. 22 A flowchart for determining a filter window position according to an MRL index associated with a corresponding template block region according to an embodiment of the second aspect of the present invention is illustrated;

[0133] FIG. 23 illustrates an embodiment of selecting a filter position when a CU has a rectangular shape;

[0134] Fig.24 A flowchart for determining a shift offset (in an orthogonal direction) to be applied to a filter window relative to a CU edge according to an embodiment of the second aspect of the present invention is illustrated;

[0135] FIG25 illustrates candidate filtering window positions for all samples or a suitable subset thereof covered by the window compared to the edge samples of the current CU's projection;

[0136] Fig.26 An embodiment is illustrated in which edge samples of a SAD-processed CU are indicated by the direction of a gradient calculated in a filter window;

[0137] Fig. 27 A flowchart for determining a position shift offset of each template block region according to an embodiment of the second aspect of the present invention is illustrated;

[0138] Fig.28 A plurality of samples in the upper left of the CU 256 are illustrated, which are compared with a corresponding subset of samples of the filter window in the upper left template block region;

[0139] FIG29 illustrates an embodiment of the third aspect of the present invention;

[0140] Fig.30 A block diagram of a decoder according to an embodiment of the present invention is illustrated;

[0141] Fig.31 A block diagram of an encoder according to an embodiment of the present invention is illustrated;

[0142] Fig.32 illustrates the data flow according to an embodiment of the present invention; and

[0143] Fig.33 A block diagram of an electronic device 900 according to an embodiment of the present invention is illustrated. DETAILED DESCRIPTION

[0144] Illustrative embodiments of the present invention will be described below with reference to the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding and should be regarded as illustrative only. Thus, those of ordinary skill in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present invention. Moreover, for the sake of clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0145] In the present invention, the term “and / or” is intended to cover all possible combinations and sub-combinations of the listed elements, including any one, any sub-combination or all of the elements listed individually, and does not necessarily exclude additional elements.

[0146] In the present invention, the phrase "at least one of... or..." is intended to cover any one or more of the listed elements, including any one, any subcombination or all of the elements listed individually, without necessarily excluding any additional elements, and without necessarily requiring all elements.

[0147] In the present invention, the term "codec" refers to "encoding" or "decoding", as is apparent from the context of the described embodiments. Likewise, the term "codec" refers to "encoder" or "decoder".

[0148] Fig. 9 A flow chart of a method for determining filter characteristics of a filter according to an embodiment of the present invention is illustrated, the filter being used to filter corresponding samples of a template adjacent to a coding unit of a picture so as to obtain corresponding intra-frame prediction modes from the template for deriving a decoder-side intra-frame mode derivation predictor for corresponding samples of a CU. The method comprises:

[0149] S200:

[0150] - based on one or more characteristics of the CU and / or template block, or

[0151] - Based on the information data associated with the image

[0152] Determine the size and / or position of the filter window of the filter.

[0153] In other words, instead of using filters with fixed sizes / positions, embodiments of the present invention allow the filter sizes / positions of the filters to be determined based on one or more characteristics of the CU / template, which are used to obtain one or more IPMs for deriving DIMD predictors for CU samples. In other embodiments, information about the filter sizes / positions to be used during DIMD processing can be obtained from additional data associated with the picture and generated at the encoder for use in DIMD processing at the encoder. In addition, the information data or additional data can be included in a bitstream that also carries data representing the encoding of the picture to be decoded, so that the decoder can derive the size / position from the received bitstream. Embodiments of the present invention improve DIMD IPM determination by proposing a set of filters with different sizes and / or by adjusting the center of the filter to different samples, i.e., unlike the prior art methods that only use filters with a single size and fixed position, the method of the present invention allows the size / position of the filter to be adjusted based on the corresponding characteristics of the CU and / or template block, i.e., the adjustment can be guided or adjusted by the currently processed CU and / or by the reconstructed neighborhood (the template block to which the CU is adjacent). The method of the invention is advantageous because the orientation and strength used for computing the HoG and selecting the DIMD IPM are more representative of the signal / sample to be predicted, which leads to improved compression efficiency. Additionally, embodiments of the invention can be implemented with limited additional complexity.

[0154] In the following description of embodiments of the present invention, it is noted that the terms "filter" and "convolution kernel" or "kernel" are used interchangeably.

[0155] It should be noted that the present invention is not limited to determining IPM statistics based on HoG, on which one or more IPMs are selected. According to other embodiments, other methods may be applied to determine IPM statistics, for example, a Hough transform or a gradient-based Hough transform may be used on the filter window size in order to determine the direction for each center pixel (of the filter window), and then a histogram of the found directions may allow the selection of a peak direction (and a matching IPM).

[0156] Aspect 1: Filters with different sizes

[0157] According to the first aspect of the present invention, filters with different filter sizes can be used to generate HoG instead of using only 3×3 filters, such as 3×3 Sobel filters. The filters with different filter sizes can be included in a list containing at least two filters with different filter sizes, and a filter with a suitable size can be selected from these available filters. According to the first aspect of the present invention, only the filter size used to generate HoG is modified during the DIMD process, otherwise, the DIMD process can be conventional, for example, as described above with reference to Figure 6 as described.

[0158] For example, depending on the number of samples / pixels available in the template block, a filter with a suitable filter window size may be selected. For example, if at least five pixels are available in the template block, a filter with a 5×5 filter window size may be used. For example, one or more of the following filters may be used:

[0159] -5×5 Sobel horizontal or vertical filter, such as Fig.10 As shown in

[0160] -5×5 Canny filter,

[0161] -5×5 Prewitt filter, such as Fig.11 As shown in

[0162] -5×5 Kirsch filter, such as Fig.12 As shown in

[0163] - Gaussian blur filter, such as Fig.13 As shown in , it can be applied before filtering to obtain HoG or / and

[0164] - Any other edge detection convolution filter or kernel.

[0165] FIG. 14 illustrates the currently processed CU 256 and template block 258. The template block 258 includes five samples / pixel, i.e., five samples wide in the horizontal direction and five samples high in the vertical direction. In other words, there are 5×5 samples everywhere in the template block 258, so a filter with a 5×5 convolution kernel or filter window 280 can be used. In the template block 258, the samples used for filtering are represented by corresponding circles, and the filter window 280 is centered on the middle line sample 262 of the template block 258. In FIG. 14, sample 282 is the current center sample of the filter window 280.

[0166] Fig.15 (a) and Fig.15(b) illustrates embodiments according to which the template block 258 comprises 5×5 samples only in one of the template block regions. Fig.15 In (a), the template block 258 includes 5×5 samples only in the template block area above the CU 256, i.e., in the upper and upper left template block areas, while only 3×3 samples are available in the left template block area. According to the present invention, according to such an embodiment, different filters with different filter window sizes are adopted. Fig.15 In the embodiment of (a), a filter with a 3×3 filter window 2801 is used to process the left template block area 258a, while a filter with a 5×5 filter window 2802 is used in the upper template block area 258b. The filter windows 2801 and 2802 are centered on the corresponding middle line samples 2621 and 2622 of the corresponding template block areas. For the filter windows 2801 and 2802, the corresponding center samples 2821 and 2822 are also illustrated. According to an embodiment, the upper left template block area 258c can be processed by one of the filters.

[0167] Fig.15 (b) shows the Fig.15 (a) is a similar embodiment, except that 5×5 samples are now located in the left template block area 258a, and 3×3 samples are located in the upper and upper left template block areas 258b and 258c. A filter with a 3×3 filter window 2801 is now applied to the upper template block area 258b, and a filter with a 5×5 filter window 2802 is now applied to the left template block area 258a. The upper left template block area 258c can be processed by one of the filters.

[0168] Filter size derivation using template sample availability

[0169] According to an embodiment, the filter size to be applied can be automatically derived based on the number of available samples in the template block 258, i.e., the available sample rows for the upper template block area 258b, the available columns for the left template block area 258a, and the available columns and rows for the upper left template block area 258c. Fig.15 As described, the template block 258 can be split into the three mentioned block regions, i.e., top, left, and top-left. The block can be split by extending the edge of the currently processed CU 256 into the template block 258, and for each template block region, a corresponding decision on the filter size can be performed according to the availability of samples in each template block region.

[0170] Fig.16A flowchart of filter size selection according to the availability of samples in a template block according to an embodiment of the first aspect of the present invention is illustrated. At S210, the DIMD template block and the reconstructed block 252 (see FIG. 4) are split into three template block areas defined by the edge of the currently processed CU 256, that is, split into a left template block area 258a, an upper template block area 258b, and an upper left template block area 258c. As indicated at S212, subsequent steps are performed for each template block area, and at S214, for the currently processed template block area, the number of rows and / or columns available in the template block area is determined. At S216, the available filters are sorted by size, for example, sorted by decreasing size, so that the filter with the largest size is located first in the list. As indicated at S218, the following steps are performed for each filter. At step S220, a filter is selected from the ordered list of filters, and it is determined at S222 whether the filter window of the selected filter is in the template block area. In the described embodiment, the list is sorted so that the filter with the largest filter window size is tested first, so that by repeating S218 to S222, a filter with a filter window completely contained in the template block area is eventually found. If such a filter is found, the process ends and the found filter is used for DIMD processing.

[0171] As referenced above Fig.15 (a) and Fig.15 (b) As described, the upper left template block area 258c may not have a square shape, that is, the number of available sample rows and columns is different. For example, as shown in Figures 14(a) and 14(b), the upper left template block area corresponds to 5×5 and 3×3 convolution kernels, respectively. According to an embodiment, a filter with a filter window size exceeding the dimension of the upper left template block area 258c may be selected to process the samples in this template block area, and the filter window size may be determined by the template block area to which the center sample of the filter window belongs or by the template block area with the lowest or highest number of available samples contained in the selected filter window.

[0172] With regard to the above-described embodiments of the first aspect of the present invention, it is to be noted that the method of the present invention is naturally not limited to the mentioned filters with 3×3 and 5×5 convolution kernels, but rather other embodiments may use filters with larger or smaller convolution kernels.

[0173] Filter size derivation using the dimensions of the currently processed CU

[0174] According to other embodiments, if there is no sample availability constraint in the template tile 258, i.e., for all available convolution kernel sizes, then each template tile region has enough available samples to allow it to be processed with any available convolution kernel. According to an embodiment, in this case, the decision on the filter size can be made based on the size or shape of the CU 256 currently being processed.

[0175] FIG17 illustrates an embodiment of deriving a filter size based on the dimension or shape of the currently used CU 256. In FIG17, the currently processed CU 256 and the template block 258 adjacent thereto are illustrated. It is assumed that the template block 258 has enough samples to allow processing with any available convolution kernel, but, in FIG17, only the samples in the template block 258 that are processed by the filter size actually selected are illustrated. In other words, in addition to the samples shown by the circles in the template block 258, there are additional sample rows and sample columns available, so the template block actually extends beyond the block shown in FIG17.

[0176] Figure 17(a) illustrates an embodiment, according to which the currently processed CU 256 has a square shape, and among the available filters, a filter with a 3×3 filter window or convolution kernel 280 is selected as the minimum filter size, for example, for filtering each mid-line sample of the template block 258, where the corresponding mid-line samples are selected so that the filter window 280 includes samples in rows / columns of the template block 258 that are directly adjacent to the CU 256.

[0177] FIG. 17( b) illustrates an embodiment according to which the CU 256 has a rectangular shape, with its long side adjacent to the upper template block area 258 b and its short side adjacent to the left template block area 258 a. According to the embodiment depicted in FIG. 17( b), a smaller filter size 2801 is used to filter samples along the short side (i.e., in the left template block area 258), while a larger filter size 2802 is used to filter samples along the long side (i.e., in the upper template block area 258 b). According to other embodiments, as shown in FIG. 17( c), if the currently processed CU 256 has a rectangular shape, a smaller filter size 2801 may be used to filter samples along the short side (i.e., in the upper template block area 258 b), while a larger filter size 2802 is used to filter samples along the long side (i.e., in the left template block area 258 a). The upper left template block area 258 c may use one of the filters depending on the availability of samples in this template block area. According to other embodiments, for the upper left template block area 258c, the smaller filter 2801 or the larger filter 2802 is always used.

[0178] Fig.18 The flowchart of filter size selection according to the shape or size of the CU currently being processed according to the embodiment of the first aspect of the present invention is illustrated. At S230, the reconstructed DIMD template block 258 is split into the three template block areas already mentioned, which are defined by the edge of the CU 256 currently being processed, that is, split into top, left and top left template block areas 258a to 258c. At S232, it is determined whether the width of CU 256 is equal to the height of CU 256. If the width is equal to the height, then at S234, the maximum or minimum available filter (see Figure 17 (a)) containing the filter window is selected in each available template block. If the width is different from the height, then at S236, it is determined whether the width of CU 256 is greater than the height. If the width is greater than the height, then at S238, the largest filter is assigned to the top template block area and the smaller filter is assigned to the left and top left template block areas, as shown in Figure 17 (b). If it is determined at S236 that the width is not greater than the height of the CU 256 , then at S240 the smallest filter is assigned to the upper and upper left template block regions and the largest filter is assigned to the left template block region, as shown in FIG. 17( c ).

[0179] Filter size derivation using the current MRL of neighboring blocks

[0180] According to other embodiments of the first aspect of the present invention, the pattern of neighboring blocks can be checked, and if the MRL index (such as intra_luma_ref_idx of a template block (such as an area of ​​an upper, left, or upper left template block) is 0, then a conventional 3×3 filter can be used. When relying on the MRL tool, according to an embodiment, if intra_luma_ref_idx is not received, it is understood or evaluated as an index value of 0, that is, in such embodiments, the MRL index is inferred to be 0 and a conventional 3×3 filter is used. However, if MRL is adopted and the MRL index is greater than 0, a filter with a size larger than the conventional size, such as a 5×5 filter, can be adopted. According to an embodiment, a larger or larger filter size can only be used if the received MRL index is greater than a predefined threshold (for example, greater than 1).

[0181] Fig.19A flowchart of filter size selection by MRL index status in each template block area according to an embodiment of the first aspect of the present invention is illustrated. At S250, the reconstructed DIMD template block 258 is split into the three above-mentioned template block areas bounded by the edges of the currently processed CU 256. As indicated at S252, for each template block area, determine whether intra_luma_ref_idx is greater than a threshold, as indicated at S254. If the index is lower than the threshold, as indicated at S256, a smaller filter size is used for the template block area, for example, a conventional or default 3×3 filter size. On the other hand, if the index is greater than the threshold, a larger filter size is used for the corresponding template block area, as indicated at S258.

[0182] Filter size signaling

[0183] According to a further embodiment, unlike the derived filter size as described in the previous embodiment of the first aspect, the filter size to be adopted for DIMD processing at the decoder side may be signaled in a high-level syntax structure (such as a picture parameter set, a sequence parameter set, a picture header, etc.). For example, if the flag sps_dimd_enabled_flag is set to 1, then the syntax element sps_dimd_filter_size indicates which filter and / or which filter size to use during DIMD processing. For example, certain values ​​of sps_dimd_filter_size may signal different filters as follows:

[0184] -0 indicates a 3×3 Sobel filter,

[0185] -1 indicates a 5×5 Sobel filter,

[0186] -2 indicates that two Sobel filters can be used, i.e., 3×3 and 5×5 Sobel filters,

[0187] -…

[0188] According to further embodiments, the corresponding value or further values ​​may indicate other filters with other sizes, such as those described above with reference to Figures 11 to 13 If the filter sizes are signaled for a CTU / CU / PU, rate-distortion optimization can be applied to determine which filters provide the best prediction.

[0189] Second aspect: filter window center position adjustment

[0190] According to a second aspect of the present invention, the filter size remains fixed, but the position of the center sample of the filter window is selected so that the filter window is no longer necessarily adjacent to the currently processed CU to be predicted.

[0191] Fig. 20 The use of so-called non-adjacent filtering windows is illustrated according to an embodiment of the second aspect of the invention. Fig. 20 The currently processed CU 256 and the associated template block 258 are illustrated, the template block 258 including the left template block area 258a, the upper template block area 258b and the upper left template block area 258c as described above with reference to the first aspect. In the corresponding template block area, the samples or pixels available for filtering are illustrated by circles, and a filter with a fixed size filter window 280 is used. Fig. 20 As shown in , in the upper template block area 258b, four rows of samples are available, while in the left template block area 258a, only three columns of template samples are available. Therefore, in the left template block area 258a, the filter 280 is placed so that the center sample 282 of the filter window is located on the middle line sample 262, while in the upper filter block area, the center sample 262' for the filter window 280 is such that the filter window 280 is no longer adjacent to the CU 256, but in the depicted embodiment, a row of non-filtered samples is located between the filter window 280 and the CU 256.

[0192] If all samples in the filter window are available, the position of the filter window 280 may be moved. If the filter window 280 covers a block in the template block where no samples are available, for example, when the center sample 282 of the window 280 is placed on the sample 262', the window 280 covers only six samples and does not cover the samples in the right column. In this case, the filter window 280 is not moved but remains adjacent to the current CU 256. According to other embodiments, in this case, if possible, the filter size may be reduced to a lower size of N×N samples, or the filter size may be adjusted to cover only the available samples present in the filter window, for example, by resizing to a 2×3 filter.

[0193] According to an embodiment, the position of the center sample 282 of the filter window 280 may be fixed. For example, it may be established so that the distance between the edge 256x and / or 256y of the currently processed CU 256 and the border of the filter window 280 is set to a predefined value or number, e.g. Fig. 20 In the embodiment, this number or value is 1.

[0194] Filter position derivation using the MRL associated with the template block

[0195] As mentioned above, the template block 258 can be split into top, left, and top-left template block regions. Each of the template block regions can be associated with an MRL state that corresponds to an MRL index for predicting the template block region. If MRL is not used or is not indicated or signaled for the corresponding template block region, an index value of 0 is inferred.

[0196] According to an embodiment, when the MRL index associated with a certain template block region is 0, the position of the filter window center sample 282 is selected for the entire template block 258 so that the filter window 280 is adjacent to the current CU 256, which corresponds to the conventional method when using a conventional filter. However, when the MRL index is set to a value different from 0, for example, when the MRL index value is equal to 1 for the template block region, the center sample 282 of the filter window 280 is offset vertically by one sample for the upper template block region 258b, horizontally by one sample for the left template block region 258a, and horizontally and / or vertically by one sample for the upper left template block region 258c. This means that the filter window 280 is no longer adjacent to the current CU, but is offset from the horizontal edge 256x and / or vertical edge 256y of the CU 256 by one or more sample rows / columns in the corresponding template block region.

[0197] Figure 21 illustrates a HoG filter window that is placed not adjacent to the current CU based on the MRL index according to an embodiment of the second aspect of the present invention. Figure 21 (a) illustrates the currently processed CU 256 and the template block 258, which includes reconstructed samples from the already reconstructed CU, the so-called neighboring CUs 2841 and 2842 indicated by the dotted boxes in the figure. The left template block area 258a includes samples from the neighboring CU 2841, for which MRL is not adopted, so MRL=0 is inferred, and the center sample 282 for the filter window is selected so that the filter window 280 is adjacent to the vertical edge 256y of the currently processed CU 256. On the other hand, the upper template block area 258b and the upper left template block area 258c include samples from the neighboring CU 2842 in which MRL is adopted. For samples of the upper left template block region 258c, MRL=0 is indicated so that the center sample of the filter window 280 is selected in the upper left template block region 258c so that the boundary of the window 280 is adjacent to the horizontal edge 256x and the vertical edge 256y of the CU 256. For the upper template block region 258b, MRL=1 is signaled or determined so that the center sample of the filter window 280 is selected in the upper left template block region 258c so that the boundary of the window 280 is adjacent to the horizontal edge 256x and the vertical edge 256y of the CU 256. Fig. 20The described approach adjusts the center sample of filter window 280, ie, center sample 282 is moved one row (row 262') outward from horizontal edge 256x of CU 256, as indicated by the arrow, thereby providing a one-sample-row offset between the window border and edge 256x.

[0198] FIG21( b) illustrates another embodiment similar to FIG21( a), except that neighboring CUs 2841 and 2842 employ MRL, so for each template block region, an MRL index is signaled. In the upper left template block region 258 c, MRL=0 is signaled, so that the center sample 262 for the 3×3 filter window is selected so that the filtered sample is adjacent to the horizontal edge 256 x and the vertical edge 256 y of CU 256, and there is no offset. For the upper template block region 258 b, MRL=1 is signaled, so in the same manner as described above with reference to FIG21( a), the center sample 262 ′ 1 is shifted outward from the horizontal edge 256 x of CU 256 by one row, as indicated by the arrow. For the left template block region 258a, MRL=2 is signaled so that the center sample 262'2 for the filter window 2802 is shifted so that there is an offset of 2 sample columns between the boundary of the filter window 2802 and the vertical edge 256y of the current CU.

[0199] According to a further embodiment, the offset is provided only if the MRL index of the associated template block region has a value greater than a certain threshold, eg only if the MRL is greater than 1.

[0200] Fig. 22 A flowchart of determining the filter window position according to the MRL index associated with the corresponding template block area according to an embodiment of the second aspect of the present invention is illustrated. At S270, the reconstructed DIMD template block 258 is split into corresponding template block areas, which are bounded by the corresponding edges 256x and 256y of the currently processed CU 256, thereby generating upper, left and upper left template block areas. At S272, for each template block area, intra_luma_ref_index is obtained, as indicated at S274. Center_filter_offset is set to intra_luma_ref_index, as indicated at S276, and the default value is 0, so at S278, the filter window is shifted by center_filter_offset samples in the orthogonal direction relative to the edge of the currently processed CU256.

[0201] Filter position derivation using the size / shape of the currently processed CU

[0202] According to other embodiments of the second aspect of the present invention, the position of the center sample of the filtering window may be derived according to the size / shape of the CU currently being processed.

[0203] If the CU 256 has a square shape, the filter window may be placed adjacent to the edges 256x, 256y of the current CU 256 (see FIG. 21( a)). According to other embodiments, if the CU 256 has a square shape, the filter window may be provided so that it is not adjacent to the edge of the current CU 256, but an offset may be provided between the edge of the CU 256 and the boundary of the filter window, for example, an offset of one sample row or one sample column.

[0204] 23 illustrates an embodiment of selecting a filter position when the CU 256 has a rectangular shape. As shown in FIG23( a), the position of the filter window 280 is offset by one sample column in the template block adjacent to the long side of the CU region 256, while the filter window 280 remains adjacent to the CU 256 in the template block along the short side. According to other embodiments, as shown in FIG23( b), the filter window offset may be applied in the template block along the short side, while the filter window remains adjacent to the CU in the template block along the long side of the CU 256.

[0205] According to other embodiments, the shift offset value may be calculated based on the ratio between the width and height of the CU 256. For example, the center_filter_offset described above may be determined as INT(width / height)-1, where INT(x) indicates the integer portion of x. The offset may be applied to the filter window in the template block area located near the shorter or longer dimension of the CU 256.

[0206] Fig.24A flow chart of determining a shift offset (in an orthogonal direction) to be applied to a filter window relative to the edge of a CU according to an embodiment of the second aspect of the present invention is illustrated. At S300, the reconstructed DIMD template block 258 is split into corresponding regions as described above. At S302, it is determined whether the width and height of CU 256 are the same. If the width and height of the CU are the same, that is, if the CU has a square shape, the filter position is not shifted, as indicated at S304. If the width and height of CU256 are different, then at S306, it is determined whether the width is greater than the height. If the width is greater than the height, then as indicated at S308, the filter position is shifted in the upper template block area by an offset equal to INT(cbwidth / cbheight)-1. Otherwise, if the width is lower than the height, then as indicated at S310, the filter position is shifted in the left template block area by an offset equal to INT(cbheight / cbwidth)-1.

[0207] Filter position derivation using distance metric

[0208] According to a further embodiment of the second aspect of the present invention, the filter position can be derived by using a distance metric. For example, a distance metric (e.g., the sum of absolute differences SAD) can be used to compare all samples or an appropriate subset of samples of the filter window with certain samples in CU 256 (e.g., samples orthogonally projected at the edge of the currently processed CU). In other words, the similarity of some or all samples within the filter window with samples at the edge of the CU is determined for different positions of the filter window, and the position of the filter window is selected so that the distance between the samples in the filter window and the samples at the edge of the CU is at a minimum, that is, the corresponding samples / pixels are similar.

[0209] FIG. 25 illustrates candidate filter window positions that have all or an appropriate subset of samples covered by the window compared to the edge samples of the projection of the current CU. FIG. 25( a) illustrates a first candidate filter window position with a center sample such that the boundary of the filter window 280 is offset from the edge 256x of the CU 256 by one sample row in the template block. FIG. 25( b) illustrates a second candidate filter window position with a center sample such that the boundary of the window 280 is adjacent to the edge 256x of the CU 256, i.e., there is no offset. In the embodiment described with reference to FIG. 25 , not all samples of the window 280 are used to determine the edge samples 256x. S 25(a) and 25(b) .

[0210] According to other embodiments, the projection direction (in FIG. 25 , from the sample in the filter window 280 to the edge sample 256 S ) is not orthogonal to the edge 256x of the CU 256, but may be colinear with the main direction of the gradient calculated by the convolution kernel. For example, the gradient may be calculated using the edge samples pointed to by the filter window and the projection direction in the same manner as the target pixel is determined for the conventional angular in-direction, where interpolation is performed in VVC, as described, for example, in Section B of Reference

[22] . Fig.26 An embodiment is shown according to which edge samples 256 of CU 256 S is the sample indicated by the direction of the gradient calculated in the filter window 280, as indicated by the direction from the window 280 to the edge sample 256. S Some or all samples covered by window 280 may be compared to edge samples using the distance metrics described above for different filter positions to determine the minimum distance and appropriate offset for the filtering window.

[0211] As described above, according to an embodiment, the corresponding edge samples 256 in the CU 256 may be used to obtain the S The distance is calculated by the SAD between the average of some or all samples orthogonal to the edge and present in the filter window (see Figure 25 (a) and Figure 25 (b)). The SAD can be accumulated over the entire row of the upper template block area, or accumulated over the entire column of the left template block area, or accumulated over the rows and columns of the upper left template block area.

[0212] When f is represented as the filter candidate index, N is represented as the width of the filter window, M is represented as the height of the filter window, and a i,j is represented by the sample value at position (i, j) in the filter window and c k,l When expressed as the sample value at position (k, l) in the current CU 256, for the template block area above and to the left, the SAD can be calculated as follows:

[0213]

[0214] According to other embodiments, only samples present in the middle line and in the same direction as the edge of the CU may be used for comparison with the edge samples, and in this case, the SAD for the upper and left template block regions is determined as follows:

[0215]

[0216] Fig. 27A flow chart for determining the position shift offset of each template block region according to an embodiment of the second aspect of the present invention is illustrated. As indicated at S320, first, the DIMD template block 258 is split into corresponding template block regions defined by the edge of the CU 256, as described above. Then, for each template block region, for each shift offset position from offset 0 to the maximum offset value, and for each row or column, as indicated at S322, S324, and S326, respectively, the SAD between the available projection samples on the edge of the filter midline and the CU is calculated, as indicated at S328. Once the calculation at S328 is performed for each template block region, for each shift offset position, and for each row or column, the shift offset position with the lowest SAD accumulated on the row or column for each template block region is retained at S330.

[0217] Regarding the upper left template block area, it should be noted that orthogonal projection is not possible. Therefore, according to an embodiment, the positions of the filter windows for the upper and left template block areas are first determined, and then the position of the center sample of the filter window of the upper left template block area is determined as the intersection of the row selected by the center sample of the filter window of the upper template block area and the column of the center sample of the filter window of the left template block area, thereby providing position continuity between different template block areas.

[0218] According to other embodiments, for the upper left template block area, part or all samples of the filter window in the upper left template block area may be compared with samples located at the upper left corner of the CU using, for example, SAD. Fig.28 As shown in FIG. 256 , the figure shows a plurality of upper left samples 256 of CU 256 TL , these samples are compared with the corresponding subset of samples of the filter window 280 in the upper left template block area. When applying this method, it is assumed that the upper left template block area has a Fig.28 The samples shown in FIG. 2 are more samples, so the center sample of the filter window 280 can be located at a different position within the upper left template block area so that the entire window still includes samples.

[0219] According to an embodiment, for each considered position of the filter window 280 in the upper left template block area, two horizontal and two vertical CU samples 256 TL (ie, four samples) are respectively compared with the four average values ​​of the samples in the filter window (eg Fig.28) are compared (e.g., using SAD). In order to calculate the average value, the filter window is split into four parts: the first part has coordinates from 0 to [M / 2] (rows) and from 0 to [N / 2] (columns), the second part has coordinates from [M / 2] to M-1 (rows) and from 0 to [N / 2] (columns), the third part has coordinates from 0 to [M / 2] (rows) and from [N / 2] to N-1 (columns), and the fourth part has coordinates from [M / 2] to M-1 (rows) and from [N / 2] to N-1 (columns). The average value is calculated in each part, and finally the sum of the absolute differences between each upper left CU sample and the corresponding part average value is calculated. The position of the filter window with the smallest SAD is selected.

[0220] According to another embodiment, for each considered position of the filter window in the top left template block, the average of the samples in the filter window may be compared with the value of the top left sample in CU 256, and the position of the filter window that produces the smallest difference may be selected. According to other embodiments, the average of multiple samples (e.g., the four samples in the top left corner of the CU) may be used instead of using a single top left CU sample.

[0221] According to an embodiment, when the method of the present invention is implemented at a decoder, the offset of the filter window is signaled to the decoder. Since there is no current CU available at the decoding side, the selected position offset is signaled in the bitstream at the CU, picture or sequence level. In practice, the average position offset can be determined for the entire picture or sequence.

[0222] Filter position signaling

[0223] According to the above-mentioned embodiment of the second aspect of the present invention, the filter position is derived from certain characteristics of the template block or CU, but the present invention is not limited to such embodiments. More specifically, according to a further embodiment of the second aspect of the present invention, the filter position is not derived from characteristics associated with the template block or CU, but the actual position can be signaled in a similar manner to the above description of the reference filter size. For example, an index such as the dimd_filter_shift_minus1 index can be signaled at the CU / PU or TU level to indicate the position or position shift of the center sample of the filter window orthogonal to the edge of the currently processed CU. The index can be calculated on the encoder side and can be selected by a rate-distortion optimization (RDO) process, where the filter position with the minimum cost is selected using, for example, SAD, and the corresponding entropy coding and decoding context for MRL can also be used to indicate the index of the shift of the filter position. Therefore, in addition to the aforementioned embodiments, the decoder can derive the above-mentioned index from the encoded data stream to determine the actual position or offset of the filter relative to the edge of the currently processed CU.

[0224] The third aspect: a combination of the previous aspects

[0225] According to the third aspect of the present invention, the aforementioned aspects of the present invention may be combined, so that according to an embodiment of the third aspect, the position of the filter window relative to the edge of the CU and the filter size may be changed.

[0226] According to an embodiment, the filter window position and size for each template block region can be determined as follows: for each template block region, in the upper left, left and upper template block regions, select the maximum filter that contains only available samples, and then derive the filter position according to the MRL index associated with each template block region. Finally, if the filter position is shifted, it can be checked whether the filter position still contains only available samples, otherwise, the filter size can be reduced and the shifted position can be maintained.

[0227] According to another embodiment, the filter window size may be determined by the number of samples available in the template and for the largest filter the farthest possible position from the edge of the CU may be chosen.

[0228] According to other embodiments, the position of the filtering window can be determined by selecting the position where the SAD between the filtered midline samples parallel to the edge of the CU and the orthogonal projection samples on the edge is the smallest, and selecting the maximum filter that still only contains available samples at the determined position.

[0229] Figure 29 illustrates an embodiment of the third aspect of the present invention, and in Figure 29(a), the size of the filter window 2801 associated with the left template block area 258(a) is set to 5×5, and its position is not shifted. On the other hand, the position of the filter window 2802 associated with the upper template block area 258b is shifted by one sample row, and its size is set to 3×3. The position of the filter window associated with the upper left template block is not shifted, and its size is set to 3×3, as shown in window 2803. Figure 29(b) illustrates another embodiment, which is different from the embodiment of Figure 29(a), and it sets the size of the filter window 2801 associated with the left template block area 258a to 5×5, and its position is shifted by one sample column. The other windows are the same as in Figure 29(a).

[0230] According to an embodiment, in order to determine the size and / or position according to one or more characteristics of the CU and / or the template block, when the CU is larger than a certain size, the samples in the template block are filtered by a filter positioned so that the filter window is offset from the edge of the CU by a first number of samples, otherwise the samples in the template block are filtered by a filter positioned so that the filter window is adjacent to the CU or offset from the edge of the CU by a second number of samples, and the second number of samples is different from the first number of samples.

[0231] Further Embodiments

[0232] So far, the inventive concept has been described with reference to aspects and embodiments of a method for determining a DIMD predictor. According to a further embodiment, the invention also provides an encoder / decoder operating according to the above principles.

[0233] Fig.30 The block diagram of a decoder for decoding a coded data stream including data representing a picture according to an embodiment of the present invention is illustrated. The decoder 300 includes the following modules:

[0234] 302: A first or decoding module is configured to decode a picture or a picture and information data associated with the picture from a coded data stream.

[0235] 304: A second or DIMD predictor derivation module configured to derive a decoder side intra mode derivation (DIMD) predictor for corresponding samples of a coding unit (CU) of a picture.

[0236] 304a: A first or filter size and / or position submodule, included in the second module and configured to

[0237] - based on one or more characteristics of the CU and / or template block, or

[0238] - Based on the information data associated with the image

[0239] Determine the size and / or position of the filter window of the filter.

[0240] 304b: A second or sample filtering submodule, included in the second module and configured to filter corresponding samples of a template block adjacent to the CU, thereby obtaining a corresponding intra prediction mode (IPM) from the template block for deriving a DIMD predictor.

[0241] Fig.31 The block diagram of an encoder for encoding a picture into a coded data stream according to an embodiment of the present invention is illustrated, wherein the coded data stream includes data representing the picture. The encoder 400 includes the following modules:

[0242] 402: A first or encoding module is configured to receive an original picture and encode the picture into a data stream.

[0243] 404: A second or DIMD predictor derivation module configured to derive a decoder side intra mode derivation (DIMD) predictor for corresponding samples of a coding unit (CU) of a picture.

[0244] 404a: A first or filter size and / or position submodule, included in the second module and configured to determine the size and / or position of a filter window of the filter according to one or more characteristics of the CU and / or the template block.

[0245] 404b: A second or sample filtering submodule, included in the second module and configured to filter corresponding samples of a template block adjacent to the CU, thereby obtaining a corresponding intra prediction mode (IPM) from the template block for deriving a DIMD predictor.

[0246] According to a further embodiment, the first module 402 of the encoder 400 is configured to encode information data associated with the picture into the data stream, the information data signaling the size and / or position of the filter window of the filter used for the DMID processing.

[0247] Fig.32 A data stream 500 according to an embodiment of the present invention is illustrated, in which a picture and information data associated with the picture are encoded. The information data signals the size and / or position of the filter window of the filter for DMID processing. For example, a data stream or bit stream 500 can be provided by an encoder 400, which performs the method of the present invention for encoding information data 502 into the data stream 500. The data stream 500 is transmitted to a decoder 300 via a wired or wireless transmission medium 504 (such as a cable or a radio link), and the decoder 300 decodes the information data 502 from the data stream 500.

[0248] Although some aspects of the disclosed concepts have been described in the context of an apparatus, it is clear that these aspects also represent a description of a corresponding method, where a block or device corresponds to a method step or a feature of a method step. Similarly, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus.

[0249] Fig.33900 is a block diagram illustrating an electronic device 900 according to an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are described only as examples and are not intended to limit the embodiments of the present invention described and / or claimed herein. The device 900 includes a computing unit 901 to perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 902 or loaded from a storage unit 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data for the operation of the storage device 900 can also be stored. The computing unit 901, ROM 902, and RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0250] The components in the device 900 are connected to the I / O interface 905, including: an input unit 906, such as a keyboard, a mouse; an output unit 907, such as various types of displays, speakers; a storage unit 908, such as a disk, an optical disk; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices through a computer network (such as the Internet and / or various telecommunications networks). The computing unit 901 can be composed of various general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any suitable processors, controllers, microcontrollers, etc. The computing unit 901 performs the various methods and processes described above, such as image processing methods. For example, in some embodiments, the image processing method can be implemented as a computer software program tangibly implemented on a machine-readable medium (such as a storage unit 908). In some embodiments, part or all of the computer program may be loaded and / or installed on the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the above-described image processing method may be performed. In some embodiments, the computing unit 901 may be configured to perform the image processing method in any other suitable manner (e.g., by means of firmware).

[0251] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementation in one or more computer programs that can be executed and / or interpreted on a programmable system that includes at least one programmable processor, and the programmable processor may be a special purpose or general purpose programmable processor, and may receive data and instructions from a storage system, at least one input device, and at least one output device, and may transmit data and instructions to a storage system, at least one input device, and at least one output device.

[0252] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that when the processor or controller executes the program code, the functions and / or operations specified in the flow chart and / or block diagram are executed. The program code can be executed completely on the machine, partially on the machine, partially on the machine as an independent software package and partially on a remote machine or completely on a remote machine or server.

[0253] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program that is used by or in connection with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0254] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having a display device (e.g., a cathode ray tube (CRT) or a liquid crystal display (LCD)) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user, for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and any form (including acoustic input, voice input, or tactile input) can be used to receive input from the user.

[0255] The systems and techniques described herein may be implemented on a computing system that includes a back-end component (e.g., as a data server), or on a computing system that includes a middleware component (e.g., an application server), or on a computing system that includes a front-end component (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or on a computer system that includes such back-end components, middleware components, front-end components, or any combination thereof. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0256] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in a cloud computing service system and solves the defects of difficult management and weak business scalability in traditional physical hosts and virtual private servers (referred to as "VPS"). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0257] It should be understood that steps can be reordered, added or deleted by using the various forms of processes shown above. For example, the steps described in the present invention can be performed in parallel, sequentially or in different orders, as long as the expected results of the technical solutions in the present invention can be achieved, and are not limited herein.

[0258] The specific embodiments mentioned above do not limit the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and replacements can be made according to design requirements and other factors. Any modifications, equivalent replacements and improvements made within the principles of the present invention should be included in the protection scope of the present invention.

[0259] References

[0260] [1]H.264:Advanced video coding for generic audiovisual services,https: / / www.itu.int / rec / T-REC-H.264-202108-P / en

[0261] [2]H.265: High efficiency video coding, https: / / www.itu.int / rec / T-REC-H.265-202108-P / en

[0262] [3]H.266:Versatile video coding,https: / / www.itu.int / rec / T-REC-H.266-202008-I / en

[0263] [4]AV1 Bitstream&Decoding Process Specification, http: / / aomedia.org / av1 / specification /

[0264] [5] E.Mora et al., “CE3-related: Decoder-side Intra Mode Derivation”, JVET-L0164, Macau, China, October 2018

[0265] [6]CE3:Decoder-side Intra Mode Derivation(tests 3.1.1,3.1.2,3.1.3and3.1.4),JVET-M0094,Marrakech,January 2019

[0266] [7] M. Abdoli et al., E. Mora, T. Guionnet, M. Raulet, Non-CE3: Decoder-side IntraMode Derivation with Prediction Fusion, JVET-N0342, Geneva, March 2019

[0267] [8] M. Abdoli et al., “Non-CE3: Decoder-side Intra Mode Derivation with Prediction Fusion Using Planar”, JVET-O0449, Gothenburg, July 2019

[0268] [9] J. Zhao et al., “EE2-Related: Improvements of Decoder-Side Intra Mode Derivation”, JVET-V0087, online, April 2021.

[0269]

[10] J. Zhao et al., “EE2 Test 2.2: DIMD with multiple blending modes”, JVET-W0054, online, July 2021.

[0270]

[11] X. Li et al., “EE2-related: Implicit derivation of DIMD blend modes”, JVET-W0067, online, July 2021.

[0271]

[12] X. Li et al., “EE2-related: A combination of CIIP and DIMD / TIMD”, JVET-W0068, online, July 2021.

[0272]

[13] J. Zhao et al., “EE2 Related-DIMD with implicitly derived multiple blending modes”, JVET-W0126, online, July 2021.

[0273]

[14] X. Li et al., “EE2-3.1: Combination of CIIP and DIMD / TIMD”, JVET-X0098, online, October 2021.

[0274]

[15] C. Zhou et al., “EE2-related: Optimization on the second mode derivation of DIMD blending mode”, JVET-X0115, online, October 2021.

[0275]

[16] T. Dumas et al., “AHG12: removing a discontinuity in the discrete angle comparison in DIMD,” JVET-X0139, online, October 2021.

[0276]

[17] X. Xiu et al., “AHG12: Removal of floating operations in DIMD and TIMD,” JVET-X0149, online, October 2021.

[0277]

[18] K. Cao et al., “Non-EE2: Fix for histogram of gradients derivation in DIMD mode”, JVET-X0156, online, October 2021.

[0278]

[19] K. Naser et al., “AhG12: Removed DIMD from MPM list of TIMD”, JVET-Y0097, online, January 2022.

[0279]

[20] S. Yoo et al., “EE2-related: Clean-up on DIMD,” JVET-Y0131, online, January 2022.

[0280]

[21] J. Zhao et al., “Non-EE2: DIMD Flag Signalling Clean-up”, JVET-Y0144, online, January 2022.

[0281]

[22] J. Pfaff et al., “Intra Prediction and Mode Coding in VVC”, IEEE Transactions on Circuits and Systems for Video Technology, Vol. 31, No. 10, October 2021.

Claims

1. A method for determining filter characteristics of a filter for filtering corresponding samples of a template block (258) adjacent to a coding unit CU (256) of a picture (102) in order to obtain a corresponding intra prediction mode (IPM) from the template block (258) for deriving a decoder-side intra mode derivation (DIMD) predictor for corresponding samples of the CU (256), the method comprising: - based on one or more characteristics of the CU (256) and / or the template block (258), or - Based on information data (502) associated with the picture (102) The size and / or position of the filter window (280) of the filter is determined (S200).

2. The method of claim 1, wherein: To determine the size and / or position according to one or more characteristics of the CU (256) and / or the template block (258), - said size of said filter window (280) is derived from one or more of the following ○ the number of available samples in the template block (258), ○ The dimension of the CU (256), o a multiple reference line (MRL) index associated with the template block (258), and / or - said position of said filter window (280) is derived from one or more of the following ○ the number of available samples in the template block (258), ○ The dimension of the CU (256), o a multiple reference line (MRL) index associated with the template block (258), o A distance metric to samples at the edge of the CU (256).

3. The method according to claim 1 or 2, wherein: In order to determine the size and / or position according to one or more characteristics of the CU (256) and / or the template block (258), the template block (258) includes a plurality of template block areas, the plurality of template block areas include an upper template block area (258b), a left template block area (258a), and an upper left template block area (258c), and the size of the filter window (280) for the corresponding template block area is derived according to available rows of samples in the upper template block area (258b), available columns of samples in the left template block area (258a), and available columns and rows of samples in the upper left template block area (258c), The method comprises: for each template block region, selecting a filter from a plurality of predefined filters having filter windows (280) of different sizes, the filter having a filter window (280) size corresponding to the available columns and / or rows of samples in the corresponding template block region, and Wherein, optionally, for a block in which the filter window (280) overlaps with an adjacent template block area, the filter is selected according to the following items: - the template block region to which the central sample of the filter window (280) belongs, or - The template block area with the lowest or highest number of samples contained in the filtering window (280).

4. A method as claimed in any one of the preceding claims, in, In order to determine the size and / or position according to one or more characteristics of the CU (256) and / or the template block (258), when the CU (256) is larger than a certain size, the samples in the template block (258) are filtered by a first filter having a first filter window size, otherwise, the samples in the template block (258) are filtered by a second filter having a second filter window size different from the first filter window size, and / or wherein, in order to determine the size and / or position according to one or more characteristics of the CU (256) and / or the template block (258), the template block (258) includes a plurality of template block areas, the plurality of template block areas including an upper template block area (258b), a left template block area (258a) and an upper left template block area (258c), and if the samples available in the template block area allow the use of a plurality of filters with different filter window sizes, then - when the CU (256) has a square shape, samples in all template block areas are filtered by a same filter, wherein, optionally, the filter is a filter having a maximum or minimum filtering window size among the plurality of filters, - when the CU (256) has a rectangular shape, filtering samples in a template block area adjacent to a long side of the CU (256) by a first filter, and filtering samples in a template block area adjacent to a short side of the CU (256) by a second filter, the first filter and the second filter having different filter window sizes, and wherein in order to filter the samples in the upper left template block area o selecting one of the first filter and the second filter according to the availability of samples in the upper left template block area, or One of the first filter and the second filter is always used.

5. A method as claimed in any one of the preceding claims, wherein: In order to determine the size and / or position according to one or more characteristics of the CU (256) and / or the template block (258), the template block (258) includes a plurality of template block areas, the plurality of template block areas include an upper template block area (258b), a left template block area (258a) and an upper left template block area (258c), and the size of the filter window (280) for the corresponding template block area is derived according to an MRL index associated with the corresponding template block area, such as intra_luma_ref_idx, in - when the MRL index is zero or when no MRL index is indicated, selecting a first filter having a first filter window size for the template block region, - When the MRL index is different from zero, for example has a certain value or a value exceeding a predefined threshold, then selecting a second filter with a second filter window size for the template block area, the second filter window being larger than the first filter window size.

6. A method as claimed in any one of the preceding claims, wherein: In order to determine the size and / or position according to one or more characteristics of the CU (256) and / or the template block (258), the template block includes a plurality of template block areas, the plurality of template block areas include an upper template block area (258b), a left template block area (258a) and an upper left template block area (258c), and a filter with a fixed filter window size is used for the corresponding template block area, wherein if the available rows / columns of samples in the template block area exceed the rows / columns of samples covered by the fixed filter window size, then the position of the center sample of the filter window (280) is selected so that the filter window (280) is not adjacent to the CU (256), thereby offsetting the edge of the CU (256) from the boundary of the filter window (280) by a predefined number of samples, Wherein, optionally, if the available rows / columns of samples in the template block area are equal to the rows / columns of samples covered by the fixed filter window size, then the size of the filter window (280) is reduced so that the rows and / or columns of samples in the template block area are covered by the filter window (280).

7. A method as claimed in any one of the preceding claims, in, In order to determine the size and / or position based on one or more characteristics of the CU (256) and / or the template block (258), when the CU (256) is larger than a certain size, samples in the template block (258) are filtered by a filter positioned such that the filter window (280) is offset from an edge of the CU (256) by a first number of samples, otherwise the samples in the template block (258) are filtered by a filter positioned such that the filter window (280) is adjacent to the CU (256) or offset from an edge of the CU (256) by a second number of samples, wherein the second number of samples is different from the first number of samples, and / or In order to determine the size and / or position according to one or more characteristics of the CU (256) and / or the template block (258), the template block (258) includes a plurality of template block areas, the plurality of template block areas include an upper template block area (258b), a left template block area (258a) and an upper left template block area (258c), and a filter with a fixed filter window size is used for the corresponding template block area, in, - When the CU (256) has a square shape, the filter window (280) o is not adjacent to the CU (256), such that an edge of the CU (256) is offset from a boundary of the filter window (280) by a predefined number of samples, or ○ adjacent to the CU (256) such that an edge of the CU (256) is not offset from a boundary of the filter window (280), - when the CU (256) has a rectangular shape, o In a template block region adjacent to a long side of the CU (256), the filter window (280) is not adjacent to the CU (256), such that an edge of the CU (256) is offset from a boundary of the filter window (280) by a predefined number of samples, and in a template block region adjacent to a short side of the CU (256), the filter window (280) is adjacent to the CU (256), such that an edge of the CU (256) is not offset from a boundary of the filter window (280), or o In a template block area adjacent to a short side of the CU (256), the filter window (280) is not adjacent to the CU (256), such that an edge of the CU (256) is offset from a boundary of the filter window (280) by a predefined number of samples, and in a template block area adjacent to a long side of the CU (256), the filter window (280) is adjacent to the CU (256), such that an edge of the CU (256) is not offset from a boundary of the filter window (280), and Optionally, the offset is calculated based on a ratio between a width and a height of the CU (256).

8. A method as claimed in any one of the preceding claims, wherein: In order to determine the size and / or position according to one or more characteristics of the CU (256) and / or the template block (258), the template block (258) includes a plurality of template block areas, the plurality of template block areas include an upper template block area (258b), a left template block area (258a) and an upper left template block area (258c), and a filter with a fixed filter window size is used for the corresponding template block area, wherein the position of the filter window (280) for the corresponding template block region is derived based on the MRL index associated with the corresponding template block region, such as intra_luma_ref_idx, in - when the MRL index is zero or when no MRL index is indicated, the filter window (280) is adjacent to the CU (256) such that an edge of the CU (256) is not offset from a boundary of the filter window (280), -When the MRL index is different from zero, for example, has a certain value or a value exceeding a predefined threshold, the filtering window (280) is not adjacent to the CU (256), so that the edge of the CU (256) is offset from the boundary of the filtering window (280) by a predefined number of samples.

9. A method as claimed in any one of the preceding claims, wherein: In order to determine the size and / or position according to one or more characteristics of the CU (256) and / or the template block (258), the template block (258) includes a plurality of template block areas, the plurality of template block areas include an upper template block area (258b), a left template block area (258a) and an upper left template block area (258c), and a filter with a fixed filter window size is used for the corresponding template block area, wherein the positions of the filter window (280) for the upper template block region and the left template block region are derived by, for different offsets between the edge of the CU (256) and the border of the filter window (280), comparing one or more or all samples contained in the filter window (280) with samples projected at the edge of the CU (256) using a distance metric, such as the sum of absolute differences (SAD), and by selecting the offset that produces the smallest distance, Wherein, optionally, the offset selection is performed for the entire template block area or for each center sample of the filter window (280), Wherein, optionally, the position of the filter window (280) for the upper left template block area is derived as follows: - after determining the filter window positions in the upper template block region and the left template block region, determining the center sample of the filter window (280) of the upper left template block region as the intersection of the row of the center sample of the filter window (280) of the upper template block region and the column of the center sample of the filter window (280) of the left template block region, or - for different positions of the filter window (280) in the upper left template block area, using a distance metric, such as the sum of absolute differences (SAD), compare one or more or all samples contained in the filter window (280) with a plurality of samples located at the upper left corner of the CU (256), and select the position that produces the minimum distance.

10. A method as claimed in any one of the preceding claims, wherein: When determining the size and / or position based on information data (502) associated with the picture (102), the information data (502) associated with the picture uses a high-level syntax structure, such as a picture parameter set, a sequence parameter set or a picture header, to signal the filter size and / or the filter position, and Wherein, optionally, the high-level grammatical structure includes - a syntax element, such as sps_dimd_filter_size, indicating which filter and / or which filter size to use during DIMD processing, and / or - an index, such as dimd_filter_shift_minus1, indicating the position or position shift of the center sample of the filter window (280) relative to the edge of the CU (256) in an orthogonal direction.

11. A non-transitory computer-readable storage medium comprising instructions, which, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 10.

12. An apparatus (300) for decoding an encoded data stream (500), the encoded data stream (500) comprising data representing a picture (102), the apparatus (300) comprising: a first module (302) configured to decode the picture (102) or decode the picture and information data (502) associated with the picture (102) from the coded data stream (500), and A second module (304) is configured to derive a decoder-side intra mode derivation (DIMD) predictor for corresponding samples of a coding unit CU (256) of the picture (102), the second module (304) comprising: The first submodule (304a) is configured to: - according to one or more characteristics of the CU (256) and / or the template block (258), or - according to said information data (502) associated with said picture (102), determining the size and / or position of a filter window (280) of the filter, and The second submodule (304b) is configured to filter corresponding samples of a template block (258) adjacent to the CU (256) to obtain a corresponding intra prediction mode (IPM) from the template block (258) for deriving the DIMD predictor.

13. An apparatus (400) for encoding a picture (102) into a coded data stream (500), the coded data stream (500) comprising data representing the picture (102), the apparatus (400) comprising: A first module (402) configured to receive an original picture (102) and encode the picture (102) into the data stream (500), and A second module (404) is configured to derive a decoder-side intra mode derivation (DIMD) predictor for corresponding samples of a codec unit CU (256) of the picture (102), the second module (404) comprising: A first submodule (404a) is configured to determine a size and / or position of a filter window (280) of the filter according to one or more characteristics of the CU (256) and / or the template block (258), and The second submodule (404b) is configured to filter corresponding samples of a template block (258) adjacent to the CU (256) to obtain a corresponding intra prediction mode (IPM) from the template block (258) for deriving the DIMD predictor.

14. An apparatus as claimed in claim 13, wherein the first module (404a) is configured to encode information data (502) associated with the picture (102) into the data stream (500), the information data (502) signaling a size and / or position of the filter window (280) of the filter used for the DMID processing.

15. A data stream (500) having a picture (102) encoded into the data stream (500) and information data (502) associated with the picture (102), The information data (502) signals the size and / or position of a filter window (280) of a filter used for DMID processing.

Images