A method, an apparatus and a computer program product for video coding and decoding

Abstract

The embodiments relate to method for decoding encoded samples of blocks of video sample, wherein for a block of video sample data the method comprises determining an intra-prediction information, where the determining comprises extracting prediction information from a template sample block; generating two or more representations of at least some samples in the template sample block by filtering the samples with different frequency channels; determining intra-prediction modes for each frequency channels of the template sample block; and computing intra-predicted samples for the current block based on the detected intra-prediction modes and the two or more representations of at least some template samples. The embodiments also concern technical equipment for implementing the method.

Description

A METHOD, AN APPARATUS AND A COMPUTER PROGRAM PRODUCT FOR VIDEO CODING AND DECODINGTechnical Field

[0001] The present solution generally relates to a method and an apparatus and a computer program product for video coding and decoding.Background

[0002] This section is intended to provide a background or context to the invention that is recited in the claims. The description herein may include concepts that could be pursued but are not necessarily ones that have been previously conceived or pursued. Therefore, unless otherwise indicated herein, what is described in this section is not prior art to the description and claims in this application and is not admitted to be prior art by inclusion in this section.

[0003] A video codec comprises an encoder and a decoder. The encoder transforms the input video into a compressed representation suited for storage / transmission. The decoder can un-compress the compressed video representation back into a viewable form. The encoder may discard some information in the original video sequence in order to represent the video in a more compact form (i.e., at lower bitrate).

[0004] The aim of the present solution is to provide an improved technique for intra prediction in video coding.

[0005] The scope of protection sought for various embodiments of the invention is set out by the independent claims. The embodiments and features, if any, described in this specification that do not fall under the scope of the independent claims are to be interpreted as examples useful for understanding various embodiments of the invention.

[0006] Various aspects include a method, an apparatus and a computer readable medium comprising a computer program stored therein, which are characterized by what is stated in the independent claims. Various embodiments are disclosed in the dependent claims.

[0007] According to a first aspect, there is provided an apparatus for decoding encoded samples of blocks of video sample, wherein for a block of video sample data the apparatus comprises means for determining an intra-prediction information, where the means for determining comprises meansfor extracting prediction information from a template sample block; means for generating two or more representations of at least some samples in the template sample block by filtering the samples with different frequency channels; means for determining intra-prediction modes for each frequency channels of the template sample block; and means for computing intra-predicted samples for the current block based on the detected intra-prediction modes and the two or more representations of at least some template samples.

[0008] According to a second aspect, there is provided a method for decoding encoded samples of blocks of video sample, wherein for a block of video sample data the method comprises determining an intra-prediction information, where the determining comprises extracting prediction information from a template sample block; generating two or more representations of at least some samples in the template sample block by filtering the samples with different frequency channels; determining intra-prediction modes for each frequency channels of the template sample block; and computing intra-predicted samples for the current block based on the detected intra-prediction modes and the two or more representations of at least some template samples.

[0009] According to a third aspect, there is provided an apparatus for decoding encoded samples of blocks of video sample, comprising at least one processor, memory including computer program code, wherein for a block of video sample data the memory and the computer program code configured to, with the at least one processor, cause the apparatus to determine an intra-prediction information, where the means for determining comprises extracting prediction information from a template sample block; generating two or more representations of at least some samples in the template sample block by filtering the samples with different frequency channels; determining intra-prediction modes for each frequency channels of the template sample block; and computing intra-predicted samples for the current block based on the detected intra-prediction modes and the two or more representations of at least some template samples.

[0010] According to a fourth aspect, there is provided computer program product for decoding encoded samples of blocks of video sample, comprising computer program code configured to, when executed on at least one processor, cause an apparatus or a system for a block of video sample data to determine an intra-prediction information, where the means for determining comprises extracting prediction information from a template sample block; generating two or more representations of at least some samples in the template sample block by filtering the samples with different frequency channels; determining intra-prediction modes for each frequency channels of the template sample block; and computing intra-predicted samples for the current block based on the detected intra-prediction modes and the two or more representations of at least some template samples.

[0011] According to an embodiment, at least one region within the current block is detected based on the detected intra-prediction mode and the location of the template sample; intra-prediction samples in the detected region are determined in accordance with the detected intra-prediction direction, and intra-prediction samples outside of the detected region are determined in accordance with an intraprediction mode different than the detected intra-prediction direction.

[0012] According to an embodiment, the two or more representations of samples comprises at least low-pass filtered representation and high-pass filtered representation.

[0013] According to an embodiment, sample values of the high-pass filtered representation is determined by deducting sample values of the low-pass filtered representation from sample values of the template sample block.

[0014] According to an embodiment, computing intra-predicted samples comprises determining an initial block of predicted samples for each frequency channel and blending initial blocks with predicted samples to form the final predicted block of intra-predicted samples.

[0015] According to an embodiment, determining an initial block of predicted samples for each frequency channel comprises assigning a different weight for different frequency channels.

[0016] According to an embodiment, the computer program product is embodied on a non-transitory computer readable medium.of the Drawings

[0017] In the following, various embodiments will be described in more detail with reference to the appended drawings, in which

[0018] Fig. 1 shows a simplified example of an encoding process;

[0019] Fig. 2 shows a simplified example of a decoding process;

[0020] Fig. 3 is a flowchart of a method according to an embodiment;

[0021] Fig. 4 is an illustration of an embodiment;

[0022] Fig. 5 is a flowchart of a method according to another embodiment;

[0023] Fig. 6 is an illustration of another embodiment having 3x3 template;

[0024] Fig. 7 is an illustration of another embodiment having 5x5 template;

[0025] Fig. 8 is a flowchart of a method according to yet another embodiment;

[0026] Fig. 9a, 9b are illustrations of yet other embodiments;

[0027] Fig. 10 is an illustration of a blending mask;

[0028] Fig. 11 is an illustration of blending predictors;

[0029] Fig. 12 is a flowchart of a method according to yet another embodiment;

[0030] Fig. 13 shows an example of an apparatus; and

[0031] Fig. 14 shows a schematic diagram of an example multimedia communication system within which various embodiments may be implemented.Embodiments

[0032] The following description and drawings are illustrative to discuss embodiments of the present solution with examples. The specific details are provided for understanding purposes. However, in certain instances, well-known or conventional details are not described in order to avoid obscuring the description. Reference in this specification to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the disclosure.

[0033] Before describing the embodiments further, a brief reference to evolution of video coding standardization is given. The present embodiments are suited within the context of next generation video coding standardization, e g., H.267 video coding standard and the ECM (Enhanced Compression Model) exploration.

[0034] A video codec comprises an encoder and a decoder. The encoder transforms the input video into a compressed representation suited for storage / transmission. The decoder can un-compress the compressed video representation back into a viewable form. The encoder may discard some information in the original video sequence in order to represent the video in a more compact form (i.e., at lower bitrate).

[0035] Figure 1 shows a simple example of a principle of an encoding process for 2D pictures, and Figure 2 shows a simple example of a principle of a decoding process for 2D pictures. In Figure 1, the following have been illustrated:- an image to be encoded (In);- a predicted representation of an image block (P'n);- a prediction error signal (Dn);- a reconstructed prediction error signal (D'n);- a preliminary reconstructed image (I'n);- a final reconstructed image (R'n);- a transform (T) and inverse transform (T-l);- a quantization (Q) and inverse quantization (Q-l);- entropy encoding (E);- a reference frame memory (RFM);- inter prediction (Pinter);- intra prediction (Pintra);- mode selection (MS), and- filtering (F).

[0036] In Figure 2 the following have been illustrated:- a predicted representation of an image block (P'n);- a reconstructed prediction error signal (D’n);- a preliminary reconstructed image (fn);- a final reconstructed image (R'n); an inverse transform (T-l);- an inverse quantization (Q-l);- an entropy decoding (E-l);- a reference frame memory (RFM);- a prediction (either inter or intra) (P);- and filtering (F).

[0037] An elementary unit for the input to an encoder and the output of a decoder, respectively, in most cases is a picture (also referred to as “an image”). A picture given as an input to an encoder may also be referred to as a source picture, and a picture decoded by a decoded may be referred to as a decoded picture or a reconstructed picture.

[0038] The source and decoded pictures are each comprised of one or more sample arrays, such as one of the following sets of sample arrays:- Luma (Y) only (monochrome).- Luma and two chroma (YCbCr or YCgCo).- Green, Blue and Red (GBR, also known as RGB).- Arrays representing other unspecified monochrome or tri-stimulus color samplings (for example, YZX, also known as XYZ).

[0039] A picture may be defined to be either a frame or a field. A frame comprises a matrix of luma samples and possibly the corresponding chroma samples. A field is a set of alternate sample rows of a frame and may be used as encoder input, when the source signal is interlaced. Chroma sample arrays may be absent (and hence monochrome sampling may be in use) or chroma sample arrays may be subsampled when compared to luma sample arrays.

[0040] The Advanced Video Coding standard (which may be abbreviated AVC or H.264 / AVC) was developed by the Joint Video Team (JVT) of the Video Coding Experts Group (VCEG) of the Telecommunications Standardization Sector of International Telecommunication Union (ITU-T)and the Moving Picture Experts Group (MPEG) of International Organization for Standardization (ISO) / International Electrotechnical Commission (IEC). There have been multiple versions of the H.264 / AVC standard, each integrating several extensions or features to the specification. These extensions include Scalable Video Coding (SVC) and Multiview Video Coding (MVC).

[0041] The High Efficiency Video Coding standard (which may be abbreviated HEVC or H.265 / HEVC) was developed by the Joint Collaborative Team - Video Coding (JCT-VC) of VCEG and MPEG. Extensions to H.265 / HEVC include scalable, multiview, three-dimensional, and fidelity range extensions, which may be referred to as SHVC, MV-HEVC, 3D-HEVC, and REXT, respectively.

[0042] Versatile Video Coding (which may be abbreviated VVC, H.266, or H.266 / VVC) is a video compression standard developed as the successor to HEVC. VVC is specified in ITU-T Recommendation H.266 and equivalently in ISO / IEC 23090-3, which is also referred to as MPEG- I Part 3.

[0043] ECM was developed by JVET (Joint Video Experts Team) of ITU-T VCEG and ISO / IEC MPEG, to provide a future video coding technology the compression capability of which would exceed that of the VVC.

[0044] Some key definitions, bitstream and coding structures, and concepts of H.264 / AVC, HEVC, VVC, and / or AVI and some of their extensions are described in this section as an example of a video encoder, decoder, encoding method, decoding method, and a bitstream structure, wherein the embodiments may be implemented. The aspects of various embodiments are not limited to H.264 / AVC, HEVC, VVC, and / or AVI or their extensions, but rather the description is given for one possible basis on top of which the present embodiments may be partly or fully realized.

[0045] Hybrid video codecs, for example ITU-T H.263, H.264 / AVC, HEVC, and VVC, may encode the video information in two phases. At first, pixel values in a certain picture area (or “block”) are predicted for example by motion compensation means or by spatial means.

[0046] In motion compensation based prediction (which may be referred to as inter prediction, temporal prediction or motion-compensated temporal prediction or motion-compensated prediction or MCP) an area in one of the previously coded frames that corresponds closely to the block being coded is found and used for prediction. Inter prediction may reduce temporal redundancy.

[0047] In spatial prediction pixel values around the block to be coded are used. In the first phase, predictive coding may be applied, for example, as so-called sample prediction and / or so-called syntax prediction. In the sample prediction, pixel or sample values in a certain picture area or "block" are predicted. These pixel or sample values can be predicted, for example, using one or more of motion compensation or intra prediction mechanisms.

[0048] Intra prediction, where pixel or sample values can be predicted by spatial mechanisms, involve finding and indicating a spatial region relationship. Intra prediction utilizes the fact that adjacent pixels within the same picture are likely to be correlated. Intra prediction can be performed in spatial or transform domain, i.e., either sample values or transform coefficients can be predicted. Intra prediction is typically exploited in intra coding, where no inter prediction is applied.

[0049] In motion vector prediction, motion vectors e.g., for inter and / or inter-view prediction may be coded differentially with respect to a block-specific predicted motion vector. In many video codecs, the predicted motion vectors are created in a predefined way, for example by calculating the median of the encoded or decoded motion vectors of the adjacent blocks. Another way to create motion vector predictions, sometimes referred to as advanced motion vector prediction (AMVP), is to generate a list of candidate predictions from adjacent blocks and / or co-located blocks in temporal reference pictures and signalling the chosen candidate as the motion vector predictor. In addition to predicting the motion vector values, the reference index of previously coded / decoded picture can be predicted. The reference index is typically predicted from adjacent blocks and / or co-located blocks in temporal reference picture. Differential coding of motion vectors is typically disabled across slice boundaries.

[0050] The block partitioning, e.g., from a coding tree unit (CTU) to coding units (CUs) and down to prediction units (PUs), may be predicted.

[0051] In filter parameter prediction, the filtering parameters e.g., for sample adaptive offset may be predicted. Prediction approaches using image information from a previously coded image can also be called as inter prediction methods which may also be referred to as temporal prediction and motion compensation. Prediction approaches using image information within the same image can also be called as intra prediction methods.

[0052] In the second phase of encoding, the prediction error, i.e., the difference between the predicted block of pixels and the original block of pixels, is coded. This may be done by transforming the difference in pixel values using a specified transform (e.g., Discrete Cosine Transform (DCT) or a variant of it), quantizing the coefficients and entropy coding the quantized coefficients. By varying the fidelity of the quantization process, encoder can control the balance between the accuracy of the pixel representation (picture quality) and size of the resulting coded video representation (file size of transmission bitrate).

[0053] In some video codecs, such as H.265 / HEVC, video pictures are divided into coding units (CU) covering the area of the picture. A CU consists of one or more prediction units (PU) defining the prediction process for the samples within the CU and one or more transform units (TU) defining the prediction error coding process for the samples in the said CU. A CU may consist of a squareblock of samples with a size selectable from a predefined set of possible CU sizes. A CU with the maximum allowed size is typically named as LCU (largest coding unit) or CTU (coding tree unit) and the video picture is divided into non-overlapping CTUs. A CTU can be further split into a combination of smaller CUs, e.g., by recursively splitting the CTU and resultant CUs. Each resulting CU typically has at least one PU and at least one TU associated with it. Each PU and TU can be further split into smaller PUs and TUs in order to increase granularity of the prediction and prediction error coding processes, respectively. Each PU has prediction information associated with it defining what kind of a prediction is to be applied for the pixels within that PU (e.g., motion vector information for inter predicted PUs and intra prediction directionality information for intra predicted PUs). Similarly, each TU is associated with information describing the prediction error decoding process for the samples within the said TU (including e.g., DCT coefficient information). It is typically signaled at CU level whether prediction error coding is applied or not for each CU. In the case there is no prediction error residual associated with the CU, it can be considered there are no TUs for the said CU. The division of the image into CUs, and division of CUs into PUs and TUs is typically signaled in the bitstream allowing the decoder to reproduce the intended structure of these units.

[0054] The decoder reconstructs the output video by applying prediction means similar to the encoder to form a predicted representation of the pixel blocks (using the motion or spatial information created by the encoder and stored in the compressed representation) and prediction error decoding (inverse operation of the prediction error coding recovering the quantized prediction error signal in spatial pixel domain). After applying prediction and prediction error decoding means the decoder sums up the prediction and prediction error signals (pixel values) to form the output video frame. The decoder (and encoder) can also apply additional filtering means to improve the quality of the output video before passing it for display and / or storing it as prediction reference for the forthcoming frames in the video sequence.

[0055] In many video codecs, including H.264 / AVC, HEVC, and VVC, motion information is indicated by motion vectors associated with each motion compensated image block. Each of these motion vectors represents the displacement of the image block in the picture to be coded (in the encoder) or decoded (at the decoder) and the prediction source block in one of the previously coded or decoded images (or pictures). In order to represent motion vectors efficiently those are typically coded differentially with respect to block specific predicted motion vectors. In typical video codecs the predicted motion vectors are created in a predefined way, for example calculating the median of the encoded or decoded motion vectors of the adjacent blocks. Another way to create motion vector predictions is to generate a list of candidate predictions from adjacent blocks and / or co-located blocks in temporal reference pictures and signaling the chosen candidate as the motion vector predictor. In addition to predicting the motion vector values, the reference index of previously coded / decoded picture can be predicted. The reference index may be predicted from adjacent blocks and / or or co-located blocks in temporal reference picture. Moreover, high efficiency video codecs may employ an additional motion information coding / decoding mechanism, often called merging / merge mode, where all the motion field information, which includes motion vector and corresponding reference picture index for each available reference picture list, is predicted and used without any modification / correction. Similarly, predicting the motion field information is carried out using the motion field information of adjacent blocks and / or co-located blocks in temporal reference pictures and the used motion field information is signaled among a list of motion field candidate list filled with motion field information of available adjacent / co-located blocks.

[0056] Video codecs may support motion compensated prediction from one source image (uniprediction) and two sources (bi-prediction). In the case of uni -prediction a single motion vector is applied whereas in the case of bi-prediction two motion vectors are signaled and the motion compensated predictions from two sources are averaged to create the final sample prediction. In the case of weighted prediction, the relative weights of the two predictions can be adjusted, or a signaled offset can be added to the prediction signal

[0057] In addition to applying motion compensation for inter picture prediction, similar approach can be applied to intra picture prediction. In this case the displacement vector indicates where from the same picture a block of samples can be copied to form a prediction of the block to be coded or decoded. This kind of intra block copying methods can improve the coding efficiency substantially in presence of repeating structures within the frame - such as text or other graphics.

[0058] In video codecs the prediction residual after motion compensation or intra prediction may be first transformed with a transform kernel (like DCT) and then coded. The reason for this is that often there still exists some correlation among the residual and transform can in many cases help reduce this correlation and provide more efficient coding.

[0059] Many video encoders utilize Lagrangian cost functions to find optimal coding modes, e g., the desired Macroblock mode and associated motion vectors. This kind of cost function uses a weighting factor to tie together the (exact or estimated) image distortion due to lossy coding methods and the (exact or estimated) amount of information that is required to represent the pixel values in an image area:C = D + XR (Eq. 1)Where C is the Lagrangian cost to be minimized, D is the image distortion (e.g., Mean Squared Error) with the mode and motion vectors considered, and R the number of bits needed to represent the required data to reconstruct the image block in the decoder (including the amount of data to represent the candidate motion vectors).

[0060] The phrase along the bitstream (e.g., indicating along the bitstream) may be defined to refer to out-of-band transmission, signaling, or storage in a manner that the out-of-band data is associated with the bitstream. The phrase decoding along the bitstream or alike may refer to decoding the referred out-of-band data (which may be obtained from out-of-band transmission, signaling, or storage) that is associated with the bitstream. For example, an indication along the bitstream may refer to metadata in a container file that encapsulates the bitstream.

[0061] There are number of intra-prediction modes available for existing video codecs. These intraprediction modes comprise different directional (i.e., horizontal, or vertical, or diagonal) intraprediction modes, as well as prediction modes such as DC or Planar intra-prediction

[0062] In Planar intra-prediction, interpolation processes in both the vertical and horizontal directions are performed to obtain vertical and horizontal predictors, respectively. Then, sample-by-sample mean of these two predictors can be computed. Each interpolation process may be carried out by computing weighted averages of the reference samples. As an example, when computing the vertical predictor, each predicted sample at a given horizontal coordinate is obtained as the weighted average between the two reference samples at the same horizontal coordinate extracted from the top row and bottom row of the current block, respectively. Due to the fact the samples located on the bottom row of the current block are not reconstructed when predicting the current block, these are padded using the sample on the bottom-left of the current block. Similarly, the horizontal predictor is computed as the weighted average of the reference samples extracted from the columns on the left and right of the current block.

[0063] ECM includes several modes that operate by analyzing a template around the current block in order to determine a prediction for the current block.

[0064] Decoder Side Intra Mode Derivation (DIMD) is a method to determine the Intra prediction mode of the current block using directionality of the texture of the neighboring reconstructed samples located in a template region at top, top-left, and left sides of the current block. DIMD comprises steps for- deriving two (or more) intra prediction modes and their corresponding weight factors;- generating predictors for the derived intra prediction modes, and also for an additional predetermined mode (which can be the Planar mode, or can be a block-vector based prediction mode);- generating the final prediction for the current block by combining the generated predictors by means of sample-wise or uniform weighting, using the derived weight factors.

[0065] In DIMD, directionality of texture is derived for each neighboring sample in the template region, using 3x3 neighboring samples of that sample. This derivation is performed in several steps:- calculating the horizontal and vertical direction strengths (Dx and Dy) using 3x3 neighbouring samples of the training sample;- determining the corresponding region on the Intra prediction mode (angle) using sign of Dx and Dy;- calculating the ratio of Dx / Dy;- determining the corresponding angle index using the determined ratio value and a table that maps the ratio to proper angle index;- determining the final intra prediction mode for that training sample using the region and derived angle;- deriving the amplitude (or importance) of this intra prediction mode as the sum of absolute values of Dx and Dy.

[0066] The intra prediction mode (corresponding to a given directionality) and its corresponding amplitude are derived as above for each training sample. These are then collected into a histogram of gradients (“HoG”), namely a histogram collecting for each intra prediction mode the cumulative amplitude of all neighbouring samples with that directionality. Finally, a pre-determined number of intra prediction modes are derived from the histogram of gradients, by finding the dominant intra prediction modes, namely the modes with the highest amplitudes in the histogram. The weights of each extracted intra prediction mode are determined based on these amplitudes, where higher amplitudes correspond to higher weights. A fixed weight is typically assigned to a pre-determined mode, which is blended together with the determined intra prediction modes to form the final DIMD prediction. Such pre-determined mode may be the Planar mode, or may be a block-vector based predictor.

[0067] DIMD is considered in ECM as an option signaled at the encoder side. In addition, the DIMD modes (prior to blending) are also included in the list of Most Probable Modes (MPM) for signalling as individual MPM candidates.

[0068] Recently, an extension to DIMD was proposed where smaller 2x2 kernels are used for certain block sizes. When using such approach, rather than using the traditional 3x3 Sobel operators, smaller 2x2 kernels are used in certain cases when the block size is small.

[0069] Since existing template-based schemes may not correctly capture local edges or localized texture within a block, the idea of the present embodiments is to produce an improved technique for intra-prediction for a given block by computing one or more predictors for the current block, where a predictor is computed based on detecting at least one edge around at least one given template sample at a given location within a template. The detection of the edge includes detecting a given intra-prediction direction (also referred to as an intra-prediction mode), where the predictor is computed based on the detected intra-prediction direction and the location of the at least one given template sample.

[0070] Figure 3 illustrates a flowchart of a method according to an embodiment. The method is targeted to decoding encoded samples of blocks of video sample data, wherein for a block of video sample data the method comprises determining an intra prediction information, where the determining comprises at least- determining 330 at least one intra-prediction mode for a template sample at a given location within a template; and- determining 340 intra-predicted samples for the current block based on the detected intraprediction mode and the location of the template sample.

[0071] For the computation of the intra-predicted samples, the method according to a further embodiment comprises also- detecting at least one region within the current block based on the detected intra-prediction mode and the location of the template sample;- determining intra-prediction samples in the detected region in accordance with the detected intra-prediction direction; and- determining intra-prediction samples outside of the detected region in accordance with an intra-prediction mode different than the detected intra-prediction direction.

[0072] A method operating according to the flowchart of Figure 3 produces a prediction for a given block by computing one or more predictors for the current block. The predictor is based on a prediction information computed based on detecting at least one edge, where the edge is detected on a given window of samples around at least one given template sample at a given location within a template. The edge is detected by using a conventional edge-detection technique. The edge is used to define an intra-prediction direction, i.e., intra-prediction mode, where the samples in the currentblock are predicted based on the detected intra-prediction direction and the location of the at least one given template sample. The detected intra-prediction direction can be a directional intraprediction mode such as the horizontal, or vertical, or diagonal intra prediction mode, or any other intra-prediction mode.

[0073] More specifically, after a given edge is detected for a given template sample at a given location within a template, the prediction information for the current block can be computed by propagating the detected edge within the current block. In details, according to an embodiment, an edge is detected in a template sample at a certain location; intra-prediction mode (i.e., intra-prediction direction) is determined based on the detected edge; a region is determined in the current block by propagating the direction of the edge in the current block; prediction samples are determined in the region; the prediction samples in the region are predicted with determined inter-prediction mode. The region can be determined as a collection of samples at specific locations within the current block, where the locations are computed based on the location of the template sample, and on the detected intra-prediction direction. The locations can be determined by projecting the detected edge from the location of the template sample into the current block, in accordance with the detected intra-prediction direction. Locations that are within a certain distance of the projected edge can be determined to be part of the computed region. Prediction samples that are not located in the region are not predicted in accordance to the detected intra-prediction direction. As an example, prediction samples that are not located in the region along the direction of propagation of the edge may be predicted in accordance with a pre-defined intra-prediction mode. As an example, the pre-defined prediction mode could be the Planar mode, or a Position Dependent Prediction (PDP) mode, or a Block Vector (BV) based prediction mode.

[0074] Figure 4 illustrates an example of a current block 410 and a template 420, 430 having template samples that are already coded samples. In Figure 4 template 420 has template sample 425 to which an intra-prediction mode 450 is detected. The predictor is computed by propagating the edge within the current block 410. In Figure 4, the propagation is shown in a region 415 along a direction of propagation of the edge. Locations that are within a certain distance of the projected edge may be determined to be part of the region.

[0075] Figure 5 illustrates a flowchart of a method according to another embodiment, wherein larger, 5x5, matrices are used rather than using the traditional 3x3 Sobel operators for detecting edges in larger blocks. The method comprises for a block of video sample data determining an intraprediction, where the determining comprises- analysing 520 at least one template sample in a template, where the analysing comprises:• determining a size;• selecting one or more matrices of the determined size;• multiplying coefficients in the matrices by samples in a window area of the determined size including the at least one template samplewhere the determining of the size depends on the characteristics of the current block; - detecting 530 at least one intra-prediction mode based on the analysis.

[0076] A method operating according flowchart of Figure 5 produces a prediction for a given block by computing one or more predictors for the current block, where a predictor is computed based on detecting at least one edge in at least one given template sample at a given location within a template, where the detecting of the edge includes detecting a given intra-prediction direction, where the intra-prediction direction is determined based on window matrices, where 3x3 matrices are used for blocks with an area smaller or equal than a given threshold, and matrices larger than 3x3 are used for blocks with an area larger than the given threshold. As an example, 5x5 matrices can be used for blocks with an area larger than the given threshold.

[0077] As an example, for a given template sample at a certain location within the template, an edge can be detected, where the detecting of the edge may consist of determining a certain intraprediction mode (i.e., an intra-prediction direction), which is the direction of the edge, as well as a given amplitude of that direction, which is the strength of the edge at that location.

[0078] As an example, to measure direction and amplitude for a gradient in a determined location, two 3x3 window matrices, Wx and Wy, can be used. The coefficients given for those matrices can be convolved with the samples in the 3x3 window area centered around the given template sample. As a result, corresponding horizontal and vertical direction strengths, Dx and Dy, can be derived. Finally based on Dx and Dy, a certain intra-prediction direction can be determined, as well as a given amplitude.

[0079] As an example, the Wx and Wy matrices can be determined as follows:1 0 -1 1 2 1V14 = 2 0 —2 Wy= 0 0 0.1 0 -1. -1 —2 -1.

[0080] Or, as another example, Wx and Wy matrices can be determined as:n 0 -1Wy= 2 0 0.1 0 1 .

[0081] Considering a sample located at (x,y) in the current block of dimensions WxH, with the origin (0,0) located at the top-left corner of the block, and assuming usage of 3x3 window matrices, then the template samples can be extracted from a template sample line located at (-2,1) for a vertical template, and at (1,-2) for a horizontal template. Figure 6 shows an example of this process. The first template sample 625 from the horizontal template 620 is considered at location (1, -2). The corresponding 3x3 window area 628 centered around the template sample 625 is shown, and the estimated direction of the edge is marked with an arrow 650.

[0082] The corresponding templates 620, 630 need to be formed of at least three rows of samples above the current block, and / or three columns of samples on the left of the current block.

[0083] In order to consider a larger area in the template for edge estimation, especially for larger blocks it may be beneficial to use larger window matrices. As an example, 5x5 matrices can be used for larger blocks, and smaller 3x3 matrices can be used for smaller blocks. As an example, larger 5x5 matrices can be used in case the area of the block is equal or larger than 512 samples. As an example, larger 5x5 matrices can be used in case the area of the block is equal or larger than 1024 samples. Figure 7 shows an example using a 5x5 window to determine an estimated direction for a certain sample within a template. The first template sample 725 from the horizontal template 720 is considered at location (1, -2). The corresponding 5x5 window area 728 centered around the template sample 725 is shown, and the estimated direction of the edge is marked with an arrow 750.

[0084] As an example, larger 5x5 matrices can be used. As an example, the Wx and Wy matrices can be determined as follows:5 4 0 -4 -5 -5 -8 -10 -8 -58 10 0 -10 -8 —4 -10 -20 -10 —4144 = 10 20 0 -20 -10 Wy = 0 0 0 0 08 10 0 -10 -8 4 10 20 10 4- 5 4 0 -4 -5 - 5 8 10 8 5 -

[0085] As an example, the Wx and Wy matrices can be determined as follows:1 2 0 —2 -1- ■-1 —4 — 6 —4 -14 8 0 -8 —4 —2 -8 -12 -8 —2W4. = 6 12 0 -12 -6 W / y= 0 0 0 0 04 8 0 -8 —4 2 8 12 8 2-1 2 0 -2 -1- - 1 4 6 4 1

[0086] When using larger matrices, the corresponding templates need to be extended to allow at least 5 rows of samples above the current block, and / or 3 columns of samples on the left of the current block, as illustrated in Figure 7. When extending the templates, appropriate checks may need to beperformed, to make sure that the samples within the extended region are available, i.e. they are already reconstructed. In case not all samples in the extended template have been already reconstructed, then the template region may be restricted to contain only already reconstructed samples. Otherwise, in case not all samples in the extended template have been already reconstructed, then smaller 3x3 matrices can be used. Otherwise, in case not all samples in the extended template have been already reconstructed, then these ca be padded using other already reconstructed samples.

[0087] Figure 8 illustrates a flowchart a method according to another embodiment, wherein computation of blending masks based on location of reference sample in the template is used. The method comprises for a block of video sample data determining an intra-prediction information, where the determining of the intra-prediction comprises• detecting 820 at least one intra-prediction mode for a given template sample at a given location within a template;• determining 830 a blending mask, where the blending mask depends on the location of the template sample and on the detected intra-prediction mode;• determining 840 intra-predicted samples for the current block based on the detected intra-prediction mode; and• multiplying 850 the intra-predicted samples by the coefficients in the blending mask.

[0088] A method operating according to the flowchart of Figure 8 produces a prediction for a given block by detecting at least one edge in at least one given template sample at a given location within a template, where the detecting of the edge includes detecting a given intra-prediction direction (i.e., intra-prediction mode), where the prediction for the given block is computed by propagating the edge within the block, where the edge is propagated by means of computing a given predictor and a corresponding blending mask, where the blending mask is computed depending on the location of the template sample and on the detected intra-prediction direction.

[0089] For a given edge detected at a given template sample, with a corresponding detected intraprediction direction, a predictor can be determined by applying an intra-prediction process to the current block following the determined intra-prediction direction. Such predictor can be weighted on a sample-by-sample basis, where specific weights are determined for each sample within the block in order to propagate the edge within the current block.

[0090] In particular, a blending mask can be determined based on the location of the template sample within the template, and on the detected intra-prediction direction.

[0091] As an example, the blending mask can be computed by applying an intra-prediction process to the current block, where the reference array input to the intra-prediction process is formed of a predetermined set of artificially computed values. The artificially computed values depend on the location of the corresponding template sample within the template.

[0092] Considering a sample located at (x,y) in the current block of dimensions WxH, with the origin (0,0) located at the top-left corner of the block. Consider as an example the template sample at location (1, -2) from the horizontal template. Then, an array of artificially computed reference values can be determined based on the location of the template sample. As an example, the array may be formed of all zero-valued reference samples, apart from the reference sample at the location of the template sample (1, -2), which may be assigned a pre-determined non zero value X, as illustrated in Figure 9a. As another example, non-zero values may be assigned to reference samples located next to the location of the template sample (1, -2). As an example, a pre-determined nonzero values Y and Z may be assigned to reference samples at location (0,-2) and to location (2,-2), respectively, as illustrated in Figure 9b.

[0093] Following the computation of the artificially computed reference samples, the blending mask may then be computed by applying an intra-prediction process where the process may follow the detected intra-prediction direction. As an example, assuming that a given intra-prediction direction M was detected for the template sample at location (1,-2), then a blending mask may be computed by predicting the samples within the current block using mode M on the artificially generated samples. As an example, if M is a pure vertical intra-prediction mode, and assuming that X = 4, Y = Z = 2, then a blending mask may be computed as illustrated in Figure 10. The intra-prediction process may be applied using a Multiple Reference Index which depends on the location of the template sample. As an example, if the sample is located at a location which is one line apart from the current block, then a Multiple Reference Index equal to one may be used. As an example, a Multiple Reference Index equal to one may be used if the template sample is located above at a position (X, -2), or on the left at a position (-2, Y). As an example, the Multiple Reference Index may depend on the size of the matrices used to tdetermine the edge. As an example, a Multiple Reference Index equal to one may be used if 3x3 matrices are used, whereas a Multiple Reference Index equal to two may be used if 5x5 matrices are used.

[0094] The blending mask computed for a given edge detected at a given reference sample may be used to propagate the edge within the current block. In particular, a corresponding predictor can be computed by applying the detected intra-prediction direction to the current block. Such predictor can then be weighted using the blending mask. In particular, the prediction samples in the predictor can be multiplied by the co-located weights in the blending mask.

[0095] This process may be performed for a number of template samples within the template. As an example, a given edge may be detected for any template sample within the template, and a corresponding weighted predictor may be computed for each reference sample within the template. As an example, only a pre-defined number of weighted predictors may be computed. As an example, the reference samples may be sorted based on the amplitude of the detected edge, and only the predictors corresponding to the N highest amplitudes may be considered. As an example, an amplitude for a given sample at location (i,j) can be computed as Amp(i,j) = \DX| + |Dy|. As an example, only predictors at specific pre-defined locations may be considered. As an example, only predictors at locations which result in amplitudes different than zero may be considered. As an example, only predictors at locations which result in amplitudes higher or equal than a given threshold may be considered.

[0096] For the given reference samples, the corresponding edge is detected and propagated within the current block by computing the corresponding predictor P and the blending mask B.

[0097] The weighted predictors can finally be used to form a final predictor for the current block. The predictors may be blended one with each other, in order to propagate different edges within the block, as illustrated in Figure 11, where arrows 1110, 1120, 1130 correspond to intra-prediction directions of edges is corresponding template samples.

[0098] An additional support predictor may be computed based on a pre-determined intra-prediction mode. As an example, a support predictor may be considered based on Planar intra-prediction. As an example, a support predictor may be considered based on Position Dependent Prediction (PDP). As an example, a support predictor may be considered by extracting a given array of already reconstructed samples from the current block pointed by a determined Block Vector (BV).

[0099] The final prediction may be computed by determining weights for the support predictor and for the at least one weighted predictor, where the weights may depend on the corresponding blending masks. Denote for instance the blending mask as B. As an example, a weighted predictor A may be computed where for each location within the current block, the weighted predictor at that location A(x,y) is computed by multiplying the corresponding prediction sample P(x,y) by the co-located sample in the blending mask B(x,y).A(x,y) = P(x,y) * B(x,y)

[0100] The weighting may be adjusted or scaled based on the amplitude of the detected direction. As an example, denoting the amplitude as Amp, then:A(x,y) = Amp * P(x,y) * B(x,y)

[0101] More in details, let’s assume that there are N blending masks available BO ... BN-1.

[0102] S is denoted for the matrix obtained by the sample-wise summing all blending masks together. Also, smax is denoted for the maximum element in S, namely the maximum sum of all blending weights for any location in the block.

[0103] For a given prediction sample location (x,y), the weight for the support predictor can be computed as:Wsupport(x,y) = ( smax - S(x,y) ) / smax

[0104] And the weight for a given predictor i can be calculated asWi(x,y) = Bi(x,y) / smax

[0105] In order to account for the amplitudes, the blending masks can be multiplied by the corresponding amplitude.

[0106] The weights can be scaled in order to perform operations with integer values. As an example, the weights can be scaled by a value of 64.

[0107] Figure 12 illustrates a flowchart of a method according to yet another embodiment, wherein edges are propagated using frequency -based separation. The method comprises for a block of video sample data determining an intra-prediction information, where the determining compriseso extracting 1220 prediction information from a template sample block;o generating 1230 two or more representations of at least some samples in the template sample block by filtering the samples with different frequency channels; o determining 1240 intra-prediction modes for each frequency channels of the template sample block;o computing 1250 intra-predicted samples for the current block based on the detected intra-prediction modes and the two or more representations of at least some template samples.

[0108] In an embodiment, edge structure or structures present in the reference area are propagated into the prediction block using frequency -based separation or other processes able to generate multiple classes of representations of reference samples. As an example, two or more representations of at least some reference samples can be calculated. One representation or reference samples, or a firstset of modified reference samples, can be generated by low pass filtering means. 1 -dimensional (ID) or 2-dimensional (2D) filters can be used to generate such modified reference samples. As an example, a simple ID filter:riow(i) = [ r(i-l) + 2*r(i) + r(i+l) ] / 4can be applied horizontally on the horizontal reference line above the prediction block and vertically on the vertical reference line left of the block. Another representation of the reference samples, or a second set of modified reference samples, can be generated by high pass or band pass filtering means. Again, ID or 2D filters can be used, or the high pass reference samples can be simply calculated by deducting the corresponding low pass reference sample riow(i) from the unmodified reference sample r(i):fhigh(i) = r(i) - riow(i)

[0109] It is appreciated that any kind of alternative filters can be used instead or in addition to these simple filters provided as an example. In general, there can be, for example, N classes and modified reference samples q for each class j can be determined using a class specific sample generator function j and reference sample set or reference sample vector r:rj = gj(r)

[0110] DIMD process or TIMD (i.e., Template-based Intra Mode Derivation) process, or any other technique using all reference samples or subsets of reference samples can be used to determine one or more prediction directions for the block. If there are multiple representations of reference samples for different frequence bands or other classes, an individual prediction direction can be determined for each such frequency band or class. The determination of such class specific prediction directions can be advantageously performed using reference samples associated with that specific class. This would allow determining each prediction direction in parallel and using a relatively small number of reference samples in the process.

[0111] Once prediction directions have been determined for each frequence band or class, the prediction directions and associated reference samples are used to determine predicted samples for the block. This can be performed in different ways. For example, an initial block of predicted samples Pj(x,y) can be determined using traditional directional intra prediction means for each class or frequency band using the individual prediction direction dj and reference samples for that class or frequency band:pj(x,y) = f(x, y, dj, rj)where f is a function performing intra prediction for a sample at position (x,y), using prediction direction or intra prediction mode dj and reference samples set or reference sample vector rj.

[0112] The determined initial blocks of predicted samples can then be blended together to form the final prediction block. This can naturally be done in various ways. For example, using one of the approaches described in various embodiments herein or a combination of those. As a further example, the combination can also be done by adding the initial prediction blocks together. This is possible especially in the case the different sets of modified reference samples were generated using low pass and band pass filters where the pass bands of the filters don’t overlap substantially with each other. It the case of such a selection, each initial prediction block represents prediction of textures with a certain distinct frequence range and adding those together will reproduce a prediction block with all frequency content restored in a unified representation. This can be denoted as follows for N classes or frequency bands:N-lp(x,y) = ^ PjCx.y)j=0

[0113] As another example, initial predictions with different classification or frequency content can be weighted differently when determining the final prediction block. For example, the low band prediction can be given a weight or 1, while predictions for higher frequencies can be given a lower weight. This can help improve prediction accuracy as typically the lower frequencies in image content are continuous and thus easier to predict compared to higher frequencies which represent more complex textures or even noise. Weights for each class can be pre-determined or those can be calculated in different ways. For example, variance or strength of the texture or amplitude of a gradient measure in the reference samples can be determined for each class and the weights Wj for the class j can be derived based on those and included in the calculation of the final prediction:TV— 1P(x,y) = ^ w,p;(x,y)7=o

[0114] Weights wj can further be make position and prediction direction dependent as described in various other embodiments. In such example, the predicted samples may be determined as:N-lP (x, y) = Wj (X, y, dj^pj (x, y)7=o

[0115] An apparatus according to an embodiment comprises means for determining an intra prediction information for a block of video sample data, where determining an intra prediction information is based on means for determining at least one intra-prediction mode for a template sample at a given location within a template and means for determining intra-predicted samples for the current block based on the detected intra-prediction mode and the location of the template sample.

[0116] As a further improvement, the apparatus may comprise means for detecting at least one region within the current block based on the detected intra-prediction mode and the location of the template sample; means for computing intra-prediction samples in the detected region in accordance with the detected intra-prediction direction, and means for computing intra-prediction samples outside of the detected region in accordance with an intra-prediction mode different than the detected intraprediction direction. The means comprises at least one processor, and a memory including a computer program code, wherein the processor may further comprise processor circuitry. The memory and the computer program code are configured to, with the at least one processor, cause the apparatus to perform the method of Figure 3 according to various embodiments.

[0117] According to an embodiment, complementing the previous, determining at least one intraprediction mode for a template sample at a given location within a template comprises determining an edge appearing in a window of samples including the template sample.

[0118] According to an embodiment, complementing the previous embodiments, the intra-prediction mode for samples outside the detected region is one of the following: a planar mode, a position dependent prediction mode, a block vector-based prediction mode.

[0119] An apparatus according to another embodiment comprises means for determining an intra prediction information for a block of video sample data, where determining an intra prediction information is based on means for analysing at least one template sample in a template, where the analysing comprises: determining a size; selecting one or more matrices of the determined size; multiplying coefficients in the matrices by samples in a window area of the determined size including the at least one template sample, where the determining of the size depends on the characteristics of the current block; and means for detecting at least one intra-prediction mode based on the analysis. The means comprises at least one processor, and a memory including a computer program code, wherein the processor may further comprise processor circuitry. The memory and the computer program code are configured to, with the at least one processor, cause the apparatus to perform the method of Figure 5 according to various embodiments.

[0120] According to an embodiment, complementing the previous embodiment, the means for determining a size comprises means for selecting between a number of sizes comprising at least a size equal to 3x3 and a size equal to 5x5.

[0121] According to an embodiment, complementing the previous embodiments, the characteristics of the current block comprise the area of the current block.

[0122] According to an embodiment, complementing the previous embodiments, the size is determined to be equal to 5x5 if the current block has an area larger or equal than a pre-determined threshold, and the size is determined to be smaller than 5x5 if the current lock has an area smaller than the pre-determined threshold.

[0123] According to an embodiment, complementing the previous embodiments, the pre-determined threshold is based on an area of 512 samples.

[0124] According to an embodiment, complementing the previous, the prediction information is an edge appearing in a window area around the template sample.

[0125] An apparatus according to another embodiment comprises means for determining an intra prediction information for a block of video sample data, where determining an intra prediction information is based on means for detecting at least one intra-prediction mode for a given template sample at a given location within a template; means for determining a blending mask, where the blending mask depends on the location of the template sample and on the detected intra-prediction mode; means for determining intra-predicted samples for the current block based on the detected intra-prediction mode; and means for multiplying the intra-predicted samples by the coefficients in the blending mask. The means comprises at least one processor, and a memory including a computer program code, wherein the processor may further comprise processor circuitry. The memory and the computer program code are configured to, with the at least one processor, cause the apparatus to perform the method of Figure 8 according to various embodiments.

[0126] According to an embodiment, complementing the previous embodiment, the apparatus further comprises means for determining a set of synthetic reference values, wherein the synthetic reference values are used for determining the blending mask.

[0127] According to an embodiment, complementing the previous embodiment, the apparatus further comprises means for determining the blending mask by applying an intra-prediction process to the block using the set of synthetic values as reference values.

[0128] According to an embodiment, complementing the previous embodiments, the apparatus further comprises means for determining an additional support prediction information based on one of the following: a planar intra-prediction, a position dependent prediction, usage of a block vector.

[0129] According to an embodiment, complementing the previous embodiment, the apparatus comprises means for determining the final prediction by determining weights for the support predictor and for at least one weighted predictor, where weights depend on the blending mask.

[0130] According to an embodiment, complementing the previous, the prediction information is an edge appearing in a window of samples including the template sample.

[0131] An apparatus according to another embodiment comprises means for determining an intra prediction information for a block of video sample data; where determining an intra prediction information is based on means for extracting prediction information from a template sample block; means for generating two or more representations of samples in the template sample block by filtering the samples with different frequency channels; means for determining intra-prediction modes for the each frequency channels of the template sample block; and means for computing intra-predicted samples for the current block based on the detected intra-prediction modes. The means comprises at least one processor, and a memory including a computer program code, wherein the processor may further comprise processor circuitry. The memory and the computer program code are configured to, with the at least one processor, cause the apparatus to perform the method of Figure 12 according to various embodiments.

[0132] According to an embodiment, complementing the previous embodiment, the two or more representations of samples comprises at least low-pass filtered representation and high-pass filtered representation.

[0133] According to an embodiment, complementing the previous embodiments, sample values of the high-pass filtered representation is determined by deducting sample values of the low-pass filtered representation from sample values of the template sample block.

[0134] According to an embodiment, complementing the previous embodiments, the means for computing intra-predicted samples comprises means for determining an initial block of predicted samples for each frequency channel and means for blending initial blocks with predicted samples to form the final predicted block of intra-predicted samples.

[0135] According to an embodiment, complementing the previous embodiments, the means for determining an initial block of predicted samples for each frequency channel comprises means for assigning a different weight for different frequency channels.

[0136] According to an embodiment, complementing the previous embodiments, the means for computing intra-predicted samples comprises means for determining an initial block of predicted samples for each frequency channel and means for blending initial blocks with predicted samples to form the final predicted block of intra-predicted samples.

[0137] According to an embodiment, complementing the previous embodiments, the means for determining an initial block of predicted samples for each frequency channel comprises means for assigning a different weight for different frequency channels.

[0138] Figure 13 illustrates an example of an electronic apparatus 1300, being an example of video coding system where the present embodiments can be implemented. In some embodiments, the apparatus may be a mobile terminal or a user equipment of a wireless communication system or a camera device. The apparatus 300 may also be comprised at a local or a remote server or a graphic processing unit of a computer. The apparatus may also be comprised as part of a head-mounted display device.

[0139] The apparatus may be configured to perform various functions, such as for example, gathering information by one or more sensors, encoding and / or decoding information, receiving and / or transmitting information, analyzing information gathered or received by the apparatus. An apparatus configured to encode a video scene may optionally comprise one or more microphones for capturing the scene and / or one or more cameras for capturing information about the physical environment in which the scene is captured. Alternatively, the apparatus configured for encoding may be configured to receive information about an environment in which a scene is captured and / or a simulated environment. An apparatus configured to decode and / or render the video scene may be configured to receive a bitstream comprising encoded video. An apparatus configured to decode and / or render the video scene may comprise one or more speakers / audio transducers and / or displays, and / or may be configured to transmit a decoded scene or signals to a device comprising one or more speakers / audio transducers and / or displays. An apparatus configured to decode and / or render the video scene may comprise a user equipment, a head-mounted display, or another device capable of rendering to a user an AR; VR and / or MR experience.

[0140] The apparatus 1300 comprises one or more processors 1310 and one or more memories 1320 and one or more transceivers interconnected through one or more buses. The one or more memories 1320 store computer instructions, for example in respective modules (Modulel, Module2, ModuleN). The one or more memories may store data in the form of image, video and / or audio data, and / or may also store instructions to be executed by the processors or the processor circuitry. The one or more processors may comprise a central processing unit (CPU) and / or a graphical processing unit (GPU). The one or more buses may be address, data or control buses, and may include interconnection mechanism, such as a series of lines on a motherboard or integrated circuit, fiber optics or other optical communication equipment. The apparatus also comprises a codec 630 that is configured to implement various embodiments relating to present solution. According to some embodiments, the apparatus may comprise an encoder or a decoder. The apparatus 600 also comprises a communication interface 1340 which is suitable for generating wireless communication signals for example for communication with a cellular communications network, a wirelesscommunications system, or a wireless local area network, and thus enabling data transfer over data transfer network 1350.

[0141] The apparatus 1300 may comprise a display in the form of a liquid crystal display. In other embodiments of the invention the display may be any suitable display technology suitable to display an image or video. The apparatus 1300 may further comprise a keypad. In other embodiments of the invention any suitable data or user interface mechanism may be employed. For example, the user interface may be implemented as a virtual keyboard or data entry system as part of a touch- sensitive display. The apparatus 1300 may comprise a microphone or any suitable audio input which may be a digital or analogue signal input. The apparatus 1300 may further comprise an audio output device which in embodiments of the invention may be any one of: an earpiece, speaker, or an analogue audio or digital audio output connection. The apparatus 1300 may also comprise a battery (or in other embodiments of the invention the device may be powered by any suitable mobile energy device such as solar cell, fuel cell or clockwork generator). The apparatus may further comprise a camera capable of recording or capturing images and / or video. The camera may be a multi-lens camera system having at least two camera sensors. The camera is capable of recording or detecting individual frames which are then passed to the codec 1330 or to processor 1310. The apparatus may receive the video and / or image data for processing from another device prior to transmission and / or storage.

[0142] The apparatus 1300 may further comprise e.g., the other functional units disclosed in any of the Figures 1 - 2 for implementing any of the present embodiments.

[0143] The apparatus may operate in a system, comprising multiple communication devices, which can communicate through one or more networks. The system may comprise any combination of wired or wireless networks including, but not limited to a wireless cellular telephone network (such as a GSM, UMTS, CDMA network etc.), a wireless local area network (WLAN) such as defined by any of the IEEE 802.x standards, a Bluetooth personal area network, an Ethernet local area network, a token ring local area network, a wide area network, and the Internet.

[0144] For example, the system can be a mobile telephone network enabling a connection to the internet. The connection can form, but is not limited to, long range wireless connections, short range wireless connections, and various wired connections including, but not limited to, telephone lines, cable lines, power lines, and similar communication pathways.

[0145] The example communication devices operating in the system may include, but are not limited to, an electronic device or apparatus, a combination of a personal digital assistant (PDA) and a mobile telephone, a PDA, an integrated messaging device (IMD), a desktop computer, a notebook computer, each of which can be a representative of the apparatus according to present embodiments.The apparatus according to present embodiments may be stationary or mobile when carried by an individual who is moving. The apparatus may also be located in a mode of transport including, but not limited to, a car, a truck, a taxi, a bus, a train, a boat, an airplane, a bicycle, a motorcycle, or any similar suitable mode of transport.

[0146] The apparatus may also be a set-top box; i.e. a digital TV receiver, which may / may not have a display or wireless capabilities, a tablet or (laptop) a personal computer (PC), which have hardware or software or combination of the encoder / decoder implementations, in various operating systems, or a chipset, processor, DSP and / or embedded system offering hardware / software based coding.

[0147] The apparatus according to present embodiments may send and receive calls and messages and communicate with service providers through a wireless connection to a base station. The base station may be connected to a network server that allows communication between the mobile telephone network and the internet.

[0148] The apparatus may communicate using various transmission technologies including, but not limited to, code division multiple access (CDMA), global systems for mobile communications (GSM), universal mobile telecommunications system (UMTS), time divisional multiple access (TDMA), frequency division multiple access (FDMA), transmission control protocol-internet protocol (TCP -IP), short messaging service (SMS), multimedia messaging service (MMS), email, instant messaging service (IMS), Bluetooth, IEEE 802.11 and any similar wireless communication technology. A communications device involved in implementing various embodiments of the present invention may communicate using various media including, but not limited to, radio, infrared, laser, cable connections, and any suitable connection.

[0149] Figure 14 is a graphical representation of an example multimedia communication system within which various embodiments may be implemented. A data source 1510 provides a source signal in an analog, uncompressed digital, or compressed digital format, or any combination of these formats. An encoder 1520 may include or be connected with a pre-processing, such as data format conversion and / or filtering of the source signal. The encoder 1520 encodes the source signal into a coded media bitstream. It should be noted that a bitstream to be decoded may be received directly or indirectly from a remote device located within virtually any type of network. Additionally, the bitstream may be received from local hardware or software. The encoder 1520 may be capable of encoding more than one media type, such as audio and video, or more than one encoder 1520 may be required to code different media types of the source signal. The encoder 1520 may also get synthetically produced input, such as graphics and text, or it may be capable of producing coded bitstreams of synthetic media. In the following, only processing of one coded media bitstream of one media type is considered to simplify the description. It should be noted,however, that typically real-time broadcast services comprise several streams (typically at least one audio, video and text sub-titling stream). It should also be noted that the system may include many encoders, but in the figure only one encoder 1520 is represented to simplify the description without a lack of generality. It should be further understood that, although text and examples contained herein may specifically describe an encoding process, one skilled in the art would understand that the same concepts and principles also apply to the corresponding decoding process and vice versa.

[0150] The coded media bitstream may be transferred to a storage 1530. The storage 1530 may comprise any type of mass memory to store the coded media bitstream. The format of the coded media bitstream in the storage 1530 may be an elementary self-contained bitstream format, or one or more coded media bitstreams may be encapsulated into a container file, or the coded media bitstream may be encapsulated into a Segment format suitable for DASH (or a similar streaming system) and stored as a sequence of Segments. If one or more media bitstreams are encapsulated in a container file, a file generator (not shown in the figure) may be used to store the one more media bitstreams in the file and create file format metadata, which may also be stored in the file. The encoder 1520 or the storage 1530 may comprise the file generator, or the file generator is operationally attached to either the encoder 1520 or the storage 1530. Some systems operate “live”, i.e. omit storage and transfer coded media bitstream from the encoder 1520 directly to the sender 1540. The coded media bitstream may then be transferred to the sender 1540, also referred to as the server, on a need basis. The format used in the transmission may be an elementary self-contained bitstream format, a packet stream format, a Segment format suitable for DASH (or a similar streaming system), or one or more coded media bitstreams may be encapsulated into a container file. The encoder 1520, the storage 1530, and the server 1540 may reside in the same physical device or they may be included in separate devices. The encoder 1520 and server 1540 may operate with live real-time content, in which case the coded media bitstream is typically not stored permanently, but rather buffered for small periods of time in the content encoder 1520 and / or in the server 1540 to smooth out variations in processing delay, transfer delay, and coded media bitrate.

[0151] The server 1540 sends the coded media bitstream using a communication protocol stack. The stack may include but is not limited to one or more of Real-Time Transport Protocol (RTP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), Transmission Control Protocol (TCP), and Internet Protocol (IP). When the communication protocol stack is packet-oriented, the server 1540 encapsulates the coded media bitstream into packets. For example, when RTP is used, the server 1540 encapsulates the coded media bitstream into RTP packets according to an RTP payload format. Typically, each media type has a dedicated RTP payload format. It should be againnoted that a system may contain more than one server 1540, but for the sake of simplicity, the following description only considers one server 1540.

[0152] If the media content is encapsulated in a container file for the storage 1530 or for inputting the data to the sender 1540, the sender 1540 may comprise or be operationally attached to a “sending file parser” (not shown in the figure). In particular, if the container file is not transmitted as such but at least one of the contained coded media bitstream is encapsulated for transport over a communication protocol, a sending file parser locates appropriate parts of the coded media bitstream to be conveyed over the communication protocol. The sending file parser may also help in creating the correct format for the communication protocol, such as packet headers and payloads. The multimedia container file may contain encapsulation instructions, such as hint tracks in the ISOBMFF, for encapsulation of the at least one of the contained media bitstream on the communication protocol.

[0153] The server 1540 may or may not be connected to a gateway 1550 through a communication network, which may e.g. be a combination of a CDN, the Internet and / or one or more access networks. The gateway may also or alternatively be referred to as a middle-box. For DASH, the gateway may be an edge server (of a CDN) or a web proxy. It is noted that the system may generally comprise any number gateways or alike, but for the sake of simplicity, the following description only considers one gateway 1550. The gateway 1550 may perform different types of functions, such as translation of a packet stream according to one communication protocol stack to another communication protocol stack, merging and forking of data streams, and manipulation of data stream according to the downlink and / or receiver capabilities, such as controlling the bit rate of the forwarded stream according to prevailing downlink network conditions. The gateway 1550 may be a server entity in various embodiments.

[0154] The system includes one or more receivers 1560, typically capable of receiving, de-modulating, and de-capsulating the transmitted signal into a coded media bitstream. The coded media bitstream may be transferred to a recording storage 1570. The recording storage 1570 may comprise any type of mass memory to store the coded media bitstream. The recording storage 1570 may alternatively or additively comprise computation memory, such as random-access memory. The format of the coded media bitstream in the recording storage 2170 may be an elementary self-contained bitstream format, or one or more coded media bitstreams may be encapsulated into a container file. If there are multiple coded media bitstreams, such as an audio stream and a video stream, associated with each other, a container file is typically used and the receiver 1560 comprises or is attached to a container file generator producing a container file from input streams. Some systems operate “live,” i.e. omit the recording storage 1570 and transfer coded media bitstream from the receiver 1560directly to the decoder 1580. In some systems, only the most recent part of the recorded stream, e.g., the most recent 10-minute excerption of the recorded stream, is maintained in the recording storage 1570, while any earlier recorded data is discarded from the recording storage 1570.

[0155] The coded media bitstream may be transferred from the recording storage 1570 to the decoder 1580. If there are many coded media bitstreams, such as an audio stream and a video stream, associated with each other and encapsulated into a container file or a single media bitstream is encapsulated in a container file e.g. for easier access, a file parser (not shown in the figure) is used to decapsulate each coded media bitstream from the container file. The recording storage 1570 or a decoder 1580 may comprise the file parser, or the file parser is attached to either recording storage 1570 or the decoder 1580. It should also be noted that the system may include many decoders, but here only one decoder 1580 is discussed to simplify the description without a lack of generality.

[0156] The coded media bitstream may be processed further by a decoder 1580, whose output is one or more uncompressed media streams. Finally, a Tenderer 1590 may reproduce the uncompressed media streams with a loudspeaker or a display, for example. The receiver 1560, recording storage 1570, decoder 1580, and Tenderer 1590 may reside in the same physical device or they may be included in separate devices.

[0157] A sender 1540 and / or a gateway 1550 may be configured to perform switching between different representations e.g. for switching between different viewports of 360-degree video content, view switching, bitrate adaptation and / or fast start-up, and / or a sender 1540 and / or a gateway 1550 may be configured to select the transmitted representation(s). Switching between different representations may take place for multiple reasons, such as to respond to requests of the receiver 1560 or prevailing conditions, such as throughput, of the network over which the bitstream is conveyed. In other words, the receiver 1560 may initiate switching between representations. A request from the receiver can be, e.g., a request for a Segment or a Subsegment from a different representation than earlier, a request for a change of transmitted scalability layers and / or sub-layers, or a change of a rendering device having different capabilities compared to the previous one. A request for a Segment may be an HTTP GET request. A request for a Subsegment may be an HTTP GET request with a byte range. Additionally, or alternatively, bitrate adjustment or bitrate adaptation may be used for example for providing so-called fast start-up in streaming services, where the bitrate of the transmitted stream is lower than the channel bitrate after starting or randomaccessing the streaming in order to start playback immediately and to achieve a buffer occupancy level that tolerates occasional packet delays and / or retransmissions. Bitrate adaptation may include multiple representation or layer up-switching and representation or layer down-switching operations taking place in various orders.

[0158] In telecommunications and data networks, a channel may refer either to a physical channel or to a logical channel. A physical channel may refer to a physical transmission medium such as a wire, whereas a logical channel may refer to a logical connection over a multiplexed medium, capable of conveying several logical channels. A channel may be used for conveying an information signal, for example a bitstream, from one or several senders (or transmitters) to one or several receivers.

[0159] A decoder 1580 may be configured to perform switching between different representations e.g. for switching between different viewports of 360-degree video content, view switching, bitrate adaptation and / or fast start-up, and / or a decoder 1580 may be configured to select the transmitted representation(s). Switching between different representations may take place for multiple reasons, such as to achieve faster decoding operation or to adapt the transmitted bitstream, e.g. in terms of bitrate, to prevailing conditions, such as throughput, of the network over which the bitstream is conveyed. Faster decoding operation might be needed for example if the device including the decoder 1580 is multi-tasking and uses computing resources for other purposes than decoding the video bitstream In another example, faster decoding operation might be needed when content is played back at a faster pace than the normal playback speed, e g. twice or three times faster than conventional real-time playback rate.

[0160] In the above, some embodiments have been described with reference to and / or using terminology of HEVC and / or VVC. It needs to be understood that embodiments may be similarly realized with any video encoder and / or video decoder.

[0161] In the above, where the example embodiments have been described with reference to an encoder, it needs to be understood that the resulting bitstream and the decoder may have corresponding elements in them. Likewise, where the example embodiments have been described with reference to a decoder, it needs to be understood that the encoder may have structure and / or computer program for generating the bitstream to be decoded by the decoder.

[0162] The embodiments of the invention described above describe the codec in terms of separate encoder and decoder apparatus in order to assist the understanding of the processes involved. However, it would be appreciated that the apparatus, structures and operations may be implemented as a single encoder-decoder apparatus / structure / operation. Furthermore, it is possible that the coder and decoder may share some or all common elements.

[0163] Although the above examples describe embodiments of the invention operating within a codec within an electronic device, it would be appreciated that the invention as defined in the claims may be implemented as part of any video codec. Thus, for example, embodiments of the invention maybe implemented in a video codec which may implement video coding over fixed or wired communication paths.

[0164] Thus, user equipment may comprise a video codec such as those described in embodiments of the invention above. It shall be appreciated that the term user equipment is intended to cover any suitable type of wireless user equipment, such as mobile telephones, portable data processing devices or portable web browsers.

[0165] Furthermore, elements of a public land mobile network (PLMN) may also comprise video codecs as described above.

[0166] In general, the various embodiments of the invention may be implemented in hardware or special purpose circuits, software, logic or any combination thereof. For example, some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software which may be executed by a controller, microprocessor or other computing device, although the invention is not limited thereto. While various aspects of the invention may be illustrated and described as block diagrams, flow charts, or using some other pictorial representation, it is well understood that these blocks, apparatus, systems, techniques or methods described herein may be implemented in, as non-limiting examples, hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof.

[0167] The embodiments of this invention may be implemented by computer software executable by a data processor of the mobile device, such as in the processor entity, or by hardware, or by a combination of software and hardware. Further in this regard it should be noted that any blocks of the logic flow as in the Figures may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software may be stored on such physical media as memory chips, or memory blocks implemented within the processor, magnetic media such as hard disk or floppy disks, and optical media such as for example DVD and the data variants thereof, CD.

[0168] The memory may be of any type suitable to the local technical environment and may be implemented using any suitable data storage technology, such as semiconductor-based memory devices, magnetic memory devices and systems, optical memory devices and systems, fixed memory and removable memory. The data processors may be of any type suitable to the local technical environment, and may include one or more of general-purpose computers, special purpose computers, microprocessors, digital signal processors (DSPs) and processors based on multi-core processor architecture, as non-limiting examples.

[0169] Embodiments of the inventions may be practiced in various components such as integrated circuit modules. The design of integrated circuits is by and large a highly automated process. Complex and powerful software tools are available for converting a logic level design into a semiconductor circuit design ready to be etched and formed on a semiconductor substrate.

[0170] Programs, such as those provided by Synopsys, Inc. of Mountain View, California and Cadence Design, of San Jose, California automatically route conductors and locate components on a semiconductor chip using well established rules of design as well as libraries of pre-stored design modules. Once the design for a semiconductor circuit has been completed, the resultant design, in a standardized electronic format (e g., Opus, GDSII, or the like) may be transmitted to a semiconductor fabrication facility or “fab” for fabrication.

[0171] The various embodiments can be implemented with the help of computer program code that resides in a memory and causes the relevant apparatuses to carry out the method. For example, a device may comprise circuitry and electronics for handling, receiving, and transmitting data, computer program code in a memory, and a processor that, when running the computer program code, causes the device to carry out the features of an embodiment. Yet further, a network device like a server may comprise circuitry and electronics for handling, receiving, and transmitting data, computer program code in a memory, and a processor that, when running the computer program code, causes the network device to carry out the features of various embodiments.

[0172] If desired, the different functions discussed herein may be performed in a different order and / or concurrently with other. Furthermore, if desired, one or more of the above-described functions and embodiments may be optional or may be combined.

[0173] Although various aspects of the embodiments are set out in the independent claims, other aspects comprise other combinations of features from the described embodiments and / or the dependent claims with the features of the independent claims, and not solely the combinations explicitly set out in the claims.

[0174] It is also noted herein that while the above describes example embodiments, these descriptions should not be viewed in a limiting sense. Rather, there are several variations and modifications, which may be made without departing from the scope of the present disclosure as, defined in the appended claims.

Claims

CLAIMS:

1. An apparatus for decoding encoded samples of blocks of video sample, wherein for a block of video sample data the apparatus comprises- means for determining an intra-prediction information, where the means for determining compriseso means for extracting prediction information from a template sample block, o means for generating two or more representations of at least some samples in the template sample block by filtering the samples with different frequency channels; o means for determining intra-prediction modes for each frequency channels of the template sample block; ando means for computing intra-predicted samples for the current block based on the detected intra-prediction modes and the two or more representations of at least some template samples.

2. The apparatus according to claim 1, wherein the two or more representations of samples comprises at least low-pass filtered representation and high-pass filtered representation.

3. The apparatus according to claim 2, wherein sample values of the high-pass filtered representation is determined by deducting sample values of the low-pass filtered representation from sample values of the template sample block.

4. The apparatus according to claim 1, wherein the means for computing intra-predicted samples comprises means for determining an initial block of predicted samples for each frequency channel and means for blending initial blocks with predicted samples to form the final predicted block of intra-predicted samples.

5. The apparatus according to claim 4, wherein the means for determining an initial block of predicted samples for each frequency channel comprises means for assigning a different weight for different frequency channels.

6. A method for decoding encoded samples of blocks of video sample, wherein for a block of video sample data the method comprises- determining an intra-prediction information, where the determining compriseso extracting prediction information from a template sample block, o generating two or more representations of at least some samples in the template sample block by filtering the samples with different frequency channels;o determining intra-prediction modes for each frequency channels of the template sample block; ando computing intra-predicted samples for the current block based on the detected intraprediction modes and the two or more representations of at least some template samples.

7. The method according to claim 6, wherein the two or more representations of samples comprises at least low-pass filtered representation and high-pass filtered representation.

8. The method according to claim 7, wherein sample values of the high-pass filtered representation is determined by deducting sample values of the low-pass filtered representation from sample values of the template sample block.

9. The method according to claim 6, wherein computing intra-predicted samples comprises determining an initial block of predicted samples for each frequency channel and blending initial blocks with predicted samples to form the final predicted block of intra-predicted samples.

10. The method according to claim 9, wherein determining an initial block of predicted samples for each frequency channel comprises assigning a different weight for different frequency channels.

11. An apparatus for decoding encoded samples of blocks of video sample, the apparatus comprising at least one processor, memory including computer program code, wherein for a block of video sample data the memory and the computer program code configured to, with the at least one processor, cause the apparatus to- determine an intra-prediction information, where the means for determining comprises o extracting prediction information from a template sample block,o generating two or more representations of at least some samples in the template sample block by filtering the samples with different frequency channels;o determining intra-prediction modes for each frequency channels of the template sample block; ando computing intra-predicted samples for the current block based on the detected intraprediction modes and the two or more representations of at least some template samples.

12. The apparatus according to claim 11, wherein the two or more representations of samples comprises at least low-pass filtered representation and high-pass filtered representation.

13. The apparatus according to claim 12, wherein sample values of the high-pass filtered representation is determined by deducting sample values of the low-pass filtered representation from sample values of the template sample block.

14. The apparatus according to claim 11, wherein computing intra-predicted samples comprises determining an initial block of predicted samples for each frequency channel and means for blending initial blocks with predicted samples to form the final predicted block of intra-predicted samples.

15. The apparatus according to claim 14, wherein determining an initial block of predicted samples for each frequency channel comprises assigning a different weight for different frequency channels.

Images