Method and apparatus for video decoding

Abstract

A method and apparatus for decoding encoded samples of a block of video sample data are provided, wherein for a block of video sample data, the method comprises deriving information to configure a template-based intra prediction mode derivation process, wherein the configured template-based intra prediction mode derivation process comprises calculating a distortion over a template region based on a distortion metric to determine one or more intra prediction modes from an intra prediction mode candidate list, determining one or more intra mode predictors by the determined one or more intra prediction modes, and calculating a prediction for a sample of the video sample data based on the intra mode predictors.

Description

Technical Field

[0001] This solution generally relates to a method, apparatus, and computer program product for video encoding and decoding. Background Technology

[0002] This section is intended to provide background or context for the invention as described in the claims. The description herein may include concepts that may be pursued but are not necessarily previously conceived or pursued. Therefore, unless otherwise indicated herein, what is described in this section is not prior art to the specification and claims of this application and is not admitted as prior art by virtue of its inclusion in this section.

[0003] Template-based intra-mode derivation (TIMD) is a technique on the decoder side that derives intra-predictive modes by selecting the mode with the least distortion from a candidate list of intra-predictive modes, where distortion is calculated using templates of the blocks. The candidate list can consist of a list of most probable modes (MPMs) extracted from neighboring blocks. Summary of the Invention

[0004] The purpose of this solution is to provide an improved TIMD process.

[0005] The scope of protection sought with respect to the various embodiments of the invention is defined by the independent claims. Embodiments and features (if any) described in this specification that do not fall within the scope of the independent claims should be interpreted as examples that aid in understanding the various embodiments of the invention.

[0006] Various aspects include methods, apparatus, and computer-readable media including computer programs stored therein, characterized by the content described in the independent claims. Various embodiments are disclosed in the dependent claims.

[0007] According to a first aspect, an apparatus is provided for decoding encoded samples of a block of video sample data, wherein for the block of video sample data, the apparatus includes components for deriving information to configure a template-based intra-prediction mode derivation process, wherein, in order to implement the configured template-based intra-prediction mode derivation process, the apparatus further includes: components for calculating distortion on a template region based on a distortion metric to determine one or more intra-prediction modes from a candidate list of intra-prediction modes; components for determining one or more intra-prediction mode predictors based on the determined one or more intra-prediction modes; and components for calculating predictions for the samples of the video sample data based on the intra-prediction mode predictors.

[0008] According to a second aspect, a method is provided for decoding encoded samples of a block of video sample data, wherein for the block of video sample data, the method includes derivation information to configure a template-based intra-prediction mode derivation process, wherein the configured template-based intra-prediction mode derivation process includes: calculating distortion on a template region based on a distortion metric to determine one or more intra-prediction modes from a candidate list of intra-prediction modes; determining one or more intra-prediction mode predictors using the determined one or more intra-prediction modes; and calculating predictions for samples of the video sample data based on the intra-prediction mode predictors.

[0009] According to a third aspect, an apparatus is provided for decoding encoded samples of a block of video sample data. The apparatus includes at least one processor and a memory including computer program code configured, together with the at least one processor, to cause the apparatus to derive information to configure a template-based intra-prediction mode derivation process. To implement the configured template-based intra-prediction mode derivation process, the apparatus is further configured to: calculate distortion on a template region based on a distortion metric to determine one or more intra-prediction modes from a candidate list of intra-prediction modes; determine one or more intra-prediction mode predictors using the determined one or more intra-prediction modes; and calculate predictions for samples of the video sample data based on the intra-prediction mode predictors.

[0010] According to a fourth aspect, a computer program product is provided for decoding encoded samples of blocks of video sample data. The computer program product includes computer program code configured to, when executed on at least one processor, cause a means or system to derive information to configure a template-based intra-prediction mode derivation process. To implement the configured template-based intra-prediction mode derivation process, the means is further configured to: calculate distortion on a template region based on a distortion metric to determine one or more intra-prediction modes from a candidate list of intra-prediction modes; determine one or more intra-prediction mode predictors using the determined one or more intra-prediction modes; and calculate predictions for samples of the video sample data based on the intra-prediction mode predictors.

[0011] According to one embodiment, the derived information is derived from the encoded bitstream.

[0012] According to one embodiment, the derived information is used to determine a distortion measure among multiple distortion measures.

[0013] According to one embodiment, the distortion measure is one of the following: the sum of absolute differences, the sum of absolute transformation differences, the sum of squared errors, the sum of absolute differences after removing the mean, the sum of absolute transformation differences after removing the mean, and the sum of squared errors after removing the mean.

[0014] According to one embodiment, the derived information is used to determine a set of reconstructed samples to form a template region.

[0015] According to one embodiment, the derived information is used to determine which of the at least two intra-prediction mode candidate lists is used.

[0016] According to one embodiment, a computer program product is embodied on a non-transitory computer-readable medium. Attached Figure Description

[0017] Various embodiments will now be described in more detail with reference to the accompanying drawings, in which:

[0018] Figure 1 A simplified example of the encoding process is shown;

[0019] Figure 2 A simplified example of the decoding process is shown;

[0020] Figure 3a , Figure 3b A simplified example of the template and its reference sample is shown;

[0021] Figure 4 This is a flowchart illustrating a method according to one embodiment; and

[0022] Figure 5 An apparatus according to one embodiment is shown. Detailed Implementation

[0023] Some abbreviations that may appear in this specification are defined as follows: 2D 3D AMVP Advanced Motion Vector Prediction AV1 AO Media Video 1 AVC (Advanced Video Coding) CABAC Context Adaptive Binary Arithmetic Coding CDMA Code Division Multiple Access CTU coding tree unit CU encoding unit DC Discrete Cosine Transform (DCT) Intra-frame mode derivation on the DIMD decoder side DPB decoder image buffer DSP Digital Signal Processor ECM Enhanced Compression Model FDMA (Frequency Division Multiple Access) GSM Global Mobile Communication System HEVC High Efficiency Video Coding HoG gradient histogram IEC International Electrotechnical Commission IMS Instant Messaging Service IPM Intra-Prediction Mode ISO International Organization for Standardization ISOBMFF ISO Basic Media File Format ITU-T Telecommunication Standardization Sector JCT-VC Video Coding Joint Collaboration Team JVT Joint Video Team LCU Maximum Coding Unit MCP Motion Compensation Prediction MHoG merged gradient histogram Derivation of MIMD merged intra-frame mode MMS Multimedia Messaging Service MPEG Moving Picture Experts Group MPG Most Probable Mode MR - Mean Removal MRL Multiple Reference Lines MVC (Multi-View Video Encoding) MV Multiview NAL Network Abstraction Layer PC Personal Computer PU prediction unit REXT range extension SAD (Sum of Absolute Differences) Sum of Absolute Transform Differences (SATD) SHVC (Scalable High-Efficiency Video Encoding) SMS Short Message Service SNR (Signal-to-Noise Ratio) SPGM Single Path Global Modulation Sum of squared errors of SSE SVC (Scalable Video Coding) TDMA (Time Division Multiple Access) TCP / IP Transmission Control Protocol – Internet Protocol TIMD Template-Based Inter-Frame Mode Derivation TU Transformer TMRL Template-Based Multi-Reference Lines TV UMTS (Universal Mobile Telecommunications System) VCEG (Video Coding Experts Group) VVC Universal Video Coding

[0024] The following description and accompanying drawings are illustrative, intended to discuss embodiments of the present solution by way of example. Specific details are provided for purposes of understanding. However, in some cases, well-known or conventional details have not been described to avoid obscuring the description. References to “an embodiment” or “an embodiment” in this specification mean that a particular feature, structure, or characteristic described in connection with that embodiment is included in at least one embodiment of this disclosure.

[0025] This embodiment relates to adaptive TIMD.

[0026] Before further describing the embodiments, a brief mention is made of the evolution of video coding standards. This embodiment applies to the context of next-generation video coding standards, such as the H.267 video coding standard and ECM (Enhanced Compression Model) exploration.

[0027] A video codec consists of an encoder and a decoder. The encoder converts the input video into a compressed representation suitable for storage / transmission. The decoder decompresses the compressed video representation back into a viewable form. The encoder can discard some information from the original video sequence to represent the video in a more compact form (i.e., at a lower bit rate).

[0028] Figure 1 A simple example of the principles of the encoding process for 2D images is shown, and Figure 2 A simple example illustrating the principle of the decoding process for 2D images is shown. Figure 1 The following content is illustrated in the image: - The image to be encoded (I n ); - Predicted representation of image patches (P' n ); - Prediction error signal (D n ); - Reconstructed prediction error signal (D') n ); - Preliminary reconstructed image (I' n ); - Final reconstructed image (R' n ); - Transform (T) and inverse transform (T) -1 ); - Quantization (Q) and inverse quantization (Q) -1 ); - Entropy coding (E); - Reference Frame Memory (RFM); - Inter-frame prediction (P inter ); - Intra-frame prediction (P intra ); - Mode selection (MS); and - Filtering (F).

[0029] exist Figure 2 The following content is illustrated in the image: - Predicted representation of image patches (P' n ); - Reconstructed prediction error signal (D') n ); - Preliminary reconstructed image (I' n ); - Final reconstructed image (R' n ); inverse transform (T) -1 ); - Inverse quantization (Q -1 ); - Entropy Decoding (E -1 ); - Reference Frame Memory (RFM); - Prediction (inter-frame or intra-frame) (P); - and filtering (F).

[0030] In most cases, the basic unit used for the input of the encoder and the output of the decoder is an image (also called a "picture"). The image input to the encoder can also be called the source image, and the image decoded by the decoder can be called the decoded image or the reconstructed image.

[0031] The source image and the decoded image each consist of one or more sample arrays, such as one of the following sets of sample arrays: - Luminance (Y) only (monochrome). - Brightness and two chromaticities (YCbCr or YCgCo). - Green, blue, and red (GBR, also known as RGB). - Represents an array of other unspecified monochromatic or tristimulus color samples (e.g., YZX, also known as XYZ).

[0032] An image can be defined as a frame or a field. A frame consists of a matrix of luminance samples and possible corresponding chrominance samples. A field is a set of alternating sample rows of frames and can be used as encoder input when the source signal is interlaced. The chrominance sample array may not exist (and therefore monochrome sampling may be used), or the chrominance sample array may be subsampled compared to the luminance sample array.

[0033] The Advanced Video Coding Standard (AVC or H.264 / AVC) was developed by the Video Coding Experts Group (VCEG) of the Telecommunication Standardization Sector of the International Telecommunication Union (ITU-T) and the Joint Video Team (JVT) of the Moving Picture Experts Group (MPEG) of the International Organization for Standardization (ISO) / International Electrotechnical Commission (IEC). Several versions of the H.264 / AVC standard exist, each integrating several extensions or features into the specification. These extensions include Scalable Video Coding (SVC) and Multi-View Video Coding (MVC).

[0034] The High Efficiency Video Coding Standard (which can be abbreviated as HEVC or H.265 / HEVC) was developed by the Joint Collaboration Team for Video Coding of VCEG and MPEG (JCT-VC). Extensions to H.265 / HEVC include Scalable, Multi-View, 3D, and Fidelity Range extensions, which can be referred to as SHVC, MV-HEVC, 3D-HEVC, and REXT, respectively.

[0035] Universal Video Coding (which can be abbreviated as VVC, H.266, or H.266 / VVC) is a video compression standard developed as a successor to HEVC. VVC is specified in ITU-T Recommendation H.266 and is equivalently specified in ISO / IEC 23090-3, also known as MPEG-I Part 3.

[0036] ECM was developed by ITU-T VCEG and ISO / IEC MPEG's JVET (Joint Video Experts Group) to provide compression capabilities that will surpass those of VVC, a future video coding technology.

[0037] This section describes some key definitions, bitstreams, and encoding structures and concepts of H.264 / AVC, HEVC, VVC, and / or AV1 and some of their extensions, as examples of video encoders, decoders, encoding methods, decoding methods, and bitstream structures on which this embodiment can be implemented. Aspects of the various embodiments are not limited to H.264 / AVC, HEVC, VVC, and / or AV1 or their extensions, but are given as a possible basis on which this embodiment can be partially or completely implemented.

[0038] Hybrid video codecs (such as ITU-T H.263, H.264 / AVC, HEVC, and VVC) can encode video information in two stages. First, pixel values ​​in specific image regions (or “blocks”) are predicted, for example, through motion compensation or spatial means.

[0039] In motion-compensated prediction (also known as inter-frame prediction, temporal prediction, motion-compensated temporal prediction, or motion-compensated prediction or MCP), a region closely corresponding to the block being encoded is found in one of the previously encoded frames and used for prediction. Inter-frame prediction can reduce temporal redundancy.

[0040] In spatial prediction, pixel values ​​surrounding the block to be encoded are used. In the first stage, predictive coding, such as so-called sample prediction and / or so-called syntax prediction, can be applied. In sample prediction, pixel values ​​or sample values ​​are predicted within a specific image region or "block." For example, one or more of motion compensation or intra-frame prediction mechanisms can be used to predict these pixel values ​​or sample values.

[0041] Intra-frame prediction (where pixel values ​​or sample values ​​can be predicted via spatial mechanisms) involves finding and indicating spatial region relationships. Intra-frame prediction leverages the fact that neighboring pixels within the same image may be correlated. Intra-frame prediction can be performed in the spatial domain or the transform domain; that is, sample values ​​or transform coefficients can be predicted. Intra-frame prediction is typically used in intra-frame coding, where inter-frame prediction is not applied.

[0042] In motion vector prediction, motion vectors, such as those used for inter-frame and / or inter-view prediction, can be differentially encoded relative to block-specific predicted motion vectors. In many video codecs, predicted motion vectors are created in a predefined manner, such as by calculating the median of the encoded or decoded motion vectors of neighboring blocks. Another way to create motion vector predictions, sometimes called Advanced Motion Vector Prediction (AMVP), is to generate a list of candidate predictions from co-located blocks in neighboring blocks and / or a time-referenced image, and to signal the selected candidates to the motion vector predictor. In addition to predicting motion vector values, reference indices from previously encoded / decoded images can also be predicted. Reference indices are typically predicted from co-located blocks in neighboring blocks and / or a time-referenced image. Differential encoding of motion vectors is typically disabled across stripe boundaries.

[0043] Block partitioning, such as from coding tree unit (CTU) to coding unit (CU) and down to prediction unit (PU), can be predicted.

[0044] In filter parameter prediction, filter parameters, for example, used for sample adaptive offset, can be predicted. Prediction methods that use image information from previously encoded images can also be called inter-frame prediction methods, or temporal prediction and motion compensation methods. Prediction methods that use image information within the same image can also be called intra-frame prediction methods.

[0045] In the second stage of encoding, the prediction error (i.e., the difference between the predicted pixel block and the original pixel block) is encoded. This can be done by transforming the difference in pixel values ​​using a specified transform (e.g., Discrete Cosine Transform (DCT) or a variant thereof), quantizing the coefficients, and entropy encoding the quantized coefficients. By varying the fidelity of the quantization process, the encoder can control the balance between the accuracy of the pixel representation (image quality) and the size of the resulting encoded video representation (file size at the transmission bitrate).

[0046] In some video codecs (such as H.265 / HEVC), video images are divided into coding units (CUs) covering image regions. A CU consists of one or more prediction units (PUs) defining a prediction process for samples within the CU, and one or more transform units (TUs) defining a prediction error coding process for samples within the CU. A CU can consist of square blocks of samples with sizes selectable from a predefined set of possible CU sizes. CUs with the largest allowed size are typically named LCUs (Maximum Coding Units) or CTUs (Coding Tree Units), and video images are divided into non-overlapping CTUs. CTUs can be further subdivided into combinations of smaller CUs, for example, by recursively subdividing CTUs and resulting CUs. Each resulting CU typically has at least one PU and at least one TU associated with it. Each PU and TU can be further subdivided into smaller PUs and TUs accordingly to increase the granularity of the prediction process and the prediction error coding process. Each PU has associated prediction information that defines what prediction to apply to the pixels within that PU (e.g., motion vector information for inter-frame prediction PUs and intra-frame prediction directionality information for intra-frame prediction PUs). Similarly, each TU is associated with information describing the prediction error decoding process for samples within that TU (including, for example, DCT coefficient information). Typically, at the CU level, it is signaled whether prediction error coding is applied to each CU. In the absence of a prediction error residual associated with a CU, the CU can be considered to have no TU. The partitioning of the image to CUs, and the partitioning of CUs to PUs and TUs, is typically signaled in the bitstream, allowing the decoder to reproduce the intended structure of these units.

[0047] The decoder reconstructs the output video by applying prediction techniques similar to those of the encoder, forming a predicted representation of pixel blocks (using motion or spatial information created by the encoder and stored in the compressed representation) and prediction error decoding (the inverse operation of prediction error encoding, restoring the quantized prediction error signal to the spatial pixel domain). After applying the prediction and prediction error decoding techniques, the decoder adds the prediction and prediction error signals (pixel values) to form the output video frame. The decoder (and encoder) may also apply additional filtering techniques to improve the quality of the output video before passing it on for display and / or storing it as a prediction reference for subsequent frames in a video sequence.

[0048] In many video codecs, including H.264 / AVC, HEVC, and VVC, motion information is indicated by motion vectors associated with each motion-compensated image block. Each of these motion vectors represents the displacement of the picture block in the image to be encoded (in the encoder) or decoded (at the decoder) relative to a predicted source block in one of the previously encoded or decoded pictures (or images). To efficiently represent motion vectors, they are typically differentially encoded relative to block-specific predicted motion vectors. In typical video codecs, predicted motion vectors are created in a predefined manner, such as by calculating the median of the encoded or decoded motion vectors of neighboring blocks. Another way to create motion vector predictions is to generate a list of candidate predictions from co-located blocks in neighboring blocks and / or a time-referenced image, and to signal the selected candidates to the motion vector predictor. In addition to predicting motion vector values, reference indices of previously encoded / decoded images can also be predicted. Reference indices can be predicted from co-located blocks in neighboring blocks and / or a time-referenced image. Furthermore, efficient video codecs can employ additional motion information encoding / decoding mechanisms (often referred to as merging / merging modes), where all motion field information (including motion vectors and corresponding reference image indices for each available list of reference images) is predicted and used without any modification / correction. Similarly, motion field information prediction is performed using motion field information from neighboring blocks and / or co-located blocks in temporal reference images, and the motion field information used is signaled in a motion field candidate list populated with motion field information from available neighboring / co-located blocks.

[0049] The video codec can support motion-compensated prediction from a single source image (one-way prediction) and motion-compensated prediction from two source images (two-way prediction). In the case of one-way prediction, a single motion vector is applied, while in the case of two-way prediction, two motion vectors are signaled, and the motion-compensated predictions from the two sources are averaged to create the final sample prediction. In the case of weighted prediction, the relative weights of the two predictions can be adjusted, or the offset signaled by the signal can be added to the prediction signal.

[0050] Besides applying motion compensation to inter-image predictions, similar methods can be applied to intra-image predictions. In this case, the displacement vector indicates where sample blocks can be copied from the same image to form the prediction of the block to be encoded or decoded. This intra-frame block copying method can significantly improve coding efficiency when there are repeating structures (such as text or other graphics) within the frame.

[0051] In video codecs, the prediction residuals after motion compensation or intra-frame prediction can first be transformed using a transform kernel (such as DCT) before being encoded. This is done because some correlation often remains between the residuals, and in many cases, the transform can help reduce this correlation and provide more efficient coding.

[0052] Many video encoders utilize a Lagrange cost function to find the optimal coding pattern, such as the desired macroblock pattern and associated motion vectors. This cost function uses a weighting factor λ to correlate the (exact or estimated) image distortion resulting from lossy coding methods with the (exact or estimated) amount of information needed to represent the pixel values ​​in the image region: C = D + λR (Equation 1)

[0053] Where C is the Lagrange cost to be minimized, D is the image distortion (e.g., mean square error) under the considered mode and motion vectors, and R is the number of bits required to reconstruct the image patch in the decoder (including the amount of data representing the candidate motion vectors).

[0054] The term "along the bitstream" (e.g., "along the bitstream indication") can be defined as referring to out-of-band transmission, signaling, or storage in a manner that associates out-of-band data with the bitstream. The term "along the bitstream decoding" or similar terminology can refer to decoding the mentioned out-of-band data associated with the bitstream (which may be obtained from out-of-band transmission, signaling, or storage). For example, an "along the bitstream indication" can refer to metadata in a container file encapsulating the bitstream.

[0055] Existing video codecs offer a variety of available intra-prediction modes. These include different directional intra-prediction modes, as well as prediction modes such as DC or Planar intra-prediction.

[0056] In Planar intra-frame prediction, interpolation processes are performed in both the vertical and horizontal directions to obtain a vertical predictor and a horizontal predictor, respectively. The sample-by-sample mean of these two predictors can then be calculated. Each interpolation process can be performed by calculating a weighted average of reference samples. As an example, when calculating the vertical predictor, each predicted sample at a given horizontal coordinate is obtained as a weighted average between two reference samples at the same horizontal coordinate, extracted accordingly from the top and bottom rows of the current block. Since the samples located in the bottom row of the current block have not yet been reconstructed when predicting the current block, these samples are filled using the bottom-left sample of the current block. Similarly, the horizontal predictor is calculated as a weighted average of reference samples extracted from the columns to the left and right of the current block.

[0057] ECM provides improved coding tools, such as Template-Based Intra-Frame Mode Derivation (TIMD). TIMD relies on an inference process that operates on regions (i.e., templates) formed by already reconstructed samples, such as samples surrounding the current block. The intra-frame predicted mode of a block is derived at both the encoder and decoder using a template-based approach, rather than being signaled to the decoder. Figure 3a The illustration shows an example of template 310 for the current block 300 and a reference sample 320 for template 310. In TIMD, distortion is calculated between the predicted and reconstructed samples of the template. One or more intra-prediction modes with the minimum distortion are selected as TIMD modes from a list of intra-prediction mode candidates and used to compute the final intra-prediction for the current block.

[0058] Current versions of TIMD technology rely on MPM intra-mode list construction to form a candidate list of intra-predictive modes for performing intra-mode template search. Specifically, for a given block, a template formed from reconstructed samples is considered. Depending on the availability of reference samples, an L-shaped template (e.g., ...) is used around the current block. Figure 3b Template 350 in the middle (if adjacent reconstructed samples are fully available), or use a rectangular template (such as) on the left and / or top of the current block. Figure 3a Template 310 in the template (depending on the block size, different template sizes can be used). Then, the intra-modes present in the MPM list are searched one by one. Reconstructed samples available around the template are used as reference samples, with samples in the template used as targets. Subsequently, the distortion between the prediction and the reconstructed template samples is calculated using the SATD distortion metric. The intra-prediction mode with the minimum SATD is selected as the TIMD mode and used for intra-prediction of the current block.

[0059] TIMD, in its current form, relies on a template formed by reconstructed samples surrounding the current block. Instead of signaling the correct intra-mode for the current block, the decoder uses samples within the template as targets to search across different intra-modes and predicts inter-frame prediction directions. The reconstructed samples surrounding the template are used as reference samples. The decoder uses the reference samples to perform intra-frame predictions in different modes, then compares the resulting predictions with the target template and ultimately selects the intra-mode that minimizes a predefined distortion metric.

[0060] The TIMD method can successfully reduce the overhead required to signal a given intra-prediction mode, but the resulting predictions are often suboptimal because the template region may not be strongly correlated with the content in the current block. TIMD in its current form is also limited to using a distortion metric, a list of intra-prediction mode candidates, and a template size that depends only on the current block size.

[0061] Therefore, the purpose of this embodiment is to provide an improved version of TIMD.

[0062] According to the method of operation of the present invention, intra-frame prediction is applied to a block, wherein the intra-frame prediction process includes determining one or more intra-frame prediction modes to compute a predictor. Wherein the intra-frame prediction mode is determined based on the distortion in the template region calculated using a distortion metric; and where at least one of the following applies: Distortion metrics are determined based on information extracted from the bitstream and / or based on information inferred from the decoder; or The template region is formed by a set of reconstructed samples, which is determined based on information extracted from the bitstream and / or based on information inferred by the decoder; or Intra-prediction modes are extracted from an intra-prediction mode candidate list based on the distortion in the computed template region, wherein the candidate list is determined based on information extracted from the bitstream and / or based on information inferred by the decoder. Use of distortion measurement

[0063] As described above, the method according to this embodiment can be based on calculating the template region (such as) using a distortion metric. Figure 3b The distortion in template 350 is used to determine the intra-frame prediction mode, where the distortion metric is determined based on information extracted from the bitstream and / or information inferred by the decoder.

[0064] As an example, the distortion between the reconstructed samples in template region 350 and the intra-frame predicted samples can be calculated. As an example, several possible distortion measures can be considered to calculate the distortion. As an example, the distortion measure can be determined as SAD, SATD, or SSE.

[0065] As an example, when calculating distortion using a given distortion metric, the mean difference between the reconstructed sample in the template region and the intra-predicted sample can be used. As an example, when calculating the distortion of a given sample, the mean difference can be subtracted. As an example, the mean difference can be determined as MR-SAD, MR-SATD, or MR-SSE. As an example, the use of removing the mean difference can be determined by reading flags from the encoded bitstream.

[0066] According to one embodiment, multiple different distortion measures can be combined to obtain distortion. As an example, the sum of two distortions obtained by using two different distortion measures can be calculated. As an example, the sum of two or more distortions determined using different distortion measures can be used to derive intra-prediction modes on a template region.

[0067] For example, information indicating which distortion metric to use can be encoded into the bitstream. As another example, a flag indicating whether distortion is calculated using SAD, SATD, SSE, MR-SAD, MR-SATD, MR-SSE, or another metric can be encoded into the bitstream. As yet another example, an index used to determine a given distortion metric from a predetermined list of available distortion metrics can be encoded into the bitstream. The list of available distortion metrics can be calculated based on characteristics of the current block, such as its size. The decoder is configured to extract this information, flag, or index from the encoded bitstream.

[0068] As an example, the distortion metric to be used by the decoder can be determined based on information inferred by the decoder. As an example, the decoder can determine the distortion in a template region to determine the distortion metric. As an example, the distortion metric can be determined by measuring the distortion relative to a given, predetermined threshold. As an example, distortion can be determined for two or more distortion metrics, and a final distortion metric can be determined based on these distortions.

[0069] For each distortion metric, a set of intra-prediction modes with corresponding distortion can be determined, and then the final set of intra-prediction modes to be used to predict the current block can be determined based on the distortion among these different set of modes. Use of the determined set of reconstructed samples

[0070] As described above, the method according to this embodiment can determine the intra-frame prediction mode based on the distortion in the template region calculated using a distortion metric. The template region is formed by a determined set of reconstructed samples, wherein the decoder is configured to determine the set of reconstructed samples based on information from the bitstream. This means that the decoder is able to determine how the template should be created.

[0071] For example, to instruct the decoder on how to create a template, a flag may have been encoded into the bitstream indicating which template region to use between two pre-defined template regions. This flag can be extracted by the decoder. For example, the flag could indicate to the decoder that it should use a smaller or larger template. For instance, the flag could indicate that the decoder should use a smaller template (depending on the block size) formed by samples that have been reconstructed from 2 or 4 rows and / or columns, or a larger template (depending on the block size) formed by samples that have been reconstructed from 4 or 8 rows and / or columns.

[0072] Alternatively, the flag can indicate whether the number of samples in the template region should be equal to a predetermined number N, or whether the number of samples should be equal to a different number of samples M=2N. This flag can be extracted by the decoder.

[0073] According to one embodiment, the set of reconstructed samples in the template region can be determined based on information inferred by the decoder. The set of reconstructed samples in the template region can also be determined based on a distortion metric used to calculate distortion in the template region. As an example, if a distortion metric different from SATD is used, a larger number of samples in the template region can be used. Use of candidate list

[0074] As described above, the method according to this embodiment can determine the intra-prediction mode from the intra-prediction mode candidate list based on the distortion in the calculated template region.

[0075] Information about the candidate list can be encoded in a bitstream for extraction by a decoder.

[0076] For example, a flag might be encoded into the bitstream indicating whether a first candidate list or a second candidate list should be used. The second candidate list can be formed based on information extracted from the first candidate list. For instance, the second candidate list can be formed by calculating the distortion in the template region based on patterns extracted from the first candidate list. As another example, the second candidate list can be formed in such a way that patterns with minimal distortion in the first candidate list are excluded from the second candidate list. As yet another example, the second candidate list can be determined based on patterns with minimal distortion in the first candidate list.

[0077] In one embodiment, an index may have been encoded into the bitstream to indicate the method used to form the candidate list from among multiple available methods. Such an index can be extracted from the bitstream by a decoder.

[0078] According to one embodiment, the method in Figure 4The method is illustrated below. It is intended for decoding encoded samples of blocks of video sample data. For a block of video sample data, the method generally includes: deriving 410 information to configure a template-based intra-prediction mode derivation process, wherein the configured template-based intra-prediction mode derivation process includes: calculating 420 distortion on a template region based on a distortion metric to determine one or more intra-prediction modes from a candidate list of intra-prediction modes; determining 430 one or more intra-prediction mode predictors based on the determined one or more intra-prediction modes; and calculating 440 predictions for the samples of the video sample data based on the intra-prediction mode predictors. Each step can be implemented by a corresponding module of a computer system.

[0079] An apparatus is provided for decoding encoded samples of a block of video sample data. For the block of video sample data, the apparatus includes components for deriving information to configure a template-based intra-prediction mode derivation process. To implement the configured template-based intra-prediction mode derivation process, the apparatus further includes: components for calculating distortion on a template region based on a distortion metric to determine one or more intra-prediction modes from a candidate list of intra-prediction modes; components for determining one or more intra-prediction mode predictors based on the determined one or more intra-prediction modes; and components for calculating predictions for the samples of the video sample data based on the intra-prediction mode predictors. The components include at least one processor and a memory including computer program code, wherein the processor may further include processor circuitry. The memory and computer program code are configured, together with the at least one processor, to cause the apparatus to perform various embodiments. Figure 4 The method.

[0080] Figure 5 An example of electronic device 500 is illustrated, which is an example of a video encoding system that can implement this embodiment. In some embodiments, the device may be a mobile terminal or user equipment or camera device of a wireless communication system. Device 500 may also be included at a graphics processing unit of a local server, a remote server, or a computer. The device may also be included as part of a head-mounted display device.

[0081] The device can be configured to perform various functions, such as collecting information via one or more sensors, encoding and / or decoding information, receiving and / or transmitting information, and analyzing information collected or received by the device. The device configured to encode a video scene may optionally include one or more microphones for capturing the scene and / or one or more cameras for capturing information about the physical environment of the captured scene. Alternatively, the device configured for encoding may be configured to receive information about the environment and / or simulated environment of the captured scene. The device configured to decode and / or render a video scene may be configured to receive a bitstream including encoded video. The device configured to decode and / or render a video scene may include one or more speaker / audio transducers and / or displays, and / or may be configured to transmit the decoded scene or signal to a device including one or more speaker / audio transducers and / or displays. The device configured to decode and / or render a video scene may include a user device, a head-mounted display, or another device capable of rendering AR, VR, and / or MR experiences to a user.

[0082] Device 500 includes one or more processors 510, one or more memories 520, and one or more transceivers interconnected via one or more buses. The one or more memories 520 store computer instructions, for example, in corresponding modules (module 1, module 2, module N). The one or more memories may store data in the form of image, video, and / or audio data, and / or may also store instructions to be executed by a processor or processor circuitry. The one or more processors may include a central processing unit (CPU) and / or a graphics processing unit (GPU). The one or more buses may be address buses, data buses, or control buses, and may include interconnect mechanisms such as a series of lines on a motherboard or integrated circuit, optical fibers, or other optical communication devices. The device also includes a codec 530 configured to implement various embodiments related to this solution. According to some embodiments, the device may include an encoder or a decoder. Device 500 also includes a communication interface 540 adapted to generate wireless communication signals, for example, for communication with a cellular communication network, wireless communication system, or wireless local area network, and thus enable data transmission via a data transmission network 550.

[0083] Device 500 may include a display in the form of a liquid crystal display. In other embodiments of the invention, the display may be any suitable display technology suitable for displaying images or video. Device 500 may also include a keyboard. In other embodiments of the invention, any suitable data or user interface mechanism may be employed. For example, the user interface may be implemented as a virtual keyboard or data input system as part of a touch-sensitive display. Device 500 may include a microphone or any suitable audio input, which may be a digital or analog signal input. Device 500 may also include an audio output device, which in embodiments of the invention may be any of the following: a handset, a speaker, or an analog or digital audio output connection. Device 500 may also include a battery (or in other embodiments of the invention, the device may be powered by any suitable mobile energy device, such as a solar cell, fuel cell, or clockwork generator). The device may also include a camera capable of recording or capturing images and / or video. The camera may be a multi-lens camera system with at least two camera sensors. The camera is capable of recording or detecting individual frames, which are then passed to codec 530 or processor 510. The device may receive video and / or image data from another device for processing before transmission and / or storage.

[0084] The device 500 may also include, for example, Figures 1 to 2 Other functional units disclosed in any of the figures in this embodiment may be used to implement any of the embodiments in this embodiment.

[0085] This device can operate in a system comprising multiple communication devices that can communicate over one or more networks. The system may include any combination of wired or wireless networks, including but not limited to wireless cellular telephone networks (such as GSM, UMTS, CDMA networks, etc.), wireless local area networks (WLANs) defined by any of the IEEE 802.x standards, Bluetooth personal area networks, Ethernet LANs, token ring LANs, wide area networks, and the Internet.

[0086] For example, the system could be a mobile phone network that enables internet connectivity. This connectivity could include, but is not limited to, long-range wireless connectivity, short-range wireless connectivity, and various wired connections, including but not limited to telephone lines, power lines, and similar communication paths.

[0087] Example communication devices operating in this system may include, but are not limited to, electronic devices or apparatuses, combinations of personal digital assistants (PDAs) and mobile phones, PDAs, integrated messaging devices (IMDs), desktop computers, and laptop computers, each of which may be representative of a device according to this embodiment. A device according to this embodiment may be stationary or mobile when carried by a moving individual. The device may also be located in a vehicle, including but not limited to cars, trucks, taxis, buses, trains, ships, airplanes, bicycles, motorcycles, or any similar suitable means of transport.

[0088] The device can also be a set-top box; that is, a digital TV receiver, tablet computer, or (laptop) personal computer (PC) with or without a display or wireless capability, having hardware or software or a combination thereof implemented in encoders / decoders in various operating systems, or providing chipsets, processors, DSPs, and / or embedded systems based on hardware / software encoding.

[0089] The apparatus according to this embodiment can send and receive calls and messages, and communicate with service providers via a wireless connection to a base station. The base station can be connected to a network server that allows communication between the mobile phone network and the Internet.

[0090] This device can communicate using various transmission technologies, including but not limited to Code Division Multiple Access (CDMA), Global System for Mobile Communications (GSM), Universal Mobile Telecommunications System (UMTS), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Transmission Control Protocol-Internet Protocol (TCP-IP), Short Message Service (SMS), Multimedia Messaging Service (MMS), email, Instant Messaging Service (IMS), Bluetooth, IEEE 802.11, and any similar wireless communication technologies. Communication devices implementing various embodiments of the invention can communicate using various media, including but not limited to radio, infrared, laser, cable connections, and any suitable connection.

[0091] In telecommunications and data networks, a channel can refer to a physical channel or a logical channel. A physical channel can refer to a physical transmission medium such as a wire, while a logical channel can refer to a logical connection on a multiplexed medium capable of transmitting several logical channels. A channel can be used to transmit information signals (such as bit streams) from one or more senders (or transmitters) to one or more receivers.

[0092] The MPEG-2 transport stream (TS), as defined in ISO / IEC 13818-1 or equivalently in ITU-T Recommendation H.222.0, is a format used to carry audio, video, and other media, as well as program metadata or other metadata, in a multiplexed stream. A packet identifier (PID) is used to identify the elementary stream (also known as a packetized elementary stream) within the TS. Therefore, logical channels within an MPEG-2 TS can be considered to correspond to specific PID values.

[0093] Available media file format standards include the ISO Basic Media File Format (ISO / IEC 14496-12, which can be abbreviated as ISOBMFF) and the file format for NAL unit structured video (ISO / IEC 14496-15), the latter of which is derived from ISOBMFF.

[0094] Various embodiments can be implemented using computer program code residing in memory and causing related devices to execute the method. For example, a device may include circuitry and electronics for processing, receiving, and transmitting data, computer program code in memory, and a processor that, when running the computer program code, causes the device to perform the features of the embodiments. Furthermore, a network device, such as a server, may include circuitry and electronics for processing, receiving, and transmitting data, computer program code in memory, and a processor that, when running the computer program code, causes the network device to perform the features of various embodiments.

[0095] If necessary, the different functions discussed herein may be executed in different orders and / or simultaneously with other functions. Furthermore, if necessary, one or more of the functions and embodiments described above may be optional or may be combined.

[0096] Although various aspects of the embodiments are set forth in the independent claims, other aspects include other combinations of features from the described embodiments and / or dependent claims with features of the independent claims, not just combinations expressly set forth in the claims.

[0097] It should also be noted that while exemplary embodiments have been described above, these descriptions should not be considered limiting. Rather, various variations and modifications may be made without departing from the scope of this disclosure as defined by the appended claims.

Claims

1. An apparatus for decoding encoded samples of a block of video sample data, wherein, for a block of video sample data, the apparatus comprises: Components for deriving information to configure a template-based intra-frame prediction mode derivation process, wherein, in order to implement the configured template-based intra-frame prediction mode derivation process, the apparatus further includes: A component used to calculate distortion on a template region based on a distortion metric to determine one or more intra-prediction modes from a list of intra-prediction mode candidates; Components for determining one or more intra-frame mode predictors based on the determined one or more intra-frame prediction modes; and A component for calculating predictions for samples of the video sample data based on the intra-frame pattern predictor.

2. The apparatus of claim 1, wherein the derived information is derived from an encoded bit stream.

3. The apparatus according to claim 1 or 2, further comprising: A component for determining a distortion metric among multiple distortion metrics using the derived information.

4. The apparatus of claim 3, wherein the distortion measure is one of the following: the sum of absolute differences, the sum of absolute transformation differences, the sum of squared errors, the sum of absolute differences after removing the mean, the sum of absolute transformation differences after removing the mean, and the sum of squared errors after removing the mean.

5. The apparatus according to claim 1 or 2, further comprising: Components used to determine the set of reconstructed samples to form the template region using the derived information.

6. The apparatus according to claim 1 or 2, further comprising: A component for using the derived information to determine which of the at least two intra-prediction mode candidate lists is used.

7. A method for decoding encoded samples of a block of video sample data, wherein, for a block of video sample data, the method includes: The derivation information is used to configure a template-based intra-prediction mode derivation process, wherein the configured template-based intra-prediction mode derivation process includes: Distortion on the template region is calculated based on distortion metrics to determine one or more intra-prediction modes from the intra-prediction mode candidate list; One or more intra-frame mode predictors are determined by the determined intra-frame prediction modes; and The intra-frame pattern predictor is used to calculate predictions for samples of the video sample data.

8. The method of claim 7, wherein the derived information is derived from the encoded bitstream.

9. The method according to claim 7 or 8, further comprising: The derived information is used to determine the distortion measure among multiple distortion measures.

10. The method of claim 9, wherein the distortion measure is one of the following: sum of absolute differences, sum of absolute transformation differences, sum of squared errors, sum of mean-free absolute differences, sum of mean-free absolute transformation differences, and sum of mean-free squared errors.

11. The method according to claim 7 or 8, comprising: The derived information is used to determine the set of reconstructed samples to form the template region.

12. The method according to claim 7 or 8, further comprising: The derived information is used to determine which of the at least two intra-prediction mode candidate lists is used.

Images