Template-based intra mode derivation merge

Abstract

An apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to: determine at least one intra prediction mode; and form an intra prediction for a current block of video data by determining the at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting intra-prediction information from at least one different block, where the intra-prediction information extracted from the at least one different block is determined based on a computation of a distortion in a template.

Description

TEMPLATE-BASED INTRA MODE DERIVATION MERGETECHNICAL FIELD

[0001] The examples and non-limiting embodiments relate generally to multimedia transport and, more particularly, to template-based intra mode derivation merge.BACKGROUND

[0002] It is known to perform data compression and decoding in a multimedia system.BRIEF DESCRIPTION OF THE DRAWINGS

[0003] The foregoing embodiments and other features are explained in the following description, taken in connection with the accompanying drawings, wherein:

[0004] FIG. 1 shows schematically an electronic device employing embodiments of the examples described herein.

[0005] FIG. 2 shows schematically a user equipment suitable for employing embodiments of the examples described herein.

[0006] FIG. 3 further shows schematically electronic devices employing embodiments of the examples described herein connected using wireless and wired network connections.

[0007] FIG. 4 shows schematically a block chart of an encoder used for data compression on a general level.

[0008] FIG. 5 is a block diagram illustrating a system in accordance with an example.

[0009] FIG. 6 is an example apparatus configured to implement the examples described herein.

[0010] FIG. 7 shows a representation of an example of non-volatile memory media used to store instructions that implement the examples described herein.

[0011] FIG. 8 is an example method, based on the examples described herein.

[0012] FIG. 9 is an example method, based on the examples described herein.

[0013] FIG. 10 is an example method, based on the examples described herein.

[0014] FIG. 11 is an example method, based on the examples described herein.DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

[0015] Described herein is a method and apparatus for implementing a template-based intra mode derivation merge.

[0016] The following describes in detail a suitable apparatus and possible mechanisms for a video / image encoding process according to embodiments. In this regard reference is first made to FIG. 1 and FIG. 2, where FIG. 1 shows an example block diagram of an apparatus 50. The apparatus may be an Internet of Things (loT) apparatus configured to perform various functions, such as for example, gathering information by one or more sensors, receiving or transmitting information, analyzing information gathered or received by the apparatus, or the like. The apparatus may comprise a video coding system, which may incorporate a codec. FIG. 2 shows a layout of an apparatus according to an example embodiment. The elements of FIG. 1 and FIG. 2 are explained next.

[0017] The electronic device 50 may for example be a mobile terminal or user equipment of a wireless communication system, a sensor device, a tag, or other lower power device. However, it would be appreciated that embodiments of the examples described herein may be implemented within any electronic device or apparatus which may process data by neural networks.

[0018] The apparatus 50 may comprise a housing 30 for incorporating and protecting the device. The apparatus 50 further may comprise a display 32 in the form of a liquid crystal display. In other embodiments of the examples described herein the display may be any suitable display technology suitable to display an image or video. The apparatus 50 may further comprise a keypad 34. In other embodiments of the examples described herein any suitable data or user interface mechanism may be employed. For example the user interface may be implemented as a virtual keyboard or data entry system as part of a touch-sensitive display.

[0019] The apparatus may comprise a microphone 36 or any suitable audio input which may be a digital or analog signal input. The apparatus 50 may further comprise an audio output device which in embodiments of the examples described herein may be any one of: an earpiece 38, speaker, or an analog audio or digital audio output connection. The apparatus 50 may also comprise a battery (or in other embodiments of the examples described herein the device may be powered by any suitable mobile energy device such as solar cell, fuel cell or clockwork generator). The apparatus may further comprise a camera 42 capable of recording or capturing images and / or video. The apparatus 50 may further comprise an infrared port for short range line of sight communication to other devices. In other embodiments the apparatus 50 may further comprise any suitable short range communication solution such as for example a Bluetooth wireless connection or a USB / firewire wired connection.

[0020] The apparatus 50 may comprise a controller 56, processor or processor circuitry for controlling the apparatus 50. The controller 56 may be connected to memory 58 which in embodiments of the examples described herein may store both data in the form of image and audio data and / or may also store instructions for implementation on the controller 56. The controller 56 may further be connected to codec circuitry 54 suitable for carrying out coding and / or decoding of audio and / or video data or assisting in coding and / or decoding carried out by the controller.

[0021] The apparatus 50 may further comprise a card reader 48 and a smart card 46, for example a UICC and UICC reader for providing user information and being suitable for providing authentication information for authentication and authorization of the user at a network.

[0022] The apparatus 50 may comprise radio interface circuitry 52 connected to the controller and suitable for generating wireless communication signals for example for communication with a cellular communications network, a wireless communications system or a wireless local area network. The apparatus 50 may further comprise an antenna 44 connected to the radio interface circuitry 52 for transmitting radio frequency signals generated at the radio interface circuitry 52 to other apparatus(es) and / or for receiving radio frequency signals from other apparatus(es).

[0023] The apparatus 50 may comprise a camera capable of recording or detecting individual frames which are then passed to the codec 54 or the controller for processing. The apparatus may receive the video image data for processing from another device prior to transmission and / or storage. The apparatus 50 may also receive either wirelessly or by a wired connection the image for coding / decoding. The structural elements of apparatus 50 described above represent examples of means for performing a corresponding function.

[0024] With respect to FIG. 3, an example of a system within which embodiments of the examples described herein can be utilized is shown. The system 10 comprises multiple communication devices which can communicate through one or more networks. The system 10 may comprise any combination of wired or wireless networks including, but not limited to a wireless cellular telephone network (such as a GSM, UMTS, CDMA, LTE, 4G, 5G network etc.), a wireless local area network (WLAN) such as defined by any of the IEEE 802.x standards, a Bluetooth personal area network, an Ethernet local area network, a token ring local area network, a wide area network, and the Internet.

[0025] The system 10 may include both wired and wireless communication devices and / or apparatus 50 suitable for implementing embodiments of the examples described herein.

[0026] For example, the system shown in FIG. 3 shows a mobile telephone network 11 and a representation of the internet 28. Connectivity to the internet 28 may include, but is not limited to, long range wireless connections, short range wireless connections, and various wired connections including, but not limited to, telephone lines, cable lines, power lines, and similar communication pathways.

[0027] The example communication devices shown in the system 10 may include, but are not limited to, an electronic device or apparatus 50, a combination of a personal digital assistant (PDA) and a mobile telephone 14, a PDA 16, an integrated messaging device (IMD) 18, a desktop computer 20, a notebook computer 22, or a head-mounted apparatus 21, which head-mounted apparatus 21 may be a head-mounted display (HMD), or glasses having a camera or other device used for processing images and / or video. The apparatus 50 may be stationary or mobile when carried by an individual who is moving. The apparatus 50 may also be located in a mode of transport including, but not limited to, a car, a truck, a taxi, a bus, a train, a boat, an airplane, a bicycle, a motorcycle or any similar suitable modeof transport.

[0028] The embodiments may also be implemented in a set-top box; e.g. a digital TV receiver, which may / may not have a display or wireless capabilities, in tablets or (laptop) personal computers (PC), which have hardware and / or software to process neural network data, in various operating systems, and in chipsets, processors, DSPs and / or embedded systems offering hardware / software based coding.

[0029] Some or further apparatus may send and receive calls and messages and communicate with service providers through a wireless connection 25 to a base station 24. The base station 24 may be connected to a network server 26 that allows communication between the mobile telephone network 11 and the internet 28. The system may include additional communication devices and communication devices of various types.

[0030] The communication devices may communicate using various transmission technologies including, but not limited to, code division multiple access (CDMA), global systems for mobile communications (GSM), universal mobile telecommunications system (UMTS), time divisional multiple access (TDMA), frequency division multiple access (FDMA), transmission control protocol-internet protocol (TCP -IP), short messaging service (SMS), multimedia messaging service (MMS), email, instant messaging service (IMS), Bluetooth, IEEE 802.11, 3GPP Narrowband loT and any similar wireless communication technology. A communications device involved in implementing various embodiments of the examples described herein may communicate using various media including, but not limited to, radio, infrared, laser, cable connections, and any suitable connection.

[0031] In telecommunications and data networks, a channel may refer either to a physical channel or to a logical channel. A physical channel may refer to a physical transmission medium such as a wire, whereas a logical channel may refer to a logical connection over a multiplexed medium, capable of conveying several logical channels. A channel may be used for conveying an information signal, for example a bitstream, from one or several senders (or transmitters) to one or several receivers.

[0032] The embodiments may also be implemented in so-called loT devices. The Internet of Things (loT) may be defined, for example, as an interconnection of uniquely identifiableembedded computing devices within the existing Internet infrastructure. The convergence of various technologies has and may enable many fields of embedded systems, such as wireless sensor networks, control systems, home / building automation, etc. to be included in the Internet of Things (loT). In order to utilize the Internet loT devices are provided with an IP address as a unique identifier. loT devices may be provided with a radio transmitter, such as a WLAN or Bluetooth transmitter or a RFID tag. Alternatively, loT devices may have access to an IP -based network via a wired network, such as an Ethernet-based network or a power-line connection (PLC).

[0033] An MPEG-2 transport stream (TS), specified in ISO / IEC 13818-1 or equivalently in ITU-T Recommendation H.222.0, is a format for carrying audio, video, and other media as well as program metadata or other metadata, in a multiplexed stream. A packet identifier (PID) is used to identify an elementary stream (a.k.a. packetized elementary stream) within the TS. Hence, a logical channel within an MPEG-2 TS may be considered to correspond to a specific PID value.

[0034] Available media file format standards include ISO base media file format (ISO / IEC 14496-12, which may be abbreviated ISOBMFF) and file format for NAL unit structured video (ISO / IEC 14496-15), which derives from the ISOBMFF.

[0035] FIG. 4 shows a block diagram of a general structure of a video encoder. FIG. 4 presents an encoder for two layers, but it would be appreciated that presented encoder could be similarly extended to encode more than two layers. FIG. 4 illustrates a video encoder comprising a first encoder section 500 for a base layer and a second encoder section 502 for an enhancement layer. Each of the first encoder section 500 and the second encoder section 502 may comprise similar elements for encoding incoming pictures. The encoder sections 500, 502 may comprise a pixel predictor 302, 402, prediction error encoder 303, 403 and prediction error decoder 304, 404. FIG. 4 also shows an embodiment of the pixel predictor 302, 402 as comprising an inter-predictor 306, 406 (Pinter), an intra-predictor 308, 408 (Pint™), a mode selector 310, 410, a filter 316, 416 (F), and a reference frame memory 318, 418 (RFM). The pixel predictor 302 of the first encoder section 500 receives 300 base layer images (Io,n) of a video stream to be encoded at both the inter-predictor 306 (which determines the difference between the image and a motion compensated reference frame 318) and the intra-predictor 308 (which determines a prediction for an image block basedonly on the already processed parts of the current frame or picture). The output of both the inter-predictor and the intra-predictor are passed to the mode selector 310. The intrapredictor 308 may have more than one intra-prediction modes. Hence, each mode may perform the intra-prediction and provide the predicted signal to the mode selector 310. The mode selector 310 also receives a copy of the base layer picture 300. Correspondingly, the pixel predictor 402 of the second encoder section 502 receives 400 enhancement layer images (Ii,n) of a video stream to be encoded at both the inter-predictor 406 (which determines the difference between the image and a motion compensated reference frame 418) and the intra-predictor 408 (which determines a prediction for an image block based only on the already processed parts of the current frame or picture). The output of both the inter-predictor and the intra-predictor are passed to the mode selector 410. The intra- predictor 408 may have more than one intra-prediction modes. Hence, each mode may perform the intra-prediction and provide the predicted signal to the mode selector 410. The mode selector 410 also receives a copy of the enhancement layer picture 400.

[0036] Depending on which encoding mode is selected to encode the current block, the output of the inter-predictor 306, 406 or the output of one of the optional intra-predictor modes or the output of a surface encoder within the mode selector is passed to the output of the mode selector 310, 410. The output of the mode selector is passed to a first summing device 321, 421. The first summing device may subtract the output of the pixel predictor 302, 402 from the base layer picture 300 / enhancement layer picture 400 to produce a first prediction error signal 320, 420 (Dn) which is input to the prediction error encoder 303, 403.

[0037] The pixel predictor 302, 402 further receives from a preliminary reconstructor 339, 439 the combination of the prediction representation of the image block 312, 412 (P’n) and the output 338, 438 (D’n) of the prediction error decoder 304, 404. The preliminary reconstructed image 314, 414 (I’n) may be passed to the intra-predictor 308, 408 and to the filter 316, 416. The filter 316, 416 receiving the preliminary representation may filter the preliminary representation and output a final reconstructed image 340, 440 (R’n) which may be saved in a reference frame memory 318, 418. The reference frame memory 318 may be connected to the inter-predictor 306 to be used as the reference image against which a future base layer picture 300 is compared in inter-prediction operations. Subject to the base layer being selected and indicated to be the source for inter-layer sample prediction and / or inter-layer motion information prediction of the enhancement layer according to some embodiments, the reference frame memory 318 may also be connected to the inter-predictor 406 to be used as the reference image against which a future enhancement layer picture 400 is compared in inter-prediction operations. Moreover, the reference frame memory 418 may be connected to the inter-predictor 406 to be used as the reference image against which a future enhancement layer picture 400 is compared in inter-prediction operations.

[0038] Filtering parameters from the filter 316 of the first encoder section 500 may be provided to the second encoder section 502 subject to the base layer being selected and indicated to be the source for predicting the filtering parameters of the enhancement layer according to some embodiments.

[0039] The prediction error encoder 303, 403 comprises a transform unit 342, 442 (T) and a quantizer 344, 444 (Q). The transform unit 342, 442 transforms the first prediction error signal 320, 420 to a transform domain. The transform is, for example, the DCT transform. The quantizer 344, 444 quantizes the transform domain signal, e.g. the DCT coefficients, to form quantized coefficients.

[0040] The prediction error decoder 304, 404 receives the output from the prediction error encoder 303, 403 and performs the opposite processes of the prediction error encoder 303, 403 to produce a decoded prediction error signal 338, 438 which, when combined with the prediction representation of the image block 312, 412 at the second summing device 339, 439, produces the preliminary reconstructed image 314, 414. The prediction error decoder 304, 404 may be considered to comprise a dequantizer 346, 446 (Q'1), which dequantizes the quantized coefficient values, e.g. DCT coefficients, to reconstruct the transform signal and an inverse transformation unit 348, 448 (T'1), which performs the inverse transformation to the reconstructed transform signal wherein the output of the inverse transformation unit 348, 448 includes reconstructed block(s). The prediction error decoder may also comprise a block filter which may filter the reconstructed block(s) according to further decoded information and filter parameters.

[0041] The entropy encoder 330, 430 (E) receives the output of the prediction error encoder 303, 403 and may perform a suitable entropy encoding / variable length encoding on the signal to provide error detection and correction capability. The outputs of the entropyencoders 330, 430 may be inserted into a bitstream e.g. by a multiplexer 508 (M).

[0042] The examples described herein fit within the context of next generation video coding standardization, including efforts towards developing the H.267 video coding standard, and the ECM exploration reference software model.

[0043] Template-based Intra Mode Derivation (TIMD) fusion is a method used in ECM to derive the prediction of the current block as a fusion of up to two intra prediction modes derived from the available reconstructed samples template. A list of possible candidate modes is considered, formed of Most Probable Modes (MPMs), as well as possibly wide- angle modes and extended-precision angular modes. Then a TIMD search is performed, where for each mode the distortion (or TIMD cost) between the prediction and reconstruction samples of the template is calculated. Depending on the block position within the frame and the reconstructed samples availability, templates can be formed of samples from the top and or the left region adjacent to the current block. TIMD templates in ECM contain 4 lines of reconstructed neighboring luma samples, however, for blocks with sides smaller than 16, the template contains only 2 reconstructed neighboring luma samples. As a result of the TIMD search, the first two intra prediction modes with minimum TIMD costs (computed using SATD in ECM) are selected as the TIMD modes, where the second mode is only considered if its cost is close enough to the cost found for the primary mode; otherwise, only the primary TIMD mode is used as predictor for the current block. If the two modes are fused, the blending weights are computed based on their TIMD costs. TIMD is considered in ECM as an option signaled at the encoder side.

[0044] In the existing TIMD method it is unlikely that two blocks with different neighboring reference samples used as TIMD template will generate identical intra prediction modes and weights which are fused in the TIMD prediction. However, it may be beneficial in some cases (e.g. neighboring blocks being part of the same object or texture) to propagate the TIMD modes and weights to neighboring blocks. It may also be beneficial to mix the TIMD information found in more neighboring blocks to compute new modes and weights to predict the current block.

[0045] A method operating according to the invention produces an intra-prediction for a given block using an intra-prediction method, where the prediction for the current block isobtained by means of at least one determined intra-prediction mode, where the at least one intra prediction mode used to predict the current block is determined by using a Templatebased Intra Mode Derivation (TIMD) process applied to at least one different block, where the TIMD process applied to at least one different block comprises the computation of a distortion between predicted and reconstructed samples in a template.

[0046] A method operating according to the invention produces an intra-prediction for a given block using an intra-prediction method, where the prediction for the current block is obtained by means of blending at least two predictors using weights, where the predictors are computed based on determined intra-prediction modes, where the intra prediction modes are determined by using a Template-based Intra Mode Derivation (TIMD) process applied to at least one different block, where the TIMD process applied to at least one different block comprises the computation of a distortion between predicted and reconstructed samples in a template of a different block, where the weights used to blend the at least two predictors for the current block are computed based on the distortion.

[0047] A method operating according to the invention produces an intra-prediction for a given block using an intra-prediction method, where the intra prediction modes and weights used to predict the current block are obtained as a result of combining the TIMD information extracted from a number of different blocks.

[0048] A method operating according to the invention produces an intra-prediction for a given block using an intra-prediction method, where the intra prediction modes and weights used to predict the current block are derived from the TIMD information extracted from a different block.

[0049] A method operating according to the invention produces an intra-prediction for a given block using an intra-prediction method, where the intra prediction modes and weights used to predict the current block are determined based on a TIMD process applied to the current block, where the TIMD process comprises using the TIMD information extracted from at least one different block.

[0050] A method operating according to the invention produces an intra-prediction for a given block using an intra-prediction method, where the prediction for the current block isobtained by means of blending up to two predictors obtained with determined intraprediction modes using determined weights, where the intra prediction modes and weights used to predict the current block are determined by using a Template-based Intra Mode Derivation (TIMD) process applied to at least one different block. As an example, the different blocks are blocks that are neighboring blocks to the current block. As an example, two neighboring blocks are considered, namely a block located above the current block, and a block located on the left of the current block. As an example, up to 7 neighboring blocks are considered, including a block located on the top-left of the current block, one or two blocks located on the top of the current block, a block located on the top-right of the current block, one or two blocks located on the left of the current block, a block located on the bottom-left of the current block. As an example, the neighboring blocks which are considered are the ones which contain the following pixel positions (relative to the top-left pixel of the current block position): (0, -1), (W - 1, -1), (W, -1), (-1, -1), (-1, 0), (-1, H - 1), (-1, H). As an example, different blocks that are not adjacent to the current block are considered. As an example, different blocks that are separated by a given number of lines of pixels above or on the left of the current block are considered. As an example, only different blocks that are encoded using the TIMD mode are considered as valid blocks. As an example, only different blocks that are encoded using the TIMD Merge mode are considered as valid neighbors. As an example, a block is considered to be coded with the TIMD mode also if it is coded using the TIMD Merge mode.

[0051] The TIMD information from different blocks is extracted and merged (or inherited) for usage in the current block. As an example, in case exactly one of the considered different blocks is encoded with the TIMD mode, its TIMD intra modes and weights can be directly merged to be used in the current block. As an example, in case more than one of the considered different blocks are encoded with TIMD mode, then the different block with larger size can be merged. As an example, in case more than one different blocks are encoded with TIMD mode, then a determined block can be considered, depending on its position with respect to the current block. As an example, in case any of different blocks are coded using TIMD Merge mode, then they can be considered as if they are coded with the TIMD mode. As an example, if a different block was coded using TIMD Merge mode, resulting in a number of intra prediction modes and weights determined to predict that different block, those intra prediction modes and weights may be merged for usage in thecurrent block. As an example, if a different block was coded using TIMD Merge mode, resulting in a number of intra prediction modes and costs associated to these intra prediction modes, those intra prediction modes and costs may be used to determine the intra prediction modes to predict the current block.

[0052] As an additional example, the shape of the candidate blocks can be considered if more than one of the considered different blocks are coded with the TIMD mode in order to select at least one block as a source block for merging the TIMD information. As an example, if a given different block is selected as a source for merging the TIMD information, then its TIMD modes and weights may be used in the current block. As an example, if a given different block is selected as a source for merging the TIMD information, then its TIMD modes and TIMD costs may be used to determine the intra prediction modes to predict the current block. As an example, the horizontal distances of the left borders of the candidate blocks on the left side of the current block and vertical distances of the top borders of the candidate blocks above the current block can be determined with respect to the borders for the current block, and the candidate with smallest distance can be selected. As an example, if there is a candidate block of size 8x8 samples on the left side of the current block with its further away border having a distance of 8 samples from the border of the current block, and there is a candidate block of size 16x4 samples above the current block with its further away border having a distance of 4 samples from the border of the current block, the latter one can be selected due to its template region being closer to the current block.

[0053] As an example, the usage of this method can be signaled at the block level under the following conditions: an additional TIMD Merge flag is signaled directly after the TIMD flag only if the current block is encoded using the TIMD mode and if at least one of the considered different blocks is encoded with TIMD mode. As an example, an additional TIMD merge flag is signaled directly after the TIMD flag only if the current block is encoded using the TIMD mode and if at least one of the neighboring blocks is encoded with TIMD Merge mode. As an example, usage of this method can be inferred. As an example, usage of this method may be inferred from the fact that the current block is coded with the TIMD mode and that at least one of the considered different blocks is encoded with TIMD mode.

[0054] As another example, the usage of this method can be signaled at the block level under the following conditions: an additional TIMD Merge flag is signaled only if at least one of the considered different blocks is encoded with TIMD mode or the TIMD Merge mode. As another example, the usage of this method can be signaled at the frame level. As another example, the usage of this method can be signaled at the block level.

[0055] As another example, a method operating according to the invention produces an intra-prediction for a given block using intra-prediction, where a number of intra-prediction modes and weights are used to predict the content of the current block, where the intra prediction modes and weights used to predict the current block are obtained as a result of combining the TIMD information extracted from a number of different blocks. As an example, up to two intra-prediction modes and weights are used. As an example, up to three intra-prediction modes and weights are used. As an example, up to five intra-prediction modes and weights are used. As an example, the intra-prediction modes and weights determined by combining the TIMD information extracted from a number of different blocks are combined with a pre-determined intra-prediction mode. As an example, the predetermined intra-prediction mode is the Planar or the DC mode. A method operating according to the invention produces an intra-prediction for a given block using intraprediction, where a number of intra-prediction modes and weights are used to predict the content of the current block, where the intra-prediction modes and weights are derived by means of computing parameters, where the computed parameters for the current block are obtained by merging or combining the TIMD parameters extracted from a number of different blocks. As an example, in case only one of the considered different block is encoded with the TIMD mode, then its TIMD parameters can be merged for usage in the current block. As an example, in case only one of the considered different block was encoded with the TIMD mode, then its derived modes and their corresponding weights can be merged to the current block.

[0056] As an example, if more than one of the considered different blocks are encoded with the TIMD mode, their corresponding TIMD modes and TIMD costs can be combined to derive new modes and costs for the current block. As an example, at least some of the TIMD modes and associated TIMD costs extracted from the different blocks can be considered. As an example, the TIMD modes associated with the two or more minimumTIMD costs can be selected. In order to take into account the fact that TIMD costs from different blocks are computed on templates of different sizes, the TIMD costs can be scaled. As an example, the scaling can be performed based on the size of the different blocks, or the scaling can be performed based on the size of the TIMD template used on a different block. As another example, the TIMD costs can be scaled based on the distance of the different block relative to the current block, where the distance can be computed as an example based on the top-left coordinate of the current block and the top-left coordinate of the different block. As another example, the TIMD costs can be modified by means of a bias factor. As an example, a bias could be added to the TIMD costs to account for the fact that some TIMD costs may be equal to zero. As an example, the bias may be computed based on characteristics of the current block. As another example, the bias may be computed based on characteristics of the different blocks.

[0057] The different blocks considered could comprise the different blocks that are coded using TIMD mode and or are coded using the TIMD Merge mode. As an example, only a subset of the considered different blocks may be used to determine the intra-prediction information for the current block. As an example, the subset of different blocks may be computed based on characteristics of the current block. As another example, the subset of different blocks may be computed based on characteristics of different blocks. As an example, only different blocks that are at a given distance from the current block may be considered, where the distance can be computed as an example based on the top-left coordinate of the current block and the top-left coordinate of the different block.

[0058] As another example, if more than one different blocks were encoded with the TIMD mode, then one different block may be selected to extract the TIMD information to use in the current block, where the neighboring block could be selected based on the TIMD cost. As an example, the different block with smallest combined TIMD cost could be selected. As an example, the combined TIMD cost could be computed as the average between the primary TIMD cost and the secondary TIMD cost. As an example, the average could be weighted. As an example, a higher weight could be assigned to the primary TIMD cost. As another example, other combinations could be used, as an example the sum of the primary TIMD cost and the secondary TIMD cost could be used.

[0059] As another example, the TIMD modes and TIMD costs extracted from differentblocks can be combined with other intra-prediction modes extracted from other different blocks that are not encoded with the TIMD mode, to derive new modes and costs for the current block. As an example, different blocks coded with the Decoder-side Intra Mode Derivation (DIMD) mode can be considered. As an example, the TIMD and DIMD modes extracted from different blocks can be combined to derive at least one intra-prediction mode for the current block. A an example, the histogram of gradients computed in a DIMD block can be used in combination with the TIMD costs computed in a TIMD block to determine at least one intra-prediction mode for the current block. As an example, the amplitudes extracted from the histogram of gradients computed in a DIMD block can be used in combination with the TIMD costs computed in a TIMD block to determine at least one intraprediction mode for the current block.

[0060] As another example, a method operating according to the invention produces an intra-prediction for a given block using intra-prediction, where the intra prediction modes and weights used to predict the current block are determined based on a TIMD process applied to the current block, where the TIMD process comprises using the TIMD information extracted from at least one different block. As an example, when searching for the best mode using distortions computed between predicted and reconstructed samples in the template of the current block, additional modes can be considered with associated costs, where the modes and costs are extracted from different blocks. As an example, the TIMD modes and costs from different blocks can be used. As an example, the TIMD costs from neighboring blocks can be scaled when including them in the TIMD search for the current block.

[0061] As another example, the Multiple Reference Line (MRL) index of a neighboring block can be extracted and used to determine the MRL index to use on the current block. As an example, in case only one neighboring block is considered to determine the intra prediction for the current block, then the MRL index of the current block can be used as the MRL index to predict the current block. As another example, in case more than one neighboring blocks are considered, the MRL indexes of the neighboring blocks can be combined to determine the MRL index to predict the current block. As an example, the minimum MRL index can be used. As an example, the maximum MRL index can be used. As an example, the average MRL index can be used. As an example, the MRL index can becomputed based on whether the MRL indexes of neighboring blocks are equal to each other.

[0062] As another example, a method operating according to the invention produces an intra-prediction for a given block using intra-prediction, where the intra prediction modes and weights used to predict the current block are determined based on a decoder-side intra mode derivation process applied to a different block, where spatial blending is used to blend the intra-prediction modes to predict the current block, where the weights used in the spatial blending depend on characteristics of the different blocks. As an example, the location of the different blocks with respect to the current block are used to determine the weights to use to spatial blend the intra-prediction modes. As an example, if a neighboring block located above the current block is coded with a given TIMD or Decoder-side Intra Mode Derivation (DIMD) mode, then higher weights could be assigned to the top portion of the current block when blending this mode to compute the intra-prediction for the current block. As an example, if a neighboring block located on the left of the current block is coded with a given TIMD or DIMD mode, then higher weights could be assigned to the left portion of the current block when blending this mode to compute the intra-prediction for the current block.

[0063] In an embodiment, an apparatus includes: at least one processor; and at least one non-transitory memory storing instructions, that when executed by the at least one processor, cause the apparatus at least to: form an intra prediction for a current block of video data by determining at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting intra-prediction information from at least one different block, where the intra-prediction information extracted from the at least one different block is determined based on the computation of a distortion in a template. The at least one different block comprises at least one neighboring block. The apparatus may operate based on signaling, where the presence of signaling is inferred based on whether any block of the at least one different block is coded using a template based intra mode derivation process.

[0064] FIG. 5 is a block diagram illustrating a system 500 in accordance with an example. In the example, the encoder 530 is used to encode video from the scene 515, and the encoder 530 is implemented in a transmitting apparatus 580. The encoder 530 produces a bitstream 510 comprising signaling that is received by the receiving apparatus 582, which implementsa decoder 540. The encoder 530 sends the bitstream 510 that comprises the herein described signaling. The decoder 540 forms the video for the scene 515-1, and the receiving apparatus 582 would present this to the user, e.g., via a smartphone, television, or projector among many other options.

[0065] In some examples, the transmitting apparatus 580 and the receiving apparatus 582 are at least partially within a common apparatus, and for example are located within a common housing 550. In other examples the transmitting apparatus 580 and the receiving apparatus 582 are at least partially not within a common apparatus and have at least partially different housings. Therefore in some examples, the encoder 530 and the decoder 540 are at least partially within a common apparatus, and for example are located within a common housing 550. For example the common apparatus comprising the encoder 530 and decoder 540 implements a codec. In other examples the encoder 530 and the decoder 540 are at least partially not within a common apparatus and have at least partially different housings, but when together still implement a codec.

[0066] 3D media from the capture (e.g., volumetric capture) at a viewpoint 512 of the scene 515, which includes a person 513) is converted via projection to a series of 2D representations with occupancy, geometry, and attributes. Additional atlas information is also included in the bitstream to enable inverse reconstruction. For decoding, the received bitstream 510 is separated into its components with atlas information; occupancy, geometry, and attribute 2D representations. A 3D reconstruction is performed to reconstruct the scene 515-1 created looking at the viewpoint 512-1 with a “reconstructed” person 513-1. The “- 1” are used to indicate that these are reconstructions of the original. As indicated at 520, the decoder 540 performs an action or actions based on the received signaling.

[0067] FIG. 6 is an example apparatus 600, which may be implemented in hardware, configured to implement the examples described herein. The apparatus 600 comprises at least one processor 602 (e.g., an FPGA and / or CPU), one or more memories 604 including computer program code 605, the computer program code 605 having instructions to carry out the methods described herein, wherein the at least one memory 604 and the computer program code 605 are configured to, with the at least one processor 602, cause the apparatus 600 to implement circuitry, a process, component, module, or function (implemented with control module 606) to implement the examples described herein, including a template-based intra mode derivation merge. Optionally included encoder 630 of the control module 606 performs encoding, and optionally included decoder 640 implements decoding. The memory 604 may be a non-transitory memory, a transitory memory, a volatile memory (e.g. RAM), or a non-volatile memory (e.g., ROM).

[0068] The apparatus 600 includes a display and / or I / O interface 608, which includes user interface (UI) circuitry and elements, that may be used to display features or a status of the methods described herein (e.g., as one of the methods is being performed or at a subsequent time), or to receive input from a user such as with using a keypad, camera, touchscreen, touch area, microphone, biometric recognition, one or more sensors, etc. The apparatus 600 includes one or more communication e.g. network (N / W) interfaces (I / F(s)) 610. The communication I / F(s) 610 may be wired and / or wireless and communicate over the Internet / other network(s) via any communication technique including via one or more links 624. The communication I / F(s) 610 may comprise one or more transmitters or one or more receivers.

[0069] The transceiver 616 comprises one or more transmitters 618 and one or more receivers 620. The transceiver 616 and / or communication I / F(s) 610 may comprise standard well-known components such as an amplifier, filter, frequency-converter, (de)modulator, and encoder / decoder circuitries and one or more antennas, such as antennas 614 used for communication over wireless link 626.

[0070] The control module 606 of the apparatus 600 comprises one of or both parts 606- 1 and / or 606-2, which may be implemented in a number of ways. The control module 606 may be implemented in hardware as control module 606-1, such as being implemented as part of the one or more processors 602. The control module 606-1 may be implemented also as an integrated circuit or through other hardware such as a programmable gate array. In another example, the control module 606 may be implemented as control module 606-2, which is implemented as computer program code (having corresponding instructions) 605 and is executed by the one or more processors 602. For instance, the one or more memories 604 store instructions that, when executed by the one or more processors 602, cause the apparatus 600 to perform one or more of the operations as described herein. Furthermore, the one or more processors 602, one or more memories 604, and example algorithms (e.g., as flowcharts and / or signaling diagrams), encoded as instructions, programs, or code, aremeans for causing performance of the operations described herein.

[0071] The apparatus 600 to implement the functionality of control 606 may correspond to any of the apparatuses depicted herein. Alternatively, apparatus 600 and its elements may not correspond to any of the other apparatuses depicted herein, as apparatus 600 may be part of a self-organizing / optimizing network (SON) node or other node, such as a node in a cloud.

[0072] The apparatus 600 may also be distributed throughout the network (e.g. internet 28) including within and between apparatus 600 and any network element (such as a base station 24 and / or apparatus 50).

[0073] Interface 612 enables data communication and signaling between the various items of apparatus 600, as shown in FIG. 6. For example, the interface 612 may be one or more buses such as address, data, or control buses, and may include any interconnection mechanism, such as a series of lines on a motherboard or integrated circuit, fiber optics or other optical communication equipment, and the like. Computer program code (e.g. instructions) 605, including control 606 may comprise object-oriented software configured to pass data or messages between objects within computer program code 605. The apparatus 600 need not comprise each of the features mentioned, or may comprise other features as well. The various components of apparatus 600 may at least partially reside in a common housing 628, or a subset of the various components of apparatus 600 may at least partially be located in different housings, which different housings may include housing 628.

[0074] FIG. 7 shows a schematic representation of non-volatile memory media 700a (e.g. computer / compact disc (CD) or digital versatile disc (DVD)) and 700b (e.g. universal serial bus (USB) memory stick) and 700c (e.g. cloud storage for downloading instructions and / or parameters 702 or receiving emailed instructions and / or parameters 702) storing instructions and / or parameters 702 which when executed by a processor allows the processor to perform one or more of the operations of the methods described herein.

[0075] FIG. 8 is an example method 800, based on the example embodiments described herein. At 810, the method includes determining at least one intra prediction mode. At 820, the method includes forming an intra prediction for a current block of video data bydetermining the at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting intra-prediction information from at least one different block of video data, where the intra-prediction information extracted from the at least one different block is determined based on a computation of a distortion in a template. Method 800 may be performed with a codec or decoder, such as apparatus 50, apparatuses depicted in FIG. 3 and FIG. 4, receiving apparatus 582 with decoder 540, apparatus 600, or other apparatuses described herein.

[0076] FIG. 9 is an example method 900, based on the example embodiments described herein. At 910, the method includes determining intra-prediction information based on computation of a distortion in a template. At 920, the method includes wherein the intraprediction information is configured to be used to form an intra prediction for a current block of video data. At 930, the method includes where the intra-prediction is formed by determining at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting the intra-prediction information from at least one different block, where the intra-prediction information extracted from the at least one different block is determined based on the computation of the distortion in the template. At 940, the method includes encoding the intra-prediction information as signaling into or along a bitstream. Method 900 may be performed with a codec or encoder, such as apparatus 50, apparatuses depicted in FIG. 3 and FIG. 4, transmitting apparatus 580 with encoder 530, apparatus 600, or other apparatuses described herein.

[0077] FIG. 10 is an example method 1000, based on the example embodiments described herein. At 1010, the method includes encoding a current block of video data, wherein the encoding of the current block of video data comprises: forming an intra prediction for a current block of video data by determining the at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting intra-prediction information from at least one different block of video data, where the intra-prediction information extracted from the at least one different block is determined based on a computation of a distortion in a template. At 1020, the method includes signaling the encoding of the current block of video data into or along a bitstream. Method 1000 may be performed with a codec or encoder, such as apparatus 50, apparatuses depicted in FIG. 3 and FIG. 4, transmitting apparatus 580 with encoder 530, apparatus 600, or other apparatuses described herein.

[0078] FIG. 11 is an example method 1100, based on the example embodiments described herein. Method 1100 includes forming an intra prediction for a current block of video data by determining at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting intra-prediction information from at least one different block of video data, where the intra-prediction information extracted from the at least one different block is determined based on a computation of a distortion in a template. Method 1100 may be performed with a codec or decoder, such as apparatus 50, apparatuses depicted in FIG. 3 and FIG. 4, receiving apparatus 582 with decoder 540, apparatus 600, or other apparatuses described herein.

[0079] The following examples are provided and described herein.

[0080] Example 1. An apparatus including: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to: determine at least one intra prediction mode; and form an intra prediction for a current block of video data by determining the at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting intraprediction information from at least one different block of video data, where the intraprediction information extracted from the at least one different block is determined based on a computation of a distortion in a template.

[0081] Example 2. The apparatus of example 1, wherein the at least one different block comprises at least one neighboring block.

[0082] Example 3. The apparatus according to any one of examples 1-2, wherein the apparatus operates based on signaling.

[0083] Example 4. The apparatus of example 3, where the presence of signaling is inferred based on whether at least one block of the at least one different block is encoded using a template based intra mode derivation process.

[0084] Example 5. The apparatus of any of examples 1 to 4, where the at least one different block is encoded using a template-based intra-prediction mode derivation process.

[0085] Example 6. The apparatus of example 5, wherein the at least one different block isdetermined to be encoded using the template-based intra mode derivation process when the at least one different block has been encoded based on: determining at least one intra prediction mode, and forming an intra prediction for the at least one different block of video data by determining the at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting intra-prediction information from at least one other different block, where the intra-prediction information extracted from the at least one other different block is determined based on a computation of a distortion in a template.

[0086] Example 7. The apparatus of any of examples 5 to 6, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: determine that the at least one different block is encoded using the template-based intra mode derivation process, when the at least one different block has been encoded based on: determining at least one intra prediction mode, and forming an intra prediction for the at least one different block of video data by determining the at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting intra-prediction information from at least one other different block, where the intra-prediction information extracted from the at least one other different block is determined based on a computation of a distortion in a template.

[0087] Example 8. The apparatus of any of examples 1 to 7, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: form the template from at least one sample of the at least one different block.

[0088] Example 9. The apparatus of any of examples 1 to 8, wherein the distortion in the template comprises a distortion between predicted samples in the template and reconstructed samples in the template, and the computation of the distortion in the template comprises computation of the distortion between the predicted samples in the template and the reconstructed samples in the template.

[0089] Example 10. The apparatus of any of examples 1 to 9, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: form the intra prediction by blending at least two predictors using weights, where the predictors are computed based on determined intra-prediction modes, where the intra prediction modes are determined using a template-based intra mode derivation process applied to the at least onedifferent block, where the template-based intra mode derivation process applied to the at least one different block comprises the computation of the distortion between predicted and reconstructed samples in a template of the at least one different block.

[0090] Example 11. The apparatus of example 10, where the weights used to blend the at least two predictors for the current block are computed based on the distortion.

[0091] Example 12. The apparatus of any of examples 10 to 11, where the weights used to blend the at least two predictors for the current block are computed based on scaling a plurality of the distortions.

[0092] Example 13. The apparatus of example 12, where the scaling operates in accordance to the size of the different block.

[0093] Example 14. The apparatus of any of examples 12 to 13, where the scaling operates in accordance to the size of the current block.

[0094] Example 15. The apparatus of any of examples 10 to 14, where the intra-prediction modes are determined based on the distortions determined using the template-based intra mode derivation process applied to the at least one different block.

[0095] Example 16. The apparatus of example 15, where the intra-prediction modes are determined by selecting the intra-prediction modes resulting in minimum distortions between the predicted samples in the template and the reconstructed samples in the template of the at least one different block.

[0096] Example 17. The apparatus of any of examples 1 to 16, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: extract templatebased intra mode derivation information from a number of different blocks; determine intra prediction modes and weights by combining the extracted template-based intra mode derivation information; and predict the current block using the determined intra prediction modes and weights determined by combining the extracted template-based intra mode derivation information.

[0097] Example 18. The apparatus of any of examples 1 to 17, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: determine adifferent block to be valid based on whether the different block has been encoded using a template-based intra mode derivation mode; and determine a neighboring block to be valid based on whether the neighboring block has been encoded using a template-based intra mode derivation merge mode.

[0098] Example 19. The apparatus of any of examples 1 to 18, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: determine the different block from a number of different blocks based on the size of the different blocks.

[0099] Example 20. The apparatus of any of examples 1 to 19, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: determine the different block from a number of different blocks based on the position of the different blocks with respect to the current block.

[0100] Example 21. The apparatus of any of examples 1 to 20, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: determine the different block from a number of different blocks based on the shape of the different blocks.

[0101] Example 22. The apparatus of any of examples 1 to 21, where the at least one intra prediction mode is determined by combining the intra-prediction information extracted from the at least one different block encoded using a template-based intra-prediction mode derivation process, with intra-prediction information extracted from at least one another different block that is not encoded using a template-based intra-prediction mode derivation process.

[0102] Example 23. An apparatus including: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to: determine intra-prediction information based on computation of a distortion in a template; wherein the intra-prediction information is configured to be used to form an intra prediction for a current block of video data; where the intra-prediction is formed by determining at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting the intra-prediction information from at least one different block, where the intra-prediction information extracted from the at least one different block is determined based on the computation of the distortion in the template; andencode the intra-prediction information as signaling into or along a bitstream.

[0103] Example 24. The apparatus of example 23, wherein the at least one different block comprises at least one neighboring block.

[0104] Example 25. The apparatus of any of examples 23 to 24, wherein the instructions, when executed by the at least one processor, cause the apparatus to encode the intraprediction information as signaling into or along a bitstream based on if at least one different block is encoded using a template based intra mode derivation process.

[0105] Example 26. The apparatus of any of examples 23 to 25, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: determine weights configured to be used to blend at least at least two predictors to form the intra prediction; determine intra prediction modes using a template-based intra mode derivation process applied to the at least one different block, where the template-based intra mode derivation process applied to the at least one different block comprises the computation of the distortion between predicted and reconstructed samples in the template of the at least one different block; and compute the predictors based on the determined intra prediction modes.

[0106] Example 27. The apparatus of example 26, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: compute the weights configured to be used to blend the at least two predictors for the current block based on the distortion.

[0107] Example 28. The apparatus of any of examples 23 to 27, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: determine intra prediction modes and weights by combining template-based intra mode derivation information from a number of different blocks; wherein the intra prediction modes and weights are configured to be used to predict the current block of video data.

[0108] Example 29. A method including: determining at least one intra prediction mode; and forming an intra prediction for a current block of video data by determining the at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting intra-prediction information from at least one different block of video data, wherethe intra-prediction information extracted from the at least one different block is determined based on a computation of a distortion in a template.

[0109] Example 30. A method including: determining intra-prediction information based on computation of a distortion in a template; wherein the intra-prediction information is configured to be used to form an intra prediction for a current block of video data; where the intra-prediction is formed by determining at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting the intra-prediction information from at least one different block, where the intra-prediction information extracted from the at least one different block is determined based on the computation of the distortion in the template; and encoding the intra-prediction information as signaling into or along a bitstream.

[0110] Example 31. An apparatus including: means for determining at least one intra prediction mode; and means for forming an intra prediction for a current block of video data by determining the at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting intra-prediction information from at least one different block of video data, where the intra-prediction information extracted from the at least one different block is determined based on a computation of a distortion in a template.

[0111] Example 32. An apparatus including: means for determining intra-prediction information based on computation of a distortion in a template; wherein the intra-prediction information is configured to be used to form an intra prediction for a current block of video data; where the intra-prediction is formed by determining at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting the intra-prediction information from at least one different block, where the intra-prediction information extracted from the at least one different block is determined based on the computation of the distortion in the template; and means for encoding the intra-prediction information as signaling into or along a bitstream.

[0112] Example 33. A non-transitory program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine for performing operations, the operations including: determining at least one intra prediction mode; and forming an intra prediction for a current block of video data by determining the at least oneintra prediction mode, where the at least one intra prediction mode is determined by extracting intra-prediction information from at least one different block of video data, where the intra-prediction information extracted from the at least one different block is determined based on a computation of a distortion in a template.

[0113] Example 34. A non-transitory program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine for performing operations, the operations including: determining intra-prediction information based on computation of a distortion in a template; wherein the intra-prediction information is configured to be used to form an intra prediction for a current block of video data; where the intra-prediction is formed by determining at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting the intra-prediction information from at least one different block, where the intra-prediction information extracted from the at least one different block is determined based on the computation of the distortion in the template; and encoding the intra-prediction information as signaling into or along a bitstream.

[0114] Example 35. An apparatus including: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to: encode a current block of video data, wherein the encoding of the current block of video data comprises: forming an intra prediction for a current block of video data by determining the at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting intra-prediction information from at least one different block of video data, where the intra-prediction information extracted from the at least one different block is determined based on a computation of a distortion in a template; and signal the encoding of the current block of video data into or along a bitstream.

[0115] Example 36. An apparatus including: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to: form an intra prediction for a current block of video data by determining the at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting intra-prediction information from at least one different block of video data, where the intra-prediction information extracted from the at least one different block is determined based on a computation of a distortion in a template.

[0116] Example 37. A method including: encoding a current block of video data, wherein the encoding of the current block of video data comprises: forming an intra prediction for a current block of video data by determining the at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting intra-prediction information from at least one different block of video data, where the intra-prediction information extracted from the at least one different block is determined based on a computation of a distortion in a template; and signaling the encoding of the current block of video data into or along a bitstream.

[0117] Example 38. A method including: forming an intra prediction for a current block of video data by determining at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting intra-prediction information from at least one different block of video data, where the intra-prediction information extracted from the at least one different block is determined based on a computation of a distortion in a template.

[0118] Example 39. An apparatus including: means for encoding a current block of video data, wherein the encoding of the current block of video data comprises: forming an intra prediction for a current block of video data by determining the at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting intraprediction information from at least one different block of video data, where the intraprediction information extracted from the at least one different block is determined based on a computation of a distortion in a template; and means for signaling the encoding of the current block of video data into or along a bitstream.

[0119] Example 40. An apparatus including: means for forming an intra prediction for a current block of video data by determining at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting intra-prediction information from at least one different block of video data, where the intra-prediction information extracted from the at least one different block is determined based on a computation of a distortion in a template.

[0120] Example 41. A non-transitory program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine for performing operations, the operations including: encoding a current block of video data, wherein theencoding of the current block of video data comprises: forming an intra prediction for a current block of video data by determining the at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting intra-prediction information from at least one different block of video data, where the intra-prediction information extracted from the at least one different block is determined based on a computation of a distortion in a template; and signaling the encoding of the current block of video data into or along a bitstream.

[0121] Example 42. A non-transitory program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine for performing operations, the operations including: forming an intra prediction for a current block of video data by determining at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting intra-prediction information from at least one different block of video data, where the intra-prediction information extracted from the at least one different block is determined based on a computation of a distortion in a template.

[0122] References to a ‘computer’, ‘processor’, etc. should be understood to encompass not only computers having different architectures such as single / multi-processor architectures and sequential / parallel architectures but also specialized circuits such as field- programmable gate arrays (FPGAs), application specific circuits (ASICs), signal processing devices and other processing circuitry. References to computer program, instructions, code etc. should be understood to encompass software for a programmable processor or firmware such as, for example, the programmable content of a hardware device such as instructions for a processor, or configuration settings for a fixed-function device, gate array or programmable logic device, etc.

[0123] As used herein, the term ‘circuitry’, ‘circuit’ and variants may refer to any of the following: (a) hardware circuit implementations, such as implementations in analog and / or digital circuitry, and (b) combinations of circuits and software (and / or firmware), such as (as applicable): (i) a combination of processor(s) or (ii) portions of processor(s) / software including digital signal processor(s), software, and one or more memories that work together to cause an apparatus to perform various functions, and (c) circuits, such as a microprocessor s) or a portion of a microprocessor s), that require software or firmware for operation, even when the software or firmware is not physically present. As a furtherexample, as used herein, the term ‘circuitry’ would also cover an implementation of merely a processor (or multiple processors) or a portion of a processor and its (or their) accompanying software and / or firmware. The term ‘circuitry’ would also cover, for example and when applicable to the particular element, a baseband integrated circuit or applications processor integrated circuit for a mobile phone or a similar integrated circuit in a server, a cellular network device, or another network device. Circuitry or circuit may also be used to mean a function or a process used to execute a method.

[0124] It should be understood that the foregoing description is only illustrative. Various alternatives and modifications may be devised by those skilled in the art. For example, features recited in the various dependent claims could be combined with each other in any suitable combination(s). In addition, features from different embodiments described above could be selectively combined into a new embodiment. Accordingly, the description is intended to embrace all such alternatives, modifications and variances which fall within the scope of the appended claims.

[0125] The following acronyms and abbreviations that may be found in the specification and / or the drawing figures are defined as follows (the abbreviations may be appended with each other or with other characters using e.g. a hyphen or dash (-), and may be case insensitive):2D two-dimensional3D three-dimensional3GPP 3rd generation partnership project4G fourth generation of broadband cellular network technology5G fifth generation cellular network technology802.x family of IEEE standards dealing with local area networks and metropolitan area networksASIC application specific integrated circuitCDMA code-division multiple accessCPU central processing unitDC direct currentDCT discrete cosine transformDIMD decoder-side intra mode derivationDSP digital signal processorECM enhanced compression modelFDMA frequency division multiple accessFPGA field programmable gate arrayGSM global system for mobile communicationsH.222.0 MPEG-2 systems, standard for the generic coding of moving pictures and associated audio informationH.2xx family of video coding standards in the domain of the ITU-T (e.g.H.263, H.264, H.266)HMD head-mounted displayIEC International Electrotechnical CommissionIEEE Institute of Electrical and Electronics EngineersI / F interfaceIMD integrated messaging deviceIMS instant messaging serviceI / O input / output loT internet of thingsIP internet protocolISO International Organization for StandardizationISOBMFF ISO base media file formatITU International Telecommunication UnionITU-T ITU Telecommunication Standardization SectorLTE long-term evolutionMMS multimedia messaging serviceMPEG-2 moving picture experts group, H.222 / H.262 as defined by the ITUMPM most probable modeMRL multiple reference lineNAL network abstraction layerN / W networkPC personal computerPDA personal digital assistantPID packet identifierPLC power line communicationRAM random access memoryRFID radio frequency identificationRFM reference frame memory SATD sum of absolute transformed differencesSMS short messaging serviceSON self-organizing / optimizing networkTCP-IP transmission control protocol-internet protocolTDMA time divisional multiple access TIMD template-based intra mode derivationTS transport streamTV televisionUI user interfaceUICC universal integrated circuit card UMTS universal mobile telecommunications systemUSB universal serial bus

Claims

CLAIMSWhat is claimed is:

1. An apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to: determine at least one intra prediction mode; and form an intra prediction for a current block of video data by determining the at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting intra-prediction information from at least one different block of video data, where the intra-prediction information extracted from the at least one different block is determined based on a computation of a distortion in a template.

2. The apparatus of claim 1, wherein the at least one different block comprises at least one neighboring block.

3. The apparatus according to any one of claims 1-2, wherein the apparatus operates based on signaling.

4. The apparatus of claim 3, where the presence of signaling is inferred based on whether at least one block of the at least one different block is encoded using a template based intra mode derivation process.

5. The apparatus of any of claims 1 to 4, where the at least one different block is encoded using a template-based intra-prediction mode derivation process.

6. The apparatus of claim 5, wherein the at least one different block is determined to be encoded using the template-based intra mode derivation process when the at least onedifferent block has been encoded based on: determining at least one intra prediction mode, and forming an intra prediction for the at least one different block of video data by determining the at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting intra-prediction information from at least one other different block, where the intra-prediction information extracted from the at least one other different block is determined based on a computation of a distortion in a template.

7. The apparatus of any of claims 5 to 6, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: determine that the at least one different block is encoded using the templatebased intra mode derivation process, when the at least one different block has been encoded based on: determining at least one intra prediction mode, and forming an intra prediction for the at least one different block of video data by determining the at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting intra-prediction information from at least one other different block, where the intra-prediction information extracted from the at least one other different block is determined based on a computation of a distortion in a template.

8. The apparatus of any of claims 1 to 7, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: form the template from at least one sample of the at least one different block.

9. The apparatus of any of claims 1 to 8, wherein the distortion in the template comprises a distortion between predicted samples in the template and reconstructed samples in the template, and the computation of the distortion in the template comprises computation of the distortion between the predicted samples in the template and the reconstructed samples in the template.

10. The apparatus of any of claims 1 to 9, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: form the intra prediction by blending at least two predictors using weights,where the predictors are computed based on determined intra-prediction modes, where the intra prediction modes are determined using a template-based intra mode derivation process applied to the at least one different block, where the template-based intra mode derivation process applied to the at least one different block comprises the computation of the distortion between predicted and reconstructed samples in a template of the at least one different block.

11. The apparatus of claim 10, where the weights used to blend the at least two predictors for the current block are computed based on the distortion.

12. The apparatus of any of claims 10 to 11, where the weights used to blend the at least two predictors for the current block are computed based on scaling a plurality of the distortions.

13. The apparatus of claim 12, where the scaling operates in accordance to the size of the different block.

14. The apparatus of any of claims 12 to 13, where the scaling operates in accordance to the size of the current block.

15. The apparatus of any of claims 10 to 14, where the intra-prediction modes are determined based on the distortions determined using the template-based intra mode derivation process applied to the at least one different block.

16. The apparatus of claim 15, where the intra-prediction modes are determined by selecting the intra-prediction modes resulting in minimum distortions between the predicted samples in the template and the reconstructed samples in the template of the at least one different block.

17. The apparatus of any of claims 1 to 16, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: extract template-based intra mode derivation information from a number of different blocks; determine intra prediction modes and weights by combining the extractedtemplate-based intra mode derivation information; and predict the current block using the determined intra prediction modes and weights determined by combining the extracted template-based intra mode derivation information.

18. The apparatus of any of claims 1 to 17, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: determine a different block to be valid based on whether the different block has been encoded using a template-based intra mode derivation mode; and determine a neighboring block to be valid based on whether the neighboring block has been encoded using a template-based intra mode derivation merge mode.

19. The apparatus of any of claims 1 to 18, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: determine the different block from a number of different blocks based on the size of the different blocks.

20. The apparatus of any of claims 1 to 19, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: determine the different block from a number of different blocks based on the position of the different blocks with respect to the current block.

21. The apparatus of any of claims 1 to 20, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: determine the different block from a number of different blocks based on the shape of the different blocks.

22. The apparatus of any of claims 1 to 21, where the at least one intra prediction mode is determined by combining the intra-prediction information extracted from the at least one different block encoded using a template-based intra-prediction mode derivationprocess, with intra-prediction information extracted from at least one another different block that is not encoded using a template-based intra-prediction mode derivation process.

23. An apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to: determine intra-prediction information based on computation of a distortion in a template; wherein the intra-prediction information is configured to be used to form an intra prediction for a current block of video data; where the intra-prediction is formed by determining at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting the intraprediction information from at least one different block, where the intra-prediction information extracted from the at least one different block is determined based on the computation of the distortion in the template; and encode the intra-prediction information as signaling into or along a bitstream.

24. The apparatus of claim 23, wherein the at least one different block comprises at least one neighboring block.

25. The apparatus of any of claims 23 to 24, wherein the instructions, when executed by the at least one processor, cause the apparatus to encode the intra-prediction information as signaling into or along a bitstream based on if at least one different block is encoded using a template based intra mode derivation process.

26. The apparatus of any of claims 23 to 25, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to:determine weights configured to be used to blend at least at least two predictors to form the intra prediction; determine intra prediction modes using a template-based intra mode derivation process applied to the at least one different block, where the template-based intra mode derivation process applied to the at least one different block comprises the computation of the distortion between predicted and reconstructed samples in the template of the at least one different block; and compute the predictors based on the determined intra prediction modes.

27. The apparatus of claim 26, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: compute the weights configured to be used to blend the at least two predictors for the current block based on the distortion.

28. The apparatus of any of claims 23 to 27, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: determine intra prediction modes and weights by combining template-based intra mode derivation information from a number of different blocks; wherein the intra prediction modes and weights are configured to be used to predict the current block of video data.

29. A method comprising: determining at least one intra prediction mode; and forming an intra prediction for a current block of video data by determining the at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting intra-prediction information from at least one different block of video data, where the intra-prediction information extracted from the at least one different block is determined based on a computation of a distortion in a template.

30. A method comprising: determining intra-prediction information based on computation of a distortion in a template; wherein the intra-prediction information is configured to be used to form an intra prediction for a current block of video data; where the intra-prediction is formed by determining at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting the intraprediction information from at least one different block, where the intra-prediction information extracted from the at least one different block is determined based on the computation of the distortion in the template; and encoding the intra-prediction information as signaling into or along a bitstream.

31. An apparatus comprising: means for determining at least one intra prediction mode; and means for forming an intra prediction for a current block of video data by determining the at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting intra-prediction information from at least one different block of video data, where the intra-prediction information extracted from the at least one different block is determined based on a computation of a distortion in a template.

32. An apparatus comprising: means for determining intra-prediction information based on computation of a distortion in a template; wherein the intra-prediction information is configured to be used to form an intra prediction for a current block of video data; where the intra-prediction is formed by determining at least one intra predictionmode, where the at least one intra prediction mode is determined by extracting the intraprediction information from at least one different block, where the intra-prediction information extracted from the at least one different block is determined based on the computation of the distortion in the template; and means for encoding the intra-prediction information as signaling into or along a bitstream.

33. A non-transitory program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine for performing operations, the operations comprising: determining at least one intra prediction mode; and forming an intra prediction for a current block of video data by determining the at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting intra-prediction information from at least one different block of video data, where the intra-prediction information extracted from the at least one different block is determined based on a computation of a distortion in a template.

34. A non-transitory program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine for performing operations, the operations comprising: determining intra-prediction information based on computation of a distortion in a template; wherein the intra-prediction information is configured to be used to form an intra prediction for a current block of video data; where the intra-prediction is formed by determining at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting the intraprediction information from at least one different block, where the intra-prediction information extracted from the at least one different block is determined based on the computation of the distortion in the template; andencoding the intra-prediction information as signaling into or along a bitstream.

35. An apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to: encode a current block of video data, wherein the encoding of the current block of video data comprises: forming an intra prediction for a current block of video data by determining the at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting intra-prediction information from at least one different block of video data, where the intra-prediction information extracted from the at least one different block is determined based on a computation of a distortion in a template; and signal the encoding of the current block of video data into or along a bitstream.

36. An apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to: form an intra prediction for a current block of video data by determining the at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting intra-prediction information from at least one different block of video data, where the intra-prediction information extracted from the at least one different block is determined based on a computation of a distortion in a template.

37. A method comprising: encoding a current block of video data, wherein the encoding of the currentblock of video data comprises: forming an intra prediction for a current block of video data by determining the at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting intra-prediction information from at least one different block of video data, where the intra-prediction information extracted from the at least one different block is determined based on a computation of a distortion in a template; and signaling the encoding of the current block of video data into or along a bitstream.

38. A method comprising: forming an intra prediction for a current block of video data by determining at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting intra-prediction information from at least one different block of video data, where the intra-prediction information extracted from the at least one different block is determined based on a computation of a distortion in a template.

39. An apparatus comprising: means for encoding a current block of video data, wherein the encoding of the current block of video data comprises: forming an intra prediction for a current block of video data by determining the at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting intra-prediction information from at least one different block of video data, where the intra-prediction information extracted from the at least one different block is determined based on a computation of a distortion in a template; and means for signaling the encoding of the current block of video data into or along a bitstream.

40. An apparatus comprising: means for forming an intra prediction for a current block of video data by determining at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting intra-prediction information from at least one different block of video data, where the intra-prediction information extracted from the at least one different block is determined based on a computation of a distortion in a template.

41. A non-transitory program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine for performing operations, the operations comprising: encoding a current block of video data, wherein the encoding of the current block of video data comprises: forming an intra prediction for a current block of video data by determining the at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting intra-prediction information from at least one different block of video data, where the intra-prediction information extracted from the at least one different block is determined based on a computation of a distortion in a template; and signaling the encoding of the current block of video data into or along a bitstream.

42. A non-transitory program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine for performing operations, the operations comprising: forming an intra prediction for a current block of video data by determining at least one intra prediction mode, where the at least one intra prediction mode is determined by extracting intra-prediction information from at least one different block of video data, where the intra-prediction information extracted from the at least one different block is determined based on a computation of a distortion in a template.

Images