Template-based intra mode derivation from proximate decoded reference samples

Abstract

A method and apparatus for encoding or decoding a video are provided. At least one derived intra prediction mode for a video block is obtained using template-based intra prediction mode derivation. Template-based intra prediction mode derivation includes predicting a template of a video block based on a first intra prediction mode using reference samples of the template of the video block. A template of a video block includes a top portion above the video block and a left portion left of the video block. A first set of reference samples is used based on a top portion of the first intra prediction mode prediction template and a second set of reference samples is used based on a left portion of the first intra prediction mode prediction template, the first set of reference samples being different from the second set of reference samples. A block predictor for the video block is obtained based on the at least one derived intra prediction mode, and the video block is decoded based on the block predictor.

Description

[0001] This application claims priority to European application number 23305549.0, filed on 12 April 2023, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This embodiment generally relates to video compression. This embodiment relates to methods and apparatus for encoding or decoding images or videos. More specifically, this embodiment relates to improved template-based intra-frame prediction. Background Technology

[0003] To achieve high compression efficiency, image and video coding schemes typically employ prediction and transform to utilize spatial and temporal redundancy in video content. Intra-frame or inter-frame prediction is usually used to develop intra-frame or inter-frame image correlations, followed by transforming, quantizing, and entropy coding of the difference between the original block and the predicted block (typically represented as prediction error or prediction residual). In inter-frame prediction, the motion vectors used for motion compensation are typically predicted from a motion vector predictor. To reconstruct the video, the compressed data is decoded through the inverse process corresponding to entropy coding, quantization, transform, and prediction. Summary of the Invention

[0004] According to one aspect, a method for decoding video is provided. The method includes: obtaining at least one derived intra-prediction mode for a video block using template-based intra-prediction mode derivation, wherein the template-based intra-prediction mode derivation includes predicting a template of the video block based on a first intra-prediction mode using reference samples of the template, wherein the template of the video block includes a top portion above the video block and a left portion to the left of the video block, and wherein the top portion of the template is predicted based on the first intra-prediction mode using a first set of reference samples, and the left portion of the template is predicted based on the first intra-prediction mode using a second set of reference samples, the first set of reference samples and the second set of reference samples being different; obtaining a block predictor for the video block based on the at least one derived intra-prediction mode; and decoding the video block based on the block predictor.

[0005] According to another aspect, an apparatus for decoding video is provided. The apparatus includes one or more processors operable to: obtain at least one derived intra-prediction mode for a video block using template-based intra-prediction mode derivation, wherein the template-based intra-prediction mode derivation includes predicting a template of the video block based on a first intra-prediction mode using reference samples of the template, wherein the template of the video block includes a top portion above the video block and a left portion to the left of the video block, and wherein the top portion of the template is predicted based on the first intra-prediction mode using a first set of reference samples, and the left portion of the template is predicted based on the first intra-prediction mode using a second set of reference samples, the first set of reference samples and the second set of reference samples being different; obtain a block predictor for the video block based on the at least one derived intra-prediction mode; and decode the video block based on the block predictor.

[0006] According to one aspect, a method for encoding video is provided. The method includes: obtaining at least one derived intra-prediction mode for a video block using template-based intra-prediction mode derivation, wherein the template-based intra-prediction mode derivation includes predicting a template of the video block based on a first intra-prediction mode using reference samples of the template, wherein the template of the video block includes a top portion above the video block and a left portion to the left of the video block, and wherein the top portion of the template is predicted based on the first intra-prediction mode using a first set of reference samples, and the left portion of the template is predicted based on the first intra-prediction mode using a second set of reference samples, the first set of reference samples and the second set of reference samples being different; obtaining a block predictor for the video block based on the at least one derived intra-prediction mode; and encoding the video block based on the block predictor.

[0007] According to another aspect, an apparatus for encoding video is provided. The apparatus includes one or more processors operable to: obtain at least one derived intra-prediction mode for a video block using template-based intra-prediction mode derivation, wherein the template-based intra-prediction mode derivation includes predicting a template of the video block based on a first intra-prediction mode using reference samples of the template, wherein the template of the video block includes a top portion above the video block and a left portion to the left of the video block, and wherein the top portion of the template is predicted based on the first intra-prediction mode using a first set of reference samples, and the left portion of the template is predicted based on the first intra-prediction mode using a second set of reference samples, the first set of reference samples and the second set of reference samples being different; obtain a block predictor for the video block based on the at least one derived intra-prediction mode; and encode the video block based on the block predictor.

[0008] This document describes other embodiments that can be used alone or in combination.

[0009] One or more embodiments also provide a computer program including instructions that, when executed by one or more processors, cause the one or more processors to perform a method for encoding / decoding video according to any embodiment described herein. One or more of these embodiments also provide a non-transitory computer-readable medium and / or a computer-readable storage medium storing instructions thereon for encoding / decoding video according to the method described herein.

[0010] One or more embodiments also provide a computer-readable storage medium storing a bit stream generated according to the method described herein. One or more embodiments also provide methods and apparatus for transmitting or receiving a bit stream generated according to the described method. Attached Figure Description

[0011] Figure 1 The diagram illustrates a block diagram of a system in which various aspects of this embodiment can be implemented.

[0012] Figure 2 A block diagram illustrating an embodiment of a video encoder in which various aspects of this embodiment can be implemented is shown.

[0013] Figure 3 A block diagram illustrating an embodiment of a video decoder in which various aspects of this embodiment can be implemented is shown.

[0014] Figure 4A and Figure 4B The illustration shows an example of the angle intra-frame prediction mode in VVC.

[0015] Figure 5A and Figure 5B The illustration shows the relationship between the breadth of the set of decoded reference samples around the WxH block to be predicted and the range of allowed intra-frame prediction angles.

[0016] Figure 6 The illustration shows an example of an angled mode that has been replaced by a wide-angle mode for non-square blocks whose width is strictly greater than their height in VVC and ECM.

[0017] Figure 7A The illustration shows an example of a template for a block to be encoded or decoded, and a decoded reference sample for that template.

[0018] Figure 7B The illustration shows an example of the range of permissible directional intra-prediction modes for a block of width × height during the TIMD derivation step in the case of TIMD utilizing a wide-angle intra-prediction mode.

[0019] Figure 8 An example of adapting a set of reference samples for a block whose prediction is to be derived using a template-based intra-frame prediction mode, according to an embodiment, is illustrated.

[0020] Figure 9 The illustration shows another example of an adaptation of a set of reference samples for a block whose prediction is to be derived using a template-based intra-frame prediction mode, according to an embodiment.

[0021] Figure 10 The illustration shows an example of adapting a set of reference samples for a block whose prediction is to be derived using a template-based intra-frame prediction mode, according to an embodiment, when the left side of the template is unavailable.

[0022] Figure 11 The illustration shows an example of adapting a set of reference samples for a block whose prediction is to be derived using a template-based intra-frame prediction mode, according to an embodiment, when the top portion of the template is unavailable.

[0023] Figure 12 An example of adapting a set of reference samples for a block whose prediction is to be derived using a template-based intra-frame prediction mode, according to another embodiment, is illustrated.

[0024] Figure 13 An example of adapting a set of reference samples for a block whose prediction is to be derived using a template-based intra-frame prediction mode, according to another embodiment, is illustrated.

[0025] Figure 14 An example of adapting a set of reference samples for a block whose prediction is to be derived using a template-based intra-frame prediction mode, according to another embodiment, is illustrated.

[0026] Figure 15 An example of adapting a set of reference samples for a block whose prediction is to be derived using a template-based intra-frame prediction mode, according to another embodiment, is illustrated.

[0027] Figure 16 An example of adapting a set of reference samples for a block whose prediction is to be derived using a template-based intra-frame prediction mode, according to another embodiment, is illustrated.

[0028] Figure 17 An example of a method for encoding video blocks according to an embodiment is illustrated.

[0029] Figure 18 An example of a method for decoding video blocks according to an embodiment is illustrated.

[0030] Figure 19 An example of a template-based intra-frame pattern prediction derivation method for encoding or decoding video blocks, according to an embodiment, is illustrated.

[0031] Figure 20 An example of a template-based intra-frame pattern prediction derivation method for encoding or decoding video blocks, according to another embodiment, is illustrated.

[0032] Figure 21 The illustration shows a block diagram of a system according to another embodiment in which aspects of this embodiment can be implemented.

[0033] Figure 22 The illustration shows two remote devices communicating via a communication network, based on an example of this principle.

[0034] Figure 23 The syntax of an example signal based on this principle is shown. Detailed Implementation

[0035] This application describes various aspects, including tools, features, embodiments, models, methods, etc. Many of these aspects are described in detail and, at least to illustrate individual characteristics, are generally described in a manner that may sound restrictive. However, this is for the purpose of clarity and does not limit the application or scope of these aspects. In fact, all the different aspects can be combined and interchanged to provide other aspects. Furthermore, these aspects can also be combined and interchanged with aspects described in previous documents.

[0036] The aspects described and envisioned in this application can be realized in many different forms. The following... Figure 1 , 2 Sections 3 and 4 provide some embodiments, but other embodiments are conceivable. Figure 1 , 2The discussion in section 3 does not limit the breadth of implementations. At least one aspect generally relates to video encoding and decoding, and at least one other aspect generally relates to the transmission of generated or encoded bitstreams. These and other aspects can be implemented as methods, apparatus, computer-readable storage media having instructions stored thereon for encoding or decoding video data according to any of the described methods, and / or computer-readable storage media having a bitstream generated according to any of the described methods stored thereon.

[0037] In this application, the terms “reconstruction” and “decoding” are used interchangeably, the terms “pixel” and “sample” are used interchangeably, and the terms “image”, “picture” and “frame” are used interchangeably.

[0038] This document describes various methods, and each method includes one or more steps or actions for implementing the described method. Unless the correct operation of the method requires a specific order of steps or actions, the order and / or use of specific steps and / or actions can be modified or combined. Additionally, terms such as "first," "second," etc., may be used in various embodiments to modify elements, components, steps, operations, etc., such as, for example, "first decoding" and "second decoding." Unless specifically required, the use of these terms does not imply an ordering of the modified operations. Therefore, in this example, the first decoding need not be performed before the second decoding, and may occur, for example, before, during, or in a time period overlapping with the second decoding.

[0039] This aspect is not limited to VVC or HEVC, and can be applied to, for example, other standards and recommendations, whether pre-existing or developed in the future, as well as extensions of any such standards and recommendations (including VVC and HEVC). Unless otherwise stated or technically excluded, the aspects described in this application may be used individually or in combination.

[0040] Figure 1 illustrates a block diagram of an example system that can implement various aspects and embodiments. System 100 may be embodied as a device including the various components described below and configured to perform one or more aspects described in this application. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, networked home appliances, and servers. The elements of system 100 may be embodied individually or in combination in a single integrated circuit, multiple ICs, and / or discrete components. For example, in at least one embodiment, the processing and encoder / decoder elements of system 100 are distributed across multiple ICs and / or discrete components. In various embodiments, system 100 is communicatively coupled to other systems or other electronic devices via, for example, a communication bus or through dedicated input and / or output ports. In various embodiments, system 100 is configured to implement one or more aspects described in this application.

[0041] System 100 includes at least one processor 110 configured to execute instructions loaded thereon to implement various aspects described herein, such as those described. Processor 110 may include embedded memory, input / output interfaces, and various other circuitry known in the art. System 100 includes at least one memory 120 (e.g., a volatile memory device and / or a non-volatile memory device). System 100 includes a storage device 140, which may include non-volatile memory and / or volatile memory, including but not limited to EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash memory, disk drives, and / or optical disk drives. As a non-limiting example, storage device 140 may include internal storage devices, attached storage devices, and / or network-accessible storage devices.

[0042] System 100 includes an encoder / decoder module 130, which is configured to, for example, process data to provide encoded or decoded video, and the encoder / decoder module 130 may include its own processor and memory. Encoder / decoder module 130 represents one or more modules that may be included in a device to perform encoding and / or decoding functions. As is known, a device may include one or both of the encoding and decoding modules. Furthermore, encoder / decoder module 130 may be implemented as a separate element of system 100, or it may be incorporated into processor 110 as a combination of hardware and software known to those skilled in the art.

[0043] Program code to be loaded into processor 110 or encoder / decoder 130 to execute the various aspects described in this application may be stored in storage device 140 and subsequently loaded onto memory 120 for execution by processor 110. According to various embodiments, one or more of processor 110, memory 120, storage device 140, and encoder / decoder module 130 may store one or more different items during the execution of the processes described in this application. These stored items may include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results of equations, formulas, operations, and arithmetic logic processing.

[0044] In some embodiments, the memory within processor 110 and / or encoder / decoder module 130 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, external memory (e.g., processor 110 or encoder / decoder module 130) is used for one or more of these functions. External memory may be memory 120 and / or storage device 140, such as volatile memory and / or non-volatile flash memory. In several embodiments, external non-volatile flash memory is used to store the television's operating system. In at least one embodiment, a fast external dynamic volatile memory (such as RAM) is used as working memory for video encoding and decoding operations, such as for MPEG-2 (MPEG stands for Moving Picture Experts Group; MPEG-2 is also known as ISO / IEC 13818, and 13818-1 is also known as H.222, and 13818-2 is also known as H.262), HEVC (HEVC stands for High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2), or VVC (Various Video Coding, a standard developed by JVET (Joint Video Experts Group)).

[0045] Inputs to the various components of system 100 may be provided by a variety of input devices indicated in block 105. Such input devices include, but are not limited to: (i) an RF section that receives radio frequency (RF) signals, for example, wirelessly transmitted by a broadcasting company; (ii) component input terminals (or a set of COMP input terminals); (iii) universal serial bus (USB) input terminals; and / or (iv) high-definition multimedia interface (HDMI) input terminals. Figure 1 Other examples not shown include composite video.

[0046] In various embodiments, the input device of block 105 has been associated with corresponding input processing elements known in the art. For example, the RF section may be associated with elements suitable for: (i) selecting a desired frequency (also known as selecting a signal or limiting a signal band to a certain band); (ii) down-converting the selected signal; (iii) further limiting the band to a narrower band to select, for example, a signal band (which may be referred to as a channel in some embodiments); (iv) demodulating the down-converted and band-limited signal; (v) performing error correction; and (vi) performing demultiplexing to select a desired data packet stream. The RF section in various embodiments includes one or more elements performing these functions, such as frequency selectors, signal selectors, band limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF section may include a tuner that performs various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., intermediate frequency or near-baseband frequency) or down-converting it to baseband. In one set-top box embodiment, the RF section and its associated input processing elements receive RF signals transmitted via a wired (e.g., cable) medium and perform frequency selection by filtering, down-converting, and re-filtering to the desired frequency band. Various embodiments may rearrange the order of the above-described (and other) elements, remove some of these elements, and / or add other elements that perform similar or different functions. Adding elements may include inserting elements between existing elements, such as inserting amplifiers and analog-to-digital converters. In various embodiments, the RF section includes an antenna.

[0047] Furthermore, the USB and / or HDMI terminals may include corresponding interface processors for connecting system 100 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Reed-Solomon error correction) may be implemented as needed, for example, within a separate input processing IC or within processor 110. Similarly, various aspects of USB or HDMI interface processing may be implemented as needed, either within a separate interface IC or within processor 110. The demodulated, error-corrected, and demultiplexed streams are provided to various processing elements, including, for example, processor 110 and encoder / decoder 130, which cooperate with memory and storage elements to process the data streams as needed for presentation on the output device.

[0048] The various components of system 100 can be provided within an integrated housing. Within this integrated housing, the various components can be interconnected and transmit data therebetween using suitable connection means 115, such as internal buses known in the art, including I2C buses, lines, and printed circuit boards.

[0049] System 100 includes a communication interface 150, which enables communication with other devices via a communication channel 190. The communication interface 150 may include, but is not limited to, a transceiver configured to transmit and receive data via the communication channel 190. The communication interface 150 may include, but is not limited to, a modem or network interface card (NIC), and the communication channel 190 may be implemented, for example, in a wired and / or wireless medium.

[0050] In various embodiments, data is streamed to system 100 using a Wi-Fi network (such as IEEE 802.11, IEEE stands for Institute of Electrical and Electronics Engineers). In these embodiments, the Wi-Fi signal is received via a communication channel 190 and a communication interface 150 suitable for Wi-Fi communication. The communication channel 190 in these embodiments is typically connected to an access point or router that provides access to external networks (including the Internet) to allow streaming applications and other OTT (over-the-top) communications. Other embodiments use a set-top box to provide streaming data to system 100, delivering data via an HDMI connection of input block 105. Still other embodiments use an RF connection of input block 105 to provide streaming data to system 100. As described above, various embodiments provide data in a non-streaming manner. Additionally, various embodiments use wireless networks other than Wi-Fi, such as cellular networks or Bluetooth networks.

[0051] System 100 can provide output signals to various output devices, including a display 165, a speaker 175, and other peripheral devices 185. The display 165 in various embodiments includes one or more of, for example, a touchscreen display, an organic light-emitting diode (OLED) display, a flexible display, and / or a foldable display. The display 165 can be used in a television, tablet computer, laptop, cellular phone (mobile phone), or other device. The display 165 can also be integrated with other components (e.g., as in a smartphone) or separate (e.g., an external monitor for a laptop). In various examples of embodiments, other peripheral devices 185 include one or more of a standalone digital video disc (or digital multifunction disc) (these two terms are collectively referred to as DVR), a disc player, a stereo system, and / or a lighting system. Various embodiments utilize one or more peripheral devices 185 that provide functionality based on the output of system 100. For example, a disc player performs the function of playing the output of system 100.

[0052] In various embodiments, signal transmission, such as AV.link, CEC, or other communication protocols that enable device-to-device control with or without user intervention, is used to transmit control signals between system 100 and display 165, speaker 175, or other peripheral devices 185. Output devices may be communicatively coupled to system 100 via dedicated connections through corresponding interfaces 160, 170, and 180. Alternatively, output devices may be connected to system 100 via communication interface 150 using communication channel 190. Display 165 and speaker 175 may be integrated into a single unit along with other components of system 100 in an electronic device, such as a television. In various embodiments, display interface 160 includes a display driver, such as a timing controller (TCon) chip.

[0053] Alternatively, the display 165 and speaker 175 may be separate from one or more other components, for example, in the case where the RF portion of input 105 is part of a separate set-top box. In various embodiments where the display 165 and speaker 175 are external components, the output signal may be provided via a dedicated output connection, which may include, for example, an HDMI port, a USB port, or a COMP output.

[0054] The embodiments can be executed by computer software implemented by processor 110, by hardware, or by a combination of hardware and software. As a non-limiting example, the embodiments can be implemented by one or more integrated circuits. Memory 120 can be of any type suitable for the technical environment and can be implemented using any suitable data storage technology, such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory, as non-limiting examples. As a non-limiting example, processor 110 can be of any type suitable for the technical environment and can include one or more of microprocessors, general-purpose computers, special-purpose computers, and processors based on multi-core architectures.

[0055] Figure 2 A video encoder 200 is illustrated. Variations of this encoder 200 are envisioned, but for clarity, encoder 200 is described below, without describing all anticipated variations.

[0056] In some embodiments, Figure 2 The illustration also shows encoders that improve upon the HEVC or VVC standards, or encoders that employ technologies similar to HEVC or VVC, such as the ECM encoder developed by JVET (Joint Video Exploration Group).

[0057] Before being encoded, the video sequence may undergo pre-coding processing (201), such as applying color transformations to the input color image (e.g., conversion from RGB 4:4:4 to YCbCr 4:2:0), performing remapping of the input image components to obtain a signal distribution more resilient to compression (e.g., using histogram equalization of the color components), or resizing the image (e.g., downsizing). Metadata may be associated with pre-processing and attached to the bitstream.

[0058] In encoder 200, the image is encoded by encoder elements as described below. The image to be encoded is segmented (202) and processed in units such as CU (coding unit) or block. In this disclosure, different expressions may be used to refer to such units or blocks resulting from the segmentation of the image. This terminology may be coding unit or CU, coding block or CB, luminance CB, block or video block. CTU (coding tree unit) may refer to a set of blocks or a set of units. In some embodiments, CTU may be considered as a block or unit itself.

[0059] Each unit is encoded using, for example, intra-frame or inter-frame modes. When a unit is encoded in intra-frame mode, it performs intra-frame prediction (260). In inter-frame mode, motion estimation (275) and compensation (270) are performed. The encoder determines (205) which of the intra-frame or inter-frame modes is used to encode the unit and indicates the intra-frame / inter-frame decision by, for example, a prediction mode flag. The encoder can also mix (263) intra-frame prediction results and inter-frame prediction results, or mix results from different intra-frame / inter-frame prediction methods. For example, prediction residuals are calculated by subtracting (210) the prediction block from the original image block.

[0060] The motion refinement module (272) uses an already available reference image to refine the motion field of a block without referencing the original block. The motion field of a region can be considered as the set of motion vectors of all pixels within that region. If the motion vectors are based on sub-blocks, the motion field can also be represented as the set of motion vectors of all sub-blocks within that region (all pixels within a sub-block have the same motion vector, and the motion vectors can vary between sub-blocks). If a single motion vector is used for the region, the motion field of that region can also be represented by a single motion vector (the same motion vector of all pixels in that region).

[0061] The predicted residual is then transformed (225) and quantized (230). The quantized transform coefficients, along with the motion vector and other syntax elements, are entropy-encoded (245) to output a bitstream. The encoder can skip the transform and apply the quantization directly to the untransformed residual signal. The encoder can bypass both the transform and quantization, i.e., directly encode the residual without applying the transform or quantization process.

[0062] The encoder decodes the coded block to provide a reference for further prediction. The quantized transform coefficients are dequantized (240) and inverse transformed (250) to decode the prediction residual. The decoded prediction residual and the prediction block are combined (255) to reconstruct the image block. A loop filter (265) is applied to the reconstructed image to perform, for example, deblocking / SAO (Sample Adaptive Shift) filtering to reduce coded artifacts. The filtered image is stored at the reference image buffer (280).

[0063] Figure 3 A block diagram of a video decoder 300 is shown. In decoder 300, the bitstream is decoded by decoder elements, as described below. Video decoder 300 typically performs... Figure 2 The decoding rounds are the inverse of the encoding rounds described herein. Encoder 200 typically also performs video decoding as part of the encoded video data.

[0064] Specifically, the input to the decoder includes a video bitstream that can be generated by the video encoder 200. The bitstream is first entropy decoded (330) to obtain transform coefficients, motion vectors, and other encoded information. Image segmentation information indicates how the image is segmented. Therefore, the decoder can segment (335) the image based on the decoded image segmentation information. The transform coefficients are dequantized (340) and inverse transformed (350) to decode the prediction residuals. The decoded prediction residuals and prediction blocks are combined (355) to reconstruct image blocks. Prediction blocks (370) can be obtained from intra-frame prediction (360) or motion-compensated prediction (i.e., inter-frame prediction) (375). A loop filter (365) is applied to the reconstructed image. The filtered image is stored at a reference image buffer (380).

[0065] The (370) prediction block can be obtained from intra-frame prediction (360) or motion-compensated prediction (i.e., inter-frame prediction) (375). The decoder can mix (373) the intra-frame prediction results and the inter-frame prediction results, or mix the results from multiple intra-frame / inter-frame prediction methods. Before motion compensation, the motion field can be refined (372) by using an already available reference image. An in-loop filter (365) is applied to the reconstructed image. The filtered image is stored at the reference image buffer (380).

[0066] The decoded image can also undergo post-decoding processing (385), such as inverse color transformation (e.g., conversion from YCbCr 4:2:0 to RGB 4:4:4) or inverse remapping, which performs the reverse of the remapping process performed in the pre-encoding process (201), or resizing the reconstructed image (e.g., enlarging). Post-decoding processing can use metadata derived in the pre-encoding process and signaled in the bitstream.

[0067] The embodiments described herein relate to intra-frame prediction, and more specifically to template-based intra-frame mode derivation (TIMD).

[0068] Any of the embodiments described herein can be implemented, for example, in the intra-prediction module 260 of the video encoder 200 or the intra-prediction module 360 ​​of the video decoder 300.

[0069] To capture arbitrary edge orientations presented in natural video, the number of directional intra-prediction modes in VVC is expanded from 33 used in HEVC to 65. New directional modes not in HEVC are also included. Figure 4A The image is depicted as dotted arrows. These denser directional intra-prediction modes are applicable to all block sizes and for both luma and chroma intra-prediction. From HEVC to VVC, the planar and DC modes remain unchanged, except for the following minor modifications. In HEVC, each intra-coded block has a square shape, and the length of each side is a power of 2. Therefore, division is not required to generate the intra-predictor using DC. In VVC, blocks can have rectangular shapes, which in general requires division per block. To avoid division for DC prediction, only the longer side is used to calculate the average of non-square blocks.

[0070] In ECM, the core structure of the 67 intra-prediction modes is inherited from the core structure of VVC. This core structure is refined in ECM as follows: the four-tap interpolation of the directional intra-prediction modes from VVC is changed to six-tap interpolation, and the position-dependent intra-prediction combination (PDPC) is supplemented with gradient PDPC.

[0071] In VVC and ECM, for non-square blocks, several regular angular intra-prediction modes are replaced by wide-angle modes. The replaced modes are signaled using the original method and remapped to the wide-angle mode index after resolution. The total number of core intra-prediction modes remains unchanged at 67. The angular and wide-angle modes in VVC... Figure 4B The diagram in the middle illustrates this. For the current WxH block to be predicted, Figure 5A and Figure 5B The diagram illustrates a set of decoded reference samples consisting of an array of top decoded reference samples of length 2W+1 and an array of left decoded reference samples of length 2H+1. Figure 5A and Figure 5BThe relationship between the breadth of decoded reference samples around the current WxH block and the range of allowed intra-prediction angles is also shown. The number of modes replaced in wide-angle directional mode depends on the block's aspect ratio. Table 1 below presents an example of the indices of intra-prediction modes replaced by wide-angle modes in VVC and ECM, depending on the size WxH of the current block to be predicted. This replacement of intra-prediction modes by wide-angle modes is also required in the following wide-angle conversion rules.

[0072] Table 1: Index of intra-prediction modes replaced by wide-angle mode in VVC and ECM (67 core intra-prediction modes). Aspect Ratio Replaced intra-prediction mode index W / h==2 2， 3， 4， 5， 6， 7, W / h==4 2， 3， 4， 5， 6， 7， 8， 9， 10， 11 W / h==8 2， 3， 4， 5， 6， 7， 8， 9， 10， 11， 12， 13 W / h==16 2， 3， 4， 5， 6， 7， 8， 9， 10， 11， 12， 13， 14， 15 W / h==32 2， 3， 4， 5， 6， 7， 8， 9， 10， 11， 12， 13， 14， 15， 16 W / h==1 No H / w==2 61， 62， 63， 64， 65， 66 H / w==4 57， 58， 59， 60， 61， 62， 63， 64， 65， 66 H / w==8 55， 56， 57， 58， 59， 60， 61， 62， 63， 64， 65， 66 H / w==16 53， 54， 55， 56， 57， 58， 59， 60， 61， 62， 63， 64， 65， 66 H / w==32 52， 53， 54， 55， 56， 57， 58， 59， 60， 61， 62， 63， 64， 65， 66

[0073] Figure 6 This example illustrates how a wide-angle mode replaces an intra-angle mode for a non-square block whose width is strictly greater than its height. In this example, mode 2 is replaced by wide-angle mode 67. Mode 3 is replaced by wide-angle mode 68. For example, if the current block to be predicted is 8x4, this replacement process will proceed incrementally until mode 7 is replaced by wide-angle mode 72.

[0074] For example, in ECM, to convert an intra-prediction mode index to its corresponding wide-angle mode, the following can be applied. In the case of predicting WxH blocks, non-TIMD, i.e., using one of the 65 directional intra-prediction modes, i Indicates the index of the current intra-prediction mode. i wide Let's denote the index of the current intra-frame prediction mode based on the wide-angle conversion, where DC_IDX=1 and VDIA_IDX=66. i wide This can be obtained through the following pseudocode: if (i>DC_IDX) { modeShift[] = {0, 6, 10, 12, 14, 15}; deltaSize = | log2(W) - log2(H) |; if (W>H&& i <2 + modeShift[deltaSize]) { i wide = i + (VDIA_IDX - 1); } else if (H>W&& i>VDIA_IDX - modeShift[deltaSize]) { i wide = i - (VDIA_IDX - 1); } else { i wide = i } } It should be noted that wide-angle conversion is not performed for plane and D indexes, and wide-angle conversion is not performed if H equals W (the current block is a square).

[0075] Template-based intra-mode derivation (TIMD) can be used to predict the current block to be encoded / decoded. TIMD relies on the assumption that, for a given block, a given intra-prediction mode that predicts the pixel region surrounding the block (also known as the block's template) with high quality is likely to predict the block with high quality. Using this assumption and the same region of available decoded pixels around the block on both the encoder and decoder sides, a search can be constructed for the intra-prediction mode that predicts the template with the highest quality (i.e., the intra-prediction mode most likely to predict the block with the highest quality), which is identical on both the encoder and decoder sides. Due to this principle, instead of signaling one or more intra-prediction mode indices, the entire prediction of a given block can be signaled from the encoder to the decoder via TIMD through a single flag specifying whether TIMD is used. This results in a considerable saving of bits in intra-prediction signaling. All explanations below apply in the same way on both the encoder and decoder sides.

[0076] In TIMD within ECM (e.g., ECM-7.0), for the current block to be encoded / decoded, TIMD follows a two-step process described in YangWang, Li Zhang, Kai Zhang, Zhipin Deng, Na Zhang. EE2-related: template-based intra-mode derivation using MPMs. Contribution JVET-V0098 at the 22nd JVET meeting in teleconference, from 20 to 28 April 2021. First, the derivation step of the template for the decoded pixels of the block returns indices of the two intra-prediction modes that predict the template with the highest quality (and therefore are most likely to predict the current block with the highest quality according to the TIMD assumption). Then, the current block is predicted using the two intra-prediction modes provided by the previous derivation step using these indices.

[0077] The derivation steps are as follows.

[0078] In TIMD within ECM, for the current block, the derivation step begins by extracting the block's template and a set of decoded reference samples for that template. For example, in... Figure 7A In (a), the left portion (700) of the iTw×height of the template of the current width×height block (703) is extracted from the currently decoded luminance channel, along with the top portion (701) of the template. Furthermore, the template's... A set of 702 decoded reference samples. In ECM-7.0, the template size is defined as follows: If width ≤ 8, then iTw = 2; otherwise, iTw = 4. If height≤8, then iTh=2; otherwise, iTh=4.

[0079] Then, create a list of indices for the intra-prediction modes to be tested on this template. .For example, A list of the most likely modes (MPMs) corresponding to the current block. If these indices do not already exist in the list of MPMs, they are supplemented with DC mode DC_IDX, horizontal mode HOR_IDX, and vertical mode VER_IDX.

[0080] Note that for TIMD, by... Figure 4AThe direction is inserted between each plain black arrow and its adjacent dashed arrow, expanding the set of directional intra-prediction modes from 65 to 129. This means that the set of possible intra-prediction modes derived via TIMD includes 131 modes.

[0081] Since the number of directional intra-prediction modes spanning the direction from "bottom left to top right" to "top right to bottom left" is twice the number of corresponding directional modes in VVC for TIMD, this means that the directional intra-prediction mode index in TIMD belongs to... ,so The index of each directional intra-prediction mode is remapped. If i express If the index of the directional intra-prediction mode in the frame is given, then its remapping can be represented as: Therefore, Table 1 above, which indicates the directional intra-prediction modes that are replaced by wide-angle directional intra-prediction modes based on block aspect ratio, is adapted to this remapping. The wide-angle conversion rules are based on block aspect ratio, but Table 1 becomes Table 2 below. For example, for a given 8x4 block using TIMD, mode 2 is replaced by wide-angle mode 131, mode 3 by wide-angle mode 132, mode 4 by wide-angle mode 133, ... mode 12 by wide-angle mode 141.

[0082] Table 2: Index of intra-prediction modes replaced by wide-angle mode in TIMD in ECM. condition Replaced intra-prediction mode index W / h==2 [|2， 12|] W / h==4 [|2， 20|] W / h==8 [|2， 24|] W / h==16 [|2， 28|] W / h==32 [|2， 30|] W / h==1 No H / w==2 [|120， 130|] H / w==4 [|112， 130|] H / w==8 [|108， 130|] H / w==16 [|104， 130|] H / w==32 [|102， 130|]

[0083] For example, to convert an intra-prediction mode index to its corresponding wide-angle mode, the following can be applied when using TIMD. When predicting the current WxH block, TIMD, which uses one of 129 directional intra-prediction modes, i Indicates the index of the current intra-prediction mode. i wide Let EXT_VDIA_IDX = 130, representing the index of the current intra-prediction mode based on the wide-angle conversion. i wide This can be obtained through the following pseudocode: if (i>DC_IDX) { modeShift[] = {0, 11, 19, 23, 27, 29}; deltaSize = | log2(W) - log2(H) |; if (W>H&&i<2 + modeShift[deltaSize]) { i wide = i + (EXT_VDIA_IDX - 1); } else if (H>W&&i>EXT_VDIA_IDX - modeShift[deltaSize]) { i wide = i - (EXT_VDIA_IDX - 1); } else { i wide = i } } Then, in the so-called "first pass", for For each intra-prediction mode indexed in the middle, a prediction for the templates (700 and 701) for that mode is obtained from the reference sample (702), and the SATD between the prediction and the template is calculated. The two intra-prediction modes that produce the two minimum prediction SATDs are retained. i 0、 i 1.

[0084] Then, in the so-called "second round", for the two intra-frame prediction modes retained by the "first round" i 0、 i For each of the steps in 1, if the mode is neither planar nor DC, TIMD also tests its two closest directional intra-prediction modes against the template of the current block based on the predicted SATD. Based on these additional tests, the indices of the two intra-prediction modes that produce the two minimum predicted SATDs can be updated. i 0、 i 1.

[0085] Finally, the blending conditions are defined, and two blending weights are potentially determined. Let timdIsBlend denote a flag indicating whether TIMD blending is applicable. Let satd0 and satd1 be the values ​​of the retained intra-prediction modes, respectively. i 0、 i 1. The first and second minimum predictions of SATD are associated.

[0086] When timdIsBlend is true, the index of the intra-prediction mode that produces the minimum prediction SATD. i Weights associated with 0 w 0 equals This division has an integer form. Since the expression for this integer form of the division is implementation-dependent (and has changed, for example, from two consecutive versions of ECM), it will not be described in detail here. Then, it is compared with the index of the intra-prediction mode that produces the second minimum prediction SATD. i 1. Associated weights w 1 equals 64 - w 0.

[0087] Note that the above explanation of the TIMD derivation steps assumes that both the left and top portions of the current block's template are available. In the case where only the top portion of the current block's template is available... Figure 7A (a) becomes Figure 7A (b) In the case where only the left portion of the template for the current block is available, Figure 7A (a) becomes Figure 7A (c)

[0088] information It is passed to the prediction step described below.

[0089] The prediction process for blocks in TIMD is as follows.

[0090] If timdIsBlend is false, then by index i The intra-frame prediction mode of 0 performs a prediction (predFinal) on the current block from its set of decoded reference samples.

[0091] Otherwise, use an index. i The intra-prediction mode for frame 0 obtains the first prediction for the block, predTimd0, from the set of decoded reference samples for that block. This is achieved using an index. i The intra-prediction mode of 1 obtains the second prediction predTimd1 for the block from the set of decoded reference samples for that block. The prediction predFinal for the current block is generated by mixing the first prediction predTimd0 and the second prediction predTimd1. in These are the coordinates of the predicted sample for that block.

[0092] The signaling for TIMD in the current block is performed as follows.

[0093] In ECM-7.0, for the current block, TIMD is signaled via a TIMD flag, which is placed after the flag of matrix-based intra-prediction (MIP) in the decision tree of signaling that is selected for predicting the intra-prediction mode of the block.

[0094] From ECM-7.0 to ECM-8.0, TIMD with extended wide-angle intra-prediction modes was integrated, as described in Kévin Reuzé, Thierry Dumas, Karam Naser, Ya Chen. EE2-1.10: optimizing the use of reference samples. Contribution JVET-AC0094 at the 29th JVET meeting inteleconference, from 11 to 20 January 2023. Prior to this integration, directional intra-prediction modes were always selected from 129 allowed directional intra-prediction modes, regardless of the size of the current block using TIMD. As mentioned above, in TIMD, the number 129 is generated by a predefined density of directional intra-prediction modes spanning directions from "bottom left to top right" to "top right to bottom left" (e.g., ...). Figure 4B As shown, it is twice the size of the one in VVC.

[0095] Note that the wide-angle rule does not change the number 129. For example, if the current block is 8×8, the index of the directional intra-prediction mode belongs to... If the current block is 8×4, then the index of the directional intra-prediction mode belongs to If the current block is 16×4, then the index of the directional pattern belongs to... .

[0096] After expanding the wide-angle intra-prediction modes by adding more wide-angle intra-prediction modes to be evaluated by TIMD, for directional intra-prediction modes with the same density across the direction from "bottom left to top right" to "top right to bottom left" (e.g., Figure 4B (As shown), select the directional intra-prediction mode to be evaluated in TIMD from a variable number of allowed directional intra-prediction modes. This variable number is greater than 129. Obtain additional allowed directional intra-prediction modes from the newly introduced wide-angle directional intra-prediction modes for blocks with sizes close to the current block size.

[0097] Precisely, for the current width×height block, let and .if Then it will be predicted after the last allowed vertical positive intra-frame prediction mode. The index of each wide-angle vertical positive intra-prediction mode is added to the range of allowed directional intra-prediction mode indices. If Then it will predict the mode before the first allowed horizontal positive frame. An index of a wide-angle horizontal positive intra-prediction mode is added to the range of allowed directional intra-prediction mode indices.

[0098] For example, Figure 7B The range of allowed directional intra-prediction modes for the current 8×4 block (703) during the TIMD derivation step is presented. The dashed arrow (704) indicates the direction of the first allowed horizontal positive intra-prediction mode before applying the wide-angle rule to the 8×4 block, i.e., the direction of index 2. The dashed arrow (706) indicates the direction of the last allowed vertical positive intra-prediction mode before applying the wide-angle rule, i.e., the direction of index 130. Therefore, the number of allowed directional modes before applying the wide-angle rule is 129.

[0099] exist Figure 7B Above, the black arrow (705) indicates the direction of the first horizontal positive intra-prediction mode allowed after applying the wide-angle rule to the 8×4 block, namely the direction of index 13. The black arrow (707) indicates the direction of the last vertical positive intra-prediction mode allowed after applying the wide-angle rule, namely the direction of index 141. Therefore, after applying the wide-angle rule, the number of allowed directional modes remains 129.

[0100] exist Figure 7B Above, the gray arrow (708) indicates the direction of the last vertical positive intra-prediction mode allowed after applying the wide-angle intra-prediction mode extension as described above, i.e., the direction of index 153. Therefore, after applying the wide-angle intra-prediction mode extension described above, the number of allowed directional modes for block 8×4 is 141.

[0101] For the current width×height block, during the TIMD derivation step, between the "first round" and the "second round," for each intra-prediction mode whose index belongs to the addition range of allowed directional intra-prediction mode indices, the intra-prediction mode is tested on the template of the current block with a step size of 5. For example, in Figure 7B In the case that the index belongs to For each intra-prediction mode, the intra-prediction mode is tested on the templates collected (700) and (701) when the index range is resolved using a step size of 5.

[0102] For the current block using TIMD, during its prediction, the derived intra-prediction modes belonging to the added range of allowed directional intra-prediction modes are applied as is. This means, for example, that the wide-angle intra-prediction mode of index 153 retains the same characteristics during the TIMD derivation step and during the actual prediction of the current block, particularly its parameters characterizing its orientation, i.e., in the ECM. and .

[0103] Position-related Pixel Combination (PDPC) in VVC or ECM is a post-processing tool in intra-frame prediction. It aims to remove discontinuities caused by the initial intra-frame prediction of certain prediction modes at the boundaries of target blocks adjacent to reference samples. This is achieved by using a weighted combination of the initial prediction value and one or more nearby reference samples. It is also enabled in purely horizontal and purely vertical modes, as well as in angular modes in directions from the lower left corner of the block to the upper right corner (and vice versa), in addition to the two non-angular modes (i.e., planar mode and DC mode). Depending on the prediction direction, either regular PDPC or gradient PDPC is applied. PDPC is also used as post-processing in the derivation process of intra-frame prediction modes for the TIMD-coded modes described above.

[0104] For the above TIMD assumptions to hold true—that is, to ensure that a given intra-prediction mode of the template predicting the current block with high quality is likely to be converted to the same intra-prediction mode predicting the current block with high quality—all pixels involved in the TIMD derivation step must be highly correlated. Therefore, in the TIMD derivation step, block pixels and template pixels must be highly correlated. Furthermore, template pixels and the set of decoded reference samples of the template must also be strongly correlated. However, as... Figure 7A As shown in (a), the template of the current block is characterized by a "hole" (in) between its left-hand portion and its "above" portion. Figure 7A (A) is shown as a white square. Due to this "hole", the set of decoded reference samples of the current block's template and the template pixels are not as strongly correlated as they would ideally be.

[0105] In some embodiments, a method and apparatus are provided for encoding or decoding a block using a TIMD process, wherein the “hole” is suppressed.

[0106] Note that all aspects of the embodiments described below are applied in the same way on both the encoder and decoder sides.

[0107] According to one aspect of this disclosure, a method for encoding or decoding video blocks is provided, wherein a template intra-prediction mode derivation (TIMD) is adapted. More specifically, in some embodiments, for video blocks using TIMD, during the TIMD derivation step, instead of a set of reference samples defining the template (which is common to the upper and left portions of the template), each portion of the template has a different set of reference samples. This allows for the removal of gaps between the template and its reference samples. The reference samples used to predict the template are closer to the template, thus improving prediction. Furthermore, this adaptation of the reference sample set allows maintaining the same dimensions as the template used in TIMD within the ECM, thus allowing the reuse of the same prediction units. For example, the two portions of the template can still have dimensions that are powers of 2.

[0108] According to the embodiments provided herein, holes in the template are eliminated, thereby improving template prediction.

[0109] As a beneficial side effect of eliminating the hole, PDPC using the TIMD derivation step and gradient PDPC can also become more efficient. As mentioned above, if PDPC is allowed, it enables the removal of some discontinuities between the predicted sample and the reference sample around the boundary of the predicted block (regardless of whether the block to be predicted is a template or a block). Therefore, the closer the predicted sample is to the reference sample around the boundary of the predicted block, the better PDPC works.

[0110] The following provides an example in which Template Intra-Predictive Mode Derivation (TIMD) is adapted such that each part of the template has its own set of reference samples.

[0111] Figure 8 (a.1) and (a.2) show the current block (1800) to be predicted during the derivation step of TIMD, its template, and a reference sample of the template. Figure 8 (a.1) illustrates the templates (1801 and 1802) used in the ECM and their reference sample (1803), while Figure 8 (a.2) Illustrate embodiments of the templates (1801 and 1802) and their reference samples (1810, 1811) provided herein adapted for TIMD. Figure 8 In (a.1) and (a.2), the top and left portions of the template of the current block (1800) are available.

[0112] exist Figure 8 In (a.1), for the given derivation steps of TIMD Block (1800), for the template to be in (1800) (including its The left side (1801) and its For a given intra-prediction mode tested on the upper part (1802), the template's Some of the reference samples in the set of reference samples (1803) are used to predict both (1801) and (1802). (1812) represents the template present in the prediction of the template used in ECM and its reference samples. hole.

[0113] exist Figure 8 In (a.2), for a given intra-prediction mode to be tested on the template (1800), some reference samples in the reference sample set (1810) of the left template part (1801) are used to predict the left template part (1801), while some reference samples in the reference sample set (1811) of the upper template part (1802) are used to predict the upper template part (1802).

[0114] and Figure 8 (a.1) different, Figure 8 (a.2) No holes are contained between the template (1800) and its reference sample.

[0115] exist Figure 8 In (a.1) and (a.2), to provide an example of which reference samples are used to predict the template (1800), the black dotted arrow indicates the direction in which the reference samples of the template are extrapolated into the template for the directional intra-prediction mode at index 48. In other words, for a given sample to be predicted in the template portion, the tail of the arrow passing through that sample positions the reference sample at the center of the directional interpolation filter used to compute the prediction for that template sample.

[0116] This example reveals the reference sample set of the template (1800) from Figure 8 (a.1) to Figure 8 The change in (a.2) modifies the template prediction. For example, in Figure 8 In (a.2), during the prediction of (1801) from (1810), the reference sample (1813) is accessed. However, in Figure 8 In (a.1), (1813) is not involved in the prediction of (1801). Note that, for example, the black fill dot at the end of the marker in (1813) indicates that the marker marks a single pixel rather than a set of pixels with a shared color.

[0117] Similarly, in Figure 8 In (a.2), during the prediction of (1802) from (1811), the reference sample (1814) is accessed. However, in Figure 8 In (a.1), (1814) is not mentioned in the prediction of (1802).

[0118] As another example, Figure 9 copy Figure 8 In addition to replacing the intra-prediction mode at index 48 with the intra-prediction mode at index 112.

[0119] exist Figure 9 In (a.2), the reference sample (1913) is accessed during the prediction of the template of block (1900) from the reference sample set (1910) in the left portion (1901). However, in Figure 9 In (a.1), (1913) is not mentioned in the prediction of (1901).

[0120] Figure 10(a) An embodiment is presented where only the upper portion (2002) of the template for the current block (2000) is available. During the TIMD derivation step, the set of reference samples (2003) for the template is completed with the reference samples colored in black. Since these reference samples are not available, they are generated by filling from the reference sample (2010).

[0121] Figure 11 (a) An embodiment is presented where only the left-hand portion (2101) of the template for the current block (2100) is available. During the TIMD derivation step, the set of reference samples (2103) for the template is completed with the reference samples colored in black. Since these reference samples are not available, they are generated by filling from the reference sample (2110).

[0122] like Figure 10 (a) and Figure 11 As shown in (a), during the derivation steps of TIMD, the design of the reference sample of the template of the current block in the ECM and the design in the variant embodiments described in these figures correspond to the same design when only one of the two template parts is available.

[0123] Figure 8 (b.1) and (b.2), Figure 9 (b.1) and (b.2), Figure 10 (b) and Figure 11 (b) shows that once the TIMD derivation steps have returned to the primary TIMD mode and the secondary TIMD mode, the reference sample used to predict the current block (1800, 1900, 2000, 2100) shares the same design in the ECM as in the above embodiments during the prediction of the current block (1800, 1900, 2000, 2100).

[0124] Note that for Figure 8 (a.1) in (1803), Figure 8 (a.2) in (1810) and (1811), Figure 9 (a.1) in (1903), Figure 9 (a.2) in (1910) and (1911), Figure 10 (2003) and Figure 11 In (2103), the relationship shown between (pair) {size of the current block, size of its template} and the right-hand extension of the template's reference sample set can be adapted according to the evolution of TIMD. Similarly, the relationship shown between the pair {size of the current block, size of its template} and the bottom-oriented extension of the template's reference sample set can also be adapted according to the evolution of TIMD.

[0125] For example, as described above, from ECM-7.0 to ECM-8.0, due to the addition of the wide-angle intra-frame prediction mode to the TIMD test starting with ECM-8.0, the template's reference sample set has been expanded to the right and bottom by a factor of 4. This expansion can be directly applied to this variant embodiment.

[0126] The above about Figure 8-11 In the variant of the described embodiment of TIMD utilizing a template-adapted set of reference samples, TIMD follows the same rules for defining wide-angle intra-prediction modes as in TIMD used in ECM. More specifically, in ECM, for a given TIMD... During the TIMD derivation step, for a given intra-prediction mode to be tested on a template of that block, the potential transformation of that intra-prediction mode to its wide-angle version depends only on the width and height of the block. Similarly, in this variant of the embodiments provided herein, in the case of TIMD utilizing a template-adapted set of reference samples, for TIMD using... During the TIMD derivation step, the conversion of the intra-prediction mode to its wide-angle version follows the same rules based on the width and height of the current block.

[0127] Figure 12 (a.1) presents the method used in ECM during the derivation steps of TIMD for a given... An example of TIMD for block (2200). In this example, the intra-prediction mode from the template is completed via index 12. The set of 2203 reference samples includes its Left side 2201 and its Prediction of the template in the upper part (2202). In this example, with width=8 and height=4, the intra-prediction mode of index 12 is converted to the wide-angle mode of index 141 according to the wide-angle conversion rule before template prediction.

[0128] Figure 12 (a.2) describes the process during the derivation of TIMD for a given set of adapted reference samples using a template. The block (2200) is an example of TIMD as provided herein in the embodiments. In this example, the intra-prediction mode is performed from its reference sample set (2210) via index 12. The predictions from the left template portion (2201) and from its reference sample set (2211) Prediction of the upper template section (2202). Again, with width=8 and height=4, the intra-prediction mode of index 12 is converted to the wide-angle mode of index 141 before the prediction of these two template sections (2201 and 2202). Figure 12 In (a.1) and (a.2), the black dotted arrow indicates the direction of the wide-angle mode at index 141.

[0129] To illustrate another example, Figure 13 (a.1) Copying Figure 12 (a.1), and Figure 13 (a.2) Copying Figure 12 (a.2), except that the intra-prediction mode at index 13 replaces the intra-prediction mode at index 12. Figure 13 In (a.1) and (a.2), with width=8 and height=4, the intra-prediction mode of index 13 does not undergo any wide-angle conversion.

[0130] In a variation of the above embodiments, for a given block of TIMD using an adapted set of reference samples utilizing the template described herein, during the TIMD derivation step, each of the two reference sample sets of the template portion is extended to the right-hand side and / or the bottom, making prediction feasible for each of the two template portions. In other words, the reference sample set is extended as needed, depending on the intra-prediction mode being tested. If the extended portion of the reference sample set of the template portion includes unavailable pixels, padding in VVC / ECM is used to fill the extended portion.

[0131] For example, in Figure 12 (a.2) and Figure 13 In (a.2), the reference sample set (2210) includes 2-width reference samples above and to the upper right of the left side of the template (2201), making it always possible to predict the left side of the template (2201) using the above embodiments. Similarly, the reference sample set (2211) includes 2-height reference samples to the left and lower left of the top of the template (2202), making it always possible to manage the prediction of the top of the template (2202) using the above embodiments.

[0132] Notice, Figure 12 (a.2) and Figure 13 The extended example in (a.2) relies on the TIMD template shape defined in the ECM. This means that... if ,but ,otherwise , if ,but ,otherwise .

[0133] Other extensions are also possible, for example, if the shape of the TIMD template is changed to ensure that predicting (2201) from (2210) and predicting (2202) from (2211) is always feasible for the intra-frame prediction mode of the test.

[0134] In a variant embodiment, instead of reusing the wide-angle rule for TIMD used in ECM, this paper provides a variant in which the wide-angle rule is modified and based on the size of the portion of the template to be predicted.

[0135] Figure 14 (a.1) illustrates the given An example of the derivation steps of TIMD used in ECM for block (2400). In this example, the templates (2401, 2402) of block (2400) are predicted from the reference sample set (2403) of the template via the intra-prediction mode at index 12, as follows. Figure 12 As in (a.1). With width=8 and height=4, before template prediction, the intra-prediction mode of index 12 is converted to the wide-angle mode of index 141 according to the wide-angle conversion rules defined in the ECM.

[0136] Figure 14 (a.2) presents the given Block (2400) provides an example of the derivation steps of TIMD based on an embodiment utilizing an adapted set of reference samples. In this example, the intra-frame prediction mode at index 12 is used to perform the derivation from its reference set (2410). The predictions from the left template portion (2401) and from its reference sample set (2411) Prediction of the template section above (2402).

[0137] According to this embodiment, as and Prior to the prediction at (2401), no wide-angle conversion was applied to the intra-prediction mode at index 12. However, as and Prior to the prediction at (2402), the intra-prediction mode of index 12 is converted to the wide-angle mode of index 141. Therefore, in this embodiment, the wide-angle conversion rule is the same as in ECM, but the rule is applied based on the size of the template portion to be predicted rather than the size of block (2400).

[0138] To illustrate another example, Figure 15 (a.1) Copying Figure 14 (a.1), and Figure 15 (a.2) Copying Figure 14(a.2), where Figure 15 Intra-prediction mode replacement at index 130 Figure 14 The intra-prediction mode using index 12 is employed. Figure 15 In (a.1), as and The intra-prediction mode at index 130 does not undergo any wide-angle conversion. Figure 15 In (a.2), as and Before predicting the left portion of the template for the current block, the intra-prediction mode at index 130 is converted to the wide-angle mode at index 1. and No wide-angle transformation is applied to the intra-prediction mode at index 130 before predicting the upper portion of the template of the current block.

[0139] Note that in Figure 14 (a.2) and Figure 15 In (a.2), for each of the two parts of the template of a block, the extension of the reference sample set of the shown template part to the right and bottom is a simple example applicable to prediction via any intra-frame prediction mode within this variant embodiment. This shown extension can be modified without affecting the purpose of the current variant embodiment, namely, a wide-angle rule based on the size of the template part to be predicted.

[0140] In ECM, the wide-angle rule does not convert directional intra-prediction modes to precisely opposite-direction intra-prediction modes. To address this issue, in a variant embodiment, the wide-angle rule depends on the size of the portion of the template to be predicted, and this wide-angle rule always converts directional intra-prediction modes to precisely opposite-direction intra-prediction modes. For example, this can be accomplished by manipulating the mode index as follows. In the case of TIMD, i.e., 129 directional intra-prediction modes, relative to the index... i v The index of the directional intra-prediction mode in the opposite direction to the vertical positive directional intra-prediction mode is... .in other words, This means that the index increment is preserved relative to the "reference" diagonal pattern index. (Relative to the index) i h The index of the horizontal positive directional intra-prediction mode, and the index of the opposite directional intra-prediction mode are... . Figure 16 Bundle Figure 15 Adapted to the current variant implementation. (and) Figure 15 Unlike (a.2), in Figure 16 In (a.2), as and Before predicting the left portion of the template for the current block, the intra-prediction mode at index 130 is converted to the intra-prediction mode at index 2. This is because the intra-prediction modes at index 130 and index 2 have precisely opposite directions. Figure 16 In (a.2), for the prediction of the left portion of the template of the current block, the gray dotted arrow indicates the direction of the intra-prediction mode at index 130. The black dotted arrow indicates the direction of the intra-prediction mode at index 2.

[0141] Any of the embodiments described above regarding TIMD using a template-adapted set of reference samples can be integrated into the TIMD derivation process implemented in a video codec (e.g., ECM).

[0142] For example, Figure 17 An example of method 2700 for encoding video blocks using TIMD according to any of the embodiments described herein is illustrated. At 2701, according to the present document regarding... Figure 8-16 Any of the described embodiments determines a set of reference samples for each part of the template. More specifically, each part of the template has its own set of reference samples. The reference sample sets may share some common samples and have other samples that differ from the other set. However, each set of reference samples is defined based on the part of the template that the set is intended to predict. For this purpose, the set of reference samples defined for a given part of the template is a set of reconstructed samples located above and to the left of the given part of the template. In some variations, the reference samples of the set are adjacent to the given part of the template.

[0143] Depending on the variants described above, the reference sample set can be extended to the right-hand side and / or lower left of the left or upper portion of the template, as per [reference to...]. Figure 12-13 As stated above.

[0144] At 2702, using the reference sample set of the left and upper template portions determined at 2701, the intra-prediction mode derivation process of TIMD is used to derive one or more intra-prediction modes based on the template of the block. Depending on the variant described above used, the intra-prediction modes evaluated during this derivation process can undergo wide-angle transformations if necessary, such as regarding... Figure 12-16 As stated above.

[0145] At 2703, a block predictor is obtained based on one or more intra-frame prediction modes derived at 2702 from a template-based derivation process to obtain video blocks.

[0146] At position 2704, the video block is encoded based on the block predictor.

[0147] Figure 18An example of method 2800 for decoding video blocks using TIMD according to any embodiment described herein is illustrated. At 2801, according to the present document regarding... Figure 8-16 Any of the described embodiments determines a set of reference samples for each portion of the template. The same embodiments used at the encoder will be used on the decoder side. At 2802, one or more intra-prediction modes are derived based on the template of the block using the reference sample sets of the left and upper template portions determined at 2801, employing the intra-prediction mode derivation process of TIMD. The derivation process is similar to the derivation process performed on the encoder side. At 2803, a block predictor is obtained based on one or more intra-prediction modes derived from the template-based derivation process at 2802 for the video block. At 2804, the video block is reconstructed based on the block predictor.

[0148] Figure 19 The diagram illustrates the following: Figure 8 a.2 or Figure 9 Example of the workflow for the TIMD derivation (2900) step using an adapted set of reference samples of a template, as shown in embodiment a.2. The following steps are performed for the current block 1800 to be encoded or decoded.

[0149] At position 2901, the set of intra-prediction modes to be evaluated is determined. For example, it can be derived from the current... The set of intra-prediction modes is obtained from the list of most probable modes (MPM) for block (1800). If these indices are not already in the list, they can be supplemented with intra-prediction modes DC_IDX (for DC mode), VER_IDX, and HOR_IDX (for vertical and horizontal modes). At 2902, a set of reference samples for each part of the template (left and top parts) is determined. The size is... The left portion (1801) of the template uses the reference sample set (1810). The size is... The top portion (1802) of the template uses a set of reference samples (1811). These sets of reference samples are extracted from the current channel (e.g., the luminance or chrominance channel).

[0150] At 2903, the set of intra-prediction modes to be evaluated on the template is looped through. For each intra-prediction mode index in the list collected at (2901). i The process proceeds to steps (2904 and 2906) for the left side of the template and steps (2905 and 2907) for the top part of the template.

[0151] At position 2904, via the pattern index i Predictions for the left-hand portion (1801) of the template determined from the reference sample set (1810) At position 2906, the left-hand portion of the calculated template (1801) is compared with the predicted value. SATD between .

[0152] At position 2905, via the pattern index i Predicting the upper left portion (1802) of the template from the reference sample set (1811). At position 2907, the top portion of the template (1802) is calculated and compared with the predicted value. SATD between .

[0153] Once all intra-prediction modes in the set have been evaluated, at (2908), one or more intra-prediction mode indices are determined based on the costs evaluated in 2906 and 2907. For example, the set The two minimum values ​​in the table give the index of the main TIMD pattern. Indexes in secondary TIMD mode .

[0154] In addition, respectively from and Determine two mixed weights and .

[0155] Prediction steps for current block 1800 ( Figure 8 Step b.2) uses two predictions to obtain the current block (1800). and To complete this, use the reference sample (1804) defined for the current block (1800) to obtain the prediction for the current block (1800). and Combine these two predictions to produce the final prediction for the current block (1800).

[0156] Figure 20 The diagram illustrates the target Figure 14 (a.2) shows another example of the workflow for the derivation (3000) step of the TIMD using a modified version of the embodiment and an adapted set of reference samples of the template. The workflow is not limited to this embodiment; similar workflows can be applied to... Figure 8-16 Other embodiments illustrated.

[0157] At (3001), a set of intra-prediction modes to be evaluated by TIMD is obtained for the current block to be predicted, with the size of block (2400) being... .

[0158] For example, it can be derived from the current The set of intra-prediction modes is obtained from the list of most probable modes (MPM) for block (2400). If these indices are not already in the list, the list can be supplemented with intra-prediction modes DC_IDX, VER_IDX, and HOR_IDX.

[0159] At (3002), extract the template of block (2400) from the current channel. The reference sample set (2410) of the left part (2401) and the template of the block (2400) The reference sample set (2411) of the upper part (2402).

[0160] At 3003, the set of intra-prediction modes to be evaluated on the template is looped through. For each intra-prediction mode index in the list collected at (3001)... i The process proceeds to steps (3014, 3004 and 3006) for the left side of the template and steps (3015, 3005 and 3007) for the top part of the template.

[0161] At 3014, the parameters for the left portion (2401) of the prediction template are derived. The intra-prediction mode index is determined based on the variant used and the angle of the intra-prediction mode. i Is it necessary to base on and Convert to its associated wide-angle mode index If so, then the intra-frame prediction mode index. i It is converted into its associated wide-angle mode index. .

[0162] If using Figure 12-13 The variant shown implements workflow 3000, and at 3014, the transformation is based on the width and height of the current block.

[0163] At position 3004, via the pattern index i or index The pattern (if transformed at 3014) is determined from the reference sample set (2410) and the prediction (2401) is obtained. In 3006, calculate (2401) and SATD between .

[0164] At 3015, derive the parameters used to predict the top portion of the template (2402), which, if necessary, includes adjusting the values ​​based on width and iTh. i Convert to its associated wide-angle mode index If using Figure 12-13The variant shown implements workflow 3000, and at 3015, the transformation is based on the width and height of the current block.

[0165] At position 3005, via the pattern index i or index The pattern (if transformed) is used to calculate the prediction of (2402) from its reference sample set (2411). .

[0166] At position 3007, calculate (2402) and SATD between .

[0167] Once all intra-prediction modes in the set have been evaluated, at 3008, one or more intra-prediction mode indices are determined based on the costs evaluated in 3006 and 3007. For example, the set The two minimum values ​​in the table give the index of the main TIMD pattern. Indexes in secondary TIMD mode .

[0168] In addition, respectively from and Determine two mixed weights and .

[0169] During the prediction step of the current block (2400), use and To obtain two predictions for the current block (2400). Use and Combine these two predictions to produce the final prediction for the current block (2400).

[0170] exist Figure 19 and Figure 20 The workflow for TIMD derivation using a template-adapted reference sample set, within steps 2900 or 3000, is provided for illustrative purposes only. Some steps can be interchanged without affecting the TIMD derivation process. For example, in... Figure 19 In the middle, (2901) and (2902) can be exchanged.

[0171] In ECM (e.g., ECM-8.0), the TIMD derivation step is part of many template-based coding tools. For example, the TIMD derivation step occurs in Intra-Block Copy (IBC), Geometric Partition Mode (GPM), and Combined Intra-Inter-Frame Prediction (CIIP).

[0172] In a variant embodiment, for one or more template-based encoding tools involving TIMD derivation steps, the conventional TIMD derivation steps in ECM are replaced by TIMD derivation steps utilizing an adapted set of reference samples with templates, as described in any of the embodiments provided herein.

[0173] In another variant embodiment, for all template-based coding tools involving TIMD derivation steps, the conventional TIMD derivation steps in ECM are replaced by TIMD derivation steps utilizing a template-adapted set of reference samples as described in any of the embodiments provided herein.

[0174] In some embodiments, position-dependent intra-prediction combination (PDPC) is included in the derivation of TIMD modes. Therefore, any of the embodiments described herein can also be used when applying PDPC in the derivation of TIMD modes. As mentioned above, for some directional intra-prediction modes, PDPC is a tool for removing discontinuities caused by intra-prediction at block boundaries adjacent to reference samples. This is achieved by using a weighted combination of initial prediction values ​​and one or more auxiliary reference samples located in a reference array of reconstructed reference samples. In classic TIMD, when PDPC is applied during template prediction, these auxiliary reference samples are located in a set of reference samples defined for the template, such as... Figure 7A Array 702 in (a). As shown in the figure, the presence of “holes” between the template components affects the efficiency of the PDPC.

[0175] When implementing TIMD using any of the embodiments described herein, where the reference sample set is adapted for each part of the template, the auxiliary reference samples used in the PDPC tool when predicting the template are located in the reference sample set defined for each part of the template, such as... Figure 9 As shown in a.2. Therefore, in these embodiments, PDPC efficiency is improved.

[0176] Figure 21 The illustration shows a block diagram of a system according to another embodiment in which aspects of this embodiment can be implemented. Figure 21 An embodiment of an apparatus 1200 for encoding or decoding video according to any of the embodiments described herein is shown. The apparatus includes a processor 1210 and is interconnected to a memory 1220 via at least one port. Both the processor 1210 and the memory 1220 may also have one or more additional interconnects to external connections.

[0177] Processor 1220 is also configured to: obtain a predictor block of a video block based on an intra-frame prediction mode using at least one primary reference sample; modify the predictor block using a position-dependent pixel combination, the position-dependent pixel combination using a weighted combination of values ​​determined from at least two secondary reference samples for at least one pixel; and encode or decode the video block, at least based on the modified predictor block, using any of the embodiments described herein. For example, processor 1220 is configured using a computer program product including code instructions implementing any of the embodiments described herein.

[0178] exist Figure 22 In the embodiment shown, in a transmission scenario between two remote devices A and B via a communication network NET, device A includes a processor associated with RAM and ROM, configured to implement methods for encoding video, as described above. Figure 1-20 The device B includes a processor associated with RAM and ROM, configured to implement methods for decoding video, as described above. Figure 1-20 As described in the example, the network is a broadcast network adapted to broadcast / transmit encoded video from device A to decoding devices including device B.

[0179] Figure 23 An example of the syntax for a signal transmitted via a packet-based transport protocol is shown. Each transmitted data packet P includes a header H and a payload PAYLOAD. In some embodiments, the payload PAYLOAD may include video data according to any of the embodiments described above.

[0180] Various implementations involve decoding. As used herein, “decoding” can encompass, for example, all or part of a process performed on a received encoded sequence to produce a final output suitable for display. In various embodiments, such a process includes one or more processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various embodiments, such a process also includes, or alternatively includes, processes performed by a decoder of the various implementations described herein, such as entropy decoding of a sequence of binary symbols to reconstruct image or video data.

[0181] As further examples, in one embodiment, "decoding" refers only to entropy decoding; in another embodiment, "decoding" refers only to differential decoding; in yet another embodiment, "decoding" refers to a combination of entropy decoding and differential decoding; and in yet another embodiment, "decoding" refers to the entire image reconstruction process including entropy decoding. Whether the phrase "decoding process" is intended to specifically refer to a subset of operations or generally to a broader decoding process will be clear based on the specific context of the description and is considered well understood by those skilled in the art.

[0182] Various implementations involve encoding. In a manner similar to the discussion of “decoding” above, as used herein, “encoding” can encompass all or part of the process performed on an input video sequence to produce an encoded bitstream. In various embodiments, such a process includes one or more processes typically performed by an encoder, such as segmentation, differential coding, transform, quantization, and entropy coding. In various embodiments, such a process also includes, or alternatively includes, processes performed by an encoder of the various implementations described herein, such as determining resampling filter coefficients and resampling the decoded image.

[0183] As a further example, in one embodiment, "encoding" refers only to entropy encoding; in another embodiment, "encoding" refers only to differential encoding; and in yet another embodiment, "encoding" refers to a combination of differential and entropy encoding. Whether the phrase "encoding process" is intended to specifically refer to a subset of operations or generally to a broader encoding process will be clear based on the specific context of the description and is considered well understood by those skilled in the art.

[0184] Note that the grammatical elements used in this article are descriptive terms. Therefore, the use of other grammatical element names is not excluded.

[0185] This disclosure describes various types of information, such as, for example, transmissible or storable information, including syntax. This information can be encapsulated or arranged in various ways, including those common in video standards, such as placing the information in SPS, PPS, NAL units, headers (e.g., NAL unit headers or slice headers), or SEI messages. Other methods are also available, including those common in system-level or application-level standards, such as placing the information in one or more of the following: a. SDP (Session Description Protocol), a format for describing multimedia communication sessions, used for session announcements and invitations, such as those described in RFCs and used in conjunction with RTP (Real-Time Transport Protocol) transmission. b. DASH MPD (Media Presentation Description) descriptors, such as those used in DASH and transmitted via HTTP. A descriptor is associated with one or more representations to provide additional characteristics for content representation. c. RTP header extensions, for example, used during RTP streaming. d. ISO-based media file formats, such as those used in OMAF, and using boxes as object-oriented building blocks (also referred to as “atoms” in some specifications) defined by unique type identifiers and lengths. e. An HLS (HTTP Live Streaming) manifest transmitted over HTTP. The manifest can be associated with, for example, versions or sets of versions of content to provide the characteristics of that version or set of versions.

[0186] When the accompanying drawings are presented as flowcharts, it should be understood that they also provide block diagrams of the corresponding apparatus. Similarly, when the accompanying drawings are presented as block diagrams, it should be understood that they also provide flowcharts of the corresponding methods / processes.

[0187] Some embodiments involve rate distortion optimization. Specifically, during the encoding process, a balance or trade-off between rate and distortion is typically considered, given computational complexity constraints. Rate distortion optimization is generally planned to minimize a rate distortion function, which is a weighted sum of rate and distortion. Different approaches exist to address the rate distortion optimization problem. For example, these approaches can be based on extensive testing of all encoding options, including all considered modes or encoding parameter values, where their encoding costs and the associated distortion of the reconstructed signal after encoding and decoding are fully evaluated. Faster methods can also be used to save encoding complexity, particularly in the case of computing approximate distortion based on predicting or predicting the residual signal rather than the reconstructed signal. A hybrid of these approaches can also be used, such as by using approximate distortion only for some possible encoding options and full distortion for others. Other methods evaluate only a subset of possible encoding options. More generally, many methods employ any of a variety of techniques to perform optimization, but optimization is not necessarily a complete evaluation of both encoding costs and associated distortion.

[0188] The implementations and aspects described herein can be implemented, for example, as methods or processes, apparatuses, software programs, data streams, or signals. Even if discussed only in the context of a single implementation (e.g., discussed only as a method), the implementation of the features in question can be implemented in other forms (e.g., apparatuses or programs). Apparatuses can be implemented, for example, as appropriate hardware, software, and firmware. Methods can be implemented, for example, in a processor, which generally refers to a processing device, including, for example, a computer, microprocessor, integrated circuit, or programmable logic device. Processors also include communication devices, such as, for example, computers, cellular phones, portable / personal digital assistants (PDAs), and other devices that facilitate information communication between end users.

[0189] The references to "an embodiment," "an embodiment," "an implementation," or "an implementation," and other variations thereof, refer to the inclusion of a specific feature, structure, characteristic, etc., described in relation to that embodiment in at least one embodiment. Therefore, the phrases "in an embodiment," "in an embodiment," "in an implementation," or "in an implementation," and any other variations appearing in various places throughout this application, do not necessarily refer to the same embodiment.

[0190] Furthermore, this application may refer to "determining" various types of information. Determining information may include, for example, one or more of the following: estimated information, calculated information, predicted information, or information retrieved from memory.

[0191] Furthermore, this application may involve "accessing" various types of information. Accessing information may include, for example, one or more of the following: receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0192] Furthermore, this application may involve "receiving" various types of information. "Receiving," like "accessing," is a broad term. Receiving information may include, for example, one or more of the following: accessing information or retrieving information (e.g., from memory). Additionally, "receiving" is generally referred to in one or more ways during operations such as: storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0193] It should be understood that the use of any of the following “ / ”, “and / or” and “at least one” (e.g., in the cases of “A / B”, “A and / or B”, and “at least one of A and B”) is intended to cover selecting only the first listed option (A), or only the second listed option (B), or both options (A and B). As a further example, in the cases of “A, B, and / or C” and “at least one of A, B, and C”, such expressions are intended to cover selecting only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A, B, and C). It will be clear to those skilled in the art and related fields that this can be extended to all listed items.

[0194] Furthermore, as used herein, the term "signal" refers, among other meanings, to instructing the corresponding decoder to do something. Thus, in embodiments, the same parameters are used on both the encoder and decoder sides. Therefore, for example, the encoder can transmit (explicitly signal) a specific parameter to the decoder so that the decoder can use the same specific parameter. Conversely, if the decoder already possesses the specific parameter and other information, signaling can be used to inform the decoder of and select the specific parameter without transmission (implicit signaling). Bit savings are achieved in various embodiments by avoiding the transmission of any actual function. It should be understood that signaling can be implemented in various ways. For example, in various embodiments, one or more syntax elements, flags, etc., are used to signal information to the corresponding decoder. Although the verb form of the word "signal" has been used above, the word "signal" can also be used as a noun herein.

[0195] It will be apparent to those skilled in the art that various implementations can generate various signals, which are formatted to carry, for example, storable or transmissible information. This information may include, for example, instructions for performing a method or data generated by one of the implementations. For example, the signal may be formatted to carry a bitstream of the embodiments. Such a signal may be formatted as, for example, electromagnetic waves (e.g., using the radio frequency portion of the spectrum) or as a baseband signal. Formatting may include, for example, encoding the data stream and using the encoded data stream to modulate a carrier wave. The information carried by the signal may be, for example, analog or digital information. It is well known that signals can be transmitted via various wired or wireless links. The signal may be stored on a processor-readable medium.

[0196] Several embodiments have been described above. The features of these embodiments may be provided individually or in any combination across various claim classes and types.

Claims

1. A method comprising: At least one derived intra-prediction mode for a video block is obtained using template-based intra-prediction mode derivation, wherein the template-based intra-prediction mode derivation includes: predicting the template of the video block based on a first intra-prediction mode of a reference sample using the template of the video block. The template for the video block includes a top portion located above the video block and a left portion located to the left of the video block. Furthermore, the top portion of the template predicted based on the first intra-frame prediction mode uses a first set of reference samples, and the left portion of the template predicted based on the first intra-frame prediction mode uses a second set of reference samples. The first set of reference samples and the second set of reference samples are different. A block predictor for the video block is obtained based on at least one derived intra-frame prediction mode. The video block is decoded based on the block predictor.

2. An apparatus comprising one or more processors, wherein the one or more processors are operable to: At least one derived intra-prediction mode for a video block is obtained using template-based intra-prediction mode derivation, wherein, The template-based intra-frame prediction mode derivation includes: predicting the template of the video block based on a first intra-frame prediction mode of a reference sample using the template of the video block. The template for the video block includes a top portion located above the video block and a left portion located to the left of the video block. Furthermore, the top portion of the template predicted based on the first intra-frame prediction mode uses a first set of reference samples, and the left portion of the template predicted based on the first intra-frame prediction mode uses a second set of reference samples. The first set of reference samples and the second set of reference samples are different. A block predictor for the video block is obtained based on at least one derived intra-frame prediction mode. The video block is decoded based on the block predictor.

3. A method comprising: At least one derived intra-prediction mode for a video block is obtained using template-based intra-prediction mode derivation, wherein the template-based intra-prediction mode derivation includes: predicting the template of the video block based on a first intra-prediction mode of a reference sample using the template of the video block. The template for the video block includes a top portion located above the video block and a left portion located to the left of the video block. Furthermore, the top portion of the template predicted based on the first intra-frame prediction mode uses a first set of reference samples, and the left portion of the template predicted based on the first intra-frame prediction mode uses a second set of reference samples. The first set of reference samples and the second set of reference samples are different. A block predictor for the video block is obtained based on at least one derived intra-frame prediction mode. The video blocks are encoded based on the block predictor.

4. An apparatus comprising one or more processors, wherein the one or more processors are operable to: At least one derived intra-prediction mode for a video block is obtained using template-based intra-prediction mode derivation, wherein, The template-based intra-frame prediction mode derivation includes: predicting the template of the video block based on a first intra-frame prediction mode of a reference sample using the template of the video block. The template for the video block includes a top portion located above the video block and a left portion located to the left of the video block. Furthermore, the top portion of the template predicted based on the first intra-frame prediction mode uses a first set of reference samples, and the left portion of the template predicted based on the first intra-frame prediction mode uses a second set of reference samples. The first set of reference samples and the second set of reference samples are different. A block predictor for the video block is obtained based on at least one derived intra-frame prediction mode. The video blocks are encoded based on the block predictor.

5. The method according to claim 1 or 3, or the apparatus according to claim 2 or 4, wherein, The first set of reference samples includes one or more reconstructed samples of the left side portion of the template, and / or the second set of reference samples includes one or more reconstructed samples of the top portion of the template.

6. The method according to any one of claims 1, 3, or 5, or the apparatus according to any one of claims 2, 4-5, wherein, The first set of reference samples includes one or more reconstructed samples located in the row immediately above the top portion of the template and the column immediately to the left of the top portion of the template, and / or the second set of reference samples includes one or more reconstructed samples located in the row immediately above the left portion of the template and the column immediately to the left of the left portion of the template.

7. The method according to any one of claims 1, 3, or 5-6, or the apparatus according to any one of claims 2, 4-6, wherein, In order to predict the template, according to the first intra-frame prediction mode, at least one of the first set or the second set is extended to the upper right of the top portion or the left portion of the template, or extended to the lower left of the top portion or the left portion of the template, or extended to both the upper right and the lower left.

8. The method according to any one of claims 1, 3, or 5-7, or the apparatus according to any one of claims 2, 4-7, wherein, The first intra-frame prediction is a directional intra-frame prediction mode, and predicting the template includes: determining, based on the size of one of the top portion or the left portion of the template, whether to apply a wide-angle conversion to the first intra-frame prediction mode before predicting the top portion of the left portion.

9. The method or apparatus according to claim 8, wherein, In response to determining that a wide-angle conversion should be applied to the first intra-frame prediction mode, the first intra-frame prediction mode is converted to a second intra-frame prediction mode based on the size of one of the top portion or the left portion of the template, and a prediction for one of the top portion or the left portion of the template is determined based on the second intra-frame prediction mode.

10. The method or apparatus according to claim 8 or 9, wherein, The second intra-prediction mode is an intra-prediction mode in the opposite direction to the first intra-prediction mode.

11. A computer program product comprising instructions for causing one or more processors to perform the method according to any one of claims 1, 3, or 5-10.

12. A non-transitory computer-readable medium storing executable program instructions for causing a computer executing the program instructions to perform the method according to any one of claims 1, 3, or 5-10.

13. A bitstream comprising data representing video encoded using the method according to any one of claims 1, 3, or 5-10.

14. A non-transitory computer-readable medium storing the bit stream according to claim 13.

15. An apparatus comprising: The apparatus according to any one of claims 2 or 5-10; as well as At least one of the following: (i) an antenna configured to receive or transmit a signal including data representing the video block; (ii) a band limiter configured to limit the signal to a band including the data representing the video block; or (iii) a display configured to display the video block.

16. The device of claim 15, wherein the device comprises at least one of a television, a cellular phone, a tablet computer, and a set-top box.

Images