Method and apparatus for encoding and decoding image or video using combined intra mode

Abstract

Disclosed are a method and apparatus for encoding or decoding a block of an image or video. For at least one block of an image or video, a first intra prediction mode and a second intra prediction mode are obtained. At least one block is encoded or decoded based on a combination of the first intra prediction mode and the second intra prediction mode. In an embodiment, the first intra prediction mode is a non-direction-based intra prediction mode, and the second intra prediction mode is a directional intra prediction mode.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to European application number 22305953.6 filed on June 30, 2022, which is incorporated herein by reference in its entirety. Technical Field

[0003] The present embodiments generally relate to video compression.The present embodiments relate to a method and apparatus for encoding and decoding blocks of an image or video based on a combination of intra-prediction modes. Background Art

[0004] To achieve high compression efficiency, image and video coding schemes usually use prediction and transform to balance the spatial and temporal redundancy in video content. Typically, intra-frame or inter-frame prediction is used to exploit intra-frame or inter-frame picture correlation, and then the difference between the original block and the predicted block - usually denoted as prediction error or prediction residual - is transformed, quantized, and entropy coded. To reconstruct the video, the compressed data is decoded by the inverse process corresponding to entropy coding, quantization, transform, and prediction. Summary of the invention

[0005] According to one aspect, a method for encoding at least one block of an image or video is provided, wherein the method comprises: for at least one block, obtaining a first intra-frame prediction mode, obtaining a second intra-frame prediction mode, and encoding the at least one block based on a combination of the first intra-frame prediction mode and the second intra-frame prediction mode.

[0006] According to one aspect, a method for decoding at least one block of an image or video is provided, wherein the method comprises: for at least one block, obtaining a first intra-frame prediction mode, obtaining a second intra-frame prediction mode, and decoding the at least one block based on a combination of the first intra-frame prediction mode and the second intra-frame prediction mode.

[0007] According to another aspect, a device for encoding at least one block of an image or video is provided, wherein the device includes one or more processors, and the one or more processors are operable to obtain a first intra-frame prediction mode, obtain a second intra-frame prediction mode, and encode the at least one block based on a combination of the first intra-frame prediction mode and the second intra-frame prediction mode for at least one block of the image or video.

[0008] According to another aspect, a device for decoding at least one block of an image or video is provided, wherein the device includes one or more processors, and the one or more processors are operable to obtain a first intra-frame prediction mode, obtain a second intra-frame prediction mode, and decode the at least one block based on a combination of the first intra-frame prediction mode and the second intra-frame prediction mode for at least one block of the image or video.

[0009] According to another aspect, a method for encoding at least one block of an image or video is provided, wherein the method comprises: for at least one block, in response to a determination that a first intra-frame prediction mode is to be combined with a second intra-frame prediction mode, obtaining a first intra-frame prediction mode, obtaining a second intra-frame prediction mode, and encoding the at least one block based on a combination of the first intra-frame prediction mode and the second intra-frame prediction mode.

[0010] According to another aspect, a method for decoding at least one block of an image or video is provided, wherein the method comprises: for at least one block, in response to a determination that a first intra-frame prediction mode is to be combined with a second intra-frame prediction mode, obtaining a first intra-frame prediction mode, obtaining a second intra-frame prediction mode, and decoding the at least one block based on a combination of the first intra-frame prediction mode and the second intra-frame prediction mode.

[0011] According to another aspect, a device for encoding at least one block of an image or video is provided, wherein the device includes one or more processors, and the one or more processors are operable to, for at least one block of the image or video, obtain a first intra-frame prediction mode in response to a determination that a first intra-frame prediction mode is to be combined with a second intra-frame prediction mode, obtain a second intra-frame prediction mode, and encode the at least one block based on a combination of the first intra-frame prediction mode and the second intra-frame prediction mode.

[0012] According to another aspect, a device for decoding at least one block of an image or video is provided, wherein the device includes one or more processors, and the one or more processors are operable to, for at least one block of the image or video, obtain a first intra-frame prediction mode in response to a determination that a first intra-frame prediction mode is to be combined with a second intra-frame prediction mode, obtain a second intra-frame prediction mode, and decode the at least one block based on a combination of the first intra-frame prediction mode and the second intra-frame prediction mode.

[0013] In some embodiments, the first intra prediction mode is obtained from a first set of intra prediction modes and the second intra prediction mode is obtained from a second set of intra prediction modes, the first and second sets of intra prediction modes being different.

[0014] In some embodiments, the first intra-prediction mode set includes a non-directional intra-prediction mode and the second intra-prediction mode set includes a directional intra-prediction mode. One of the first intra-prediction mode and the second intra-prediction mode is obtained from the first intra-prediction mode set, and the other of the first intra-prediction mode and the second intra-prediction mode is obtained from the second intra-prediction mode set.

[0015] In some embodiments, the first intra prediction mode is one of a planar prediction mode, a DC prediction mode, an intra block copy prediction mode, a matrix-based intra prediction mode. In some embodiments, the second intra prediction mode is one of a directional intra prediction mode.

[0016] In some embodiments, the second intra prediction mode is derived from reconstructed samples neighboring the at least one block. In a variation, the second intra prediction mode is obtained from at least one of a decoder-side intra mode derivation or a template-based intra mode derivation.

[0017] In some embodiments, the combination is a weighted average of a prediction obtained from a first intra-frame prediction mode and a prediction obtained from a second intra-frame prediction mode. In some variants, the weight used in the combination depends on the first intra-frame prediction mode, or at least one of an indicator signaled in the bitstream indicating a weight among a set of weights used for prediction according to the first intra-frame prediction mode, or a cost obtained when determining the second intra-frame prediction mode, or a ranking of the second prediction mode in a list of intra-frame prediction modes. In another variant, the weight used in the combination is derived from the cost obtained when determining the second intra-frame prediction mode. In another variant, the weight varies with the position of the sample in at least one block.

[0018] In one embodiment, one or more processors are operable to encode the image or video to which the block belongs. In one embodiment, one or more processors are operable to decode the image or video to which the block belongs. Additional embodiments are described herein that can be used alone or in combination.

[0019] One or more embodiments also provide a computer program including instructions, which, when executed by one or more processors, causes the one or more processors to perform a method for block encoding / decoding of an image or video according to any embodiment described herein. One or more embodiments also provide a non-transitory computer-readable medium and / or a computer-readable storage medium having instructions stored thereon, the instructions being used to

[0020] Blocks of an image or video are encoded / decoded according to the methods described herein.

[0021] One or more embodiments further provide a computer-readable storage medium on which a bitstream generated according to the method described herein is stored. One or more embodiments further provide a method and apparatus for transmitting or receiving a bitstream generated according to the above method. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A block diagram of a system is illustrated in which aspects of the present embodiments may be implemented.

[0023] Figure 2 Illustrated is a block diagram of an embodiment of a video encoder within which aspects of the present embodiments may be implemented.

[0024] Figure 3 Illustrated is a block diagram of an embodiment of a video decoder within which aspects of the present embodiments may be implemented.

[0025] Figure 4 Examples of 67 intra prediction modes in the VVC standard and ECM under development are illustrated.

[0026] Figure 5A and 5B An example of deriving a generic MPM list for luma CBs belonging to an internal slice in an ECM is illustrated.

[0027] Fig. 6A and 6B An example of signaling of an intra prediction mode for predicting luma CB in an ECM is illustrated.

[0028] Figure 7 An example of signaling of an intra prediction mode for predicting a pair of chroma CBs in an ECM is illustrated.

[0029] Figure 8 An example of the relationship between the range of the decoded reference sample set around the WxH block to be predicted and the range of allowed intra prediction angles is illustrated.

[0030] Fig. 9 An example of an angular pattern replaced by a wide-angle pattern for a non-square block whose width is strictly greater than its height in VVC and ECM is illustrated.

[0031] Fig.10 An example of a template of a luma CB and decoded reference samples of the template used in template-based intra mode derivation (TIMD) is illustrated.

[0032] Fig.11 An example of a matrix-based intra prediction process for an input block of height H and width W is illustrated.

[0033] Fig.12 An example of GPM splitting grouped by the same angle is shown.

[0034] Fig.13 An example of blending weight w0 used in geometric partitioning mode is illustrated.

[0035] Fig.14 Examples of available intra prediction modes that can be used in geometric partition mode are illustrated.

[0036] Fig.15An example of the current CTU (coding tree unit) processing order and its available reference samples in the current and left CTUs is illustrated.

[0037] Fig.16 An example of reference regions for IBC mode is shown when the CTU size is 128 × 128. Grey blocks indicate available reference regions, while white blocks indicate invalid reference regions.

[0038] Fig.17 An example of a reference area for IBC when a CTU is 256×256 is illustrated.

[0039] Fig.18 An example of top and left neighboring blocks used in the derivation of CIIP weights is illustrated.

[0040] Fig.19A An example of a method for encoding a block of an image or a video according to an embodiment is illustrated.

[0041] Fig.19B An example of a method for decoding a block of an image or a video according to an embodiment is illustrated.

[0042] Fig. 20A An example of a method for encoding a block of an image or a video according to another embodiment is illustrated.

[0043] Fig. 20B An example of a method for decoding a block of an image or a video according to another embodiment is illustrated.

[0044] Fig. 20C An example of a method for decoding a block of an image or video according to a further embodiment is illustrated.

[0045] Fig.21A An example of a method for signaling or decoding one or more syntax elements in or from a bitstream that provide for determining one or more intra-prediction modes for predicting a block of an image or video according to an embodiment is illustrated.

[0046] Fig. 21B An example of a method for signaling or decoding one or more syntax elements in or from a bitstream that provide for determining one or more intra-prediction modes for predicting a block of an image or video according to another embodiment is illustrated.

[0047] Fig. 21C An example of a method for signaling or decoding one or more syntax elements in or from a bitstream that provide for determining one or more intra-prediction modes for predicting a block of an image or video according to another embodiment is illustrated.

[0048] Fig.21D An example of a method for signaling or decoding one or more syntax elements in or from a bitstream that provide for determining one or more intra-prediction modes for predicting a block of an image or video according to another embodiment is illustrated.

[0049] Fig.21E An example of a method for signaling or decoding one or more syntax elements in or from a bitstream that provide for determining one or more intra-prediction modes for predicting a block of an image or video according to another embodiment is illustrated.

[0050] Fig. 22 An example of a method for signaling or decoding one or more syntax elements in or from a bitstream that provide for determining one or more intra-prediction modes for predicting a block of an image or video according to another embodiment is illustrated.

[0051] Fig.23A and 23B An example of a method for signaling or decoding one or more syntax elements in or from a bitstream that provide for determining one or more intra-prediction modes for predicting a block of an image or video according to another embodiment is illustrated.

[0052] Fig.24 A block diagram of a system according to another embodiment is illustrated within which aspects of the present embodiment may be implemented.

[0053] Fig.25 Illustrated are two remote devices communicating over a communication network, according to an example of the present principles.

[0054] Fig.26 The syntax of a signal according to an example of the present principles is illustrated. DETAILED DESCRIPTION

[0055] The present application describes a number of aspects, including tools, features, embodiments, models, methods, etc. Many of these aspects are specifically described, and at least in order to illustrate individual characteristics, are usually described in a manner that may sound limited. However, this is for the purpose of describing clarity, and does not limit the application or scope of these aspects. In fact, all different aspects can be combined and interchanged to provide other aspects. In addition, the aspects can also be combined and interchanged with the aspects described in the earlier submission.

[0056] The aspects described and contemplated in this application can be implemented in many different forms. Figure 1 , Figure 2 and Figure 3 Some embodiments are provided, but other embodiments are also contemplated and are intended to Figure 1 , Figure 2 and Figure 3 The discussion does not limit the breadth of implementation. At least one aspect generally relates to video encoding and decoding, and at least one other aspect generally relates to transmitting a generated or encoded bitstream. These and other aspects can be implemented as methods, apparatus, computer-readable storage media having stored thereon instructions for encoding or decoding video data according to any of the methods described, and / or computer-readable storage media having stored thereon bitstreams generated according to any of the methods described.

[0057] In this application, the terms “reconstruction” and “decoding” may be used interchangeably, the terms “pixel” and “sample” may be used interchangeably, and the terms “image”, “picture” and “frame” may be used interchangeably.

[0058] Various methods are described herein, and each method includes one or more steps or actions for realizing the described method.Unless the correct operation of the method requires a specific order of steps or actions, otherwise the order and / or use of specific steps and / or actions can be modified or combined.Additionally, terms such as "first", "second" etc. can be used to modify elements, parts, steps, operations, etc. in various embodiments, such as, for example, "first decoding" and "second decoding".Unless specifically required, the use of such terms does not imply the sequencing of the operation to modification.Therefore, in this example, the first decoding does not need to be performed before the second decoding, and can occur in, for example, before, during or in a time period overlapping with the second decoding.

[0059] The present aspects are not limited to VVC or HEVC, and may be applied, for example, to other standards and recommendations, whether existing earlier or developed in the future, and extensions of any such standards and recommendations (including VVC and HEVC). Unless otherwise indicated or technically excluded, the aspects described in this application may be used alone or in combination.

[0060] Figure 1A block diagram of an example of a system in which various aspects and embodiments can be implemented is illustrated. System 100 can be embodied as a device including various components described below, and is configured to perform one or more aspects described in this application. Examples of such devices include, but are not limited to, various electronic devices, such as personal computers, laptop computers, smart phones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances and servers. The elements of system 100 can be embodied in a single integrated circuit, multiple ICs and / or discrete components, either alone or in combination. For example, in at least one embodiment, the processing and encoder / decoder elements of system 100 are distributed across multiple ICs and / or discrete components. In various embodiments, system 100 is coupled to other systems or other electronic devices via, for example, a communication bus or by a dedicated input and / or output port communication. In various embodiments, system 100 is configured to implement one or more aspects described in this application.

[0061] The system 100 includes at least one processor 110, which is configured to execute instructions loaded therein, for implementing various aspects described in the present application, for example. The processor 110 may include embedded memory, input and output interfaces, and various other circuit systems as known in the art. The system 100 includes at least one memory 120 (e.g., a volatile memory device and / or a non-volatile memory device). The system 100 includes a storage device 140, which may include a non-volatile memory and / or a volatile memory, including but not limited to an EEPROM, a ROM, a PROM, a RAM, a DRAM, an SRAM, a flash memory, a disk drive, and / or an optical drive. As a non-limiting example, the storage device 140 may include an internal storage device, an attached storage device, and / or a network accessible storage device.

[0062] The system 100 includes an encoder / decoder module 130, which is configured to process data, for example, to provide encoded video or decoded video, and the encoder / decoder module 130 may include its own processor and memory. The encoder / decoder module 130 represents a module (one or more) that may be included in a device to perform encoding and / or decoding functions. As is known, a device may include one or both of the encoding and decoding modules. Additionally, the encoder / decoder module 130 may be implemented as a separate element of the system 100, or may be incorporated into the processor 110 as a combination of hardware and software as known to those skilled in the art.

[0063] Program code to be loaded onto the processor 110 or the encoder / decoder 130 to perform various aspects described in the present application may be stored in the storage device 140 and subsequently loaded onto the memory 120 for execution by the processor 110. According to various embodiments, one or more of the processor 110, the memory 120, the storage device 140, and the encoder / decoder module 130 may store one or more of various items during the execution of the processes described in the present application. Such stored items may include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results from the processing of equations, formulas, operations, and operational logic.

[0064] In some embodiments, memory internal to the processor 110 and / or encoder / decoder module 130 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, memory external to the processing device (e.g., the processing device may be the processor 110 or the encoder / decoder module 130) is used for one or more of these functions. The external memory may be memory 120 and / or storage device 140, such as dynamic volatile memory and / or non-volatile flash memory. In several embodiments, the external non-volatile flash memory is used to store the operating system of the television. In at least one embodiment, fast external dynamic volatile memory such as RAM is used as working memory for video encoding and decoding operations, such as for MPEG-2 (MPEG refers to Moving Picture Experts Group, MPEG-2 is also known as ISO / IEC13818, and 13818-1 is also known as H.222, and 13818-2 is also known as H.262), HEVC (HEVC refers to High Efficiency Video Coding, also known as H.265 and MPEG-H Part2), or VVC (Versatile Video Coding, a new standard being developed by JVET (Joint Video Experts Group)).

[0065] As indicated in block 105, input to the elements of system 100 may be provided through various input devices. Such input devices include, but are not limited to, (i) a radio frequency (RF) section that receives an RF signal transmitted over the air, for example, by a broadcaster, (ii) a component (COMP) input terminal (or a set of COMP input terminals), (iii) a universal serial bus (USB) input terminal, and / or (iv) a high-definition multimedia interface (HDMI) input terminal. Figure 1 Other examples not shown include composite video.

[0066] In various embodiments, the input device of block 105 has associated corresponding input processing elements as known in the art. For example, the RF section may be associated with elements suitable for the following: (i) selecting a desired frequency (also referred to as selecting a signal, or band limiting a signal to a frequency band), (ii) down-converting the selected signal, (iii) again band-limiting to a narrower frequency band to select a signal band that may be referred to as a channel in some embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired data packet stream. The RF section of various embodiments includes one or more elements that perform these functions, such as a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a down-converter, a demodulator, an error corrector, and a demultiplexer. The RF section may include a tuner that performs various functions in these functions, including, for example, down-converting a received signal to a lower frequency (e.g., an intermediate frequency or a near-baseband frequency) or baseband. In a set-top box embodiment, the RF part and its associated input processing element receive the RF signal transmitted by wired (for example, cable) medium, and filter to the desired frequency band to perform frequency selection by filtering, down-conversion and filtering again. Various embodiments rearrange the order of above-mentioned (and other) elements, remove some in these elements, and / or add other elements of similar or different functions. Adding element can include inserting element between existing element, for example, inserting amplifier and analog-to-digital converter. In various embodiments, the RF part includes antenna.

[0067] Additionally, the USB and / or HDMI terminals may include corresponding interface processors for connecting the system 100 to other electronic devices across the USB and / or HDMI connections. It will be appreciated that various aspects of input processing, such as Reed-Solomon error correction, may be implemented as desired, for example, within a separate input processing IC or within the processor 110. Similarly, various aspects of USB or HDMI interface processing may be implemented as desired, within a separate interface IC or within the processor 110. The demodulated, error-corrected, and demultiplexed streams are provided to various processing elements, including, for example, the processor 110 and the encoder / decoder 130, which operate in combination with memory and storage elements to process the data streams as desired for presentation on an output device.

[0068] The various elements of system 100 may be provided within an integrated housing within which the various elements may be interconnected and transmit data therebetween using a suitable connection arrangement 115, such as an internal bus as known in the art, including an I2C bus, wiring, and printed circuit boards.

[0069] The system 100 includes a communication interface 150 that enables communication with other devices via a communication channel 190. The communication interface 150 may include, but is not limited to, a transceiver configured to send and receive data through the communication channel 190. The communication interface 150 may include, but is not limited to, a modem or a network card, and the communication channel 190 may be implemented in, for example, a wired and / or wireless medium.

[0070] In various embodiments, a Wi-Fi network such as IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers) is used to stream data to the system 100. The Wi-Fi signal of these embodiments is received by a communication channel 190 and a communication interface 150 suitable for Wi-Fi communication. The communication channel 190 of these embodiments is usually connected to an access point or a router, which provides access to an external network including the Internet to allow streaming applications and other over-the-top communications. Other embodiments use a set-top box to provide streaming data to the system 100, which delivers data through the HDMI connection of the input block 105. Still other embodiments use the RF connection of the input block 105 to provide streaming data to the system 100. As indicated above, various embodiments provide data in a non-streaming manner. Additionally, various embodiments use a wireless network other than Wi-Fi, such as a cellular network or a Bluetooth network.

[0071] The system 100 can provide output signals to various output devices, including a display 165, a speaker 175, and other peripherals 185. The display 165 of various embodiments includes, for example, one or more of a touch screen display, an organic light emitting diode (OLED) display, a curved display, and / or a foldable display. The display 165 can be used for a television, a tablet computer, a laptop computer, a cellular phone (mobile phone), or other devices. The display 165 can also be integrated with other components (for example, in a smart phone), or it can be separate (for example, an external monitor for a laptop computer). In various examples of embodiments, the other peripherals 185 include one or more of a stand-alone digital video disk (or digital versatile disk) (DVR, both terms), a disk player, a stereo system, and / or a lighting system. Various embodiments use one or more peripherals 185 that provide functions based on the output of the system 100. For example, a disk player performs the function of playing the output of the system 100.

[0072] In various embodiments, control signals are communicated between the system 100 and the display 165, speaker 175, or other peripheral device 185 using signaling, such as AV.Link, CEC, or other communication protocols that enable device-to-device control with or without user intervention. The output devices can be communicatively coupled to the system 100 via dedicated connections through the corresponding interfaces 160, 170, and 180. Alternatively, the output devices can be connected to the system 100 using a communication channel 190 via the communication interface 150. The display 165 and speaker 175 can be integrated into a single unit with other components of the system 100 in an electronic device (e.g., a television). In various embodiments, the display interface 160 includes a display driver, such as a timing controller (TCon) chip.

[0073] For example, if the RF portion of input 105 is part of a separate set-top box, the display 165 and speaker 175 may alternatively be separate from one or more other components. In various embodiments where the display 165 and speaker 175 are external components, the output signals may be provided via dedicated output connections including, for example, an HDMI port, a USB port, or a COMP output.

[0074] The embodiments may be implemented by computer software implemented by the processor 110 or by hardware or by a combination of hardware and software. As a non-limiting example, the embodiments may be implemented by one or more integrated circuits. The memory 120 may be of any type suitable for the technical environment and may be implemented using any suitable data storage technology, such as, as non-limiting examples, optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory. As a non-limiting example, the processor 110 may be of any type suitable for the technical environment and may include one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture.

[0075] Figure 2 An encoder 200 is illustrated. Variations of the encoder 200 are contemplated, but for clarity, the encoder 200 is described below without describing all contemplated variations.

[0076] In some embodiments, Figure 2 Also illustrated is an encoder that improves the HEVC standard or the VVC standard, or an encoder that adopts a technology similar to HEVC or VVC, such as the encoder ECM developed by JVET (Joint Video Exploration Team).

[0077] Before being encoded, the video sequence may undergo pre-encoding processing (201), for example, applying a color transform to the input color picture (e.g., conversion from RGB 4:4:4 to YCbCr 4:2:0), or performing a remapping of the input picture components to obtain a signal distribution that is more resilient to compression (e.g., using histogram equalization of the color components), or resizing the picture (e.g., downscaling). Metadata may be associated with the pre-processing and attached to the bitstream.

[0078] In encoder 200, a picture is encoded by encoder elements as described below. The picture to be encoded is partitioned (202) and processed in units such as CUs (coding units) or blocks. In the present disclosure, different expressions may be used to refer to such units or blocks resulting from the partitioning of the picture. Such terms may be coding unit or CU, coding block or CB, luma CB or block... A CTU (coding tree unit) may refer to a group of blocks or a group of units. In some embodiments, a CTU may be considered as a block or as a unit itself.

[0079] Each unit is encoded using, for example, intra or inter mode. When the unit is encoded in intra mode, it performs intra prediction (260). In inter mode, motion estimation (275) and compensation (270) are performed. The encoder decides (205) which of intra mode or inter mode to use to encode the unit, and indicates the intra / inter decision by, for example, a prediction mode flag. The encoder can also mix (263) intra prediction results and inter prediction results, or mix results from different intra / inter prediction methods. For example, the prediction residual is calculated by subtracting (210) the prediction block from the original image block.

[0080] The motion refinement module (272) uses the already available reference pictures in order to refine the motion field of the block without referring to the original block. The motion field for a region can be considered as the set of motion vectors for all pixels in the region. If the motion vectors are sub-block based, the motion field can also be represented as the set of motion vectors for all sub-blocks in the region (all pixels within a sub-block have the same motion vector, and the motion vector can vary from sub-block to sub-block). If a single motion vector is used for the region, the motion field for the region can also be represented by the single motion vector (the same motion vector is used for all pixels in the region).

[0081] The prediction residual is then transformed (225) and quantized (230). The quantized transform coefficients along with motion vectors and other syntax elements are entropy encoded (245) to output a bitstream. The encoder may skip the transform and apply quantization directly to the untransformed residual signal. The encoder may bypass both the transform and quantization, i.e. the residual is directly encoded without applying the transform or quantization process.

[0082] The encoder decodes the coded block to provide a reference for further prediction. The quantized transform coefficients are dequantized (240) and inverse transformed (250) to decode the prediction residual. The decoded prediction residual and the prediction block are combined (255) to reconstruct the image block. An in-loop filter (265) is applied to the reconstructed picture to perform, for example, deblocking / SAO (sample adaptive offset) filtering to reduce coding artifacts. The filtered image is stored at a reference picture buffer (280).

[0083] Figure 3 A block diagram of a video decoder 300 is shown. In the decoder 300, the bitstream is decoded by the decoder elements as described below. The video decoder 300 generally performs the same Figure 2 The decoding process is the reverse of the encoding process described in . The encoder 200 also typically performs video decoding as part of encoding the video data.

[0084] In particular, the input to the decoder includes a video bitstream, which may be generated by the video encoder 200. The bitstream is first entropy decoded (330) to obtain transform coefficients, motion vectors and other encoding information. The picture partition information indicates how the picture is partitioned. Therefore, the decoder can divide (335) the picture according to the decoded picture partition information. The transform coefficients are dequantized (340) and inverse transformed (350) to decode the prediction residual. The decoded prediction residual and the prediction block are combined (355) to reconstruct the image block.

[0085] The prediction block may be obtained (370) from intra prediction (360) or motion compensated prediction (i.e., inter prediction) (375). The decoder may blend (373) intra prediction results and inter prediction results, or blend results from multiple intra / inter prediction methods. Prior to motion compensation, the motion field may be refined (372) by using already available reference pictures. An in-loop filter (365) is applied to reconstruct the image. The filtered image is stored at a reference picture buffer (380).

[0086] The decoded picture may further undergo post-decoding processing (385), such as inverse color transform (e.g., conversion from YCbCr 4:2:0 to RGB 4:4:4) or inverse remapping, which performs the inverse of the remapping process performed in the pre-encoding process (201), or resizing the reconstructed picture (e.g., upscaling). The post-decoding processing may use metadata derived in the pre-encoding process and signaled in the bitstream.

[0087] Some embodiments described herein relate to intra prediction used in image or video coding. As an example, for a given block to be predicted, the encoder selects the best intra prediction mode based on rate distortion and signals the selected intra prediction mode index to the decoder. In this way, the decoder can perform the same prediction for the block. Signaling the index of the selected intra prediction mode can add additional overhead, thereby reducing the coding gain from the intra portion of the encoded video. An example of encoding the index of the intra prediction mode selected to predict a given block is to create a list of most likely modes (MPMs), and if the index of the selected intra prediction mode belongs to the list, the signaling overhead is reduced thereby. This is a classic method for signaling an intra prediction mode index, called MPM list-based signaling. For example, this method is adopted in VVC and HEVC. The method is extended in ECM (Enhanced Compression Model), where two MPM lists are used instead of one. In the following, for the sake of simplicity, the MPM list-based signaling for signaling of the mode index is referred to as the signaling of the mode.

[0088] To further limit the signaling overhead, ECM also features two tools that derive, from the decoded pixels surrounding a given block, the intra-prediction mode that is most likely to be the best intra-prediction mode to predict the given block in terms of rate-distortion. For each of these two tools, the reduction in signaling overhead is due to the fact that the signaling of the tool alone enables the decoder to obtain the index of the most likely "best" intra-prediction mode.

[0089] The first tool is called decoder-side intra mode derivation (DIMD). The second tool is called template-based intra mode derivation (TIMD). More specifically, in DIMD, the template of the decoded pixels above and to the left of the current block to be predicted is analyzed to derive the directionality of the template, from which two directional intra prediction modes are selected. A prediction signal is generated by mixing these two intra prediction modes with a planar mode. In TIMD, several intra prediction modes are tested on a template of decoded pixels above and to the left of the current block. A cost (the sum of absolute transform differences (SATD)) is determined for each intra prediction mode tested between the decoded samples of the template and the predicted samples of the template using the tested intra prediction mode. The two intra prediction modes that produce the two minimum costs are retained. A prediction signal is generated by applying the intra prediction mode with the minimum SATD or mixing the two intra prediction modes that provide the two minimum costs.

[0090] In VVC and ECM, there are other intra prediction tools such as intra block copy (IBC) and matrix-based intra prediction (MIP). In IBC, a reference block from the reconstructed part of the current frame is copied and used as a prediction for the current block. In MIP, a matrix-vector multiplication is performed between a reference sample and a matrix selected from a set of available matrices to compute a prediction for the current block.

[0091] In HEVC and VVC, there are also inter prediction tools that predict the current block from the reconstructed frame. Some of these tools use bi-prediction and predict the current block by taking a weighted average of two inter predictions. Some tools such as combined intra inter prediction (CIIP) also perform a weighted average between intra prediction and inter prediction.

[0092] Some embodiments of the present disclosure relate to a method for encoding or decoding a block of an image or video, wherein a weighted average between two intra prediction modes is used to predict the block, thereby allowing to increase the performance of intra compression in ECM and VVC.

[0093] Combine the following Figure 4-18 An example of the intra prediction mode is described.

[0094] In this document, the term "intra mode" or "intra prediction mode" may refer to any of the intra prediction tools described herein, such as MIP, IBC, CIIP, or any of the core 67 intra prediction modes as known from HEVC, VVC or ECM, or any other mode that generates a prediction of a unit from reconstructed samples of the picture to which the unit belongs.

[0095] Core 67 intra prediction modes:

[0096] To capture arbitrary edge directions present in natural videos, the number of directional intra prediction modes is extended to 65 in VVC from 33 as used in HEVC. Figure 4 The directional intra prediction modes in VVC are illustrated, the directional modes from HEVC are shown with solid black arrows, and the new directional modes of VVC that are not in HEVC are depicted as black dashed arrows. These more dense directional intra prediction modes apply to all block sizes and both luma and chroma intra prediction. In VVC, in addition to the 65 directional intra prediction modes, planar mode and DC mode are provided, bringing the number of core intra prediction modes to 67.

[0097] From HEVC to VVC, the planar mode and DC mode remain unchanged except for the following minor modifications. In HEVC, each intra-coded block has a square shape and the length of each of its sides is a power of 2. Therefore, no division operation is required to generate the intra predictor using DC. In VVC, blocks can have a rectangular shape, which makes it necessary to use a division operation for each block in the general case. To avoid the division operation for DC prediction, only the longer sides are used to calculate the average of non-square blocks.

[0098] In ECM, the core structure of 67 intra prediction modes is inherited from that in VVC. This core structure is refined in ECM, where the four-tap interpolation for directional intra prediction modes from VVC becomes a six-tap interpolation in ECM. Position-dependent intra prediction combining (PDPC) is supplemented in ECM with gradient PDPC.

[0099] Intra prediction mode signaling in ECM

[0100] Intra-prediction mode signaling in terms of luma

[0101] In ECM (currently ECM-4.0), if the intra prediction mode selected to predict the current luma coding block (CB) is neither DIMD mode, nor matrix-based intra prediction (MIP) mode, nor TIMD mode, that is, it is one of the 67 intra prediction modes mentioned above, the MPM list of the CB is used to signal its index.

[0102] In ECM (currently ECM-4.0), the list of common MPMs is broken down into a list of 6 major MPMs and a list of 16 minor MPMs, as shown in Figure 5A and Figure 5B The general MPM list is constructed by sequentially adding the candidate intra prediction mode indices from the one most likely to be the selected intra prediction mode for predicting the current luma CB to the least likely one. Figure 5A From left to right, the sequential addition of candidate intra prediction mode indices is shown in the case where the current luma CB to be predicted belongs to an intra slice, and the current luma CB has a width W and a height H. The candidate intra prediction modes are inserted into the primary list or the secondary list in a specific order, and some intra prediction modes depend on Figure 5A and Figure 5BThe conditions illustrated in are inserted into one of the lists. For the primary list, the first MPM is a planar intra prediction mode, and then the subsequent intra prediction modes inserted are the intra prediction modes of the neighboring CBs used to predict the current luma CB. For the secondary list, two intra prediction modes obtained from DIMD are inserted, and then other intra prediction modes are inserted into the secondary list by considering the neighboring intra prediction modes of one or more intra prediction modes inserted into the primary list until the size limit of the secondary list is reached. When the size limit is not reached, some default modes can be added.

[0103] It will be noted that there is no redundancy in the generic list of MPMs, meaning that it cannot contain two identical intra-prediction mode indices. For readability, Figure 5A and Figure 5B The diagram illustrates the case where each candidate intra-frame prediction mode index is different from each other. However, in general, the slots with indices 0 to i-1 included in the general list of the MPM are already filled. If the current candidate intra-frame prediction mode index already exists in the current general list of the MPM, the candidate is skipped, and if the next candidate intra-frame prediction mode does not exist in the general list of the MPM, it is inserted into the slot of index i. Otherwise, if the current intra-frame prediction mode index does not exist in the general list of the MPM, the current intra-frame prediction mode index is inserted at the slot of index i, and the next candidate intra-frame prediction mode is inserted at the slot of index i+1.

[0104] exist Fig. 6A and 6B The signaling of the intra prediction mode selected to predict the current luma CB in an ECM (currently ECM-4.0) is depicted in FIG. Fig. 6A and 6B The signaling of the intra prediction mode selected to predict the current luma CB on the encoder side is described. But the same applies to the decoder side. Fig. 6A and 6BIn the , MRL indicates multiple reference lines. If the TIMD flag is equal to 1, the MRL index belongs to {0, 1, 3}. An MRL index of 0 means that MRL is not used to predict the current brightness CB. An MRL index of 1 means that the second row of decoded reference samples above the current brightness CB and the second column of decoded reference samples to the left of the current brightness CB are used for prediction. An MRL index of 3 means that the fourth row of decoded reference samples above the current brightness CB and the fourth column of decoded reference samples to the left of the current brightness CB are used for prediction. If the TIMD flag is equal to 0, the MRL index belongs to {0, 1, 3, 5, 7, 12}. ISP indicates intra-frame sub-partitions. The ISP mode index belongs to {0, 1, 2}. An ISP mode index of 0 means that ISP is not used for the current brightness CB. An ISP mode index of 1 indicates that the current brightness CB is horizontally divided into brightness transform blocks (TBs). An ISP mode index of 2 indicates that the current brightness CB is vertically divided into brightness TBs. In this figure, the intra prediction modes BDPCM (Intra Block DPCM), TMP (Intra Prediction Based on Template Matching), IBC (Intra Block Coding) and Palette are omitted because these tools are exclusively turned on for specific video sequences.

[0105] Intra prediction mode signaling for chroma

[0106] Figure 7 The signaling of the intra prediction mode selected in an ECM (currently ECM-4.0) to predict the current chrominance CB pair, which is the concatenation of the current luma CB, Cb and Cr CB. Figure 7 In , if the direct mode (DM) flag is equal to 1, the four possibilities for the current intra-frame prediction mode index are the index of the planar mode, the index of the horizontal mode, the index of the vertical mode, and the index of the DC. To avoid any redundancy, if DM is one of the four modes mentioned above, the index of the redundant mode is replaced by the index of the vertical diagonal mode in the set of the four modes. Note that in ECM (currently ECM-4.0), the cross-component linear model (CCLM) collects six different intra-frame prediction modes, labeled LM, MMLM, MDLM_L, MDLM_T, MMLM_L, and MMLM_T, while in VVC, CCLM only collects three intra-frame prediction modes.

[0107] Wide Angle Intra Prediction (WAIP)

[0108] In VVC and ECM, for non-square blocks, several conventional angular intra prediction modes are replaced by wide-angle modes. The replaced modes are signaled using the original method and remapped to the index of the wide-angle mode after parsing. The total number of core intra prediction modes remains unchanged, i.e. 67.

[0109] For the current WxH block to be predicted, Figure 8 A decoded reference sample set is shown, consisting of an array of top decoded reference samples of length 2W+1 and an array of left decoded reference samples of length 2H+1. Figure 8 The relationship between the range of decoded reference samples around the current WxH block and the range of allowed intra prediction angles is also shown.Table 1 presents the index of the intra prediction mode replaced by the wide mode in VVC and ECM, depending on the size of the current block to be predicted.

[0110]

[0111] Table 1: Indexes of intra prediction modes replaced by wide mode in VVC and ECM (67 core intra prediction modes).

[0112] Fig. 9 An example of how angular intra modes are replaced by wide modes for non-square blocks whose width is strictly greater than their height is shown. In this example, mode 2 is replaced by wide mode 67. Mode 3 is replaced by wide mode 68. For example, if the current block to be predicted is 8×4, this replacement process will proceed incrementally until mode 7 is replaced by wide mode 72.

[0113] Template-based Intra Mode Derivation (TIMD)

[0114] for Fig.10 Given a luma CB (1003) in (a), the following mode derivation via TIMD is applied in the same way on the encoder and decoder sides. For each intra prediction mode in the MPM list of this luma CB, supplemented with a default mode if necessary, the TIMD mode determines the prediction of the template (1000 and 1001) of the luma CB from the decoded reference samples of the template (1002), and determines the SATD between the prediction reference samples of the template and the decoded reference samples of the template of the luma CB. In the first pass, the two intra prediction modes with the smallest SATD are selected as TIMD modes. This means that the set of possible intra prediction modes derived via TIMD collects 131 modes. After retaining two intra prediction modes in the first pass involving the MPM list supplemented with the default mode, for each of the two selected intra prediction modes, if the mode is neither a planar mode nor a DC mode, TIMD also checks the two closest extended direction intra prediction modes of the selected intra prediction mode based on the SATD cost. In this second pass, by Figure 4 By inserting a direction between each black arrow in and its adjacent black dashed arrow, the directional intra prediction mode set is expanded from 65 to 129, thereby providing an extended directional intra prediction mode.

[0115] Note that in the above description, it is assumed that the template of the brightness CB does not exceed the boundary of the current frame. In the case where at least a part of the template of the brightness CB exceeds the boundary of the current frame, Fig.10 (b) and (c) illustrate how the template is adapted.

[0116] exist Fig.10 In (a), the current W×H brightness CB (1003) is surrounded by its fully available template, which is composed of w t ×H part (1000) and the W×h above it t During the TIMD derivation step, the tested intra prediction mode is selected from the template's 1+2w t +2W+2h t +2H decoded reference sample set (1002) predicts the template of the current brightness CB. In the current version of ECM (ECM-4.0), if W≤8, then w t is equal to 2, otherwise w t is equal to 4. If H≤8, then h t is equal to 2, otherwise h t is equal to 4. Fig.10 In (b), the current W×H brightness CB (1003) is surrounded by its template, where only the W×h t The above part (1001) is available. During the TIMD derivation step, the tested intra prediction mode is derived from the template's 1+2W+2h t +2H decoded reference sample set (1002) predicts the template of the current brightness CB. Fig.10 In (c), the current W×H brightness CB (1003) is surrounded by its template, where only its left side has w t The ×H part (1000) is available. During the TIMD derivation step, the tested intra prediction mode is selected from the template's 1+2w t The +2W+2H decoded reference sample set (1002) predicts the template of the current brightness CB.

[0117] To predict the current luma CB via TIMD, after applying PDPC, the two intra predictions obtained from the two TIMD modes selected in the first or second pass for the luma CB are fused with weights. The weights used depend on the predicted SATD of the two TIMD modes.

[0118] Because the set of directional intra prediction modes is extended from 65 to 129 for TIMD, the intra prediction mode replacement in WAIP is adapted. Table 1 is adapted to Table 2. For example, for a given 8×4 luma CB using TIMD, mode 2 is replaced by wide mode 131, mode 3 is replaced by wide mode 132, mode 4 is replaced by wide mode 133, ..., mode 12 is replaced by wide mode 141.

[0119] condition Alternative intra prediction mode index W / H==2 [|2，12|] W / H==4 [|2，20|] W / H==8 [|2，24|] W / H==16 [|2，28|] W / H==32 [|2，30|] W / H==1 none H / W==2 [|120，130|] H / W==4 [|112，130|] H / W==8 [|108，130|] H / W==16 [|104，130|] H / W==32 [|102，130|]

[0120] Table 2: Index of intra prediction modes replaced by wide mode in TIMD in ECM.

[0121] Matrix-based Intra Prediction (MIP)

[0122] The matrix-based intra prediction (MIP) method is a newly added intra prediction technique to VVC. To predict samples of a rectangular block of width W and height H, MIP takes as input a column of H reconstructed neighboring boundary samples to the left of the current block and a row of W reconstructed neighboring boundary samples above the current block. If the reconstructed samples are not available, they are generated like conventional intra prediction. The generation of the prediction signal is based on the following three steps: optional averaging of the reconstructed neighboring boundary samples, matrix-vector multiplication between the MIP weight matrix and the averaged neighboring boundary samples, and optional linear interpolation from the results of the previous multiplications, as in Fig.11 as shown in .

[0123] In ECM, until ECM-4.0, MIP has not been modified with respect to its implementation in VVC.

[0124] Decoder-side intra mode derivation (DIMD)

[0125] For a given luma CB to be predicted, DIMD derives two intra prediction modes from templates of reconstructed neighboring samples surrounding the luma CB, and combines these two intra predictors with a planar mode predictor using weights derived from gradients determined from the templates. The division operations in the weight derivation are performed using the same lookup table (LUT)-based integration scheme used by the cross-component linear model (CCLM). For example, the division operations in the orientation calculation

[0126] Orient=G y / G x

[0127] Computed by the following LUT-based scheme:

[0128] x = Floor(Log2(Gx))

[0129] normDiff=((Gx<<4)>>x)&15

[0130] x+=(3+(normDiff!=0)?1:0)

[0131] Orient=(Gy*(DivSigTablenormDiff]|8)+(1<<(x-1)))>>X

[0132] in

[0133] DivSigTable

[16] ={0, 7, 6, 5, 5, 4, 4, 3, 3, 2, 1, 1, 11, 0}.

[0134] The two derived intra prediction modes are included in the main list of MPMs. Therefore, for a given luma CB to be predicted, the DIMD process is performed before creating the MPM list. For a given luma CB, the main intra prediction mode derived via DIMD is stored and used for the MPM list construction of the neighboring luma CBs.

[0135] Geometric Partitioning Model (GPM)

[0136] In VVC and ECM, geometric partition mode is supported for inter prediction. A CU-level flag is used as a merge mode to signal the geometric partition mode, where other merge modes include normal merge mode, MMVD mode, CIIP mode, and sub-block merge mode. For each possible CU size w×h=2 m ×2 n , where m, n ∈ {3…6} excluding 8x64 and 64x8, the geometric partitioning mode supports a total of 64 partitions.

[0137] When using GPM mode, the CU is split into two parts by a geometrically positioned straight line. Fig.12 Some examples are illustrated in Figure 1. The position of the partition line is derived mathematically from the angle and offset parameters of the specific partition. Each part of the geometric partition in the CU uses its own motion for inter-frame prediction; each partition only allows uni-prediction, i.e., each part has one motion vector and one reference index. Uni-prediction motion constraints are applied to ensure that only two motion compensated predictions are required for each CU, as with conventional bi-prediction.

[0138] If geometry partition mode is used for the current CU, a geometry partition index indicating the partition mode (angle and offset) of the geometry partition and two merge indices (one for each partition) are further signaled. The number of maximum GPM candidate sizes is explicitly signaled in the SPS and the syntax for specifying the GPM merge indices is binarized. After predicting each part of the geometry partition, a hybrid process with adaptive weights is used to adjust the sample values ​​along the edges of the geometry partitions. This is the prediction signal for the entire CU and the transform and quantization process is applied to the entire CU as in other prediction modes.

[0139] Blending along geometric partition edges

[0140] After predicting each part of a geometric partition using its own motion, blending is applied to the two prediction signals to derive samples around the geometric partition edges. The blending weight for each position of the CU is derived based on the distance between the individual position and the partition edge.

[0141] The distance from position (x, y) to the edge of the partition is derived as follows

[0142]

[0143] where i, j are the indices of the angle and offset of the geometry partition, depending on the signaled geometry partition index. x，j and ρ y，j The sign of depends on the angle index i.

[0144] The weights for each part of the geometric partition are derived as follows:

[0145]

[0146] partIdx depends on the angle index i. Fig.13 An example of weight w0 is illustrated in FIG.

[0147] GPM Intra

[0148] In the exploratory experiments in ECM-3.1, for ECM-4.0, a GPM intra prediction mode is provided, where the intra prediction mode is added to GPM to combine inter prediction with intra prediction. Four tests were conducted.

[0149] In a first variant (test a), a GPM with inter and intra prediction is provided. In the GPM with inter and intra prediction, the final prediction samples are generated by weighting the inter prediction samples and the intra prediction samples for each GPM separation region. The inter prediction samples are derived by the same scheme as the GPM in the current ECM, while the intra prediction samples are derived from the intra prediction mode (IPM) candidate list and the index signaled from the encoder. The IPM candidate list size is predefined as 3. The available IPM candidates are parallel angle mode (parallel mode) relative to the GPM block boundary, vertical angle mode (vertical mode) relative to the GPM block boundary, and planar mode, as Fig.14 (a)-(c). In addition, Fig.14 The GPM with intra and intra prediction shown in (d) is restricted to the GPM with intra to reduce the signaling overhead of IPM and avoid the increase in the size of the intra prediction circuit on the hardware decoder. In addition, direct motion vector and IPM storage on the GPM mixed region is introduced to further improve the coding performance.

[0150] In the second variant (test b), the following two modifications are introduced into the first variant to achieve higher coding performance: a combination of IPM derivation based on DIMD and neighboring patterns, GPM-Intra and GPM-MMVD.

[0151] The IPM candidate list size is the same as the first variant, and the parallel mode is registered first. Therefore, if the same IPM candidate does not exist in the list, a maximum of two IPM candidates derived from the decoder-side intra mode derivation (DIMD) method and / or neighboring blocks in ECM-3.1 can be registered. As for the neighboring mode derivation, there are up to five positions for the available neighboring blocks, but they are restricted by the angle of the GPM block boundaries, as shown in Table 3 below, which has been used for GPM with template matching (GPM-TM) in ECM-3.1.

[0152] Unlike the first variant, GPM with intra prediction (GPM-Intra) can be used together with GPM with merging with motion vector differences (GPM-MMVD) already implemented in ECM-3.1.

[0153] GPM Angle 0 2 3 4 5 8 11 12 13 14 First Division A A A A L+A L+A L+A L+A A A Second Division L+A L+A L+A L L L L L+A L+A L+A GPM Angle 16 18 19 20 21 24 27 28 29 30 First Division A A A A L+A L+A L+A L+A A A Second Division L+A L+A L+A L L L L L+A L+A L+A

[0154] Table 3: Location of available neighboring blocks for IPM candidate derivation based on the angle of the GPM block boundary. A and L indicate the top and left of the prediction block.

[0155] In the third variant (test c), the template-based intra mode derivation (TIMD) in ECM-3.1 can be additionally used for IPM candidates within the GPM-frame to further improve the coding performance. The IPM candidate list size is also the same as the first variant. The parallel mode can be registered first, and then the TIMD, DIMD and IPM candidates of the adjacent blocks are registered in sequence.

[0156] In a fourth variation (test d), in addition to the third variation, the application rate of the GPM-Intra blocks may be increased by utilizing GPM-Intra with GPM with template matching (GPM-TM).

[0157] Intra Block Copy (IBC)

[0158] Intra-block copy (IBC) is a tool adopted in the HEVC extension on SCC. The IBC tool significantly improves the coding efficiency of screen content materials. Since the IBC mode is implemented as a block-level coding mode, block matching (BM) is performed at the encoder to find the best block vector (or motion vector) for each CU. Here, the block vector is used to indicate the displacement from the current block to the reference block, which has been reconstructed inside the current picture. The luminance block vector of the IBC-encoded CU is integer precision. The chrominance block vector is also rounded to integer precision. When used in combination with AMVR, the IBC mode can switch between 1 pixel and 4 pixel motion vector precision. The IBC-encoded CU is regarded as a third prediction mode different from the intra or inter prediction mode. The IBC mode is applicable to CUs in which both the width and height are less than or equal to 64 luminance samples.

[0159] On the encoder side, hash-based motion estimation is performed for IBC. The encoder performs RD checks on blocks where the width or height is not greater than 16 luma samples. For non-merge mode, a block vector search is first performed using a hash-based search. If the hash search does not return a valid candidate, a local search based on block matching is performed.

[0160] In hash-based search, the hash key matching (32-bit CRC) between the current block and the reference blocks is extended to all allowed block sizes. The hash key calculation for each position in the current picture is based on 4×4 sub-blocks. For larger-sized current blocks, the hash key is determined to be the hash key of the matching reference block when all hash keys of all 4×4 sub-blocks match the hash key in the corresponding reference position. If the hash keys of multiple reference blocks are found to match the hash key of the current block, the block vector cost of each matching reference is calculated and the block vector cost with the minimum cost is selected.

[0161] In the block matching search, the search range is set to cover both the previous and current CTUs.

[0162] At CU level, IBC mode is signaled with a flag, and it can be signaled as IBC AMVP mode or IBC Skip / Merge mode as follows:

[0163] IBC skip / merge mode: The merge candidate index is used to indicate which block vector from the list of neighboring candidate IBC coded blocks is used to predict the current block. The merge list consists of spatial, HMVP and pairwise candidates.

[0164] IBC AMVP mode: Block vector differences are encoded in the same way as motion vector differences. The block vector prediction method uses two candidates as predictors, one from the left neighbor and one from the top neighbor (if IBC coded). When either neighbor is not available, the default block vector will be used as the predictor. A signaling flag is used to indicate the block vector predictor index.

[0165] IBC Reference Area

[0166] To reduce memory consumption and decoder complexity, IBC in VVC only allows reconstruction of parts of predefined areas - including the area of ​​the current CTU and some areas to the left of the CTU. Fig.15 An example of a reference area for IBC mode is shown, where each block represents a 64×64 unit of luma samples. The current block Curr to be predicted is shown with stripes, the grey blocks correspond to the reconstructed blocks, and the blocks marked with an X are blocks in the reconstructed area that are not available for IBC. Fig.15 As shown in , depending on the location of the current coded CU position within the current CTU, the following applies:

[0167] If the current block falls into the upper left 64×64 block of the current CTU, in addition to the samples already reconstructed in the current CTU, using the CPR mode, it can also refer to the reference samples in the lower right 64×64 block of the left CTU. Using the CPR mode, the current block can also refer to the reference samples in the lower left 64×64 block of the left CTU and the reference samples in the upper right 64×64 block of the left CTU.

[0168] If the current block falls into the upper right 64×64 block of the current CTU, in addition to the samples already reconstructed in the current CTU, if the brightness position (0, 64) relative to the current CTU has not been reconstructed, the CPR mode is used, and the current block can also refer to the reference samples in the lower left 64×64 block and the lower right 64×64 block of the left CTU; otherwise, the current block can also refer to the reference samples in the lower right 64x64 block of the left CTU.

[0169] If the current block falls into the lower left 64×64 block of the current CTU, in addition to the samples that have been reconstructed in the current CTU, if the luma position (64, 0) relative to the current CTU has not been reconstructed, the CPR mode is used, and the current block can also refer to the reference samples in the upper right 64×64 block and the lower right 64×64 block of the left CTU. Otherwise, using the CPR mode, the current block can also refer to the reference samples in the lower right 64×64 block of the left CTU.

[0170] If the current block falls into the lower right 64×64 block of the current CTU, the CPR mode is used, which can only refer to the samples that have been reconstructed in the current CTU.

[0171] This restriction allows the IBC mode to use local on-chip memory for hardware implementation.

[0172] IBC adaptation of camera captured content

[0173] IBC is an effective tool for encoding screen content. It also shows the improvement of coding efficiency of some camera-captured content at the cost of a significant increase in encoding time. The following describes an IBC adaptation scheme based on EE2-3.2 software. The adaptation shows that IBC can achieve good coding gain with a controllable increase in encoding time.

[0174] When the CTU size is 128×128, the decoder is exactly the same as EE2-3.2. This means that the reference area of ​​​​the IBC is extended to two CTU rows above the current CTU, such as Fig.16 Specifically, for a CTU (m, n) to be encoded, the reference area includes CTUs with indices (m-2, n-2) ... (W, n-2), (0, n-1) ... (W, n-1), (0, n) ... (m, n), where W indicates the maximum horizontal index in the current picture.

[0175] However, when the CTU size is 256x256, the two additional rows of CTUs above may require additional memory. To prevent IBC from using additional memory, when the CTU size is 256×256, the reference area is as follows Fig.17 Specifically, assuming that the current CTU index is (m, n), the reference region includes CTUs with indices (0, n) ... (m, n) and (m-1, n-1) ... (W, n-1), such as Fig.17 The lighter gray block is shown in the figure, and the darker gray block is the current CTU (m, n).

[0176] When the CTU size is 256×256, in addition to the changes in the EE2-3.2 decoder, the EE2-3.2 encoder is modified to limit the per-sample block vector search (or local search) range to horizontally [–12, 12] and vertically [–12, 12] centered on the first block vector predictor of each IBC block.

[0177] Combined Intra-Inter Prediction (CIIP)

[0178] In VVC and ECM, when a CU is encoded in merge mode, if the CU contains at least 64 luma samples (i.e., the CU width multiplied by the CU height is equal to or greater than 64), and if both the CU width and the CU height are less than 128 luma samples, an additional flag is signaled to indicate whether the combined inter / intra prediction (CIIP) mode is applied to the current CU. As its name indicates, CIIP prediction combines the inter prediction signal with the intra prediction signal. The inter prediction signal P in CIIP mode is derived using the same inter prediction process applied to the regular merge mode. inter ; and the intra-frame prediction signal P intra is derived after the conventional intra prediction process using planar mode. The intra and inter prediction signals are then combined using weighted averaging, where the weight values ​​are calculated depending on the coding mode of the top and left neighboring blocks as follows (in Fig.18 (depicted in):

[0179] If the top neighbor is available and is intra-coded, set isIntraTop to 1, otherwise set isIntraTop to 0;

[0180] If the left neighbor is available and is intra-coded, set isIntraLeft to 1, otherwise set isIntraLeft to 0;

[0181] If (isIntraLeft+isIntraTop) is equal to 2, set wt to 3;

[0182] Otherwise, if (isIntraLeft+isIntraTop) is equal to 1, set wt to 2;

[0183] Otherwise, set wt to 1.

[0184] The CIIP forecast is formed as follows:

[0185] PCIIP=((4-wt)*P inter +wt*P intra +2)>>2.

[0186] Embodiments of methods and apparatus for encoding or decoding a block of an image or video are described herein in conjunction with FIGS. 19-26 , wherein the block is predicted based on a combination of a first predictor block and a second predictor block, the first and second predictor blocks being obtained from a first intra prediction mode and a second intra prediction mode, respectively. Hereinafter, a mode of predicting a block based on such a combination in any of the embodiments described herein is referred to as luma intra fusion.

[0187] Any of the embodiments described herein may be implemented in an intra prediction module of an image or video encoder / decoder, such as the intra prediction module 260 of the encoder 200 and the intra prediction module 360 ​​of the decoder 300 .

[0188] In an embodiment, the first intra prediction mode is obtained from a first intra prediction mode set and the second intra prediction mode is obtained from a second intra prediction mode set, the first and second intra prediction modes being different. In a variant, the first and second intra prediction mode sets are also different.

[0189] In one embodiment, the first intra-prediction mode set includes non-directional intra-prediction modes, and the second intra-prediction mode set includes directional intra-prediction modes (IPMs). One of the first intra-prediction mode and the second intra-prediction mode is obtained from the first intra-prediction mode set, and the other of the first intra-prediction mode and the second intra-prediction mode is obtained from the second intra-prediction mode set.

[0190] For example, the first intra-frame prediction mode set includes one or more of the following intra-frame prediction modes: planar mode, DC mode, MIP mode, IBC mode. In a variant, the MIP mode, when included in the first intra-frame prediction mode set, is an adapted MIP mode using a matrix specially trained for not capturing the directional characteristics of the block. In another variant, the first intra-frame prediction mode set may also include any other intra-frame prediction mode that does not capture the directional characteristics of the block or only captures the lower frequency of the signal of the block.

[0191] For example, the second intra-frame prediction mode set includes one or more of the directional intra-frame prediction modes, such as one or more of the following intra-frame prediction modes: one or more of the 67 directional intra-frame prediction modes of VVC or ECM, one or more of the extended directional intra-frame prediction modes of TIMD mode, one or more wide-angle intra-frame prediction modes, directional intra-frame prediction modes parallel to the partition edges of the geometric partitioning pattern, directional intra-frame prediction modes perpendicular to the partition edges of the geometric partitioning pattern, an adapted MIP mode using a matrix trained to collect only directional features of a block, or an intra-frame prediction mode provided by a DIMD or TIMD process. The second intra-frame prediction mode set may also include any other intra-frame prediction mode that captures one or more directional features of a block or captures the high frequency of a signal of a block.

[0192] It is understood that, unless otherwise stated, any one of the embodiments described above and below may be combined with any one or more of the other embodiments described above and below.

[0193] Fig.19A An example of a method 1900 for encoding a block of an image or video according to an embodiment is illustrated. At 1901, a first predictor block is obtained from a first intra-frame prediction mode. In a variation, the first intra-frame prediction mode can be determined by testing the intra-frame prediction mode of the first intra-frame prediction mode set, determining the rate-distortion cost of each intra-frame prediction mode in the first intra-frame prediction mode set, and selecting the intra-frame prediction mode that is best in terms of rate-distortion. At 1902, a second predictor block is obtained from a second intra-frame prediction mode. In a variation, the second intra-frame prediction mode can be determined by testing the intra-frame prediction mode of the second intra-frame prediction mode set in the same manner as the first intra-frame prediction mode. In another variation, the second intra-frame prediction mode can be derived from reconstructed samples adjacent to the block. Other variations for determining the second prediction mode are further described below. At 1903, a luma intra-frame fusion prediction is obtained, that is, a prediction for a block is obtained by combining the first block predictor and the second block predictor. An embodiment for combining the first block predictor and the second block predictor is further described below. At 1904, the block is encoded using prediction from luma intra fusion.

[0194] Fig.19B An example of a method 1910 for decoding a block of an image or video according to an embodiment is illustrated. At 1911, a first predictor block is obtained from a first intra prediction mode. As an example, the first intra prediction mode can be determined by decoding a syntax element indicating the first intra prediction mode from a bitstream. At 1912, a second predictor block is obtained from a second intra prediction mode. In a variation, the second intra prediction mode can be determined by decoding one or more syntax elements that provide the second intra prediction mode. In another variation, the second intra prediction mode can be derived on the decoder side in the same manner as on the encoder side. Some variations for determining the second intra prediction mode are further described below. At 1913, a luma intra fusion prediction of the block is obtained by combining a first block predictor and a second block predictor. An embodiment for combining a first block predictor and a second block predictor is further described below. At 1914, a prediction from luma intra fusion is used to decode / reconstruct the block.

[0195] Fig. 20AAn example of a method 2000 for encoding a block of an image or video according to another embodiment is illustrated. In this embodiment, a determination of prediction using luma intra fusion is responsive to a determination of whether two intra prediction modes are to be combined. At 2001, a first predictor block is obtained from a first intra prediction mode. In a variation, the first intra prediction mode may be combined with Fig.19A In 2002, a second predictor block is obtained from a second intra-frame prediction mode. In a variant, the second intra-frame prediction mode can be combined with Fig.19A The same manner as in the described embodiments or using any other variants described further below is used to determine. At 2003, it is determined whether the first intra prediction mode is to be combined or merged with the second intra prediction mode. In other words, it is determined whether the first predictor block is to be combined with the second predictor block.

[0196] For example, a first intra-frame prediction mode is determined to be combined with a second intra-frame prediction mode based on a cost. In this variant, the cost is obtained when determining the second intra-frame prediction mode, for example, when testing each intra-frame prediction mode in the second intra-frame prediction mode set or when determining the second intra-frame prediction mode using a TIMD process. Other variants may be used to determine whether to combine the first and second intra-frame prediction modes.

[0197] If it is determined at 2003 that the first intra prediction mode is to be combined with the second intra prediction, a prediction for the block is obtained by combining the first block predictor and the second block predictor at 2004. Embodiments for combining the first block predictor and the second block predictor are further described below.

[0198] If it is determined at 2003 that the first intra prediction mode is not combined with the second intra prediction, a prediction for the block is obtained without combining the first block predictor and the second block predictor at 2005. The prediction for the block may be obtained from the first block predictor or any other prediction, such as the second intra prediction mode or another intra prediction mode or an inter prediction mode.

[0199] In 2006, the block is encoded using the prediction obtained in 2004 or 2005.

[0200] exist Fig. 20A In another variation, before determining the second intra prediction mode, a determination is made whether to combine the two intra prediction modes. For example, the determination may be made based on the size of the block or based on the first intra prediction mode, or other variations described below may be used.

[0201] In this further variation, the second intra prediction mode is determined only if it is determined that two intra prediction modes are to be combined.

[0202] Fig. 20B An example of a method 2010 for decoding a block of an image or video according to another embodiment is illustrated. At 2011, a first predictor block is obtained from a first intra prediction mode. As an example, the first intra prediction mode may be determined by decoding one or more syntax elements indicating the first intra prediction mode from a bitstream.

[0203] At 2012, it is determined whether the first intra prediction mode is to be combined or merged with the second intra prediction mode. This determination may be made based on the size of the block or based on the first intra prediction mode, or based on one or more syntax elements decoded from the bitstream, or based on other variations described below.

[0204] If it is determined in 2012 that the first intra prediction mode is to be combined with the second intra prediction mode, then in 2013, the second intra prediction mode is determined and a second predictor block is obtained from the second intra prediction mode. In a variation, the second intra prediction mode may be combined with Fig.19B The first block predictor and the second block predictor are then combined to obtain a prediction for the block. Embodiments for combining the first block predictor and the second block predictor are further described below.

[0205] If it is determined at 2012 that the first intra prediction mode is not combined with the second intra prediction, then at 2015, a prediction for the block is obtained without combining the first block predictor and the second block predictor. The prediction for the block may be obtained from the first block predictor or any other prediction, such as the second intra prediction mode or another intra prediction mode or an inter prediction mode. In any case, the prediction mode for the block is the same as the prediction mode used when encoding the block.

[0206] In 2016, the block is decoded / reconstructed using the prediction obtained in 2014 or 2015.

[0207] Fig. 20C An example of a method 2020 for decoding a block of an image or video according to another embodiment is illustrated. Fig. 20B In a similar manner, a first predictor block is obtained from the first intra prediction mode. In this embodiment, after obtaining the second intra prediction mode at 2022, a determination is made at 2023 whether to combine the two intra prediction modes. Depending on the variant, the second intra prediction mode can be determined by decoding one or more syntax elements that provide the second intra prediction mode, or can be derived on the decoder side in the same manner as on the encoder side. Some variants for determining the second intra prediction mode are further described below. Then, a second block predictor is obtained from the second intra prediction mode that has been determined.

[0208] At 2023, it is determined whether the first intra prediction mode is to be combined or merged with the second intra prediction mode. Fig. 20A The determination is made in the same manner or based on other variations described below.

[0209] If it is determined in 2023 that the first intra prediction mode is to be combined with the second intra prediction mode, then in 2024, a prediction for the block is obtained by combining the first block predictor and the second block predictor. An embodiment for combining the first block predictor and the second block predictor is further described below. If it is determined in 2023 that the first intra prediction mode is not to be combined with the second intra prediction mode, then in 2025, a prediction for the block is obtained without combining the first block predictor and the second block predictor. The prediction for the block may be obtained from the first block predictor or any other prediction, such as the second intra prediction mode or another intra prediction mode or an inter prediction mode. In any case, the prediction mode for the block is the same as the prediction mode used when encoding the block.

[0210] At 2026 , the block is decoded / reconstructed using the prediction obtained at 2024 or 2025 .

[0211] Some embodiments for determining the second intra prediction mode are described below.

[0212] In some embodiments, the second intra prediction mode is a directional intra prediction mode.

[0213] In some embodiments, the directional intra prediction modes to be combined are derived from (one or more) decoder-based processes, i.e., decoder-side intra mode derivation (DIMD) or template-based intra mode derivation (TIMD), to avoid having to signal the IPM index of the second intra prediction mode. In the following, it is assumed that the directional intra prediction modes to be combined are derived from (one or more) decoder-based processes.

[0214] In a variation in which the first prediction mode is determined to be the planar mode, the second intra prediction mode may be derived using the following variation.

[0215] Classically, the DIMD process combines two IPMs with planar mode. Therefore, in embodiments using DIMD to derive the second intra prediction mode (directional mode), the combination of two intra prediction modes (hereinafter also referred to as luma intra fusion) may not be applicable to luma CB using planar mode.

[0216] In embodiments where TIMD is used to derive a second intra prediction mode (an additional directional intra prediction mode), planar mode may be removed from the TIMD search.

[0217] In an embodiment, the second intra prediction mode may be determined as the second MPM (MPM[1]) of the list. Thus, in this embodiment, the first and second MPMs (MPM[0] and MPM[1]) from the MPM list are combined because the plane is MPM[0] and MPM[1] is typically a direction-based mode. In a variant, such a combination of the first and second MPMs of the MPM list is only done under certain conditions. For example, this may be done when the neighbors to the left and above are close: i.e., the same IPM is used to the left and above, or the indices to the left and above have an absolute difference of 1.

[0218] In a variation in which the first prediction mode is determined to be the DC mode, the second intra prediction mode may be derived using the following variation.

[0219] In embodiments where TIMD is used to derive a second intra prediction mode (an additional directional intra prediction mode), DC may be removed from the TIMD search.

[0220] In a variation in which the first prediction mode is determined to be a MIP mode, the following variation may be used to derive a second intra prediction mode or adapt the MIP mode. As described above, the MIP mode uses matrices from a trained matrix set to generate predictions for a block. In order to efficiently apply a direction-based IPM (a second intra prediction mode) on top of the MIP mode (used as the first intra prediction mode), the matrices used in the MIP mode may be retrained to not capture the direction of the block. This means that the retrained matrices will only predict non-directional information in the block, leaving the directional information to be predicted from the luma intra fusion mode that combines the first and second intra prediction modes.

[0221] In another embodiment, only a subset (eg, half) of the matrices are retrained to not predict the direction(s), while the other matrices are retrained in the same manner as described above with respect to the classical MIP mode.

[0222] In those embodiments, when MIP mode is used as the first intra prediction mode of a combination of two intra prediction modes (luminance intra fusion), only matrices that have been trained not to predict directional information may be selected in MIP mode.

[0223] In a variant, the combination of the two intra prediction modes (luminance intra fusion) is always applied to the retrained matrix, in such a way that no additional signaling is required. In fact, "no additional signaling is required" because (1) the identification of the directional intra prediction modes involved in luma intra fusion does not incur signaling costs when the index of the directional intra prediction mode (the second intra prediction mode) is derived by the decoder-based process, and (2) since luma intra fusion is always applied to the MIP mode (the first intra prediction mode), there is no need to signal luma intra fusion.

[0224] In a variation in which the first prediction mode is determined to be the IBC mode, the second intra prediction mode may be derived using the following variation.

[0225] In an embodiment, in case luma intra blending is used on a luma CB using IBC as the first intra prediction mode, the second intra prediction mode uses the intra mode used by the original luma CB pointed to by the IBC motion vector.

[0226] In an embodiment, the application of luma intra blending is restricted to the case where the block pointed to by the used IBC motion vector is a regular intra coded block using directional IPM.

[0227] In an embodiment, luma intra blending is also applied when the pointed-to block is not coded using conventional intra but the propagated intra information is a directional mode.

[0228] Depending on the embodiment for luma intra fusion (i.e., combining the first and second intra prediction modes), one or more syntax elements are signaled in the bitstream along with the coded data representing the block. The one or more syntax elements may signal one or more of the following: the first intra prediction mode, the second intra prediction mode, an indicator indicating whether the first intra prediction mode is to be combined with the second intra prediction mode, an indicator indicating whether the second intra prediction mode is derived from a decoder-side intra mode or a template-based intra mode derivation, an indicator indicating a weight among a set of weights to be used for the first predictor block when combining the first and second predictor blocks.

[0229] Some embodiments for signaling one or more syntax elements are described.

[0230] Fig.21A An example of a method 2100 according to an embodiment is illustrated for signaling in a bitstream or decoding from a bitstream one or more syntax elements that provide for determining one or more intra-prediction modes for predicting a block of an image or video. For example, the method 2100 may be combined with Fig.19A and Fig.19B The embodiments described herein are combined. It is assumed here that the block is encoded using a prediction combining a first intra prediction mode and a second prediction mode, as explained in any of the embodiments described herein. At 2101, one or more syntax elements are encoded in a bitstream, respectively decoded from the bitstream, indicating a first intra prediction mode. At 2102, one or more syntax elements are encoded in a bitstream, respectively decoded from the bitstream, indicating a second intra prediction mode.

[0231] Fig. 21BAn example of a method 2110 for signaling or decoding one or more syntax elements in or from a bitstream according to another embodiment, wherein the one or more syntax elements provide one or more intra-prediction modes for determining a block of a predicted image or video. For example, the method 2110 may be combined with Figures 20A-20B -20C described embodiments are combined. It is assumed here that the block is encoded using a prediction combining a first intra-frame prediction mode and a second prediction mode, as explained in any one of the embodiments described herein. At 2111, one or more syntax elements are encoded in a bitstream and decoded from the bitstream, respectively, indicating the first intra-frame prediction mode. At 2112, one or more syntax elements are encoded in a bitstream and decoded from the bitstream, respectively, indicating whether the first intra-frame prediction mode will be combined with the second intra-frame prediction mode. In other words, it is signaled here whether the luma intra-frame fusion mode is used for the block. In a variant, the second intra-frame prediction mode is not signaled in the bitstream. Therefore, if the first intra-frame prediction mode is to be combined with the second intra-frame prediction mode, the second intra-frame prediction mode is derived at the decoder in the same manner as in the encoder. In another variant, one or more syntax elements can be signaled to indicate the second intra-frame prediction mode.

[0232] Fig. 21C An example of a method 2120 for signaling or decoding one or more syntax elements in or from a bitstream according to another embodiment is shown, wherein the one or more syntax elements provide one or more intra-prediction modes for determining a block of a predicted image or video. For example, the method 2120 may be used in conjunction with Figures 19A-19B or Figures 20A-20B-20C described embodiment combination. It is assumed here that the block is encoded using a prediction combining a first intra prediction mode and a second prediction mode, as explained in any one of the embodiments described herein. At 2121, one or more syntax elements are encoded in a bitstream and decoded from the bitstream, respectively, which indicate a first intra prediction mode. At 2122, it is determined whether to signal the use of a luma intra fusion mode in the bitstream. In other words, it is determined whether to signal in the bitstream whether the first intra prediction mode will be combined with the second intra prediction. As described in some embodiments above, the use of a combination of the first and second intra prediction modes can be disabled or enabled based on the size of the block or based on the first intra prediction mode. In another variant, the use of a combination of the first and second intra prediction modes can be disabled or enabled based on the cost evaluated when determining the second intra prediction mode (e.g., using a TIMD search). The determination of whether to signal the use of the combination in the bitstream is performed in the same manner at the encoder and the decoder. If it is determined that the use of the combination of the first and second intra prediction modes is disabled, it is not necessary to signal whether the combination is used because the same determination is performed on both the encoder and the decoder.

[0233] If it is determined that a combination of the first and second intra prediction modes (use of luma intra fusion) is to be signaled, then the use of the combination of the first and second intra prediction modes is signaled to indicate whether the block is predicted by the combination of the first and second intra prediction modes or is effectively predicted by another prediction. Then at 2123, one or more syntax elements are encoded into the bitstream and decoded from the bitstream, respectively, and the one or more syntax elements indicate whether the first intra prediction mode is to be combined with the second intra prediction mode. In other words, whether the luma intra fusion mode is used for the block is signaled here. The same variants for signaling or deriving the above-mentioned second intra prediction are also possible.

[0234] Fig.21D An example of a method 2130 for signaling or decoding one or more syntax elements in or from a bitstream according to another embodiment that provide one or more intra-prediction modes for determining a block of a picture or video is shown. For example, the method 2130 may be used in conjunction with Figures 19A-19B or Figures 20A-20B-20C. It is assumed here that the block is encoded using a prediction that combines a first intra-frame prediction mode and a second prediction mode, as explained in any of the embodiments described herein. At 2131, one or more syntax elements are encoded in a bitstream and decoded from the bitstream, respectively, indicating a first intra-frame prediction mode. At 2132, one or more syntax elements are encoded in a bitstream and decoded from the bitstream, respectively, indicating a mode used to derive a second intra-frame prediction mode. For example, one or more syntax elements indicate whether DIMD or TIMD is used to derive the second intra-frame prediction mode. This embodiment can be combined with Figures 21A-21B -21C described embodiment combination.

[0235] Fig.21E An example of a method 2140 for signaling or decoding one or more syntax elements in or from a bitstream according to another embodiment is shown, wherein the one or more syntax elements provide one or more intra-prediction modes for determining a block of a predicted image or video. For example, the method 2140 may be used in conjunction with Figures 19A-19B or Figures 20A-20B -20C. It is assumed that the block is encoded using a prediction that combines the first intra-frame prediction mode and the second prediction mode, as explained in any of the embodiments described herein. At 2141, one or more syntax elements are encoded in the bitstream, respectively decoded from the bitstream, indicating the first intra-frame prediction mode. At 2142, one or more syntax elements are encoded in the bitstream, respectively decoded from the bitstream, and the one or more syntax elements provide for deriving weights for combining the first and second intra-frame prediction modes, as further described below. This embodiment can be combined with Figures 21A-21B - Combination of the embodiments described in 21C-21D.

[0236] In the above embodiments, the first intra prediction mode is signaled to the decoder using one or more syntax elements. However, in some embodiments, the first intra prediction mode is not explicitly signaled to the decoder. For example, one or more syntax elements may be used to signal the use of luma intra blending for a block, where the first intra prediction mode and the second intra prediction mode are derived at the decoder. For example, the decoder knows the combination of the first and second intra prediction modes.

[0237] In another embodiment, only the first intra prediction mode is signaled to the decoder and the use of the combination of the first and second intra prediction modes is always activated. The second intra prediction mode is determined based on the first intra prediction mode or is derived from a DIMD or TIMD process.

[0238] Further embodiments related to signaling are described below, which may be combined with the above embodiments.

[0239] In some embodiments, the SPS flag is used to indicate whether the use of luma intra fusion can be used on a slice. In embodiments where TIMD (respectively DIMD) is used to derive the second intra prediction mode for luma intra fusion, this flag is not transmitted and is inferred to be 0 when TIMD (respectively DIMD) is not enabled.

[0240] In some embodiments, the use of luma intra-frame blending is restricted to certain block sizes. Specifically, luma intra-frame blending may be disabled for blocks that are too small (e.g., if their width multiplied by their height is less than 32, or if their width or height is below a certain value, such as 8) to reduce latency issues for smaller blocks. Additionally, in some embodiments, luma intra-frame blending may be disabled for blocks that are considered too large for better performance (e.g., if their width multiplied by their height is greater than 1024, or if their width or height is above a certain value, such as 32).

[0241] In another embodiment, referred to as "per-block embodiment signaling", when allowed in a slice and on a block, if the intra-prediction mode selected to predict the block can be the first intra-prediction mode of luma intra fusion, i.e., one of the non-directional intra modes mentioned above, such as normal intra-plane, MIP, IBC, the luma intra fusion process is always signaled on the block.

[0242] For example, when the luma intra blending process is always signaled on a block, if the intra prediction mode selected to predict the block is MIP mode, then "per-block embodiment signaling" applies. Fig. 22 An example of updating the signaling of the intra prediction mode selected to predict the current luma CB in ECM-4.0 is shown. Fig. 22 In the example, the luma intra fusion flag (MIP) is encoded with (one or more) CABAC context models. However, Fig. 22 is an example. The luma intra-frame fusion flag (MIP) can be bypassed for encoding. Also note that in Fig. 22 In , the luma intra-frame fusion flag (MIP) is placed after the truncated binary code of the MIP matrix index. However, the luma intra-frame fusion flag (MIP) can be placed between the MIP transpose flag and the truncated binary code of the MIP matrix index. The luma intra-frame fusion flag (MIP) can also be placed between the MIP flag and the MIP transpose flag. For example, in Fig. 22 In , TIMD is systematically used as a decoder-based process to derive an index of a second intra prediction mode, such as a directional intra prediction mode, involved in luma intra fusion.

[0243] As another example, when the luma intra fusion process is always signaled on a block, if the intra mode selected to predict the block is MIP mode or planar mode, then "per-block embodiment signaling" is applied, Fig.23A and 23B An example of updating the signaling of the intra prediction mode selected to predict the current luma CB in ECM-4.0 is shown. Fig.23A and 23B In the example of , the luma intra fusion flag (plane) is encoded with (one or more) CABAC context models. However, the luma intra fusion flag (plane) can be bypassed for encoding. For example, in Fig.23A and 23B In , TIMD is systematically used as a decoder-based process to derive an index of a second intra prediction mode, such as a directional intra prediction mode, involved in luma intra fusion.

[0244] Fig.23A and 23B Figure 10. Signaling of the intra prediction mode selected to predict the current luma CB in ECM-4.0, when the luma intra fusion process is always signaled on a block, in the case of the "per block embodiment signaling" diagram. If the intra mode selected to predict the block is MIP mode or planar mode, then the luma intra fusion flag (planar) is not signaled and is inferred to be 0. Note also that if MRL is selected to predict the current block, then planar mode cannot be the intra prediction mode selected to predict the current block, so the luma intra fusion flag (planar) is inferred to be 0.

[0245] In some embodiments, the luma intra fusion process is always applied without additional signaling. In some embodiments, for specific cases, the process is always applied and not signaled. For example, in embodiments where the MIP matrix has a retrained subset dedicated to luma intra fusion, the luma intra fusion process is always applied to blocks using matrices from the retrained subset.

[0246] In some embodiments, both TIMD and DIMD may be used to derive directional intra prediction modes. In those embodiments, a flag may be used to indicate which derivation process should be used.

[0247] In some embodiments, there is no decoder-side derivation process and the directional mode for luma intra blending is signaled to the decoder.

[0248] In some embodiments, when a block to be encoded is eligible for luma intra-frame fusion, a cost evaluation, such as, for example, a TIMD process, is always applied, and luma intra-frame fusion is performed if and only if the SATD cost obtained by the evaluation is below a certain threshold. In some embodiments, the threshold is used to determine whether the luma intra-frame fusion process should be transmitted: for example, if the SATD cost is above the threshold, luma intra-frame fusion is never applied, otherwise it is signaled whether it is used; or if the SATD cost is below the threshold, it is always applied, otherwise it is signaled whether it is applied.

[0249] In the following, some embodiments are described for determining weights used when combining the first and second intra prediction modes in luma intra fusion mode.

[0250] Before applying luma intra fusion, we call predA the first predictor block resulting from predictions from a first intra prediction mode of the luma intra fusion mode (i.e., predictions resulting from conventional planar, MIP, IBC, or other modes eligible for luma intra fusion), and predB the second predictor block resulting from predictions from a second intra prediction mode, such as a directional mode selected for luma intra fusion (i.e., predictions from an intra mode selected, for example, by a TIMD or DIMD process). The final prediction block resulting from luma intra fusion is called predF, where

[0251] predF[x][y]=α[x][y]×predA[x][y]+β[x][y]×predB[x][y],

[0252] So that α+β=1.

[0253] In some embodiments, α and β have fixed values ​​throughout the block. For example, in some embodiments, α = β = 0.5 for all samples, or in some other embodiments, α = 0.75 and β = 0.25.

[0254] In some embodiments, the weights used depend on the first intra prediction mode used for luma intra fusion. For example, in some embodiments, if luma intra fusion is used on top of MIP, the weights are α=β=0.5, and otherwise α=0.75 and β=0.25.

[0255] In some embodiments, there is a set of weights, and after signaling the use of luma intra fusion on a block, the selected weights are transmitted to the decoder. For example, the set of weights for α may be (1 / 4, 1 / 2, 3 / 4} (where β = 1-α). In some embodiments, the weights are different depending on the first intra prediction mode used for luma intra fusion. For example, when luma intra fusion is used on top of MIP, the weights for α may be {1 / 4, 1 / 2, 3 / 4}, but when used on top of IBC may be {3 / 8, 1 / 2, 5 / 8}.

[0256] In some embodiments, a process similar to that used to determine whether luma intra-frame fusion should be transmitted may be applied to determine the weights to be used. For example, if the cost estimate of the additional mode (e.g., the SATD given by the TIMD derivation process) is below a certain threshold, the weights may be, for example, α=0.75 and β=0.25, and otherwise α=β=0.5. Similarly, in some embodiments, if the SATD cost is above a certain threshold, the weights may be, for example, α=0.25 and β=0.75, and otherwise α=β=0.5.

[0257] In some embodiments, weights for luma intra fusion may be derived for each block. For example, in some embodiments using TIMD for luma intra fusion, the SATD cost costA of the first intra prediction mode may be calculated on the TIMD template. Where costB is the cost of the best direction-based mode found by TIMD, the final weights may be derived as α=costA / (costA+costB) and β=1-α.

[0258] In some embodiments, the weights may vary within a block. For example, for each sample at position (x, y), where (0, 0) is the top left corner and (width, height) is the bottom right corner, the weight may be

[0259]

[0260] In some embodiments, the weight α may depend on the rank r in the MPM list of the second intra prediction mode (directional-based IPM). For example, in some embodiments, when the directional-based mode selected by TIMD is within the top N modes in the MPM list, α may be calculated as Where N<M and β=1-α. In some embodiments, N=8 and M=16.

[0261] Embodiments described herein provide a new image or video compression tool that combines predictions of two intra prediction modes. In some embodiments, the tool is used by combining non-directional intra tools such as planar mode, matrix-based intra prediction (MIP) or intra block copy (IBC) with directional intra prediction mode (IPM). In order to reduce signaling costs, tools such as template-based intra mode derivation (TIMD) or decoder-side intra mode derivation (DIMD) can be used at the decoder to derive directional-based modes.

[0262] Fig.24 A block diagram of a system according to another embodiment is illustrated within which aspects of the present embodiment may be implemented. Fig.24 An embodiment of an apparatus 2400 for encoding or decoding a block of an image or video is shown, wherein the block is predicted using a luma fusion mode as described according to any of the embodiments described herein. The apparatus includes a processor 2410 and may be interconnected to a memory 2420 via at least one port. Both the processor 2410 and the memory 2420 may also have one or more additional interconnections to external connections.

[0263] The processor 2420 is also configured to use any of the embodiments described herein, obtain a first intra-frame prediction mode, obtain a second intra-frame prediction mode, and encode or decode at least one block based on a combination of the first intra-frame prediction mode and the second intra-frame prediction mode. For example, the processor 2421 is configured using a computer program product that includes code instructions that implement any of the embodiments described herein.

[0264] exist Fig.25 In the embodiment illustrated in FIG. 1 , in a transmission environment between two remote devices A and B on a communication network NET, device A includes a processor associated with a memory RAM and a ROM, which is configured to implement the Figure 1 , 2 , 19A, 20A, 21A-E described method for encoding a block of an image or video, and device B includes a processor associated with a memory RAM and ROM, which is configured to implement as described with respect to Figure 1 , 3 , 19B, 20B, 20C, 21A-E described method for decoding a block of an image or video. Depending on the embodiment, the devices A and B are also configured to be used as described with respect to Figure 4-23B The described block is predicted using the luma fusion mode.

[0265] According to an example, the network is a broadcast network adapted to broadcast / transmit encoded images or videos from device A to decoding devices including device B.

[0266] Fig.26 An example of the syntax of a signal transmitted via a packet-based transport protocol is shown. Each transmitted packet P includes a header H and a payload PAYLOAD. In some embodiments, the payload PAYLOAD may include encoded image or video data according to any of the above embodiments. In a variant, the signal includes data representing any of the following items:

[0267] - the first intra prediction mode,

[0268] - the second intra prediction mode,

[0269] - An indicator indicating whether the second intra prediction mode is obtained from decoder-side intra mode derivation or from template based intra mode derivation is signaled in the bitstream.

[0270] - for at least one block in the bitstream, an indicator is signaled indicating whether the first intra prediction mode is to be combined with the second intra prediction mode,

[0271] - an indicator indicating the use of luma blending mode for the block,

[0272] - an indicator indicating a matrix to be used by said first intra prediction mode, said first intra prediction mode being a MIP mode used in luma merge mode,

[0273] - an indicator indicating a weight from among a set of weights used for the prediction obtained from the first intra prediction mode in luma merge mode.

[0274] Various implementations involve decoding. "Decoding" as used in this application may include, for example, all or part of the processing performed on a received coded sequence to produce a final output suitable for display. In various embodiments, such a process includes one or more processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various embodiments, such a process also or alternatively includes a process performed by a decoder of the various implementations described in this application, for example, decoding resampling filter coefficients, resampling decoded pictures.

[0275] As another example, in one embodiment, "decoding" refers only to entropy decoding, in another embodiment, "decoding" refers only to differential decoding, and in another embodiment, "decoding" refers to a combination of entropy decoding and differential decoding, and in another embodiment, "decoding" refers to the entire reconstructed picture process including entropy decoding. Based on the context of the specific description, whether the phrase "decoding process" is intended to refer specifically to a subset of operations or to a broader decoding process will be clear and is considered to be well understood by those skilled in the art.

[0276] Various implementations involve encoding. In a manner similar to the discussion above about "decoding", "encoding" as used in this application may include, for example, all or part of the processing performed on an input video sequence to produce an encoded bitstream. In various embodiments, such a process includes one or more processes typically performed by an encoder, such as partitioning, differential encoding, transforms, quantization, and entropy encoding. In various embodiments, such a process also or alternatively includes a process performed by an encoder of the various implementations described in this application, such as determining resampling filter coefficients, resampling decoded pictures.

[0277] As another example, in one embodiment, "encoding" refers only to entropy encoding, in another embodiment, "encoding" refers only to differential encoding, and in another embodiment, "encoding" refers to a combination of differential encoding and entropy encoding. Based on the context of the specific description, whether the phrase "encoding process" is intended to refer specifically to a subset of operations or to a broader encoding process will be clear and is considered to be well understood by those skilled in the art.

[0278] Note that the grammatical elements as used herein are descriptive terms. As such, they do not exclude the use of other grammatical element names.

[0279] This disclosure has described various pieces of information that can be transmitted or stored, such as syntax, for example. The information can be packaged or arranged in a variety of ways, including, for example, ways that are common in video standards, such as putting the information in an SPS, PPS, NAL unit, header (e.g., a NAL unit header or a slice header), or SEI message. Other ways are also available, including, for example, ways that are common to system-level or application-level standards, such as putting the information in one or more of the following:

[0280] a. SDP (Session Description Protocol), a format for describing multimedia communication sessions, used for the purpose of session announcement and session invitation, for example as described in RFC, and used in conjunction with RTP (Real-time Transport Protocol) transport.

[0281] b. DASH MPD (Media Presentation Description) descriptor, e.g. as used in DASH and transmitted over HTTP, a descriptor is associated with a representation or a set of representations to provide additional characteristics to the content representation.

[0282] c. RTP header extension, eg as used during RTP streaming.

[0283] d. ISO Base Media File Format, such as used in OMAF, and uses object-oriented building blocks defined by a unique type identifier and a length (also called "atoms" in some specifications).

[0284] e. HLS (HTTP Live Streaming) manifests transmitted over HTTP. A manifest may be associated with, for example, a version or set of versions of content to provide characteristics of the version or set of versions.

[0285] When a figure is represented as a flow chart, it should be understood that it also provides a block diagram of the corresponding apparatus. Similarly, when a figure is presented as a block diagram, it should be understood that it also provides a flow chart of the corresponding method / process.

[0286] Some embodiments relate to rate-distortion optimization. In particular, during the encoding process, a balance or trade-off between rate and distortion is usually considered, usually given a constraint on computational complexity. Rate-distortion optimization is usually formulated as minimizing a rate-distortion function, which is a weighted sum of rate and distortion. There are different methods to solve the rate-distortion optimization problem. For example, a method can be based on extensive testing of all coding options, including all considered modes or coding parameter values, and a complete evaluation of their coding costs and the associated distortion of the reconstructed signal after encoding and decoding. Faster methods can also be used to save coding complexity, in particular, based on approximate distortion calculations of predictions or prediction residual signals rather than reconstructed signals. A mixture of these two methods can also be used, such as by using approximate distortions only for some possible coding options and using full distortions for other coding options. Other methods only evaluate a subset of possible coding options. More generally, many methods use any one of a variety of techniques to perform optimization, but optimization is not necessarily a complete evaluation of both coding costs and associated distortions.

[0287] The implementations and aspects described herein can be implemented in, for example, methods or processes, devices, software programs, data streams, or signals. Even if only discussed in the context of a single implementation form (e.g., discussed only as a method), the implementation of the features discussed can also be implemented in other forms (e.g., devices or programs). Devices can be implemented in, for example, appropriate hardware, software, and firmware. The method can be implemented in, for example, a processor, which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. The processor also includes a communication device, such as, for example, a computer, a cellular phone, a portable / personal digital assistant ("PDA"), and other devices that facilitate information communication between end users.

[0288] Reference to "one embodiment" or "an embodiment" or "an implementation" or "implementation" and other variations thereof mean that a particular feature, structure, characteristic, etc. described in connection with the embodiment is included in at least one embodiment. Thus, the appearances of the phrase "in one embodiment" or "in an embodiment" or "in an implementation" or "in an implementation" and any other variations appearing in various places throughout the application are not necessarily all referring to the same embodiment.

[0289] Additionally, the present application may involve “determining” various pieces of information. Determining information may include, for example, one or more of estimating information, calculating information, predicting information, or retrieving information from a memory.

[0290] Furthermore, the present application may refer to "accessing" various pieces of information. Accessing information may include, for example, one or more of receiving information, retrieving information (e.g., from a memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0291] Additionally, the present application may involve "receiving" various pieces of information. Like "accessing," receiving is intended to be a broad term. Receiving information may include, for example, one or more of accessing information or retrieving information (e.g., from a memory). Furthermore, "receiving" is generally referred to in one way or another during operations such as, for example, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0292] It will be appreciated that, for example, in the case of "A / B," "A and / or B," and "at least one of A and B," any of the following uses of " / ," "and / or," and "at least one of..." are intended to include selecting only the first listed option (A), or only the second listed option (B), or both options (A and B). As a further example, in the case of "A, B, and / or C" and "at least one of A, B, and C," such wording is intended to include selecting only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A and B and C). This can be extended to as many items as are listed, as will be apparent to one of ordinary skill in this and related arts.

[0293] In addition, as used herein, the word "signal" refers to, among other things, indicating something to a corresponding decoder. Thus, in one embodiment, the same parameters are used on both the encoder side and the decoder side. Therefore, for example, the encoder can transmit (explicitly signal) specific parameters to the decoder so that the decoder can use the same specific parameters. On the contrary, if the decoder already has specific parameters and other parameters, signaling can be used without transmission (implicitly signaling) to simply allow the decoder to know and select specific parameters. By avoiding the transmission of any actual function, bit saving is achieved in various embodiments. It will be appreciated that signaling can be implemented in a variety of ways. For example, in various embodiments, one or more grammatical elements, flags, etc. are used to signal information to the corresponding decoder. Although the verb form of the word "signal" is previously related to, the word "signal" can also be used as a noun in this article.

[0294] As will be clear to one of ordinary skill in the art, implementations may generate a variety of signals formatted to carry information that may be stored or transmitted, for example. The information may include, for example, instructions for executing the method, or data generated by one of the described implementations. For example, a signal may be formatted to carry a bitstream of the described embodiments. Such a signal may be formatted as, for example, an electromagnetic wave (e.g., using a radio frequency portion of a spectrum) or a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, the signal may be transmitted over a variety of different wired or wireless links. The signal may be stored on a processor-readable medium.

[0295] A number of embodiments have been described above. The features of these embodiments may be provided alone or in any combination across the various claim categories and types.

Claims

1. A method comprising, for at least one block of an image or video: decoding an indicator indicating a first intra prediction mode, In response to determining that the first intra prediction mode is to be combined with the second intra prediction mode, obtaining the second intra prediction mode, The at least one block is decoded based on a combination of the first intra prediction mode and the second intra prediction mode.

2. An apparatus comprising one or more processors, wherein the one or more processors are operable to, for at least one block of an image or video: decoding an indicator indicating a first intra prediction mode, In response to determining that the first intra prediction mode is to be combined with the second intra prediction mode, obtaining the second intra prediction mode, The at least one block is decoded based on a combination of the first intra prediction mode and the second intra prediction mode.

3. A method comprising, for at least one block of an image or video: - encoding an indicator indicating a first intra prediction mode, - in response to determining that the first intra prediction mode is to be combined with the second intra prediction mode, obtaining the second intra prediction mode, - encoding the at least one block based on a combination of the first intra prediction mode and the second intra prediction mode.

4. An apparatus comprising one or more processors, wherein: The one or more processors are operable to, for at least one block of an image or video: encoding an indicator indicating a first intra prediction mode, In response to determining that the first intra prediction mode is to be combined with the second intra prediction mode, obtaining the second intra prediction mode, The at least one block is encoded based on a combination of the first intra prediction mode and the second intra prediction mode.

5. The method according to claim 1 or 3 or the device according to claim 2 or 4, wherein: The first intra prediction mode is obtained from a first intra prediction mode set, and the second intra prediction mode is obtained from a second intra prediction mode set, the first and second intra prediction mode sets being different.

6. The method according to any one of claims 1, 3 or 5 or the device according to any one of claims 2, 4 or 5, wherein: One of the first intra prediction mode or the second intra prediction mode is obtained from a first intra prediction mode set including a non-directional intra prediction mode, and the other of the first intra prediction mode and the second intra prediction mode is obtained from a second intra prediction mode set including a directional intra prediction mode.

7. The method according to any one of claims 1, 3 or 5-6 or the device according to any one of claims 2, 4 or 5-6, wherein: The first intra prediction mode is one of a planar prediction mode, a DC prediction mode, an intra block copy prediction mode, and a matrix-based intra prediction mode.

8. The method according to any one of claims 1, 3 or 5-7 or the device according to any one of claims 2, 4 or 5-7, wherein: The second intra prediction mode is one of the directional intra prediction modes.

9. The method according to any one of claims 1, 3 or 5-8 or the device according to any one of claims 2, 4 or 5-8, wherein: At least one of the first intra prediction mode or the second intra prediction mode is signaled in the bitstream.

10. The method according to any one of claims 1, 3 or 5-9 or the device according to any one of claims 2, 4 or 5-9, wherein: Obtaining the second intra prediction mode includes deriving the second intra prediction mode from reconstructed samples neighboring the at least one block.

11. The method according to any one of claims 1, 3 or 5-10 or the device according to any one of claims 2, 4 or 5-10, wherein: The second intra prediction mode is obtained from at least one of a decoder-side intra mode derivation or a template-based intra mode derivation.

12. The method or apparatus according to claim 11, wherein: An indicator indicating whether the second intra prediction mode is obtained from decoder-side intra mode derivation or from template based intra mode derivation is signaled in the bitstream.

13. The method according to any one of claims 3 or 5-12 or the device according to any one of claims 4-12, wherein: Determining that the first intra prediction mode is to be combined with the second intra prediction mode is based on an indicator signaled for the at least one block in the bitstream.

14. A method according to any one of claims 3 or 5-13 or an apparatus according to any one of claims 4-13, wherein: For the at least one block, determining that the first intra prediction mode is to be combined with the second intra prediction mode is based on a cost obtained in determining the second intra prediction mode.

15. The method or apparatus according to claim 13, wherein: The signaling of the indicator indicating whether the first intra prediction mode is to be combined with the second intra prediction mode is based on the cost obtained when determining the second intra prediction mode.

16. A method or apparatus according to any one of claims 3 or 5-15 or an apparatus according to any one of claims 4-15, wherein: For the at least one block, determining that the first intra prediction mode is to be combined with the second intra prediction mode is based on a size of the at least one block.

17. The method according to any one of claims 3 or 5-16 or the device according to any one of claims 4-16, wherein: The first intra prediction mode is a matrix-based intra prediction mode using one matrix from a set of matrices, and wherein at least one matrix from the set is trained to predict non-directional information within a block to be predicted.

18. The method or apparatus according to claim 17, wherein: For the at least one block, determining that the first intra prediction mode is to be combined with the second intra prediction mode is based on an indicator signaled in the bitstream, the indicator indicating a matrix to be used by the first intra prediction mode.

19. The method according to any one of claims 1, 3 or 5-18 or the apparatus according to any one of claims 2, 4 or 5-18, wherein: The combination is a weighted average of the first intra prediction mode and the second intra prediction mode.

20. The method or apparatus of claim 19, wherein: The weight used in the combination depends on the first intra-frame prediction mode, an indicator signaled in the bitstream indicating a weight from a weight set used for the first intra-frame prediction mode, a cost obtained when determining the second intra-frame prediction mode, and at least one of the ranking of the second prediction mode in the intra-frame prediction mode list.

21. The method or apparatus of claim 19, wherein: The weight used in the combination is derived from the cost obtained when the second intra prediction mode is determined.

22. The method or apparatus according to any one of claims 19 to 21, wherein: The weights used in the combining vary with the sample positions in the at least one block.

23. A computer program product comprising instructions for causing one or more processors to carry out the method of any one of claims 1, 3 or 5-22.

24. A non-transitory computer-readable medium storing executable program instructions, the executable program instructions causing a computer executing the instructions to perform the method according to any one of claims 1, 3, or 5-22.

25. A bitstream comprising data representing at least one block of an image or video encoded using the method of any of claims 1, 3 or 5-22.

26. A non-transitory computer readable medium storing the bitstream of claim 25.

27. An apparatus comprising: - a device according to any one of claims 2 or 4-22; as well as - At least one of: (i) an antenna configured to receive a signal, the signal comprising data representing an image or video, (ii) a band limiter configured to limit the signal comprising data representing an image or video to a frequency band, or (iii) a display configured to display an image or video.

28. The apparatus of claim 27, wherein: The device includes at least one of a television, a cellular phone, a tablet computer, and a set-top box.

Images