Improving intra prediction modes based on available reference samples

Abstract

In one embodiment, the wide-angle process can be modified to prohibit intra-prediction modes (IPMs) that use padded reference samples, especially if other IPMs use available reference samples. More generally, certain intra-prediction modes can be removed or added, or MPMs can be reordered, depending on whether reference samples are available. The signaling process can be modified to handle added or removed modes, especially if the number of IPMs can vary. Intra-mode storage and propagation can also be modified to handle changes made to IPMs.

Description

[Technical Field]

[0001] The present embodiments generally relate to methods and apparatus for intra prediction in video encoding and decoding. [Background technology]

[0002] To achieve high compression efficiency, image and video coding schemes typically use prediction and transformation to exploit spatial and temporal redundancies in video content. Generally, intra- or inter-prediction is used to exploit intra- or inter-frame image correlation, and the difference between the original block and the predicted block, often called the prediction error or prediction residual, is transformed, quantized, and entropy coded. To reconstruct the video, the compressed data is decoded by an inverse process corresponding to the entropy coding, quantization, transformation, and prediction. Summary of the Invention

[0003] According to one embodiment, a method of video decoding is presented, comprising: identifying the availability of one or more reference samples for a block to be decoded in an image; obtaining a set of intra prediction modes for the block depending on the availability of the one or more reference samples; obtaining an intra prediction mode from the set of intra prediction modes; and performing intra prediction of the block to be decoded based on the intra prediction mode of the block to form a predictive block of the block.

[0004] According to another embodiment, a method for video encoding is presented, comprising: identifying the availability of one or more reference samples for a block to be encoded in an image; obtaining a set of intra prediction modes for the block depending on the availability of the one or more reference samples; selecting an intra prediction mode from the set of intra prediction modes; and performing intra prediction of the block to be encoded based on the intra prediction mode of the block to form a predictive block of the block.

[0005] According to another embodiment, an apparatus for video decoding is presented, including one or more processors configured to identify availability of one or more reference samples for a block to be decoded in an image, obtain a set of intra prediction modes for the block depending on the availability of the one or more reference samples, obtain an intra prediction mode from the set of intra prediction modes, and perform intra prediction for the block to be decoded to form a predictive block for the block based on the intra prediction mode of the block.

[0006] According to another embodiment, a video encoding apparatus is presented, including one or more processors configured to identify availability of one or more reference samples for a block to be encoded in an image, obtain a set of intra prediction modes for the block depending on the availability of the one or more reference samples, select an intra prediction mode from the set of intra prediction modes, and perform intra prediction of the block to be encoded to form a predictive block for the block based on the intra prediction mode of the block.

[0007] One or more embodiments also provide a computer program comprising instructions that, when executed by one or more processors, cause the one or more processors to perform an encoding or decoding method according to any of the embodiments described herein. One or more of the present embodiments also provide a computer-readable storage medium having stored thereon instructions for video encoding or decoding according to the methods described herein.

[0008] One or more embodiments also provide a computer-readable storage medium having stored thereon video data generated according to the above methods. One or more embodiments also provide methods and apparatus for transmitting or receiving video data generated according to the methods described herein. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 shows a block diagram of a system in which various aspects of the present embodiments may be implemented. [Figure 2] FIG. 1 illustrates a block diagram of an embodiment of a video encoder. [Figure 3] FIG. 2 illustrates a block diagram of one embodiment of a video decoder. [Figure 4] FIG. 10 illustrates reference samples for intra prediction. [Figure 5A] FIG. 1 illustrates reference sample substitution for intra prediction. [Figure 5B] FIG. 1 illustrates reference sample substitution for intra prediction. [Figure 6] FIG. 1 illustrates the process of reference sample substitution for intra prediction. [Figure 7A] FIG. 1 is a diagram illustrating intra-prediction directions in HEVC. [Figure 7B] FIG. 1 is a diagram illustrating intra-prediction directions in VVC. [Figure 7C] FIG. 1 illustrates horizontal and vertical, positive and negative intra prediction modes. [Figure 8A] FIG. 10 is a diagram illustrating wide-angle intra prediction. [Figure 8B] FIG. 10 is a diagram illustrating wide-angle intra prediction. [Figure 8C] FIG. 10 is a diagram illustrating wide-angle intra prediction. [Figure 9] FIG. 1 is a diagram showing all available intra-prediction directions in VVC. [Figure 10] FIG. 10 is a diagram illustrating a planar mode. [Figure 11] FIG. 10 is a diagram showing reference samples used for each intra prediction mode (IPM) on a PU with aspect ratio W / H=4. [Figure 12A] FIG. 10 is a diagram showing CBs within an intra-slice. [Figure 12B] FIG. 10 shows an unavailable reference sample. [Figure 12C] FIG. 10 illustrates a removed intra-prediction mode. [Figure 13A] FIG. 10 illustrates another CB within an intra-slice. [Figure 13B] FIG. 10 shows an unavailable reference sample. [Figure 13C] FIG. 10 illustrates a removed intra-prediction mode. [Figure 14] FIG. 10 illustrates a workflow for signaling the index of the intra-prediction mode selected to predict a current block of WxH at the encoder side, according to an embodiment. [Figure 15] FIG. 1 illustrates a workflow for decoding an index of an intra-prediction mode selected to predict a current block of WxH at the decoder side, according to an embodiment. [Figure 16] FIG. 10 illustrates the identification of unavailable decoded reference samples surrounding a current block using a search of decoded blocks surrounding the current block. [Figure 17] FIG. 10 illustrates a workflow for signaling the index of the intra-prediction mode selected to predict a current block of WxH at the encoder side according to another embodiment. [Figure 18]FIG. 10 illustrates a workflow for decoding an index of an intra-prediction mode selected to predict a current block of WxH at the decoder side according to another embodiment. [Figure 19] FIG. 10 illustrates the generation of a general list of 22 MPMs for the current luminance CB of an ECM. [Figure 20] FIG. 10 illustrates a modified generation of a general list of 22 MPMs for the current brightness CB of an ECM, according to one embodiment. [Figure 21] FIG. 10 illustrates a modified generation of the general list of 22 MPMs for the current brightness CB of an ECM according to another embodiment. [Figure 22] FIG. 10 illustrates a modified generation of the general list of 22 MPMs for the current brightness CB of an ECM according to another embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] FIG. 1 illustrates a block diagram of an example system in which various aspects and embodiments may be implemented. System 100 may be embodied as a device including the various components described below and configured to perform one or more aspects described herein. Examples of such devices include various electronic devices, such as, but not limited to, personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. The elements of system 100, alone or in combination, may be embodied in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one embodiment, the processing elements and encoder / decoder elements of system 100 are distributed across multiple ICs and / or discrete components. In various embodiments, system 100 is communicatively coupled to other systems or other electronic devices, for example, via a communication bus or through dedicated input and / or output ports. In various embodiments, system 100 is configured to implement one or more aspects described herein.

[0011] System 100 includes at least one processor 110 configured to execute loaded instructions to implement various aspects described herein, for example. Processor 110 may include embedded memory, input / output interfaces, and various other circuitry known in the art. System 100 includes at least one memory 120 (e.g., a volatile memory device and / or a non-volatile memory device). System 100 includes storage device 140, which may include non-volatile memory and / or volatile memory including, but not limited to, EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash, magnetic disk drives, and / or optical disk drives. Storage device 140 may include, by way of non-limiting example, internal storage, attached storage, and / or network-accessible storage.

[0012] System 100 includes, for example, an encoder / decoder module 130 configured to process data to provide encoded or decoded video, which may include its own processor and memory. Encoder / decoder module 130 represents a module that may be included in a device to perform encoding and / or decoding functions. As is known, a device may include one or both of an encoding module and a decoding module. Furthermore, encoder / decoder module 130 may be implemented as a separate element of system 100 or may be incorporated within processor 110 as a combination of hardware and software, as is known to those skilled in the art.

[0013] Program code to be loaded into processor 110 or encoder / decoder 130 to implement various aspects described herein may be stored in storage device 140 and loaded onto memory 120 for execution by processor 110. According to various embodiments, one or more of processor 110, memory 120, storage device 140, and encoder / decoder module 130 may store one or more of various items during performance of the processes described herein. Such stored items may include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results of processing of equations, formulas, arithmetic, and operational logic.

[0014] In some embodiments, memory internal to the processor 110 and / or the encoder / decoder module 130 is used to store instructions and provide working memory for the processing required for encoding or decoding. However, in other embodiments, memory external to the processing device (e.g., the processing device may be the processor 110 or the encoder / decoder module 130) is used for one or more of these functions. The external memory may be the memory 120 and / or the storage device 140, such as dynamic volatile memory and / or non-volatile flash memory. In some embodiments, the external non-volatile flash memory is used to store the television's operating system. In at least one embodiment, high-speed external dynamic volatile memory, such as RAM, is used as working memory for video encoding and decoding operations, such as for MPEG-2, HEVC, or VVC.

[0015] Input to the elements of system 100 may be provided through various input devices shown in block 105. Such input devices include, but are not limited to, (i) an RF section for receiving RF signals transmitted over the air by, for example, a broadcast station, (ii) a composite input, (iii) a USB input, and / or (iv) an HDMI® input.

[0016] In various embodiments, the input devices of block 105 have associated respective input processing elements as known in the art. For example, the RF section may be associated with elements suitable for (i) selecting a desired frequency (also referred to as selecting a signal or limiting a signal to a band of frequencies), (ii) downconverting the selected signal, (iii) bandlimiting again to a narrower band of frequencies to select a signal frequency band, which in some embodiments may be referred to as a channel (for example), (iv) demodulating the downconverted and bandlimited signal, (v) performing error correction, and (vi) demultiplexing to select a desired stream of data packets. The RF section of various embodiments includes one or more elements for performing these functions, such as a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a downconverter, a demodulator, an error corrector, and a demultiplexer. The RF section may include a tuner for performing various of these functions, including, for example, downconverting a received signal to a lower frequency (e.g., an intermediate frequency or near-baseband frequency) or to baseband. In one set-top box embodiment, the RF section and its associated input processing elements receive RF signals transmitted over a wired (e.g., cable) medium and perform frequency selection by filtering, downconverting, and filtering again to a desired frequency band. Various embodiments rearrange the order of the above (and other) elements, remove some of these elements, and / or add other elements that perform similar or different functions. Adding elements may include inserting elements between existing elements, such as inserting amplifiers and analog-to-digital converters. In various embodiments, the RF section includes an antenna.

[0017] Additionally, the USB and / or HDMI terminals may include respective interface processors for connecting system 100 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of the input processing, e.g., Reed-Solomon error correction, may be implemented, for example, within a separate input processing IC or within processor 110, as desired. Similarly, aspects of the USB or HDMI interface processing may be implemented, for example, within a separate interface IC or within processor 110, as desired. The demodulated, error corrected, and demultiplexed stream is provided to various processing elements, including, for example, processor 110 and encoder / decoder 130 operating in combination with memory and storage elements, to process the data stream as desired for presentation on an output device.

[0018] The various elements of system 100 may be provided within an integrated housing in which the various elements are interconnected and capable of transmitting data therebetween using suitable connection arrangements 115, e.g., internal buses known in the art, including I2C buses, wiring, and printed circuit boards.

[0019] System 100 includes a communication interface 150 that enables communication with other devices over a communication channel 190. Communication interface 150 may include, but is not limited to, a transceiver configured to transmit and receive data over communication channel 190. Communication interface 150 may include, but is not limited to, a modem or a network card, and communication channel 190 may be implemented in a wired and / or wireless medium, for example.

[0020] In various embodiments, data is streamed to system 100 using a Wi-Fi network, such as IEEE 802.11. The Wi-Fi signal in these embodiments is received via communication channel 190 and communication interface 150, which are adapted for Wi-Fi communication. Communication channel 190 in these embodiments is typically connected to an access point or router that provides access to external networks, including the Internet, enabling streaming applications and other over-the-top communications. Other embodiments provide streaming data to system 100 using a set-top box that sends data through an HDMI connection in input block 105. Still other embodiments provide streaming data to system 100 using an RF connection in input block 105.

[0021] System 100 can provide output signals to various output devices, including display 165, speakers 175, and other peripherals 185. Other peripherals 185, in various example embodiments, include one or more of a standalone DVR, a disc player, a stereo system, a lighting system, and other devices that provide functionality based on the output of system 100. In various embodiments, control signals are communicated between system 100 and display 165, speakers 175, or other peripherals 185 using AV-like signaling. Links, CEC, or other communication protocols enable control between devices with or without user intervention. Output devices may be communicatively coupled to system 100 via dedicated connections through respective interfaces 160, 170, and 180. Alternatively, output devices may be connected to system 100 using communication channel 190 via communication interface 150. Display 165 and speakers 175 may be integrated in a single unit with other components of system 100 within an electronic device, such as a television. In various embodiments, the display interface 160 includes a display driver, such as a timing controller (T Con) chip.

[0022] Alternatively, display 165 and speakers 175 may be separate from one or more of the other components, for example, if the RF portion of input 105 is part of a separate set-top box. In various embodiments where display 165 and speakers 175 are external components, the output signals may be provided via dedicated output connections including, for example, an HDMI port, a USB port, or a COMP output.

[0023] 2 shows an example video encoder 200, such as a Versatile Video Coding (VVC) encoder. FIG. 2 may also show an encoder that improves on the VVC standard or employs techniques similar to VVC.

[0024] In this application, the terms "reconstructed" and "decoded" can be used interchangeably, the terms "encoded" and "coded" can be used interchangeably, and the terms "image", "picture" and "frame" can be used interchangeably. Typically, but not necessarily, the term "reconstructed" is used on the encoder side, and the term "decoded" is used on the decoder side.

[0025] Before being encoded, the video sequence may go through pre-encoding processing (201), which may, for example, apply a color transformation to the input color picture (e.g., converting from RGB 4:4:4 to YCbCr 4:2:0) or perform a remapping of the input picture components to make the signal distribution more resilient to compression (e.g., using histogram equalization of one of the color components). Metadata may be associated with the pre-processing and attached to the bitstream.

[0026] In the encoder 200, a picture is coded by the encoder elements as described below. The picture to be coded is divided (202) and processed, for example, in units of coding units (CUs). Each unit is coded, for example, using either intra mode or inter mode. If the unit is coded in intra mode, intra prediction is performed (260). In inter mode, motion estimation (275) and motion compensation (270) are performed. The encoder determines (205) whether to use intra mode or inter mode to code the unit and indicates the intra mode / inter mode decision, for example, by a prediction mode flag. A prediction residual is calculated, for example, by subtracting (210) the prediction block from the original image block.

[0027] The prediction residual is transformed (225) and quantized (230). The quantized transform coefficients, motion vectors, and other syntax elements, such as image segmentation information, are entropy coded (245) to output a bitstream. As a non-limiting example, CABAC (context-based adaptive binary arithmetic coding) can be used to code the syntax elements into the bitstream.

[0028] The encoder can skip the transform and apply quantization directly to the untransformed residual signal. The encoder can bypass both the transform and the quantization, i.e., the residual is coded directly without applying a transform or quantization process.

[0029] The encoder decodes the coded block to provide a reference for further prediction. The quantized transform coefficients are inverse quantized (240) and inverse transformed (250) to decode the prediction residual. The decoded prediction residual is combined with the prediction block (255) to reconstruct an image block. An in-loop filter (265) is applied to the reconstructed image to perform, for example, deblocking / Sample Adaptive Offset (SAO) filtering to reduce coding artifacts. The filtered image is stored in a reference picture buffer (280).

[0030] 3 shows a block diagram of an exemplary video decoder 300. In the decoder 300, the bitstream is decoded by decoder elements as described below. The video decoder 300 generally performs a decoding pass that is the inverse of the encoding pass as described in FIG. 2. The encoder 100 also generally performs video decoding as part of encoding the video data.

[0031] In particular, the decoder's input includes a video bitstream, which may be generated by the video encoder 200. The bitstream is first entropy decoded (330) to obtain transform coefficients, prediction modes, motion vectors, and other coding information. Image partition information indicates how the picture is partitioned. The decoder can then partition the picture according to the decoded image partition information (335). The transform coefficients are inverse quantized (340) and inverse transformed (350) to decode the prediction residual. The decoded prediction residual is combined with a prediction block (355) to reconstruct an image block. The prediction block can be obtained from intra prediction (360) or motion-compensated prediction (i.e., inter prediction) (370). An in-loop filter (365) is applied to the reconstructed image. The filtered image is stored in a reference picture buffer (380). Note that for a given picture, the contents of the reference picture buffer 380 on the decoder 300 side are identical to the contents of the reference picture buffer 280 on the encoder 200 side for the same picture.

[0032] The decoded image may further pass through post-decoding processing (385), such as an inverse color conversion (e.g., converting from YCbCr 4:2:0 to RGB 4:4:4) or an inverse remapping that performs the inverse of the remapping process performed in the pre-encoding process (201). The post-decoding process may use metadata derived in the pre-encoding process and signaled in the bitstream.

[0033] Intra prediction and reference samples substitution

[0034] The intra prediction process in HEVC and VVC consists of three steps: · Reference sample generation; Intra-sample prediction; and Post-processing of predicted samples

[0035] The reference sample generation process is shown in Figure 4. The pixel value at coordinate (x, y) is denoted by P(x, y). The reference sample ref[] is also known as L-shaped. For a prediction unit (PU) of size NxN, the top row of (2N+2*refIdx) decoded samples is formed from the previously reconstructed top pixel and the top-right pixel. Similarly, the left column of (2N+2*refIdx) samples is formed from the reconstructed left pixel and the bottom-left pixel. In VVC, the reference line and column of samples can be at a distance (d=refIdx) of more than one sample from the current block, as shown in Figure 4. An index "mrlIdx" is signaled to indicate which value of "d" should be used.

[0036] The pixel in the upper left corner is also used to fill the gap between the reference in the row above and the reference in the column to the left. In Figures 5A and 5B, dashed areas correspond to unavailable image regions (e.g., out-of-bounds or not-yet-reconstructed regions), and missing reference samples are indicated by dotted lines. As shown in Figure 6, if some of the samples on the top or left side are unavailable (610), for example, because the corresponding CU is not in the same slice, or the current CU is at a frame boundary, as shown in Figure 5A, or because the current CU is at the bottom right after quadtree split, as shown in Figure 5B, a method called reference sample replacement is performed (630), in which missing samples are copied from available samples in a clockwise and counterclockwise direction. These copied samples are also called "padded reference samples," and if reconstructed samples are used as reference samples, they are also called "unpadded reference samples."

[0037] If a reconstructed top / left reference sample is available, the reconstructed reference sample is copied to a reference sample buffer (620). After the reference sample substitution process, intra-sample prediction is performed (640). Depending on the current CU size and prediction mode, the reference sample is filtered using a specified filter.

[0038] Intra-sample prediction involves predicting pixels of a target CU based on reference samples. There are various prediction modes. Planar and DC prediction modes are used to predict smooth, gradually changing regions, while angular (defined as an angle from 45 degrees to -135 degrees clockwise) prediction mode is used to capture different directional structures. For square blocks, HEVC supports 33 directional prediction modes, indexed from 2 to 34. These prediction modes correspond to different prediction directions, as shown in Figure 7A. The numbers in the figure indicate the prediction mode index associated with the corresponding direction. Modes 2 to 17 indicate horizontal prediction (H-26 to H+32), and modes 18 to 34 indicate vertical prediction (V-32 to V+32).

[0039] In VVC, there are 65 angular prediction modes corresponding to the 33 angular directions defined in HEVC, plus 32 directions, each corresponding to a direction halfway between adjacent pairs, as shown in Figure 7B. For square blocks, modes below 34 indicate horizontal prediction, and modes above 34 indicate vertical prediction.

[0040] As mentioned above, the angle direction can be classified as either a vertical direction or a horizontal direction. As shown in FIG. 7C, the horizontal prediction mode uses either only the left reference sample or several left reference samples and several top reference samples. Similarly, the vertical prediction mode uses either only the top reference sample or several top reference samples and several left reference samples. The horizontal positive direction uses only the left reference sample for prediction. Similarly, the vertical positive direction uses only the top reference sample for prediction. The negative horizontal and vertical directions use both the left and top reference samples for prediction.

[0041] In VVC, for non-square blocks, disallowed normal directional intra prediction modes are replaced with wide-angle intra prediction modes, as shown in Figures 8A, 8B, and 8C. Table 1 shows the replaced intra modes and added wide-angle modes for various aspect ratios. Note that although 32 block ratios are included in Table 1, they cannot actually be used because partitioning does not allow W / H=32 or H / W=32. In Figure 9, the dashed line indicates WAIP (Wide Angle Intra Prediction Mode). Note that in ECM, the indices -1 to -14 presented in Figure 9 are remapped to move from 1 to -12 so that the angular mode indices are continuous. Mode -15 (remapped to -13) and mode 81 are not shown in Figure 9 because there are no block sizes that can use them, but these modes are processed by the reference software.

[0042] [Table 1]

[0043] For a given angular prediction mode, the predictor samples on the reference array are copied along the corresponding direction in the target PU. Some predictor samples may have integer positions, in which case they coincide with the corresponding reference samples, while other predictor positions have fractional parts, indicating that their positions fall between two reference samples. In the latter case, the predictor samples are interpolated using the nearest reference samples (post-processing of predicted samples). In HEVC, a linear interpolation of the two nearest reference samples is performed to calculate the predictor sample value. In VVC, a 4-tap filter fT[], selected depending on the intra-mode direction, is used to interpolate the predictor samples.

[0044] In addition to the directional modes, the DC mode embeds predictions with the average of L-shaped samples (except for rectangular CUs, which use the average of the reference samples on the long side), and the Planar mode spatially interpolates the reference samples as shown in Figure 10.

[0045] Intra-prediction mode coding

[0046] Because multiple intra-prediction modes are available, a decoder requires mode information to form a prediction for an intra-coded CU. The encoder encodes the mode information using one or more Most Probable Mode (MPM) sets. For example, Enhanced Compression Model 5.0 (ECM-5.0) uses a first MPM list (having 6 MPMs) and a second MPM list (having 16 MPMs). The first MPM list is constructed by sequentially adding candidate intra-prediction mode indexes based on the intra-prediction mode indexes used next to the current luma coding block, with the first MPM index reserved for the planar mode. The neighboring indexes added are the left neighbor, top neighbor, bottom-left neighbor, top-right neighbor, and top-left neighbor.

[0047] The second MPM list is constructed by first adding the indices of the first and second Decoder-side Intra Mode Derivation (DIMD) modes of the current luma coding block, and then adding the increment and decrement indices of the MPM of the first angle (mpm[1]+1, mpm[1]-1, mpm[1]+2, mpm[1]-2, mpm[1]+3, mpm[1]-3, mpm[1]+4, mpm[1]-4, mpm[2]+1, mpm[2]-1, mpm[2]+2, mpm[2]-2, mpm[2]+3, mpm[2]-3, mpm[2]+4, mpm[2]-4, etc.) such that the redundant mode index is not present in the MPM list, i.e., not present in either the first or second MPM list. If the selected intra-prediction mode does not belong to the first or second MPM list, the remaining intra-prediction modes (IPMs) are coded using 45-symbol truncated binary coding.

[0048] In one embodiment, it is proposed to modify the intra prediction process so that the wide-angle intra prediction mode is selected not only based on the block aspect ratio but also based on the availability of reference samples. In this way, IPMs that typically use primarily padded reference samples are disallowed, and IPMs that normally would not be allowed but actually use non-padded reference samples are allowed. The signaling is modified to take advantage of the change. This can be done by adding context to the CABAC coding bins that use this change, or by modifying the code to handle added or removed modes if the total number of modes is different from the original 67 indexes.

[0049] Intra mode propagation has also been changed to propagate more accurate mode values ​​and handle cases where neither the adjacent intra mode nor its 180-degree counterpart is available in the current block.

[0050] Selecting an available intra mode based on the availability of adjacent reference samples

[0051] Figure 11 shows which samples are tested for availability to know which IPM to use with padded reference samples, according to one embodiment. The solid lines (1110, 1115) show the angular IPMs used by ECM-6.0 for PUs with a W / H ratio of 4 (modes 12-76). The dashed lines (1120, 1125) show the modes used by ECM-6.0 for PUs with a W / H ratio of 2 (modes 8-72). The dotted lines (1130, 1135) show the modes used by ECM-6.0 for PUs with a W / H ratio of 1 (modes 2-66).

[0052] As shown in Figure 11, if a set of modes desires to use only available reference samples for intra prediction, it is possible to determine which samples need to be available for this set of modes for a particular block width / height ratio. For example, as shown in Figure 11, if reference sample A is unavailable (and the sample to the left of A is also unavailable), at least one mode from modes 73 to 76 (i.e., modes used for W / H = 4 but not for W / H < 4) uses at least one padded sample. If reference sample B is unavailable (and all samples between A and B are unavailable), all modes 73 to 76 use at least one padded reference sample.

[0053] If it is determined that reference samples for an intra prediction mode are unavailable, it may be determined to remove this intra prediction mode because the prediction quality will be reduced. Furthermore, if all reference samples for these intra prediction modes are available, it may be possible to check whether several other intra prediction modes can be added. Note that some of the intra prediction modes we propose to add are modes that are removed by WAIP. However, because these intra prediction modes use reconstructed reference samples (not padded reference samples), the prediction quality may still be good. For example, as illustrated in FIG. 11, if X is available (all reference samples above X are available), modes 8 to 11 (i.e., modes used for W / H=2 but not for W / H>2) can use non-padded reference samples and be added. If Y is available (all reference samples above Y are available), modes 2 to 7 (i.e., modes used for W / H=1 but not for W / H>1) can use non-padded reference samples and be added. It should be noted that it is possible to perform intra-mode removal without adding other intra-prediction modes, or to add some intra-prediction modes without removing any intra-modes.

[0054] Intra prediction modes can also be removed / added using the intra prediction modes listed in Table 1. In one embodiment, the allowed modes are based on a set of modes designed for another block size. For example, depending on the available neighboring reference samples, a square block may use a mode designed for W / H=2, as shown in Table 1. It should be noted here that some intra modes in the resulting set of intra modes may still use padded reference samples, but in general, the number of padded samples used will be less than in the original set of intra prediction modes. Some examples are described in detail below.

[0055] Let the position of the top-left sample of the PU to be encoded / decoded be ref[0;0], the distance of the reference line specified by mrlIdx be D, and the width and height of the current PU be W and H, respectively. If W > H, - When the upper samples from ref[-1-D;-1-D] to ref[2*W+W*D / H-1;-1-D] are available (i.e., non-padded samples), a set of modes defined for the aspect ratio W / H can be used, for example, the modes used in ECM-6.0 as described in Table 1. [E001] - Otherwise (if at least one of those samples is not available), when all the left samples from ref[-1-D;-1-D] to ref[-1-D;H+W / 2+H*D / (W / 2)-1] are available, a set of modes defined for the aspect ratio (W / 2) / H can be used. Otherwise (if at least one of those left samples is not available), a set of modes defined for the aspect ratio W / H can be used. [E002] Otherwise, if W < H, - When the left samples from ref[-1-D;-1-D] to ref[-1-D;2*H+H*D / W-1] are available (i.e., non-padded samples), a set of modes defined for the aspect ratio W / H can be used, for example, the modes used in ECM-6.0. [E003] - Otherwise (if at least one of those samples is not available), when all the upper samples from ref[-1-D;-1-D] to ref[W+H / 2+W*D / (H / 2)-1;-1-D] are available, a set of modes defined for the aspect ratio W / (H / 2) can be used. Otherwise (if at least one of those upper samples is not available), a set of modes defined for the aspect ratio W / H can be used. [E004]

[0056] The conditions E001 and E003 check whether all samples are available in WAIP mode. If at least one sample is not available, then based on the subsequent conditions, normal ECM mode (i.e., the mode defined in Table 1) may not be used. In some embodiments, this first condition is more conservative, and normal ECM mode is not used only if none of the samples it uses are available. In such embodiments, E001 is written as follows: - If at least one upper sample between ref[W+W / 2+(W / 2)*D / H;-1-D] and ref[2*W-1+W*D / H;-1-D] is available (i.e., a non-padded sample), the set of modes defined for aspect ratio W / H can be used. And E003 is written as follows: - If at least one left sample from ref[-1-D;H+H / 2+(H / 2)*D / W] to ref[-1-D;2*H-1+H*D / W] is available (i.e., a non-padded sample), the set of modes defined for the aspect ratio W / H can be used.

[0057] Conditions E002 and E004 check whether samples are available for modes not normally used in ECM, and those modes are added only if all samples are available. In some embodiments, this condition is relaxed and a mode is added if at least one of the samples is available. In such embodiments, E002 is written as follows: - Otherwise, if at least one left sample between ref[-1-D;2*H+H*D / W] and ref[-1-D;H+W / 2+H*D / (W / 2)-1] is available, then the set of modes defined for aspect ratio (W / 2) / H may be used, otherwise (if none of these samples are available), then the set of modes defined for aspect ratio W / H may be used. And E004 is written as follows: - Otherwise, if at least one upper sample between ref[2*W+W*D / H;-1-D] and ref[W+H / 2-1+W*D / (H / 2);-1-D] is available, the set of modes defined for aspect ratio W / (H / 2) MAY be used, otherwise (if none of these samples are available), the set of modes defined for aspect ratio W / H MAY be used.

[0058] In some embodiments, a combination of these conditions is used, and in some embodiments, different conditions are used depending on, but not limited to, block size, sequence size, QP, or neighbor information.

[0059] In some embodiments, more modes are added. In such embodiments, if the set of ECM modes of W / H is replaced with a set of modes of (W / 2) / H (respectively, W / (H / 2)), tests E001-E004 are run again as if the block width were W / 2 (respectively, height is H / 2). If a new set of modes is selected, this can be run again until the set of modes used no longer needs to be changed.

[0060] In some embodiments, existing availability checks of neighboring CUs used to construct the MPM list can be used to determine whether a mode should be added. For example, a test for reference sample availability is performed at positions 1 and 2, then positions 3 and 4, as shown in Figure 19. In that case, conditions [E001]-[E004] do not depend on whether W is greater than H. In one example, more modes would be added and none would be removed, which may be particularly useful when no signaling is required, such as when modes are added for TIMD or DIMD.

[0061] In one example, if the top samples from ref[-1;-1] to ref[W;-1] are available (i.e., unpadded samples up to position 1), all modes defined for an aspect ratio of 2*W / H as described in Table 1 can be used in addition to the modes already allowed for this CU. If the top samples ref[2*W;-1] are also available (i.e., unpadded samples), all modes defined for an aspect ratio of 4*W / H as described in Table 1 can also be used in addition to the modes already allowed for this CU. If the left samples from ref[-1;-1] to ref[-1;H] are available (i.e., unpadded samples up to position 2), all modes defined for an aspect ratio of W / (H*2) as described in Table 1 can be used in addition to the modes already allowed for this CU. If the top samples ref[-1;2*H] are also available (i.e., unpadded samples), all modes defined for an aspect ratio of W / (H*4) as described in Table 1 can also be used in addition to the modes already allowed for this CU.

[0062] Intramode signaling

[0063] In some embodiments, the number of available IPMs is maintained at 67 at all times, with a fixed number of 65 angle IPMs. In such embodiments, no signaling changes are required. In other embodiments, syntax elements associated with intra-mode signaling, such as the CABAC coding bin context for the first MPM flag, the second MPM flag, or the first MPM index flag, may be modified to take into account the available modes. For example, one of the three context model indexes will be selected depending on the following conditions: 1) The IPMs used are the same as those currently used in ECM-6.0. For example, for a PU with W / H=1, they are angular modes 2 to 66, Planar, and DC. 2) The IPM used is changed, typically to allow for the use of IPMs available in modes with larger W / H ratios. For example, for a PU with W / H=1, this means using angle modes I to I+64 with I>2, as well as Planar and DC. 3) The IPM used is changed, typically to allow for the use of IPMs available in modes with smaller W / H ratios. For example, for a PU with W / H=1, this means using angle modes I to I+64 with I<2, as well as Planar and DC.

[0064] In some embodiments, the flag intra_luma_mpm_flag used to identify whether the intra mode used in the current luminance CB is in the MPM list changes its syntax as follows, according to Table 128 of the VTM specification text:

[0065] [Table 2] The value of intra_mode_set_diff is derived as follows: If the set of modes used for the current luminance CB of position posX, width and height posY, W and H is the set designed for a block of ratio W / H as defined in Table 1, intra_mode_set_diff[posX][posY] is set to 0. Otherwise, if the set of modes is designed for blocks with a ratio entirely larger than W / H, intra_mode_set_diff[posX][posY] is set to 1. Otherwise, intra_mode_set_diff[posX][posY] is set to 2.

[0066] The initialization of the context model, as defined in Table 70 of the VTM specification text, is as follows:

[0067]

Table 3

[0068] In some embodiments, the three contexts are 1) The IPM used is the same as the IPM currently used in ECM-6.0. For example, in a PU with W / H = 1, they are in angular modes 2 to 66 and become Planar and DC. 2) The IPM used is changed so that it can usually use an IPM available in a mode with a larger abs(log2(W / H)) ratio. For a PU with W / H ≥ 1 (respectively, W / H ≤ 1), this means using the angular modes from I to I + 64 where I > J (respectively, I < J), and J is the smallest angular mode normally available for this PU as well as Planar and DC. 3) The IPM used is changed so that it can usually use an IPM available in a mode with a larger abs(log2(W / H)) ratio. For a PU with W / H > 1 (respectively, W / H < 1), this means using the angular modes from I to I + 64 where I < J (respectively, I > J), and J is the smallest angular mode normally available for this PU as well as Planar and DC and normally used.

[0069] In some embodiments, the above rules can be reduced to use only two CABAC contexts (i.e., depending on whether the mode is changed). In some embodiments, these rules can be combined with other information such as, but not limited to, the PU size used, sequence size, QP, prediction tool, etc.

[0070] In some embodiments, the number of available modes varies from PU to PU, and therefore the signaling changes to account for the different number of modes. In one embodiment, the first and second MPM lists retain the original number of encoded flags, and the remaining IPMs are encoded using truncated binary encoding for N-22 symbols, where N is the number of modes available for this PU (N=45 means using the same signaling as ECM-6.0, N>45 means more modes are available than ECM-6.0, and N<45 means fewer modes are available than ECM-6.0).

[0071] In some embodiments, restrictions on the availability of IPMs are limited to Template-based Intra Mode Derivation (TIMD) and / or DIMD, and / or other decoder-side tools, so as not to require signaling changes. In such embodiments, the TIMD search (respectively, the DIMD search, and / or other decoder-side tool) may be limited to the IPMs that are deemed available.

[0072] In embodiments where the TIMD can use additional wide angles, only a subset of modes are added to reduce the increase in search complexity. The additional modes can be included in the first part of the search, for example, as described below.

[0073] Let W and H be the width and height of the block to be coded, minOrg and maxOrg be the minimum and maximum angular intra mode values ​​available in the current block, using the TIMD values ​​of 131 modes (e.g., from Table 1, angular modes for W / H=2 are 8 to 72, corresponding to minOrg=13 and maxOrg=141), and let newMin and newMax be the new minimum and maximum angular intra mode values ​​available in the current block, determined, for example, from E001 to E004 (e.g., if all adjacent samples are available in a block with W / H=2, modes from -4 to 7 and modes from 73 to 78 are added according to Table 1, i.e., using the TIMD values ​​of 131 modes, newMin=-9 and newMax=153). Starting from newMin+1 with a step size of N (e.g., N=5), if a mode is not between minOrg and maxOrg, it is added to the first part of the TIMD search; otherwise, no mode is added.

[0074] Certain modes can also be chosen to always be added to the search. For example, newMin+1, orgMin-1, orgMax+1, newMax+1 can be chosen to always be added to the first part of the search. These modes can be the only modes added to the search to reduce design complexity, or they can be added on top of modes already added to maximize compression gain. The second part of the TIMD search (the refinement part) can be done similarly to ECM.

[0075] In some embodiments, if, before decoding the IPM index, it is found that several additional modes are available, an additional flag is decoded to indicate whether the original 67 modes allowed for the current block size are used, or whether one of N additional modes is used. If one of N additional modes is used, the additional index is decoded using a truncated binary code for N symbols.

[0076] Propagation of the intra direction

[0077] In ECM, if the left neighboring block has W / H=2 and uses angular mode 67 for prediction, the mode index used to construct the MPM list of the current block is 2. However, if the current luma block is square, index 2 corresponds to angular mode 2. Therefore, using the mode index of the neighboring mode to construct the MPM list of the current block may lead to adding a mode to the MPM list that was not actually used and therefore should not be considered the "most probable" for decoding the current luma-coded block. Furthermore, in ECM, each index can correspond to two different angular modes (e.g., index 2 is either angular mode 2 or angular mode 67, index 3 is either angular mode 3 or angular mode 68, etc.), but the two different modes are not 180-degree opposite.

[0078] The fact that the two modes are 180 degrees opposite means that they predict the same directional texture (angle modes 2 and 66 both predict a 45 degree direction), but from different references (angle mode 2 predicts from bottom left to top right, and angle mode 66 predicts from top right to bottom left).

[0079] In some embodiments, the generation of the MPM list is done from the actual modes used by neighboring blocks instead of the index used. For example, when constructing the MPM list, if a neighboring mode is not available, the mode is replaced by the corresponding mode at 180 degrees, i.e., if the mode IPM is less than 34, the mode is replaced by IPM+64, otherwise the mode is replaced by IPM-64. This is always possible in ECMs up to ECM-6.0, since there is always a 180-degree angle mode range.

[0080] This may not be the case for encoders or decoders that remove some modes without adding others, such as those described in some of the previous embodiments. If, during MPM list construction, a mode that is not available at the current luminance CB needs to be added, that mode is replaced with the closest available mode. In some embodiments, the modes are selected to always have a range of 65 angle modes to ensure that any angle is available and it is always possible to replace a mode with its 180-degree counterpart.

[0081] Beyond the wide-angle intra modes

[0082] The selection of available intra-prediction modes based on the availability of neighboring decoded reference samples can be extended to a rule that no longer involves Table 1. This means that for a given block, depending on the availability of its neighboring decoded reference samples, the set of available intra-prediction modes for its effective ratio W / H (see Table 1) is no longer replaced with a different set of available intra-prediction modes associated with width and height ratios close to the effective ratio W / H. Instead, for a given availability of neighboring decoded reference samples for a given block, a given number of intra-prediction modes may be eliminated.

[0083] For example, the rules for potentially suppressing intra prediction modes may be as follows: For a given W×H block, if none of the W decoded reference samples located on the upper right side are available, the last n0∈N positive vertical intra prediction mode is not allowed (e.g., n0=4). The "last" n0 positive vertical intra prediction mode refers to the n0 positive vertical intra prediction mode with the largest absolute angle relative to the vertical axis. According to ECM nomenclature, the "last" n0 positive vertical intra prediction mode refers to the n0 positive vertical intra prediction mode with the largest index. If none of the H decoded reference samples located on the lower left side are available, the first n1∈N positive horizontal intra prediction mode is not allowed (e.g., n1=4). The "first" n1 positive horizontal intra prediction mode refers to the n1 positive horizontal intra prediction mode with the largest absolute angle relative to the horizontal axis. According to ECM nomenclature, the "first" n1 positive horizontal intra-prediction mode refers to the n1 positive horizontal intra-prediction mode with the smallest index. These indices can be negative in the case of wide-angle intra-prediction.

[0084] As an example, this rule can be illustrated in Figure 12 for a given WxH luma coding block (CB) belonging to an ECM-5.0 intra slice. For this given luma CB (1201) in an ECM-5.0 intra slice, disallowed intra prediction modes are identified from the partitioning history of (1201), where W = 16 and H = 8. The indices 0, 1, 2, and 3 indicate the order of encoding / decoding the first four luma CBs belonging to the first 64x64 luma CB (1200) resulting from the QT partitioning of the considered luma CTB.

[0085] In this example, it is important to note that for a given W×H luma CB, at encoding time, the availability of its neighboring decoded reference samples can be fully specified before writing any bits of its parent luma coding tree block (CTB) partition to the bitstream, i.e., before writing any bits associated with intra prediction in its parent luma CTB. For a given W×H luma CB, at decoding time, the availability of its neighboring decoded reference samples can be fully specified immediately after reading bits of its parent luma CTB partition from the bitstream, i.e., before reading any bits associated with intra prediction in its parent luma CTB. In the following example, in ECM-5.0, the CTU size is set to 128, as in VVC, to obtain an example that is more similar to the version in VVC.

[0086] On the encoder side, a given 128x128 luma CTB is split into four 64x64 luma CBs via a quad-tree (QT). For example, consider the first 64x64 luma CB (1200). A split at a specific depth is characterized by (typeSplit, idxChild). "typeSplit" refers to the split type: {Quad-Tree (QT), Binary-Tree Horizontal (BT_H), Binary-Tree Vertical (BT_V), Ternary-Tree Horizontal (TT_H), Ternary-Tree Vertical (TT_V)}, and "idxChild" indicates the index in the encoding order of the child CB of interest resulting from this split. The considered partition of the luminance CB (1201) is completely described by its partition tree {(QT,0), (QT,0), (BT_V,1), (TT_H,2)} as shown in Figure 12A. This partition tree reveals the fact that none of the decoded references of W on the top right side of (1201) are available, and none of the decoded reference samples of H on the bottom left side of (1201) are available.

[0087] These two portions of unavailable decoded reference samples are summarized as (1202) in Figure 12B. For example, this is indicated by the flags "is_above_right_full" and "is_below_left_full" added to (1201) being 0. The last n0 positive vertical intra-prediction modes are disallowed, with (1204) and (1203) representing the direction of the smallest indexed intra-prediction mode and the direction of the largest indexed intra-prediction mode, respectively, in this set of disallowed modes. The first n1∈N positive horizontal intra-prediction modes are disallowed, with (1205) and (1206) representing the direction of the smallest indexed intra-prediction mode and the direction of the largest indexed intra-prediction mode, respectively, in this set of disallowed modes, as shown in Figure 12C.

[0088] Finally, the signaling of the index of the intra-prediction mode selected for prediction (1201) is adjusted to take into account the removed intra-prediction mode. For example, if the selected intra-prediction mode is not a template-based intra-prediction (TMP), decoder-side intra-mode derivation (DIMD), template-based intra-mode derivation (TIMD), or matrix-based intra-prediction (MIP) mode, does not use an MRL, and is not an MPM, its index is truncated binary coded with a code length of N-n0-n1. Note that since ECM-5.0 includes 67 common intra-prediction modes (65 directional, planar, and DC), 6 primary MPMs, and 16 secondary MPMs, the index of a common intra-prediction mode that is not an MPM may have N=45 possibilities.

[0089] On the decoder side, the process follows the encoder side process except for signaling the index of the intra-prediction mode selected for prediction (1201). Considering the example above, if the selected intra-prediction mode is not TMP, DIMD, TIMD, or MIP mode, does not use MRL, and is not MPM, then the index is decoded with a truncated binary code for 45-n0-n1 possible symbols.

[0090] Another example of this embodiment can be seen in Figure 13 for a given WxH luma coding block (CB) belonging to an ECM-5.0 intra slice. For this given WxH luma CB (1301) in an ECM-5.0 intra slice, disallowed intra prediction modes are identified from the partitioning history of (1301), where W = 8 and H = 16. The indices 0 to 7 indicate the order of encoding / decoding the first eight luma CBs belonging to the first 64x64 luma CB (1300) resulting from the QT partitioning of the considered luma CTB.

[0091] On the encoder side, a given 128x128 luma CTB is split into four 64x64 luma CBs via QT. For example, consider the first 64x64 luma CB (1300). The splitting of the considered luma CB (1301) is completely described by its splitting tree {(QT,0), (QT,1), (BT_H,1), (BT_V,0), (BT_V,1)} as shown in Figure 13A.

[0092] From this partitioning tree, it is easy to infer that all decoded references for W on the upper right side of (1301) are available, but none of the decoded reference samples for H on the lower left side of (1301) are available, as shown in (1302) of Figure 13B. For example, this is indicated by the flags "is_above_right_full" and "is_below_left_full" added to (1301) being 1 and 0, respectively. The first n1∈N positive horizontal intra-prediction modes are disallowed, and (1303) and (1304) represent the direction of the intra-prediction mode with the smallest index and the direction of the intra-prediction mode with the largest index, respectively, in this set of disallowed modes, as shown in Figure 13C. Finally, the signaling of the index of the intra-prediction mode selected for prediction (1301) is adjusted to take into account the removed intra-prediction modes. For example, if the selected intra prediction mode is not a TMP, DIMD, TIMD, or MIP mode, does not use MRL, and is not MPM, then the index is encoded with a truncated binary code over 45-n1 possible symbols.

[0093] On the decoder side, the process follows the encoder side process except for signaling the index of the intra mode selected for prediction (1301). Considering the example above, if the selected intra prediction mode is not TMP, DIMD, TIMD, or MIP mode, does not use MRL, and is not MPM, then the index is decoded with a truncated binary code for 45-n1 possible symbols.

[0094] The above example can be adapted to any other block in another channel / slice. Furthermore, the rule for suppressing intra prediction modes depending on the availability of neighboring decoded reference samples of the current block can be easily modified. For example, the rule can be: "For a given W×H block, if none of the rightmost W / 2 decoded reference samples located on the upper right side are available, then the last n0∈N positive vertical intra prediction modes are not allowed. If none of the bottommost H / 2 decoded reference samples located on the lower left side are available, then the first n1∈N positive horizontal intra prediction modes are not allowed."

[0095] The workflow for encoding the index of the selected intra-prediction mode at the encoder side according to this embodiment for a given W×H block is summarized in Figure 14. In this embodiment, the rules indicating the conditional relationship between the availability of neighboring reference samples of a given block and which intra-prediction modes are removed for the given block are known at the encoder side. In particular, for the current block, the encoder identifies unavailable reference samples from the segmentation history of the current block (1410). For example, for the segmentation history of {(QT,0),(QT,0),(BT_V,1),(TT_H,2)}, is_above_right_full=false and is_below_left_full=false in Figure 12. Based on the identified unavailable reference samples and the rules for removing intra-prediction modes, the encoder removes some intra-prediction modes (1420) (these intra-prediction modes are unavailable for the current block). For example, if is_above_right_full=false and is_below_left_full=false, then in FIG. 12 the last n0 positive vertical intra prediction modes and the first n1 positive horizontal intra prediction modes are not allowed.

[0096] Taking the removed mode into account, the encoder adapts signaling of the index of the intra-prediction mode selected to predict the current block (1430). Typically, the total number of available intra-prediction modes is adjusted by reducing it by the amount of removed intra-prediction modes. For example, if the selected intra-prediction mode is not a TMP, DIMD, TIMD, or MIP mode, does not use MRL, and is not MPM, its index is truncated binary coded with code length 45-n0-n1 in FIG. 12. The coded index is written to the bitstream (1440).

[0097] For this W×H block, the workflow for decoding the index of the selected intra-prediction mode at the decoder side according to this embodiment is summarized in Figure 15. The steps (1510, 1520, 1530, 1540) on the decoder side correspond to the steps on the encoder side.

[0098] These two figures show the workflow for only a single block. When multiple blocks are considered, the order of the steps in Figures 14 and 15 will vary depending on the codec in question.

[0099] 12-15, for a given W×H block, unavailable decoded reference samples are identified from the segmentation history of this block. In other embodiments, unavailable decoded reference samples may be identified using a function that searches for already decoded blocks surrounding the current block.

[0100] For example, as shown in FIG. 16, for a given W×H CB (1602) of an ECM-5.0 intra slice, the function “getCURestricted” receives a pixel position “pos,” e.g., “posAR” (1603) or “posBL” (1604), a coding unit (CU) “curCu” of the given W×H CB (1602), and a channel type “chType” of the CB (1602), and returns a pointer to an already decoded CB containing the pixel located at “pos.” If the pixel located at “pos” does not belong to any CB or belongs to a CB that has not yet been decoded, “getCURestricted” may return a pointer NULL, e.g., “nullptr” in C++. In FIG. 16, CBs (1600), (1601), and (1602) are obtained from the last two splits BT_V and BT_H in the current state of encoding / decoding. For example, in FIG. 16, "posAR" belongs to a CB that has not yet been decrypted, so "getCURestricted(posAR,curCu,chType)" returns "nullptr".

[0101] The workflow for encoding the index of the selected intra-prediction mode at the encoder side according to this embodiment for a given W×H block is summarized in Figure 17. In particular, for the current block, the encoder identifies (1710) unavailable reference samples from the segmentation history of the current block. For example, in Figure 16, if getCURestricted(posAR,curCu,chType)=nullptr, then is_above_right_full=false, and if getCURestricted(posBL,curCu,chType)=nullptr, then is_below_left_full=false. Based on the identified unavailable reference samples and the rules for removing intra-prediction modes, the encoder removes (1720) some intra-prediction modes (these intra-prediction modes are unavailable for the current block). For example, if is_above_right_full=false and is_below_left_full=false, then the last n0 positive vertical intra prediction modes and the first n1 positive horizontal intra prediction modes are not allowed.

[0102] Taking the removed mode into account, the encoder adapts (1730) the signaling of the index of the intra-prediction mode selected to predict the current block. For example, if the selected intra-prediction mode is not TMP, DIMD, TIMD, or MIP mode, does not use MRL, and is not MPM, its index is truncated binary coded with code length 45-n0-n1. The coded index is written to the bitstream (1740).

[0103] For this W×H block, the workflow for decoding the index of the selected intra-prediction mode at the decoder side according to this embodiment is summarized in Figure 18. The steps (1810, 1820, 1830, 1840) on the decoder side correspond to the steps on the encoder side.

[0104] In another embodiment, instead of removing a given number of intra-prediction modes for a given block depending on the availability of its neighboring decoded reference samples, the list of its MPMs can be rearranged to focus on unavailable decoded reference samples for prediction, and for some intra-prediction modes that have their indexes in this list of MPMs, their indexes can be moved toward the end of this list of MPMs. Intra-prediction modes whose indexes are moved toward the end of the list of MPMs for a given block are considered to be less likely to be selected as intra-prediction modes to predict the given block.

[0105] An embodiment of a current W×H luma CB in an intra slice of ECM-5.0 is shown in Figure 19. In particular, Figure 19 shows, for a given current W×H luma CB, the generation of a general list of 22 MPMs for this luma CB. The first 6 MPMs in the general list of MPMs correspond to the list of the first MPM, while the last 16 MPMs in the general list of MPMs become the list of the second MPM.

[0106] In FIG. 19, the planar mode is first added to the general list of the MPM (1900). Next, the indices of the intra prediction modes selected to predict the left, top, bottom-left, top-right, and top-left luminance CBs are added to the general list of the MPM (1901-1905). Next, the indices of the two intra prediction modes derived via DIMD for the current luminance CB are added to the general list of the MPM (1906, 1907). Next, if the current second MPM is neither planar nor DC, the indices of its eight neighboring angular intra prediction modes are added to the general list of the MPM (1908). Next, if the current third MPM is neither planar nor DC, the indices of its eight neighboring angular intra prediction modes are added to the general list of the MPM (1909).

[0107] After potentially adding more angular intra-prediction mode indices according to (1910), the index of the default mode is inserted into the general list of MPMs to reach 22 MPMs (1911). Note that each of the above insertions is applied under the condition that there is no redundancy in the general list of MPMs. That is, when inserting the index of the current intra-prediction mode into the general list of MPMs, if this index already exists in this list, the insertion is skipped.

[0108] Figure 20 illustrates the generation of a general list of 22 MPMs for the same current luma CB according to one embodiment. In Figure 20, the generation of the general list of 22 MPMs for the current W x H luma CB follows the workflow of Figure 19, except that reordering may be introduced. For example, a function f may receive as a first argument an index of a candidate intra-prediction mode to be added to the general list of MPMs, and as a second argument an array "res" of reserved mode indices. Then, if the candidate intra-prediction mode is "valid" under a condition that depends on the availability of decoded reference samples for the current W x H luma CB, f may enter the index of the candidate intra-prediction mode into the general list of MPMs. Otherwise, f may add the index of this intra-prediction mode to "res."

[0109] The planar mode is first added to the general list of the MPM (2000). Next, the indices of the intra prediction modes selected to predict the left, top, bottom left, top right, and top left luminance CBs are added to the general list of the MPM under the validity conditions defined by f(2001-2005). Next, all intra prediction mode indices stored in "res" are added to the general list of the MPM (2006). Next, the indices of the two intra prediction modes derived via DIMD for the current luminance CB are added to the general list of the MPM (2007, 2008). The final steps (2009), (2010), (2011), and (2012) follow steps (1908), (1909), (1910), and (1911) in FIG. 19, respectively.

[0110] In another embodiment, f may (or may not) be applied to the index of any candidate intra-prediction mode that may be added to the general list of MPMs for the current luma CB. Furthermore, the step of adding all intra-prediction mode indices stored in "res" to the general list of MPMs for the current luma CB may occur at any time during the generation of the general list of MPMs.

[0111] For example, in the embodiment shown in Figure 21, the addition of all intra prediction modes stored in "res" (2108) is performed after potentially placing the indices of the two intra prediction modes derived via DIMD for the current luminance CB (2106, 2107) into the general list of the MPM.

[0112] For example, in another embodiment shown in Figure 22, the addition of all intra prediction modes stored in "res" (2208) is performed after potentially placing the indices of the two intra prediction modes derived by DIMD for the current luminance CB into the general list of the MPM, under the conditions of validity defined by f (2206, 2207).

[0113] In one embodiment, the validity condition defined by f for the intra-prediction mode index passed as the first argument may be that if none of the W decoded reference samples on the top right side of the current W×H block are available, then the last n0∈N positive vertical intra-prediction modes are invalid (e.g., n0=8). If none of the H decoded reference samples on the bottom left side of the current W×H block are available, then the first n1∈N positive horizontal intra-prediction modes are invalid (e.g., n1=8).

[0114] In another embodiment, the validity condition defined by f for the intra-prediction mode index passed as the first argument may be that if none of the right-most W / 2 decoded reference samples on the top-right side of the current W×H block are available, then the last n0∈N positive vertical intra-prediction modes are invalid (e.g., n0=4). If none of the bottom-most H / 2 decoded reference samples on the bottom-left side of the current W×H block are available, then the first n1∈N positive horizontal intra-prediction modes are invalid (e.g., n1=4).

[0115] In yet another embodiment, the validity condition defined by f for the intra-prediction mode index passed as the first argument may be that if the decoded reference samples above (including top-left, top, and top-right) the current W×H block are unavailable, then the last q0∈N vertical intra-prediction modes are invalid (e.g., q0=8). If the decoded reference samples to the left (including top-left, left, and bottom-left) of the current W×H block are unavailable, then the first q1∈N horizontal intra-prediction modes are invalid (e.g., q1=8).

[0116] In yet another embodiment, the validity condition defined by f for the index of the intra prediction mode passed as the first argument may be a combination of several conditions depending on various states of the availability of decoded reference samples of the current W×H block. For example, if the decoded reference samples above (including the top-left, top, and top-right) the current block are unavailable, the last q0∈N vertical intra prediction modes are invalid (e.g., q0=8). Otherwise, the following condition is checked: If none of the W decoded reference samples on the top-right side of the current block are available, the last n0∈N positive vertical intra prediction modes are invalid (e.g., n0=5). As another example, if the decoded reference samples on the left side (including the top-left, left, and bottom-left) of the current block are available, the first q1∈N horizontal intra prediction modes are invalid (e.g., q1=5). Otherwise, the following condition is checked: If none of the H decoded reference samples on the bottom-left side of the current block are available, the first n1∈N positive horizontal intra prediction modes are invalid (eg, n1=5).

[0117] Various methods are described herein, each of which includes one or more steps or actions for achieving the described method. Unless a particular order of steps or actions is necessary for the proper operation of a method, the order and / or use of particular steps and / or actions may be varied or combined. Additionally, terms such as “first,” “second,” and the like may be used in various embodiments to vary elements, components, steps, operations, etc., such as, for example, “first decode” and “second decode.” The use of such terms does not imply a varied order of operations unless specifically required. Thus, in this example, the first decode need not be performed before the second decode, but could occur, for example, before, during, or within an overlapping period with the second decode.

[0118] Various methods and other aspects described in this application can be used to modify modules of the video encoder 200 and decoder 300, such as the intra-prediction modules (260, 360), as shown in Figures 2 and 3. Furthermore, the aspects are not limited to ECM, VVC, or HEVC, but can be applied to, for example, other standards and recommendations, and extensions of any such standards and recommendations. Unless otherwise specified or technically precluded, the aspects described in this application can be used individually or in combination.

[0119] Various numerical values ​​are used in this application. The specific values ​​are for illustrative purposes and the described aspects are not limited to these specific values.

[0120] Various implementations include decoding. As used herein, "decoding" can include all or part of the processes performed on a received encoded sequence, for example, to generate a final output suitable for display. In various embodiments, such processes include one or more processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. Whether the term "decoding process" is intended to refer specifically to a subset of operations or to the broader decoding process generally will be clear based on the context of a particular description and will be well understood by those skilled in the art.

[0121] Various implementations include encoding. Similar to the above description of "decoding," "encoding" as used herein may include, for example, all or part of the processes performed on an input video sequence to generate an encoded bitstream.

[0122] The implementations and aspects described herein may be implemented as, for example, a method or process, an apparatus, a software program, a data stream, or a signal. Even if described in the context of only one type of implementation (e.g., described only as a method), the described functionality may also be implemented in other forms (e.g., an apparatus or a program). An apparatus may be implemented, for example, in appropriate hardware, software, and firmware. The methods may be implemented in a processor, which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include, for example, communication devices such as computers, mobile phones, portable / personal digital assistants (PDAs), and other devices that facilitate communication of information between end users.

[0123] References to "one embodiment" or "embodiment" or "one implementation" or "implementation," as well as other variations thereof, mean that a particular feature, structure, characteristic, etc. described in connection with an embodiment is included in at least one embodiment. Thus, appearances of the phrases "in one embodiment" or "in an embodiment" or "in one implementation" or "in an implementation" appearing in various places in this application, as well as any other variations thereof, are not necessarily all referring to the same embodiment.

[0124] Additionally, this application may refer to "determining" various pieces of information. Determining information may include, for example, one or more of estimating information, calculating information, predicting information, or retrieving information from memory.

[0125] Additionally, the present application may refer to "accessing" various pieces of information. Accessing information may include, for example, one or more of receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or inferring information.

[0126] Additionally, the present application may refer to "receiving" various information. Receiving, like "accessing," is intended as a broad term. Receiving information may include, for example, one or more of accessing information or retrieving information (e.g., from a memory). Furthermore, "receiving" is typically involved in some way in an operation such as, for example, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or inferring information.

[0127] For example, in the cases of "A / B," "A and / or B," and "at least one of A and B," the use of any of " / ," "and / or," and "at least one of" should be understood to be intended to include the selection of only the first listed option (A), or the selection of only the second listed option (B), or the selection of both options (A and B). As a further example, in the cases of "A, B, and / or C" and "at least one of A, B, and C," such language is intended to include the selection of only the first listed option (A), or the selection of only the second listed option (B), or the selection of only the third listed option (C), or the selection of only the first and second listed options (A and B), or the selection of only the first and third listed options (A and C), or the selection of only the second and third listed options (B and C), or the selection of all three options (A, B, and C). This can be expanded as many times as there are listed items, as would be apparent to one of ordinary skill in this and related arts.

[0128] Also, as used herein, the term "signal" refers to, among other things, indicating something to a corresponding decoder. For example, in certain embodiments, an encoder signals a quantization matrix for inverse quantization. Thus, in one embodiment, the same parameters are used at both the encoder and decoder sides. Thus, for example, an encoder can send a specific parameter to a decoder (explicit signaling) so that the decoder can use the same specific parameter. Conversely, if the decoder already has a specific parameter as well as other parameters, the encoder can use signaling (implicit signaling) without sending the specific parameter so that the decoder can recognize and select the specific parameter. By avoiding sending actual functions, bit savings are achieved in various embodiments. It should be understood that signaling can be achieved in various ways. For example, one or more syntax elements, flags, etc. are used to signal information to a corresponding decoder in various embodiments. While the above refers to the verb form of the word "signal," the word "signal" can also be used as a noun herein.

[0129] As will be apparent to those skilled in the art, implementations can generate various signals formatted to carry information that can be stored or transmitted, for example. The information can include, for example, instructions for performing a method or data generated by one of the described implementations. For example, a signal can be formatted to carry a bitstream of the described embodiments. Such a signal can be formatted, for example, as an electromagnetic wave (e.g., using the radio frequency portion of the spectrum) or as a baseband signal. Formatting can include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information carried by the signal can be, for example, analog or digital information. The signal can be transmitted over a variety of different wired or wireless links, as is known. The signal can be stored on a processor-readable medium.

Claims

1. 1. A method of decoding video, comprising: identifying the availability of one or more reference samples for a block to be decoded in a picture; obtaining a set of intra-prediction modes for the block according to availability of the one or more reference samples; obtaining an intra-prediction mode from the set of intra-prediction modes; performing intra prediction on the block to be decoded based on an intra prediction mode of the block to form a predicted block for the block; A method comprising:

2. 1. A method of encoding video, comprising: identifying the availability of one or more reference samples for a block to be encoded in a picture; obtaining a set of intra-prediction modes for the block according to availability of the one or more reference samples; selecting an intra-prediction mode from the set of intra-prediction modes; performing intra prediction on the block to be encoded based on an intra prediction mode of the block to form a predicted block for the block; A method comprising:

3. The method of claim 1 or claim 2, wherein the availability of the one or more reference samples is identified based on the partitioning history of the block.

4. Obtaining the set of intra-prediction modes includes: Obtaining a first set of intra-prediction modes; adjusting the first set of intra-prediction modes to the set of intra-prediction modes in response to availability of the one or more reference samples; 4. The method of claim 1, comprising:

5. The method according to claim 1 , wherein the first set of intra prediction modes is obtained based on an aspect ratio of the block.

6. The method of claim 5 , wherein the set of intra-prediction modes is obtained based on an aspect ratio different from the aspect ratio of the block.

7. The method of claim 6 , wherein the different aspect ratio is half or double the aspect ratio of the block.

8. The adjusting step comprises: The method of claim 1 , comprising removing multiple vertical positive intra-prediction modes in response to one or more top right reference samples being unavailable.

9. The adjusting step comprises: The method of claim 1 , comprising removing multiple horizontal positive intra-prediction modes in response to one or more bottom-left reference samples being unavailable.

10. The adjusting step comprises: The method of claim 1 , further comprising adding multiple vertical positive intra-prediction modes depending on the availability of one or more top right reference samples.

11. The adjusting step comprises: The method of claim 1 , further comprising adding multiple horizontal positive intra-prediction modes depending on the availability of one or more bottom-left reference samples.

12. The method according to claim 1 , wherein the index corresponding to the intra-prediction mode is signaled depending on the number of intra-prediction modes in the set of intra-prediction modes.

13. 13. The method of claim 12, wherein, depending on which intra-prediction mode belongs to a set of remaining modes, the set of remaining modes excludes Most Probable Modes (MPMs), and the index is coded using truncated binary coding of N-M symbols, where N is the number of intra-prediction modes in the set of intra-prediction modes and M is the number of MPMs.

14. The method of claim 1 , wherein a context index of a syntax element depends on the set of intra-prediction modes.

15. The method of claim 14 , wherein the syntax element is used to signal the intra-prediction mode.

16. 1. An apparatus for decoding video, comprising one or more processors, The one or more processors: identifying the availability of one or more reference samples for a block to be decoded in a picture; obtaining a set of intra-prediction modes for the block according to availability of the one or more reference samples; obtaining an intra-prediction mode from the set of intra-prediction modes; performing intra prediction on the block to be decoded based on an intra prediction mode of the block to form a predicted block for the block; 2. An apparatus configured to perform the steps of:

17. 1. An apparatus for encoding video, comprising one or more processors, The one or more processors: identifying the availability of one or more reference samples for a block to be encoded in a picture; obtaining a set of intra-prediction modes for the block according to availability of the one or more reference samples; selecting an intra-prediction mode from the set of intra-prediction modes; performing intra prediction on the block to be encoded based on an intra prediction mode of the block to form a predicted block for the block; 2. An apparatus configured to perform the steps of:

18. 18. The apparatus of claim 16 or claim 17, wherein the availability of the one or more reference samples is identified based on a partitioning history of the block.

19. The one or more processors: Obtaining a first set of intra-prediction modes; adjusting the first set of intra-prediction modes to the set of intra-prediction modes in response to availability of the one or more reference samples; and obtaining the set of intra-prediction modes by 19. Apparatus according to any one of claims 16 to 18.

20. The device of claim 16 , wherein the first set of intra prediction modes is obtained based on an aspect ratio of the block.

21. A signal containing video data formed by carrying out a method according to any one of claims 2 to 15.

22. A computer-readable storage medium having stored thereon instructions for encoding or decoding video according to the method of any one of claims 1 to 15.

Images