Template-based filtering for intra prediction

Abstract

In one implementation, prediction of a block is filtered using a filter learned on a template. To learn the filter, a first template is generated from a decoded sample set in a region adjacent to the current block, and a second template is generated from a prediction sample set in the same adjacent region, where the prediction sample set is obtained based on an intra prediction mode for predicting the current block. Filter parameters are calculated by minimizing a loss function between the de-coded sample set and the filtered prediction sample set. To reduce computational complexity, intra filtering may be enabled only when an intra prediction mode is from an MPM list, obtained from TIMD or DIMD, or if at least one intra mode in a template region is a neighbor to the current intra prediction mode.

Description

Technical Field

[0001] The present embodiments generally relate to methods and apparatus for intra-frame prediction in video encoding and decoding. Background Art

[0002] In order to achieve high compression efficiency, image and video coding schemes usually use prediction and transformation to balance the spatial and temporal redundancy in video content. Typically, intra-frame or inter-frame prediction is used to exploit intra-frame or inter-frame picture correlation, and then the difference between the original block and the predicted block (usually expressed as a prediction error or prediction residual) is transformed, quantized, and entropy encoded. In order to reconstruct the video, the compressed data is decoded by the inverse process corresponding to entropy coding, quantization, transformation, and prediction. Summary of the invention

[0003] According to one embodiment, a video decoding method is proposed, comprising: obtaining an intra-frame prediction mode of a block to be decoded in a picture; obtaining a prediction block of the block based on the intra-frame prediction mode of the block; obtaining a decoded sample set in an area adjacent to the block; obtaining a prediction sample set in the area adjacent to the block; obtaining one or more filter parameters of a filter based on the decoded sample set and the prediction sample set in the area adjacent to the block; applying the filter to the prediction block of the block to form a filtered prediction block of the block; and decoding the block based on the filtered prediction block of the block.

[0004] According to another embodiment, a video encoding method includes: obtaining an intra-frame prediction mode of a block to be encoded in a picture; obtaining a prediction block of the block based on the intra-frame prediction mode of the block; obtaining a reconstructed sample set in an area adjacent to the block; obtaining a prediction sample set in the area adjacent to the block; obtaining one or more filter parameters of a filter based on the reconstructed sample set and the prediction sample set in the area adjacent to the block; applying the filter to the prediction block of the block to form a filtered prediction block of the block; and encoding the block based on the filtered prediction block of the block.

[0005] According to another embodiment, a device for video decoding is provided, comprising one or more processors and at least one memory coupled to the one or more processors, wherein the one or more processors are configured to: obtain an intra-frame prediction mode of a block to be decoded in a picture; obtain a prediction block of the block based on the intra-frame prediction mode of the block; obtain a decoded sample set in an area adjacent to the block; obtain a prediction sample set in the area adjacent to the block; obtain one or more filter parameters of a filter based on the decoded sample set and the prediction sample set in the area adjacent to the block; apply the filter to the prediction block of the block to form a filtered prediction block of the block; and decode the block based on the filtered prediction block of the block.

[0006] According to another embodiment, a device for video encoding is provided, comprising one or more processors and at least one memory coupled to the one or more processors, wherein the one or more processors are configured to: obtain an intra-frame prediction mode of a block to be encoded in a picture; obtain a prediction block of the block based on the intra-frame prediction mode of the block; obtain a reconstructed sample set in a region adjacent to the block; obtain a prediction sample set in the region adjacent to the block; obtain one or more filter parameters of a filter based on the reconstructed sample set and the prediction sample set in the region adjacent to the block; apply the filter to the prediction block of the block to form a filtered prediction block of the block; and encode the block based on the filtered prediction block of the block.

[0007] One or more embodiments also provide a computer program including instructions, which, when executed by one or more processors, cause one or more processors to perform the encoding method or decoding method according to any embodiment described herein. One or more embodiments of the present embodiment also provide a computer-readable storage medium having stored thereon instructions for performing video encoding or decoding according to the method described herein.

[0008] One or more embodiments further provide a computer-readable storage medium on which video data generated according to the above method is stored. One or more embodiments further provide a method and apparatus for sending or receiving video data generated according to the method described herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 A block diagram of a system is shown in which aspects of the present embodiments may be implemented.

[0010] Figure 2 A block diagram of an embodiment of a video encoder is shown.

[0011] Figure 3A block diagram of an embodiment of a video decoder is shown.

[0012] Figure 4 The 67 core intra prediction modes in VVC and ECM are shown.

[0013] Figure 5 A sample for calculating horizontal and vertical gradients is shown.

[0014] Fig. 6A , 6B and 6C show the template of the current luminance CB to be encoded / decoded and the decoded reference samples of the template.

[0015] Figure 7 The input of the spatial 5-tap components of the filter is shown when the CCCM predicts the current chroma CB to be encoded / decoded from a potentially downsampled version of the reconstructed luma CB concatenated with the current chroma CB.

[0016] Fig. 8A shows the reconstructed luma CB concatenated with the current W×H chroma CB to be encoded / decoded, Figure 8B The downsampled reconstructed luminance CB and luminance reference region are shown, and Figure 8C Its chromaticity reference region is shown.

[0017] Fig.9A and Fig. 9B The templates of decoded reference samples and predicted samples of the current W×H luma CB to be encoded / decoded are shown respectively.

[0018] Fig.10 Two-step template-based filtering of luma intra prediction at the encoder side for a current luma CB predicted in intra mode is shown according to one embodiment.

[0019] Fig.11 Two-step template-based filtering of luma intra prediction at the decoder side for a current luma CB predicted in intra mode is shown according to one embodiment.

[0020] Fig.12 Template shapes used for TIMD and possibly for deriving intra filters are shown.

[0021] Fig.13A and Fig. 13B Overlapping blocks in the template are shown.

[0022] Fig.14 A two-step template-based filtering of luma intra prediction at the decoder side is shown for an intra-predicted current luma CB according to one embodiment.

[0023] Fig.15Two-step template-based filtering of luma intra prediction at the encoding side for a current luma CB intra predicted via TIMD according to one embodiment is shown.

[0024] Fig.16A , 16B and 16C show examples of TIMD with two derived intra-prediction modes.

[0025] Fig.17 Two-step template-based filtering of luma intra prediction at the decoder side is shown for a current luma CB intra predicted via TIMD according to one embodiment. DETAILED DESCRIPTION

[0026] Figure 1 A block diagram of an example of a system in which various aspects and embodiments can be implemented is shown. System 100 can be implemented as a device including various components described below, and is configured to perform one or more aspects described in this application. Examples of such devices include, but are not limited to, various electronic devices, such as personal computers, laptop computers, smart phones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected household appliances and servers. The elements of system 100 can be embodied in a single integrated circuit, multiple ICs and / or discrete components, either alone or in combination. For example, in at least one embodiment, the processing and encoder / decoder elements of system 100 are distributed over multiple ICs and / or discrete components. In various embodiments, system 100 is coupled to other systems or other electronic devices via, for example, a communication bus or by a dedicated input and / or output port communication. In various embodiments, system 100 is configured to implement one or more aspects described in this application.

[0027] The system 100 includes at least one processor 110, which is configured to execute instructions loaded therein, for implementing various aspects described in the present application, for example. The processor 110 may include embedded memory, input and output interfaces, and various other circuits known in the art. The system 100 includes at least one memory 120 (e.g., a volatile memory device and / or a non-volatile memory device). The system 100 includes a storage device 140, which may include a non-volatile memory and / or a volatile memory, including but not limited to an EEPROM, a ROM, a PROM, a RAM, a DRAM, an SRAM, a flash memory, a disk drive, and / or an optical drive. As a non-limiting example, the storage device 140 may include an internal storage device, an attached storage device, and / or a network accessible storage device.

[0028] The system 100 includes an encoder / decoder module 130, which is configured to, for example, process data to provide encoded video or decoded video, and the encoder / decoder module 130 may include its own processor and memory. The encoder / decoder module 130 represents a module that may be included in a device to perform encoding and / or decoding functions. As is known, a device may include one or both of the encoding and decoding modules. In addition, the encoder / decoder module 130 may be implemented as an independent element of the system 100, or may be incorporated into the processor 110 as a combination of hardware and software, as known to those skilled in the art.

[0029] Program code to be loaded onto the processor 110 or the encoder / decoder 130 to perform various aspects described in this application may be stored in the storage device 140 and subsequently loaded onto the memory 120 for execution by the processor 110. According to various embodiments, one or more of the processor 110, the memory 120, the storage device 140, and the encoder / decoder module 130 may store one or more of various items during the execution of the processes described in this application. Such stored items may include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results from the processing of equations, formulas, operations, and operational logic.

[0030] In several embodiments, memory internal to the processor 110 and / or encoder / decoder module 130 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, memory external to the processing device (e.g., the processing device may be the processor 110 or the encoder / decoder module 130) is used for one or more of these functions. The external memory may be a memory 120 and / or a storage device 140, such as a dynamic volatile memory and / or a non-volatile flash memory. In several embodiments, the external non-volatile flash memory is used to store the operating system of the television. In at least one embodiment, a fast external dynamic volatile memory such as RAM is used as working memory for video encoding and decoding operations, such as for MPEG-2, HEVC, or VVC.

[0031] As shown in block 105, input may be provided to the elements of system 100 through various input devices. Such input devices include, but are not limited to, (i) an RF section that receives an RF signal transmitted over the air, such as by a broadcaster, (ii) a composite input terminal, (iii) a USB input terminal, and / or (iv) an HDMI input terminal.

[0032] In various embodiments, the input device of block 105 has associated corresponding input processing elements known in the art. For example, the RF portion may be associated with elements suitable for the following operations: (i) selecting a desired frequency (also referred to as selecting a signal, or band limiting a signal to a frequency band), (ii) down-converting the selected signal, (iii) again band-limiting to a narrower frequency band to select a signal frequency band that may be referred to as a channel in some embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired data packet stream. The RF portion of various embodiments includes one or more elements that perform these functions, such as a frequency selector, a signal selector, a frequency band limiter, a channel selector, a filter, a down-converter, a demodulator, an error corrector, and a demultiplexer. The RF portion may include a tuner that performs various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., an intermediate frequency or a near-baseband frequency) or baseband. In a set-top box embodiment, the RF part and its associated input processing element receive the RF signal transmitted by wired (for example, cable) medium, and filter to the desired frequency band again to perform frequency selection by filtering, down-conversion and filtering. Various embodiments rearrange the order of above-mentioned (and other) elements, remove some in these elements, and / or add other elements that perform similar or different functions. Adding element can include inserting element between existing element, for example, inserting amplifier and analog-to-digital converter. In various embodiments, the RF part includes antenna.

[0033] In addition, the USB and / or HDMI terminals may include corresponding interface processors for connecting the system 100 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing, such as Reed-Solomon error correction, may be implemented, for example, within a separate input processing IC or within the processor 110 as desired. Similarly, various aspects of USB or HDMI interface processing may be implemented within a separate interface IC or within the processor 110 as desired. The demodulated, error-corrected, and demultiplexed streams are provided to various processing elements, including, for example, the processor 110 and the encoder / decoder 130, which operate in conjunction with memory and storage elements to process the data streams as needed for presentation on an output device.

[0034] The various components of the system 100 may be disposed within an integrated housing. Within the integrated housing, the various components may be interconnected and transmit data therebetween using suitable connection means 115, such as an internal bus known in the art, including an I2C bus, wiring, and a printed circuit board.

[0035] The system 100 includes a communication interface 150 that enables communication with other devices through a communication channel 190. The communication interface 150 may include, but is not limited to, a transceiver configured to send and receive data through the communication channel 190. The communication interface 150 may include, but is not limited to, a modem or a network card, and the communication channel 190 may be implemented in, for example, a wired and / or wireless medium.

[0036] In various embodiments, data is streamed to the system 100 using a Wi-Fi network such as IEEE 802.11. The Wi-Fi signals of these embodiments are received through a communication channel 190 and a communication interface 150 suitable for Wi-Fi communications. The communication channel 190 of these embodiments is typically connected to an access point or router that provides access to external networks including the Internet to allow streaming applications and other over-the-top communications. Other embodiments provide streaming data to the system 100 using a set-top box that delivers data through an HDMI connection of the input block 105. Other embodiments provide streaming data to the system 100 using an RF connection of the input block 105.

[0037] The system 100 can provide output signals to various output devices, including a display 165, a speaker 175, and other peripherals 185. In various examples of embodiments, the other peripherals 185 include one or more of a stand-alone DVR, a disc player, a stereo system, a lighting system, and other devices that provide functions based on the output of the system 100. In various embodiments, control signals are transmitted between the system 100 and the display 165, the speaker 175, or other peripherals 185 using signaling such as AV.Link, CEC, or other communication protocols that can achieve device-to-device control with or without user intervention. The output devices can be communicatively coupled to the system 100 via dedicated connections through their respective interfaces 160, 170, and 180. Alternatively, the output devices can be connected to the system 100 using a communication channel 190 via the communication interface 150. The display 165 and the speaker 175 can be integrated into a single unit with other components of the system 100 in an electronic device (e.g., a television). In various embodiments, the display interface 160 includes a display driver, such as a timing controller (T Con) chip.

[0038] For example, if the RF portion of input 105 is part of a separate set-top box, the display 165 and speaker 175 may alternatively be separate from one or more other components. In various embodiments where the display 165 and speaker 175 are external components, the output signals may be provided via dedicated output connections, including, for example, an HDMI port, a USB port, or a COMP output.

[0039] Figure 2 An example video encoder 200 is shown, such as a VVC (Versatile Video Coding) encoder. Figure 2 Also shown is an encoder that improves the VVC standard or an encoder that uses a technique similar to the VVC standard.

[0040] In this application, the terms "reconstruction" and "decoding" can be used interchangeably, the terms "encoded" or "coded" can be used interchangeably, and the terms "image", "picture" and "frame" can be used interchangeably. Usually, but not necessarily, the term "reconstruction" is used on the encoder side, while "decoding" is used on the decoder side.

[0041] Before being encoded, the video sequence may undergo a pre-encoding process (201), for example, applying a color transform to the input color picture (e.g., conversion from RGB 4:4:4 to YCbCr 4:2:0), or performing a remapping of the input image components in order to obtain a signal distribution that is more resilient to compression (e.g., using histogram equalization of one of the color components). Metadata may be associated with the pre-processing and attached to the bitstream.

[0042] In encoder 200, a picture is encoded by encoder elements as described below. The picture to be encoded is partitioned (202) and processed in units such as Cu (coding units). Each unit is encoded using, for example, intra or inter mode. When a unit is encoded in intra mode, it performs intra prediction (260). In inter mode, motion estimation (275) and compensation (270) are performed. The encoder decides (205) which of intra mode or inter mode to use to encode the unit and indicates the intra / inter decision by, for example, a prediction mode flag. For example, the prediction residual is calculated by subtracting (210) the predicted block from the original image block.

[0043] The prediction residual is then transformed (225) and quantized (230). The quantized transform coefficients, as well as motion vectors and other syntax elements, such as picture partition information, are entropy encoded (245) to output a bitstream. The encoder can skip the transform and directly quantize the untransformed residual signal. The encoder can bypass both the transform and quantization, i.e., the residual is directly encoded without applying the transform or quantization process.

[0044] The encoder decodes the coded block to provide a reference for further prediction. The quantized transform coefficients are dequantized (240) and inverse transformed (250) to decode the prediction residual. The decoded prediction residual and the prediction block are combined (255) to reconstruct the image block. An in-loop filter (265) is applied to the reconstructed picture to perform, for example, deblocking / SAO (sample adaptive offset) / ALF (adaptive loop filter) filtering to reduce coding artifacts. The filtered image is stored in a reference picture buffer (280).

[0045] Figure 3 A block diagram of an example video decoder 300 is shown. In the decoder 300, the bitstream is decoded by decoder elements as described below. The video decoder 300 generally performs the same Figure 2 The decoding process is the reverse of the encoding process described in . The encoder 200 also typically performs video decoding as part of encoding the video data.

[0046] In particular, the input to the decoder includes a video bitstream, which may be generated by the video encoder 200. The bitstream is first entropy decoded (330) to obtain transform coefficients, prediction mode, motion vectors, and other encoding information. Picture partition information indicates how the picture is partitioned. Therefore, the decoder can divide (335) the picture according to the decoded picture partition information. The transform coefficients are dequantized (340) and inverse transformed (350) to decode the prediction residual. The decoded prediction residual and the prediction block are combined (355) to reconstruct the image block. The prediction block can be obtained (370) from intra-frame prediction (360) or motion compensated prediction (i.e., inter-frame prediction) (375). An in-loop filter (365) is applied to reconstruct the image. The filtered image is stored in a reference picture buffer (380). Note that for a given picture, the contents of the reference picture buffer 380 on the decoder 300 side are the same as the contents of the reference picture buffer 280 on the encoder 200 side for the same picture.

[0047] The decoded picture may further undergo post-decoding processing (385), such as an inverse color transform (e.g., conversion from YCbCr 4:2:0 to RGB 4:4:4) or inverse remapping, which performs the inverse of the remapping process performed in the pre-encoding process (201). The post-decoding processing may use metadata derived in the pre-encoding process and signaled in the bitstream.

[0048] The present disclosure relates to intra prediction. In the following, we first present the main features of the core intra prediction in the Enhanced Compression Model (ECM), which is currently being studied in JVET (Joint Video Experts Group). Then, some template-based tools within intra prediction in ECM are introduced in detail.

[0049] Inherited from the core 67 intra prediction modes of Versatile Video Coding (VVC)

[0050] To capture arbitrary edge directions present in natural videos, VVC features 65 directional intra prediction modes. In addition, to predict blocks with smoothly changing textures, VVC uses planar and DC modes. These 67 core intra prediction modes are applicable to all block sizes and to both luminance and chrominance intra prediction. Figure 4 The 67 core intra prediction modes are described. For a given luma block and for a given directional intra prediction mode, under certain conditions, Position Dependent Intra Prediction Combining (PDPC) filters the prediction for that block using decoded reference samples located on the opposite side of the decoded reference samples extrapolated via the mode. When a given luma block is predicted by planar or DC, under certain conditions, PDPC filters the prediction using decoded reference samples to the left of the block and decoded reference samples above the block.

[0051] In ECM, the core set of 67 intra prediction modes is inherited from VVC. Refining intra prediction in ECM: The four-tap interpolation for directional intra prediction mode becomes a six-tap interpolation; and PDPC is supplemented with gradient PDPC.

[0052] Intra prediction mode signaling in ECM

[0053] In ECM-4.0, if the intra prediction mode selected to predict the current luma coding block (CB) is neither decoder-side intra mode derivation (DIMD), nor matrix-based intra prediction (MIP) mode or template-based intra mode derivation (TIMD), i.e., it is Figure 4 If one of the 67 intra prediction modes shown is selected, the most probable mode (MPM) list of the CB is used to signal its index. Note that in the previous considerations, BDPCM, template-based intra prediction (TMP), intra block copy (IBC), and palette (Palette) modes are ignored because these tools are exclusively activated for specific video sequences (e.g., screen content).

[0054] In ECM-4.0, the general MPM list is decomposed into six main MPM lists and sixteen auxiliary MPM lists. The general MPM list is constructed by adding candidate intra prediction mode indices sequentially from the one most likely to be the selected intra prediction mode for predicting the current luma CB to the one least likely. Note that there is no redundancy in the general list of MPMs, which means that it cannot contain two identical intra prediction mode indices.

[0055] Intra prediction mode signaling in chroma

[0056] When the intra prediction mode selected to predict the current chrominance Cb pair (i.e., juxtaposed Cb and Cr CB) is signaled, in ECM-4.0, if the direct mode (DM) flag is equal to 1, the four possibilities for the current intra prediction mode index are the index of the planar mode, the index of the horizontal mode, the index of the vertical mode, and the index of the DC mode. To avoid any redundancy, if DM is one of the above four modes, in this set of four modes, the index of the redundant mode is replaced by the index of the vertical diagonal mode. Note that in ECM-4.0, the cross-component linear model (CCLM) collects six different intra prediction modes, denoted as LM, MMLM, MDLM_L, MDLM_T, MMLM_L, and MMLM_T, while in VVC, CCLM only collects three intra prediction modes.

[0057] Template-based intra prediction tool in ECM

[0058] Decoder-side intra mode derivation (DIMD)

[0059] In ECM-4.0, DIMD derives indices of two intra prediction modes from the gradients in the template of the decoded reference samples of the current luma CB to be encoded / decoded, which may be the two best intra prediction modes for predicting the current luma CB in terms of rate distortion. Later, the current luma CB is predicted by blending the two prediction blocks obtained by applying the two derived intra prediction modes with the prediction block obtained by applying the plane. The weights involved in the blending are derived from the gradients in the template.

[0060] More specifically, for the current luma CB, the indices of the two intra prediction modes are derived from the gradients in the template. First, a histogram of oriented gradients (HOG) with 65 bins corresponding to the 65 directional intra prediction modes is initialized to 0. Then, for each decoded reference sample in the middle row or middle column of the template for the three rows of decoded reference samples above the current luma CB and the three columns of decoded reference samples to its left, the horizontal and vertical gradients (G HOR , G VER ), and the corresponding HOG of the index increases by |G HOR |+|G VER |. The indices of the two largest HOG bins are the indices of the two derived intra prediction modes.

[0061] Template-based Intra Mode Derivation (TIMD)

[0062] Like DIMD, TIMD follows a two-step process for a current luma CB to be encoded / decoded: an intra prediction mode index derivation step involving a template of decoded reference samples of the current luma CB, and a step of actually predicting the current luma CB.

[0063] exist Fig. 6A In the example, the current W×H brightness CB (603) is surrounded by its fully available template, which consists of the w t ×H part (600) and the W×h t During the TIMD derivation step, the intra prediction mode to be tested is based on the template's 1+2w t +2W+2h t +2H decodes the set of reference samples (602) to predict the template of the current brightness CB. In ECM-4.0, if W≤8, w t is equal to 2, and otherwise w t equal to 4; if H≤8, h t is equal to 2, and otherwise h t is equal to 4. Figure 6B In the example, the current W×H brightness CB (603) is replaced by only the W×h t The part (601) of the available templates is enclosed. During the TIMD derivation step, the intra prediction mode tested is based on the template's 1+2W+2h t +2H decodes the set of reference samples (602) to predict the template of the current brightness CB. Figure 6C In the example, the current W×H brightness CB (603) is represented by only the left side w t The available templates for the ×H part (600) are bounded. During the TIMD derivation step, the tested intra prediction modes are determined according to the 1+2w t +2W+2H decodes a set of reference samples (602) to predict the template of the current brightness CB.

[0064] for Fig. 6A For a given luma CB (603) in FIG. 5 , the following mode derivation via TIMD is applied in the same way on the encoder and decoder side. For each intra prediction mode in the MPM list of this luma CB, supplemented with a default mode if necessary, the encoder / decoder computes a prediction of the template (600 and 601) of this luma CB from the decoded reference samples of the template (602), and computes the SATD between the prediction and the template of this luma CB. The two intra prediction modes with the smallest SATD are selected as TIMD modes. Note that for TIMD, by Figure 4By inserting a direction between each solid arrow and its adjacent dashed arrow in , the set of directional intra prediction modes is expanded from 65 to 129. This means that the possible intra prediction modes derived via TIMD have collected 131 modes.

[0065] After retaining the two intra prediction modes from the first pass of the test involving the MPM list supplemented with the default mode, for each of these two modes, if the mode is neither planar mode nor DC mode, TIMD also tests its two closest extended direction intra prediction modes according to the predicted SATD. Note that in the above description, it is assumed that the template of the luma CB does not exceed the boundaries of the current frame. In the case that at least part of the template of the luma CB exceeds the boundaries of the current frame, these parts are considered unavailable, such as Figure 6B and Figure 6C shown.

[0066] To predict the current luma CB via TIMD, after applying PDPC, the two predictions of the luma CB via the two TIMD modes generated from the two-pass test are merged with weights. The weights used depend on the predicted SATD of the two TIMD modes.

[0067] Convolutional Cross-Component Model (CCCM).

[0068] In the Exploration Experiments (EE) on top of ECM-4.0, the Convolutional Cross Component Model (CCCM) predicts the current chroma CB to be encoded / decoded by applying a convolution filter to a potentially downsampled version of the reconstructed luma CB collocated with the current chroma CB. When chroma subsampling is used, this downsampling is performed so that the resolution of the downsampled collocated reconstructed luma CB matches the resolution of the chroma grid.

[0069] The CCCM convolutional 7-tap filter consists of a 5-tap plus-shaped spatial component, a nonlinear term, and a bias term. The input to the spatial 5-tap component of the filter consists of the center (C) luma sample concatenated with the current chroma sample to be predicted and its upper / north (N), lower / south (S), left / west (W), and right / east (E) neighbors, as Figure 7 shown.

[0070] The nonlinear term P is expressed as a power of two of the center luminance sample C and is scaled to the sample value range of the content P = (C*C+midVal) >> bit depth Where bitDepth represents the pixel bit depth, and midVal represents the middle value of the bit depth range.

[0071] For example, for 10 bits of content, the calculation is as follows P=(C*C+512)>>10. The bias term B represents a scalar offset between the input and output. B is set to the middle chroma value, e.g., 512 for 10-bit content.

[0072] c0, c1, c2, c3, c4, c5 and c6 are called the seven coefficients of the 7-tap filter, and the current predicted chrominance sample "predChromaVal" is expressed as: predChromaVal=clip(c0C+c1N+c2S+c3E+c4W+c5P+c6B) where "clip" is clipped to the range of valid chroma sample values.

[0073] The filter coefficients c0, c1, c2, c3, c4, c5 and c6 are calculated by minimizing the mean square error (MSE) between predicted chroma samples generated by applying a convolutional 7-tap filter to reconstructed luma samples in the luma reference region (802) and potentially downsampled versions of reconstructed chroma samples in the chroma reference region (803), as shown in Figure 8B and 8C shown.

[0074] Fig. 8A A reconstructed luma CB is shown concatenated with a current WxH chroma CB (801) to be encoded / decoded. Figure 8B A downsampled reconstructed luma CB (800) is shown juxtaposed with the chroma CB and a luma reference region (802) in the case where the chroma format is 4:2:0, i.e., the resolution of each chroma channel is divided by 2 via subsampling before encoding. Figure 8C Its chromaticity reference region (803) is shown.

[0075] The luma reference region (802) consists of six rows / columns of potentially downsampled reconstructed luma samples above and to the left of a potentially downsampled version of the reconstructed luma CB (800) juxtaposed with the current chroma CB. The chroma reference region (803) consists of six rows / columns of reconstructed chroma samples above and to the left of the current chroma CB (801) to be encoded / decoded. Each reference region extends one CB width to the right and one CB height below the CB boundary. Each reference region is adjusted to include only available decoded reference samples. The extension of the area filled with black in Figure 8 is required to support the edge sampling of the plus shaped spatial filter and is filled when in an unavailable area.

[0076] MSE minimization is performed by computing the autocorrelation matrix of the luma input and the cross-correlation vector between the luma input and the chroma output. The autocorrelation matrix is ​​LDL decomposed and back-substitution is used to compute the final filter coefficients. This process follows the calculation of the adaptive linear filtering (ALF) filter coefficients in ECM, except that LDL decomposition is chosen instead of Cholesky decomposition to avoid the use of square root operations. This calculation uses only integer arithmetic.

[0077] Note that either a single model or a multi-model variant of CCCM can be used. The multi-model variant uses two models, one derived for samples above the average luma reference value and the other for the remaining samples. The multi-model CCCM mode can be selected for coding units (CUs) containing at least 128 available decoded reference samples.

[0078] Note also that the "reference region" was chosen to match the standard nomenclature of CCCM. However, the reference region for a given CB is equivalent to the template for that CB.

[0079] Note that CCCM is not included in the intra prediction mode signaling for chroma, because CCCM is currently studied in EE and is therefore not part of ECM yet.

[0080] As mentioned above, DIMD, TIMD and CCCM are new template-based intra prediction modes. This means that DIMD, TIMD and CCCM fully determine the intra prediction model used to predict a given block. Note that TIMD can be used in conjunction with MRL and ISP. Therefore, for a given luma CB, TIMD fully determines the intra prediction model (a combination of intra prediction modes and potentially mode mixing) used to predict the CB, which is optionally replicated on sub-partitions of the CB, and the model optionally uses reference samples decoded from another row.

[0081] In contrast, in the present disclosure, the template of a given luma CB defines filtering that will be applied to predictions of that CB generated using a given intra prediction mode. In other words, the present disclosure proposes a method for smoothing a signal in order to improve the quality of the predictions (similar to the purpose of PDPC). Unlike PDPC, where the decision to turn on / off filtering and the filtering behavior depend only on the size of the current luma CB and the index of the intra prediction mode that predicts the current luma CB, in the present disclosure, the parameterization of the filtering depends on the strength of the decoded reference samples in the template of the current luma CB.

[0082] Filtering for luma intra prediction learned on templates

[0083] For a given W×H luma CB to be encoded / decoded and predicted via a given intra prediction mode, the proposed filtering of the prediction of the CB is decomposed into two steps: learning of the filter and applying the learned filter to the prediction of the CB. It should be noted that the proposed method of intra prediction filtering can also be applied to chroma components.

[0084] Learning Filters

[0085] Fig.9A and Fig. 9B The predicted sample template (902) and the decoded reference sample template (901) of the current WxH luma CB (900) to be encoded / decoded are shown.

[0086] like Fig.9A As shown, the current W×H brightness CB (900) has a first template (901) consisting of n a The row decoded reference sample and the nth one located to the left of the current brightness CB l The template area can be of other shapes or sizes and can generally be any reconstructed area close to the current block. The current brightness CB also has a second template (902) consisting of n columns of decoded reference samples located above the current brightness CB. a The row prediction sample and the nth row to the left of the current brightness CB l The prediction samples in the second template are obtained based on the given intra prediction mode. That is, intra prediction is performed on the template area using the selected intra prediction direction. The reference samples used for prediction can be the same as those of the current block, or can be new reference samples above and to the left of the template.

[0087] On the encoding side, multiple intra-frame prediction modes can be tested to select the intra-frame prediction mode actually used to encode the block. If the proposed filtering is enabled for a potential intra-frame prediction mode, the prediction samples in the second template are obtained based on the potential intra-frame prediction mode. Therefore, when filtering is allowed for multiple potential intra-frame prediction modes, prediction samples are generated for each of these potential intra-frame prediction modes during the intra-frame mode decision process. On the decoder side, the intra-frame prediction mode is explicitly decoded or implicitly decoded from the bitstream, and the prediction samples of the second template are obtained based on the decoded intra-frame prediction mode.

[0088] Given the encoding / decoding partition history leading to the encoding / decoding of the current luma CB, the first template (901) of decoded reference samples can be adjusted such that unavailable decoded reference samples are excluded from (901). The second template (902) of prediction samples can be adjusted in the same way, i.e., unavailable prediction samples are excluded from (902). For example, in Fig.9A, adjust (901) to exclude the n unavailable decoded reference samples at the bottom thereof b ∈[0,H] row (904) and the n unavailable decoded reference samples on its right hand side r ∈[0,W] column (903). Similarly, in Fig. 9B In (902), it is adjusted to exclude the n unavailable prediction samples at its bottom. b ∈[0,H] row (904) and the n unavailable prediction samples on its right hand side r ∈[|0, W|] columns (903). Typically, the first template of the decoded reference samples and the second template of the predicted samples cover the same area in the picture.

[0089] Then, different possible filter parameters are tested to select the filter parameter θ. In particular, the possible filter parameters are applied to a second template of the prediction sample. The difference (e.g., MSE) between the filtered prediction sample and the decoded reference sample is calculated. The filter parameter θ can be learned by minimizing the MSE between the filtered luminance samples in the template (902) of the prediction sample and the decoded reference sample in the template (901) of the decoded reference sample. If necessary, the template (901) of the decoded reference sample and the template (902) of the prediction sample can be padded in the same manner using a p-pixel padding boundary, as shown in FIG. 9. Padding can include filling a padding area with a given value. Alternatively, padding can include copying the value of an available spatially adjacent sample to a given sample to be filled.

[0090] Apply the filter to the prediction of the current brightness CB

[0091] The filtering with the learned parameters θ can be applied to the prediction of the current luma CB, where the prediction is generated using a given intra prediction mode.

[0092] Fig.10 A two-step template-based filtering of luma intra prediction at the encoder side is shown for a current luma CB predicted in intra mode according to one embodiment.

[0093] exist Fig.10 In the figure, the dotted line indicates that on the encoder side, the learning step must be performed before running the filtering of the prediction of the current luma CB, but not necessarily right before. In practice, several processes can be placed between the learning step and filtering the prediction of the current luma CB. For example, the learning step can be done once the template of the decoded reference samples is reconstructed for the current luma CB.

[0094] Specifically, in step 1010, the encoder extracts a template of a decoded reference sample from an adjacent reconstruction area of ​​the current block, and the encoder also obtains a template of a prediction sample of the current block. Then, the current brightness CB can be predicted using a given intra prediction mode. In step 1020, based on the template of the decoded reference sample and the template of the prediction sample, the filter parameter θ is learned.

[0095] As described above, the prediction samples in the second template can be generated by using a given intra prediction mode (the mode used for the current block). Alternatively, in order to save calculations, the prediction samples can be extracted from the template area, that is, the prediction samples of the neighboring blocks (based on the intra prediction mode used to encode / decode the neighboring blocks) are stored and can be directly extracted for use in the second template.

[0096] Then, the prediction of the current block can be Perform (1030) filtering of the current brightness CB to produce a filtered prediction Calculate (1040) the filtered prediction The difference between and the original block (X) is used to obtain the prediction residual. Then, as Figure 2 As shown, the residual can be quantized, transformed and entropy encoded.

[0097] Fig.11 A two-step template-based filtering of luma intra prediction at the decoder side for an intra-predicted current luma CB is shown according to one embodiment.

[0098] Similar to the encoder side, Fig.11 In , the dotted line indicates that at the decoder side, the learning step must be performed before starting the filtering of the prediction of the current luma CB, but not necessarily right before.

[0099] Specifically, at step 1110, the decoder extracts a template of a decoded reference sample from an adjacent decoded area of ​​the current block, and the decoder also obtains a template of a prediction sample of the current block. The template is generated in the same way as the encoder side. Then, the current brightness can be predicted by using the decoded intra prediction mode. At step 1120, based on the template of the decoded reference sample and the template of the prediction sample, the filter parameter θ is learned.

[0100] Then, the prediction of the current block can be Perform (1130) filtering of the current brightness CB to produce a filtered prediction The reconstructed residual of the current brightness CB from the inverse transform With the filtered prediction Combine (1140) to form the reconstructed current brightness CB(). Figure 3 As shown, the reconstructed block may be further filtered.

[0101] The filter is a convolutional 7-tap filter, as in CCCM.

[0102] In this embodiment, the filter may be a convolutional 7-tap filter in CCCM. In this case, the input to the spatial 5-tap component of the filter may include the center (C) luminance prediction sample and its upper / north (N), lower / south (S), left / west (W), and right / east (E) neighbors. In addition, θ={c0, c1, c2, c3, c4, c5, c6}, and the definitions of the nonlinear terms and bias terms may follow those described for CCCM. Any other definitions of nonlinear terms or bias terms may also be applicable.

[0103] The filter is a convolutional 5-tap filter

[0104] In this embodiment, the filter may be a convolutional 5-tap filter. This case may be equivalent to that proposed for CCCM, but with the nonlinear terms and the bias term θ = {c0, c1, c2, c3, c4} removed.

[0105] A filter consisting of a single weight and a single bias

[0106] In this embodiment, the filter may include a single weight a and a single bias b. In this case, the input prediction 10,000 brightness samples and the output filtered brightness samples The relationship between them is:

[0107] The filter is a piecewise linear function

[0108] In this embodiment, the filter may be a The piecewise linear function f of the segment. Note that since the piecewise linear function can be expressed in different forms, its parameter set can take different forms. For example, the piecewise linear function can be expressed by giving the linear segment with index i a slope α i and offset β i and predefine its boundaries (b i , b i+1 ). In this case, θ={(α i , β i )) i∈[0，ρ-1] For example, if ρ = 4, b0 = 0, b1 = 120, b2 = 570, b3 = 1003, b4 = 1024, for a given input predicted luma sample and a filtered version of its output

[0109] Reduced template size

[0110] Larger template sizes correspond to higher complexity on the encoder and decoder side. Moreover, larger templates contain decoded pixels that are far away from the current block. That is why these decoded pixels and the pixels of the current block may exhibit different statistical properties. Therefore, the trained filter coefficients will probably not be optimal for improving the prediction of the current block.

[0111] In template-based intra coding (DIMD and TIMD), the template used to derive the intra prediction mode index contains 2 to 4 rows of decoded pixels. Fig.12 Template shapes for TIMD and possibly for deriving intra filters are shown. The current WxH block (1201) has a template (1202) consisting of two fragments.

[0112] In TIMD, Fig.12 The template shape shown is optimal for capturing the local statistics of the current block. The same template shape can be used for the proposed intra-frame filtering derivation. However, in one example, for better applicability of the filter, the template size should be greater than or equal to 3, which requires access to surrounding pixels.

[0113] The number of filters reduced

[0114] The encoder may need to derive intra filter coefficients for each intra prediction mode tested. Thus, for a given luma block to be encoded, this may require deriving 67 filters on the encoder side, which significantly increases the encoder runtime.

[0115] To reduce this complexity, the following are recommended: 1. Intra-frame filtering is only applicable when an MPM belonging to the main list of MPMs is used to predict the current luma block. That is, at the encoder side, only 6 filters need to be trained to encode a given luma block. This also reduces the signaling overhead because intra-frame filtering is only used when an MPM belonging to the main list of MPMs is selected. 2. Extend point 1 to both the primary and auxiliary MPM lists. For a given luma block, the coefficients of the filter have to be learned for each of the additional 16 intra prediction modes belonging to the auxiliary MPM list. 3. Intra-mode grouping: For example, training of intra-prediction modes in horizontal, vertical, diagonal, and anti-diagonal directions. That is, a given intra-prediction mode is grouped into one of four groups of directional intra-prediction modes, and a single filter is trained for each group. The same idea can be used by grouping adjacent intra-prediction modes into one mode.

[0116] Intra-frame filtering for TIMD / DIMD

[0117] In this embodiment, instead of training the filter for each intra prediction mode, filter training is performed only for TIMD / DIMD mode. That is, when TIMD / DIMD is called to derive the best intra prediction mode index, the corresponding filter is trained. This embodiment has two advantages: 1. Reduced encoder complexity. This is because filter training is performed only for DIMD / TIMD modes instead of for all 67 intra prediction modes 2. Reduced signaling. This is because the DIMD / TIMD flag can be used to indicate that the filter has been used. That is, whenever DIMD / TIMD is signaled, the decoder infers that the intra-frame filter is used.

[0118] Mixing original and filtered predictions

[0119] The filter parameters are trained from pairs of predicted and reconstructed samples near the upper and lower boundaries of the current luma block. This makes the filter more effective in improving the prediction in this area than in the lower bottom area of ​​the current luma block. Therefore, it is recommended to gradually switch from the filtered prediction to the regular prediction. This can be done via a hybrid process of fusing the original prediction (PredOrg) and the filtered prediction (PredFil) so that the final prediction (PredFin) is PredFin(i,j)=M(i,j)*PredFil(i,j)+(1-M(i,j))*PredOrg(i,j) Where M(i, j) is a mixing function that starts at 1 and gradually becomes zero as i and j increase.

[0120] Intra-frame filtering for similar adjacent intra-frame prediction modes

[0121] In this embodiment, intra filtering is applied only if, for at least one block overlapping with the template of the current luma block to be encoded / decoded, the intra prediction selected mode for predicting this block (the index of which is signaled to the decoder side in the bitstream) is a neighbor of the selected intra prediction mode for predicting the current luma block. A block is considered to overlap with a template if some or all samples of the block are included in the template area. For example, if a block B selected to predict a block overlapping with the template of the current luma block is i Index The intra prediction mode is the index of the luma block selected to predict the current luma block Neighbors of the intra prediction mode, then there may be a function that returns true For example, is the circular distance.

[0122] exist Fig.13A and Fig. 13BIn the example, the prediction sample template (1302) and the decoding reference sample template (1301) of the current W×H luminance CB (1300) to be encoded / decoded include three blocks B0, B1 and B2. Each block B i The index it chooses The intra prediction mode (whose index is signaled to the decoder side in the bitstream) is associated with the template of the current luma CB. For readability, the padding samples of the template of the current luma CB are not shown.

[0123] Fig.14 A two-step template-based filtering of luma intra prediction at the decoder side is shown for an intra-predicted current luma CB according to this embodiment. represents the reconstructed residual of the current luminance CB from the inverse transform. Represents the reconstructed current brightness CB.

[0124] For the index used to predict the current brightness CB For a given intra prediction mode, if and and If false (1410) is returned, no learning of the filter coefficients is performed and the prediction of the current brightness CB is not filtered. Fig.14 As shown, the prediction of the current block Reconstructed residual with the current block Combine (1460) to form the reconstructed current brightness

[0125] Otherwise, if or Return true (1410), then in step 1420, the decoder extracts the template of the decoded reference samples from the adjacent decoded area of ​​the current block, and the decoder also obtains the template of the prediction samples of the current block. For each block B overlapping with the template of the current brightness CB i If the index it chooses is The intra prediction mode is the index selected to predict the current brightness CB The neighbor of the intra prediction mode, then the template area and B i The internal prediction samples and decoded reference samples are included in the extraction (1425). Otherwise, B is excluded from the extraction. i All prediction samples and decoded reference samples within (1425).

[0126] In step 1430, filter parameters θ are learned based on the template of the decoded reference sample and the template of the predicted sample. Then, the prediction of the current block can be Perform (1440) filtering of the current brightness CB to produce a filtered prediction The reconstructed residual of the current brightness CB With the filtered prediction Combine (1450) to form the reconstructed current brightness like Figure 3 As shown, the reconstructed block may be further filtered.

[0127] Described relative to the decoder side Fig.14 On the encoder side, steps similar to 1410, 1420, and 1430 may be performed to adjust the template region and learn the filter parameters.

[0128] According to one embodiment, if a given intra prediction mode is a core intra prediction mode (i.e., a planar or DC or directional intra prediction mode) and MRL and ISP are not used, the proposed filtering may be allowed for the prediction of the current luma CB generated via the given intra prediction mode. Specifically, according to this embodiment, if the given mode is a MIP mode, the proposed filtering is not allowed for the prediction of the current luma CB generated via the given intra prediction mode.

[0129] According to one embodiment, if a given intra prediction mode is a core intra prediction mode (i.e., a planar or DC or directional intra prediction mode) and MRL is not used, the proposed filtering of the prediction of the current luma CB generated via the given intra prediction mode may be allowed. In this case, if ISP is used, the process of learning filter parameters and applying the learned filter to the prediction of the current luma block may be repeated for each luma transform block (TB) within the current luma CB.

[0130] In this case, the method is similar to Fig.14 except that this method is performed at the TB level.

[0131] This embodiment features two main advantages. First, on the encoder side, for a given luminance block to be encoded, the learning of the filter coefficients is performed for each intra prediction mode belonging to a small subset of all tested intra prediction modes. This reduces the complexity of the proposed method on the encoder side. Second, the proposed filtering does not cause additional signaling, since the intra prediction mode chosen to predict a block overlapping with the template of the current luminance block to be encoded / decoded defines whether the proposed filtering is applicable.

[0132] Fig.15 A method for filtering the prediction of the current luma CB using a learned filter at the encoder side according to an embodiment is shown. This embodiment can be applied to the case where a template of prediction samples is obtained by applying an intra prediction tool selected to predict the current luma CB on the template area, and may be particularly relevant given its small complexity overhead.

[0133] In this embodiment, for the current brightness CB that selects TIMD for intra-frame prediction, after completing the TIMD derivation step, in the filter learning step, the index of the derived TIMD mode and the derived TIMD weight can be used to generate a template of the prediction sample of the current brightness CB.

[0134] Specifically, Fig.15 This embodiment is depicted where the current WxH luma CB selects the TIMD for intra prediction at the encoder side. Fig.15 Assume that, in TIMD, as shown in the example of FIG16 , two intra prediction modes are derived, as in ECM-4.0. The TIMD derivation step returns the first derived TIMD mode of index i0, the second derived TIMD mode of index i1, and their respective weights W0 and w1. As in ECM-4.0, if the allowed directional intra prediction modes of TIMD correspond to a set of directional intra prediction modes that is twice as dense as that of VVC, If weight normalization in ECM-4.0 is used, w0 and w1 are positive and w0+w1=64.

[0135] Return to reference Fig.15 , at 1510, the decoded reference samples (1620) are used by the first derived TIMD pattern with index i0 to fill the template T0 (1600) of the prediction samples of the current W×H luma CB (1610). This can be done by extrapolating the decoded reference samples to T0 following the direction of the pattern with index i0. At 1520, the decoded reference samples (1620) are used by the second derived TIMD pattern with index i1 to fill the template T1 (1601) of the prediction samples of the current W×H luma CB (1610). This can be done by extrapolating the decoded reference samples to T1 following the direction of the pattern with index i1. At 1530, T0 with weight W0 and T1 with weight w1 are mixed to produce the final template T1 of the prediction samples of the current W×H luma CB (1610). f (1602). For example, the mixture can be given by the following equation: T f (x, y)=(w0T0(x, y)+w1T1(x, y)+32)>>6 Where (x, y) represents T f The internal coordinates of the current prediction sample.

[0136] At 1550, a template (1630) for decoding reference samples of the current luminance CB(1610) is extracted. At 1540, filter parameters θ are learned based on the template (1602) of the prediction samples and the template (1630) of the decoding reference samples. At 1560, a prediction P0 of X is calculated via a first derived TIMD mode with index i0, and a prediction P1 of X is calculated via a second derived TIMD mode with index i1. At 1570, a mixture of P0 with weight W0 and P1 with weight w1 returns a prediction of the current luminance CBX For example, the mixture can be written as where (x, y) represents the coordinates of the current prediction sample within the prediction block At 1580, is filtered using the learned parameter θ to produce a filtered prediction of the current luminance CB Finally, is subtracted from X (1590) This provides the residual R to be further encoded

[0137] Fig.17 Illustrated is a method for filtering a prediction of the current luminance CB using a learned filter at the decoder side, corresponding to Fig.15 the method of the encoder shown in Fig.15 , Fig.17 it is assumed that in TIMD, two intra prediction modes are derived Fig.17 Steps 1710, 1720, 1730, 1740, 1750, 1760, 1770, and 1780 in Fig.15 are respectively the same as steps 1510, 1520, 1530, 1540, 1550, 1560, 1570, and 1580 in Finally, is added to the reconstructed residual (1790), which gives the reconstruction of the current luminance CB

[0138] In Fig.15 and Fig.17 In this embodiment shown, during the filter learning step, the template (1602) of the prediction samples around the current W×H luminance CB may include two template parts used during the TIMD derivation step. For example, during the TIMD derivation step, the template region for calculating the SATD of the prediction for each tested intra prediction mode may consist of a W×h t template part located above the current W×H luminance CB and a w t ×H template part located to the left of the current W×H luminance CB. During the filter learning step, the template (1602) of the prediction samples may include a W×h tTemplate part, w located to the left of the current W×H brightness CB t ×H template part and the w located at the upper left of the current W×H brightness CB t ×h t part. In this way, the prediction of the two template parts via each of the two derived TIMD modes during the TIMD derivation step can be reused during the filter learning step. This saves computation time. Another advantage of this design is that exactly the same decoded reference samples (1620) can be used in the TIMD derivation step and the filter learning step. Therefore, no additional storage buffer is required to store different decoded reference samples for the TIMD derivation step and the filter learning step.

[0139] exist Fig.15 and Fig.17 In TIMD, as in ECM-4.0, two intra prediction modes are derived. However, this method can be applied to derive And the case of intra-frame prediction mode where n>2.

[0140] Various methods are described herein, and each method includes one or more steps or actions for realizing the method.Unless the correct operation of the method requires a specific order of steps or actions, otherwise the order and / or use of specific steps and / or actions can be modified or combined.In addition, terms such as "first", "second" etc. can be used to modify elements, components, steps, operations, etc. in various embodiments, for example, "first decoding" and "second decoding".Unless specifically required, the use of these terms does not mean the sequencing of the operation to modification.Therefore, in this example, the first decoding does not need to be performed before the second decoding, and can occur in, for example, before, during or in a time period overlapping with the second decoding.

[0141] The various methods and other aspects described in this application can be used to modify Figure 2 and Figure 3 Modules of the video encoder 200 and decoder 300 shown, such as the intra prediction module (260, 360). In addition, the present aspects are not limited to ECM, VVC or HEVC, and can be applied to, for example, other standards and recommendations, and extensions of any such standards and recommendations. Unless otherwise specified or technically excluded, the aspects described in this application can be used alone or in combination.

[0142] Various numerical values ​​are used in this application. The specific values ​​are for illustrative purposes, and the described aspects are not limited to these specific values.

[0143] Various implementations involve decoding. "Decoding" as used in this application may include, for example, all or part of the processing performed on a received coded sequence to produce a final output suitable for display. In various embodiments, such a process includes one or more processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. Based on the context of the specific description, whether the phrase "decoding process" is intended to refer specifically to a subset of operations or to a broader decoding process will be clear and is considered to be well understood by those skilled in the art.

[0144] Various implementations involve encoding.In a manner similar to the discussion above regarding "decoding", "encoding" as used in this application may include, for example, all or part of the processing performed on an input video sequence to produce an encoded bitstream.

[0145] The implementations and aspects described herein can be implemented in, for example, methods or processes, devices, software programs, data streams, or signals. Even if only discussed in the context of a single implementation form (e.g., discussed only as a method), the implementation of the features discussed can also be implemented in other forms (e.g., devices or programs). Devices can be implemented in, for example, appropriate hardware, software, and firmware. The method can be implemented in, for example, a device, such as a processor, which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. The processor also includes a communication device, such as a computer, a cellular phone, a portable / personal digital assistant ("PDA"), and other devices that facilitate information communication between end users.

[0146] References to "one embodiment" or "an embodiment" or "one implementation" or "implementation" and other variations thereof mean that a particular feature, structure, characteristic, etc. described in connection with the embodiment is included in at least one embodiment. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" or "in an implementation" or "in an implementation", and any other variations appearing in various places throughout the application, do not necessarily all refer to the same embodiment.

[0147] Additionally, this application may refer to "determining" various pieces of information. Determining information may include, for example, one or more of estimating information, calculating information, predicting information, or retrieving information from a memory.

[0148] Additionally, the application may refer to "accessing" various pieces of information. Accessing information may include, for example, one or more of receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0149] Furthermore, the application may refer to "receiving" various pieces of information. Like "accessing," receiving is a broad term. Receiving information may include, for example, one or more of accessing information or retrieving information (e.g., from a memory). Furthermore, during operations such as storing information, processing information, sending information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information, "receiving" is often involved in one way or another.

[0150] It should be understood that, for example, in the case of "A / B," "A and / or B," and "at least one of A and B," any of the following uses of " / ," "and / or," and "at least one of" are intended to include selecting only the first listed option (A), or only the second listed option (B), or both options (A and B). As another example, in the case of "A, B, and / or C" and "at least one of A, B, and C," such wording is intended to include selecting only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A and B and C). As will be apparent to one of ordinary skill in this and related arts, this can be extended to many of the items listed.

[0151] In addition, as used herein, the word "signaling" refers in particular to indicating something to a corresponding decoder. For example, in some embodiments, the encoder signals a quantization matrix for dequantization. Thus, in one embodiment, the same parameters are used on the encoder side and the decoder side. Thus, for example, the encoder can send (explicit signaling) specific parameters to the decoder so that the decoder can use the same specific parameters. On the contrary, if the decoder already has specific parameters as well as other parameters, signaling can be used without sending (implicit signaling) to simply allow the decoder to know and select specific parameters. Bit savings are achieved in various embodiments by avoiding the transmission of any actual function. It should be understood that signaling can be implemented in a variety of ways. For example, in various embodiments, one or more syntax elements, flags, etc. are used to signal information to the corresponding decoder. Although the verb form of the word "signal" is mentioned above, the word "signal" can also be used as a noun here.

[0152] It will be apparent to one of ordinary skill in the art that embodiments may generate a variety of signals formatted to carry information that may be stored or transmitted, for example. The information may include, for example, instructions for executing a method, or data generated by one of the described embodiments. For example, a signal may be formatted to carry a bitstream of the described embodiments. Such a signal may be formatted as, for example, an electromagnetic wave (e.g., using a radio frequency portion of a spectrum) or a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is well known, a signal may be transmitted over a variety of different wired or wireless links. The signal may be stored on a processor readable medium.

Claims

1. A method for video decoding, comprising: Obtaining an intra prediction mode for a block to be decoded in a picture; Obtaining a prediction block for the block based on the intra prediction mode of the block; Obtaining a decoded sample set in a region adjacent to the block; Obtaining a prediction sample set in the area adjacent to the block; obtaining one or more filter parameters of a filter based on the decoded sample set and the predicted sample set in the region adjacent to the block; applying the filter to the prediction block for the block to form a filtered prediction block for the block; as well as The block is decoded based on the filtered prediction block for the block.

2. A video encoding method, comprising: Obtaining an intra prediction mode for a block to be encoded in a picture; Obtaining a prediction block for the block based on the intra prediction mode of the block; Obtaining a decoded sample set in a region adjacent to the block; Obtaining a prediction sample set in the area adjacent to the block; obtaining one or more filter parameters of a filter based on the decoded sample set and the predicted sample set in the region adjacent to the block; applying the filter to the prediction block for the block to form a filtered prediction block for the block; as well as The block is encoded based on the filtered prediction block for the block. 3 . The method according to claim 1 , wherein the set of prediction samples is obtained based on the intra prediction mode of the block.

4. The method according to any one of claims 1 to 3, wherein: Based on the template-based intra mode derivation, at least two intra prediction modes are obtained for the block, wherein at least two corresponding predictors are formed for the block, corresponding to the at least two intra prediction modes of the block and mixed to form the predicted block of the block, and wherein at least another two corresponding predictors are formed for the region adjacent to the block, corresponding to the at least two intra prediction modes of the block and mixed to form the set of prediction samples in the region adjacent to the block.

5. The method according to claim 4, wherein: At least part of the prediction in the template based intra mode derivation is reused when forming the at least two further predictors for the region neighboring the block.

6. The method according to any one of claims 1 to 5, wherein: The one or more filter parameters are obtained by minimizing a loss function between the decoded sample set and a filtered version of the prediction sample set in the region neighboring the block.

7. The method according to any one of claims 1 to 6, wherein the decoded sample set comprises samples above the block and samples to the left of the block.

8. The method according to any one of claims 1-7, wherein the filter is a convolution filter.

9. The method of any one of claims 1-8, wherein the one or more filter parameters include a single weight and a single bias.

10. The method according to any one of claims 1-8, wherein the filter corresponds to a piecewise linear filter.

11. The method according to any one of claims 1 to 9, wherein: The prediction block is filtered in response to the intra prediction mode belonging to a most probable mode (MPM) list.

12. The method of claim 11, wherein the MPM list is a master list of MPMs.

13. The method of claim 11, wherein the MPM list comprises a primary list of MPMs and an auxiliary list of MPMs.

14. The method according to any one of claims 1 to 13, further comprising: An intra-prediction mode group to which the intra-prediction mode of the block belongs is determined, wherein a single filter is obtained for the intra-prediction mode group.

15. The method according to any one of claims 1-14, wherein the filter is applied only when TIMD (Template-based Intra Mode Derivation) is invoked.

16. The method according to any one of claims 1-14, wherein the filter is applied only when DIMD (Decoder-side Intra Mode Derivation) is invoked.

17. The method of any one of claims 1-15, further comprising mixing the prediction block and the filtered prediction block.

18. The method according to any one of claims 1 to 17, further comprising: obtaining an intra prediction mode selected to predict a portion of the region adjacent to the block, wherein only the selected intra prediction mode in response to the portion being a neighbor of the intra prediction mode of the block, the portion is included in the region adjacent to the block.

19. The method according to any one of claims 1-18, wherein the block is a coding block or a transform block.

20. An apparatus for video decoding, comprising one or more processors and at least one memory, wherein the one or more processors are configured to: Obtaining an intra prediction mode for a block to be decoded in a picture; Obtaining a prediction block for the block based on the intra prediction mode of the block; Obtaining a decoded sample set in a region adjacent to the block; Obtaining a prediction sample set in the area adjacent to the block; obtaining one or more filter parameters of a filter based on the decoded sample set and the predicted sample set in the region adjacent to the block; applying the filter to the prediction block for the block to form a filtered prediction block for the block; as well as The block is decoded based on the filtered prediction block for the block.

21. An apparatus for video encoding, comprising one or more processors and at least one memory, wherein the one or more processors are configured to: Obtaining an intra prediction mode for a block to be encoded in a picture; Obtaining a prediction block for the block based on the intra prediction mode of the block; Obtaining a decoded sample set in a region adjacent to the block; Obtaining a prediction sample set in the area adjacent to the block; obtaining one or more filter parameters of a filter based on the decoded sample set and the predicted sample set in the region adjacent to the block; applying the filter to the prediction block for the block to form a filtered prediction block for the block; as well as The block is encoded based on the filtered prediction block for the block.

22. The apparatus according to claim 20 or 21, wherein the set of prediction samples is obtained based on the intra prediction mode of the block.

23. The device according to any one of claims 20 to 22, wherein: Based on the template-based intra mode derivation, at least two intra prediction modes are obtained for the block, wherein at least two corresponding predictors are formed for the block, corresponding to the at least two intra prediction modes of the block and mixed to form the predicted block of the block, and wherein at least another two corresponding predictors are formed for the region adjacent to the block, corresponding to the at least two intra prediction modes of the block and mixed to form the set of prediction samples in the region adjacent to the block.

24. The device according to claim 23, wherein: At least part of the prediction in the template based intra mode derivation is reused when forming the at least two further prediction values ​​for the region neighbouring the block.

25. The device according to any one of claims 20 to 24, wherein: The one or more filter parameters are obtained by minimizing a loss function between the set of decoded samples and a filtered version of the set of prediction samples in the region adjacent to the block.

26. The device according to any one of claims 20 to 25, wherein: The one or more processors are further configured to determine an intra-prediction mode group to which the intra-prediction mode of the block belongs, wherein a single filter is obtained for the intra-prediction mode group.

27. The apparatus according to any one of claims 20-26, wherein the filter is applied only when TIMD (Template-based Intra Mode Derivation) is invoked.

28. The device according to any one of claims 20 to 26, wherein: The filter is only applied when DIMD (decoder side intra mode derivation) is invoked.

29. A signal comprising video data formed by performing the method of any one of claims 1-19.

30. A computer-readable storage medium having stored thereon instructions for encoding or decoding a video according to the method according to any one of claims 1-19.

Images