Template-based intra mode derivation (TIMD) and decoder-side intra mode derivation (DIMD)

Abstract

At least one method and apparatus are presented for efficiently encoding or decoding video using TIMD or DIMD. For example, the method can include determining that an intra mode is to be derived for predicting a current block, deriving a plurality of intra modes based on a template associated with the current block, obtaining a plurality of predictions based on the plurality of intra modes, blending the plurality of predictions to obtain a prediction block for the current block, and encoding / decoding the block based on the prediction block. In variations, the plurality of intra modes can include more than 2, e.g., 3 to 5, angular modes. In another variation, the plurality of intra modes includes a first mode, a second mode, and at least one additional BV mode. In yet other variations, a reference line used for prediction is adapted.

Description

Cross-reference to related applications

[0001] This application claims the benefit of European Patent Application No. 23307109.1, filed on November 30, 2023, which is incorporated herein by reference in its entirety. Technical Field

[0002] This embodiment generally relates to a method and apparatus for mixing intra-prediction modes derived in template-based intra-mode derivation (TIMD) or decoder-side intra-mode derivation (DIMD) during video encoding and decoding. Background Technology

[0003] To achieve high compression efficiency, image and video coding schemes typically employ prediction and transform to take advantage of spatial and temporal redundancy in the video content. Generally, intra-frame or inter-frame prediction is used to fully utilize intra-frame or inter-frame image correlations, and then the difference between the original block and the predicted block (often called the prediction error or prediction residual) is transformed, quantized, and entropy-coded. To reconstruct the video, the compressed data is decoded through the inverse process corresponding to entropy coding, quantization, transform, and prediction. Summary of the Invention

[0004] According to a first aspect, a video encoding / decoding method is disclosed, comprising: determining an intra-frame mode to be derived for predicting a current block; deriving multiple intra-frame modes based on a template associated with the current block; obtaining multiple predictions based on the multiple intra-frame modes; mixing the multiple predictions to obtain a prediction block of the current block; and encoding / decoding the block based on the prediction block.

[0005] According to one specific feature, intra-frame mode derivation is Template Intra-Frame Mode Derivation (TIMD). According to another specific feature, the multiple intra-frame modes include more than two angular modes. According to yet another variation, one of the additional angular modes corresponds to a decoder-side intra-frame mode derivation (DIMD) mode.

[0006] According to another specific feature, the plurality of intra-frame modes include a first mode, a second mode, a third non-angular mode, and at least one additional non-angular mode obtained from a neighboring IBC / IntraTMP block.

[0007] Based on another specific feature, intra-frame mode derivation is a decoder-side intra-frame mode derivation (DIMD) mode.

[0008] Based on another specific feature, a reference line is selected above the reference line chosen for the previous mode to be used to obtain a prediction using a given intra-frame mode within any intra-frame mode derivation (i.e., DIMD or TIMD).

[0009] One or more embodiments also provide apparatus for encoding or decoding according to the methods described herein.

[0010] One or more embodiments also provide a computer program including instructions that, when executed by one or more processors, cause the one or more processors to perform the encoding or decoding methods described in any embodiment herein. One or more embodiments of this disclosure also provide a computer-readable storage medium storing instructions thereon for encoding or decoding video according to the methods described herein.

[0011] One or more embodiments also provide a computer-readable storage medium storing video data generated according to the method described herein. One or more embodiments also provide methods and apparatus for transmitting or receiving video data generated according to the method described herein. Attached Figure Description

[0012] Figure 1 A block diagram of a system that can implement several aspects of this embodiment is shown.

[0013] Figure 2 A block diagram of an embodiment of a video encoder is shown.

[0014] Figure 3 A block diagram of a video decoder embodiment is shown.

[0015] Figure 4 illustrates the TIMD process according to existing technology.

[0016] Figure 5 The TIMD process for the number of extended angle fusion modes according to the first embodiment is illustrated.

[0017] Figure 6a and Figure 6b The TIMD process for the number of conditionally extended angle fusion modes according to the first embodiment is illustrated.

[0018] Figure 7 The TIMD process according to the second embodiment is illustrated by expanding the number of non-angle fusion modes by including other block vectors from neighboring blocks encoded with intra-block copy.

[0019] Figure 8 Template-based intra-frame mode derivation (TIMD) according to an embodiment is shown.

[0020] Figure 9 The reference line selection for TIMD mode or DIMD mode according to an embodiment is shown. Detailed Implementation

[0021] Figure 1A block diagram illustrating a system example that can implement various aspects and embodiments is shown. System 100 can be implemented as a device including a variety of components and configured to perform one or more aspects described in this application. Examples of such devices include, but are not limited to, a variety of electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, networked home appliances, and servers. Elements of system 100 can be implemented individually or in combination as a single integrated circuit, multiple ICs, and / or discrete components. For example, in at least one embodiment, the processing and encoder / decoder elements of system 100 are distributed across multiple ICs and / or discrete components. In various embodiments, system 100 is communicatively coupled to other systems or other electronic devices, for example, via a communication bus or via dedicated input and / or output ports. In various embodiments, system 100 is configured to implement one or more aspects described in this application.

[0022] System 100 includes at least one processor 110 configured to execute instructions loaded thereon to implement various aspects, such as those described in this application. Processor 110 may include embedded memory, input / output interfaces, and various other circuitry known in the art. System 100 includes at least one memory 120 (e.g., a volatile memory device and / or a non-volatile memory device). System 100 includes a storage device 140, which may include non-volatile memory and / or volatile memory, including but not limited to EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash memory, disk drives, and / or optical disk drives. Storage device 140 may include internal storage devices, attached storage devices, and / or network-accessible storage devices, as non-limiting examples.

[0023] System 100 includes an encoder / decoder module 130 configured to, for example, process data to provide encoded or decoded video, and the encoder / decoder module 130 may include its own processor and memory. The encoder / decoder module 130 represents a module that can be included in a device to perform encoding and / or decoding functions. As is known, a device may include one or both of an encoding module and a decoding module. Alternatively, the encoder / decoder module 130 may be implemented as a separate element of system 100 or may be incorporated into processor 110 as a combination of hardware and software known to those skilled in the art.

[0024] Program code to be loaded onto processor 110 or encoder / decoder 130 to execute the various aspects described in this application may be stored in storage device 140 and subsequently loaded into memory 120 for execution by processor 110. According to various embodiments, one or more of processor 110, memory 120, storage device 140, and encoder / decoder module 130 may store one or more items of various kinds during the execution of the processes described in this application. These stored items may include, but are not limited to, input video, decoded video or portions thereof, bitstreams, matrices, variables, and intermediate or final results of processing from formulas, operations, and operational logic.

[0025] In several embodiments, memory within processor 110 and / or encoder / decoder module 130 is used to store instructions and provide working memory for processing during encoding or decoding. However, in other embodiments, external memory (e.g., processor 110 or encoder / decoder module 130) is used for one or more of these functions. External memory may be memory 120 and / or storage device 140, such as volatile memory and / or non-volatile flash memory. In several embodiments, external non-volatile flash memory is used to store the television's operating system. In at least one embodiment, fast external volatile memory (such as RAM) is used as working memory for video encoding and decoding operations, such as for MPEG-2, HEVC, or VVC.

[0026] Inputs to the components of system 100 can be provided by a variety of input devices as shown in box 105. These input devices include, but are not limited to: (i) an RF section that receives, for example, RF signals transmitted over the air by a broadcaster; (ii) a composite input terminal; (iii) a USB input terminal; and / or (iv) an HDMI input terminal.

[0027] In various embodiments, the input device of block 105 has associated corresponding input processing elements known in the art. For example, the RF section may be associated with elements suitable for: (i) selecting a desired frequency (also known as selecting a signal, or band-limiting a signal to a frequency band); (ii) down-converting the selected signal; (iii) band-limiting it again to a narrower frequency band to select, for example, a signal band (which may be referred to as a channel in some embodiments); (iv) demodulating the down-converted and band-limited signal; (v) performing error correction; and (vi) demultiplexing to select a desired data packet stream. The RF section in various embodiments includes one or more elements (e.g., frequency selectors, signal selectors, band limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers) to perform these functions. The RF section may include a tuner that performs multiple of these functions (including, for example, down-converting a received signal to a lower frequency (e.g., intermediate frequency or near-baseband frequency) or baseband). In one set-top box embodiment, the RF section and its associated input processing elements receive RF signals transmitted via a wired (e.g., cable) medium and perform frequency selection by filtering, down-converting, and re-filtering to a desired frequency band. Various embodiments rearrange the order of the above (and other) components, remove some of these components, and / or add other components that perform similar or different functions. Adding components may include inserting components between existing components, such as inserting amplifiers and analog-to-digital converters. In various embodiments, the RF section includes an antenna.

[0028] Additionally, USB and / or HDMI terminals may include corresponding interface processors for connecting system 100 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Reed-Solomon error correction) may be implemented, for example, within a separate input processing IC or within processor 110 as needed. Similarly, aspects of USB or HDMI interface processing may be implemented, as needed, within a separate interface IC or within processor 110. The demodulated, error-corrected, and demultiplexed stream is provided to various processing elements, including, for example, processor 110 and encoder / decoder 130, which operate in combination with memory and storage elements to process the data stream as needed for presentation on the output device.

[0029] Various components of system 100 can be housed within an integrated housing. Within the integrated housing, various components can be interconnected and transmit data therebetween using suitable connection means 115 (e.g., internal buses known in the art, including I2C buses, wiring, and printed circuit boards).

[0030] System 100 includes a communication interface 150 that enables communication with other devices via a communication channel 190. The communication interface 150 may include, but is not limited to, a transceiver configured to send and receive data via the communication channel 190. The communication interface 150 may include, but is not limited to, a modem or network interface card (NIC), and the communication channel 190 may be implemented, for example, within a wired and / or wireless medium.

[0031] In various embodiments, a Wi-Fi network (e.g., IEEE 802.11) is used to stream data to system 100. In these embodiments, the Wi-Fi signal is received via a communication channel 190 and a communication interface 150 adapted for Wi-Fi communication. The communication channel 190 in these embodiments is typically connected to an access point or router that provides access to external networks, including the Internet, to allow streaming applications and other over-the-top services to communicate. Other embodiments use a set-top box to provide streaming data to system 100, which transmits data via an HDMI connection to input box 105. Still other embodiments use an RF connection to input box 105 to provide streaming data to system 100.

[0032] System 100 can provide output signals to a variety of output devices, including a display 165, a speaker 175, and other peripheral devices 185. In various embodiment examples, other peripheral devices 185 include one or more of a standalone DVR, an optical disc player, a stereo system, a lighting system, and other devices that provide functionality based on the output of system 100. In various embodiments, signaling is used to transmit control signals between system 100 and the display 165, speaker 175, or other peripheral devices 185 using signaling of communication protocols such as AV.Link, CEC, or others that implement device-to-device control (with or without user intervention). Output devices can be communicatively coupled to system 100 via dedicated connections through corresponding interfaces 160, 170, and 180. Alternatively, output devices can be connected to system 100 via communication interface 150 using communication channel 190. The display 165 and speaker 175 can be integrated into a single unit within an electronic device (e.g., a television set) along with other components of system 100. In various embodiments, display interface 160 includes a display driver, such as a timing controller (TCon) chip.

[0033] Alternatively, the display 165 and speaker 175 can be separated from one or more other components, for example, if the RF portion of input 105 is part of a separate set-top box. In various embodiments where the display 165 and speaker 175 are external components, the output signal can be provided via a dedicated output connection including, for example, an HDMI port, a USB port, or a COMP output.

[0034] Figure 2 An exemplary video encoder 200, such as a VVC (Various Video Coding) encoder, is shown. Figure 2 It may also show encoders that improve upon the VVC standard or encoders that employ VVC-like technology.

[0035] In this application, the terms "reconstruction" and "decoding" are used interchangeably, as are the terms "encoding" and "coded," and the terms "image," "picture," and "frame." Typically, but not necessarily, the term "reconstruction" is used on the encoder side, while "decoding" is used on the decoder side.

[0036] Before encoding, the video sequence may undergo pre-coding processes (201), such as applying color transformations to the input color images (e.g., a conversion from RGB 4:4:4 to YCbCr 4:2:0), or performing remapping of the input image components to obtain a signal distribution that is more robust to compression (e.g., using histogram equalization on one of the color components). Metadata may be associated with pre-processing and appended to the bitstream.

[0037] In encoder 200, the image is encoded by encoder elements as described below. The image to be encoded is divided (202) and processed in units, for example, CUs. Each unit is encoded using, for example, an intra-frame mode or an inter-frame mode. When a unit is encoded in intra-frame mode, intra-frame prediction (260) is performed. In inter-frame mode, motion estimation (275) and compensation (270) are performed. The encoder determines (205) which of the intra-frame mode or inter-frame mode to use to encode the unit, and indicates the intra-frame / inter-frame decision, for example, by a prediction mode flag. After prediction, prediction enhancement (285) is applied to the predicted block. For example, the prediction residual is calculated by subtracting (210) the predicted block from the original image block.

[0038] The predicted residual is then transformed (225) and quantized (230). The quantized transform coefficients, along with the motion vector and other syntax elements, are entropy encoded (245) to output a bitstream. The encoder can skip the transform and apply quantization directly to the untransformed residual signal. The encoder can bypass both the transform and quantization, i.e., encode the residual directly without applying the transform or quantization process.

[0039] The encoder decodes the encoded blocks to provide a reference for further prediction. The quantized transform coefficients are dequantized (240) and inverse transformed (250) to decode the prediction residuals. The image blocks are reconstructed by combining (255) the decoded prediction residuals and the predicted blocks. A loop filter (265) is applied to the reconstructed image to perform, for example, deblocking / SAO (sample adaptive offset) filtering to reduce coding artifacts. The filtered image is stored in a reference image buffer (280).

[0040] Figure 3 A block diagram of an exemplary video decoder 300 is shown. In decoder 300, the bitstream is decoded by decoder elements as described below. Video decoder 300 typically performs operations similar to... Figure 2 The encoding process is the inverse of the decoding process. Encoder 200 also typically performs video decoding as part of the video data encoding.

[0041] Specifically, the input to the decoder includes a video bitstream, which can be generated by the video encoder 200. The bitstream is first entropy decoded (330) to obtain transform coefficients, motion vectors, and other encoded information. Image segmentation information indicates how the image is segmented. The decoder can therefore segment (335) the image based on the decoded image segmentation information. The transform coefficients are dequantized (340) and inverse transformed (350) to decode the prediction residual. Image blocks are reconstructed by combining (355) the decoded prediction residual and the predicted blocks. The predicted blocks can be obtained from intra-frame prediction (360) or motion-compensated prediction (i.e., inter-frame prediction) (375) (370). After prediction, prediction enhancement (390) is applied to the predicted blocks. A loop filter (365) is applied to the reconstructed image. The filtered image is stored in a reference image buffer (380).

[0042] The decoded image can undergo further post-decoding processing (385), such as inverse color transformation (e.g., conversion from YCbCr4:2:0 to RGB 4:4:4) or inverse remapping of the inverse of the remapping process performed in pre-encoding processing (201). Post-decoding processing can utilize metadata derived in pre-encoding processing and signaled in the bitstream.

[0043] This principle proposes to improve intra-prediction blocks by modifying the mixing process of intra-prediction modes in the TIMD or DIMD process. More specifically, the TIMD fusion process in current designs is limited to using two angular modes and one non-angular mode. Including more angular modes (e.g., those obtained from DIMD modes) advantageously enhances the TIMD fusion process. Furthermore, other non-angular modes from neighboring intra-block copy blocks can be added to the fusion process. Further, the current TIMD mixing process uses two reference lines to obtain the prediction signal for mixing. The DIMD mixing process uses only one reference line. This principle improves the process by expanding the number of reference lines in the DIMD mixing process and further proposes a method for carefully selecting reference lines for each intra-prediction mode.

[0044] For intra-coded blocks, the encoder selects the best prediction mode from several modes, which can be categorized as: regular modes (which may include angled modes, DC, and Planar), matrix-based modes (called MIP), or block-copy-based modes (which may include IBC and IntraTMP). It is noted that the fusion of different modes can lead to improved prediction. Recent additions to video compression techniques include various industry standards, versions of reference software, and / or documentation, such as the Enhanced Compression Model (ECM) developed by the JVET (Joint Video Exploration Team). The aim is to further improve upon existing VVC (Video Coding Universal) standards. For example, in newer video codec schemes, TIMD and DIMD combine multiple regular modes derived from templates. Recently, TIMD fusion and IntraTMP fusion have also been adopted into ECM, demonstrating some benefits of the fusion approach.

[0045] Recently, several enhancements to the TIMD fusion design have been proposed in ECM. For example, TIMD fusion has been extended to use IBC and IntraTMP candidates obtained from neighboring blocks as potential TIMD candidates. Additionally, a third intra-predicted non-angular mode can optionally be fused during TIMD fusion, provided that the first two intra-predicted modes differ from the third intra-predicted mode and the SATD cost of this non-angular mode is below a given threshold. The fusion weights are calculated based on the SATD cost. Combinations of these two enhancements have also been proposed in recent ECMs and have shown promising results.

[0046] Recognizing the advantages of fusion, this invention proposes several improvements to existing TIMD fusion designs. According to a first embodiment, angular patterns can be added to enhance the final prediction. According to a second embodiment, non-angular patterns can be added to enhance the final prediction by considering neighboring block vector (BV) patterns. According to a third embodiment, a new reference sample selection process is proposed for new angular patterns to increase the diversity of reference samples used to obtain predictions. Finally, according to a fourth embodiment, TIMD fusion also calculates a merged gradient histogram (MHoG) and uses the top five intra-frame patterns with the highest amplitudes to obtain predictions, blending them based on weights calculated according to the histogram amplitudes, and further blending them with planar patterns. To increase the diversity of prediction samples, the reference line selection process can be modified similarly to TIMD.

[0047] The following sections introduce some of the latest tools in ECM research that may require improvement.

[0048] Decoder-side intra-frame mode derivation (DIMD) In the latest codec experiments, when applying DIMD, up to five intra-frame modes are derived from reconstructed neighboring samples, and these five predictors are combined with a planar mode predictor using weights derived from gradient histograms. The division operations in the weight derivation utilize the same lookup table (LUT)-based integerization scheme used by CCLM. For example, division operations in direction calculation... (1) The following LUT-based method is used for calculation: x = Floor( Log2( Gx ) ) (2) normDiff = ( ( Gx<<4 )>>x )&15 (3) x += ( 3 + ( normDiff != 0 ) ? 1 : 0 ) (4) Orient = (Gy ( DivSigTable[ normDiff ] | 8 ) + ( 1<<( x-1 ) ) )>>x (5) in DivSigTable

[16] = { 0, 7, 6, 5 ,5, 4, 4, 3, 3, 2, 2, 1, 1, 1, 1, 0} (6) For size If the magnitude of the histogram above or to the left of a block is more than twice that of the other, then the weight of each of the five derivation patterns is modified. In this case, the weights are position-dependent and calculated as follows: If the histogram above is twice the size of the one on the left, then: (7) If the histogram on the left is twice the size of the one above, then: (8) in It is the unmodified uniform weight of DIMD. It is predefined and set to 10.

[0049] The derived intra-frame mode is included in the master list of most probable intra-frame modes (MPMs), so the DIMD process is performed before the MPM list is built. The master derived intra-frame mode of the DIMD block is stored with the block and used for the MPM list construction of neighboring blocks.

[0050] Finally, the neighboring reconstructed sample regions used to calculate the gradient histogram are modified based on the availability of reconstructed samples. The decoded reference sample region of the current WxH brightness CB is expanded to the upper right, up to W additional columns, if available. If available, it is expanded to the lower left, up to H additional rows.

[0051] DIMD merge mode When using DIMD merging, DIMD information extracted from neighboring blocks is used to compute intra-prediction for the current block. A new merged gradient histogram (MHoG) is computed for the current block based on the HoG of neighboring blocks. Only neighboring blocks encoded with DIMD or DIMD merging are considered.

[0052] When a single DIMD or a DIMD-merged neighboring block is available, its gradient histogram is used to form the MHoG for the current block. If more than one DIMD or DIMD-merged neighboring block is available, the corresponding histograms are combined by averaging the magnitudes to derive the MHoG. A maximum of 13 blocks surrounding the current block are considered to extract DIMD information.

[0053] Finally, the MHoG is used to compute intra-frame prediction modes and weights, as in regular DIMD. The directional modes and their weights corresponding to the five highest amplitudes in the MHoG are selected, and the corresponding predictors are mixed as in regular DIMD.

[0054] Fusion for Template-Based Intra-Frame Mode Derivation (TIMD) For each intra-prediction mode in the MPM, and for the wide-angle mode when reference samples are available in the upper right and / or lower left, the SATD between the predicted and reconstructed samples of the template is calculated. The top two intra-prediction modes with the minimum SATD are selected as TIMD modes. After applying the PDPC procedure, these two TIMD modes are fused with weights, and this weighted intra-prediction is used to encode the current block. The derivation of the TIMD modes includes position-dependent intra-prediction combination (PDPC).

[0055] The costs of the two selected modes are compared with a threshold, and cost factor 2 is applied in the test as follows: costMode2<2 costMode1 (9) If the condition is true, then apply fusion; otherwise, use only mode 1.

[0056] The weight of the pattern is calculated based on its SATD cost as follows: weight1 = costMode2 / (costMode1+ costMode2) (10) weight2 = 1 - weight1 (11) Division is performed using the same lookup table (LUT)-based integerization scheme used by CCLM.

[0057] TIMD Fusion Improvement Based on a current approach in ECM that employs IntraTMP and IBC candidates for TIMD, this paper proposes adding available IntraTMP and IBC patterns from neighboring blocks to the TIMD candidate list to predict the current block template. Up to 50 patterns can be appended to a list including MPM, DC, horizontal, and vertical patterns. Each prediction from IBC and IntraTMP is included in a template matching cost comparison to derive the two best TIMD patterns. If selected by the TIMD process, an IntraTMP / IBC prediction is performed on the current block, and a TIMD mixing process is used.

[0058] Based on another existing technical approach studied in ECM, namely the TIMD fusion process using a non-angle intra-frame mode, a third non-angle intra-frame prediction mode is introduced into the TIMD fusion process. .

[0059] In this scheme, if the non-angle intra-frame mode has the minimum SATD cost (In DC and Planar) with two selected TIMD intra-frame modes ( and Different and (in If so, add it to the TIMD fusion process.

[0060] Weights used in TIMD fusion Based on SATD cost calculations: (12) in (13) The logic behind the non-angle prediction activation condition is based on the calculated SATD cost. Additionally, the location-dependent sample-based mixing of the DIMD fusion process is reused, but the location-dependent criterion applied to the selected predictor amplitude is replaced by a SATD cost-based criterion. The location-dependent criterion is determined based on the ratio of the normalized SATD of the selected TIMD predictor, calculated in the ABOVE (top) and LEFT (left) template regions, as shown below: if ,but (Vertical) (14) Otherwise if ,but (Level) (15) otherwise, (Diagonal) in: - This is a position-dependent parameter value associated with the i-th selected TIMD pattern (e.g., i belongs to [0;2]). A value of 0 indicates no position dependency (diagonal sample-based mixing), a value of 1 indicates the vertical position dependency of the i-th selected TIMD pattern, and a value of 2 indicates the horizontal position dependency of the i-th selected TIMD pattern. - This is the normalized SATD cost associated with the selected TIMD candidate and calculated in the ABOVE template: (16) - It is the normalized SATD cost associated with the selected TIMD candidate and calculated in the LEFT template: (17) in: - The height of the (ABOVE) template area. - The width of the (LEFT) template area. - The current height of the CU - The current width of the CU.

[0061] In addition, block size adaptation is performed. The maximum weight bias (range) of the mixture based on the samples is also considered. The positional dependency becomes size-dependent, with larger blocks (greater than 128 samples) exhibiting smaller bias. Conversely, for smaller block sizes (width or height less than 8 samples), the positional dependency is forced to zero.

[0062] Ultimately, if one of the two templates is unavailable, the position dependency state is set accordingly based on the existing template. If only the LEFT template exists, the position dependency is set to 2 (horizontal); otherwise, if only the ABOVE template exists, the position dependency is set to 1 (vertical).

[0063] Based on another existing technical approach, the optimization of the TIMD mixing process was studied in ECM. For the TIMD modes employing mixing, the first mode uses a reference line. As for the second mode, whether or not to use reference lines... or It depends on the following conditions: - Select if all of the following conditions are true. : ●The current block is not an ISP block. ● The second mode is the angle prediction mode. ●The second mode does not represent a non-fractional angle. - Otherwise, choose .

[0064] Recently, modifications to the above process have been proposed. For the second mode, the modified conditions for determining the reference line are as follows (as shown in the underline): - Select if all of the following conditions are true. : ●The current block is not an ISP block. ● Both the first and second modes are angle prediction modes. ●All of the following conditions are false: › abs(predModeIntra 1 - predModeIntra 2 (greater than) Threshold . Threshold The value is set to 8 Or 4.

[0065] › (predModeIntra 1 - EXT_HOR_IDX) (predModeIntra 2 - EXT_HOR_IDX) is less than 0.

[0066] › (predModeIntra 1 - EXT_VER_IDX) (predModeIntra 2 - EXT_VER_IDX) is less than 0.

[0067] - Otherwise, choose .

[0068] This adjustment provides a small but consistent gain when evaluating the initial proposal.

[0069] Intra-frame prediction fusion According to another approach currently being studied in ECM, an intra-frame prediction method derives the predicted sample as a weighted combination of multiple predictors generated from different reference lines. In this process, multiple intra-frame predictors are generated and then fused by a weighted average. The process for deriving the predictors to be used in the fusion process is described below: 1) For intra-angle prediction modes including single-mode cases of TIMD and DIMD, the proposed method uses weighted summaries to represent the following: Intra-prediction is derived from intra-frame predictions obtained from multiple reference lines, where It is an intra-frame prediction from the default reference line. This is a prediction from the line above the default reference line. The weights are set to... and .

[0070] 2) For the use of a hybrid TIMD mode, For the first mode ( , ),and For the second mode ( , ).

[0071] 3) For the hybrid DIMD mode, the number of predictors selected for the weighted average is increased from 3 to 6.

[0072] When the intra-frame mode has a non-integer slope (requiring reference sample interpolation) and the block size is greater than 16, the intra-frame prediction fusion method is applied to the luma block, used in conjunction with MRL, and not applied to ISP-encoded blocks. In the method studied in subtest a, PDPC is applied to the intra-frame prediction mode using the reference line closest to the current block.

[0073] It has been demonstrated that the mixing of intra-prediction modes derived in TIMD or DIMD processes often yields gains, but there is still room for further improvement. This document introduces a novel approach to the TIMD fusion process for intra-prediction that increases the number of intra-prediction angle modes used for fusion. According to an additional embodiment, the reference line selection process is adjusted to consider more angle modes. Furthermore, the reference line selection process is extended to DIMD modes, which can also benefit from the increased diversity of prediction samples when performing mixing / merging.

[0074] The present invention covers the following main embodiments, including increasing the number of intra-prediction angle modes in the TIMD fusion process, increasing the number of non-angle intra-prediction modes (i.e., IBC / IntraTMP BV) in the TIMD fusion process, adapted reference line selection for new intra-prediction angle modes, and extending reference line selection to DIMD modes to increase diversity.

[0075] According to a general embodiment, a video decoding method includes: determining intra-frame patterns to be derived for predicting a current block; deriving multiple intra-frame patterns based on a template associated with the current block; obtaining multiple predictions based on the multiple intra-frame patterns; mixing the multiple predictions to obtain a prediction block for the current block; and decoding the block based on the prediction block. According to another general embodiment, a video coding method includes: determining intra-frame patterns to be derived for predicting a current block; deriving multiple intra-frame patterns based on a template associated with the current block; obtaining multiple predictions based on the multiple intra-frame patterns; mixing the multiple predictions to obtain a prediction block for the current block; and encoding the block based on the prediction block. This derivation of multiple intra-frame patterns based on a template associated with the current block may correspond to a TIMD fusion process or a DIMD fusion process, and various embodiments are described in detail below.

[0076] Increase the number of fusion angle modes in the TIMD fusion process In one embodiment, intra-frame mode derivation includes Template Intra-Frame Mode Derivation (TIMD). Depending on specific characteristics, the plurality of intra-frame modes includes more than two angular modes. In a variant, the plurality of intra-frame modes includes three to five angular modes.

[0077] Figure 4 illustrates the TIMD process according to the prior art. In TIMD mode, two intra-prediction modes (IPM1 and IPM2) that provide the minimum SATD cost are selected. In ECM, the intra-prediction mode only considers the conventional prediction mode. Recently, an extension to use unconventional prediction modes has been proposed in ECM, as shown in Figure 4. Furthermore, it is proposed in ECM to optionally fuse a third non-angular IPM3 when this IPM differs from the previous two IPMs.

[0078] Figure 5 The TIMD process for expanding the number of angular fusion modes according to a first embodiment is illustrated. The first embodiment expands the concept of fusion by increasing the number of angular intra-frame prediction modes used for fusion. For example, considering the preceding [predicted modes] that meet a threshold condition... The intra-frame angle modes with the lowest SATD cost are fused together, among which It can be 3, 4, or 5. As an example, The maximum value can be set to 5. The threshold conditions are as follows: costAngularMode n <2 costMode1, where n>1 (18) The weights used in the fusion process are calculated using the SATD cost per IPM.

[0079] As an alternative, the above threshold condition is modified as follows: costAngularMode n <2 costAngularMode1, where n>1 (19) Where costAngularMode1 is the SATD cost of the first angular mode. In equation (18), an angular mode is added when its cost is less than twice the cost of the first mode in the list (which can be a non-angular mode). In equation (19), an angular mode is added when its cost is less than twice the cost of the first angular mode (which is not necessarily the first mode). Therefore, (19) provides a more lenient alternative to (18).

[0080] In a variation of this first embodiment, the number of angle modes is conditionally increased. For example, multiple intra-frame mode (IPM) includes a first mode, a second mode, and at least one additional angle mode, wherein the number of additional angle modes depends on the first mode and the second mode.

[0081] Figure 6a and Figure 6b The TIMD process for conditionally expanding the number of angular fusion modes according to a first embodiment is illustrated. In this variation, the number of angular modes considered depends on whether the first and second candidates are angular or non-angular. Figure 6a In the example, when one of the first two IPMs is non-angular, IPM3 can optionally be added as an angular mode if the SATD cost threshold condition (Equation (18)) is met. Figure 6b In another example shown, when the first two IPMs are non-angular, IPM3 and IPM4 can optionally be added as angular modes if the threshold condition is met. Therefore, in this variant, the maximum number of TIMD modes considered for fusion is 4.

[0082] In another variation of this first embodiment, the additional modes correspond to the modes derived by DIMD. For example, in another example, multiple intra-frame modes may include a first mode, a second mode, and at least one additional angle mode, one of which corresponds to a decoder-side intra-frame mode derivation (DIMD) mode. Instead of deriving more modes for TIMD, ready-made modes using DIMD are proposed. Specifically, DIMD is computed before TIMD to populate the MPM list. The resulting DIMD modes (up to five) can be used as additional modes for TIMD fusion.

[0083] As a variant, additional modes derived from DIMD are added based on template cost. Specifically, intra-prediction modes are first mapped from 64 prediction directions to 135 prediction directions. Next, the TIMD template cost derivation process is applied to calculate the SATD between the reconstructed template and the predicted template. Intra-prediction modes that meet the following threshold conditions are added as TIMD fusion candidates.

[0084] costAngularMode n <k costMode1 (20) Where n>2 and k is 2 (as an unrestricted example).

[0085] As another variation of the above, a small offset is added to the angle pattern. The template cost is then used to determine the optimal DIMD mode for refinement, and the refinement of the DIMD intra-prediction mode is performed. For example, if the optimal DIMD intra-mode is 17, the refinement process involves calculating the SATD cost of intra-modes 16, 17, and 18 to determine the optimal intra-mode for the TIMD process. Thus, in another example, multiple intra-modes include a first mode, a second mode, and at least one additional angle mode, one of which corresponds to a decoder-side intra-mode derivation (DIMD) mode.

[0086] Non-angular fusion is enhanced by adding IBC / IntraTMP block vector-based predictions during the TIMD fusion process. Number of patterns In one embodiment, the plurality of intra-frame modes include a first mode, a second mode, a third non-angle mode, and at least one additional non-angle mode obtained from a neighboring IBC / IntraTMP block.

[0087] Figure 7 A TIMD process according to a second embodiment is illustrated, which expands the number of non-angular fusion modes by adding block vectors from neighboring IBC / IntraTmp coded blocks. In this embodiment, additional BV predictors from neighboring blocks are proposed as potential candidates for TIMD fusion. BV predictors are selected based on the template cost calculated between the template of the reference block and the template of the current block. When the template cost is below a threshold, a BV predictor is added. For example, up to five BV predictors can be added to the TIMD fusion process.

[0088] Select a reference line in TIMD to compute predictions from intra-frame prediction modes. In a third embodiment, in response to a criterion, a reference line above a reference line selected for a previous angular mode is selected to be used to obtain a prediction using the given angular mode. In another variation, obtaining multiple predictions based on multiple intra-frame modes includes: for each angular mode, averaging the prediction based on a first neighboring reference line and the prediction based on a second reference line above the first reference line. In yet another variation, the same principle is applied to non-angular modes, wherein, in response to a criterion, a reference line above a reference line selected for a previous intra-frame mode is used for the currently additional non-angular intra-frame mode.

[0089] When expanding the number of angle modes as described in the first embodiment, this embodiment proposes a method to further enhance the diversity of the fusion process by optimizing the existing reference line selection process.

[0090] Figure 8 This illustrates the template-based intra-frame mode derivation (TIMD) process based on the current design in the ECM. (Size: [size not specified]) The current luminance CB template is shown in light gray. The set of decoded reference samples for the template is shown in dark gray. All possible intra-frame prediction directions (including wide-angle mode) are tested by calculating the SATD cost between the reconstructed template (light gray 810) and the predicted template using the reference samples (dark gray 820). It is further noted that the number of reference lines above and to the left of the luminance CB is 2 or 4 based on the block size.

[0091] Figure 9 The reference line selection for TIMD mode is shown according to the current design in the ECM, where only the two reference lines above and to the left of the current brightness CB are considered for TIMD mode.

[0092] Currently in ECM, the Multiple Reference Line (MRL) list has been expanded to provide access to more reference lines above the current block. For TIMD, the full MRL is not used; instead, only the first two reference line candidates are used. The current reference line selection process for TIMD in ECM uses the reference line for the first intra-frame mode. Furthermore, a reference line is used for the second intra-frame mode based on the aforementioned criteria, which serve as a variant description for optimizing the TIMD blending process. or Advantageously, this feature increases diversity when two intra-frame modes are angular modes and are close to each other.

[0093] The third embodiment proposes a reference line selection process in the TIMD fusion mode that is further optimized for the additional intra-frame angle modes considered in the first embodiment. Specifically, for each additional angle mode considered for TIMD fusion, a reference line above the reference line selected for the previous angle mode is selected when the stated criterion is met. The rationale for the criterion is simple and based on the proximity of the two intra-frame angle modes. When the current angle mode and the previous angle mode are close to each other (e.g., determined by a threshold), a reference line above the reference line selected for the previous angle mode is selected to determine the prediction mode. Advantageously, this embodiment allows for increased diversity for prediction modes that are not far from each other.

[0094] As a variation, the above process can be extended by checking the distance to each previous intra-frame angle pattern. Once an angle pattern is closer than any previous intra-frame angle pattern (e.g., determined by a threshold), the next reference line is selected. As a non-limiting example, when the additional angle pattern added to the TIMD fusion is a third pattern, the reference line selection process can be modified as follows: - Select if all of the following conditions are true. : ● The current block is not an ISP block. ● The first, second, and third modes are angle prediction modes. ● Second mode usage Reference line ● All of the following conditions are false: ›abs(predModeIntra2- predModeIntra3) is greater than Threshold . Threshold The value is set to 8 or 4.

[0095] ›(predModeIntra2- EXT_HOR_IDX) (predModeIntra3-EXT_HOR_IDX) is less than 0.

[0096] ›(predModeIntra2- EXT_VER_IDX) (predModeIntra3-EXT_VER_IDX) is less than 0.

[0097] - Otherwise, select if the following conditions are met. : ● Second mode usage Reference line ● All of the following conditions are false. ›abs(predModeIntra1-predModeIntra3) is greater than Threshold . Threshold The value is set to 8 or 4.

[0098] ›(predModeIntra1- EXT_HOR_IDX) (predModeIntra3-EXT_HOR_IDX) is less than 0.

[0099] ›(predModeIntra1- EXT_VER_IDX) (predModeIntra3-EXT_VER_IDX) is less than 0.

[0100] - Otherwise, choose .

[0101] Similarly, the above process applies to the first... The intra-frame angle pattern repeats to One of the reference lines.

[0102] In a variant of the third embodiment, the reference line selection process can also be adapted when the first two TIMD modes include non-angle modes. As a variant, an additional intra-frame angle mode is added as a third mode and used based on the following criteria. Reference line: ● The current block is not an ISP block. ● The first or second mode is a non-angle mode. ● At least one of the first two modes is an angle mode. ● All of the following conditions are false: ›abs(predModeIntra 角度 - `predModeIntra3` is greater than the threshold. The threshold is set to 4 or 8. ›(predModeIntra 角度 - EXT_HOR_IDX) (predModeIntra3 - EXT_HOR_IDX) is less than 0. ›(predModeIntra 角度 - EXT_VER_IDX) (predModeIntra3 - EXT_VER_IDX) is less than 0. - Otherwise, use .

[0103] Another variation is when an additional intra-frame prediction angle mode is added as a fourth mode during the fusion process, based on the following guidelines. Reference line: ● The current block is not an ISP block. ● Both the first and second modes are non-angle modes. ● The third mode added is the angle mode. ● All of the following conditions are false: ›abs(predModeIntra3-predModeIntra4) is greater than the threshold. The threshold should be set to 4 or 8. ›(predModeIntra3- EXT_HOR_IDX) (predModeIntra3 - EXT_HOR_IDX) is less than 0. ›(predModeIntra3- EXT_VER_IDX) (predModeIntra3 - EXT_VER_IDX) is less than 0. - Otherwise, use .

[0104] In another variation of the third embodiment, the reference line selection process can also be adapted for TIMD fusion processes with non-angular intra-frame modes.

[0105] As described in the TIMD fusion process for non-angular intra-frame modes, it is proposed to optionally fuse a third intra-frame mode into a non-angular mode. In this scenario, a reference line can be used for this mode when the following conditions are met. : ● The third non-angular mode is DC or Planar. ● The current block is not an ISP block. ● The first two modes have already been used When using reference lines (this adds variety). - Otherwise, use Used as a reference line.

[0106] As a variation of the above, when the first two modes are non-angular modes (but not IBC or IntraTMP), then to increase diversity, the second mode is used when the following conditions are met. Reference line: ● The current block is not an ISP block. ● Both the first and second modes are non-angle modes, and at least one is not an IBC or IntraTMP mode. - Otherwise, use Used as a reference line.

[0107] In yet another variation of the third embodiment, the predicted fusion can be obtained from multiple reference lines.

[0108] The TIMD fusion process blends prediction samples from two intra-prediction modes. For the first mode, a reference line is used. For the second mode, in some cases, the choice is... In this variant, the use of and The two reference lines are used to calculate the forecast. Then, the two forecasts are averaged. Finally, based on the calculated weights, this average forecast is fused with the forecast using the first model.

[0109] In yet another variation, using and The averaging of the predicted samples is performed only for non-angular DC and planar modes.

[0110] Select a reference line in DIMD mode to compute predictions from intra-prediction mode. In another embodiment, when intra-frame mode derivation includes a decoder-side intra-frame mode derivation (DIMD) mode, in response to a criterion, a reference line above a reference line selected for a previously chosen angle mode is selected to obtain a prediction using the given angle mode. Further, obtaining multiple predictions based on multiple intra-frame modes in DIMD may include, for each angle mode, averaging the prediction based on a first neighboring reference line and the prediction based on a second reference line above the first reference line.

[0111] The DIMD mode and the DIMD merged mode first calculate the HoG (Histogram of Histograms). Then, it selects the five prediction modes with the highest amplitudes in the HoG histogram and blends them together using weights calculated from the same HoG. Finally, it blends with the planar mode.

[0112] All five prediction modes use the same reference line to construct prediction blocks. To increase diversity, the reference line selection process for intra-angle modes described in the previous section for TIMD is extended to DIMD and DIMD merging modes.

[0113] As a variation, multiple predicted signals are constructed using different reference lines and then mixed by simple averaging. The averaged predicted signals are then further merged, as is typically done in DIMD.

[0114] This document describes various methods, and each method includes one or more steps or actions for implementing the described method. The order and / or use of specific steps and / or actions may be modified unless a specific order of steps or actions is required for the method to operate correctly. Furthermore, terms such as "first" and "second" may be used in various embodiments to modify elements, components, steps, operations, etc., e.g., "first decoding" and "second decoding." The use of such terms does not imply an ordering of the modified operations unless specifically required. Therefore, in this example, the first decoding does not need to be performed before the second decoding and may occur, for example, before, during, or in a period overlapping with the second decoding.

[0115] The various methods and other aspects described in this application can be used to modify the module, for example, such as Figure 2The motion compensation module (270) of the video encoder 200 shown is illustrated. Furthermore, this aspect is not limited to ECM and VVC, and can be applied to, for example, other standards and recommendations, as well as any extensions of such standards and recommendations. Unless otherwise stated or technically impractical, the aspects described in this application may be used individually or in combination.

[0116] Various numerical values ​​are used in this application. The specific values ​​are for illustrative purposes, and the aspects described are not limited to these specific values.

[0117] Various implementations involve decoding. As used herein, “decoding” can include all or part of a process performed on a received encoded sequence, for example, to produce a final output suitable for display. In various embodiments, such a process includes one or more processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. Whether the phrase “decoding process” is intended to specifically refer to a subset of operations or to refer generally to a broader decoding process will be clear based on the context of the specific description and should be well understood by those skilled in the art.

[0118] Various implementations involve encoding. Similar to the discussion of "decoding" above, "encoding" as used in this application can include, for example, all or part of a process performed on an input video sequence to produce an encoded bitstream.

[0119] Note that the grammatical elements used in this article are descriptive terms. Therefore, the use of other grammatical element names is not excluded.

[0120] The implementations and aspects described herein can be implemented, for example, as methods or procedures, apparatus, software programs, data streams, or signals. Even if discussed only in the context of a single form of implementation (e.g., discussed only as a method), the implementation of the features in question can be implemented in other forms (e.g., apparatus or program). Apparatus can be implemented, for example, as appropriate hardware, software, and firmware. Methods can be implemented, for example, as apparatuses (e.g., processors), where processor generally refers to processing devices, including, for example, computers, microprocessors, integrated circuits, or programmable logic devices. Processors also include communication devices, such as computers, mobile phones, portable / personal digital assistants (“PDAs”), and other devices that facilitate information communication between end users.

[0121] References to “an embodiment” or “an embodiment” or “an implementation” or “an implementation” and other variations are intended to include the specific features, structures, characteristics, etc., described in connection with at least one embodiment. Therefore, the phrases “in an embodiment” or “in an embodiment” or “in an implementation” or “in an implementation” appearing in various places in this application, as well as any other variations, do not necessarily refer to the same embodiment.

[0122] Furthermore, this application may involve "determining" various types of information. Determining information may include, for example, one or more of the following: estimation information, calculation information, prediction information, or information retrieved from memory.

[0123] Furthermore, this application may involve "accessing" various types of information. Accessing information may include, for example, receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information, or one or more of these.

[0124] Furthermore, this application may relate to "receiving" various types of information. Like "access," "receiving" is intended to be a broad term. Receiving information may include, for example, access information or retrieval information (e.g., from memory) or one or more of them. Further, "receiving" generally involves, in some way during operation, storing information, processing information, sending information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0125] It should be understood that the use of any one of “ / ”, “and / or”, and “…at least one of…”, such as in “A / B”, “A and / or B”, and “at least one of A and B”, is intended to cover selecting only the first listed option (A), or only the second listed option (B), or both options (A and B). As a further example, in the cases of “A, B, and / or C” and “at least one of A, B, and C”, such wording is intended to cover selecting only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A, B, and C). This can be extended to the number of items listed, as is clear to those skilled in the art and related fields.

[0126] Additionally, as used herein, the term "signal" specifically refers to instructing the corresponding decoder to do something. For example, in some embodiments, the encoder signals the quantization matrix used for dequantization. In this way, in embodiments, the same parameters are used on both the encoder and decoder sides. Thus, for example, the encoder can send (explicit signaling) specific parameters to the decoder so that the decoder can use the same specific parameters. Conversely, if the decoder already has the specific parameters as well as other parameters, signaling can be used without transmission (implicit signaling) to simply allow the decoder to know and select the specific parameters. Bit savings are achieved in many embodiments by avoiding the transmission of any actual functionality. It should be understood that signaling can be done in various ways. For example, in many embodiments, one or more syntax elements, flags, etc., are used to signal information to the corresponding decoder. Although the foregoing refers to the verb form of the term "signal," the word "signal" can also be used as a noun herein.

[0127] As will be apparent to those skilled in the art, implementations can generate signals in various formats to carry information, which can be stored or transmitted, for example. The information may include, for example, instructions for performing a method, or data generated by one of the described implementations. For example, the signal may be formatted to carry a bitstream of the described embodiments. Such a signal may be formatted, for example, as an electromagnetic wave (e.g., using the radio frequency portion of the spectrum) or as a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier wave with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, the signal can be transmitted via a variety of different wired or wireless links. The signal may be stored on a processor-readable medium.

[0128] We have described several embodiments. The features of these embodiments may be provided individually or in any combination across multiple claim classes and types. Furthermore, embodiments may include one or more of the following features, devices, or aspects, individually or in any combination across multiple claim classes and types.

Claims

1. A video decoding method, comprising: Determine which intra-frame mode to derive for predicting the current block; Multiple intra-frame modes are derived based on the template associated with the current block; Multiple predictions are obtained based on the aforementioned multiple intra-frame modes; The multiple predictions are combined to obtain the prediction block for the current block; as well as The block is decoded based on the predicted block.

2. The method of claim 1, wherein intra-frame mode derivation includes template intra-frame mode derivation (TIMD).

3. The method of claim 2, wherein the plurality of intra-frame modes includes more than two angle modes.

4. The method of claim 2, wherein the plurality of intra-frame modes includes 3 to 5 angle modes.

5. The method of claim 2, wherein the plurality of intra-frame modes include a first mode, a second mode, and at least one additional angle mode, wherein the number of additional angle modes depends on the first mode and the second mode.

6. The method of claim 2, wherein the plurality of intra-frame modes include a first mode, a second mode, and at least one additional angle mode, one of the additional angle modes corresponding to a decoder-side intra-frame mode derivation (DIMD) mode.

7. The method of claim 2, wherein the plurality of intra-frame modes includes a first mode, a second mode, and at least one additional angle mode, one of the additional angle modes corresponding to a decoder-side intra-frame mode derivation (DIMD) mode, and one of the additional angle modes corresponding to a decoder-side intra-frame mode derivation (DIMD) mode with offset.

8. The method of claim 2, wherein the plurality of intra-frame modes include a first mode, a second mode, a third non-angular mode, and at least one additional non-angular mode obtained from a neighboring IBC / IntraTMP block.

9. The method of claim 3, wherein, in response to a criterion, a reference line above a reference line selected for a previous angle pattern is selected for using the given angle pattern to obtain the prediction.

10. The method of claim 3, wherein obtaining the plurality of predictions based on the plurality of intra-frame modes comprises: For each angle pattern, the predictions based on the first neighboring reference line and the predictions based on the second reference line above the first reference line are averaged.

11. The method of claim 1, wherein intra-frame mode derivation includes a decoder-side intra-frame mode derivation (DIMD) mode, and wherein, in response to a criterion, a reference line above a reference line selected for a previous angle mode is selected for obtaining the prediction using the given angle mode.

12. The method of claim 1, wherein intra-frame mode derivation includes decoder-side intra-frame mode derivation (DIMD) modes, and wherein obtaining multiple predictions based on the plurality of intra-frame modes includes: For each angle pattern, the predictions based on the first neighboring reference line and the predictions based on the second reference line above the first reference line are averaged.

13. The method of claim 8, wherein, in response to a criterion, a reference line above a reference line selected for a previous intra-frame mode is used for the current additional non-angular intra-frame mode.

14. A video coding method, comprising: Determine which intra-frame mode to derive for predicting the current block; Multiple intra-frame modes are derived based on the template associated with the current block; Multiple predictions are obtained based on the aforementioned multiple intra-frame modes; The multiple predictions are combined to obtain the prediction block for the current block; as well as The block is decoded based on the predicted block.

15. The method of claim 14, wherein intra-frame mode derivation includes template intra-frame mode derivation (TIMD).

16. The method of claim 15, wherein the plurality of intra-frame modes includes more than two angular modes.

17. The method of claim 15, wherein the plurality of intra-frame modes comprises 3 to 5 angle modes.

18. The method of claim 15, wherein the plurality of intra-frame modes include a first mode, a second mode, and at least one additional angle mode, wherein the number of additional angle modes depends on the first mode and the second mode.

19. The method of claim 15, wherein the plurality of intra-frame modes include a first mode, a second mode, and at least one additional angle mode, one of the additional angle modes corresponding to a decoder-side intra-frame mode derivation (DIMD) mode.

20. The method of claim 15, wherein the plurality of intra-frame modes includes a first mode, a second mode, and at least one additional angle mode, one of the additional angle modes corresponding to a decoder-side intra-frame mode derivation (DIMD) mode, and one of the additional angle modes corresponding to a decoder-side intra-frame mode derivation (DIMD) mode with offset.

21. The method of claim 15, wherein the plurality of intra-frame modes include a first mode, a second mode, a third non-angular mode, and at least one additional non-angular mode obtained from a neighboring IBC / IntraTMP block.

22. The method of claim 16, wherein, in response to a criterion, a reference line above a reference line selected for a previous angle pattern is selected for using the given angle pattern to obtain the prediction.

23. The device of claim 16, wherein obtaining the plurality of predictions based on the plurality of intra-frame modes comprises: For each angle pattern, the predictions based on the first neighboring reference line and the predictions based on the second reference line above the first reference line are averaged.

24. The method of claim 14, wherein intra-frame mode derivation includes a decoder-side intra-frame mode derivation (DIMD) mode, and wherein, in response to a criterion, a reference line above a reference line selected for a previous angle mode is selected for obtaining the prediction using the given angle mode.

25. The method of claim 14, wherein intra-frame mode derivation includes decoder-side intra-frame mode derivation (DIMD) modes, and wherein obtaining multiple predictions based on the plurality of intra-frame modes includes: For each angle pattern, the predictions based on the first neighboring reference line and the predictions based on the second reference line above the first reference line are averaged.

26. The method of claim 21, wherein, in response to a criterion, a reference line above a reference line selected for a previous intra-frame mode is used for the current additional non-angular intra-frame mode.

27. An apparatus comprising one or more processors, wherein the one or more processors are configured to perform the method as claimed in any one of claims 1-26.

28. A signal comprising video data, formed by performing the method as described in any one of claims 14-26.

29. A computer-readable storage medium having stored thereon instructions for performing video decoding or encoding according to any one of claims 1-26.

Images