Fix for intra prediction fusion

Abstract

One or more intra-mode predictors of the current block are determined, and the one or more intra-mode predictors are derived based on neighboring samples of the current block via either decoder-side intra-mode derivation (DIMD) or template-based intra-mode derivation (TIMD). Whether position-dependent intra-prediction combining (PDPC) is applied is determined based on the one or more intra-mode predictors. In response to the determination that PDPC is applied, a fusion process is performed on the one or more intra-mode predictors to combine the one or more intra-mode predictors into a single intra-mode predictor. The PDPC is applied based on the single intra-mode predictor to generate a modified predictor of the current block. The current block is reconstructed based on the modified predictor generated by the PDPC.

Description

[Technical field]

[0001] This application claims benefit of priority to U.S. Provisional Application No. 17 / 943,275, "MODIFICATION ON FUSION OF INTRA PREDICTION," filed September 13, 2022, which claims benefit of priority to U.S. Provisional Application No. 63 / 252,600, "Modification on Fusion of Intra Prediction," filed October 5, 2021. The entire disclosures of the prior applications are hereby incorporated by reference in their entireties.

[0002] This disclosure describes embodiments that relate generally to video coding. [Background technology]

[0003] The background provided herein is intended to generally present the context of the present disclosure. The work of the present inventors is not admitted, expressly or impliedly, as prior art to the present disclosure to the extent that their work is described in this Background section, and aspects of the description that may not otherwise qualify as prior art at the time of filing.

[0004] Video coding and decoding may be performed using inter-picture prediction with motion compensation. Uncompressed digital video may include a sequence of pictures, each having spatial dimensions of, for example, 1920x1080 luma samples and associated chroma samples. The sequence of pictures may have a fixed or variable picture rate, for example, 60 pictures per second or 60 Hz. Uncompressed video has specific bitrate requirements. For example, 1080p60 4:2:0 video (1920x1080 luma sample resolution at a frame rate of 60 Hz) at 8 bits per sample requires a bandwidth approaching 1.5 Gbit / s. One hour of such video requires more than 600 Gbytes of storage space.

[0005] One goal of video coding and decoding is the reduction of redundancy in the input video signal through compression. Compression can help reduce the aforementioned bandwidth and / or storage space needs by more than two orders of magnitude in some cases. Both lossless and lossy compression, as well as combinations thereof, can be used. Lossless compression refers to techniques where an exact copy of the original signal can be reconstructed from the compressed original signal. When using lossy compression, the reconstructed signal may not be identical to the original signal, but the distortion between the original signal and the reconstructed signal is small enough to make the reconstructed signal useful for the intended application. For video, lossy compression is widely used. The amount of distortion that is tolerated depends on the application, for example, a user of a particular consumer streaming application may tolerate higher distortion than a user of a television distribution application. The achievable compression ratio can reflect that a higher tolerable / acceptable distortion can result in a higher compression ratio.

[0006] Video encoders and decoders can utilize techniques from a number of broad categories, including, for example, motion compensation, transformation, quantization, and entropy coding.

[0007] Video codec techniques can include a technique known as intra-coding. In intra-coding, sample values ​​are represented without reference to samples or other data from previously reconstructed reference pictures. In some video codecs, a picture is spatially subdivided into blocks of samples. When all blocks of samples are coded in intra mode, the picture can be an intra picture. Intra pictures and their derivatives, such as independent decoder refresh pictures, can be used to reset the decoder state and therefore can be used as the first picture in a coded video bitstream and video session or as a still image. Samples of an intra block can be subjected to a transform, and the transform coefficients can be quantized before entropy coding. Intra prediction can be a technique that minimizes sample values ​​in the pre-transform domain. In some cases, the smaller the DC value and the smaller the AC coefficients after the transform, the fewer bits are required for a given quantization step size to represent the block after entropy coding.

[0008] Traditional intra-coding, e.g., as known from MPEG-2 generation coding techniques, does not use intra-prediction. However, some newer video compression techniques include techniques that attempt to do so from surrounding sample data and / or metadata, e.g., obtained during encoding and / or decoding of spatially adjacent and preceding data blocks in decoding order. Such techniques are hereafter referred to as "intra-prediction" techniques. Note that, at least in some cases, intra-prediction uses only reference data from the current picture being reconstructed, and not from reference pictures.

[0009] There may be various forms of intra prediction. If one or more of such techniques can be used in a given video coding technique, the technique in use may be coded in an intra prediction mode. In certain cases, a mode may have sub-modes and / or parameters that may be coded separately or may be included in a mode codeword. Which codeword to use for a given mode, sub-mode, and / or parameter combination may affect the coding efficiency gain through intra prediction and may also affect the entropy coding technique used to convert the codeword into a bitstream.

[0010] A specific mode of intra prediction was introduced in H.264, improved in H.265, and further improved in newer coding techniques such as the Joint Search Model (JEM), Generic Video Coding (VVC), and Benchmark Set (BMS). A predictor block may be formed using neighboring sample values ​​belonging to already available samples. The sample values ​​of the neighboring samples are copied to the predictor block according to the direction. The reference to the direction in use may be coded in the bitstream or may itself be predicted.

[0011] Referring to FIG. 1, depicted at the bottom right is a subset of 9 known predictor directions from the 33 possible predictor directions of H.265 (corresponding to the 33 angular modes of the 35 intra modes). The point where the arrows converge (101) represents the sample being predicted. The arrows represent the direction in which the sample is predicted. For example, arrow (102) indicates that sample (101) is predicted from a sample or samples at an angle of 45 degrees from the horizontal toward the top right. For example, arrow (103) indicates that sample (101) is predicted from a sample or samples at an angle of 22.5 degrees from the horizontal toward the bottom left.

[0012] Continuing to refer to FIG. 1, at the top left, a square block (104) of 4×4 samples is depicted (indicated by a dashed bold line). The square block (104) contains 16 samples, each labeled with "S", its position in the Y dimension (e.g., row index), and its position in the X dimension (e.g., column index). For example, sample S21 is the second sample (from the top) in the Y dimension and the first sample (from the left) in the X dimension. Similarly, sample S44 is the fourth sample in the block (104) in both the Y and X dimensions. Since the size of the block is 4×4 samples, S44 is at the bottom right. Additionally, reference samples are shown following a similar numbering scheme. The reference samples are labeled with R, their Y position (e.g., row index), and X position (column index) relative to the block (104). In both H.264 and H.265, the predicted samples are adjacent to the block being reconstructed, so there is no need to use negative values.

[0013] Intra-picture prediction can work by copying reference sample values ​​from neighboring samples as appropriate according to the signaled prediction direction. For example, assume that the coded video bitstream includes signaling for this block indicating a prediction direction that coincides with the arrow (102). That is, the sample is predicted from the prediction sample or samples at an angle of 45 degrees from the horizontal toward the upper right. In that case, samples S41, S32, S23, and S14 are predicted from the same reference sample R05. Then sample S44 is predicted from reference sample R08.

[0014] In certain cases, especially when the orientation is not evenly divisible by 45 degrees, the values ​​of multiple reference samples may be combined, for example through interpolation, to calculate the reference sample.

[0015] As video coding technology develops, the number of possible directions increases. In H.264 (2003), nine different directions could be represented. This increased to 33 in H.265 (2013), and at the time of disclosure, JEM / VVC / BMS can support up to 65 directions. Experiments are performed to identify the most likely directions, and certain techniques in entropy coding are used to represent those possible directions with a small number of bits, allowing certain penalties for less likely directions. Furthermore, the direction itself can sometimes be predicted from neighboring directions used in neighboring, already decoded blocks.

[0016] FIG. 2 shows a schematic (201) depicting 65 intra prediction directions with JEM to illustrate the increasing number of prediction directions over time.

[0017] The mapping of intra-prediction direction bits in a coded video bitstream representing directions can vary between video coding techniques, and can range, for example, from simple direct mapping of prediction directions to complex adaptation schemes involving intra-prediction modes, codewords, most likely modes, and similar techniques. However, in any given case, there may be certain directions that are statistically less likely to occur in the video content than certain other directions. Since the goal of video compression is to reduce redundancy, in well-behaved video coding techniques, less likely directions are represented by more bits than more likely directions. Summary of the Invention

[0018] Aspects of the present disclosure provide a method and apparatus for video encoding / decoding. In some examples, an apparatus for video decoding includes receiving circuitry and processing circuitry.

[0019] According to one aspect of the present disclosure, a method of video decoding performed in a video decoder is provided. In the method, coded information of a current block and neighboring samples of the current block may be received from a coded video bitstream. One or more intra-mode predictors of the current block may be determined, and the one or more intra-mode predictors may be derived based on neighboring samples of the current block via either decoder-side intra-mode derivation (DIMD) or template-based intra-mode derivation (TIMD). Whether a position-dependent intra-prediction combining (PDPC) is applied may be determined based on the one or more intra-mode predictors. In response to a determination that the PDPC is applied, a fusion process may be performed on the one or more intra-mode predictors, and the fusion process may combine the one or more intra-mode predictors to generate a single intra-mode predictor. The PDPC may be applied based on the single intra-mode predictor, and may generate a modified predictor of the current block following the fusion process on the one or more intra-mode predictors. The current block may be reconstructed based on the modified predictor generated by the PDPC.

[0020] In some embodiments, the fusion process may include generating a single intra-mode predictor based on a weighted combination of the one or more intra-mode predictors. In some embodiments, the fusion process may further include generating a modified predictor based on a weighted combination of (i) one or more of the neighboring samples and (ii) the single intra-mode predictor.

[0021] In response to the one or more intra-mode predictors being derived based on the DIMD, a first intra-mode predictor of the one or more intra-mode predictors may be determined based on a first intra-mode. The first intra-mode may be determined based on neighboring samples and associated with a highest histogram of gradient (HoG) of the neighboring samples. A second intra-mode predictor of the one or more intra-mode predictors may be determined based on a second intra-mode. The second intra-mode may be determined based on neighboring samples and associated with a second highest HoG of the neighboring samples. A third intra-mode predictor of the one or more intra-mode predictors may be determined based on a planar mode.

[0022] In some embodiments, whether the PDPC is applicable may be determined based on the first intra mode derived based on the DIMD.

[0023] In response to the first intra mode and the second intra mode being angular modes, it may be determined whether PDPC is applicable based on average angular values ​​of the first intra mode and the second intra mode. In response to one of the first intra mode and the second intra mode being angular modes, it may be determined whether PDPC is applicable based on one of the first intra mode and the second intra mode being an angular mode.

[0024] In some embodiments, the PDPC may be determined to be applicable in response to at least one of the one or more intra-mode predictors being determined based on a planar mode or a DC mode.

[0025] In response to a first intra-mode predictor of the one or more intra-mode predictors being derived based on the TIMD, a first intra-mode predictor of the one or more intra-mode predictors may be determined based on a first intra-mode, the first intra-mode may be determined based on neighboring samples and associated with a minimum cost value of the neighboring samples, a second intra-mode predictor of the one or more intra-mode predictors may be determined based on a second intra-mode, the second intra-mode may be determined based on neighboring samples and associated with a second minimum cost value of the neighboring samples.

[0026] In some embodiments, whether the PDPC is applicable may be determined based on the first intra mode derived based on the TIMD.

[0027] In response to the first intra mode and the second intra mode being angular modes, it may be determined whether PDPC is applicable based on an average angular value of the first intra mode and the second intra mode. In response to one of the first intra mode and the second intra mode being an angular mode, it may be determined whether PDPC is applicable based on one of the first intra mode and the second intra mode being an angular mode.

[0028] In some embodiments, in response to at least one of the first intra mode and the second intra mode being a planar mode or a DC mode, it may be determined that the PDPC is applicable.

[0029] According to another aspect of the present disclosure, an apparatus is provided having processing circuitry, the processing circuitry may be configured to perform any method for video coding.

[0030] Aspects of the present disclosure also provide a non-transitory computer-readable medium storing instructions that, when executed by a computer, cause the computer to perform any of the methods for video decoding. [Brief description of the drawings]

[0031] Further features, nature and various advantages of the disclosed subject matter will become more apparent from the following detailed description and the accompanying drawings.

[0032] [Figure 1] FIG. 2 is a schematic diagram of an example subset of intra-prediction modes.

[0033] [Diagram 2] FIG. 2 is a diagram of an example intra-prediction direction.

[0034] [Diagram 3] FIG. 3 is a schematic diagram of a simplified block diagram of a communication system (300) according to one embodiment.

[0035] [Figure 4] FIG. 4 is a schematic diagram of a simplified block diagram of a communication system (400) according to one embodiment.

[0036] [Diagram 5] FIG. 2 is a schematic diagram of a simplified block diagram of a decoder according to one embodiment.

[0037] [Figure 6] FIG. 2 is a schematic diagram of a simplified block diagram of an encoder according to one embodiment.

[0038] [Figure 7] 4 shows a block diagram of an encoder according to another embodiment;

[0039] [Figure 8] 4 shows a block diagram of a decoder according to another embodiment;

[0040] [Figure 9A] FIG. 2 is a first example diagram of a position-dependent intra-prediction combined (PDPC) mode, according to some embodiments of the present disclosure.

[0041] [Figure 9B] FIG. 13 is a second example diagram of a PDPC mode, according to some embodiments of the present disclosure.

[0042] [Figure 9C] FIG. 13 is a third example diagram of a PDPC mode, according to some embodiments of the present disclosure.

[0043] [Figure 9D] FIG. 4 is a fourth exemplary diagram of a PDPC mode, according to some embodiments of the present disclosure.

[0044] [Figure 10] FIG. 1 is a schematic diagram of decoder-side intra mode derivation (DIMD) with fusion according to some embodiments of the present disclosure.

[0045] [Figure 11] FIG. 1 is a schematic diagram of template-based intra-mode derivation (TIMD) in accordance with some embodiments of the present disclosure.

[0046] [Figure 12] FIG. 1 is a schematic diagram of a TIMD with fusion, according to some embodiments of the present disclosure.

[0047] [Figure 13] FIG. 1 is a schematic diagram of DIMD with fusion according to some embodiments of the present disclosure.

[0048] [Figure 14] FIG. 1 is a schematic diagram of a TIMD with fusion, according to some embodiments of the present disclosure.

[0049] [Figure 15A]1 is an example process for DIMD with fusion according to some embodiments of the present disclosure.

[0050] [Figure 15B] 1 is an example process for a TIMD mode, according to some embodiments of the present disclosure.

[0051] [Figure 16] 1 shows a flowchart outlining an exemplary decoding process according to some embodiments of the present disclosure.

[0052] [Figure 17] 1 shows a flowchart outlining an exemplary encoding process according to some embodiments of the present disclosure.

[0053] [Figure 18] FIG. 1 is a schematic diagram of a computer system according to one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0054] FIG. 3 illustrates a simplified block diagram of a communication system (300) according to one embodiment of the present disclosure. The communication system (300) includes a plurality of terminal devices that can communicate with each other, for example, via a network (350). For example, the communication system (300) includes a first pair of terminal devices (310) and (320) interconnected via the network (350). In the example of FIG. 3, the first pair of terminal devices (310) and (320) perform a unidirectional transmission of data. For example, the terminal device (310) may code video data (e.g., a stream of video pictures captured by the terminal device (310)) for transmission to another terminal device (320) via the network (350). The encoded video data may be transmitted in the form of one or more coded video bitstreams. The terminal device (320) may receive the coded video data from the network (350), decode the coded video data to reconstruct the video pictures, and display the video pictures according to the reconstructed video data. Unidirectional data transmission is commonplace, such as in media service applications.

[0055] In another example, the communication system (300) includes a second pair of terminal devices (330) and (340) for bidirectional transmission of coded video data, such as may occur during a video conference. For the bidirectional transmission of data, in one example, each of the terminal devices (330) and (340) may code video data (e.g., a stream of video pictures captured by the terminal device) for transmission to the other of the terminal devices (330) and (340) over the network (350). Each of the terminal devices (330) and (340) may also receive coded video data transmitted by the other of the terminal devices (330) and (340), decode the coded video data to recover the video pictures, and display the video pictures on an accessible display device in accordance with the recovered video data.

[0056] In FIG. 3, the terminal devices (310), (320), (330), and (340) may be depicted as servers, personal computers, and smartphones, although the principles of the present disclosure are not so limited. The embodiments of the present disclosure find application with laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. The network (350) represents any number of networks that carry coded video data between the terminal devices (310), (320), (330), and (340), including, for example, wired and / or wireless communication networks. The communication network (350) may exchange data in circuit-switched and / or packet-switched channels. Exemplary networks include telecommunications networks, local area networks, wide area networks, and / or the Internet. For purposes of this description, the architecture and topology of the network (350) is not important to the operation of the present disclosure, unless otherwise described below.

[0057] 4 shows an arrangement of a video encoder and a video decoder in a streaming environment as an example for application of the disclosed subject matter. The disclosed subject matter is similarly applicable to other video-enabled applications including, for example, video conferencing, digital TV, storage of compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0058] The streaming system may include a video source (401) that creates a stream of uncompressed video pictures (402), a capture subsystem (413) that may include, for example, a digital camera. In one example, the stream of video pictures (402) includes samples taken by a digital camera. The stream of video pictures (402), depicted as a thick line to highlight a high data volume when compared to the encoded video data (404) (or coded video bitstream), may be processed by an electronic device (420) that includes a video encoder (403) coupled to the video source (401). The video encoder (403) may include hardware, software, or a combination thereof to enable or implement aspects of the disclosed subject matter, as described in more detail below. The encoded video data (404) (encoded bitstream (404)), depicted as a thin line to highlight a lower data volume when compared to the stream of video pictures (402), may be stored in a streaming server (405) for future use. One or more streaming client subsystems, such as the client subsystems (406) and (408) of FIG. 4, can access the streaming server (405) to retrieve copies (407, 409) of the encoded video data (404). The client subsystem (406) can include a video decoder (410), for example within an electronic device (430). The video decoder (410) decodes an incoming copy (407) of the encoded video data and creates an outgoing stream (411) of video pictures that can be rendered on a display (412) (e.g., a display screen) or other rendering device (not shown). In some streaming systems, the encoded video data (404), (407), and (409) (e.g., a video bitstream) can be encoded according to a particular video coding / compression standard. Examples of these standards include ITU-T Recommendation H.265.In one example, the developing video coding standard is informally known as Versatile Video Coding (VVC), and the disclosed subject matter may be used in the context of VVC.

[0059] It is noted that the electronic devices (420) and (430) can include other components (not shown). For example, the electronic device (420) can include a video decoder (not shown), and the electronic device (430) can also include a video encoder (not shown).

[0060] 5 shows a block diagram of a video decoder (510) according to one embodiment of the present disclosure. The video decoder (510) may be included in an electronic device (530). The electronic device (530) may include a receiver (531) (e.g., receiving circuitry). The video decoder (510) may be used in place of the video decoder (410) in the example of FIG. 4.

[0061] The receiver (531) may receive one or more coded video sequences to be decoded by the decoder (510), or in the same or another embodiment, one coded video sequence at a time, with the decoding of each coded video sequence being independent of the other coded video sequences. The coded video sequences may be received from a channel (501), which may be a hardware / software link to a storage device that stores the encoded video data. The receiver (531) may receive the encoded video data together with other data, such as coded audio data and / or auxiliary data streams, which may be transferred using respective entities (not shown). The receiver (531) may separate the coded video sequences from the other data. To combat network jitter, a buffer memory (515) may be coupled between the receiver (531) and the entropy decoder / analyzer (520) (hereinafter "analyzer (520)"). In certain applications, the buffer memory (515) is part of the video decoder (510). In other cases, it may be external to the video decoder (510) (not shown). In yet other cases, there may be a buffer memory (not shown) external to the video decoder (510), e.g., to combat network jitter, and additionally, there may be another buffer memory (515) within the video decoder (510), e.g., to handle playback timing. When the receiver (531) is receiving data from a storage / forwarding device with sufficient bandwidth and controllability, or from an isochronous network, the buffer (515) may not be needed or may be small. For use with best-effort packet networks such as the Internet, the buffer memory (515) may be needed and may be relatively large, advantageously of adaptive size, and may be at least partially implemented in an operating system or similar element (not shown) external to the video decoder (510).

[0062] The video decoder (510) may include a parser (520) for reconstructing symbols (521) from the coded video sequence. These categories of symbols include information used to manage the operation of the video decoder (510) and potential information for controlling a rendering device that is not an integral part of the electronic device (530), such as a rendering device (512) (e.g., a display screen), but may be coupled to the electronic device (530) as shown in FIG. 5. The rendering device control information may be in the form of Supplemental Enhancement Information (SEI) messages or video usability information (VUI) parameter set fragments (not shown). The parser (520) may parse / entropy decode the received coded video sequence. The coding of the coded video sequence may follow a video coding technique or standard and may follow various principles including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The analyzer (520) may extract from the coded video sequence a set of subgroup parameters for at least one of the subgroups of pixels in the video decoder based on the at least one parameter corresponding to the group. The subgroups may include groups of pictures (GOPs), pictures, tiles, slices, macroblocks, coding units (CUs), blocks, transform units (TUs), prediction units (PUs), etc. The analyzer (520) may also extract information from the coded video sequence, such as transform coefficients, quantization parameter values, motion vectors, etc.

[0063] The parser (520) may perform an entropy decoding / parsing operation on the video sequence received from the buffer memory (515) to produce symbols (521).

[0064] The reconstruction of the symbols (521) can involve several different units, depending on the type of coded video picture or portion thereof (e.g., inter-picture and intra-picture, inter-block and intra-block) and other factors. Which units are involved and how they are involved can be controlled by subgroup control information parsed from the coded video sequence by the parser (520). The flow of such subgroup control information between the parser (520) and the following units is not depicted for clarity.

[0065] In addition to the functional blocks already mentioned, the decoder (510) may be conceptually subdivided into several functional units, as described below. In a practical implementation operating under commercial constraints, many of these units will interact closely with each other and may be at least partially integrated with each other. However, for purposes of describing the disclosed subject matter, a conceptual subdivision into the following functional units is appropriate:

[0066] The first unit is a scalar / inverse transform unit (551), which receives the quantized transform coefficients as symbols (521) from the analyzer (520), along with control information including the transform to use, block size, quantization factor, quantization scaling matrix, etc. The scalar / inverse transform unit (551) can output blocks containing sample values ​​that can be input to an aggregator (555).

[0067] In some cases, the output samples of the scaler / inverse transform (551) may relate to intra-coded blocks, i.e., blocks that do not use prediction information from a previously reconstructed picture, but may use prediction information from a previously reconstructed portion of the current picture. Such prediction information may be provided by an intra-picture prediction unit (552). In some cases, the intra-picture prediction unit (552) generates blocks of the same size and shape of the block being reconstructed using surrounding information fetched from a current picture buffer (558) that has already been reconstructed. The current picture buffer (558) buffers, for example, a partially reconstructed current picture and / or a fully reconstructed current picture. The aggregator (555) adds the prediction information generated by the intra-prediction unit (552) to the output sample information provided by the scaler / inverse transform unit (551), possibly on a sample-by-sample basis.

[0068] In other cases, the output samples of the scaler / inverse transform unit (551) may be inter-coded and potentially related to a motion compensated block. In such cases, the motion compensated prediction unit (553) may access the reference picture memory (557) to fetch samples used for prediction. After motion compensation of the fetched samples according to the symbols (521) related to the block, these samples may be added by the aggregator (555) to the output of the scaler / inverse transform unit (551) (in this case referred to as residual samples or residual signals) to generate output sample information. The addresses in the reference picture memory (557) from which the motion compensation unit (553) fetches the prediction samples may be controlled by motion vectors available to the motion compensation unit in the form of symbols (521), which may have, for example, X, Y, and reference picture components. Motion compensation may also include interpolation of sample values ​​fetched from the reference picture memory (557) when sub-sample accurate motion vectors are in use, motion vector prediction mechanisms, etc.

[0069] The output samples of the aggregator (555) can be subjected to various loop filtering techniques in the loop filter unit (556). Video compression techniques are controlled by parameters contained in the coded video sequence (also called a coded video bitstream) and made available to the loop filter unit (556) as symbols (521) from the analyzer (520), but can include in-loop filter techniques that can be responsive to meta-information obtained during decoding of previous portions (in decoding order) of the coded picture or coded video sequence, and can also be responsive to previously reconstructed loop filtered sample values.

[0070] The output of the loop filter unit (556) may be a sample stream that can be output to a rendering device (512) and also stored in a reference picture memory (557) for use in future inter-picture prediction.

[0071] A particular coded picture, once fully reconstructed, may be used as a reference picture for future prediction. For example, once a coded picture corresponding to a current picture is fully reconstructed and the coded picture is identified as a reference picture (e.g., by the analyzer (520)), the current picture buffer (558) may become part of the reference picture buffer (557), and a fresh current picture buffer may be reallocated before beginning reconstruction of the next coded picture.

[0072] The video decoder (510) may perform decoding operations according to a predetermined video compression technique in a standard such as ITU-T Rec. H.265. A coded video sequence may conform to the syntax specified by the video compression technology or standard being used in the sense that the coded video sequence conforms to both the syntax of the video compression technology or standard and to a profile documented in the video compression technology or standard. In particular, a profile may select a particular tool from all tools available in the video compression technology or standard as the only tool available for use under that profile. Also, what is required for compliance may be that the complexity of the coded video sequence is within a range defined by a level of the video compression technology or standard. In some cases, the level limits the maximum picture size, maximum picture rate, maximum reconstruction sample rate (e.g., measured in megasamples per second), maximum reference picture size, etc. The limits set by the level may in some cases be further restricted through metadata for HRD (Hypothetical Reference Decoder) usage and buffer management signaled in the coded video sequence.

[0073] In one embodiment, the receiver (531) may receive additional (redundant) data along with the encoded video. The additional data may be included as part of the coded video sequence. The additional data may be used by the video decoder (510) to properly decode the data and / or to more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0074] 6 shows a block diagram of a video encoder (603) according to one embodiment of the present disclosure. The video encoder (603) is included in an electronic device (620). The electronic device (620) includes a transmitter (640) (e.g., transmitting circuitry). The video encoder (603) may be used in place of the video encoder (403) in the example of FIG. 4.

[0075] The video encoder (603) may receive video samples from a video source (601) (not part of the electronic device (620) in the example of FIG. 6) that may capture video images to be coded by the video encoder (603). In another example, the video source (601) is part of the electronic device (620).

[0076] The video source (601) may provide a source video sequence to be coded by the video encoder (603) in the form of a digital video sample stream that may be of any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, ...), any color space (e.g., BT.601 YCrCB, RGB, ...), and any suitable sampling structure (e.g., YCrCb 4:2:0, YCrCb 4:4:4). In a media serving system, the video source (601) may be a storage device that stores previously prepared video. In a video conferencing system, the video source (601) may be a camera that captures local image information as a video sequence. The video data may be provided as a number of individual pictures that give motion when viewed in sequence. The pictures themselves may be organized as a spatial array of pixels, each pixel may contain one or more samples depending on the sampling structure, color space, etc. in use. Those skilled in the art can easily understand the relationship between pixels and samples. The following description focuses on samples.

[0077] According to one embodiment, the video encoder (603) may code and compress pictures of a source video sequence into a coded video sequence (643) in real-time or under any other time constraint required by an application. Enforcing an appropriate coding rate is one function of the controller (650). In some embodiments, the controller (650) controls and is operatively coupled to other functional units as described below. Coupling is not shown for clarity. Parameters set by the controller (650) may include rate control related parameters (picture skip, quantizer, lambda value for rate distortion optimization techniques, ...), picture size, group of pictures (GOP) layout, maximum motion vector search range, etc. The controller (650) may be configured with other suitable functions related to the video encoder (603) that are optimized for a particular system design.

[0078] In some embodiments, the video encoder (603) is configured to operate in a coding loop. As a rather simplified description, in one example, the coding loop can include a source coder (630) (e.g., responsible for generating symbols, such as a symbol stream, based on an input picture to be coded and a reference picture) and a (local) decoder (633) embedded in the video encoder (603). The decoder (633) reconstructs the symbols to create sample data in a manner similar to that created by the (remote) decoder (so that any compression between the symbols and the coded video bitstream is lossless in the video compression techniques considered in the disclosed subject matter). The reconstructed sample stream (sample data) is input to a reference picture memory (634). Since the decoding of the symbol stream produces bit-for-bit accurate results that are independent of the decoder location (local or remote), the contents in the reference picture memory (634) are also bit-for-bit accurate between the local and remote encoders. In other words, the prediction part of the encoder "sees" exactly the same sample values ​​as the reference picture samples that the decoder "sees" when using prediction during decoding. This basic principle of reference picture synchrony (e.g., if synchrony cannot be maintained due to channel errors, drift will result) is used in some related technologies as well.

[0079] The operation of the "local" decoder (633) can be the same as that of a "remote" decoder, such as the video decoder (510), already described in detail above in conjunction with Figure 5. With brief reference again to Figure 5, because symbols are available and the encoding / decoding of symbols into a coded video sequence can be lossless by the entropy coder (645) and analyzer (520), the entropy decoding portion of the video decoder (510), including the buffer memory (515) and analyzer (520), may not be fully implemented in the local decoder (633).

[0080] At this point, it is recognized that any decoder technique other than analysis / entropy decoding present in a decoder must necessarily be present in a corresponding encoder in substantially the same functional form. For this reason, the disclosed subject matter focuses on the decoder operation. A description of the encoder technique can be omitted since it is the inverse of the decoder technique described generically. Only in certain areas is a more detailed description necessary and is provided below.

[0081] As part of its operation, the source coder (630) may perform motion compensated predictive coding, which predictively codes the input picture with reference to one or more previously coded pictures from the video sequence designated as “reference pictures.” In this manner, the coding engine (632) codes differences between pixel blocks of the input picture and pixel blocks of reference pictures that may be selected as predictive references for the input picture.

[0082] The local video decoder (633) may decode the coded video data of pictures that may be designated as reference pictures based on the symbols created by the source coder (630). The operation of the coding engine (632) may advantageously be a lossy process. When the coded video data may be decoded in a video decoder (not shown in FIG. 6), the reconstructed video sequence may be a replica of the source video sequence, typically with some errors. The local video decoder 633 may replicate the decoding process that may be performed on the reference pictures by the video decoder, such that the reconstructed reference pictures are stored in a reference picture cache (634). In this way, the video encoder (603) may locally store copies of reconstructed reference pictures that have common content with the reconstructed reference pictures that would be obtained by the far-end video decoder (in the absence of transmission errors).

[0083] The predictor (635) may perform a predictive search for the coding engine (632). That is, for a new picture to be coded, the predictor (635) may search the reference picture memory (634) for sample data (as candidate reference pixel blocks) or specific metadata, such as reference picture motion vectors, block shapes, etc., that serve as suitable predictive references for the new picture. The predictor (635) may operate on a sample block-by-pixel block basis to find suitable predictive references. In some cases, as determined by the results obtained by the predictor (635), the input picture may have predictive references drawn from multiple reference pictures stored in the reference picture memory (634).

[0084] The controller (650) may manage the coding operations of the source coder (630), including, for example, setting the parameters and subgroup parameters used to encode the video data.

[0085] The output of all the aforementioned functional units is subjected to entropy coding in an entropy coder (645), which converts the symbols produced by the various functional units into a coded video sequence by lossless compression according to techniques such as Huffman coding, variable length coding, arithmetic coding, etc.

[0086] The transmitter (640) may buffer the coded video sequence created by the entropy coder (645) and prepare it for transmission over a communication channel (660), which may be a hardware / software link to a storage device that will store the encoded video data. The transmitter (640) may merge the coded video data from the video coder (603) with other data to be transmitted, such as coded audio data and / or auxiliary data streams (sources not shown).

[0087] A controller (650) may manage the operation of the video encoder (603). During coding, the controller (650) may assign a particular coded picture type to each coded picture, which may affect the coding technique that may be applied to the respective picture. For example, pictures are often assigned as one of the following picture types:

[0088] An Intra picture (I-picture) may be one that can be coded and decoded without using any other picture in a sequence as a source of prediction. Some video codecs allow different types of Intra pictures, including, for example, Independent Decoder Refresh Pictures. Those skilled in the art understand these variations of I-pictures and their respective applications and characteristics.

[0089] A predictive picture (P picture) may be one that can be coded and decoded using intra- or inter-prediction, using at most one motion vector and reference index to predict the sample values ​​of each block.

[0090] Bidirectionally predicted pictures (B-pictures) may be coded and decoded using intra- or inter-prediction, using at most two motion vectors and reference indices to predict the sample values ​​of each block. Similarly, multiple predicted pictures may use more than two reference pictures and associated metadata for the reconstruction of a block.

[0091] A source picture is typically spatially subdivided into multiple sample blocks (e.g., blocks of 4x4, 8x8, 4x8, or 16x16 samples each) and may be coded on a block-by-block basis. Blocks may be predictively coded with reference to other (already coded) blocks as determined by the coding assignment applied to the respective picture of the block. For example, blocks of an I-picture may be non-predictively coded, or they may be predictively coded with reference to already coded blocks of the same picture (spatial or intra prediction). Pixel blocks of a P-picture may be predictively coded via spatial prediction or via temporal prediction with reference to one previously coded reference picture. Blocks of a B-picture may be coded via spatial prediction or via temporal prediction with reference to one or two previously coded reference pictures.

[0092] The video encoder (603) may perform coding operations in accordance with a given video coding technique or standard, such as ITU-T Rec. H.265. In operation, the video encoder (603) may perform various compression operations, including predictive coding operations that exploit temporal and spatial redundancy in the input video sequence, so that the coded video data may conform to a syntax specified by the video coding technology or standard being used.

[0093] In one embodiment, the transmitter (640) may transmit additional data along with the encoded video. The source coder (630) may include such data as part of the coded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant pictures and slices, SEI messages, VUI parameter set fragments, etc.

[0094] A video may be captured as multiple source pictures (video pictures) in a time sequence. Intra-picture prediction (often abbreviated as intra-prediction) exploits spatial correlation in a given picture, while inter-picture prediction exploits correlation (temporal or other) between pictures. In one example, a particular picture under coding / decoding, called the current picture, is partitioned into blocks. When a block in the current picture resembles a reference block in a previously coded and still buffered reference picture in the video, the block in the current picture may be coded by a vector called a motion vector. A motion vector points to a reference block in a reference picture and may have a third dimension that identifies the reference picture when multiple reference pictures are in use.

[0095] In some embodiments, bi-prediction techniques may be used for inter-picture prediction. According to bi-prediction techniques, two reference pictures, such as a first reference picture and a second reference picture, both of which are previous in decoding order (but may be past and future, respectively, in display order) of a current picture in a video, are used. A block in the current picture may be coded by a first motion vector that points to a first reference block in the first reference picture and a second motion vector that points to a second reference block in the second reference picture. A block may be predicted by a combination of the first reference block and the second reference block.

[0096] Furthermore, to improve coding efficiency, merge mode techniques may be used for inter-picture prediction.

[0097] According to some embodiments of the present disclosure, predictions such as inter-picture prediction and intra-picture prediction are performed on a block-by-block basis. For example, according to the HEVC standard, pictures in a sequence of video pictures are partitioned into coding tree units (CTUs) for compression, and the CTUs within a picture have the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. In general, a CTU includes three coding tree blocks (CTBs), one luma CTB and two chroma CTBs. Each CTU may be recursively 4-tree split into one or more coding units (CUs). For example, a CTU of 64×64 pixels may be split into one CU of 64×64 pixels, four CUs of 32×32 pixels, or 16 CUs of 16×16 pixels. In one example, each CU is analyzed to determine a prediction type for the CU, such as an inter-prediction type or an intra-prediction type. A CU is split into one or more prediction units (PUs) depending on temporal and / or spatial predictability. In general, each PU includes a luma prediction block (PB) and two chroma PBs. In one embodiment, prediction operations in coding (encoding / decoding) are performed in units of prediction blocks. Taking a luma prediction block as an example of a prediction block, the prediction block includes a matrix of values ​​(e.g., luma values) for pixels, such as 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, etc.

[0098] 7 shows a diagram of a video encoder (703) according to another embodiment of the present disclosure. The video encoder (703) is configured to receive a processed block (e.g., a predictive block) of sample values ​​in a current video picture in a sequence of video pictures and to encode the processed block into a coded picture that is part of a coded video sequence. In one example, the video encoder (703) is used in place of the video encoder (403) in the example of FIG. 4.

[0099] In an HEVC example, the video encoder (703) receives a matrix of sample values ​​for a processing block, such as a predictive block of 8x8 samples. The video encoder (703) determines whether the processing block is best coded using intra mode, inter mode, or bi-predictive mode, e.g., using rate-distortion optimization. When the processing block is coded in intra mode, the video encoder (703) may use intra prediction techniques to encode the processing block into a coded picture, and when the processing block is coded in inter mode or bi-predictive mode, the video encoder (703) may use inter prediction techniques or bi-predictive techniques, respectively, to encode the processing block into a coded picture. In certain video coding techniques, the merge mode may be an inter-picture prediction sub-mode in which a motion vector is derived from one or more motion vector predictors without the benefit of coded motion vector components outside the predictors. In certain other video coding techniques, there may be motion vector components applicable to the current block. In one embodiment, the video encoder (703) includes other components, such as a mode decision module (not shown) for determining the mode of the processing block.

[0100] In the example of FIG. 7, the video encoder (703) includes an inter-encoder (730), an intra-encoder (722), a residual calculator (723), a switch (726), a residual encoder (724), a general controller (721), and an entropy encoder (725), coupled together as shown in FIG.

[0101] The inter-encoder (730) is configured to receive samples of a current block (e.g., a processing block), compare the block to one or more reference blocks in a reference picture (e.g., blocks in a previous picture and a subsequent picture), generate inter-prediction information (e.g., a description of redundant information due to the inter-encoding technique, motion vectors, merge mode information), and calculate an inter-prediction result (e.g., a prediction block) based on the inter-prediction information using any suitable technique. In some examples, the reference picture is a reference picture that has been decoded based on the encoded video information.

[0102] The intra encoder (722) receives samples of a current block (e.g., a processing block), possibly compares the block to previously coded blocks in the same picture, generates transformed and quantized coefficients, and possibly also generates intra prediction information (e.g., intra prediction direction information according to one or more intra encoding techniques). In one example, the intra encoder (722) also calculates intra prediction results (e.g., prediction blocks) based on the intra prediction information and reference blocks in the same picture.

[0103] The generic controller (721) is configured to determine generic control data and control other components of the video encoder (703) based on the generic control data. In one example, the generic controller (721) determines the mode of the block and provides a control signal to the switch (726) based on the mode. For example, when the mode is an intra mode, the generic controller 721 controls the switch 726 to select the intra mode result used by the residual calculator 723, and controls the entropy encoder 725 to select the intra prediction information and include the intra prediction information in the bitstream, and when the mode is an inter mode, the generic controller 721 controls the switch 726 to select the inter prediction result used by the residual calculator 723, and controls the entropy encoder 725 to select the inter prediction information and include the inter prediction information in the bitstream.

[0104] The residual calculator (723) is configured to calculate a difference (residual data) between a received block and a prediction result selected from the intra-encoder (722) or the inter-encoder (730). The residual encoder (724) operates on the residual data and is configured to encode the residual data to generate transform coefficients. In one example, the residual encoder (724) is configured to transform the residual data from a spatial domain to a frequency domain to generate transform coefficients. The transform coefficients then undergo a quantization process to obtain quantized transform coefficients. In various embodiments, the video encoder (703) also includes a residual decoder (728). The residual decoder (728) is configured to perform an inverse transform and generate decoded residual data. The decoded residual data may be suitably used by the intra-encoder (722) and the inter-encoder (730). For example, the inter-encoder (730) can generate decoded blocks based on the decoded residual data and the inter-prediction information, and the intra-encoder (722) can generate decoded blocks based on the decoded residual data and the intra-prediction information. The decoded blocks are suitably processed to generate decoded pictures, which may be buffered in a memory circuit (not shown) and used as reference pictures in some examples.

[0105] The entropy encoder (725) is configured to format a bitstream to include the encoded block. The entropy encoder (725) is configured to include various information according to a suitable standard, such as the HEVC standard. In one example, the entropy encoder (725) is configured to include general control data, selected prediction information (e.g., intra-prediction information or inter-prediction information), residual information, and other suitable information in the bitstream. It is noted that, according to the disclosed subject matter, when coding a block in a merged sub-mode of either an inter mode or a bi-prediction mode, no residual information is present.

[0106] 8 shows a diagram of a video decoder (810) according to another embodiment of the present disclosure. The video decoder (810) is configured to receive coded pictures that are part of a coded video sequence and to decode the coded pictures to generate reconstructed pictures. In one example, the video encoder (810) is used in place of the video encoder (410) in the example of FIG. 4.

[0107] In the example of FIG. 8, the video decoder (810) includes an entropy decoder (871), an inter decoder (880), a residual decoder (873), a reconstruction module (874), and an intra decoder (872), coupled together as shown in FIG.

[0108] The entropy decoder (871) may be configured to reconstruct from the coded picture certain symbols representing syntax elements of which the coded picture is composed. Such symbols may include, for example, prediction information (e.g., intra- or inter-prediction information) that may identify the mode in which the block is coded (e.g., intra- or inter-prediction mode, merged submode or the latter two in another submode), certain samples or metadata used for prediction by the intra- or inter-decoder (872) or (880), respectively, residual information, e.g., in the form of quantized transform coefficients, etc. As an example, when the prediction mode is an inter- or bi-prediction mode, the inter-prediction information is provided to the inter-decoder (880), and when the prediction type is an intra-prediction type, the intra-prediction information is provided to the intra-decoder (872). The residual information may undergo inverse quantization and is provided to the residual decoder (873).

[0109] The inter decoder (880) is configured to receive the inter prediction information and to generate inter prediction results based on the inter prediction information.

[0110] The intra decoder (872) is configured to receive intra prediction information and to generate a prediction result based on the intra prediction information.

[0111] The residual decoder (873) is configured to perform inverse quantization to extract inverse quantized transform coefficients, and process the inverse quantized transform coefficients to transform the residual from the frequency domain to the spatial domain. The residual decoder (873) may also require certain control information, including Quantizer Parameters (QP), which may be provided by the entropy decoder (871) (this may be only low volume control information, so the data path is not depicted).

[0112] The reconstruction module (874) is configured to combine, in the spatial domain, the residual as output by the residual decoder (873) and the prediction result (possibly as output by an inter- or intra-prediction module) to form a reconstructed block, which may be part of a reconstructed picture, which may be part of a reconstructed video. It should be noted that other suitable operations, such as a deblocking operation, may be performed to improve visual quality.

[0113] It should be noted that the video encoders (403), (603), and (703) and the video decoders (410), (510), and (810) may be implemented using any suitable technology. In one embodiment, the video encoders (403), (603), and (703) and the video decoders (410), (510), and (810) may be implemented using one or more integrated circuits. In another embodiment, the video encoders (403), (603), and (603) and the video decoders (410), (510), and (810) may be implemented using one or more processors executing software instructions.

[0114] This disclosure includes improvements related to decoder-side intra mode derivation (DIMD).

[0115] ITU-T VCEG (Q6 / 16) and ISO / IEC MPEG (JTC 1 / SC 29 / WG 11) published the H.265 / HEVC (High Efficiency Video Coding) standard in 2013 (version 1), 2014 (version 2), 2015 (version 3), and 2016 (version 4). In 2015, these two standardization organizations jointly formed the Joint Video Exploration Team (JVET) to explore the possibility of developing the next video coding standard beyond HEVC. In April 2018, JVET officially started the standardization process for the next generation of video coding beyond HEVC. This new standard was named Versatile Video Coding (VVC), and JVET was renamed Joint Video Expert Team. In July 2020, H.266 / VVC version 1 was completed. In January 2021, an ad hoc group was formed to investigate enhanced compression beyond VVC capabilities.

[0116] In VVC, the results of intra prediction based on DC, planar, and some angular modes may be further modified by the Position-Dependent Intra Prediction Combined (PDPC) method. PDPC is a combined intra prediction method with boundary reference samples and HEVC-style intra prediction using filtered boundary reference samples. PDPC may be applied without signaling to intra modes of planar, DC, intra angles below horizontal, and intra angles below (i) vertical, and (ii) 80. PDPC may not be applied if the current block is under block-based delta pulse code modulation (BDPCM) mode. PDPC may not be applied if the current block is under multiple reference line (MRL) intra prediction mode and the MRL index is greater than 0.

[0117] According to PDPC, a predicted sample (e.g., pred(x′,y′)) of a current sample (x,y) may be predicted using an intra prediction mode (e.g., DC, planar, or angular) and a linear combination of reference samples of the current block according to Equation 1 as follows:

number

[0118] When PDPC is applied to DC, planar, horizontal, and vertical intra modes, additional boundary filters may not be required. However, additional boundary filters, such as DC mode boundary filters or horizontal / vertical mode edge filters, may be required in HEVC. The additional boundary filters may be DC mode boundary filters or horizontal / vertical mode edge filters. The PDPC process for DC and planar modes may be identical. For angular modes, the left or top reference samples may not be used if the current angular mode is horizontal (e.g., HOR_IDX) or vertical (e.g., VER_IDX), respectively. The weights and scale factors of the PDPC may depend on the prediction mode and block size. PDPC may be applied to blocks with both width and height of four or more.

[0119] 9A to 9D show the reference sample (R (x,-1) and R (-1,y) ) is shown. A predicted sample pred(x′, y′) may be located at (x′, y′) in the prediction block 902. R (-1,y) , R (x,-1) , and R (-1,-1)may represent reference samples located to the left, top, and top left of the prediction block 902, respectively. Figure 9A illustrates a diagonal top-right mode of PDPC. Figure 9B illustrates a diagonal bottom-left mode of PDPC. Figure 9C illustrates an adjacent diagonal top-right mode of PDPC. Figure 9D illustrates an adjacent diagonal bottom-left mode of PDPC. In one example as shown in Figure 9A, for the diagonal mode, the reference sample R (x,-1) The coordinate x of is given by x = x´ + y´ + 1, and the reference sample R -1,y The coordinate y of can be given by y=x′+y′+1. For other angular modes, as shown in FIGS. 9C and 9D, the reference sample R (x,-1) and R (-1,y) can be located at fractional sample positions. (x,-1) and R (-1,y) When lies at a fractional sample position, the sample value at the nearest integer sample position may be used.

[0120] In some embodiments, the reference sample R (x,-1) It may be applied when PDPC is not applied due to the non-availability of secondary reference samples such as. Gradient-based PDPC as provided in JVET-Q0391 may be an intra-prediction mode extended from horizontal / vertical mode. Thus, the PDPC weights (e.g., wT and wL) and the scale parameters (e.g., nScale) for determining the attenuation in the PDPC weights with respect to the distance from the left / top boundary, respectively, may be set equal to the corresponding parameters in horizontal / vertical mode. Bilinear interpolation may be applied when the secondary reference samples are at fractional sample positions.

[0121] In DIMD, intra modes may be derived using associated syntax elements signaled in the bitstream, or intra modes may be derived at the decoder side without using associated syntax elements signaled in the bitstream. Many methods may be used to derive decoder-side intra modes, but the term "decoder-side intra mode derivation" is not limited to the methods described in this disclosure.

[0122] In DIMD, two intra modes from multiple candidate intra modes for the current CU / PU may be derived from reconstructed neighboring samples of the current CU / PU. Texture gradient analysis may be performed at both the encoder and decoder to generate multiple candidate intra modes based on the reconstructed neighboring samples. Each of the multiple candidate intra modes may be associated with a respective history of gradients (or a respective gradient). Two intra modes (e.g., intraMode1 and intraMode2) with the highest history of gradients (or highest gradient of histogram diagram) may be selected. The intra mode predictors of the selected two intra modes (e.g., intraMode1 and intraMode2) may be combined with the planar mode predictor using a weighted sum. A final intra mode predictor for the current CU / PU may be formed based on the combination of intraMode1, intraMode2, and planar.

[0123] Table 1 shows an exemplary DIMD signaling process. As shown in Table 1, a DIMD flag (e.g., DIMD_flag) may be signaled before an ISP flag (e.g., ISP_flag). When the DIMD flag is 1 or true, it may indicate that the current CU / PU uses DIMD, and the ISP flag may be further parsed to verify whether ISP applies to the current CU / PU. When the DIMD flag is not 1 (or false), it may indicate that the current CU / PU does not use DIMD. Thus, syntax elements related to other intra-coding tools (e.g., MIP, MRL, MPM, etc.) may be parsed in the decoder.

[0124] The context modeling of the DIMD flag may depend on the neighboring CU / PU. For example, the context modeling of the DIMD flag may depend on (i) the availability of the left neighboring CU / PU or the top neighboring CU / PU, and (ii) whether the left neighboring CU / PU or the top neighboring CU / PU also uses DIMD. If either the left neighboring CU / PU or the top neighboring CU / PU exists and uses DIMD, the context index (e.g., ctxIdx) may be 1. When both the left neighboring CU / PU and the top neighboring CU / PU exist and use DIMD, the context index (e.g., ctxIdx) may be 2. Otherwise, the context index (e.g., ctxIdx) may be 0. [Table 1]

[0125] An exemplary fusion process (1000) of intraMode1, intraMode2, and planar based on DIMD is shown in FIG. 10. As shown in FIG. 10, a first intra mode (e.g., intraMode1) and a second intra mode (e.g., intraMode2) may be obtained based on DIMD. Furthermore, a first angle prediction (1002) may be obtained based on the first intra mode, a second angle prediction (1004) may be obtained based on the second intra mode, and a third prediction (1006) may be obtained based on a third intra mode (e.g., planar mode). When PDPC is applicable, a first PDPC process (1008) may be applied to the first prediction (1002) to generate a refined first prediction. A second PDPC process (1010) may be applied to the second prediction (1004) to generate a refined second prediction. An applicable PDPC is applicable to the planar mode, but a PDPC does not have to be applied in the fusion process (1000). A fusion process (1012) can be performed to generate a weighted combination of the refined first prediction, the refined second prediction, and the third prediction.

[0126] Template-based intra mode derivation (TIMD) can use a reference sample of the current CU as a template and select an intra mode from a set of candidate intra prediction modes associated with the TIMD. The selected intra mode can be determined as the best intra mode based on a cost function, for example. As shown in FIG. 11, a reconstructed sample in a neighborhood of the current CU (1102) can be used as a template (1104). The reconstructed sample in the template (1104) can be compared with a predicted sample of the template (1104). The predicted sample can be generated using a reference sample (1106) of the template (1104). The reference sample (1106) can be a reconstructed sample in a neighborhood around the template (1104). A cost function (e.g., sum of absolute differences (SAD) or sum of absolute transform differences (SATD)) can be used to calculate the cost (or distortion) between the predicted sample in the template (1104) and the reconstructed sample based on each of the candidate intra prediction modes. The intra prediction mode with the least cost (or distortion) may be selected as the intra prediction mode (eg, the best intra prediction mode) for inter predicting the current CU (1102).

[0127] Table 2 shows an example coding process associated with TIMD. As shown in Table 2, when the DIMD flag (e.g., DIMD_flag) is not 1 (or not true), the TIMD flag (e.g., TIMD_flag) may be signaled. When the DIMD flag is 1 (or true), the current CU / PU uses DIMD, and the ISP flag (e.g., ISP_flag) may be parsed to determine whether ISP is used for the current CU / PU. When the DIMD flag is not 1 (or false), the TIMD flag may be parsed. When the TIMD flag is 1, TIMD may be applied to the current CU / PU without applying other intra-coding tools (e.g., ISP is not allowed when TIMD is used). When the TIMD flag is not 1 (or false), syntax elements related to other intra-coding tools (e.g., MIP, MRL, MPM, etc.) may be parsed at the decoder. [Table 2]

[0128] According to TIMD, the first two intra prediction modes with the smallest SATD may be selected as the TIMD mode. These two TIMD modes may be fused with a weight to generate a weighted intra prediction. The weighted intra prediction may be used to code the current CU. The PDPC may also be included in the derivation of the TIMD mode.

[0129] The costs (or cost values) of the two selected modes (eg, intramode1 and intramode2) may be thresholded, and a cost factor of 2 may be applied in Equation 2 as follows:

number

number

[0130] FIG. 12 illustrates a TIMD mode with fusion. As shown in FIG. 12, a first prediction (1202) for a current block may be generated based on a first intra mode (e.g., intraMode1), and a second prediction (1204) for a current block may be generated based on a second intra mode (e.g., intraMode2). The first intra mode and the second intra mode may be obtained based on TIMD. In step (S1206), when the first intra mode is an angular mode, a PDPC process (1210) may be applied to the first prediction (1202) to generate a refined first prediction. In step (S1208), when the second intra mode is an angular mode, the second prediction (1204) may receive a PDPC process (1212) to generate a refined second prediction. The refined first prediction and the refined second prediction may be fused or combined at (1214) to generate a first combined prediction. The first combined prediction may be a weighted combination of the refined first prediction and the refined second prediction. In response to the first intra mode being a DC mode or a planar mode, a PDPC process (1218) may be applied to the first combined prediction to generate a second combined prediction. Furthermore, the current block may be intra predicted based on the second combined prediction.

[0131] 12, in step (S1206), when the first intra mode is not an angular mode, the PDPC process (1210) may not be applied to the first prediction (1202). Similarly, in step (S1208), when the second intra mode is not an angular mode, the PDPC process (1212) may not be applied to the second prediction (1204). Furthermore, when the first intra mode is not a DC mode or a planar mode, the PDPC process (1218) may not be applied to the first joint prediction.

[0132] Decoder-side intra prediction can use explicitly signaled intra modes and decoder-side derived intra modes to perform intra prediction at the decoder side, for example, the decoder can apply DIMD to derive the intra mode and use the derived intra mode for intra prediction.

[0133] In this disclosure, the PDPC process may be performed after a fusion process such as a DIMD fusion process. An example process (1300) may be provided in FIG. 13. As shown in FIG. 13, a first predictor (1302) may be generated by performing intra prediction on the current block based on a first intra mode (e.g., intraMode1), a second predictor (1304) may be generated by performing intra prediction on the current block based on a second intra mode (e.g., intraMode2), and a third predictor (1306) may be generated by performing intra prediction on the current block based on a planar mode. The first intra mode and the second intra mode may be obtained based on DIMD. The first predictor, the second predictor, and the third predictor may be combined by a fusion process (1308) to generate a combined predictor. The combined predictor may be a weighted combination of the first predictor, the second predictor, and the second predictor. A PDPC process (1310) may be applied to the combined predictor to generate a refined predictor, so that the current block may be reconstructed at the decoder side based on the refined predictor.

[0134] Please note that Figure 13 is only an example. The number of predictors is not limited to three. Any number of predictors may be applied in the process (1300). Additionally, the PDPC process may include the gradient-based PDPC described above. The gradient-based PDPC may be implemented without a secondary reference sample.

[0135] In an embodiment, only one intra mode or predictor is used to determine whether a PDPC process can be applied. For example, a first intra mode (e.g., intraMode1) may be used to determine whether a PDPC process is applicable to the process (1300).

[0136] In another embodiment, the average value of the intra-angular modes may be used to determine whether the PDPC process is applied. When the first intra-mode and the second intra-mode are angular modes, the average of the angular values ​​of the first intra-mode and the second intra-mode may be used to determine whether the PDPC process is applicable to the process (1300). For example, when the first intra-mode (e.g., intraMode1) is intra-mode 60 and the second intra-mode (e.g., intraMode2) is intra-mode 62, the average angular value of the first intra-mode and the second intra-mode is (60+62)>>1=61. When PDPC is applicable to intra-angular mode 61, the PDPC process (1310) may be applied after the fusion process (1308) of the process (1300).

[0137] In another example, when only one of the first intra mode and the second intra mode is an intra-angular mode, it may be determined whether PDPC is applicable based on one of the first intra mode and the second intra mode that is an intra-angular mode.

[0138] In yet another embodiment, the PDPC process may be determined to be applicable to a particular predictor, such as a planar or DC predictor. For example, the PDPC process may always be applicable in process (1300) when one of the first intra mode and the second intra mode is a planar mode or a DC mode.

[0139] In this disclosure, the PDPC process may be performed after the TIMD fusion process. An example process (1400) may be shown in FIG. 14. As shown in FIG. 14, a first predictor (1402) may be generated by performing intra prediction on the current block based on a first intra mode (e.g., intraMode1), and a second predictor (1404) may be generated by performing intra prediction on the current block based on a second intra mode (e.g., intraMode2). The first intra mode and the second intra mode may be obtained based on the TIMD. The first predictor and the second predictor may be combined by a fusion process (1406) to generate a combined predictor. The combined predictor may be a weighted combination of the first predictor and the second predictor. A PDPC process (1408) may be applied to the combined predictor to generate a refined predictor. Thus, the current block may be reconstructed based on the refined modified predictor at the decoder side.

[0140] Note that the process (1400) can include any number of predictors. Additionally, the PDPC process can include gradient-based PDPC, which can be implemented without a secondary reference sample.

[0141] In an embodiment, only one intra mode or predictor is used to determine whether a PDPC process can be applied. For example, a first intra mode (e.g., intraMode1) may be used to determine whether a PDPC process is applicable to the process (1400).

[0142] In another embodiment, an average value of the intra-angle modes may be used to determine whether the PDPC process is applied. When the first intra-mode and the second intra-mode are angular modes, an average of the angular values ​​of the first intra-mode and the second intra-mode may be used to determine whether the PDPC process is applicable to the process (1400).

[0143] For example, when a first intra mode (e.g., intraMode1) is intra mode 60 and a second intra mode (e.g., intraMode2) is intra mode 62, the average angular value of the first intra mode and the second intra mode is (60+62)>>1=61. When intra angular mode 61 is PDPC applicable, a PDPC process (1408) may be applied after the fusion process (1406) of process (1400).

[0144] In another example, when only one of the first intra mode and the second intra mode is an intra-angular mode, it may be determined whether PDPC is applicable based on one of the first intra mode and the second intra mode that is an intra-angular mode.

[0145] In yet another embodiment, the PDPC process may be determined to be applicable to a particular predictor, such as a planar or DC predictor. For example, the PDPC process may always be applicable in process (1400) when one of the first intra mode and the second intra mode is a planar mode or a DC mode.

[0146] In the present disclosure, the PDPC process may be skipped after the fusion process. For example, the PDPC process may be skipped for DIMD and / or TIMD. FIG. 15A shows an example process (1500A) in which a final intra predictor may be generated by performing a fusion process on a first predictor (1502), a second predictor (1504), and a third predictor (1506). The first predictor (1502) may be obtained based on a first intra mode, the second predictor (1504) may be obtained based on a second intra mode, and the third predictor (1506) may be obtained based on a planar mode. The first intra mode and the second intra mode may be generated based on DIMD.

[0147] 15B shows an example process (1500B) in which a final intra predictor may be generated by performing a fusion process on a first predictor (1510) and a second predictor (1512). The first predictor (1510) may be obtained based on a first intra mode, and the second predictor (1512) may be obtained based on a second intra mode. The first intra mode and the second intra mode may be generated based on TIMD.

[0148] It should be noted that the number of predictors for the fusion shown in Figures 15A and 15B is not limited to three or two, but can be any number.

[0149] FIG. 16 shows a flow chart outlining an exemplary decoding process (1600) according to some embodiments of the present disclosure. FIG. 17 shows a flow chart outlining an exemplary encoding process (1700) according to some embodiments of the present disclosure. The proposed processes may be used separately or in combination in any order. Furthermore, each of the processes (or embodiments), the encoder, and the decoder may be implemented by processing circuitry (e.g., one or more processors, or one or more integrated circuits). In one example, the one or more processors execute a program stored on a non-transitory computer-readable medium.

[0150] In embodiments, any of the operations of the process (e.g., 1600 and 1700) may be combined or arranged in any quantity or order as desired. In embodiments, two or more of the operations of the process (e.g., 1600 and 1700) may be performed in parallel.

[0151] The processes (e.g., (1600) and (1700)) may be used in the reconstruction and / or encoding of blocks to generate a prediction block for a block being reconstructed. In various embodiments, the processes (e.g., (1600) and (1700)) are performed by processing circuitry in terminal devices (310), (320), (330), and (340), processing circuitry performing the functions of a video encoder (403), processing circuitry performing the functions of a video decoder (410), processing circuitry performing the functions of a video decoder (510), processing circuitry performing the functions of a video encoder (503), etc. In some embodiments, the processes (e.g., (1600) and (1700)) are implemented with software instructions, such that the processing circuitry performs the processes (e.g., (1600) and (1700)) when the processing circuitry executes the software instructions.

[0152] As shown in Figure 16, the process (1600) may start at (S1601) and proceed to (S1610), where coded information for a current block and neighboring samples of the current block may be received from a coded video bitstream.

[0153] At (S1620), one or more intra-mode predictors for the current block may be determined, and the one or more intra-mode predictors may be derived based on neighboring samples of the current block via either DIMD or TIMD.

[0154] At (S1630), whether PDPC is applied may be determined based on one or more intra-mode predictors.

[0155] At (S1640), in response to a determination that PDPC is applied, a fusion process may be performed on one or more intra-mode predictors, which may combine the one or more intra-mode predictors to generate a single intra-mode predictor.

[0156] At (S1650), PDPC may be applied based on a single intra-mode predictor to generate a modified predictor for the current block following a fusion process for one or more intra-mode predictors.

[0157] At (S1660), the current block may be reconstructed based on the modified predictor generated by the PDPC.

[0158] In some embodiments, the fusion process may include generating a single intra-mode predictor based on a weighted combination of the one or more intra-mode predictors. In some embodiments, the fusion process may further include generating a modified predictor based on a weighted combination of (i) one or more of the neighboring samples and (ii) the single intra-mode predictor.

[0159] In response to the one or more intra-mode predictors being derived based on the DIMD, a first intra-mode predictor of the one or more intra-mode predictors may be determined based on a first intra-mode. The first intra-mode may be determined based on neighboring samples and associated with a highest HoG among the neighboring samples. A second intra-mode predictor of the one or more intra-mode predictors may be determined based on a second intra-mode. The second intra-mode may be determined based on adjacent samples and associated with a second highest HoG among the adjacent samples. A third intra-mode predictor of the one or more intra-mode predictors may be determined based on a planar mode.

[0160] In some embodiments, whether the PDPC is applicable may be determined based on the first intra mode derived based on the DIMD.

[0161] In response to the first intra mode and the second intra mode being angular modes, it may be determined whether PDPC is applicable based on an average angular value of the first intra mode and the second intra mode. In response to one of the first intra mode and the second intra mode being an angular mode, it may be determined whether PDPC is applicable based on one of the first intra mode and the second intra mode being an angular mode.

[0162] In some embodiments, the PDPC may be determined to be applicable in response to at least one of the one or more intra-mode predictors being determined based on a planar mode or a DC mode.

[0163] In response to a first intra-mode predictor of the one or more intra-mode predictors being derived based on the TIMD, a first intra-mode predictor of the one or more intra-mode predictors may be determined based on a first intra-mode, the first intra-mode may be determined based on neighboring samples and associated with a minimum cost value of the neighboring samples, a second intra-mode predictor of the one or more intra-mode predictors may be determined based on a second intra-mode, the second intra-mode may be determined based on neighboring samples and associated with a second minimum cost value of the neighboring samples.

[0164] In some embodiments, whether the PDPC is applicable may be determined based on the first intra mode derived based on the TIMD.

[0165] In response to the first intra mode and the second intra mode being angular modes, it may be determined whether PDPC is applicable based on an average angular value of the first intra mode and the second intra mode. In response to one of the first intra mode and the second intra mode being an angular mode, it may be determined whether PDPC is applicable based on one of the first intra mode and the second intra mode being an angular mode.

[0166] In some embodiments, in response to at least one of the first intra mode and the second intra mode being a planar mode or a DC mode, it may be determined that the PDPC is applicable.

[0167] As shown in Figure 17, the process (1700) can start at (S1701) and proceed to (S1710). At (S1710), one or more intra-mode predictors of a current block can be determined, and the one or more intra-mode predictors can be derived based on neighboring samples of the current block via either DIMD or TIMD.

[0168] At (S1720), whether position-dependent intra-prediction combining (PDPC) is applied may be determined based on one or more intra-mode predictors.

[0169] In response to determining that the PDPC is applied, at (S1730), a fusion process may be performed on one or more intra-mode predictors. The fusion process may combine one or more intra-mode predictors to generate a single intra-mode predictor.

[0170] At (S1740), PDPC may be applied based on a single intra-mode predictor to generate a modified predictor for the current block following a fusion process for one or more intra-mode predictors.

[0171] At (S1750), intra prediction may be performed on the current block based on the modified predictor.

[0172] At (S1760), a coded bitstream may be generated to include coded information of the current block and neighboring samples of the current block.

[0173] The techniques described above may be implemented as computer software using computer readable instructions and physically stored on one or more computer readable media. For example, Figure 18 illustrates a computer system (1800) suitable for implementing certain embodiments of the disclosed subject matter.

[0174] Computer software may be coded using any suitable machine code or computer language that may be subject to assembly, compilation, linking, or similar mechanisms to create code containing instructions that may be executed by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc., either directly or through interpretation, microcode execution, etc.

[0175] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smart phones, gaming devices, Internet of Things devices, and the like.

[0176] 18 for computer system (1800) are exemplary in nature and are not intended to suggest any limitation as to the scope of use or functionality of the computer software implementing the embodiments of the present disclosure. The arrangement of components should not be construed as having any dependency or requirement regarding any one or combination of components illustrated in the exemplary embodiment of computer system (1800).

[0177] The computer system (1800) may include certain human interface input devices. Such human interface input devices may be responsive to input by one or more human users, for example, through tactile input (e.g., keystrokes, swipes, data glove movements), audio input (e.g., voice, clapping), visual input (e.g., gestures), olfactory input (not depicted). Human interface devices may also be used to capture certain media that are not necessarily directly related to conscious human input, such as audio (e.g., speech, music, ambient sounds), images (e.g., scanned images, photographic images obtained from still image cameras), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0178] The input human interface devices may include one or more (only one of each is depicted) of a keyboard (1801), a mouse (1802), a trackpad (1803), a touch screen (1810), a data glove (not shown), a joystick (1805), a microphone (1806), a scanner (1807), and a camera (1808).

[0179] The computer system (1800) may also include certain human interface output devices. Such human interface output devices may stimulate one or more of the senses of a human user, for example, through haptic output, sound, light, and smell / taste. Such human interface output devices may include haptic output devices (e.g., haptic feedback through a touch screen (1810), data gloves (not shown), or joystick (1805), which may be haptic feedback devices that do not function as input devices), audio output devices (e.g., speakers (1809), headphones (not depicted)), visual output devices (e.g., screens (1810), including CRT screens, LCD screens, plasma screens, OLED screens, each with or without haptic screen input capabilities, some of which may be capable of outputting two-dimensional visual output or more three-dimensional output through means such as stereoscopic output, virtual reality glasses (not depicted), holographic displays, smoke tanks (not depicted)), and printers (not depicted).

[0180] The computer system (1800) may also include human accessible storage devices, such as optical media including CD / DVD ROM / RW (1820) with CD / DVD or similar media (1821), thumb drives (1822), removable hard drives or solid state drives (1823), legacy magnetic media such as tapes and floppy disks (not depicted), specialized ROM / ASIC / PLD based devices such as security dongles (not depicted), and their associated media.

[0181] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the presently disclosed subject matter does not encompass transmission media, carrier waves, or other transient signals.

[0182] The computer system (1800) may also include an interface (1854) to one or more communication networks (1855). The networks may be, for example, wireless, wired, optical. The networks may further be local, wide area, metropolitan, vehicular and industrial, real-time, delay tolerant, etc. Examples of networks include Ethernet, WLAN, cellular networks including GSM, 3G, 4G, 5G, LTE, etc., TV wired and wireless wide area digital networks including cable TV, satellite TV, and terrestrial broadcast TV, vehicular and industrial including CANBus, etc. Certain networks typically require an external network interface adapter attached to a specific general-purpose data port or peripheral bus (1849) (e.g., a USB port on the computer system (1800)), while others are typically built into the core of the computer system (1800) by attachment to a system bus (e.g., an Ethernet interface to a PC computer system or a cellular network interface to a smartphone computer system) as described below. Using any of these networks, the computer system (1800) may communicate with other entities. Such communications may be one-way receive only (e.g., broadcast television), one-way transmit only (e.g., from a CANbus to a particular CANbus), or bidirectional, for example, to other computer systems using local or wide area digital networks. Specific protocols and protocol stacks may be used in each of these networks and network interfaces, as described above.

[0183] The aforementioned human interface devices, human accessible storage devices, and network interfaces may be attached to a core (1840) of the computer system (1800).

[0184] The cores (1840) may include one or more central processing units (CPUs) (1841), graphics processing units (GPUs) (1842), specialized programmable processing units in the form of field programmable gate areas (FPGAs) (1843), hardware accelerators for specific tasks (1844), graphics adapters (1850), etc. These devices may be connected through a system bus (1848), along with read-only memory (ROM) (1845), random access memory (1846), and internal mass storage (1847) such as non-user accessible hard drives, SSDs, etc. In some computer systems, the system bus (1848) may be accessible in the form of one or more physical plugs to allow expansion with additional CPUs, GPUs, etc. Peripheral devices may be either attached directly to the core's system bus (1848) or attached through a peripheral bus (1849). In one example, a screen 1810 may be connected to a graphics adapter 1850. Architectures for peripheral buses include PCI, USB, etc.

[0185] The CPU (1841), GPU (1842), FPGA (1843), and accelerator (1844) may combine to execute certain instructions that may constitute the aforementioned computer code. The computer code may be stored in ROM (1845) or RAM (1846). Temporary data may also be stored in RAM (1846), while persistent data may be stored in, for example, internal mass storage (1847). Rapid storage and retrieval to any of the memory devices may be enabled through the use of cache memory, which may be closely associated with one or more of the CPU (1841), GPU (1842), mass storage (1847), ROM (1845), RAM (1846), etc.

[0186] The computer-readable medium may have computer code thereon for performing various computer-implemented operations. The medium and computer code may be specially designed and constructed for the purposes of the present disclosure, or they may be of the kind well known and available to those skilled in the computer software arts.

[0187] As an example, and not by way of limitation, a computer system having the architecture (1800), and in particular the core (1840), can provide functionality as a result of a processor (including a CPU, GPU, FPGA, accelerator, etc.) executing software embodied in one or more tangible computer-readable media. Such computer-readable media can be media associated with the user-accessible mass storage introduced above, as well as specific storage of the core (1840) of a non-transitory nature, such as the core internal mass storage (1847), ROM (1845), etc. Software implementing various embodiments of the present disclosure can be stored in such devices and executed by the core (1840). The computer-readable media can include one or more memory devices or chips, depending on the particular needs. The software can cause the core (1840), and in particular the processor (including a CPU, GPU, FPGA, etc.) therein, to perform certain processes or certain portions of certain processes described herein, including defining data structures stored in RAM (1846) and modifying such data structures according to the processes defined by the software. Additionally or alternatively, the computer system may provide functionality as a result of logic hardwired or otherwise embodied in circuitry (e.g., accelerator (1844)), which may operate in place of or in conjunction with software to perform particular processes or particular portions of particular processes described herein. Reference to software may encompass logic, and vice versa, where appropriate. Reference to computer-readable medium may encompass circuitry (such as an integrated circuit (IC)) that stores software for execution, circuitry that embodies logic for execution, or both, where appropriate. The present disclosure encompasses any suitable combination of hardware and software. Appendix A: Acronyms JEM: Joint exploration model VVC: versatile video coding BMS: Benchmark set MV: Motion Vector HEVC: High Efficiency Video Coding SEI: Supplementary Enhancement Information VUI: Video Usability Information GOP: Group of Pictures TU: Transform Unit PU: Prediction Unit CTU: Coding Tree Unit CTB: Coding Tree Block PB: Prediction Block HRD: Hypothetical Reference Decoder SNR Signal Noise Ratio CPU: Central Processing Unit GPU: Graphics Processing Unit CRT: Cathode Ray Tube LCD: Liquid Crystal Display OLED: Organic Light-Emitting Diode CD: Compact Disc DVD: Digital Video Disc ROM: Read-Only Memory RAM: Random Access Memory ASIC: Application-Specific Integrated Circuit PLD: Programmable Logic Device LAN: Local Area Network GSM: Global System for Mobile communications LTE: Long-Term Evolution CANBus: Controller Area Network Bus USB: Universal Serial Bus PCI: Peripheral Component Interconnect FPGA: Field Programmable Gate Areas SSD: Solid-state drive IC: Integrated Circuit CU: Coding Unit

[0188] While this disclosure has described several exemplary embodiments, there are alterations, substitutions, and various substitute equivalents which are within the scope of this disclosure. It will thus be appreciated that those skilled in the art will be able to devise many systems and methods which, although not explicitly shown or described herein, embody the principles of this disclosure and are within the spirit and scope of this disclosure.

Claims

1. 1. A method of video decoding performed in a video decoder, comprising: receiving coded information of a current block and neighboring samples of the current block from a coded video bitstream; determining one or more intra-mode predictors for the current block, the one or more intra-mode predictors being derived based on the neighboring samples of the current block via either decoder-side intra-mode derivation (DIMD) or template-based intra-mode derivation (TIMD); determining whether position-dependent intra-prediction combining (PDPC) is applied based on the one or more intra-mode predictors; in response to determining that the PDPC applies, performing a fusion process on the one or more intra-mode predictors, the fusion process combining one or more intra-mode predictors to generate a single intra-mode predictor; applying the PDPC based on the single intra-mode predictor to generate a modified predictor for the current block following the fusion process for the one or more intra-mode predictors; reconstructing the current block based on the modified predictor generated by the PDPC; The step of determining one or more intra-mode predictors comprises: In response to the one or more intra-mode predictors being derived based on the DIMD, determining a first intra-mode predictor of the one or more intra-mode predictors based on a first intra-mode, the first intra-mode being determined based on the neighboring samples and being associated with a highest histogram of gradient (HoG) of the neighboring samples; determining a second intra-mode predictor of the one or more intra-mode predictors based on a second intra-mode, the second intra-mode being determined based on the neighboring samples and associated with a second highest HoG of the neighboring samples; determining a third intra-mode predictor of the one or more intra-mode predictors based on a planar mode.

2. performing the fusion process includes generating the single intra-mode predictor based on a weighted combination of the one or more intra-mode predictors; 2. The method of claim 1, wherein applying the PDPC further comprises generating the modified predictor based on a weighted combination of (i) one or more of the neighboring samples and (ii) the single intra-mode predictor.

3. The step of determining whether the PDPC applies comprises: The method of claim 1 , further comprising: determining whether the PDPC is applicable based on the first intra-mode derived based on the DIMD.

4. The step of determining whether the PDPC applies comprises: In response to the first intra mode and the second intra mode being angular modes, determining whether the PDPC is applicable based on an average angular value of the first intra mode and the second intra mode; 2. The method of claim 1, further comprising: in response to one of the first intra mode and the second intra mode being an angular mode, determining whether the PDPC is applicable based on one of the first intra mode and the second intra mode being the angular mode.

5. The step of determining whether the PDPC applies comprises:

2. The method of claim 1, further comprising: determining that the PDPC is applicable in response to determining based on at least one of the one or more intra-mode predictors being in a planar mode or a DC mode.

6. A method of video decoding performed in a video decoder, comprising: receiving coded information of a current block and neighboring samples of the current block from a coded video bitstream; determining one or more intra-mode predictors for the current block, the one or more intra-mode predictors being derived based on the neighboring samples of the current block via either decoder-side intra-mode derivation (DIMD) or template-based intra-mode derivation (TIMD); determining whether position-dependent intra-prediction combining (PDPC) is applied based on the one or more intra-mode predictors; in response to determining that the PDPC applies, performing a fusion process on the one or more intra-mode predictors, the fusion process combining one or more intra-mode predictors to generate a single intra-mode predictor; applying the PDPC based on the single intra-mode predictor to generate a modified predictor for the current block following the fusion process for the one or more intra-mode predictors; reconstructing the current block based on the modified predictor generated by the PDPC; The step of determining one or more intra-mode predictors comprises: In response to the one or more intra-mode predictors being derived based on the TIMD, determining a first intra-mode predictor of the one or more intra-mode predictors based on a first intra-mode, the first intra-mode being determined based on the neighboring samples and associated with a smallest cost value of the neighboring samples; determining a second intra-mode predictor of the one or more intra-mode predictors based on a second intra-mode, the second intra-mode being determined based on the neighboring samples and associated with a second smallest cost value of the neighboring samples; The step of determining whether the PDPC applies comprises: In response to the first intra mode and the second intra mode being angular modes, determining whether the PDPC is applicable based on an average angular value of the first intra mode and the second intra mode; In response to one of the first intra mode and the second intra mode being an angular mode, determining whether the PDPC is applicable based on one of the first intra mode and the second intra mode being the angular mode.

7. Apparatus comprising processing circuitry configured to carry out the method of any one of claims 1 to 6. 。

Images