Method and apparatus for improved intra prediction - Patents.com

Abstract

Flag information associated with the current block is obtained in the coding information and indicates whether the current block is intra predicted based at least in part on intra mode derivation. In response to the flag information being true, multiple candidate intra modes are determined based on neighboring samples of the current block. A first intra mode of the current block is determined based on a first syntax element included in the coding information. The first syntax element indicates the first intra mode of the current block from a most probable multiple prediction (MPM) list and an MPM residual. A second intra mode of the current block is determined based on the first intra mode and the multiple candidate intra modes. The current block is reconstructed based on the first intra mode and the second intra mode.

Description

[Technical field]

[0001] TECHNICAL FIELD This disclosure relates generally to video coding and describes embodiments thereof.

[0002] This application claims the benefit of priority to U.S. Patent Application No. 17 / 877,802, entitled "METHOD AND APPARATUS FOR IMPROVED INTRA PREDICTION," filed July 29, 2022, which claims the benefit of priority to U.S. Provisional Application No. 63 / 231,998, entitled "Improvement on Intra Prediction," filed August 11, 2021, and U.S. Provisional Application No. 63 / 228,369, entitled "Improvement on Intra Prediction," filed August 2, 2021. The entire disclosures of the prior applications are hereby incorporated by reference in their entireties. [Background technology]

[0003] The background discussion provided herein is intended to generally indicate the context of the present disclosure. Works of the presently named inventors, to the extent that they are described in this Background section, and aspects of the description that may not be admitted as prior art at the time of filing, are not admitted, expressly or impliedly, as prior art against this disclosure.

[0004] Video coding and decoding can be performed using inter-picture prediction with motion compensation. Uncompressed digital video can include a sequence of pictures, each having spatial dimensions of, for example, 1920x1080 luma samples and associated chrominance samples. The sequence of pictures can have a fixed or variable picture rate (also informally known as frame rate), for example, 60 pictures / second or 60Hz. Uncompressed video has specific bitrate requirements. For example, 1080p60 4:2:0 video (1920x1080 luma sample resolution at a frame rate of 60Hz) with 8 bits per sample requires a bandwidth approaching 1.5Gbit / s. One hour of such video requires more than 600GByte of storage space.

[0005] One of the goals of video coding and decoding is the reduction of redundancy in the input video signal through compression. Compression helps to reduce the aforementioned bandwidth and / or storage space requirements, in some cases by more than two orders of magnitude. Both lossless and lossy compression, as well as combinations thereof, can be employed. Lossless compression refers to techniques that can reconstruct an exact copy of the original signal from the compressed original signal. When using lossy compression, the reconstructed signal may not be identical to the original signal, but the distortion between the original and reconstructed signals is small enough so that the reconstructed signal is useful for the intended application. For video, lossy compression is widely adopted. The amount of distortion that is tolerated varies from application to application. For example, a user of a particular consumer streaming application may tolerate higher distortion than a user of a television distribution application. The achievable compression ratio can reflect the following: The more distortion that is allowed / tolerated, the higher the compression ratio that can be obtained.

[0006] Video encoders and decoders can utilize techniques from a number of broad categories, including, for example, motion compensation, transform, quantization, and entropy coding.

[0007] Video codec techniques can include a technique known as intra coding. In intra coding, sample values ​​are represented without reference to samples or other data from previously reconstructed reference pictures. In some video codecs, a picture is spatially divided into blocks of samples. If all blocks of samples are coded in intra mode, the picture may be an intra picture. Intra pictures and their derivatives, such as independent decoder refresh pictures, can be used to reset the decoder state and therefore can be used as the first picture of a coded video bitstream and video session or as a still picture. Samples within a block can be exposed to a transform and the transform coefficients can be quantized before entropy coding. Intra prediction is a technique that minimizes sample values ​​in the pre-transformed region. In some cases, the smaller the DC value and the smaller the AC coefficients after the transform, the fewer bits are needed for a given quantization step size to represent the block after entropy coding.

[0008] Conventional intra-coding, for example as known from MPEG-2 generation coding techniques, does not use intra-prediction. However, some newer video compression techniques include techniques that attempt to predict intra-prediction, for example from surrounding sample data and / or metadata acquired during encoding and / or decoding of spatially adjacent and preceding data blocks in decoding order. Such techniques are referred to hereinafter as "intra prediction" techniques. Note that in at least some cases, intra-prediction uses only reference data from the current picture being reconstructed, and not from the reference picture.

[0009] There are many different forms of intra-prediction. If one or more of such techniques can be used in a given video coding technique, the technique in use can be coded with an intra-prediction mode. In certain cases, the mode can have sub-modes and / or parameters, which can be coded separately or included in a mode codeword. The codeword used for a given mode, sub-mode, and / or parameter combination can affect the coding efficiency improvement through intra-prediction, and the entropy coding technique used to convert the codeword into a bitstream can also be affected.

[0010] Certain modes of intra prediction were introduced in H.264, refined in H.265, and further refined in new coding techniques such as the joint exploration model (JEM), versatile video coding (VVC), and benchmark set (BMS). A predictor block can be formed using neighboring sample values ​​belonging to already available samples. The sample values ​​of the neighboring samples are copied into the predictor block according to the direction. The reference to the direction in use can be coded in the bitstream or can be predicted itself.

[0011] Referring to FIG. 1, shown at the bottom right is a subset of 9 known prediction directions from the 33 possible prediction directions of H.265 (corresponding to the 33 angular modes out of the 35 intra modes). The point where the arrows converge (101) represents the sample to be predicted. The arrows represent the direction in which the sample is predicted. For example, arrow (102) indicates that sample (101) is predicted from a sample at an angle of 45 degrees from the horizontal to the top right. Similarly, arrow (103) indicates that sample (101) is predicted from a sample at an angle of 22.5 degrees from the horizontal to the bottom left of sample (101).

[0012] 1, at the top left is depicted a square block (104) of 4x4 samples (indicated by bold dashed lines). The square block (104) contains 16 samples, each labeled with an "S" to indicate its location in the Y dimension (e.g., row index) and its location in the X dimension (e.g., column index). For example, sample S21 is the second sample (from the top) in the Y dimension and the first sample (from the left) in the X dimension. Similarly, sample S44 is the fourth sample of the block (104) in both the Y and X dimensions. Since the size of the block is 4x4 samples, S44 is at the bottom right. Additionally, reference samples are shown following a similar numbering scheme. The reference samples are labeled with an R and are the Its Y position (eg, row index) and X position (column index). In both H.264 and H.265, the prediction samples are adjacent to the block being reconstructed, so negative values ​​do not need to be used.

[0013] Intra-picture prediction can work by copying reference sample values ​​from adjacent samples as required by the signaled prediction direction. For example, assume that the coded video bitstream includes signaling for this block that indicates a prediction direction consistent with the arrow (102). That is, the sample is predicted from the prediction sample to the upper right at an angle of 45 degrees from the horizontal. Then samples S41, S32, S23, S14 are predicted from the same reference sample R05. Sample S44 is then predicted from reference sample R08.

[0014] In some cases, values ​​of multiple reference samples may be combined, for example through interpolation, to calculate a reference sample, especially when the orientation is not evenly divisible by 45 degrees.

[0015] As video coding techniques develop, the number of possible directions is increasing. In H.264 (2003), nine different directions could be represented. This increased to 33 in H.265 (2013), and JEM / VVC / BMS can support up to 65 directions at the time of publication. Experiments have been performed to identify the most likely directions, and certain techniques in entropy coding are used to represent those likely directions with a small number of bits, accepting certain penalties for the less likely directions. Furthermore, the direction itself can sometimes be predicted from the neighboring directions used in neighboring, already decoded, blocks.

[0016] FIG. 2 shows a schematic diagram (201) depicting 65 intra prediction directions according to JEM to illustrate the increase in the number of prediction directions over time.

[0017] The mapping of intra-prediction direction bits in a coded video bitstream to represent the directions may vary between video coding techniques and may range, for example, from a simple direct mapping of the prediction direction to an intra-prediction mode, to a codeword, to a complex adaptation scheme that includes most possible modes, and similar techniques. In any case, however, there may be certain directions that are statistically less likely to occur in the video content than certain other directions. Because the goal of video compression is to reduce redundancy, in well-performing video coding techniques, these less likely directions are represented with more bits than the more likely directions. Summary of the Invention

[0018] Aspects of the present disclosure provide a method and apparatus for video encoding / decoding. In some examples, an apparatus for video decoding includes a receiving circuit and a processing circuit.

[0019] According to one aspect of the disclosure, a video decoding method is provided that is executed in a video decoder. In the method, coding information of a current block and neighboring samples of the current block may be received from a coded video bitstream. Flag information associated with the current block in the coding information may be obtained. The flag information may indicate whether the current block is intra predicted based at least in part on an intra mode derivation, in which a plurality of candidate intra modes are derived based on the neighboring samples. In response to the flag information indicating that the current block is intra predicted based at least in part on the intra mode derivation, the plurality of candidate intra modes may be determined based on the neighboring samples. A first intra mode of the current block may be determined based on a first syntax element included in the coding information. The first syntax element may indicate the first intra mode of the current block from a Most Probable Mode (MPM) list and an MPM remainder. A second intra mode of the current block may be determined based on the first intra mode and the plurality of candidate intra modes. The current block may be reconstructed based on the first intra mode and the second intra mode.

[0020] In some embodiments, the first intra mode may be determined based on a first syntax element in response to the flag information indicating that the current block is intra predicted based at least in part on the intra mode derivation.

[0021] In the method, in response to the flag information indicating that the current block is not intra predicted based on the intra mode derivation, a third intra mode may be determined based on a second syntax element included in the coding information.

[0022] In one aspect, the intra mode derivation may include a decoder-side intra mode derivation (DIMD), in which the multiple candidate intra modes are determined based on a histogram of gradients (HoG) associated with the neighboring samples.

[0023] In another embodiment, the intra mode derivation may include a template-based intra mode derivation (TIMD) in which the multiple candidate intra modes are determined based on cost values ​​of the neighboring samples.

[0024] To determine the second intra mode of the current block, a first predicted value of a sample in the current block may be determined based on the first intra mode. A reconstructed value of the sample in the current block may be determined based on the first predicted value. A candidate predicted value of each of the samples in the current block may be determined based on each of the candidate intra modes. A plurality of template match values ​​may further be determined. Each of the plurality of template match values ​​may be equal to twice a reconstructed value minus (i) the first predicted value and (ii) the respective candidate predicted value based on a corresponding one of the plurality of candidate intra modes. A second intra mode of the current block may then be determined, where the second intra mode is a candidate intra mode of the plurality of candidate intra modes associated with a minimum template match value of the plurality of template match values.

[0025] A second predicted value of samples in the current block may be determined based on the second intra mode to reconstruct the current block. Based on the final predicted value, the current block may be further reconstructed. The final predicted value may be a weighted combination of the first predicted value and the second predicted value.

[0026] In some embodiments, the weighted combination may be equally weighted.

[0027] According to another aspect of the present disclosure, an apparatus is provided, the apparatus including a processing circuit, the processing circuit may be configured to perform any method of video coding.

[0028] Aspects of the present disclosure also provide a non-transitory computer-readable medium having stored thereon instructions that, when executed by a computer for video decoding, cause the computer to perform any method of video coding. [Brief description of the drawings]

[0029] Further features, nature and various advantages of the disclosed technical subject matter will become more apparent from the following detailed description and the accompanying drawings. [Figure 1] FIG. 1 is a schematic diagram of an example subset of intra-prediction modes. [Diagram 2] FIG. 2 is an illustration of an example intra-prediction direction. [Diagram 3] FIG. 3 is a simplified block diagram schematic of a communication system (300) according to one embodiment. [Figure 4] FIG. 4 is a simplified block diagram schematic of a communication system (400) according to one embodiment. [Diagram 5] FIG. 5 is a schematic diagram of a simplified block diagram of a decoder according to one embodiment. [Figure 6] FIG. 6 is a schematic diagram of a simplified block diagram of an encoder according to one embodiment. [Figure 7] FIG. 7 is a block diagram of an encoder according to another embodiment. [Figure 8] FIG. 8 is a block diagram of a decoder according to another embodiment. [Figure 9] FIG. 9 is a schematic diagram of template-based intra-mode derivation (TIMD) according to one embodiment. [Figure 10] FIG. 10 is a flowchart outlining an exemplary decoding process in accordance with some embodiments of the present disclosure. [Figure 11] FIG. 11 is a flowchart outlining an exemplary encoding process in accordance with some embodiments of the present disclosure. [Figure 12] FIG. 12 is a schematic diagram of a computer system according to one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0030] FIG. 3 illustrates a simplified block diagram of a communication system (300) according to one embodiment of the present disclosure. The communication system (300) includes, for example, a plurality of terminal devices that can communicate with each other via a network (350). For example, the communication system (300) includes a first pair of terminal devices (310) and (320) interconnected via the network (350). In the example of FIG. 3, the first pair of terminal devices (310) and (320) perform unidirectional transmission of data. For example, the terminal device (310) can encode video data (e.g., a stream of video pictures captured by the terminal device (310)) for transmission to another terminal device (320) via the network (350). The encoded video data can be transmitted in the form of one or more coded video bitstreams. The terminal device (320) can receive the coded video data from the network (350), decode the coded video data to recover the video pictures, and display the video pictures according to the recovered video data. One-way data transmission may be common, such as in media service applications.

[0031] In another example, the communication system (300) includes a second pair of terminal devices (330) and (340) that perform bidirectional transmission of coded video data, such as may occur during a video conference. For the bidirectional transmission of data, in one example, each of the terminal devices (330) and (340) can code video data (e.g., a stream of video pictures captured by the terminal device) for transmission to the other of the terminal devices (330) and (340) over the network (350). Each of the terminal devices (330) and (340) can also receive coded video data transmitted by the other of the terminal devices (330) and (340), decode the coded video data to recover the video pictures, and display the video pictures on an accessible display device in accordance with the recovered video data.

[0032] In the example of FIG. 3, the terminal devices (310), (320), (330), and (340) may be illustrated as servers, personal computers, and smartphones, although the principles of the present disclosure are not so limited. Embodiments of the present disclosure apply to laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. Network (350) represents any number of networks that convey coded video data between the terminal devices (310), (320), (330), and (340), including, for example, wired and / or wireless communication networks. The communication network (350) may exchange data over circuit-switched and / or packet-switched channels. Exemplary networks include telecommunications networks, local area networks, wide area networks, and / or the Internet. For purposes of this description, the architecture and topology of network (350) may not be important to the operation of the present disclosure, unless otherwise described below.

[0033] 4 shows an arrangement of a video encoder and a video decoder in a streaming environment as an example of an application of the disclosed subject matter. The disclosed subject matter can be similarly applied to other video-enabled applications, including, for example, video conferencing, digital television, storing compressed video on digital media, including CDs, DVDs, memory sticks, etc.

[0034] The streaming system may include a capture subsystem (413), which may include a video source (401), e.g., a digital camera, that creates a stream of uncompressed video pictures (402). In one example, the stream of video pictures (402) includes samples captured by the digital camera. The stream of video pictures (402), depicted as a bold line to emphasize the high amount of data when compared to the encoded video data (404) (or coded video bitstream), may be processed by an electronic device (420), which includes a video encoder (403) coupled to the video source (401). The video encoder (403) may include hardware, software, or a combination thereof to enable or implement aspects of the disclosed subject matter, as described in more detail below. The encoded video data (404) (or coded video bitstream (404)), depicted as a thin line to emphasize its lower amount of data when compared to the stream of video pictures (402), can be stored on the streaming server (405) for future use. One or more streaming client subsystems, such as the client subsystems (406 and 408) of FIG. 4, can access the streaming server (405) to obtain copies (407) and (409) of the coded video data (404). The client subsystem (406) can include a video decoder (410), for example, within the electronic device (430). The video decoder (410) decodes the incoming copy (407) of the encoded video data and creates an outgoing stream of video pictures (411) that can be rendered on a display (412) (e.g., a display screen) or other rendering device (not shown).In some streaming systems, the encoded video data (404), (407), and (409) (e.g., video bitstreams) may be encoded according to a given video coding / compression standard. Examples of these standards include ITU-T Recommendation H.265. As an example, a video coding standard under development is informally known as Versatile Video Coding (VVC). The disclosed technical subject matter may be used in the context of VVC.

[0035] The electronic devices (420) and (430) may include other components (not shown). For example, the electronic device (420) may include a video decoder (not shown), and the electronic device (430) may also include a video encoder (not shown).

[0036] 5 shows a block diagram of a video decoder (510) according to one embodiment of the present disclosure. The video decoder (510) can be included in an electronic device (530). The electronic device (530) can include a receiver (531) (e.g., receiving circuitry). The video decoder (510) can be used in place of the video decoder (410) in the example of FIG. 4.

[0037] The receiver (531) can receive one or more coded video sequences to be decoded by the video decoder (510), one coded video sequence at a time, in the same or another embodiment, with the decoding of each coded video sequence being independent of the other coded video sequences. The coded video sequences can be received from a channel (501), which can be a hardware / software link to a storage device that stores the coded video data. The receiver (531) can receive the encoded video data along with other data, such as coded audio data and / or auxiliary data streams, which can be forwarded to respective entities (not shown) for use. The receiver (531) can separate the coded video sequences from the other data. To address network jitter, a buffer memory (515) can be coupled between the receiver (531) and the entropy decoder / parser (520) (hereafter "parser (520)"). In certain applications, the buffer memory (515) is part of the video decoder (510). In other applications, it may be external to the video decoder (510) (not shown). In still other applications, there may be a buffer memory (not shown) external to the video decoder (510), e.g., to handle network jitter, and there may additionally be another buffer memory (515) internal to the video decoder (510), e.g., to handle playback timing. If the receiver (531) is receiving data from a store-and-forward device with sufficient bandwidth and controllability, or from an isosynchronous network, the buffer memory (515) may not be needed or may be small.For use with best-effort packet networks such as the Internet, the buffer memory (515) may be required to be relatively large and may advantageously be adaptively sized, and may be implemented at least in part in an operating system or similar element (not shown) external to the video decoder (510).

[0038] The video decoder (510) may include a parser (520) for reconstructing symbols (521) from the coded video sequence. These categories of symbols include information used to manage the operation of the video decoder (510) and potentially for controlling a rendering device, such as a rendering device (512) (e.g., a display screen) that is not an integral part of the electronic device (530) but may be coupled to the electronic device (530) as shown in FIG. The control information for the rendering device may be in the form of a Supplemental Enhancement Information (SEI message) or a Video Usability Information (VUI) parameter set fragment (not shown). The parser (520) may parse / entropy decode the received coded video sequence. The coding of the coded video sequence may follow a video coding technique or standard and may follow a variety of principles, including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser (520) may extract from the coded video sequence at least one set of subgroup parameters for a subgroup of pixels in the video decoder based on at least one parameter corresponding to the group. The subgroup may be a Group of Pictures (GOP), picture, tile, slice, macroblock, coding unit (CU), block, transform unit (TU), prediction unit (TU), or a combination of the above. The parser (520) may also extract from the coded video sequence information, such as transform coefficients, quantizer parameter values, motion vectors, etc.

[0039] The parser (520) can perform entropy decoding / parsing operations on the video sequence received from the buffer memory (515) to produce symbols (521).

[0040] The reconstruction of the symbols (521) can involve a number of different units, depending on the type of coded video picture or part thereof (inter-picture and intra-picture, inter-block and intra-block, etc.) and other factors. Which units are involved and how they are involved can be controlled by subgroup control information parsed from the coded video sequence by the parser (520). The flow of such subgroup control information between the parser (520) and the following units is not depicted for clarity.

[0041] Beyond the functional blocks already mentioned, the video decoder (510) may be conceptually subdivided into several functional units, as described below. In a practical implementation operating under commercial constraints, many of these units may interact closely with each other and may be, at least in part, integrated with each other. However, for purposes of describing the disclosed technical subject matter, the following conceptual subdivision into functional units is adequate:

[0042] The first unit is a scalar / inverse transform unit (551), which receives quantized transform coefficients from the parser (520) as well as control information including which transform to use, block size, quantization coefficients, quantization scaling matrix, etc. as symbols (521). The scalar / inverse transform unit (551) can output blocks that make up sample values ​​that can be input to an aggregator (555).

[0043] In some cases, the output samples of the scaler / inverse transform (551) may relate to intra-coded blocks, i.e., blocks that do not use prediction information from a previously reconstructed picture, but may use prediction information from a previously reconstructed portion of the current picture. Such prediction information may be provided by an intra-picture prediction unit (552), which in some cases generates a block of the same size and shape as the block being reconstructed using surrounding already reconstructed information fetched from a current picture buffer (558). The current picture buffer (558) may, for example, buffer a partially reconstructed current picture and / or a fully reconstructed current picture. The aggregator (555) may add the prediction information generated by the intra-prediction unit (552) to the output sample information provided by the scaler / inverse transform unit (551) on a sample-by-sample basis.

[0044] In other cases, the output samples of the scalar / inverse transform unit (551) may relate to inter-coded and potentially motion-compensated blocks. In such cases, the motion compensated prediction unit (553) may access the reference picture memory (557) to fetch samples used for prediction. After motion compensating the fetched samples according to the symbols (521) associated with the block, these samples may be added by the aggregator (555) to the output of the scalar / inverse transform unit (551) (known in this case as residual samples or residual signals) to generate output sample information. The addresses in the reference picture memory (557) from which the motion compensated prediction unit (553) fetches the prediction samples may be controlled by a motion vector and are available to the motion compensated prediction unit in the form of a symbol (521) that may have, for example, X, Y, and reference picture components. Motion compensation may also include interpolation of sample values ​​fetched from the reference picture memory (557) when sub-sample accurate motion vectors are used, motion vector prediction mechanisms, etc.

[0045] The output samples of the aggregator (555) can be subject to various loop filtering techniques in a loop filter unit (556). Video compression techniques can include in-loop filter techniques controlled by parameters contained in the coded video sequence (also referred to as the coded video bitstream) and made available to the loop filter unit (556) as symbols (521) from the parser (520), but can also be responsive to meta-information obtained during decoding of previous portions (in decoding order) of the coded picture or coded video sequence, as well as to previously reconstructed and loop filtered sample values.

[0046] The output of the loop filter unit (556) may be a sample stream that can be output to a rendering device (512) and also stored in a reference picture memory (557) for use in future inter-picture prediction.

[0047] Once a given coded picture has been fully reconstructed, it can be used as a reference picture for future predictions. For example, once a coded picture corresponding to a current picture has been fully reconstructed, and the coded picture has been identified as a reference picture (e.g., by the parser (520)), the current picture buffer (558) becomes part of the reference picture memory (557), and a new current picture buffer can be reallocated before beginning reconstruction of the next coded picture.

[0048] The video decoder (510) can perform decoding operations in accordance with predefined video compression techniques in standards such as ITU-T Rec. H.265. A coded video sequence may be compliant with the syntax specified by the video compression technique or standard being used in the sense that the coded video sequence is compliant with both the syntax of the video compression technique or standard and the profile described in the video compression technique or standard. Specifically, a profile may select a given tool from all tools available in the video compression technique or standard as the only tool available under that profile. Compliance also requires that the complexity of the coded video sequence is within a range defined by the level of the video compression technique or standard. In some cases, the level limits the maximum picture size, maximum frame rate, maximum reconstruction sample rate (e.g., measured in megasamples per second), maximum reference picture size, etc. The limits set by the level may be further limited in some cases through Hypothetical Reference Decoder (HRD) specifications and HRD buffer management metadata signaled in the coded video sequence.

[0049] In one embodiment, the receiver (531) can receive additional (redundant) data along with the encoded video. The additional data may be included as part of the coded video sequence. The additional data may be used by the video decoder (510) to properly decode the data and / or to more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0050] 6 shows a block diagram of a video encoder (603) according to one embodiment of the present disclosure. The video encoder (603) is included in an electronic device (620). The electronic device (620) includes a transmitter (640) (e.g., a transmitting circuit). The video encoder (603) can be used in place of the video encoder (403) in the example of FIG. 4.

[0051] The video encoder (603) can receive video samples from a video source (601) (which is not part of the electronic device (620) in the example of FIG. 6), which can capture video images to be encoded by the video encoder (603). In another example, the video source (601) is part of the electronic device (620).

[0052] The video source (601) can provide a source video sequence that is coded by the video encoder (603) in the form of a digital video sample stream that can be of any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, ...), any color space (e.g., BT.601 YCrCB, RGB, ...), and any suitable sampling structure (e.g., YCrCb 4:2:0, YCrCb 4:4:4). In a media serving system, the video source (601) can be a storage that archives previously prepared video. In a video conferencing system, the video source (601) can be a camera that captures local picture information as a video sequence. The video data can be provided as a number of individual pictures that, when viewed in sequence, give motion. The picture itself can be organized as a spatial array of pixels, where each pixel can constitute one or more samples, depending on the sampling structure, color space, etc., in use. Those skilled in the art can easily understand the relationship between pixels and samples. The following description focuses on samples.

[0053] According to one embodiment, the video encoder (603) can code and compress pictures of a source video sequence into a coded video sequence (643) in real-time or under any other time constraint required by an application. Enforcing an appropriate coding rate is one of the functions of the controller (650). In some embodiments, the controller (650) controls and is operatively coupled to other functional units as described below. For clarity, couplings are not depicted. Parameters set by the controller (650) include rate control related parameters (picture skip, quantization, lambda value of rate-distortion optimization techniques, ...), picture size, group of pictures (GOP) layout, maximum motion vector search range, etc. The controller (650) can be configured to have other appropriate functions associated with the video encoder (603) optimized for a given system design.

[0054] In some embodiments, the video encoder (603) is configured to operate in a coding loop. As an oversimplified explanation, in one example, the coding loop can include a source coder (630) (e.g., responsible for creating symbols, such as a symbol stream, based on an input picture to be coded and a reference picture) and a (local) decoder (633) embedded in the video encoder (603). The decoder (633) reconstructs the symbols to create sample data in a similar manner to that created by the (remote) coder (since in the video compression techniques contemplated in the disclosed subject matter, the compression between the symbols and the coded video bitstream is lossless). The reconstructed sample stream (sample data) is input to a reference picture memory (634). Since the decoding of the symbol stream leads to a bit-exact result that is independent of the location of the decoder (local or remote), the content in the reference picture memory (634) is also bit-exact between the local and remote encoders. In other words, the prediction part of the encoder "sees" exactly the same sample values ​​as the reference picture samples that the decoder would "see" if it used prediction during decoding. This basic principle of reference picture synchrony (and the resulting drift if synchrony cannot be maintained, e.g., due to channel errors) is used in several related technologies as well.

[0055] The operation of the "local" decoder (633) may be the same as a "remote" decoder, such as the video decoder (510), already described in detail in conjunction with Figure 5. Referring again briefly to Figure 5, however, because symbols are available and the encoding / decoding of symbols into a coded video sequence by the entropy coder (645) and parser (520) may be lossless, the entropy decoding portion of the video decoder (510), including the buffer memory (515), and the parser (520), may not be fully implemented in the local decoder (633).

[0056] An observation that can be made at this point is that any decoder technique, except analysis / entropy decoding, that is present in the decoder must also be present, in substantially identical functional form, in the corresponding encoder. For this reason, the disclosed technical matter focuses on the operation of the decoder. A description of the encoder techniques can be omitted, since they are the inverse of the decoder techniques that have been generically described. Only in certain areas is a more detailed description necessary, and is provided below.

[0057] During operation, in some examples, the source coder (630) may perform motion-compensated predictive coding, which predictively codes an input picture with reference to one or more previously coded pictures from the video sequence designated as “reference pictures.” In this manner, the coding engine (632) codes differences between pixel blocks of the input picture and pixel blocks of reference pictures that may be selected as predictive references for the input picture.

[0058] The local video decoder (633) can decode the coded video data of pictures that may be designated as reference pictures based on the symbols created by the source coder (630). The operation of the coding engine (632) can advantageously be a lossy process. If the coded video data can be decoded by a video decoder (not shown in FIG. 6), the reconstructed video sequence can be a replica of the source video sequence, typically with some errors. The local video decoder (633) can replicate the decoding process that may be performed on the reference pictures by the video decoder and store the reconstructed reference pictures in a reference picture cache (634). In this way, the video encoder (603) can locally store copies of reconstructed reference pictures that have common content as reconstructed reference pictures to be acquired by a far-end video decoder (no transmission errors).

[0059] A predictor (635) may perform a prediction search for the coding engine (632). That is, for a new picture to be coded, the predictor (635) may search the reference picture memory (634) for sample data (as candidate reference pixel blocks) or predefined metadata, such as reference picture motion vectors, block shapes, etc., that may serve as suitable prediction references for the new picture. The predictor (635) may operate on a sample-by-pixel block basis to find a suitable prediction reference. In some cases, as determined by the search results obtained by the predictor (635), the input picture may have prediction references drawn from multiple reference pictures stored in the reference picture memory (634).

[0060] The controller (650) can manage the coding operations of the source coder (630), including, for example, setting the parameters and subgroup parameters used for encoding the video data.

[0061] The output of all the aforementioned functional units may undergo entropy coding in an entropy coder (645), which converts the symbols produced by the various functional units into a coded video sequence by losslessly compressing the symbols according to techniques such as Huffman coding, variable length coding, arithmetic coding, etc.

[0062] The sender (640) can buffer the coded video sequence created by the entropy coder (645) in preparation for transmission over a communication channel (660), which can be a hardware / software link to a storage device that stores the encoded video data. The sender (640) can merge the coded video data from the video coder (603) with other data to be transmitted, such as coded audio data and / or auxiliary data streams (sources not shown).

[0063] The controller (650) can manage the operation of the video encoder (603). During coding, the controller (650) can assign each coded picture a predefined coded picture type, which can affect the coding technique that can be applied to the respective picture. For example, a picture can often be assigned as one of the following picture types:

[0064] An Intra picture (I-picture) is one that can be coded and decoded without using other pictures in a sequence as a source of prediction. Some video codecs can use different types of Intra pictures, including, for example, Independent Decoder Refresh (IDR) pictures. Those skilled in the art are aware of these variants of I-pictures and their respective applications and characteristics.

[0065] A predictive picture (P picture) is one that uses at most one motion vector and reference index to predict the sample values ​​of each block and can be coded and decoded using intra- or inter-prediction.

[0066] A bi-directionally predicted picture (B-picture) is one that uses up to two motion vectors and reference indices to predict the sample values ​​of each block and can be coded and decoded using intra- or inter-prediction. Similarly, a multi-predicted picture can use more than one reference picture and associated metadata for the reconstruction of a block.

[0067] A source picture is typically spatially subdivided into multiple sample blocks (e.g., 4x4, 8x8, 4x8, or 16x16 blocks of samples each) and may be coded on a block-by-block basis. Blocks may be predictively coded with reference to other (already coded) blocks as determined by a code assignment applied to the block's respective picture. For example, blocks of an I-picture may be non-predictively coded or predictively coded with reference to already coded blocks of the same picture (spatial or intra prediction). Pixel blocks of a P-picture may be predictively coded via spatial prediction or via temporal prediction with reference to one previously coded reference picture. Blocks of a B-picture may be predictively coded via spatial prediction or via temporal prediction with reference to one or two previously coded reference pictures.

[0068] The video encoder (603) may perform coding operations in accordance with a given video coding technique or standard, such as ITU-T Rec H.265. In its operation, the video encoder (603) may perform various compression operations, including predictive coding operations that exploit temporal and spatial redundancy in the input video sequence. The coded video data may therefore conform to a syntax specified by the video coding technique or standard being used.

[0069] In one embodiment, the transmitter (640) can transmit additional data along with the encoded video. The source coder (630) can include such data as part of the coded video sequence. The additional data can include temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant pictures and slices, SEI messages, VUI parameter set fragments, etc.

[0070] A video may be captured as multiple source pictures (video pictures) in a temporal sequence. Intra-picture prediction (often abbreviated as intra-prediction) exploits spatial correlation within a given picture, and inter-picture prediction exploits correlation (temporal or other) between pictures. In one example, a particular picture being encoded / decoded, referred to as a current picture, is divided into blocks. If a block of the current picture is similar to a reference block of a previously coded and still buffered reference picture in the video, the block of the current picture may be coded by a vector, referred to as a motion vector. A motion vector points to a reference block of a reference picture, and may have three dimensions that identify the reference picture when multiple reference pictures are used.

[0071] In some embodiments, bi-prediction techniques can be used in inter-picture prediction. According to bi-prediction techniques, two reference pictures are used, such as a first reference picture and a second reference picture, both of which are prior in decoding order (but may be past and future, respectively, in display order) to a current picture in a video. A block in the current picture can be coded by a first motion vector that points to a first reference block in the first reference picture and a second motion vector that points to a second reference block in the second reference picture. A block can be predicted by a combination of the first reference block and the second reference block.

[0072] Furthermore, merge mode techniques can be used for inter-picture prediction to improve coding efficiency.

[0073] According to some embodiments of the present disclosure, prediction, such as inter-picture prediction and intra-picture prediction, is performed on a block-by-block basis. For example, according to the HEVC standard, a picture in a sequence of video pictures is divided into coding tree units (CTUs) for compression, and the CTUs in a picture have the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. Generally, a CTU includes three coding tree blocks (CTBs), which are one luma CTB and two chroma CTBs. Each CTU can be recursively quadtree split into one or more coding units (CUs). For example, a CTU of 64×64 pixels can be split into one CU of 64×64 pixels, or into four CUs of 32×32 pixels, or into 16 CUs of 16×16 pixels. In one example, each CU is analyzed to determine the prediction type of the CU, such as an inter prediction type or an intra prediction type. A CU is divided into one or more prediction units (PUs) according to temporal and / or spatial predictability. Typically, each PU includes one luma prediction block (PB) and two chroma PBs. In one example, prediction operations in coding (encoding / decoding) are performed in units of prediction blocks. Taking a luma prediction block as an example of a prediction block, the prediction block includes a matrix of pixel values ​​(e.g., luma values), such as 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, etc.

[0074] 7 shows a diagram of a video encoder (703) according to another embodiment of the present disclosure. The video encoder (703) is configured to receive a processed block of sample values ​​(e.g., a predictive block) in a current video picture in a sequence of video pictures and to encode the processed block into a coded picture that is part of a coded video sequence. In one example, the video encoder (703) is used instead of the video encoder (403) in the example of FIG. 4.

[0075] In an HEVC example, the video encoder (703) receives a matrix of sample values ​​for a processing block, such as a prediction block, such as 8×8 samples. The video encoder (703) determines whether the processing block is optimally coded using intra-mode, inter-mode, or bi-prediction mode, e.g., using rate-distortion optimization. If the processing block is coded in intra-mode, the video encoder (703) can encode the processing block into a coded picture using intra-prediction techniques, and if the processing block is coded in inter-mode or bi-prediction mode, the video encoder (703) can encode the processing block into a coded picture using inter-prediction or bi-prediction techniques, respectively. In certain video coding techniques, the merge mode may be an inter-picture prediction sub-mode in which motion vectors derive from one or more motion vector predictors without the benefit of a coded motion vector component external to the predictor. In certain other video coding techniques, there may be a motion vector component applicable to the subject block. In one example, the video encoder (703) includes other components, such as a mode decision module (not shown) to determine the mode of the processing blocks.

[0076] In the example of FIG. 7, the video encoder (703) includes an inter-encoder (730), an intra-encoder (722), a residual calculator (723), a switch (726), a residual encoder (724), a general controller (721), and an entropy encoder (725), coupled together as shown in FIG.

[0077] The inter-encoder (730) is configured to receive a sample of a current block (e.g., a processing block), compare the block with one or more reference blocks in a reference picture (e.g., blocks in previous and subsequent pictures), generate inter-prediction information (e.g., a description of redundant information according to an inter-encoding technique, motion vectors, merge mode information), and calculate an inter-prediction result (e.g., a prediction block) based on the inter-prediction information using any suitable technique. In some examples, the reference picture is a decoded reference picture that is decoded based on the encoded video information.

[0078] The intra encoder (722) is configured to receive samples of a current block (e.g., a processing block), possibly compare the block with blocks already coded in the same picture, and generate quantized coefficients after transformation and possibly also intra prediction information (e.g., intra prediction direction information according to one or more intra coding techniques). In one example, the intra encoder (722) also calculates intra prediction results (e.g., prediction blocks) based on the intra prediction information and reference blocks in the same picture.

[0079] The generic controller (721) is configured to determine generic control data and control other components of the video encoder (703) based on the generic control data. In one example, the generic controller (721) determines the mode of the block and provides a control signal to the switch (726) based on the mode. For example, if the mode is an intra mode, the generic controller (721) controls the switch (726) to select the intra mode result to be used in the residual calculator (723) and controls the entropy encoder (725) to select the intra prediction information and include the intra prediction information in the bitstream. And, if the mode is an inter mode, the generic controller (721) controls the switch (726) to select the inter prediction result to be used in the residual calculator (723) and controls the entropy encoder (725) to select the inter prediction information and include the inter prediction information in the bitstream.

[0080] The residual calculator (723) is configured to calculate a difference (residual data) between the received block and a prediction result selected from the intra-encoder (722) or the inter-encoder (730). The residual encoder (724) is configured to operate on the residual data and encode the residual data to generate transform coefficients. In one example, the residual encoder (724) is configured to transform the residual data from the spatial domain to the frequency domain and generate transform coefficients. The transform coefficients are then subjected to a quantization process to obtain quantized transform coefficients. In various embodiments, the video encoder (703) also includes a residual decoder (728). The residual decoder (728) is configured to perform an inverse transform and generate decoded residual data. The decoded residual data can be used by the intra-encoder (722) and the inter-encoder (730) as appropriate. For example, the inter-encoder (730) may generate decoded blocks based on the decoded residual data and the inter-prediction information, and the intra-encoder (722) may generate decoded blocks based on the decoded residual data and the intra-prediction information. The decoded blocks are appropriately processed to generate decoded pictures, and the decoded pictures may be buffered in a memory circuit (not shown) and used as reference pictures in some examples.

[0081] The entropy encoder (725) is configured to format a bitstream to include the encoded block. The entropy encoder (725) is configured to include various information in accordance with an appropriate standard, such as the HEVC standard. In one example, the entropy encoder (725) is configured to include general control data, selected prediction information (e.g., intra-prediction information or inter-prediction information), residual information, and other appropriate information in the bitstream. Note that no residual information is present when coding a block in a merged sub-mode of either an inter-mode or a bi-prediction mode in accordance with the disclosed subject matter.

[0082] 8 shows a diagram of a video decoder (810) according to another embodiment of the present disclosure. The video decoder (810) is configured to receive coded pictures that are part of a coded video sequence and to decode the coded pictures to generate reconstructed pictures. In one example, the video decoder (810) is used in place of the video decoder (410) in the example of FIG. 4.

[0083] In the example of FIG. 8, the video decoder (810) includes an entropy decoder (871), an inter decoder (880), a residual decoder (873), a reconstruction module (874), and an intra decoder (872), coupled together as shown in FIG. 8.

[0084] The entropy decoder (871) can be configured to reconstruct from the coded picture certain symbols that represent the syntax elements for which the coded picture is created. Such symbols can include, for example, the mode in which the block is coded (e.g., intra mode, inter mode, bi-predictive mode, the latter two in merged or separate submodes), prediction information (e.g., intra prediction information or inter prediction information) that can identify certain samples or metadata used for prediction by the intra decoder (872) or the inter decoder (880), respectively, residual information, for example in the form of quantized transform coefficients, etc. In one example, if the prediction mode is an inter mode or bi-predictive mode, the inter prediction information is provided to the inter decoder (880). And, if the prediction type is an intra prediction type, the intra prediction information is provided to the intra decoder (872). The residual information can be subject to inverse quantization and provided to the residual decoder (873).

[0085] The inter decoder (880) is configured to receive the inter prediction information and to generate inter prediction results based on the inter prediction information.

[0086] The intra decoder (872) is configured to receive the intra prediction information and to generate a prediction result based on the intra prediction information.

[0087] The residual decoder (873) is configured to perform inverse quantization to extract de-quantized transform coefficients, and to process the de-quantized transform coefficients to transform the residual from the frequency domain to the spatial domain. The residual decoder (873) may also require certain control information (including quantization parameters (QPs)), which may be provided by the entropy decoder (871) (the data path not depicted in this manner may be only a small amount of control information).

[0088] The reconstruction module (874) is configured to combine, in the spatial domain, the residual as output by the residual decoder (873) and the prediction result (possibly as output by an inter- or intra-prediction module) to form a reconstructed block, which may be part of a reconstructed picture, which in turn may be part of a reconstructed video. Note that other suitable operations, such as a deblocking operation, may be performed to improve image quality.

[0089] The video encoders (403), (603), and (703) and the video decoders (410), (510), and (810) may be implemented using any suitable technique. In one embodiment, the video encoders (403), (603), and (703) and the video decoders (410), (510), and (810) may be implemented using one or more integrated circuits. In another embodiment, the video encoders (403), (603), and (603) and the video decoders (410), (510), and (810) may be implemented using one or more processors executing software instructions.

[0090] This disclosure includes improvements in intra-mode coding.

[0091] ITU-T VCEG (Q 6 / 16) and ISO / IEC MPEG (JTC1 / SC29 / WG11) published the H.265 / HEVC (High Efficiency Video Coding) standard in 2013 (version 1), 2014 (version 2), 2015 (version 3), and 2016 (version 4). In 2015, these two standardization organizations jointly formed the Joint Video Exploration Team (JVET) to explore the possibility of developing the next video coding standard beyond HEVC. In April 2018, JVET officially started the standardization process for the next generation of video coding beyond HEVC. The new standard was named Versatile Video Coding (VVC), and JVET was renamed Joint Video Expert Team. In July 2020, H.266 / VVC version 1 was completed. In January 2021, an ad hoc group was formed to investigate extended compression beyond VVC capabilities.

[0092] In decoder-side intra mode derivation (DIMD), intra modes can be derived using associated syntax elements signaled in the bitstream, or intra modes can be derived at the decoder side without associated syntax elements signaled in the bitstream. Many methods can be used to derive intra modes at the decoder side, and decoder-side intra mode derivation is not limited to the methods described in this disclosure.

[0093] In the DIMM, two intra modes from multiple candidate intra modes for the current CU / PU can be derived from the reconstructed neighboring samples of the current CU / PU. In the DIMM, texture gradient analysis can be performed on both the encoder side and the decoder side to generate multiple candidate intra modes based on the reconstructed neighboring samples. Each of the multiple candidate intra modes can be associated with a respective history of gradients (or respective gradients). Two intra modes (e.g., intra mode 1 and intra mode 2) with the highest gradient history can be selected. The intra mode predictors of the two selected intra modes (e.g., intra mode 1 and intra mode 2) can be combined with a planar mode predictor using a weighted sum. A final intra mode predictor for the current CU / PU can be formed based on a combination of intra mode 1, intra mode 2, and planar.

[0094] Table 1 shows an exemplary DIMD signaling. As shown in Table 1, a DIMD flag (e.g., DIMD_flag) can be signaled before an ISP flag (e.g., ISP_flag). If the DIMD flag is 1 (or true), it can indicate that the CU / PU is currently using DIMD, and the ISP flag can be further parsed to verify whether ISP is currently applied to the CU / PU. If the DIMD flag is not 1 (or false), it can indicate that the CU / PU is not currently using DIMD. Thus, syntax elements related to other intra-coding tools (e.g., MIP, MRL, MPM, etc.) can be parsed at the decoder.

[0095] The context modeling of the DIMD flag may depend on the neighboring CU / PU. For example, the context modeling of the DIMD flag may depend on (i) the availability of the left neighboring CU / PU or the top neighboring CU / PU, and (ii) either the left neighboring CU / PU or the top neighboring CU / PU also uses DIMD. If either the left neighboring CU / PU or the top neighboring CU / PU exists and uses DIMD, the context index (e.g., ctxIdx) may be 1. If both the left neighboring CU / PU and the top neighboring CU / PU exist and use DIMD, the context index may be 2. Otherwise, the context index may be 0. [Table 1] Table 1. DIMM signaling pseudocode

[0096] Template-based intra mode derivation (TIMD) can use a reference sample of the current CU as a template and select an intra mode from a set of candidate intra prediction modes associated with the TIMD. The selected intra mode can be determined as an optimal intra mode based on a cost function, for example. As shown in FIG. 9, neighboring reconstructed samples of the current CU (902) can be used as a template (904). The reconstructed samples of the template (904) can be compared with predicted samples of the template (904). The predicted samples can be generated using a reference sample (906) of the template (904). The reference sample (906) can be a neighboring reconstructed sample around the template (904). The cost function can be used to calculate a cost (or distortion) between the predicted sample and the reconstructed sample in the template (904) based on each of a set of candidate intra prediction modes. The intra prediction mode with the least cost (or distortion) may be selected as the intra prediction mode (eg, the best intra prediction mode) to intra predict the current CU (902).

[0097] Table 2 shows an example TIMD signaling. As shown in Table 2, the TIMD flag (e.g., TIMD_flag) can be signaled if the DIMD flag (e.g., DIMD_flag) is not 1 (or not true). If the DIMD flag is 1, the current CU / PU uses DIMD, and the ISP flag (e.g., ISP_flag) can be parsed to determine whether ISP is used for the current CU / PU. If the DIMD flag is not 1, the TIMD flag is parsed. If the TIMD flag is 1, TIMD can be applied to the current CU / PU without applying other intra-coding tools (e.g., no ISP when using TIMD). If the TIMD flag is not 1 (or false), syntax elements related to other intra-coding tools (e.g., MIP, MRL, MPM, etc.) can be parsed at the decoder. [Table 2] Table 2 Pseudocode for TIMD signaling

[0098] The decoder-side intra prediction can also use explicit signaled intra modes and decoder-side derived intra modes to perform intra prediction at the decoder side, for example, the decoder can use a DIMM to derive an intra mode, and then use the derived intra mode for intra prediction.

[0099] This disclosure may provide hybrid intra prediction for intra predicting a current block in a current picture of a video. Hybrid intra prediction may apply both bitstream-signaled intra modes and decoder-side derived intra modes to generate an intra prediction of a current block. For example, hybrid intra prediction may apply a first intra mode (e.g., intra mode 1) that is signaled using a most probable mode (MPM)-related syntax element in the bitstream, and a second intra mode (e.g., intra mode 2) that is generated based on a decoder-side intra mode derivation. The decoder-side intra mode derivation may be DIMD, TIMD, etc. Thus, the first intra mode (e.g., intra mode 1) is signaled in the bitstream, and the second intra mode (e.g., intra mode 2) is derived at the decoder side without signaling in the bitstream. It should be noted that the decoder-side intra mode derivation is not limited to DIMD or TIMD. The decoder-side intra mode derivation may be any decoder-side intra mode derivation applicable to derive an intra mode for intra predicting the current block. In addition, it should be noted that the first intra mode (e.g., intra mode 1) and the second intra mode (e.g., intra mode 2) may not be the same.

[0100] In one embodiment, a decoder-side derived intra mode (e.g., intra mode 2) may be generated first, and then a bitstream signaled intra mode (e.g., intra mode 1) may be generated subsequently. Table 3 shows exemplary pseudocode for a process of first generating a decoder-derived intra mode (e.g., intra mode 2) and then generating a bitstream signaled intra mode (e.g., intra mode 1). [Table 3] Table 3. Exemplary pseudocode for intra mode generation for hybrid intra prediction

[0101] As shown in Table 3, a decoder-side intra mode derivation flag (e.g., decoder_side_intra_mode_derivation_method_flag) may be obtained from the coded bitstream and parsed (or coded). In response to the decoder-side intra mode derivation flag being 1 (or true), a decoder-side derived intra mode (e.g., intra mode 2) may be derived for the current block. For example, the decoder-side derived intra mode (e.g., intra mode 2) may be derived based on DIMD, where two or more candidate intra modes may be determined based on a histogram of gradients (HoG) associated with neighboring samples (or templates) of the current block. The decoder-side derived intra mode (e.g., intra mode 2) may be one of the two or more candidate intra modes derived by DIMD. In another example, a decoder-side derived intra mode may be derived based on a TIMD, where two or more intra modes may be determined based on cost values ​​(e.g., sum of absolute differences (SAD) or sum of absolute transform differences (SATD)) of neighboring samples (or templates) of a current block. The decoder-side derived intra mode (e.g., intra mode 2) may be one of two or more candidate intra modes derived by the TIMD. Furthermore, syntax elements related to the MPM and the MPM remainder may be coded (or parsed) to generate a bitstream signaled intra mode (e.g., intra mode 1). The bitstream signaled intra mode may be identified in either the MPM list or the MPM remainder. In one example, the MPM list may include 22 intra modes, and the MPM remainder (or non-MPM list) may include 45 intra modes.

[0102] Further referring to Table 3, in response to the decoder-side intra mode derivation flag being 0 (or false), a decoder-side derived intra mode (e.g., intra mode 2) may not be derived and a bitstream signaled intra mode (e.g., intra mode 1) may not be determined. Instead, a syntax element associated with another intra mode may be coded (or parsed) to generate another intra mode. Thus, the current block may not be predicted based on hybrid intra prediction. The current block may be predicted based on another intra mode.

[0103] In another embodiment, a bitstream signaled intra mode (e.g., intra mode 1) may be obtained first, and a decoder-side derived intra mode (e.g., intra mode 2) may be derived thereafter. Table 4 shows an example pseudocode for a process of first generating a bitstream signaled intra mode and then generating a decoder-side derived intra mode. [Table 4] Table 4. Example pseudocode for intra mode generation for hybrid intra prediction

[0104] As shown in Table 4, syntax elements related to the MPM and MPM coset may be coded (or parsed) to first generate a bitstream signaled intra mode (e.g., intra mode 1). The bitstream signaled intra mode (e.g., mode 1) may be identified in either the MPM list or the MPM coset (or non-MPM list) according to the syntax elements related to the MPM and MPM coset. Furthermore, a decoder-side intra mode derivation flag (e.g., decoder_side_intra_mode_derivation_method_flag) may be obtained from the coded bitstream and parsed (or coded). In response to the decoder-side intra mode derivation flag being 1 (or true), a decoder-side derived intra mode (e.g., intra mode 2) may be derived for the current block. For example, the decoder-side derived intra mode (e.g., intra mode 2) may be derived based on DIMD or TIMD.

[0105] With further reference to Table 4, in response to the decoder-side intra mode derivation flag being 0 (or false), a decoder-side derived intra mode (e.g., intra mode 2) may not be generated. Instead, a syntax element associated with another intra mode may be coded (or parsed) to generate another intra mode. The current block may be predicted based on one or a combination of the bitstream signaled intra mode (e.g., intra mode 1) and the other intra modes.

[0106] When the decoder performs intra prediction, a first intra mode (e.g., intra mode 1) may be used to generate a first intra predictor (e.g., intraPredictor1). The first intra predictor may further be subtracted from twice the original sample value. The decoder-side intra mode derivation may use the subtracted value (e.g., twice the original sample value minus the first predictor) as a template and apply a template matching based method to find an optimal (or selected) matching intra mode (e.g., intra mode 2) from two or more candidate intra modes. For example, the decoder may find the optimal intra mode 2 by calculating (2×original sample value−first intra predictor−second intra predictor). Here, each intra predictor may be generated by a respective one of the two or more candidate intra modes. The candidate intra mode with the minimum value can be selected as the optimal (or selected) matching intra mode (eg, intra mode 2).

[0107] For example, the decoder-side derived intra mode (e.g., intra mode 2) can be determined from two or more candidate intra modes derived based on the decoder-side intra mode derivation, such as DIMD or TIMD. To determine the decoder-side derived intra mode (e.g., intra mode 2) from the two or more candidate intra modes, a first intra predictor (e.g., intra predictor 1) of the current block can be obtained based on the bitstream signaled intra mode (e.g., intra mode 1). The first intra predictor can be a first predicted value of a sample in the current block based on the first intra mode. Furthermore, the original sample value of the current block on the encoder side or a reconstructed value of the current block on the decoder side can be determined. The reconstructed value of the current block on the decoder side can be obtained through a predicted value of the current block. In one example, the reconstructed value of the current block can be obtained from the first predictor. In another example, the reconstructed value of the current block can be obtained from two or more candidate intra modes generated from the decoder-side intra mode derivation, such as DIMD or TIMD.

[0108] The first intra predictor (e.g., intra predictor1) can be subtracted from twice the original sample values ​​(or twice the reconstructed values) of the current block. A template can be generated by the subtraction. The template can be the difference between twice the original sample values ​​(or twice the reconstructed values) of the current block and the first intra predictor of the current block. A template matching based method can be further applied to determine a decoder-side derived intra mode (e.g., intra mode2) from two or more candidate intra modes. For example, each candidate predicted value (or each second intra predictor intraPredictor2) of the samples in the current block can be determined based on each of the two or more candidate intra modes. A plurality of template match values ​​(or cost values) may be determined based on a corresponding one of two or more candidate intra modes, where each of the plurality of template match values ​​may be equal to a template minus a respective candidate predicted value (or respective second intra predictor intraPredictor2) of a sample in the current block. An exemplary formula for the template match value may be (2×original sample value (or reconstructed sample value)−first intra predictor−second intra predictor), where the first intra predictor is generated by a bitstream signaling intra mode (e.g., intra mode 1), and the second intra predictor is a second intra predictor generated by one of the two or more candidate intra modes.

[0109] A decoder-side derived intra mode (eg, intra mode 2) may be determined as a selected (or optimal) candidate intra mode of multiple candidate intra modes associated with a minimum template match value of the multiple template match values.

[0110] Once the bitstream signaled intra mode (e.g., intra mode 1) and the decoder-side derived intra mode (e.g., intra mode 2) are determined, the current block can be predicted based on a final intra predictor (or a final predicted value). The final intra predictor (or a final predicted value) can be a weighted combination of the first predictor (or a first predicted value) and the second predictor (or a second predicted value). The first predictor can be obtained based on the first intra mode (or a bitstream signaled intra mode intraMode1), and the second predictor can be obtained based on the second intra mode (or a decoder-side derived intra mode intraMode2). In one example, the weighted combination is equally weighted. Thus, the final intra predictor can be written in Equation 1 as follows:

number

[0111] FIG. 10 is a flow chart outlining an exemplary decoding process (1000) according to some embodiments of the present disclosure. FIG. 11 is a flow chart outlining an exemplary encoding process (1100) according to some embodiments of the present disclosure. The proposed processes may be used separately or combined in any order. Furthermore, each of the processes (or embodiments), the encoder, and the decoder may be implemented by a processing circuit (e.g., one or more processors or one or more integrated circuits). In one example, the one or more processors execute a program stored in a non-transitory computer-readable medium.

[0112] In embodiments, any of the operations of the process (e.g., (1000) and (1100)) may be combined or arranged in any quantity or order as desired. In embodiments, two or more of the operations of the process (e.g., (1000) and (1100)) may be performed in parallel.

[0113] The processes (e.g., (1000) and (1100)) may be used in the reconstruction and / or encoding of a block to generate a prediction block for the block being reconstructed. In various embodiments, the processes (e.g., (1000) and (1100)) are performed by processing circuitry in terminal devices (310), (320), (330), and (340), processing circuitry performing the functions of a video encoder (303), processing circuitry performing the functions of a video decoder (410), processing circuitry performing the functions of a video decoder (510), processing circuitry performing the functions of a video encoder (603), etc. In some embodiments, the processes (e.g., (1000) and (1100)) are implemented with software instructions, such that the processing circuitry performs the processes (e.g., (1000) and (1100)) when the processing circuitry executes the software instructions.

[0114] As shown in Figure 10, the process (1000) may start at (S1001) and proceed to (S1010), where coding information for a current block and neighboring samples of the current block may be received from a coded video bitstream.

[0115] At (S1020), in the coding information, flag information associated with the current block can be obtained. The flag information can indicate whether the current block is intra predicted based at least in part on intra mode derivation, in which multiple candidate intra modes are derived based on neighboring samples.

[0116] In (S1030), in response to flag information indicating that the current block is intra predicted based at least in part on the intra mode derivation, a plurality of candidate intra modes can be determined based on the neighboring samples.

[0117] In (S1040), a first intra mode of the current block can be determined based on a first syntax element included in the coding information. The first syntax element can indicate the first intra mode of the current block from a Most Probable Mode (MPM) list and an MPM remainder.

[0118] At (S1050), a second intra mode of the current block can be determined based on the first intra mode and a plurality of candidate intra modes.

[0119] In (S1060), the current block may be reconstructed based on the first intra mode and the second intra mode.

[0120] In some embodiments, in response to flag information indicating that the current block is intra predicted based at least in part on the intra mode derivation, the first intra mode may be determined based on the first syntax element.

[0121] In the process (1000), in response to flag information indicating that the current block is not intra predicted based on the intra mode derivation, a third intra mode can be determined based on a second syntax element included in the coding information.

[0122] In one embodiment, the intra mode derivation may include decoder-side intra mode derivation (DIMD), in which multiple candidate intra modes are determined based on a histogram of gradients (HoG) associated with neighboring samples.

[0123] In another embodiment, the intra-mode derivation may include template-based intra-mode derivation (TIMD), in which multiple candidate intra-modes are determined based on cost values ​​of neighboring samples.

[0124] To determine a second intra mode of the current block, a first predicted value of the samples in the current block may be determined based on the first intra mode. A reconstructed value of the samples in the current block may be determined based on the first predicted value. A candidate predicted value of each of the samples in the current block may be determined based on each of the multiple candidate intra modes. A plurality of template match values ​​may further be determined. Each of the multiple template match values ​​may be equal to twice the reconstructed value minus (i) the first predicted value and (ii) the respective candidate predicted value based on a corresponding one of the multiple candidate intra modes. A second intra mode of the current block may subsequently be determined, where the second intra mode may be a candidate intra mode of the multiple candidate intra modes associated with a minimum template match value of the multiple template match values.

[0125] A second predicted value of the samples in the current block may be determined based on the second intra mode to reconstruct the current block. The current block may further be reconstructed based on the final predicted value. The final predicted value may be a weighted combination of the first predicted value and the second predicted value.

[0126] In some embodiments, the weighted combination may be equally weighted.

[0127] As shown in Figure 11, the process (1100) may start at (S1101) and proceed to (S1110), where multiple candidate intra modes for the current block may be determined based on neighboring samples of the current block.

[0128] At (S1120), a first intra mode for the current block may be determined from a Most Probable Mode (MPM) list and the MPM remainder.

[0129] At (S1130), a second intra mode of the current block can be determined based on the first intra mode and a plurality of candidate intra modes.

[0130] At (S1140), flag information associated with the current block may be generated. The flag information may indicate whether the current block is intra predicted based at least in part on an intra mode derivation, in which multiple candidate intra modes are derived based on neighboring samples of the current block.

[0131] In (S1150), a first syntax element may be generated. The first syntax element may indicate a first intra mode of the current block from the MPM list and the MPM remainder.

[0132] The techniques described above may be implemented as computer software using computer-readable instructions and physically stored on one or more computer-readable media. For example, FIG. 12 illustrates a computer system (1300) suitable for implementing certain embodiments of the disclosed subject matter.

[0133] Computer software may be coded using any suitable machine code or computer language. It may be assembled, compiled, linked, or otherwise processed by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc., may be subject to similar mechanisms for creating code that contains instructions that may be executed directly, or through an interpreter, microcode execution, etc.

[0134] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smart phones, gaming devices, Internet of Things, and the like.

[0135] 12 for computer system (1300) are exemplary in nature and are not intended to suggest any limitation as to the scope of use or functionality of the computer software implementing embodiments of the present disclosure, nor should the arrangement of components be interpreted as having any dependency or requirement regarding any one or combination of components illustrated in the exemplary embodiment of computer system (1300).

[0136] The computer system (1300) may include certain human interface input devices that may be responsive to input by one or more human users through, for example, tactile input (such as keystrokes, swipes, data glove movements, etc.), audio input (such as voice, clapping, etc.), visual input (such as gestures, etc.), and olfactory input (not shown). The human interface devices may also be used to capture certain media that do not necessarily involve direct human conscious input, such as audio (such as speech, music, ambient sounds, etc.), images (such as scanned images, photographic images acquired from a still image camera, etc.), and video (such as two-dimensional video, three-dimensional video, including stereoscopic vision, etc.).

[0137] The input human interface devices may include one or more of a keyboard (1301), a mouse (1302), a trackpad (1303), a touch screen (1310), a data glove (not shown), a joystick (1305), a microphone (1306), a scanner (1307), and a camera (1308) (only one of each is depicted).

[0138] The computer system (1300) may also include certain human interface output devices that can stimulate one or more of the senses of a human user through, for example, tactile output, sound, light, and smell / taste. Such human interface output devices may include haptic output devices (e.g., a touch screen (1310), data gloves (not shown), or joystick (1305), although there are also haptic feedback devices that do not function as input devices), audio output devices (such as speakers (1309), headphones (not shown)), visual output devices (such as screens (1310), including CRT screens, LCD screens, and plasma screens, each with or without touch screen input capability, and each with or without haptic feedback capability - some of which can output two-dimensional visual output or three or more dimensional output via means such as stereographic output, virtual reality glasses (not shown), holographic displays and smoke tanks (not shown), and printers (not shown).

[0139] The computer system (1300) may also include human accessible storage devices and their associated media, such as optical media including CD / DVD ROM / RW (1320) with associated media such as CD / DVD (1321), thumb-drives (1322), removable hard drives or solid state drives (1323), traditional magnetic media such as tape and floppy disks (not shown), specialized ROM / ASIC / PLD based devices such as security dongles (not shown), and the like.

[0140] Those skilled in the art should also understand that the term "computer readable media" as used in connection with the presently disclosed subject matter does not encompass transmission media, carrier waves, or other transient signals.

[0141] The computer system (1300) may also include an interface (1354) to one or more communication networks (1355). The networks may be, for example, wireless, wired, or optical. The networks may further be local, wide area, metropolitan, vehicular and industrial, real-time, delay tolerant, and the like. Examples of networks include local area networks such as Ethernet, wireless LAN, cellular networks including GSM, 3G, 4G, 5G, LTE, and the like, TV wide area or wide area digital networks including cable TV, satellite TV, and terrestrial broadcast TV, vehicular and industrial including CAN bus, and the like. Certain networks typically require an external network interface adapter (such as, for example, a USB port on the computer system (1300)) to be connected to a certain general-purpose data port or peripheral bus (1349). Others are typically integrated into the core of the computer system (1300) by attachment to a system bus, as described below (such as, for example, an Ethernet interface to a PC computer system, or a cellular network interface to a smartphone computer system). Using any of these networks, the computer system (1300) can communicate with other entities. Such communications may be unidirectional, receive-only (e.g., broadcast television), unidirectional transmit-only (e.g., a CAN bus to a given CAN bus device), or bidirectional, for example, to other computer systems using local or wide area digital networks. Predetermined protocols and protocol stacks may be used with each of these networks and network interfaces, as described above.

[0142] The aforementioned human interface devices, human accessible storage devices, and network interfaces may be attached to a core (1340) of the computer system (1300).

[0143] The cores (1340) may include one or more central processing units (CPUs) (1341), graphics processing units (GPUs) (1342), specialized programmable processing units in the form of field programmable gate arrays (FPGAs) (1343), hardware accelerators for certain tasks (1344), graphics adapters (1350), etc. These devices may be connected via a system bus (1348) along with read only memory (ROM) (1345), random access memory (1346), internal mass storage such as an internal non-user accessible hard drive, SSD, etc. (1347). In some computer systems, the system bus (1348) is accessible in the form of one or more physical plugs to allow expansion with additional CPUs, GPUs, etc. Peripheral devices may be connected directly to the core's system bus (1348) or may be connected via a peripheral bus (1349). In one example, a screen (1310) may be connected to the graphics adapter (1350). Peripheral bus architectures include PCI, USB, and the like.

[0144] The CPU (1341), GPU (1342), FPGA (1343), and accelerator (1344) can execute predetermined instructions, which in combination can constitute the computer code described above. The computer code can be stored in ROM (1345) or RAM (1346). Transient data can also be stored in RAM (1346), while permanent data can be stored in, for example, internal mass storage (1347). Rapid storage and retrieval to any of the memory devices can be made possible through the use of cache memory, which can be closely associated with one or more of the CPU (1341), GPU (1342), mass storage (1347), ROM (1345), RAM (1346), etc.

[0145] The computer-readable media can have computer code thereon for performing various computer-implemented operations. The media and computer code can be those specially designed and constructed for the purposes of the present disclosure, or they can be of the kind well known and available to those skilled in the computer software arts.

[0146] As a non-limiting example, a computer system having the architecture (1300), and in particular the cores (1340), can provide functionality as a result of a processor (including CPU, GPU, FPGA, accelerator, etc.) executing software embodied in one or more tangible, computer-readable media. Such computer-readable media may be media associated with user-accessible mass storage as described above, as well as storage associated with the cores (1340) of a non-transitory nature, such as core-internal mass storage (1347) or ROM (1345). Software implementing various embodiments of the present disclosure may be stored in such devices and executed by the cores (1340). The computer-readable media may include one or more memory devices or chips according to particular needs. The software can cause the core (1340), and in particular the processors therein (including CPUs, GPUs, FPGAs, etc.), to execute the particular processes described herein, or particular portions of the particular processes, including defining data structures stored in RAM (1346) and modifying such data structures according to the processes defined by the software. Additionally or alternatively, the computer system can provide functionality as a result of logic hardwired or otherwise embodied in circuitry (e.g., accelerators (1344)), which can operate in place of or in conjunction with software to execute the particular processes or particular portions of the particular processes described herein. Reference to software includes logic, and vice versa, as appropriate. Reference to a computer-readable medium can include circuitry (such as an integrated circuit (IC)) that stores software for execution, circuitry that embodies logic for execution, or both, as appropriate. The present disclosure encompasses any suitable combination of hardware and software.

[0147] Appendix A: Acronyms JEM: Joint Exploration Model VVC: Versatile Video Coding BMS: Benchmark Set MV: Motion Vector HEVC: High Efficiency Video Coding SEI: Supplemental Extended Information VUI: Video Usability Information GOP: Group of Pictures TU: conversion unit PU: Prediction Unit CTU: Coding Tree Unit CTB: coding tree block PB: Predicted block HRD: Hypothetical Reference Decoder SNR: Signal to Noise Ratio CPU: Central Processing Unit GPU: Graphics Processing Unit CRT: cathode ray tube LCD: Liquid crystal display OLED: Organic Light Emitting Diode CD:Compact Disc DVD: Digital Video Disc ROM: Read-only memory RAM: Random Access Memory ASIC: Application Specific Integrated Circuit PLD: Programmable Logic Device LAN: Local Area Network GSM: Global System for Mobile Communications LTE: Long Term Evolution CANBus: Controller Area Network Bus USB: Universal Serial Bus PCI: Peripheral Component Interconnect FPGA: Field Programmable Gate Array SSD: Solid State Drive IC: Integrated Circuit CU: Coding unit

[0148] While this disclosure has described several exemplary embodiments, modifications, permutations, and various substitute equivalents exist, which are within the scope of this disclosure. Thus, it will be appreciated that those skilled in the art will be able to devise numerous systems and methods which, although not explicitly shown or described herein, embody the principles of the present disclosure and are therefore within its spirit and scope.

Claims

1. 1. A video decoding method performed in a video decoder, comprising: receiving coding information for a current block and neighboring samples of the current block from a coded video bitstream; obtaining flag information associated with the current block in the coding information; the flag information indicating whether the current block is intra predicted based at least in part on an intra mode derivation, where a plurality of candidate intra modes are derived based on the neighboring samples. Steps and determining the plurality of candidate intra modes based on the neighboring samples in response to the flag information indicating that the current block is intra predicted based at least in part on the intra mode derivation; determining a first intra mode for the current block based on a first syntax element included in the coded information; the first syntax element indicates the first intra mode of the current block from a Most Probable Mode (MPM) list and an MPM remainder. Steps and determining a second intra mode of the current block based on the first intra mode and the plurality of candidate intra modes; reconstructing the current block based on the first intra mode and the second intra mode; A method comprising:

2. the first intra mode is determined based on a first syntax element in response to the flag information indicating that the current block is intra predicted based at least in part on the intra mode derivation. The method of claim 1.

3. The method further comprises: in response to the flag information indicating that the current block is not intra predicted based on the intra mode derivation, determining a third intra mode based on a second syntax element included in the coded information; The method of claim 1 , comprising:

4. the intra mode derivation includes a decoder-side intra mode derivation (DIMD), in which the plurality of candidate intra modes are determined based on a histogram of gradients (HoG) associated with the neighboring samples; The method of claim 1.

5. the intra mode derivation includes a template-based intra mode derivation (TIMD), in which the plurality of candidate intra modes are determined based on cost values ​​of the neighboring samples; The method of claim 1.

6. The step of determining the second intra mode of the current block further comprises: determining a first prediction value of a sample in the current block based on the first intra mode; determining reconstructed values ​​of the samples in the current block based on a first predicted value; determining a candidate prediction value for each of the samples in the current block based on each of the plurality of candidate intra-modes; determining a plurality of template match values; each of the plurality of template match values ​​is equal to two times a reconstructed value minus (i) a first predicted value and (ii) a respective candidate predicted value based on a corresponding one of the plurality of candidate intra modes; Steps and determining a second intra mode of the current block; the second intra mode being a candidate intra mode of the plurality of candidate intra modes associated with a minimum template match value of the plurality of template match values. Steps and The method of claim 1 , comprising:

7. The step of reconstructing the current block further comprises: determining a second prediction value of a sample in the current block based on the second intra mode; reconstructing the current block based on the final prediction; the final predicted value being a weighted combination of the first predicted value and the second predicted value. Steps and The method of claim 6, comprising:

8. The weighted combination is equally weighted. The method according to claim 7.

9. An apparatus including a processing circuit, the processing circuit comprising: receiving coding information for a current block and neighboring samples of the current block from a coded video bitstream; obtaining flag information associated with the current block in the coding information; the flag information indicating whether the current block is intra predicted based at least in part on an intra mode derivation, where a plurality of candidate intra modes are derived based on the neighboring samples; determining the plurality of candidate intra modes based on the neighboring samples in response to the flag information indicating that the current block is intra predicted based at least in part on the intra mode derivation; determining a first intra mode for the current block based on a first syntax element included in the coded information; the first syntax element indicates the first intra mode of the current block from a Most Probable Mode (MPM) list and an MPM remainder; determining a second intra mode for the current block based on the first intra mode and the plurality of candidate intra modes; and reconstructing the current block based on the first intra mode and the second intra mode; The apparatus is configured to:

10. the first intra mode is determined based on a first syntax element in response to the flag information indicating that the current block is intra predicted based at least in part on the intra mode derivation.

10. The apparatus of claim 9.

11. The processing circuitry further comprises: in response to the flag information indicating that the current block is not intra predicted based on the intra mode derivation, determining a third intra mode based on a second syntax element included in the coded information; The apparatus of claim 9 , configured to:

12. the intra mode derivation includes a decoder-side intra mode derivation (DIMD), in which the plurality of candidate intra modes are determined based on a histogram of gradients (HoG) associated with the neighboring samples; 10. The apparatus of claim 9.

13. the intra mode derivation includes a template-based intra mode derivation (TIMD), in which the plurality of candidate intra modes are determined based on cost values ​​of the neighboring samples; 10. The apparatus of claim 9.

14. The processing circuitry includes: determining a first prediction value of a sample in the current block based on the first intra mode; determining reconstructed values ​​of the samples in the current block based on the first predicted values; determining a candidate prediction value for each of the samples in the current block based on each of the plurality of candidate intra-modes; determining a plurality of template match values; based on a corresponding one of the plurality of candidate intra-modes, each of the plurality of template match values ​​is equal to twice a reconstructed value minus (i) a first predicted value and (ii) a respective candidate predicted value; and determining a second intra mode of the current block; the second intra mode being a candidate intra mode of the plurality of candidate intra modes associated with a minimum template match value of the plurality of template match values. The apparatus of claim 9 , configured to:

15. The processing circuitry includes: determining a second prediction value for samples within the current block based on the second intra mode; reconstructing the current block based on the final prediction; the final predicted value being a weighted combination of the first predicted value and the second predicted value. The apparatus of claim 14 configured as follows:

16. The weighted combination is equally weighted.

16. The apparatus of claim 15.

17. 1. A computer program comprising one or more computer-executable instructions, the instructions, when executed by at least one processor, causing the at least one processor to: receiving coding information for a current block and neighboring samples of the current block from a coded video bitstream; obtaining flag information associated with the current block in the coding information; the flag information indicating whether the current block is intra predicted based at least in part on an intra mode derivation, where a plurality of candidate intra modes are derived based on the neighboring samples. Steps and determining the plurality of candidate intra modes based on the neighboring samples in response to the flag information indicating that the current block is intra predicted based at least in part on the intra mode derivation; determining a first intra mode for the current block based on a first syntax element included in the coded information; the first syntax element indicates the first intra mode of the current block from a Most Probable Mode (MPM) list and an MPM remainder. Steps and determining a second intra mode of the current block based on the first intra mode and the plurality of candidate intra modes; reconstructing the current block based on the first intra mode and the second intra mode; A computer program that causes the

18. the first intra mode is determined based on a first syntax element in response to the flag information indicating that the current block is intra predicted based at least in part on the intra mode derivation.

18. A computer program product as claimed in claim 17.

19. The instructions, when executed by the at least one processor, cause the at least one processor to: in response to the flag information indicating that the current block is not intra predicted based on the intra mode derivation, determining a third intra mode based on a second syntax element included in the coded information; 20. The computer program product of claim 17,

20. the intra mode derivation includes a decoder-side intra mode derivation (DIMD), in which the plurality of candidate intra modes are determined based on a histogram of gradients (HoG) associated with the neighboring samples; 18. A computer program product as claimed in claim 17.

21. 1. A video encoding method performed in a video encoder, comprising: determining a first intra mode for the current block from a Most Probable Mode (MPM) list and an MPM remainder; determining whether the current block is intra predicted based at least in part on an intra mode derivation, where a plurality of candidate intra modes are derived based on neighboring samples of the current block; determining the plurality of candidate intra modes based on the neighboring samples; determining a second intra mode of the current block based on the first intra mode and the plurality of candidate intra modes; encoding the flag information into a bitstream; the flag information indicating whether the current block is intra predicted or not based at least in part on the intra mode derivation. Steps and encoding a first syntax element into the bitstream; The first syntax element indicates the first intra mode of the current block from an MPM list and an MPM remainder. Steps and encoding the current block based on the first intra mode and the second intra mode; A method comprising:

Images