Method and electronic device for video encoding

Abstract

A method and an electronic device for video coding are provided. The electronic device performs the method of updating a most probable mode candidate list for a current block of video data. The electronic device first identifies a neighboring block located at a predefined position relative to the current block and its associated matrix-based intra prediction mode. Next, the electronic device determines a regular intra prediction mode corresponding to the matrix-based intra prediction mode for the neighboring block according to a predefined mathematical relationship between the regular intra prediction mode and the matrix-based intra prediction mode. Finally, the electronic device inserts the regular intra prediction mode associated with the neighboring block into the most probable mode candidate list according to a predefined order. If the regular intra prediction mode is signaled in the semantics of a video bitstream including the current block, a video decoder will predict the current block from the reconstructed neighboring block according to the regular intra prediction mode.

Description

[0001] This application is a divisional application of the application for invention application number 202080028123.X, application date 9th April 2020, titled “Video coding method and apparatus using improved matrix-based intra prediction coding mode”. TECHNICAL FIELD

[0002] The present application relates generally to video data encoding and decoding, and in particular, to methods and systems for video coding using matrix-based intra prediction (MBIP) coding mode. BACKGROUND

[0003] Digital video is supported by a variety of electronic devices, such as digital televisions, laptop or desktop computers, tablet computers, digital recording devices, digital media players, video gaming devices, smart phones, video teleconferencing devices, video streaming devices, etc. The electronic devices transmit, receive, encode, decode, and / or store digital video data by implementing video compression / decompression standards defined by MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-4, Part 10, Advanced Video Coding (AVC), High Efficiency Video Coding (HEVC), and Versatile Video Coding (VVC) standards. Video compression typically includes performing spatial (intra-frame) prediction and / or temporal (inter-frame) prediction to reduce or remove redundancy inherent in the video data. For block-based video coding, a video frame is partitioned into one or more slices, each slice having multiple video blocks, which can also be referred to as coding tree units (CTUs). Each CTU can contain one coding unit (CU) or is recursively partitioned into smaller CUs until a predefined minimum CU size is reached. Each CU, also referred to as a leaf-CU, contains one or more transform units (TUs) and each CU also contains one or more prediction units (PUs). Each CU can be coded in intra mode, inter mode, or IBC mode. Video blocks in intra coded (I) slices of a video frame are encoded using spatial prediction with respect to reference samples in neighboring blocks within the same video frame. Video blocks in inter coded (P or B) slices of a video frame can be encoded using spatial prediction with respect to reference samples in neighboring blocks within the same video frame or temporal prediction with respect to reference samples in other previously encoded and / or future predicted video frames.

[0004] A prediction block for a current video block to be coded is derived based on spatial or temporal prediction of a previously coded reference block, e.g., a neighboring block. The process of finding the reference block can be done by block matching algorithms. Residual data representing pixel differences between the current block to be coded and the prediction block is referred to as a residual block or prediction error. An inter-coded block is coded according to a motion vector pointing to a reference block forming the prediction block in a reference frame and the residual block. The process of determining the motion vector is commonly referred to as motion estimation. An intra-coded block is coded according to an intra prediction mode and the residual block. To further compress, the residual block is transformed from the pixel domain to a transform domain, e.g., frequency domain, resulting in residual transform coefficients that can then be quantized. The quantized transform coefficients, which are initially arranged in a two-dimensional array, can be scanned to produce a one-dimensional vector, and then entropy coded into a video bitstream for even more compression.

[0005] The coded video bitstream is then saved in a computer-readable storage medium, e.g., flash memory, to be accessed by another electronic device with digital video capability or directly sent to the electronic device wired or wirelessly. The electronic device then performs video decompression, which is the reverse process of the video compression described above, e.g., obtaining the syntax elements from the bitstream by parsing the coded video bitstream, and reconstructing the digital video data from the coded video bitstream into its original format based at least in part on the syntax elements obtained from the bitstream, and renders the reconstructed digital video data on a display of the electronic device.

[0006] As the quality of digital videos increases from high definition to 4K x 2K or even 8K x 4K, the amount of video data to be encoded / decoded increases exponentially. It has been a long-standing challenge in how to code / decode video data more efficiently, while maintaining the image quality of the decoded video data.

[0007] For example, a conventional intra prediction mode performs angular prediction for a current coding block by directly copying or interpolating from reconstructed pixels from neighboring blocks. As a result, the predicted samples using the conventional intra prediction mode have limited freedom in pixel value variation, especially along the prediction direction. To further improve coding efficiency, a matrix-based intra prediction (MBIP) mode is introduced by applying a linear matrix transform to the reconstructed pixels in the neighboring blocks to predict the samples of the current coding block. However, current implementations of MBIP pose new challenges to hardware / software implementation, e.g., requiring complex look-up table operations between coding blocks using different types of intra prediction methods, and occupying a large amount of space (especially on-chip) for storing the matrix coefficients. SUMMARY

[0008] This application describes implementations related to video data encoding and decoding, and more specifically, implementations related to systems and methods of video encoding and decoding using improved matrix-based intra prediction (MBIP) coding mode.

[0009] According to a first aspect of the present application, a method of updating a most probable mode candidate list for a current block of video data is performed at an electronic device having one or more processing units and a memory storing a plurality of programs to be executed by the one or more processing units. The method includes identifying a neighboring block located at a predefined position relative to the current block and its associated matrix-based intra prediction mode; determining a regular intra prediction mode corresponding to the matrix-based intra prediction mode for the neighboring block according to a predefined mathematical relationship between the regular intra prediction mode and the matrix-based intra prediction mode; and inserting the regular intra prediction mode associated with the neighboring block into the most probable mode candidate list according to a predefined order.

[0010] According to a second aspect of the present application, an electronic device includes one or more processing units, a memory, and a plurality of programs stored in the memory. These programs, when executed by the one or more processing units, cause the electronic device to perform the method of updating a most probable mode candidate list for a current block of video data as described above.

[0011] According to a third aspect of the present application, a non-transitory computer- readable storage medium stores a plurality of programs for execution by an electronic device having one or more processing units. These programs, when executed by the one or more processing units, cause the electronic device to perform the method of updating a most probable mode candidate list for a current block of video data as described above.

[0012] According to a fourth aspect of the present application, a method of predicting a current block of video data using matrix-based intra prediction is performed at an electronic device having one or more processing units and a memory storing a plurality of programs to be executed by the one or more processing units. The method includes identifying one or more neighboring blocks relative to the current block; selecting a matrix-based intra prediction mode for predicting the current block among a plurality of matrix-based intra prediction modes; retrieving, from a storage device, coefficients of a matrix and a bias vector corresponding to the selected matrix-based intra prediction mode; and performing matrix-based intra prediction on the identified one or more neighboring blocks using the retrieved coefficients of the matrix and the bias vector.

[0013] According to a fifth aspect of the present application, an electronic device includes one or more processing units, memory, and a plurality of programs stored in the memory. The programs, when executed by the one or more processing units, cause the electronic device to perform the method of predicting a current block of video data using matrix-based intra prediction as described above.

[0014] According to a sixth aspect of the present application, a non-transitory computer- readable storage medium stores a plurality of programs for execution by an electronic device having one or more processing units. The programs, when executed by the one or more processing units, cause the electronic device to perform the method of predicting a current block of video data using matrix-based intra prediction as described above. BRIEF DESCRIPTION OF DRAWINGS

[0015] The accompanying drawings are included to provide a further understanding of implementations and are incorporated in and constitute a part of the specification. The drawings illustrate the described implementations and, together with the description, serve to explain the principles involved.

[0016] Figure 1 is a block diagram illustrating an exemplary video encoding and decoding system, in accordance with some implementations of the present disclosure.

[0017] Figure 2 is a block diagram illustrating an exemplary video encoder, in accordance with some implementations of the present disclosure.

[0018] Figure 3 is a block diagram illustrating an exemplary video decoder, in accordance with some implementations of the present disclosure.

[0019] Figures 4A to 4D is a block diagram illustrating how a frame is recursively quad-tree partitioned into video blocks of different sizes, in accordance with some implementations of the present disclosure.

[0020] Figure 5A is a block diagram illustrating 67 candidate intra prediction modes for predicting a current coding block based on reconstructed neighboring blocks, in accordance with some implementations of the present disclosure.

[0021] Figure 5B is a block diagram illustrating exemplary locations of five reconstructed neighboring blocks of a current coding block, in accordance with some implementations of the present disclosure.

[0022] Figure 6A and Figure 6B is a block diagram illustrating two matrix-based intra prediction schemes for coding blocks of different sizes, in accordance with some implementations of the present disclosure.

[0023] Figure 7is a flowchart showing an exemplary process by which a video codec implements techniques for generating a most probable mode (MPM) candidate list, according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0024] Reference will now be made in detail to specific implementations, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous non-limiting specific details are set forth in order to assist in understanding the subject matter presented herein. But it will be apparent to one of ordinary skill in the art that various alternatives can be used without departing from the scope of the claims and the subject matter can be practiced without these specific details. For example, it will be apparent to one of ordinary skill in the art that the subject matter presented herein can be implemented on many types of electronic devices with digital video capabilities.

[0025] Figure 1 is a block diagram showing an exemplary system 10 for encoding and decoding video blocks in parallel, according to some embodiments of the present disclosure. As shown in Figure 1 The system 10 includes a source device 12 that generates and encodes video data to be decoded at a later time by a destination device 14, as shown in

[0026] In some embodiments, the destination device 14 can receive encoded video data to be decoded via a link 16. The link 16 can comprise any type of communication medium or device capable of moving the encoded video data from the source device 12 to the destination device 14. In one example, the link 16 can comprise a communication medium to enable the source device 12 to transmit encoded video data directly to the destination device 14 in real-time. The encoded video data can be modulated according to a communication standard, such as a wireless communication protocol, and transmitted to the destination device 14. The communication medium can comprise any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium can form part of a packet-based network, such as a local area network, a wide-area network, or a global network such as the Internet. The communication medium can include routers, switches, base stations, or any other equipment that can be useful to facilitate communication from the source device 12 to the destination device 14.

[0027] In some other embodiments, encoded video data may be sent from output interface 22 to storage device 32. The encoded video data in storage device 32 may then be accessed by target device 14 via input interface 28. Storage device 32 may include any data storage medium of various distributed or locally accessed data storage media, such as hard disk drives, Blu-ray discs, DVDs, CD-ROMs, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In another example, storage device 32 may correspond to a file server or another intermediate storage device that holds the encoded video data generated by source device 12. Target device 14 may access the stored video data from storage device 32 via streaming or downloading. The file server may be any type of computer capable of storing and sending encoded video data to target device 14. Exemplary file servers include web servers (e.g., for websites), FTP servers, network attached storage (NAS) devices, or local disk drives. The target device 14 can access the encoded video data via any standard data connection suitable for accessing the encoded video data stored on the file server. Standard data connections include wireless channels (e.g., Wi-Fi connections), wired connections (e.g., DSL, cable modems, etc.), or a combination of both. The transmission of the encoded video data from the storage device 32 can be streaming, downloading, or a combination of both.

[0028] like Figure 1 As shown, source device 12 includes a video source 18, a video encoder 20, and an output interface 22. Video source 18 may include sources or combinations of such sources, such as: a video capture device (e.g., a camera), a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video. As an example, if video source 18 is a camera in a security monitoring system, source device 12 and target device 14 may form a camera phone or video phone. However, the embodiments described in this application are generally applicable to video encoding and decoding and can be applied to wireless and / or wired applications.

[0029] Captured, pre-captured, or computer-generated video can be encoded by video encoder 20. The encoded video data can be sent directly to target device 14 via output interface 22 of source device 12. The encoded video data can also (or alternatively) be stored on storage device 32 for later access by target device 14 or other devices for decoding and / or playback. Output interface 22 may further include a modem and / or transmitter.

[0030] Target device 14 includes an input interface 28, a video decoder 30, and a display device 34. Input interface 28 can include a receiver and / or a modem and receive encoded video data over link 16. The encoded video data transmitted over link 16, or provided on storage device 32, can include various syntax elements generated by video encoder 20 for use by video decoder 30 in decoding the video data. Such syntax elements can be included within the encoded video data transmitted on a communication medium, stored on a storage medium, or stored on a file server.

[0031] In some implementations, target device 14 can include a display device 34, which can be an integrated display device and an external display device configured to communicate with target device 14. Display device 34 displays the decoded video data to a user and can include any of various display devices such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device.

[0032] Video encoder 20 and video decoder 30 can operate according to a proprietary standard or industry standard, such as VVC, HEVC, MPEG-4, Part 10, Advanced Video Coding (AVC), or extensions of such standards. It should be understood that the application is not limited to a specific video coding / decoding standard and can apply to other video coding / decoding standards. It is generally contemplated that video encoder 20 of source device 12 can be configured to encode video data according to any of these current or future standards. Similarly, it is also generally contemplated that video decoder 30 of target device 14 can be configured to decode video data according to any of these current or future standards.

[0033] Video encoder 20 and video decoder 30 each can be implemented as any of a variety of suitable encoder circuitry, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware or any combinations thereof. When implemented partially in software, an electronic device can store instructions for the software in a suitable, non- transitory computer-readable medium and execute the instructions in hardware using one or more processors to perform the video coding / decoding operations disclosed in the present disclosure. Each of video encoder 20 and video decoder 30 can be included in one or more encoders or decoders, either of which can be integrated as part of a combined encoder / decoder (CODEC) in a respective device.

[0034] Figure 2is a block diagram illustrating an example video encoder 20, in accordance with some implementations described in this application. Video encoder 20 can perform intra-prediction coding and inter-prediction coding of video blocks within a video frame. Intra-prediction coding relies on spatial prediction to reduce or remove spatial redundancy in the video data within a given video frame or picture. Inter-prediction coding relies on temporal prediction to reduce or remove temporal redundancy in the video data within neighboring video frames or pictures of a video sequence.

[0035] As Figure 2 Video encoder 20 includes video data memory 40, prediction processing unit 41, decoded picture buffer (DPB) 64, summer 50, transform processing unit 52, quantization unit 54, and entropy encoding unit 56, as shown in FIG. 1. Prediction processing unit 41 further includes motion estimation unit 42, motion compensation unit 44, partition unit 45, intra-prediction processing unit 46, and intra-block copy (BC) unit 48. In some implementations, video encoder 20 also includes inverse quantization unit 58, inverse transform processing unit 60, and summer 62 for video block reconstruction. A deblocking filter (not shown) can be located between summer 62 and DPB 64 to filter block boundaries to remove blockiness artifacts from reconstructed video. In addition to the deblocking filter, an in-loop filter (not shown) can be used to filter the output of summer 62. Video encoder 20 can take the form of a fixed or programmable hardware unit, or can be dispersed in one or more of the illustrated fixed or programmable hardware units.

[0036] Video data memory 40 can store video data to be encoded by the components of video encoder 20. The video data in video data memory 40 can be obtained, for example, from video source 18. DPB 64 is a buffer that stores reference video data used by video encoder 20 (e.g., in intra- or inter-prediction coding modes) when encoding video data. Video data memory 40 and DPB 64 can be formed by any of a variety of memory devices. In various examples, video data memory 40 can be on-chip with other components of video encoder 20, or off-chip relative to those components.

[0037] As Figure 2As shown in FIG. 1, after receiving video data, partitioning unit 45 within prediction processing unit 41 partitions the video data into video blocks. This partitioning can also include partitioning of video frames into slices, tiles, or other larger coding units (CUs) in accordance with a predefined partitioning structure, such as a quadtree structure, associated with the video data. Video frames can be partitioned into multiple video blocks (or sets of video blocks referred to as tiles). Prediction processing unit 41 can select one of a plurality of possible predictive coding modes, such as one of a plurality of intra-predictive coding modes or one of a plurality of inter-predictive coding modes, for the current video block based on error results (e.g., rate and distortion levels). Prediction processing unit 41 can provide the resulting intra- or inter-predicted block to summer 50 to generate a residual block, and to summer 62 to reconstruct the encoded block for use as part of a reference frame at a later time. Prediction processing unit 41 also provides syntax elements, such as motion vectors, intra-mode indicators, partitioning information, and other such syntax information, to entropy encoding unit 56.

[0038] To select an appropriate intra-predictive coding mode for a current video block, intra-prediction processing unit 46 within prediction processing unit 41 can perform intra-predictive coding of the current video block relative to one or more neighboring blocks in the same frame as the current block to be coded to provide spatial prediction. Motion estimation unit 42 and motion compensation unit 44 within prediction processing unit 41 perform inter-predictive coding of the current video block relative to one or more prediction blocks in one or more reference frames to provide temporal prediction. Video encoder 20 can perform multiple coding passes, e.g., to select an appropriate coding mode for each block of video data.

[0039] In some implementations, motion estimation unit 42 determines an inter-prediction mode for a current video frame by generating motion vectors according to a predetermined pattern within a sequence of video frames, the motion vectors indicating displacement of a prediction unit (PU) of a video block within the current video frame relative to a prediction block within a reference video frame. Motion estimation performed by motion estimation unit 42 is a process of generating motion vectors that estimate motion for video blocks. A motion vector, for example, can indicate displacement of a PU of a video block within a current video frame or picture relative to a prediction block within a reference frame (or other coding unit), the prediction block being relative to a current block being coded within the current frame (or other coding unit). The predetermined pattern can designate video frames in the sequence as P-frames or B-frames. Intra-BC unit 48 can determine vectors (e.g., block vectors) for intra-BC coding in a similar manner as motion vectors are determined for inter-prediction by motion estimation unit 42, or can utilize block vectors determined by motion estimation unit 42.

[0040] In terms of pixel differences, a prediction block is a block of a reference frame that is deemed to closely match a PU of a video block being coded, and pixel differences can be determined by a sum of absolute differences (SAD), sum of squared differences (SSD), or other difference metrics. In some implementations, video encoder 20 can calculate values for sub-integer pixel positions of reference frames stored in DPB 64. For example, video encoder 20 can interpolate values for quarter-pel positions, eighth-pel positions, or other fractional-pel positions of a reference frame. Thus, motion estimation unit 42 can perform a motion search with respect to both full-pel positions and fractional-pel positions, and output motion vectors with fractional-pel precision.

[0041] Motion estimation unit 42 calculates motion vectors for PUs of a video block in an inter-predicted coded frame by comparing a location of the PU to a location of a prediction block of a reference frame selected from a first reference frame list (List 0) or a second reference frame list (List 1), each of which identifies one or more reference frames stored in DPB 64. Motion estimation unit 42 sends the calculated motion vectors to motion compensation unit 44, and then to entropy encoding unit 56.

[0042] Motion compensation performed by motion compensation unit 44 can involve extracting or generating a prediction block based on the motion vectors determined by motion estimation unit 42. Upon receiving a motion vector for a PU of a current video block, motion compensation unit 44 can locate the prediction block pointed to by the motion vector in one of the reference frame lists, retrieve the prediction block from DPB 64, and forward the prediction block to summer 50. Summer 50 then forms a residual video block of pixel difference values by subtracting the pixel values of the prediction block provided by motion compensation unit 44 from the pixel values of the current video block being coded. The pixel difference values forming the residual video block can include luma difference components or chroma difference components, or both. Motion compensation unit 44 can also generate semantic elements associated with a video block of a video frame for use by video decoder 30 in decoding the video block of the video frame. The semantic elements can include, for example, semantic elements defining motion vectors used to identify the prediction block, any flags indicating a prediction mode, or any other semantic information described herein. Note that motion estimation unit 42 and motion compensation unit 44 can be highly integrated, but are illustrated separately for conceptual purposes.

[0043] In some implementations, intra BC unit 48 can generate vectors and extract prediction blocks in a manner similar to that described above in connection with motion estimation unit 42 and motion compensation unit 44, but the prediction blocks are in the same frame as the current block being coded, and the vectors are referred to as block vectors rather than motion vectors. In particular, intra BC unit 48 can determine an intra prediction mode to use for encoding the current block. In some examples, intra BC unit 48 can encode the current block using various intra prediction modes, e.g., during a separate encoding pass, and test their performance through rate-distortion analysis. Next, intra BC unit 48 can select an appropriate intra prediction mode to use among the various tested intra prediction modes, and generate an intra mode indicator accordingly. For example, intra BC unit 48 can compute rate-distortion values for the various tested intra prediction modes using rate-distortion analysis, and select the intra prediction mode with the best rate-distortion characteristics among the tested modes as the appropriate intra prediction mode to use. Rate-distortion analysis generally determines the amount of distortion (or error) between an encoded block and the original, unencoded block that was encoded to produce the encoded block, and the bit rate (i.e., the number of bits) used to produce the encoded block. Intra BC unit 48 can compute the ratio from the distortion and the rate for various encoded blocks to determine which intra prediction mode exhibits the best rate-distortion value for the block.

[0044] In other examples, intra BC unit 48 can perform such functions for intra BC prediction according to the implementations described herein using motion estimation unit 42 and motion compensation unit 44 in whole or in part. In either case, for intra block copy, in terms of pixel difference, the prediction block can be a block that is deemed to closely match the block to be coded, the pixel difference can be determined by sum of absolute difference (SAD), sum of squared difference (SSD), or other difference metric, and the identification of the prediction block can include computing values for sub-integer pixel positions.

[0045] Whether the prediction block is from the same frame according to intra prediction or a different frame according to inter prediction, video encoder 20 can form pixel difference values by subtracting pixel values of the prediction block from pixel values of the current video block being coded, thereby forming a residual video block. The pixel difference values forming the residual video block can include both luma component differences and chroma component differences.

[0046] As an alternative to inter prediction performed by motion estimation unit 42 and motion compensation unit 44 or intra block copy prediction performed by intra BC unit 48, as described above, intra prediction processing unit 46 can intra predict the current video block. In particular, intra prediction processing unit 46 can determine an intra prediction mode to use for encoding the current block. To do so, intra prediction processing unit 46 can encode the current block using various intra prediction modes, e.g., during a separate encoding pass, and intra prediction processing unit 46 (or, in some examples, a mode selection unit) can select an appropriate intra prediction mode to use from among the intra prediction modes that have been tested. Intra prediction processing unit 46 can provide information indicating the selected intra prediction mode for the block to entropy encoding unit 56. Entropy encoding unit 56 can encode the information indicating the selected intra prediction mode in the bitstream.

[0047] After prediction processing unit 41 determines a prediction block for the current video block via inter prediction or intra prediction, summer 50 forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block can be included in one or more transform units (TUs) and provided to transform processing unit 52. Transform processing unit 52 transforms the residual video data into residual transform coefficients using a transform, such as a discrete cosine transform (DCT) or a conceptually similar transform.

[0048] Transform processing unit 52 can send the resulting transform coefficients to quantization unit 54. Quantization unit 54 quantizes the transform coefficients to further reduce bit rate. The quantization process can also reduce the bit depth associated with some or all of the coefficients. The degree of quantization can be modified by adjusting a quantization parameter. In some examples, quantization unit 54 can then perform a scan of the matrix including the quantized transform coefficients. Alternatively, entropy encoding unit 56 can perform the scan.

[0049] Following quantization, entropy encoding unit 56 entropy encodes the quantized transform coefficients using, for example, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), Probability Interval Partitioning Entropy (PIPE) coding or another entropy encoding methodology or technique, into the video bitstream. The encoded bitstream can then be transmitted to video decoder 30, or archived on a storage device 32 for later transmission to or retrieval by video decoder 30. Entropy encoding unit 56 can also entropy encode motion vectors and other syntax elements for the current video frame that are being coded.

[0050] The inverse quantization unit 58 and the inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct the residual video block in the pixel domain for generating a reference block used to predict other video blocks. As noted above, the motion compensation unit 44 can generate a motion-compensated predicted block from one or more reference blocks of frames stored in the DPB 64. The motion compensation unit 44 can also apply one or more interpolation filters to the predicted block to calculate sub-integer pixel values for use in motion estimation.

[0051] The summer 62 adds the reconstructed residual block to the motion-compensated predicted block produced by the motion compensation unit 44 to produce a reference block for storage in the DPB 64. The reference block can then be used by the intra BC unit 48, the motion estimation unit 42, and the motion compensation unit 44 as a prediction block to inter predict another video block in a subsequent video frame.

[0052] Figure 3 FIG. 1 is a block diagram illustrating an example video decoder 30, in accordance with some embodiments of the present application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, a summer 90, and a DPB 92. The prediction processing unit 81 further includes a motion compensation unit 82, an intra prediction unit 84, and an intra BC unit 85. The video decoder 30 can perform decoding processes generally reciprocal to the encoding processes described above in connection with the video encoder 20. Figure 2 The decoding processes described in relation to the video encoder 20 are generally reciprocal. For example, the motion compensation unit 82 can generate prediction data based on motion vectors received from the entropy decoding unit 80, while the intra prediction unit 84 can generate prediction data based on intra prediction mode indicators received from the entropy decoding unit 80.

[0053] In some examples, the units of the video decoder 30 can be tasked to perform embodiments of the present application. Moreover, in some examples, embodiments of the present disclosure can be dispersed among one or more of the plurality of units of the video decoder 30. For example, the intra BC unit 85 can perform embodiments of the present application, alone or in combination with other units of the video decoder 30, such as the motion compensation unit 82, the intra prediction unit 84, and the entropy decoding unit 80. In some examples, the video decoder 30 can not include the intra BC unit 85, and the functionality of the intra BC unit 85 can be performed by other components of the prediction processing unit 81, such as the motion compensation unit 82.

[0054] Video data memory 79 can store video data to be decoded by the other components of video decoder 30, such as an encoded video bitstream. The video data stored in video data memory 79 can be obtained, for example, from storage device 32, from a local video source, such as a camera, via wired or wireless network communication of video data, or by accessing physical data storage media (e.g., a flash drive or hard drive). Video data memory 79 can include a coded picture buffer (CPB) that stores encoded video data from an encoded video bitstream. Decoded picture buffer (DPB) 92 of video decoder 30 stores reference video data for use in decoding video data by video decoder 30 (e.g., in intra- or inter-coding modes). Video data memory 79 and DPB 92 can be formed by any of a variety of memory devices, such as dynamic random access memory (DRAM) including synchronous DRAM (SDRAM), magneto resistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. For Figure 3 illustrative purposes, video data memory 79 and DPB 92 are depicted as two distinct components of video decoder 30. But it will be readily apparent to one of ordinary skill in the art that video data memory 79 and DPB 92 can be provided by the same memory device or separate memory devices. In some examples, video data memory 79 can be on-chip with other components of video decoder 30, or off-chip relative to those components.

[0055] During the decoding process, video decoder 30 receives an encoded video bitstream that represents video blocks of encoded video frames and associated syntax elements. Video decoder 30 can receive the syntax elements at the video frame level and / or video block level. Entropy decoding unit 80 of video decoder 30 entropy decodes the bitstream to generate quantized coefficients, motion vectors, or intra- prediction mode indicators, among other syntax elements. Entropy decoding unit 80 then forwards the motion vectors and other syntax elements to prediction processing unit 81.

[0056] When a video frame is coded as an intra-predicted coded (I) frame or as an intra coded prediction block in other types of frames, intra-prediction unit 84 of prediction processing unit 81 can generate prediction data for a video block of the current video frame based on the signaled intra-prediction mode and reference data from previously decoded blocks of the current frame.

[0057] When a video frame is coded as an inter-predicted coded (i.e., B or P) frame, motion compensation unit 82 of prediction processing unit 81 generates one or more prediction blocks for a video block of the current video frame based on motion vectors and other syntax elements received from entropy decoding unit 80. Each of the prediction blocks can be generated from a reference frame within one of the reference frame lists. Video decoder 30 can construct the reference frame lists, List 0 and List 1, using default construction techniques based on reference frames stored in DPB 92.

[0058] In some examples, when a video block is coded according to the intra BC mode described herein, intra BC unit 85 of prediction processing unit 81 generates a prediction block for the current video block based on block vectors and other syntax elements received from entropy decoding unit 80. The prediction block can be within a reconstructed region of the same picture as the current video block, as defined by video encoder 20.

[0059] Motion compensation unit 82 and / or intra BC unit 85 determine prediction information for a video block of the current video frame by parsing the motion vectors and other syntax elements, and then use the prediction information to generate a prediction block for the current video block being decoded. For example, motion compensation unit 82 uses some of the received syntax elements to determine a prediction mode used to code the video block of the video frame (e.g., intra or inter prediction), an inter-prediction frame type (e.g., B or P), construction information for one or more of the reference frame lists for the frame, motion vectors for each inter-predicted coded video block of the frame, an inter-prediction status for each inter-predicted coded video block of the frame, and other information used to decode the video block in the current video frame.

[0060] Similarly, intra BC unit 85 can use some of the received syntax elements, such as flags, to determine whether the current video block is predicted using the intra BC mode, construction information for which video blocks of the frame are within the reconstructed region and should be stored in DPB 92, block vectors for each intra BC predicted video block of the frame, an intra BC prediction status for each intra BC predicted video block of the frame, and other information used to decode the video block in the current video frame.

[0061] Motion compensation unit 82 can also perform interpolation to calculate interpolated values for sub-integer pixels of a reference block using interpolation filters as used by video encoder 20 during encoding of the video block. In this case, motion compensation unit 82 can determine the interpolation filters used by video encoder 20 from the received syntax elements, and use these interpolation filters to generate the prediction block.

[0062] Inverse quantization unit 86 inverse quantizes quantized transform coefficients provided in the bitstream and entropy decoded by entropy decoding unit 80 using the same quantization parameter calculated by video encoder 20 for each video block in the video frame to determine a degree of quantization. Inverse transform processing unit 88 applies an inverse transform, e.g., an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process, to the transform coefficients in order to reconstruct the residual blocks in the pixel domain.

[0063] After motion compensation unit 82 or intra BC unit 85 generates a prediction block for the current video block based on the vectors and other syntax elements, adder 90 reconstructs a decoded video block for the current video block by adding the residual block from inverse transform processing unit 88 to the corresponding prediction block generated by motion compensation unit 82 and intra BC unit 85. In-loop filters (not shown) can be located between adder 90 and DPB 92 to further process the decoded video block. The decoded video block in a given frame is then stored in DPB 92, which stores reference frames for subsequent motion compensation of video blocks that follow. DPB 92, or a separate memory device from DPB 92, can also store decoded video for later presentation on a display device, such as display device 34 of FIG. 1. Figure 1

[0064] In a typical video coding process, a video sequence generally includes an ordered set of frames or pictures. Each frame can include three arrays of samples, denoted as SL, SCb, and SCr. SL is a two-dimensional array of luma samples. SCb is a two-dimensional array of Cb chroma samples. SCr is a two-dimensional array of Cr chroma samples. In other cases, the frame can be monochrome and thus include only one two-dimensional array of luma samples.

[0065] As shown in Figure 4A Video encoder 20 (or more specifically partition unit 45) generates an encoded representation of a frame by first partitioning the frame into a set of coding tree units (CTUs), as shown in FIG. 2. A video frame can include an integer number of CTUs ordered consecutively in a raster scan order from left to right and top to bottom. Each CTU is a largest logical coding unit and the width and height of the CTU are signaled by video encoder 20 in a sequence parameter set such that all CTUs in the video sequence have the same size of one of 128x128, 64x64, 32x32, and 16x16. But it should be noted that the present application is not necessarily limited to the particular sizes. As shown in Figure 4B ​As illustrated in FIG. 1, each CTU can include one coding tree block (CTB) of luma samples, two corresponding coding tree blocks of chroma samples, and syntax elements used to code the samples of the coding tree blocks. The syntax elements describe the properties of different types of units of a coded block of pixels and how a video sequence can be reconstructed at the video decoder 30, including inter or intra prediction, intra prediction mode, motion vectors, and other parameters. In monochrome pictures or pictures with three separate color planes, a CTU can include a single coding tree block and syntax elements used to code the samples of the coding tree block. A coding tree block can be an NxN block of samples.

[0066] To achieve better performance, the video encoder 20 can recursively perform tree partitioning, such as binary tree partitioning, quad tree partitioning, or a combination of both, on the coding tree blocks of a CTU and divide the CTU into smaller coding units (CUs). As Figure 4C As depicted in FIG. 4, the 64x64 CTU 400 is first divided into four smaller CUs, each having a block size of 32x32. Among the four smaller CUs, the CU 410 and the CU 420 are each divided into four 16x16 CUs in block size. The two 16x16 CUs 430 and 440 are each further divided into four 8x8 CUs in block size. Figure 4D A quad tree data structure is depicted, showing the final result of the partitioning process of the CTU 400 as Figure 4C As depicted in FIG. 4, the 64x64 CTU 400 is first divided into four smaller CUs, each having a block size of 32x32. Among the four smaller CUs, the CU 410 and the CU 420 are each divided into four 16x16 CUs in block size. The two 16x16 CUs 430 and 440 are each further divided into four 8x8 CUs in block size. Figure 4B As depicted in FIG. 4, the 64x64 CTU 400 is first divided into four smaller CUs, each having a block size of 32x32. Among the four smaller CUs, the CU 410 and the CU 420 are each divided into four 16x16 CUs in block size. The two 16x16 CUs 430 and 440 are each further divided into four 8x8 CUs in block size.

[0067] In some implementations, the video encoder 20 can further partition the coding blocks of a CU into one or more prediction blocks (PBs) of MxN. A prediction block is a rectangular (square or non-square) block of samples for which the same prediction (inter or intra prediction) is applied. A prediction unit (PU) of a CU can include a prediction block of luma samples, two corresponding prediction blocks of chroma samples, and syntax elements used to predict the prediction blocks. In monochrome pictures or pictures with three separate color planes, a PU can include a single prediction block and syntax structures used to predict the prediction block. The video encoder 20 can generate a predicted luma block, a predicted Cb block, and a predicted Cr block for the luma prediction block, the Cb prediction block, and the Cr prediction block of each PU of a CU.

[0068] Video encoder 20 can use intra prediction or inter prediction to generate the prediction block for a PU. If video encoder 20 uses intra prediction to generate the prediction block for a PU, video encoder 20 can generate the prediction block for the PU based on decoded samples of the frame associated with the PU. If video encoder 20 uses inter prediction to generate the prediction block for a PU, video encoder 20 can generate the prediction block for the PU based on decoded samples of one or more frames other than the frame associated with the PU.

[0069] After video encoder 20 generates the prediction luma blocks, prediction Cb blocks, and prediction Cr blocks for the one or more PUs of a CU, video encoder 20 can generate a luma residual block for the CU by subtracting the prediction luma blocks for the CU from the original luma coded block of the CU, such that each sample in the luma residual block for the CU indicates a difference between a luma sample in one of the prediction luma blocks for the CU and a corresponding sample in the original luma coded block of the CU. Similarly, video encoder 20 can generate Cb and Cr residual blocks for the CU, such that each sample in the Cb residual block for the CU indicates a difference between a Cb sample in one of the prediction Cb blocks for the CU and a corresponding sample in the original Cb coded block of the CU, and each sample in the Cr residual block for the CU can indicate a difference between a Cr sample in one of the prediction Cr blocks for the CU and a corresponding sample in the original Cr coded block of the CU.

[0070] Furthermore, as shown in Figure 4C Video encoder 20 can use quad-tree partitioning to split the luma residual block, the Cb residual block, and the Cr residual block of a CU into one or more luma transform blocks, Cb transform blocks, and Cr transform blocks, as shown in

[0071] Video encoder 20 can apply one or more transforms to a luma transform block of a TU to generate a luma coefficient block for the TU. A coefficient block can be a two-dimensional array of transform coefficients. A transform coefficient can be a scalar. Video encoder 20 can apply one or more transforms to a Cb transform block of a TU to generate a Cb coefficient block for the TU. Video encoder 20 can apply one or more transforms to a Cr transform block of a TU to generate a Cr coefficient block for the TU.

[0072] After generating a coefficient block (e.g., a luma coefficient block, a Cb coefficient block, or a Cr coefficient block), video encoder 20 can quantize the coefficient block. Quantization generally refers to a process in which transform coefficients are quantized to possibly reduce the amount of data used to represent the transform coefficients, providing further compression. After video encoder 20 quantizes a coefficient block, video encoder 20 can entropy encode syntax elements indicating the quantized transform coefficients. For example, video encoder 20 can perform Context- Adaptive Binary Arithmetic Coding (CABAC) on the syntax elements indicating the quantized transform coefficients. Finally, video encoder 20 can output a bitstream that includes a sequence of bits, which forms a representation of the coded frame and associated data, the bitstream being maintained in storage device 32 or sent to destination device 14.

[0073] After receiving the bitstream generated by video encoder 20, video decoder 30 can parse the bitstream to obtain syntax elements from the bitstream. Video decoder 30 can reconstruct the frames of the video data based at least in part on the syntax elements obtained from the bitstream. The process of reconstructing the video data is generally reciprocal to the encoding process performed by video encoder 20. For example, video decoder 30 can perform inverse transforms on the coefficient blocks associated with the TUs of a current CU to reconstruct residual blocks associated with the TUs of the current CU. Video decoder 30 also reconstructs the coded block of the current CU by adding the samples of the prediction block for the PUs of the current CU to corresponding samples of the transform blocks of the TUs of the current CU. After reconstructing the coded blocks for each CU of a frame, video decoder 30 can reconstruct the frame.

[0074] As noted above, video coding primarily uses two modes, namely, intra- frame prediction (or intra-prediction) and inter-frame prediction (or inter-prediction) to achieve video compression. It should be noted that IBC can be considered as a third mode of intra-prediction. Between the two modes, inter-prediction contributes more to coding efficiency than intra-prediction because motion vectors are used to predict the current video block from reference video blocks.

[0075] Figure 5Ais a block diagram illustrating 67 regular intra prediction modes for predicting a current coding block based on reconstructed neighboring blocks, according to some embodiments of the present disclosure. The 67 regular intra prediction modes include 65 angular modes (shown as mode indices 2 to 33 are "horizontal mode set", and mode indices 34 to 66 are "vertical mode set") plus two non-angular modes, referred to as "Planar mode" (mode index 0) and "DC mode" (mode index 1), collectively as "non-angular mode set". Embodiments of the present disclosure are applied to any number of angular modes for intra prediction. For example, the number of modes can be 35 as used in HEVC or some other number of modes greater than 35. Although embodiments can be applied to intra prediction mode coding for only a few selected color components, such as the luma component or the chroma components, it will be apparent to those of ordinary skill in the art that they can be applied to all available color components (luma and two chroma) or any other combination.

[0076] According to some embodiments of the present disclosure, a video coder, such as video encoder 20 or video decoder 30, can examine three or more neighboring blocks in a set of neighboring blocks to identify intra prediction modes, generating a most probable mode (MPM) candidate list for a current block. If a neighboring block is coded using an intra prediction mode, the video coder can add the intra prediction mode used to code the neighboring block to the MPM candidate list for the current block. The positions of the neighboring blocks examined by the video coder 20 can be fixed relative to the current block.

[0077] Figure 5B is a block diagram illustrating exemplary positions of five reconstructed neighboring blocks of a current coding block, according to some embodiments of the present disclosure. For example, the positions of the five neighboring blocks can include a left (L) block, an above (A) block, a below-left (BL) block, an above-right (AR) block, and / or an above-left (AL) block. Other positions of neighboring blocks can also be used. The order in which intra prediction modes from the neighboring blocks are added to the MPM candidate list can depend on many factors, such as the current block size, whether the block has a certain shape (such as rectangular or square), or based on context information (such as the size or shape of the neighboring blocks and the type or frequency of the intra prediction modes of the neighboring blocks). Note that, Figure 5B The five neighboring positions in are provided as examples, but fewer or more neighboring blocks can be considered when constructing the MPM candidate list using embodiments.

[0078] In general, a video codec can generate a MPM candidate list from different MPM types. The different types include, but are not limited to, neighbor-based intra prediction modes, derived intra prediction modes, and default intra prediction modes. A neighbor-based intra prediction mode refers to an intra prediction mode used by a neighboring block. A default intra prediction mode refers to a constant intra prediction mode that does not change with neighboring blocks. The default intra prediction mode(s) can be one of a planar mode, a DC mode, a horizontal mode, or a vertical mode. A derived intra prediction mode refers to an intra prediction mode derived from a neighbor-based intra prediction mode or a default intra prediction mode. The derived intra prediction mode can not be the actual intra prediction mode of a neighboring block. It can be an intra prediction mode derived from or in some other way derived from the actual intra prediction mode of a neighboring block. For example, the derived intra prediction mode can be the neighbor-based intra prediction mode ± 1, ± 2, etc. The derived intra prediction mode can also be generated by another already existing derived intra prediction mode.

[0079] A video codec can add intra prediction modes to a MPM candidate list according to the intra prediction mode types. For example, a video codec can first add neighbor-based intra prediction modes, then add derived intra prediction modes when the number of neighbor-based intra prediction modes is less than N, and then add default intra prediction modes when the total number of neighbor-based and derived intra prediction modes is still less than N. In another implementation, a video codec can add different types of intra prediction modes in an interleaved manner. For example, a video codec can add one or more default intra prediction modes after adding a certain number of neighbor-based intra prediction modes to the list. Alternatively, a video codec can add two neighbor-based intra prediction modes, two default intra prediction modes, and then add more neighbor-based intra prediction modes.

[0080] Only unique neighbor-based intra prediction modes will be added to the MPM candidate list. For example, if one of the neighboring blocks has the same intra prediction mode that has already been added to the MPM candidate list, such a mode is not added to the list again. The location of the neighboring blocks can be represented by a sub-block size (e.g., 4x4), meaning it is the granularity at which the intra prediction mode information is stored. In another example, the intra prediction mode information can be specified per pixel or for larger blocks such as 8x8. If the chroma components are subsampled compared to the luma components (such as in a 4:2:0 color format), the chroma component sub-block locations can be smaller, e.g., 2x2, which can correspond to luma 4x4.

[0081] In some implementations, depending on the neighboring block size, Figure 5BThe plurality of positions of the neighboring blocks depicted in the center can belong to the same CU. For example, if the neighboring block is 16x16 and the currently coded block is 8x8, the top-left position and the left position can belong to the same 16x16 neighboring block, where the intra prediction mode information would be the same for those positions.

[0082] The number of neighboring positions M can be equal to the MPM candidate list size N, but can be smaller or larger. In one example, the number M can be smaller than N to allocate some space to include other types of intra prediction modes (e.g., derived intra prediction modes or default intra prediction modes) into the MPM candidate list. The number of positions can depend on characteristics of the current block and / or the neighboring blocks, such as the block size, whether the block is square or rectangular, whether the rectangular block is a horizontally-oriented rectangular block (width is greater than height) or a vertically-oriented rectangular block (width is smaller than height), the ratio between the height and the width, and the ratio between the larger value and the smaller value among the height and the width. The number of positions can also depend on the prediction mode (e.g., intra or inter) of the neighboring blocks.

[0083] When a block is coded using an intra prediction mode, a video encoder (e.g., video encoder 20) can generate an MPM candidate list for the block. A video decoder (e.g., video decoder 30) can generate the same MPM candidate list as determined by the video encoder by implementing the same MPM candidate list generation process implemented by the video encoder. Since the video encoder and the video decoder generate the same MPM candidate list, the video encoder can signal the intra prediction mode to the video decoder by signaling an index value corresponding to a particular candidate in the MPM candidate list. Unless explicitly stated to the contrary, the MPM candidate list generation implementations described herein can be performed by either the video encoder or the video decoder.

[0084] In addition to the aforementioned regular intra prediction modes, a number of MBIP modes have been proposed for performing intra prediction for a current block from neighboring blocks by applying a linear matrix transform to the reconstructed pixels of the neighboring blocks. Figure 6A and Figure 6B are block diagrams illustrating two matrix-based intra prediction schemes for different sizes of coded blocks according to embodiments of the present disclosure.

[0085] First, the reconstructed pixels from the above and left neighboring blocks are filtered and optionally down-sampled. For example, for a 16x16 coded block, Figure 6B For the 8x8 coded block depicted in the center, the 8 pixels from the above and left neighboring blocks are down-sampled by a factor of 2 to derive 4 down-sampled pixels from each side. Likewise, for a 16x16 coded block, Figure 6AThe 4x4 coded block depicted in

[0086] Second, the down-sampled pixels are re-arranged into a one-dimensional vector, a matrix multiplication is performed on the one-dimensional vector with a pre-defined matrix, and then a bias offset vector is added. After the matrix multiplication and bias offset, the result is re-arranged back into a two-dimensional array to form a matrix-transformed two-dimensional result. Depending on the current block size, the matrix-transformed two-dimensional result can be in a sub-sampled domain. For example, if the matrix-transformed two-dimensional result has a size of 4x4, it is considered to be in the sub-sampled domain if the current block size is 8x8 (see, e.g., Figure 6B ), and it is considered not to be in the sub-sampled domain if the current block size is 4x4 (see, e.g., Figure 6A ).

[0087] Finally, for the scenario depicted in Figure 6B , the matrix-transformed two-dimensional result is up-sampled / interpolated to form the prediction values for the current coded block. But for the scenario depicted in Figure 6A , the matrix-transformed two-dimensional result already has a size of 4x4, and no up-sampling / interpolation is needed to form the prediction values for the current coded block. Notably, the examples provided in Figure 6A and Figure 6B are illustrative implementations.

[0088] While MBIP overcomes some of the problems associated with conventional intra prediction modes, it still introduces some new challenges to the design and implementation of new codec standards. For example, the 67 MBIP modes require 67 coefficient matrices and 67 bias offset vectors. Assuming each matrix / vector coefficient is a 10-bit precision number, it would take almost 8 Kbytes of memory space to store these values. This can increase the physical size and power usage of a chip-based implementation of MBIP.

[0089] In some implementations, instead of using 10-bit precision for each coefficient in the MBIP matrices and bias offset vectors, a lower precision is used to save the memory space needed to store these coefficients. For example, 9-bit or even 8-bit precision is used for each coefficient in the MBIP matrices and bias offset vectors to save at least 10% of the memory space.

[0090] In some implementations, different precisions can be used for the coefficients in the MBIP matrix and the bias offset vector, respectively, because the coding efficiency from MBIP is less sensitive to the precision of the bias offset vector than to the precision of the MBIP matrix. For example, 10-bit precision can still be used for the coefficients in the MBIP matrix, while a lower precision (e.g., 9-bit or lower) is used for the coefficients in the bias offset vector. Moreover, it is possible to skip the bias offset vector altogether. In other words, if the quality of the reconstructed pixels is satisfactory, MBIP only performs matrix multiplication on the pixels of the neighboring blocks and skips the addition of the bias vector to save more memory space.

[0091] In yet other implementations, only the MBIP matrix and bias vector corresponding to the smallest block size are stored in the memory, while other MBIP matrices and bias offset vectors corresponding to larger block sizes are upsampled and / or interpolated from those MBIP matrices and bias offset vectors of the smallest size. For example, only the matrix and bias offset vector defined for 4x4 blocks are saved. When the MBIP mode is applied to 8x8 or 16x16 blocks, the MBIP matrix and bias offset defined for 4x4 blocks are upsampled and / or interpolated (e.g., bilinear interpolation) to the size defined for 8x8 or 16x16 blocks, respectively, and then used to predict the current block.

[0092] As noted above in connection with Figure 6A and Figure 6B applying filtering and downsampling operations to the reconstructed pixels from the neighboring blocks requires additional logic and thus physical space for the codec implementation.

[0093] In some implementations, a 3-tap in-loop smoothing filter is used in the filtering and / or downsampling process in the MBIP mode, which is applied to the reconstructed pixels of the neighboring blocks in the regular intra prediction mode. In yet other implementations, no filtering operation is performed on the reconstructed pixels of the neighboring blocks before downsampling. In other words, the downsampling is directly performed on the reconstructed pixels from the neighboring blocks without any filtering operation performed in advance.

[0094] The coding efficiency of intra modes depends not only on the number of angular coding modes, but also on the range of reconstructed samples that can be accessed. In some implementations, reconstructed pixels from the above-right (AR) block, the above-left (AL) block, and the below-left (BL) block are also used in the filtering and down-sampling process in the MBIP mode for better coding efficiency. In addition, when the current block is coded using the MBIP mode, different number of reconstructed pixels from the neighboring blocks can be used from the above neighboring block (A) and from the left neighboring block (L), respectively. For example, if the aspect ratio of the current block is non-square (e.g., vertical rectangular), more reconstructed pixels from the left neighboring block (L) along the longer side of the current block can be used in the MBIP mode than from the shorter side of the current block. In some cases, when the aspect ratio of the current block exceeds a certain threshold (e.g., 3 to 1), only reconstructed pixels from the neighboring block along the longer side of the current block are used in the MBIP mode, and no reconstructed pixels from the neighboring block along the shorter side of the current block are used at all.

[0095] Matrix multiplication is a computationally expensive operation. There are certain situations where the MBIP mode is disabled because the gain associated with MBIP is not justified when compared to the complexity associated with MBIP, e.g.,

[0096] • if the width of the current block is four times the height of the current block;

[0097] • if the height of the current coding block is four times the width of the current block;

[0098] • if the width or height of the current block is greater than 64; and

[0099] • if the current block is a chroma block.

[0100] When this happens, the current block that meets one of the above situations can be predicted using a regular intra prediction mode, and the MPM candidate list is generated for the current block accordingly as described above. If the current block is predicted using the MBIP mode Figure 5B one of the five neighboring blocks of the current block depicted in FIG. 6, such MBIP mode needs to be converted to the corresponding regular intra prediction mode so as to be considered for predicting the current block.

[0101] Figure 7 is a flowchart illustrating an exemplary process by which a video codec implementation generates a most probable mode (MPM) candidate list in accordance with some embodiments of the present disclosure. As described above, this video codec can be the video encoder 20 or the video decoder 30. For illustrative purposes, the present disclosure will use the video encoder 20 as an example in the following.

[0102] Based on this, it is assumed that video encoder 20 will process the current block of video data to be encoded. It is determined that the current block will use a conventional intra-frame prediction mode (e.g., ...). Figure 5A After one of the 67 patterns described in the document is predicted, the video encoder 20 needs to pass an inspection. Figure 5B The video encoder 20 first identifies (710) neighboring blocks located at predefined positions relative to the current block and their associated matrix-based intra-prediction modes. Assuming the left neighboring block L is identified, the video encoder 20 determines that the left neighboring block L is reconstructed according to one of the MBIP modes. However, since the MBIP mode is disabled for the current block, the video encoder 20 needs to determine which regular intra-prediction mode will be associated with the left neighboring block L when generating the MPM candidate list.

[0103] In some implementations, the video encoder 20 determines (730) a regular intra-prediction mode corresponding to the MBIP mode for the left neighboring block L based on a predefined mathematical relationship between regular intra-prediction modes and matrix-based intra-prediction modes. For example, the video encoder 20 may assign a constant value (e.g., planar mode 0) to (730-1) the left neighboring block L, regardless of the actual MBIP mode used to predict the left neighboring block L. In other words, when the video encoder 20 considers neighboring blocks for updating the MPM candidate list of the current block, a constant value corresponding to one of the regular intra-prediction modes is used to represent the neighboring block.

[0104] In some implementations, MBIP modes are designed such that specific MBIP modes target block content that favors DC intra-prediction, planar intra-prediction, and / or directional intra-prediction modes, respectively. For example, an MBIP mode (e.g., mode 0) is selected to target block content that favors planar prediction, an MBIP mode (e.g., mode 1) is selected to target block content that favors DC prediction, and multiple MBIP modes (with values ​​greater than 1) are selected to target block content that favors angular prediction with different directions. As a result, the mapping between MBIP modes and regular intra-prediction modes becomes direct.

[0105] For example, suppose the MBIP intra-frame prediction mode is defined as Mode MBIP Furthermore, the standard intra-frame prediction mode is defined as Mode INTRA (Based on VVC, its range is from 0 to 66), when the total number of MBIP modes is also 67, there is the following one-to-one mapping between the regular intra-frame prediction mode and the MBIP mode:

[0106] • Map the regular intra-frame prediction mode to the MBIP mode;

[0107] • Set Mode MBIP = Mode INTRA

[0108] • Map regular intra prediction modes to MBIP modes:

[0109] • Set Mode INTRA = Mode MBIP

[0110] In other words, when the total number of regular intra prediction modes is the same as the total number of matrix-based intra prediction modes, the predefined mathematical relationship between the regular intra prediction modes and the matrix-based intra prediction modes is defined as the regular intra prediction modes having the same values as the values of the matrix-based intra prediction modes (730-3).

[0111] When the total number of MBIP modes is 35, the following mapping logic is used for mode conversion:

[0112] • Map regular intra prediction modes to MBIP modes:

[0113] • If Mode INTRA has a value less than 2, set Mode MBIP = Mode INTRA ;

[0114] • Otherwise, set Mode MBIP = (Mode INTRA - 34) / 2 + 18.

[0115] • Map MBIP modes to regular intra prediction modes:

[0116] • If Mode MBIP has a value less than 2, set Mode INTRA = Mode MBIP ;

[0117] • Otherwise, set Mode INTRA = (Mode MBIP - 18) x 2 + 34.

[0118] Similarly, when the total number of MBIP modes is 19, the following mapping logic is used for mode conversion.

[0119] • Map regular intra prediction modes to MBIP modes:

[0120] • If Mode INTRA has a value less than 2, set Mode MBIP = Mode INTRA ;

[0121] • Otherwise, set Mode MBIP = (Mode INTRA - 34) / 4 + 10.

[0122] • Map the MBIP mode to the regular intra prediction mode:

[0123] • If Mode MBIP has a value less than 2, set Mode INTRA = Mode MBIP ;

[0124] • Otherwise, set Mode INTRA = (Mode MBIP - 10) x 4 + 34.

[0125] When the total number of MBIP modes is 11, the following mapping logic is used for mode conversion:

[0126] • Map the regular intra prediction mode to the MBIP mode:

[0127] • If Mode INTRA has a value less than 2, set Mode MBIP = Mode INTRA ;

[0128] • Otherwise, set Mode MBIP = (Mode INTRA - 34) / 8 + 6.

[0129] • Map the MBIP mode to the regular intra prediction mode:

[0130] • If Mode MBIP has a value less than 2, set Mode INTRA = Mode MBIP ;

[0131] • Otherwise, set Mode INTRA = (Mode MBIP - 6) x 8 + 34.

[0132] In other words, when the total number of regular intra prediction modes is greater than the total number of matrix-based intra prediction modes, the predefined mathematical relationship between the regular intra prediction modes and the matrix-based intra prediction modes is defined as two categories: (i) non-angular intra prediction modes (less than mode 2) and (ii) angular intra prediction modes (equal to or greater than mode 2) (730-5). It should be noted, however, that in all these cases, when the MBIP mode is enabled together with the regular intra prediction mode, there is no need to store any mapping table for MPM list generation.

[0133] After determining the regular intra prediction modes, video encoder 20 inserts (750) the regular intra prediction modes associated with neighboring blocks into the most probable mode candidate list according to the predefined order as described above in connection with Figure 5B

[0134] In some implementations, position dependent intra prediction combination (PDPC) is applied to the prediction samples formed by regular intra prediction to improve the intra prediction coding efficiency. The coding gain of PDPC comes from the improved prediction value quality resulting from the combination of the intra prediction samples and the reconstructed pixels from neighboring blocks. PDPC can be performed on top of MBIP mode. In other words, the PDPC operation is performed on the prediction values of the current block formed by MBIP. Of course, such operation increases the computational complexity of the encoding / decoding operation and should be used at selective locations where the benefit outweighs the cost.

[0135] In one or more examples, the functions described can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions can be stored on or transmitted over as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media can include computer-readable storage media, which corresponds to a tangible medium such as data storage media, or communication media including any medium that facilitates transfer of a computer program from one place to another, e.g., according to a communication protocol. In this manner, computer- readable media generally can correspond to (1) tangible computer-readable storage media which is non-transitory or (2) a communication medium such as a signal or carrier wave. Data storage media can be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code and / or data structures for implementation of the techniques described in this application. A computer program product can include a computer-readable medium.

[0136] The terminology used in the description of the implementations herein is for the purpose of describing particular implementations only and is not intended to limit the scope of the claims. As used in the description of the implementations and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term “and / or” as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms “comprises” and / or “comprising,” when used in this specification, specify the presence of stated features, elements, and / or components, but do not preclude the presence or addition of one or more other features, elements, components, and / or groups thereof.

[0137] ​It will also be understood that, although the terms first, second, etc. can be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first electrode could be termed a second electrode, and, similarly, a second electrode could be termed a first electrode, without departing from the scope of the present embodiments. The first electrode and the second electrode are both electrodes, but they are not the same electrode.

[0138] The description of the present application has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the application in the form disclosed. Many modifications, variations, and alternatives of the present application will be apparent to those of ordinary skill in the art, once the above teaching has been presented. The embodiments were chosen and described in order to best explain the principles of the application and the practical application, and to enable others skilled in the art to understand the application for various embodiments with various modifications as are suited to the particular use contemplated. Therefore, it is intended that the scope of the claims be limited by the scope of the embodiments disclosed, and that modifications and other embodiments are intended to be included within the scope of the appended claims.

Claims

1.A method for video coding, comprising: dividing a video frame into a plurality of video blocks; identifying a neighboring block located at a predefined position relative to a current block in the plurality of video blocks and its associated matrix-based intra prediction mode; determining a regular intra prediction mode corresponding to the matrix-based intra prediction mode of the neighboring block according to a predefined mathematical relationship between a plurality of matrix-based intra prediction modes and corresponding regular intra prediction modes; deriving a candidate intra prediction mode based on the determined regular intra prediction mode; and inserting the derived candidate intra prediction mode into a mode candidate list according to a predefined order, wherein the predefined mathematical relationship is defined as that for each matrix-based intra prediction mode, the corresponding regular intra prediction mode is constant, wherein the corresponding regular intra prediction mode is a planar mode. 2.The method of claim 1, wherein identifying the neighboring block comprises: identifying a plurality of neighboring blocks including the neighboring block. 3.The method of claim 2, wherein a number of the plurality of neighboring blocks is less than a size of the mode candidate list. 4.The method of claim 2, wherein the plurality of neighboring blocks includes a left block and an above block. 5.The method of claim 1, wherein each matrix-based intra prediction mode has an associated matrix and a bias vector. 6.The method of claim 1, further comprising: signaling an index value corresponding to a particular candidate in the mode candidate list. 7.An electronic device comprising: one or more processing units; memory coupled to the one or more processing units; and a plurality of programs stored in the memory that, when executed by the one or more processing units, cause the electronic device to perform the method for video coding of any of claims 1-6. 8.A non-transitory computer-readable storage medium storing a plurality of programs for execution by an electronic device having one or more processing units, wherein the plurality of programs, when processed by the one or more processing units, cause the electronic device to perform the method for video coding of any of claims 1-6 and store a bitstream generated according to the method for video coding of any of claims 1-6. A bitstream generated by the method for video coding of any of claims 1-6 is stored in a non-transitory computer-readable storage medium. ​ 9. A method of storing a bitstream, comprising: ​

Images