Method, apparatus and computer program product for video encoding and video decoding

Abstract

To provide an intra-prediction method with weights and a method and an apparatus for implementing the method.SOLUTION: The method comprises: receiving a picture to be encoded by an encoder; performing at least one prediction according to a first prediction mode for samples inside a block of the picture in a current channel; deriving an intra prediction mode from at least one coded block in a reference channel; performing at least one other prediction according to the derived intra prediction mode for the samples inside the block of the picture; and determining a final prediction of the block based on the at least one first and at least one second predictions with weights.SELECTED DRAWING: Figure 9

Description

[Technical field]

[0001] The present solution generally relates to video encoding and decoding. [Background technology]

[0002] This section is intended to provide a background or context for the claimed invention. The description herein may include ideas that could be pursued, but not necessarily ideas that have been previously conceived or pursued. Thus, except as indicated herein, the subject matter described in this section is not prior art to the description and claims of this application, and is not admitted to be prior art by virtue of its inclusion in this section.

[0003] A video coding system may include an encoder that converts input video into a compressed representation suitable for storage / transmission, and a decoder that can uncompress the compressed video representation back into a viewable form. The encoder may discard some information in the original video sequence in order to represent the video in a more compact form, for example allowing the video information to be stored / transmitted at a lower bit rate than would otherwise be required. Summary of the Invention

[0004] The scope of protection sought for the various embodiments of the invention is set forth in the independent claims. To the extent that embodiments and features are described herein but are not included in the scope of the independent claims, these embodiments and features should be interpreted as examples useful for understanding the various embodiments of the invention.

[0005] Various aspects include a method, an apparatus and a computer readable medium having a computer program stored therein, characterized in what is set forth in the independent claims. Various embodiments are disclosed in the dependent claims.

[0006] According to a first aspect there is provided a method comprising: - receiving a picture to encode; - performing at least one prediction on inner samples of a block of a picture of the current channel according to a first prediction mode; - deriving an intra prediction mode from at least one coded block of a reference channel; - performing at least one other prediction on samples inside the block of the picture according to the derived intra prediction mode; and - determining a final prediction for a block based on the weighted at least one first prediction and the at least one second prediction. Includes.

[0007] According to a second aspect there is provided an apparatus comprising at least one processor and a memory containing computer program code, the memory and the computer program code operating together with the at least one processor to perform at least: - receiving a picture to encode; - performing at least one prediction on inner samples of a block of a picture of the current channel according to a first prediction mode; - deriving an intra prediction mode from at least one coded block of a reference channel; - performing at least one other prediction on samples inside the block of the picture according to the derived intra prediction mode; and - determining a final prediction for the block based on the weighted at least one first prediction and the at least one second prediction. The apparatus is configured to execute the following:

[0008] According to a third aspect there is provided an apparatus comprising: - means for receiving a picture to be encoded; - means for performing at least one prediction according to a first prediction mode on inner samples of a block of a picture of the current channel; - means for deriving an intra prediction mode from at least one coded block of a reference channel; - means for performing at least one other prediction on samples inside the block of the picture according to the derived intra prediction mode; - means for determining a final prediction for a block based on the weighted at least one first prediction and the at least one second prediction; Equipped with.

[0009] According to a fourth aspect there is provided a computer program product comprising computer program code which, when executed on at least one processor, - receiving a picture to encode; - performing at least one prediction on inner samples of a block of a picture of the current channel according to a first prediction mode; - deriving an intra prediction mode from at least one coded block of a reference channel; - performing at least one other prediction on samples inside the block of the picture according to the derived intra prediction mode; and - determining a final prediction for a block based on the weighted at least one first prediction and the at least one second prediction. The device or system is configured to execute the

[0010] According to one embodiment, the first prediction mode is performed in a cross-component linear mode.

[0011] According to one embodiment, the derived intra prediction mode is derived from at least one collocated block of a channel different from the current channel.

[0012] According to one embodiment, the derived intra prediction mode is derived from at least one neighboring block of the current channel.

[0013] According to one embodiment, the derived intra prediction mode is determined based on texture analysis from reconstructed neighboring samples of the current channel.

[0014] According to one embodiment, the texture analysis method is one of a decoder-side intra mode derivation method, a template matching-based method, and an intra block copy method.

[0015] According to one embodiment, the decision from the neighboring samples takes into account the direction of the first prediction.

[0016] According to one embodiment, the final prediction comprises the combined first and second predictions with constant equal weights for all samples of the block.

[0017] According to one embodiment, the final prediction comprises the combined first and second predictions with constant unequal weights for all samples of the block.

[0018] According to one embodiment, the final prediction comprises the first and second predictions combined, with equal or unequal per-sample weightings, where the weights of each predicted sample are different from each other.

[0019] According to one embodiment, the weighting value of a sample is determined based on a prediction direction or a mode identifier of a derived intra-prediction mode.

[0020] According to one embodiment, the weight value of a sample is determined based on the prediction direction of the cross-component linear mode, the position of the reference sample, or a mode identifier.

[0021] According to one embodiment, the weighting value of a sample is determined based on the cross-component linear prediction mode and the prediction direction of the derived prediction mode, the position of the reference sample or a mode identifier.

[0022] According to one embodiment, the weighting values ​​of the samples are determined based on the size of the block.

[0023] According to one embodiment, a computer program product is embodied on a non-transitory computer readable medium.

[0024] Various embodiments will now be described in more detail with reference to the accompanying drawings. [Brief description of the drawings]

[0025] [Figure 1] FIG. 2 illustrates an example of an encoding process. [Diagram 2] FIG. 1 illustrates an example of a decoding process. [Diagram 3] FIG. 13 is a diagram showing an example of sample positions of a current block. [Figure 4] FIG. 13 is a diagram showing an example of four reference lines adjacent to a prediction block. [Diagram 5] FIG. 2 illustrates an example of a matrix weighting intra prediction process. [Figure 6]FIG. 2 illustrates a coded block of a chroma channel and a co-located block of a luma channel. [Figure 7] FIG. 2 shows a coding block of a chroma channel and a nearby block of a collocated block of a luma channel. [Figure 8] FIG. 1 illustrates the mixing / combining process of the joint prediction method. [Figure 9] 1 is a flow diagram illustrating a method according to one embodiment. [Figure 10] FIG. 1 illustrates an apparatus according to one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0026] In the following, some embodiments are described in the context of one video coding configuration, although it should be noted that embodiments of the present invention are not necessarily limited to this particular configuration.

[0027] The Advanced Video Coding standard (sometimes abbreviated as AVC or H.264 / AVC) is a standard developed by the International Telecommunication Union Telecommunication Standardization Sector (ITU-T) Video Coding Experts Group (VCEG) and the International Organization for Standardization (ISO) / International Electrotechnical Commission (IEC) Moving Picture Experts Group (MPEG) Joint Video Team (JVT). The H.264 / AVC standard is published by both parent standards organizations and is referred to as ITU-T Recommendation H.264 and ISO / IEC International Standard 14496-10, also known as MPEG-4 Part 10 Advanced Video Coding (AVC). There are numerous editions of the H.264 / AVC standard, each of which incorporates new extensions or features to the specification. These extensions include Scalable Video Coding (SVC) and Multiview Video Coding (MVC).

[0028] The High Efficiency Video Coding standard (sometimes abbreviated HEVC or H.265 / HEVC) is a standard developed by the VCEG and MPEG's Joint Collaborative Team-Video Coding (JCT-VC). The standard is published by both parent standards organizations and is referred to as ITU-T Recommendation H.265 and ISO / IEC International Standard 23008-2, also known as MPEG-H Part 2 High Efficiency Video Coding (HEVC). Extensions to H.265 / HEVC include scalable, multi-view, three-dimensional and range of fidelity extensions, which are sometimes referred to as SHVC, MV-HEVC, 3D-HEVC and REXT, respectively. Unless otherwise indicated, references in this description to H.265 / HEVC, SHVC, MV-HEVC, 3DHEVC and REXT made for the purposes of understanding the definitions, structure or concepts of those standards specifications should be understood to be references to the most recent versions of those standards available prior to the filing date of this application.

[0029] The Versatile Video Coding standard (VVC, H.266 or H.266 / VVC) is a standard currently under development by the Joint Video Experts Team (JVET), a collaboration between ISO / IEC MPEG and ITU-T VCEG.

[0030] In this section, some key definitions, bitstreams and encoding structures, and concepts of H.264 / AVC and HEVC and some of their extension standards are described as examples of video encoders, decoders, encoding methods, decoding methods, and bitstream structures that can implement embodiments of the present invention. Some of the key definitions, bitstreams and encoding structures, and concepts of H.264 / AVC are the same as those of the HEVC standard, and therefore they are described together below. Although aspects of various embodiments are not limited to only H.264 / AVC or HEVC or their extensions, the description is provided as one possible basis for partially or completely realizing embodiments of the present invention.

[0031] A video codec may include an encoder that converts the input video into a compressed representation suitable for storage / transmission, and a decoder that can decompress the compressed video representation back into a viewable form. This compressed representation is sometimes called a bitstream or video bitstream. Furthermore, the video encoder and / or video decoder may be separate from each other; that is, they do not necessarily form a codec. The encoder may discard some information in the original video sequence in order to represent the video in a more compact form (i.e., at a lower bitrate).

[0032] An example of the encoding process is shown in Figure 1. Figure 1 shows the image to be encoded (I n ), the predicted representation of the image block (P' n ), prediction error signal (D n ), the reconstructed prediction error signal (D' n ), the reconstructed preliminary image (I' n ), the final reconstructed image (R' n ), transformation (T) and inverse transformation (T -1 ), quantization (Q) and dequantization (Q -1 ), Entropy coding (E), Reference Frame Memory (RFM), Inter prediction (P inter ), intra prediction (P intra), mode selection (MS) and filtering (F). An example of the decoding process is shown in Figure 2. Figure 2 shows a predicted representation of an image block (P' n ), the reconstructed prediction error signal (D' n ), the reconstructed preliminary image (I' n ), the final reconstructed image (R' n ), inverse transformation (T -1 ), inverse quantization (Q -1 ), Entropy Decoding (E -1 ), Reference Frame Memory (RFM), Prediction (Inter or Intra) (P), and Filtering (F).

[0033] Hybrid video codecs, such as ITU-T H.263, H.264 / AVC and HEVC, can code video information in two stages. First, pixel values ​​of a certain picture area (or "block") are predicted, for example by motion compensation means (finding and indicating an area of ​​one of the previously coded video frames that closely corresponds to the block being coded) or by spatial means (using pixel values ​​surrounding the block to be coded in a specified manner). In this first stage, predictive coding can be applied, for example as so-called sample prediction and / or so-called syntax prediction.

[0034] Sample prediction involves predicting pixel or sample values ​​for a picture area or "block." These pixel or sample values ​​may be predicted using, for example, one or more of motion compensation or intra prediction mechanisms.

[0035] A motion compensation mechanism (sometimes called inter prediction, temporal prediction or motion-compensated temporal prediction, or motion-compensated prediction, or MCP) involves finding and pointing to an area of ​​one of the previously coded video frames that closely corresponds to a block being coded. One advantage of inter prediction is that it can reduce temporal redundancy.

[0036] Intra prediction can predict pixel or sample values ​​by spatial mechanisms. Intra prediction involves finding and representing spatial domain relationships, taking advantage of the fact that adjacent pixels in the same picture are likely to be correlated. Intra prediction can be performed in the spatial or transform domain, i.e., predicting sample values ​​or transform coefficients. Intra prediction can be used in intra coding, where inter prediction is not applied.

[0037] Syntax prediction, sometimes referred to as parameter prediction, predicts syntax elements and / or syntax element values ​​and / or variables derived from syntax elements from previously (decoded) encoded syntax elements and / or previously derived variables. Non-limiting examples of syntax prediction are provided below.

[0038] In motion vector prediction, motion vectors, e.g., motion vectors for inter prediction and / or inter view prediction, can be differentially coded with respect to a predicted motion vector of a particular block. In many video codecs, the predicted motion vector is generated in a predefined manner, e.g., by calculating the median of the coded or decoded motion vectors of neighboring blocks. Another manner of generating motion vector prediction, sometimes called advanced motion vector prediction (AMVP), is to create a list of candidate predictions from neighboring and / or co-located blocks of a temporal reference picture, and signal a selected candidate as a motion vector predictor. In addition to predicting motion vector values, reference indices of previously coded / decoded pictures can be predicted. The reference indices are usually predicted from neighboring and / or co-located blocks of a temporal reference picture. Differential coding of motion vectors is usually disabled across slice boundaries.

[0039] Block partitioning, for example, from coding tree units (CTUs) to coding units (CUs) and then to prediction units (PUs), can be predicted. Partitioning is the process of dividing a set into subsets in such a way that each element of the set can be a subset. A picture can be partitioned into CTUs of maximum size 128x128, but an encoder can choose to use a smaller size, such as 64x64. First, the coding tree units (CTUs) can be partitioned by a quaternary tree (known as a quadtree) structure. The leaf nodes of the quaternary tree can then be further partitioned by a multi-type tree structure. There are four splitting types in the multitype tree structure: vertical binary splitting, horizontal binary splitting, vertical ternary splitting, and horizontal ternary splitting. The leaf nodes of the multitype tree are called coding units (CUs). CUs, PUs, and TUs (transform units) have the same block size, except when a CU is too large for the maximum transform length. The segmentation structure of the CTU is a quadtree with nested multitype trees using binary and ternary splitting, i.e., the separate CU, PU, ​​and TU concepts are not used, except when necessary for CUs with sizes too large for the maximum transform length. CUs can have square or rectangular shapes.

[0040] Filter parameter prediction allows predicting filtering parameters, for example filtering parameters for sample adaptive offset.

[0041] Prediction techniques that use picture information from previously coded pictures are sometimes called inter-prediction techniques, and these techniques are also called temporal prediction and motion compensation. Prediction techniques that use picture information within the same picture are sometimes called intra-prediction techniques.

[0042] The second stage encodes the prediction error, i.e. the difference between the predicted pixel block and the original pixel block. This can be done by transforming the pixel value differences using a specified transform (e.g. the discrete cosine transform (DCT) or a variant thereof), quantizing the coefficients, and entropy coding the quantized coefficients. By varying the fidelity of the quantization process, the encoder can control the balance between the accuracy of the pixel representation (picture quality) and the size of the resulting encoded video representation (file size at transmission bit rate).

[0043] In many video codecs, including H.264 / AVC and HEVC, the motion information is indicated by a motion vector associated with each motion-compensated image block. Each of these motion vectors represents the displacement of an image block of a picture to be coded (at an encoder) or decoded (at a decoder) and a prediction source block of one of the previously coded or decoded images (or pictures). In H.264 / AVC and HEVC, as in many other video compression standards, a picture is divided into a number of rectangular meshes, for each of which a similar block of one of the reference pictures is indicated for inter prediction. The position of the predicted block is coded as a motion vector that indicates the position of the predicted block relative to the block being coded.

[0044] Video coding standards may specify bitstream syntax and semantics, as well as a decoding process for error-free bitstreams, but may not specify the encoding process, and encoders may only be required to produce conforming bitstreams. A Hypothetical Reference Decoder (HRD) can be used to check bitstream and decoder conformance. These standards may include coding tools that help deal with transmission errors and losses, but the use of those tools in encoding may be optional, and the decoding process for erroneous bitstreams may not be specified.

[0045] A syntax element may be defined as an element of data represented in the bitstream. A syntax structure may be defined as zero or more syntax elements present together in the bitstream in a specified order.

[0046] In most cases, the basic unit of input to an encoder and output of a decoder is a picture: a picture provided as input to an encoder is sometimes called a source picture, and a picture decoded by a decoder is sometimes called a decoded picture or reconstructed picture.

[0047] The source picture and the decoded picture each consist of one or more sample arrays, for example one of the following sets of sample arrays: - Luma (Y) only (monochrome) - Luma and two chromas (YCbCr or YCgCo) - Green, Blue and Red (GBR, also known as RGB) - Arrays representing other unspecified monochrome or tristimulus color samplings (e.g. YZX, also known as XYZ)

[0048] In the following, these arrays may be referred to as luma (or L or Y) and chroma, and the two chroma arrays may be referred to as Cb and Cr, regardless of the actual color representation used. The actual color representation used may be indicated, for example, in the encoded bitstream, for example using the Video Usability Information (VUI) syntax of HEVC. A component may be defined as an array or a single sample from one of the three sample arrays (luma and two chromas), or an array or a single sample of an array that makes up a monochrome picture.

[0049] A picture can be defined as a frame or a field. A frame contains a matrix of luma samples and possibly corresponding chroma samples. A field is a set of alternating sample rows of a frame, which can be used as an encoder input when the source signal is interlaced. The chroma sample array may not be present (hence monochrome sampling is used) or it may be subsampled when compared to the luma sample array.

[0050] Some chroma formats can be summarized as follows: - In monochrome sampling, there is only one sample array, which can be nominally considered the luma array. - In 4:2:0 sampling, each of the two chroma arrays has half the height and half the width of the luma array. - In 4:2:2 sampling, each of the two chroma arrays has the same height as the luma array and half the width of the luma array. - In 4:4:4 sampling when separate color planes are not used, each of the two chroma arrays has the same height and the same width as the luma array.

[0051] An encoding format or standard may allow the sample array to be encoded as separate color planes into a bitstream, from which each encoded color plane can be decoded separately. When separate color planes are used, each of the color planes is processed separately (by the encoder and / or decoder) as a picture with monochrome sampling.

[0052] Versatile Video Coding (VVC) proposes new coding tools, such as intra-prediction, inter-picture prediction, transform, quantization and coefficient coding, entropy coding, in-loop filtering, screen content coding, 360-degree video coding, high-level syntax and parallel processing. In the following, we briefly describe these tools in detail. Intra prediction - 67 Intra modes with wide angle mode extension - Block size and mode dependent 4-tap interpolation filter - Position dependent intra prediction combination (PDPC) - Cross component linear model intra prediction (CCLM) - Multiple reference line intra prediction - Intra-subpartition - Weighted intra prediction using matrix multiplication Inter-picture prediction - Block motion copy with spatial, temporal, history-based and pairwise average merge candidates - Affine motion inter prediction - Sub-block based temporal motion vector prediction - Adaptive motion vector resolution - 8x8 block-based motion compression for temporal motion prediction - High-precision (1 / 16 pixel) motion vector storage and motion compensation using an 8-tap interpolation filter for the luma component and a 4-tap interpolation filter for the chroma component - Triangular partition - Combined intra and inter prediction - merge with motion vector difference (MVD) (MMVD) - Symmetric MVD coding - Bidirectional optical flow - Decoder-side motion vector refinement - Bi-prediction using CU-level weights Transformation, quantization and coefficient coding - Multiple linear transform options using DCT2, DCT7 and DCT8 - Quadratic transformation for low frequency zones - Sub-block transformation for predicted inter-residuals - Dependent quantization with max QP increased from 51 to 63 - Transform coefficient coding with sign data hiding - Transform skip residual coding ·Entropy coding - Arithmetic coding engine with adaptive double windows probability update In-loop filter - In-loop reshaping - Deblocking filter with stronger and longer filters - Sample Adaptive Offset - Adaptive Loop Filter Screen content encoding - Current Picture Referencing with Reference Area Constraints ·360 degree video encoding - Horizontal wraparound motion compensation High-level syntax and parallel processing - Reference picture management using direct reference picture list signaling - Tile groups including rectangular tile groups

[0053] In VVC, each picture can be partitioned into coding tree units (CTUs) similar to HEVC. Pictures can also be partitioned into slices, tiles, bricks and subpictures. A quaternary tree structure can be used to partition the CTUs into smaller CUs. Each CU can be partitioned using nested multi-type trees, including quad trees and ternary and binary partitions. There are specific rules for inferring the partitioning of picture boundaries. Redundant partitioning patterns are not allowed in nested multi-type partitioning.

[0054] In VVC, a cross-component linear model (CCLM) prediction mode is used to reduce cross-component redundancy, in which chroma samples are predicted based on the reconstructed luma samples of the same CU by using the following linear model: pred C (i,j)=α·rec L '(i,j)+β In the above formula, pred C (i,j) represents the predicted chroma sample of a CU, and rec L '(i,j) represents the downsampled reconstructed luma sample of the same CU.

[0055] The CCLM parameters (α and β) are derived using up to four adjacent chroma samples and their corresponding downsampled luma samples. Figure 3 shows an example of the positions of the left and top samples and samples of the current block included in the CCLM mode, i.e., the samples used to derive α and β. Figure 3 shows the Rec C and Rec' L is shown, and Rec' L is for the downsampled reconstructed luma samples, and Rec C is for the reconstructed chroma samples.

[0056] If the dimensions of the current chroma block are W×H, then W′ and H′ are set as follows: - When LM mode is applied, W'=W, H'=H - When LM-A mode is applied, W'=W+H - When LM-L mode is applied, H'=H+W

[0057] The top adjacent position is denoted as S[0,-1]...S[W'-1,-1], and the left adjacent position is denoted as S[-1,0]...S[-1,H'-1].

[0058] Then, four samples are selected as follows: - S[W' / 4,-1], S[3*W' / 4,-1], S[-1,H' / 4], S[-1,3*H' / 4] when LM mode is applied and both upper and left adjacent samples are available - S[W' / 8,-1], S[3*W' / 8,-1], S[5*W' / 8,-1], S[7*W' / 8,-1] when LM-A mode is applied or only upper adjacent samples are available - S[-1,H' / 8], S[-1,3*H' / 8], S[-1,5*H' / 8], S[-1,7*H' / 8] when LM-L mode is applied or only left adjacent samples are available

[0059] The four adjacent luma samples at the selected position are downsampled and compared four times to find two smaller values ​​x0A and x1A and two larger values ​​x0B and x1B. Their corresponding chroma sample values ​​are denoted as y0A, y1A, y0B and y1B. Then, Xa, Xb, Ya and Yb are derived as follows: Xa=(x0A+x1A+1)>>1 Xb=(x0B+x1B+1)>>1 Ya=(y0A+y1A+1)>>1 Yb=(y0B+y1B+1)>>1

[0060] Finally, the linear model parameters α and β are obtained according to the following equations:

number

[0061] The division operation to calculate the parameter α is performed using a lookup table. To reduce the memory required to store this table, the value "diff" (the difference between the maximum and minimum values) and the parameter α are expressed in exponential notation. For example, diff is approximated using a 4-bit significant part and an exponent part. Thus, the table of 1 / diff is reduced to 16 elements for the 16 values ​​of the significant part as follows: DivTable[]={0,7,6,5,5,4,4,3,3,2,2,1,1,1,1,0}

[0062] This may have the advantage of both reducing the computational complexity and reducing the memory size required to store the required tables.

[0063] Alternatively, in addition to the top and left templates being used to jointly calculate the linear model coefficients, they can also be used in the remaining two LM modes, referred to as the LM_A and LM_L modes.

[0064] In LM_A mode, only the top template is used to calculate the linear model coefficients. The top template is extended to (W+H) to get more samples. In LM_L mode, only the left template is used to calculate the linear model coefficients. The left template is extended to (H+W) to get more samples.

[0065] For non-square blocks, the top template is dilated to W+W and the left template is dilated to H+H.

[0066] To match the chroma sample positions for 4:2:0 video sequences, two types of downsampling filters are applied to the luma samples to achieve a 2:1 downsampling ratio in both horizontal and vertical directions. The choice of downsampling filter is specified by the SPS level flag. These two downsampling filters are as follows, corresponding to "Type 0" and "Type 2" content, respectively:

[0067]

number

number

[0068] It is understood that when the top reference line is on a CTU boundary, only one luma line (a common line buffer in intra prediction) is used to create the downsampled luma samples.

[0069] This parameter computation is performed as part of the decoding process, and not just as an encoder search operation, and as a result, no syntax is used to communicate the α and β values ​​to the decoder.

[0070] A total of eight intra modes are allowed for chroma intra mode coding, including five traditional intra modes and three cross-component linear model modes (CCLM, LM_A, and LM_L). The chroma mode signaling and derivation process is shown in Table 1 below. The chroma mode coding directly depends on the intra prediction mode of the corresponding luma block. Since separate block partitioning structures for luma and chroma components are enabled in I slices, one chroma block may correspond to multiple luma blocks. Therefore, for chroma DM mode, the intra prediction mode of the corresponding luma block that covers the center position of the current chroma block is directly carried over.

[0071] [Table 1]

[0072] As shown in Table 2 below, a single binarization table is used regardless of the value of sps_cclm_enabled_flag. [Table 2]

[0073] In Table 2, the first binary digit indicates whether it is a regular mode (0) or an LM mode (1). If the first binary digit is an LM mode, the next binary digit indicates whether it is an LM_CHROMA (0) or not. If the binary digit is not an LM_CHROMA, the next binary digit indicates whether it is an LM_L (0) or an LM_A (1). In this case, when sps_cclm_enabled_flag is 0, the first binary digit of the corresponding intra_chroma_pred_mode binarization table can be discarded before entropy encoding. Or, in other words, it is inferred that the first binary digit is 0 and therefore not coded. This single binarization table is used both when sps_cclm_enabled_flag is equal to 0 and when it is equal to 1. The first two binary digits in the table are context coded using their own context model, and the remaining binary digits are bypass coded.

[0074] Furthermore, to reduce luma-chroma latency in dual trees, when a 64x64 luma coding tree node is partitioned with Not Split (ISP is not used for 64x64 CUs) or QT, the chroma CUs of 32x32 / 32x16 chroma coding tree nodes are allowed to use CCLM in the following manner: - If a 32x32 chroma node is not split or partitioned QT partitioned, all chroma CUs in the 32x32 node can use CCLM. - If a 32x32 chroma node is partitioned with Horizontal BT and the 32x16 child node does not split but uses Vertical BT partitioning, all chroma CUs of the 32x16 chroma node can use CCLM.

[0075] In all other luma and chroma coding tree splitting conditions, CCLM is not allowed for chroma CUs.

[0076] Multiple reference line (MRL) intra prediction uses more reference lines for intra prediction. Figure 4 shows an example of four reference lines (reference lines 0, 1, 2, 3), where samples of segments A and F are not fetched from reconstructed neighboring samples but are padded with the nearest samples from segments B and E, respectively. HEVC intra prediction uses the nearest reference line (i.e., reference line 0). In MRL, two additional lines (reference line 1 and reference line 3) are used.

[0077] The index of the selected reference line (mrl_idx) can be signaled in or along the bitstream and used to generate the intra predictor. For reference line idx greater than 0, only additional reference line modes can be included in the MPM list, and only the mpm index can be signaled without the remaining modes. The reference line index can be signaled before the intra prediction modes, and Planar mode can be excluded from the intra prediction modes if a non-zero reference line index is signaled.

[0078] MRL can be disabled for the first line of a block inside the CTU to prevent the use of extended reference samples outside the current CTU line. In addition, PDPC can be disabled when additional lines are used. For MRL modes, the derivation of DC values ​​for DC intra prediction modes for non-zero reference line index is aligned with that for reference line index 0. MRL requires storage of three adjacent luma reference lines with the CTU to generate predictions. The cross component linear model (CCLM) tool also requires three adjacent luma reference lines for its downsampling filter. To reduce decoder storage requirements, the definition of MLR, which uses the same three lines, is aligned with CLM.

[0079] Intra Subpartition (ISP) splits an intra predicted luma block vertically or horizontally into two or four subpartitions depending on the block size. For example, the minimum block size for ISP is 4×8 (or 8×4). If the block size is larger than 4×8 (or 8×4), the corresponding block is split by four subpartitions. It was found that M×128 (M≦64) and 128×N (N≦64) ISP blocks can create potential problems including 64×64 VDPU. For example, M×128 CU in single tree case is composed of M×128 luma TB (transform block) and two corresponding

[0080]

number

[0081] The matrix weighted intra prediction (MIP) method is a new intra prediction technique added to VVC. To predict samples of a rectangular block of width W and height H, matrix weighted intra prediction (MIP) takes as input one line of H reconstructed adjacent boundary samples to the left of the block and one line of W reconstructed adjacent boundary samples above the block. If reconstructed samples are not available, they are generated as they would be if they were performed in conventional intra prediction. Figure 5 shows an example of the matrix weighted intra prediction process, where the generation of the prediction signal is based on three steps: averaging, matrix-vector multiplication, and linear interpolation.

[0082] One of the features of inter prediction in VVC is merging using MVD. The merge list may contain the following candidates: 1) Spatial motion vector prediction (MVP) from spatially adjacent CUs 2) Temporal MVP from the same location CU 3) History-based MVP from a FIFO table 4) Pairwise Average MVP (use candidates already in the list) 5) Zero MV

[0083] Merge mode with motion vector difference (MMVD) is to signal MVD and resolution index after signaling merge candidates.

[0084] In symmetric MVD, in the case of bi-prediction, the motion information of list-1 is derived from the motion information of list-0.

[0085] In affine prediction, several motion vectors are indicated / signaled for different corners of a block, which are used to derive the motion vectors of sub-blocks. In affine merging, the affine motion information of a block is generated based on the regular or affine motion information of neighboring blocks.

[0086] In sub-block-based temporal motion vector prediction, motion vectors of sub-blocks of the current block are predicted from appropriate sub-blocks of the reference frame indicated by the motion vectors of spatially neighboring blocks (if available).

[0087] In adaptive motion vector resolution (AMVR), the precision of the MVD is signaled per CU.

[0088] For bi-prediction with CU-level weights, an index to the weight value for the weighted average of the two predicted blocks is indicated.

[0089] Bidirectional optical flow (BDOF) refines the motion vectors in the case of bidirectional prediction. BDOF can generate two prediction blocks using the signaled motion vectors. Then, a motion refinement that minimizes the error between the two prediction blocks is calculated using the gradient values ​​of those blocks. This motion refinement and the gradient values ​​are used to refine the final prediction block.

[0090] Transform is a solution to remove spatial redundancy of prediction residual blocks for block-based hybrid video coding. Furthermore, existing directional intra prediction gives rise to directional patterns of prediction residuals, which leads to predictable patterns for transform coefficients. The predictable patterns of transform coefficients are mainly observed in low-frequency components. Therefore, low-frequency non-separable transform (LFNST) can be used to further compress redundancy among low-frequency first-order transform coefficients, which are transform coefficients from traditional directional intra prediction.

[0091] Multiple Transform Selection (MTS) relies on three triangular transforms and selects, at the encoder side, the pair of horizontal and vertical transforms that maximizes the rate-distortion cost.

[0092] In the decoder-side intra mode derivation (DIMD) method, the intra prediction direction or mode is derived from previously encoded / decoded pixels at both the encoder and decoder sides. Therefore, unlike traditional intra prediction tools, no mode signaling is required. Pixel / sample prediction using DIMD mode can be performed as follows:

[0093] For the intra prediction mode (IPM) of the decoder-side intra mode derivation block, texture gradient analysis is performed at both the encoder and decoder sides. The process starts with an empty histogram of gradients (HoG) with a certain number of entries corresponding to different angular intra prediction modes. According to one approach, 65 entries are defined. During the texture gradient analysis, the amplitudes of these entries are determined. This HoG calculation can be performed, for example, by applying horizontal and vertical Sobel filters to the pixels of a template of width 3 around the block. If pixels above the template are included in a different CTU, they are not used in this texture analysis.

[0094] In this filtering, two kernel matrices of size 3x3 are used together with the filtering window, so that pixel values ​​in the filtering window A are convolved with the matrix. One matrix generates a horizontal gradient value Gx at the central pixel of the filtering window, and the other matrix generates a vertical gradient value Gy at the central pixel of the filtering window. In other words, the central pixel and eight pixels around the central pixel are used to calculate the gradient of the central pixel. The sum of the absolute values ​​of the two gradient values ​​indicates the magnitude of the gradient, and the arctangent of the ratio Gy / Gx indicates the direction of the gradient. If the filtering window has an edge, this direction also indicates the angular intra prediction mode. The filtering window is moved to the next pixel of the template, and the above procedure is repeated. According to one approach, the calculations described above are performed for each pixel in the central row of the template region.

[0095] The cross-component linear model (CCLM) uses a linear model to predict samples of chroma channels (e.g., Cb and Cr). The model parameters are derived based on neighboring reconstructed samples of the chroma block, co-located adjacent samples of the luma block, and co-located inner reconstructed samples of the luma block.

[0096] The purpose of CCLM is to find the correlation of samples between two or more channels. However, the linear model of the CCLM method cannot always provide the exact correlation between the luma and chroma channels, and therefore its performance is suboptimal.

[0097] Therefore, an objective of an embodiment of the present invention is to improve the prediction performance of Cross Component Linear Model (CCLM) prediction by providing joint intra prediction in chroma coding. Joint intra prediction uses a combination of CCLM and an intra prediction mode derived from a reference channel. This means that for a current block of a chroma channel, the derived intra prediction mode can be inherited from the co-located block of the luma channel. Alternatively, the derived mode can be based on the prediction modes of reconstructed neighboring blocks of the chroma channels (e.g. Cb and Cr).

[0098] The final prediction for a chroma block is achieved by combining the CCLM and the derived prediction mode with some weighting.

[0099] In the following, the embodiment of the present invention will be discussed in more detail. A joint prediction method according to the embodiment combines CCLM and derived intra-prediction mode prediction. The joint prediction method is configured to predict samples of a block based on CCLM prediction and traditional spatial intra-prediction. The traditional intra-prediction mode can be derived from a co-located block of a reference channel (e.g., luma channel) of the CCLM mode or from a region within the co-located block.

[0100] The derived traditional intra mode is used to find additional correlation between samples of the two channels. Figure 6 shows an example of a coding block 610 of a chroma channel 601 and a corresponding co-located block 620 of a luma channel 602. When the block segmentation of different channels does not correspond to each other, the co-located block 620 can be determined by mapping a position of the chroma channel 601 to a position of the luma channel 602, and the co-located block 620 uses the block of the determined luma position as the co-located block 620. For example, the upper left corner, the lower right corner or the center point of the chroma block can be used as the reference chroma position in this process.

[0101] According to an alternative approach, the mode derived from the reference channel is not always the co-located block. The derived mode can be determined based on the prediction mode of at least one of the blocks in the co-located expanded area. This is illustrated in FIG. 7, which shows a co-located block 720 and a co-located neighborhood 725 for a coding block 710. In this case, the derived mode can be determined based on the rate-distortion (RD) performance of two or more prediction modes. As another example, the prediction mode with the largest sample area of ​​the expanded co-located neighborhood or the prediction mode associated with the largest luma block of the expanded co-located neighborhood can also be selected as the derived mode.

[0102] The overall process of the method according to one embodiment includes the following steps: a first prediction, which includes predicting samples inside the block using a CCLM mode; - deriving an intra prediction mode from a coded block of a reference channel; - a second prediction, comprising predicting samples inside the block based on the derived intra prediction mode; and - determining a final prediction for the block based on the first and second predictions with predetermined weightings; Includes.

[0103] 8 shows an example of a joint prediction process that combines a first prediction and a second prediction. The first prediction 810 is a prediction using the CCLM mode, and the second prediction 820 is a prediction using the derived mode. When combining 850, both the first prediction and the second prediction are weighted.

[0104] The weighting approach for the combination 850 can be one of the following: - The first and second predictions can be combined with a constant equal weight for all samples of the block. The first and second predictions can be combined with constant unequal weights for all samples of the block. The first and second predictions can be combined using equal / unequal sample-wise weighting, where each predicted sample can be weighted differently than the other samples. The weighting values ​​of the samples may be determined based on the prediction direction or mode identifier of the derived mode. The weight value of a sample may be determined based on the prediction direction of the CCLM mode, the position of the reference sample, or the mode identifier. The weight value of the sample may be determined based on the CCLM mode and the prediction direction of the derived mode, the position of the reference sample or the mode identifier. - The weight values ​​of the samples can be determined based on the size of the block, e.g., samples on the larger side of the block can use larger weights for the derived modes and smaller weights for the CCLM modes, or vice versa. The predicted block weight value can be set to zero for some block positions, e.g., the weight of a block generated using a derived prediction mode can be set to zero when its distance from the top or left edge of the block is greater than a threshold.

[0105] The joint prediction process according to these embodiments can be applied to a variety of scenarios, which are described below.

[0106] This joint prediction can be applied to one chroma channel (e.g., Cb or Cr), while the other channel can be predicted based on CCLM modes only or derived modes only. The selection of the channel to which the joint prediction is applied can be fixed or can be based on a rate-distortion process in the codec.

[0107] Alternatively, each of the chroma channels can be predicted using one mode. For example, one channel can be predicted based on a CCLM mode and the other channel can be predicted based on a derived intra mode. The selection of the prediction mode for each channel can be determined based on a rate-distortion process or can be fixed.

[0108] The derived mode for the second prediction may be determined based on the prediction mode of a neighboring block of a corresponding chroma channel.

[0109] The derived mode can be set to a predefined mode, such as a planar prediction mode or a DC prediction mode. The derived mode can also be indicated using higher level signaling, such as a higher level signaling including a syntax element that determines the derived mode in a slice or picture header or in a bitstream parameter set. Alternatively, the derived mode can be indicated separately or jointly for these different chroma channels at the transform unit, prediction unit, or coding unit level.

[0110] According to one embodiment, the derived modes for the chroma channels are different. For example, the derived mode for one channel (e.g., Cb or Cr) can be determined based on a co-located block of a reference channel (e.g., luma channel), and the derived mode for the other chroma channel can be determined based on the prediction mode of an adjacent block of that channel.

[0111] Any syntax elements required for the embodiments of the present invention can be signaled in or along the bitstream. This signaling can be done under certain conditions such as CCLM direction, derived mode direction, block position and size, etc. Alternatively, the syntax elements can be determined on the decoder side, for example by checking the availability of CCLM mode, derived mode, block size, etc.

[0112] In another embodiment, the derived mode can be determined from reconstructed adjacent samples of the encoded channel based on texture analysis methods, for which a certain number of reconstructed adjacent samples (or a template of samples) can be considered.

[0113] According to another embodiment, the texture analysis method for deriving the intra prediction mode may be one or more of a decoder-side intra mode derivation (DIMD) method, a template matching-based (TM-based) method, an intra block copy (IBC) method, etc.

[0114] The mode derivation from the neighboring samples can take into account the direction of the CCLM mode, for example, if the CCLM mode uses only the upper neighboring samples, the mode can be derived according to only the upper neighboring samples, or vice versa.

[0115] In the case where the derived modes are achieved by reconstructed adjacent samples, one mode per channel can be derived based on corresponding adjacent samples combining using CCLM modes. Alternatively, the derived modes can be common for both chroma channels, and the derived modes can be derived according to reconstructed adjacent samples of one or both channels.

[0116] As with the joint prediction in the previous case, the derived modes achieved from texture analysis of adjacent samples can be applied to one channel, while the other channel can be predicted using only CCLM modes. Alternatively, the joint prediction can be applied to only one channel, while the other channel can be predicted based on only CCLM or only derived modes.

[0117] The weight values ​​for combining the two predictions can be determined based on texture analysis of the reconstructed neighboring samples. For example, the intra prediction modes derived using DIMD modes include certain weights in the derivation process of the respective modes. These weights or some mapping of these weights can be considered for the weight determination of the derived modes and CCLM modes.

[0118] According to another embodiment, a transform selection (such as multiple transform selection (MTS), low frequency non-separable transform (LFNST)), or an index of a transform in the LFNST, may be determined based on one or both of the derived mode and the CCLM mode.

[0119] It should be understood that embodiments of the present invention are not limited to combining two predictions. A final prediction can be achieved by combining three or more predictions. For example, a final prediction can be calculated using one or more CCLM modes and one or more derived modes.

[0120] A method according to an embodiment is illustrated in the flow chart of Figure 9. The method generally includes: receiving a picture to be coded 910, performing at least one prediction according to a first prediction mode for inner samples of a block of the picture of a current channel 920, deriving an intra prediction mode from at least one coded block of a reference channel 930, performing at least one other prediction according to the derived intra prediction mode for inner samples of the block of the picture 940, and determining a final prediction of a block based on the weighted at least one first prediction and the at least one second prediction 950. Each of these steps can be implemented by a corresponding respective module of a computer system.

[0121] An apparatus according to an embodiment comprises means for receiving a picture to be coded, means for performing at least one prediction according to a first prediction mode on inner samples of a block of the picture of a current channel, means for deriving an intra prediction mode from at least one coded block of a reference channel, means for performing at least one other prediction according to the derived intra prediction mode on inner samples of the block of the picture, and means for determining a final prediction of a block based on the weighted at least one first prediction and the at least one second prediction. These means comprise at least one processor and a memory including computer program code, which may further comprise a processor circuit. The memory and the computer program code are configured to cause the apparatus to execute the method of FIG. 9 according to various embodiments.

[0122] An example of the device is shown in Figure 10. The generalized structure of the device is described according to the functional blocks of the system. Some functions can be performed by a single physical device. For example, all the computational steps can be performed by a single processor if desired.

[0123] The data processing system of the device according to the example of FIG. 10 comprises a main processing unit 100, a memory 102, a storage device 104, an input device 106, an output device 108 and a graphics subsystem 110, which are connected to each other via a data bus 112. The main processing unit 100 is a processing unit arranged to process data in the data processing system. The main processing unit 100 can comprise or be implemented as one or more processors or processor circuits. The memory 102, the storage device 104, the input device 106 and the output device 108 can include other components known to those skilled in the art. The memory 102 and the storage device 104 store data in the data processing system 100. In the memory 102 there is computer program code, for example computer program code for implementing neural network training or other machine learning processes. The input device 106 inputs data into the system and the output device 108 receives data from the data processing system and transfers the data, for example to a display. Although the data bus 112 is shown as a single line, it can be any combination of a processor bus, a PCI bus, a graphical bus, an ISA bus, etc. Thus, those skilled in the art will readily appreciate that the device can be any data processing device, such as a computer device, a personal computer, a server computer, a mobile phone, a smart phone, or an internet access device, e.g., an internet table computer.

[0124] Various embodiments may be implemented with the aid of computer program code present in a memory and causing an associated device to perform the method. For example, a device may comprise circuitry and electronic components for processing, receiving and transmitting data, computer program code in a memory, and a processor that, when executed, causes the device to perform the features of the embodiments. Additionally, a network device, such as a server, may comprise circuitry and electronic components for processing, receiving and transmitting data, computer program code in a memory, and a processor that, when executed, causes the network device to perform the features of the embodiments. The computer program code includes one or more operating characteristics, the operating characteristics being defined by configuration by the computer based on the type of the processor, and a system may be connected to the processor by a bus, the programmable operating characteristics of the system being for implementing the method according to various embodiments.

[0125] The computer program product according to one embodiment may be embodied on a non-transitory computer readable medium. According to another embodiment, the computer program product may be downloaded in the form of data packets over a network.

[0126] If desired, the various functions discussed herein may be performed in a different order and / or simultaneously with other functions. Furthermore, if desired, one or more of the above-described functions and embodiments may be optional or combined.

[0127] While the independent claims set out various aspects of the embodiments, other aspects include other combinations of the features of the described embodiments and / or the dependent claims with the features of the independent claims, and are not limited to only the combinations explicitly set out in the claims.

[0128] It should also be noted that while exemplary embodiments have been described above, these descriptions should not be construed in a limiting sense, rather there are several variations and modifications that can be made without departing from the scope of the present disclosure, as defined in the appended claims.

Claims

1. receiving a picture to be encoded, performing at least one prediction according to a first prediction mode on samples inside a block of said picture of the current channel; - deriving an intra prediction mode from at least one coded block of a reference channel; - performing at least one other prediction on the samples inside the block of the picture according to the derived intra prediction mode; and determining a final prediction for said block based on weighted said at least one first prediction and said at least one second prediction; The method includes:

2. The method of claim 1 , wherein the first prediction mode is a cross-component linear mode.

3. The method of claim 1 , wherein the derived intra-prediction mode is derived from at least one co-located block of a channel different from the current channel.

4. The method of claim 1 , wherein the derived intra-prediction mode is derived from at least one neighboring block of the current channel.

5. The method of claim 1 , wherein the derived intra-prediction mode is determined based on texture analysis from reconstructed adjacent samples of the current channel.

6. The method of claim 5 , wherein the texture analysis method is one of a decoder-side intra mode derivation method, a template matching based method, and an intra block copying method.

7. The method of claim 5 , wherein the determination from the neighboring samples takes into account a direction of the first prediction.

8. The method of claim 1 , wherein the final prediction comprises the first and second predictions combined with constant equal weights for all samples of the block.

9. The method of claim 1 , wherein the final prediction comprises a combined first and second prediction with constant unequal weights for all samples of the block.

10. 2. The method of claim 1, wherein the final prediction comprises the first and second predictions combined, with equal or unequal per-sample weightings, where the weights of each predicted sample are different from each other.

11. The method of claim 1 , further comprising determining weight values ​​for the samples based on a prediction direction or a mode identifier of a derived intra-prediction mode.

12. The method of claim 1 , further comprising determining the weight value of the sample based on a prediction direction of a cross-component linear mode, a reference sample position, or a mode identifier.

13. The method of claim 1 , further comprising determining the weight values ​​of the samples based on a cross-component linear mode and a prediction direction of a derived prediction mode, a reference sample position, or a mode identifier.

14. The method of claim 1 , further comprising determining weight values ​​for the samples based on a size of the block.

15. 1. An apparatus comprising at least one processor and a memory containing computer program code, the memory and the computer program code operating in conjunction with the at least one processor to perform at least: receiving a picture to be encoded, performing at least one prediction according to a first prediction mode on samples inside a block of said picture of the current channel; - deriving an intra prediction mode from at least one coded block of a reference channel; - performing at least one other prediction on the samples inside the block of the picture according to the derived intra prediction mode; and determining a final prediction for said block based on weighted said at least one first prediction and said at least one second prediction; The apparatus is configured to cause the apparatus to execute.

16. The apparatus of claim 15 , wherein the first prediction mode is performed in a cross-component linear mode.

17. The apparatus of claim 15 , wherein the derived intra-prediction mode is derived from at least one co-located block of a channel different from the current channel.

18. The apparatus of claim 15 , wherein the derived intra-prediction mode is derived from at least one neighboring block of the current channel.

19. The apparatus of claim 15 , wherein the derived intra-prediction mode is determined based on texture analysis from reconstructed adjacent samples of the current channel.

20. 20. The apparatus of claim 19, wherein the texture analysis method is one of a decoder-side intra mode derivation method, a template matching based method, and an intra block copying method.

21. The apparatus of claim 19 , wherein the determination from the neighboring samples takes into account a direction of the first prediction.

22. The apparatus of claim 15 , wherein a final prediction comprises a combined first and second prediction with constant equal weights for all samples of the block.

23. The apparatus of claim 15 , wherein a final prediction comprises a combined first and second prediction with constant unequal weights for all samples of the block.

24. 16. The apparatus of claim 15, wherein the final prediction comprises the first and second predictions combined, with equal or unequal per-sample weightings, where the weights of each predicted sample are different from one another.

25. The apparatus of claim 15 , further comprising: determining weight values ​​of the samples based on a prediction direction or a mode identifier of a derived intra-prediction mode.

26. The apparatus of claim 15 , further comprising: determining weight values ​​of the samples based on a prediction direction of a cross-component linear mode, a reference sample position, or a mode identifier.

27. The apparatus of claim 15 , further comprising: determining weight values ​​of the samples based on a cross-component linear mode and a prediction direction of a derived prediction mode, a reference sample position, or a mode identifier.

28. The apparatus of claim 15 , further comprising: determining weight values ​​for the samples based on a size of the block.

Images