Intra-frame prediction methods and devices

Abstract

An intra-frame prediction method using cross-component liner prediction mode (CCLM) includes: determining a luminance block corresponding to a current chroma block; obtaining luminance reference pixels of the luminance block based on L available chroma reference pixels of the current chroma block, wherein the obtained luminance reference pixels of the luminance block are downsampled luminance reference pixels; calculating linear model coefficients based on the luminance reference pixels and the chroma reference pixels corresponding to the luminance reference pixels; and obtaining a predicted value of the current chroma block based on the linear model coefficients and the downsampled luminance block value of the luminance block.

Description

[0001] Cross-referencing related applications

[0002] This application is a divisional application. The original application has the application number 201980065676.X and the original application date is September 30, 2019. The entire contents of the original application are incorporated herein by reference.

[0003] The original application claims the benefit of U.S. Provisional Patent Application No. 62 / 742,266, filed October 5, 2018; U.S. Provisional Patent Application No. 62 / 742,355, filed October 6, 2018; U.S. Provisional Patent Application No. 62 / 742,275, filed October 6, 2018; and U.S. Provisional Patent Application No. 62 / 742,356, filed October 6, 2018. The entire contents of the original application are incorporated herein by reference. Technical Field

[0004] The embodiments of the present invention generally relate to the field of video decoding, and more specifically to the field of intra-frame prediction using cross-component linear model prediction (CCLM). Background Technology

[0005] Even with shorter videos, a large amount of video data needs to be described, which can be challenging when the data needs to be sent over communication networks with limited bandwidth or transmitted otherwise. Therefore, video data is typically compressed before transmission over modern telecommunications networks. Video size can also be an issue when storing video on storage devices due to potentially limited memory resources. Video compression devices typically use software and / or hardware on the source side to encode video data before transmission or storage, thereby reducing the amount of data needed to represent digital video images. The compressed data is then received at the destination by a video decompression device that decodes the video data. Given limited network resources and the growing demand for higher video quality, there is a need for improved compression and decompression techniques that can increase compression ratios with minimal impact on image quality. Efficient Video Decoding, the latest video compression technology published by the ISO / IEC Moving Picture Experts Group and the ITU-T Video Coding Experts Group, known as ISO / IEC 23008-2 MPEG-H Part 2 or ITU-T H.265, approximately doubles the data compression ratio at the same video quality level, or significantly improves video quality at the same bitrate. Summary of the Invention

[0006] Examples of the present invention provide intra-frame prediction apparatus and methods for encoding and decoding images, which can improve the efficiency of cross-component linear model prediction (CCLM), thereby improving the decoding efficiency of video signals. The invention is described in detail in the examples and claims of this document.

[0007] The above and other objectives are achieved through the subject matter claimed in the independent claims. Other implementations are apparent from the dependent claims, the specification, and the drawings.

[0008] Specific embodiments are set forth in the appended independent claims, and other embodiments are set forth in the dependent claims.

[0009] According to a first aspect, the present invention relates to a method for performing intra-frame prediction using a linear model. The method includes: determining a luma block corresponding to a current chroma block; obtaining luma reference pixels of the luma block based on L available chroma reference pixels of the current chroma block, wherein the obtained luma reference pixels of the luma block are downsampled luma reference pixels; calculating linear model coefficients based on the luma reference pixels and the chroma reference pixels corresponding to the luma reference pixels; and obtaining a predicted value of the current chroma block based on the linear model coefficients and the downsampled value of the luma block. The chroma reference pixels of the current chroma block include neighboring reconstructed pixels of the current chroma block. The L available chroma reference pixels are determined from the neighboring reconstructed pixels. Similarly, the neighboring pixels of the luma block are also neighboring reconstructed pixels of the luma block (i.e., neighboring reconstructed luma pixels). In one example, the luma reference pixels of the luma block are obtained by downsampling the neighboring reconstructed pixels selected based on the available chroma reference pixels.

[0010] In existing methods, the availability of reference pixels is determined using the luminance reference pixels to determine the linear model coefficients. However, in some scenarios, available luminance reference pixels do not have corresponding chrominance reference pixels, leading to decoding errors. The technique presented herein addresses this problem by checking the availability of the chrominance reference pixels to determine their availability. In some examples, a chrominance reference pixel is available if it is within the current image, slice, or tile, and has already been reconstructed. In other examples, a chrominance reference pixel is available if it is within the current image, slice, or tile, has already been reconstructed, and is not omitted according to the encoding decision, etc. The available reference pixels for the current chrominance block can be available reconstructed neighboring pixels of the chrominance block. The luminance reference pixels corresponding to the available chrominance reference pixels are used to determine the linear model coefficients.

[0011] According to the first aspect, in one possible implementation of the method, determining the L available chroma reference pixels includes: determining that the L upper adjacent chroma pixels of the current chroma block are available, where 1≤L≤W2, W2 represents the range of upper reference pixels, and L and W2 are positive integers, and using the L upper adjacent chroma pixels as the available chroma reference pixels.

[0012] According to the first aspect or any of the above implementations of the first aspect, in one possible implementation of the method, W2 is equal to 2×W or W+H, where W represents the width of the current chroma block and H represents the height of the current chroma block.

[0013] According to the first aspect or any of the above implementations of the first aspect, in one possible implementation of the method, determining the L available chroma reference pixels includes: determining that the L left adjacent chroma pixels of the current chroma block are available, where 1≤L≤H2, H2 represents the range of left reference pixels, and L and H2 are positive integers, and using the L left adjacent chroma pixels as the available chroma reference pixels.

[0014] According to the first aspect or any of the above implementations of the first aspect, in one possible implementation of the method, H2 is equal to 2×H or W+H, where W represents the width of the current chroma block and H represents the height of the current chroma block.

[0015] According to the first aspect or any of the above implementations of the first aspect, in one possible implementation of the method, determining L available chroma reference pixels includes: determining that L1 upper adjacent chroma pixels and L2 left adjacent chroma pixels of the current chroma block are available, wherein 1≤L1≤W2, 1≤L2≤H2, W2 represents the range of upper reference pixels, H2 represents the range of left reference pixels, L1, L2, W2 and H2 are positive integers and L1+L2=L, and the L1 upper adjacent chroma pixels and the L2 left adjacent chroma pixels are used as the available chroma reference pixels.

[0016] According to the first aspect or any of the above implementations of the first aspect, in one possible implementation of the method, the luminance reference pixel is obtained by downsampling only the adjacent pixels selected according to the L available chroma reference pixels above the luminance block, or only the adjacent pixels selected according to the L available chroma reference pixels to the left of the luminance block. For example, if L equals 4, the luminance reference pixel is obtained by downsampling 24 adjacent pixels selected according to 4 available chroma reference pixels above the luminance block, or by downsampling 24 adjacent pixels selected according to 4 available chroma reference pixels to the left of the luminance block, wherein a 6-tap filter is used in the downsampling process.

[0017] According to the first aspect or any of the above implementations of the first aspect, in one possible implementation of the method, a downsampled luminance block of the luminance block is obtained by downsampling the reconstructed luminance block of the luminance block corresponding to the current chroma block.

[0018] According to the first aspect or any of the above implementations of the first aspect, in one possible implementation of the method, when the luminance reference pixel is obtained only from the adjacent pixels above the luminance block and when the top row of the current chroma block overlaps with the top row of the current coding tree unit (CTU), the luminance reference pixel is obtained by using only one row of adjacent reconstructed luminance pixels of the reconstructed luminance block of the luminance block.

[0019] According to the first aspect or any of the above implementations of the first aspect, in one possible implementation of the method, calculating the linear model coefficients based on the luminance reference pixel and the chrominance reference pixel corresponding to the luminance reference pixel includes: determining a maximum luminance value and a minimum luminance value based on the luminance reference pixel; obtaining a first chrominance value at least partially based on the position of the luminance reference pixel related to the maximum luminance value; obtaining a second chrominance value at least partially based on the position of the luminance reference pixel related to the minimum luminance value; and calculating the linear model coefficients based on the first chrominance value, the second chrominance value, the maximum luminance value, and the minimum luminance value.

[0020] According to the first aspect or any of the above implementations of the first aspect, in one possible implementation of the method, obtaining the first chromaticity value based at least partially on the position of a luminance reference pixel related to the maximum luminance value includes: obtaining the first chromaticity value based at least partially on one or more positions of one or more luminance reference pixels related to the maximum luminance value; obtaining the second chromaticity value based at least partially on the position of a luminance reference pixel related to the minimum luminance value includes: obtaining the second chromaticity value based at least partially on one or more positions of one or more luminance reference pixels related to the minimum luminance value.

[0021] According to the first aspect or any of the above-described implementations of the first aspect, in one possible implementation of the method, the linear model coefficients α and β are calculated according to the following equation:

[0022]

[0023] β=y A -αx A

[0024] Where, x B This represents the maximum brightness value, y B This represents the first chromaticity value, x. A This represents the minimum brightness value, y A This represents the second chromaticity value.

[0025] According to the first aspect or any of the above implementations of the first aspect, in one possible implementation of the method, the predicted value of the current chroma block is obtained according to the following equation:

[0026] pred C (i,j)=α·rec L ′ (i,j)+β,

[0027] Among them, predC (i,j) represents the predicted value of the chroma pixel in the current chroma block, rec′ L (i,j) represents the pixel value of the corresponding luminance pixel in the downsampled luminance block of the reconstructed luminance block of the luminance block.

[0028] According to the first aspect or any of the above implementations of the first aspect, in one possible implementation of the method, when the luminance reference pixel is obtained only from the adjacent pixels to the left of the luminance block and the current chroma block is located at the left boundary of the current coding tree unit (CTU), the luminance reference pixel is obtained by using only a column of adjacent reconstructed luminance pixels of the reconstructed luminance block.

[0029] According to the first aspect or any of the above implementations of the first aspect, in one possible implementation of the method, the linear model includes a multi-directional linear model (MDLM).

[0030] According to a second aspect, the present invention relates to a method for performing intra-frame prediction using a linear model. The method includes: determining a luminance block corresponding to a current chroma block; obtaining luminance reference pixels of the luminance block based on L available chroma reference pixels of the current chroma block, wherein the obtained luminance reference pixels of the luminance block are downsampled luminance reference pixels obtained by downsampling neighboring pixels (i.e., neighboring reconstructed luminance pixels) of the luminance block corresponding to the L available chroma reference pixels; calculating linear model coefficients based on the luminance reference pixels and the chroma reference pixels corresponding to the luminance reference pixels; and obtaining a predicted value of the current chroma block based on the linear model coefficients and the value of the downsampled luminance block of the luminance block. The chroma reference pixels of the current chroma block include neighboring reconstructed pixels of the current chroma block. The L available chroma reference pixels are determined from the neighboring reconstructed pixels. Similarly, the neighboring pixels of the luminance block are also neighboring reconstructed pixels (i.e., neighboring reconstructed luminance pixels) of the luminance block. The luminance reference pixel of the luminance block is obtained by downsampling the adjacent reconstructed pixels corresponding to the available chromaticity reference pixel.

[0031] Regarding "the adjacent reconstructed pixels corresponding to the available chroma reference pixels," the correspondence between the adjacent reconstructed luminance pixels and the available chroma reference pixels is not limited to a one-to-one correspondence; it can also be understood that the correspondence between the adjacent reconstructed luminance pixels and the available chroma reference pixels can be an "M to N correspondence." For example, if a 6-tap filter is used for downsampling, then M = 24 and N = 4.

[0032] According to a third aspect, the present invention relates to an apparatus for encoding video data. The apparatus includes a video data storage device and a video encoder. The video encoder is configured to: determine a luminance block corresponding to a current chroma block; obtain luminance reference pixels of the luminance block based on (or by) determining L available chroma reference pixels of the current chroma block, wherein the obtained luminance reference pixels of the luminance block are downsampled luminance reference pixels, the downsampled luminance reference pixels being obtained by downsampling adjacent pixels (i.e., adjacent reconstructed luminance pixels) of the luminance block corresponding to the L available chroma reference pixels; calculate linear model coefficients of a linear model based on the luminance reference pixels and the chroma reference pixels corresponding to the luminance reference pixels; and obtain a predicted value of the current chroma block based on the linear model coefficients and the value of the downsampled luminance block of the luminance block. For example, if L equals 4, then the luminance reference pixel of the luminance block is 4 downsampled luminance reference pixels, which are obtained by downsampling 24 neighboring pixels of the luminance block corresponding to the 4 available chrominance reference pixels, wherein a 6-tap filter is used in the downsampling process.

[0033] Fourthly, the present invention relates to an apparatus for decoding video data. The apparatus includes a video data storage device and a video decoder. The video decoder is configured to: determine a luminance block corresponding to a current chroma block; obtain luminance reference pixels of the luminance block based on L available chroma reference pixels of the current chroma block, wherein the obtained luminance reference pixels of the luminance block are downsampled luminance reference pixels; calculate linear model coefficients based on the luminance reference pixels and the chroma reference pixels corresponding to the luminance reference pixels; and obtain a predicted value of the current chroma block based on the linear model coefficients and the downsampled value of the luminance block.

[0034] In existing methods, the availability of a pixel is determined using the luminance reference pixel to determine the linear model coefficients. However, in some scenarios, an available luminance reference pixel may not have a corresponding chrominance reference pixel, leading to decoding errors. The technique presented herein addresses this problem by checking the availability of the chrominance reference pixel to determine its availability. In some examples, a chrominance reference pixel is available if it is within the current image, stripe, or block, and has already been reconstructed. In other examples, a chrominance reference pixel is available if it is within the current image, stripe, or block, has already been reconstructed, and is not omitted according to the encoding decision, etc. The available chrominance reference pixel for the current chrominance block can be a usable reconstructed neighbor pixel of the chrominance block (i.e., a usable reconstructed neighbor chrominance pixel). The luminance reference pixel corresponding to the available chrominance reference pixel is used to determine the linear model coefficients.

[0035] According to the third or fourth aspect, in one possible implementation of the device, determining the L available chroma reference pixels includes: determining that the L upper adjacent chroma pixels of the current chroma block are available, where 1≤L≤W2, W2 represents the range of upper reference pixels, and L and W2 are positive integers, and using the L upper adjacent chroma pixels as the available chroma reference pixels.

[0036] According to the third or fourth aspect or any of the above-described implementations of the third or fourth aspect, in one possible implementation of the device, W2 is equal to 2×W or W+H, where W represents the width of the current chroma block and H represents the height of the current chroma block.

[0037] According to the third or fourth aspect or any of the above implementations of the third or fourth aspect, in one possible implementation of the device, determining the L available chroma reference pixels includes: determining that the L left adjacent chroma pixels of the current chroma block are available, where 1≤L≤H2, H2 represents the range of left reference pixels, and L and H2 are positive integers, and using the L left adjacent chroma pixels as the available chroma reference pixels.

[0038] According to the third or fourth aspect or any of the above-described implementations of the third or fourth aspect, in one possible implementation of the device, H2 is equal to 2×H or W+H, where W represents the width of the current chroma block and H represents the height of the current chroma block.

[0039] According to the third or fourth aspect or any of the above implementations of the third or fourth aspect, in one possible implementation of the device, determining L available chroma reference pixels includes: determining that L1 upper adjacent chroma pixels and L2 left adjacent chroma pixels of the current chroma block are available, wherein 1≤L1≤W2, 1≤L2≤H2, W2 represents the range of upper reference pixels, H2 represents the range of left reference pixels, L1, L2, W2 and H2 are positive integers and L1+L2=L, and the L1 upper adjacent chroma pixels and the L2 left adjacent chroma pixels are used as the available chroma reference pixels.

[0040] According to the third or fourth aspect or any of the above implementations of the third or fourth aspect, in one implementation of the device, the luminance reference pixel is obtained by: downsampling only the adjacent pixels above the luminance block selected according to the L available chromaticity reference pixels, or downsampling only the adjacent pixels to the left of the luminance block selected according to the L available chromaticity reference pixels.

[0041] According to the third or fourth aspect or any of the above-described implementations of the third or fourth aspect, in one possible implementation of the device, a downsampled luminance block of the luminance block is obtained by downsampling the reconstructed luminance block of the luminance block corresponding to the current chroma block.

[0042] According to the third or fourth aspect or any of the above implementations of the third or fourth aspect, in one possible implementation of the device, when the luminance reference pixel is obtained only from the adjacent pixels above the luminance block and when the top row of the current chroma block overlaps with the top row of the current coding tree unit (CTU), the luminance reference pixel is obtained by using only one row of adjacent reconstructed luminance pixels of the reconstructed luminance block of the luminance block.

[0043] According to the third or fourth aspect or any of the above-described implementations of the third or fourth aspect, in one possible implementation of the device, the linear model coefficients α and β are calculated according to the following equation:

[0044]

[0045] β=y A -αx A

[0046] Where, x B This represents the maximum brightness value, y B x represents the first chromaticity value. A This represents the minimum brightness value, yA This represents the second chromaticity value.

[0047] According to the third or fourth aspect or any of the above-described implementations of the third or fourth aspect, in one possible implementation of the device, the predicted value of the current chroma block is obtained according to the following equation:

[0048] pred C (i,j)=α·rec L ′ (i,j)+β,

[0049] Among them, pred C (i,j) represents the predicted value of the chroma pixel in the current chroma block, rec′ L (i,j) represents the pixel value of the corresponding luminance pixel in the downsampled luminance block of the reconstructed luminance block of the luminance block.

[0050] According to the third or fourth aspect or any of the above implementations of the third or fourth aspect, in one possible implementation of the device, when the luminance reference pixel is obtained only from the adjacent pixels to the left of the luminance block and the current chroma block is located at the left boundary of the current coding tree unit (CTU), the luminance reference pixel is obtained by using only a column of adjacent reconstructed luminance pixels of the reconstructed luminance block.

[0051] According to the third or fourth aspect or any of the above-described implementations of the third or fourth aspect, in one possible implementation of the device, the linear model includes a multi-directional linear model (MDLM).

[0052] According to a fifth aspect, the present invention relates to a method for decoding an intra-frame chroma prediction mode into a bitstream of a video signal. The method includes: performing intra-frame prediction on chroma blocks in the video signal according to the intra-frame chroma prediction mode, wherein the intra-frame chroma prediction mode is selected from a first set of modes, a second set of modes, or a third set of modes, the second set of modes including at least one of a CCLM_L mode or a CCLM_T mode; generating a bitstream of the video signal by carrying syntax elements indicating the intra-frame chroma prediction mode, wherein the number of bits of the syntax elements when selecting the intra-frame chroma prediction mode from the first set of modes is less than the number of bits of the syntax elements when selecting the intra-frame chroma prediction mode from the second set of modes, and the number of bits of the syntax elements when selecting the intra-frame chroma prediction mode from the second set of modes is less than the number of bits of the syntax elements when selecting the intra-frame chroma prediction mode from the third set of modes.

[0053] The provided method for encoding the intra-frame chroma prediction mode allows CCLM_L and CCLM_T to be represented by binary strings and carried in the bitstream of the video signal.

[0054] According to the fifth aspect, in one possible implementation of the method, the first set of modes includes at least one of a derived mode (DM) or a cross-component linear model (CCLM) prediction mode, and the third set of modes includes at least one of a vertical mode, a horizontal mode, a DC mode, or a planar mode.

[0055] According to the fifth aspect or any of the above-described implementations of the fifth aspect, in one possible implementation of the method, the syntax element of the DM mode is 0; the syntax element of the CCLM mode is 10; the syntax element of the CCLM_L mode is 1110; the syntax element of the CCLM_T mode is 1111; the syntax element of the planar mode is 11000; the syntax element of the vertical mode is 11001; the syntax element of the horizontal mode is 11010; and the syntax element of the DC mode is 11011.

[0056] According to the fifth aspect or any of the above-described implementations of the fifth aspect, in one possible implementation of the method, the syntax element of the DM mode is 00; the syntax element of the CCLM mode is 10; the syntax element of the CCLM_L mode is 110; the syntax element of the CCLM_T mode is 111; the syntax element of the planar mode is 0100; the syntax element of the vertical mode is 0101; the syntax element of the horizontal mode is 0110; and the syntax element of the DC mode is 0111.

[0057] According to a sixth aspect, the present invention relates to a method for decoding an intra-frame chroma prediction mode in a bitstream of a video signal. The method includes: parsing a plurality of syntax elements from the bitstream of the video signal; determining an intra-frame chroma prediction mode based on one of the plurality of syntax elements, wherein the intra-frame chroma prediction mode is determined from a first set of modes, a second set of modes, or a third set of modes, the second set of modes including at least one of a CCLM_L mode or a CCLM_T mode, the number of bits of the syntax element selected from the first set of modes being less than the number of bits of the syntax element selected from the second set of modes, and the number of bits of the syntax element selected from the second set of modes being less than the number of bits of the syntax element selected from the third set of modes; and performing intra-frame prediction on a current chroma block in the video signal based on the intra-frame chroma prediction mode.

[0058] According to a seventh aspect, the present invention relates to an apparatus for encoding video data. The apparatus includes a video data storage device and a video encoder. The video encoder is configured to: perform intra-frame prediction on chroma blocks in the video signal according to an intra-frame chroma prediction mode, wherein the intra-frame chroma prediction mode is selected from a first set of modes, a second set of modes, or a third set of modes, the second set of modes including at least one of a CCLM_L mode or a CCLM_T mode; and generate a bitstream of the video signal by carrying syntax elements indicating the intra-frame chroma prediction mode, wherein the number of bits of the syntax elements when selecting the intra-frame chroma prediction mode from the first set of modes is less than the number of bits of the syntax elements when selecting the intra-frame chroma prediction mode from the second set of modes, and the number of bits of the syntax elements when selecting the intra-frame chroma prediction mode from the second set of modes is less than the number of bits of the syntax elements when selecting the intra-frame chroma prediction mode from the third set of modes.

[0059] According to an eighth aspect, the present invention relates to an apparatus for decoding video data. The apparatus includes a video data storage and a video decoder. The video decoder is configured to: parse a plurality of syntax elements from a bitstream of a video signal; determine an intra-frame chroma prediction mode based on one of the plurality of syntax elements, wherein the intra-frame chroma prediction mode is determined from a first set of modes, a second set of modes, or a third set of modes, the second set of modes including at least one of a CCLM_L mode or a CCLM_T mode, wherein the number of bits of the syntax element selected from the first set of modes is less than the number of bits of the syntax element selected from the second set of modes, and the number of bits of the syntax element selected from the second set of modes is less than the number of bits of the syntax element selected from the third set of modes; and perform intra-frame prediction on a current chroma block in the video signal based on the intra-frame chroma prediction mode.

[0060] The provided apparatus for encoding and decoding video data enables CCLM_L and CCLM_T to be represented using binary strings, carried in the bitstream of the video signal at the encoding device side, and decoded at the decoding device side. According to the seventh and eighth aspects, in one possible implementation of the apparatus, the first set of modes includes at least one of a derived mode (DM) or a cross-component linear model (CCLM) prediction mode, and the third set of modes includes at least one of a vertical mode, a horizontal mode, a DC mode, or a planar mode.

[0061] According to the seventh and eighth aspects or any of the above-described implementations of the seventh and eighth aspects, in one possible implementation of the device, the syntax element of the DM mode is 0; the syntax element of the CCLM mode is 10; the syntax element of the CCLM_L mode is 1110; the syntax element of the CCLM_T mode is 1111; the syntax element of the planar mode is 11000; the syntax element of the vertical mode is 11001; the syntax element of the horizontal mode is 11010; and the syntax element of the DC mode is 11011.

[0062] According to the seventh and eighth aspects or any of the above-described implementations of the seventh and eighth aspects, in one possible implementation of the method, the syntax element of the DM mode is 00; the syntax element of the CCLM mode is 10; the syntax element of the CCLM_L mode is 110; the syntax element of the CCLM_T mode is 111; the syntax element of the planar mode is 0100; the syntax element of the vertical mode is 0101; the syntax element of the horizontal mode is 0110; and the syntax element of the DC mode is 0111.

[0063] According to a ninth aspect, the present invention relates to a method for performing intra-frame prediction using a cross-component linear model (CCLM). The method includes: determining a luminance block corresponding to a current chroma block; obtaining luminance reference pixels of the luminance block by downsampling neighboring pixels of the luminance block, wherein the luminance reference pixels include only luminance reference pixels obtained based on neighboring pixels above the luminance block or only luminance reference pixels obtained based on neighboring pixels to the left of the luminance block; determining a maximum luminance value and a minimum luminance value based on the luminance reference pixels; obtaining a first chroma value at least partially based on one or more positions of one or more luminance reference pixels associated with the maximum luminance value; obtaining a second chroma value at least partially based on one or more positions of one or more luminance reference pixels associated with the minimum luminance value; calculating linear model coefficients based on the first chroma value, the second chroma value, the maximum luminance value, and the minimum luminance value; and generating a predicted value for the current chroma block based on the linear model coefficients and the downsampled luminance block value of the luminance block.

[0064] According to the ninth aspect, in one possible implementation of the method, the number of luminance reference pixels is greater than or equal to the width of the current chroma block or greater than or equal to the height of the current chroma block.

[0065] According to the ninth aspect or any of the above-described implementations of the ninth aspect, in one possible implementation of the method, the luminance reference pixel used in determining the maximum luminance value and the minimum luminance value is the available luminance reference pixel of the luminance block.

[0066] According to the ninth aspect or any of the above-described implementations of the ninth aspect, in one possible implementation of the method, the available luminance reference pixel of the luminance block is determined based on the available chroma reference pixel of the current chroma block.

[0067] According to the ninth aspect or any of the above implementations of the ninth aspect, in one possible implementation of the method, when the luminance reference pixel is obtained only from the adjacent pixel above the luminance block, a maximum of 2×W luminance reference pixels are used to derive the linear model coefficients, where W represents the width of the current chroma block.

[0068] According to the ninth aspect or any of the above implementations of the ninth aspect, in one possible implementation of the method, when the luminance reference pixel is obtained only from the adjacent pixel on the left side of the luminance block, a maximum of 2×H luminance reference pixels are used to derive the linear model coefficients, where H represents the height of the current chroma block.

[0069] According to the ninth aspect or any of the above implementations of the ninth aspect, in one possible implementation of the method, at most N available luminance reference pixels are used to derive the linear model coefficients, where N is the sum of W and H, W represents the width of the current chroma block, and H represents the height of the current chroma block.

[0070] According to any of the above-described implementations of the ninth aspect or the first aspect, in one possible implementation of the method, the linear model coefficients α and β are calculated according to the following equation:

[0071]

[0072] β=y A -αx A

[0073] Where, x B This represents the maximum brightness value, y B x represents the first chromaticity value. A This represents the minimum brightness value, y A This represents the second chromaticity value.

[0074] According to the ninth aspect or any of the above-described implementations of the ninth aspect, in one possible implementation of the method, the predicted value of the current chroma block is obtained according to the following equation:

[0075] pred C (i,j)=α·rec L ′ (i,j)+β,

[0076] Among them, pred C (i,j) represents the predicted value of the chroma pixel in the current chroma block, rec′ L(i,j) represents the pixel value of the corresponding luminance pixel in the downsampled reconstructed luminance block of the luminance block.

[0077] According to the ninth aspect or any of the above-described implementations of the ninth aspect, in one possible implementation of the method, the downsampled luminance block of the luminance block is obtained by downsampling the reconstructed luminance block of the luminance block corresponding to the current chroma block.

[0078] According to the ninth aspect or any of the above implementations of the ninth aspect, in one possible implementation of the method, when the luminance reference pixel is obtained only from the adjacent pixels above the luminance block, and when the top row of the current chroma block overlaps with the top row of the current coding tree unit (CTU) or the current chroma block is located above the upper boundary of the current coding tree unit (CTU), the luminance reference pixel is obtained by using only one row of adjacent reconstructed luminance pixels of the reconstructed luminance block of the luminance block.

[0079] According to the ninth aspect or any of the above implementations of the ninth aspect, in one possible implementation of the method, when the luminance reference pixel is obtained only from the adjacent pixels to the left of the luminance block and the current chroma block is located at the left boundary of the current coding tree unit (CTU), the luminance reference pixel is obtained by using only a column of adjacent reconstructed luminance pixels of the reconstructed luminance block.

[0080] According to the ninth aspect or any of the above-described implementations of the ninth aspect, in one possible implementation of the method, the CCLM includes a multi-directional linear model (MDLM).

[0081] According to a tenth aspect, the present invention relates to an encoder. The encoder is used to perform a method according to the ninth aspect or any of the above-described implementations of the ninth aspect.

[0082] According to an eleventh aspect, the present invention relates to a decoder. The decoder is used to perform a method according to the ninth aspect or any of the above-described implementations of the ninth aspect.

[0083] Eleventhly, the present invention relates to an intra-frame prediction method using cross-component liner prediction mode (CCLM). The method includes:

[0084] Obtain the reference pixel of the current brightness block, wherein the reference pixel belongs only to the template above the current brightness block;

[0085] Based on the reference pixel, the maximum and minimum brightness values ​​are obtained;

[0086] The first chromaticity value is obtained based on the pixel position of the pixel with the maximum brightness value;

[0087] The second chromaticity value is obtained based on the pixel position of the minimum brightness value;

[0088] Calculate the linear model coefficients based on the first chromaticity value, the second chromaticity value, the maximum luminance value, and the minimum luminance value;

[0089] Based on the linear model coefficients, the predicted value of the current chroma block is obtained, wherein the current chroma block corresponds to the current luminance block.

[0090] According to the eleventh aspect, in one possible implementation of the method, the number of reference pixels is greater than or equal to the width of the current chroma block.

[0091] According to the eleventh aspect or any of the above implementations of the eleventh aspect, in one possible implementation of the method, the reference pixel point is available.

[0092] According to the eleventh aspect or any of the above implementations of the eleventh aspect, in one possible implementation of the method, the method further includes:

[0093] The availability of the reference pixel is checked within a range, wherein the length of the range is 2×W or the length of the range is the sum of W and H, where W represents the width of the current chroma block and H represents the height of the current chroma block.

[0094] According to the eleventh aspect or any of the foregoing implementations of the eleventh aspect, in a possible implementation of the method, up to 2×W available reference pixels are used to derive the linear model coefficients, where W represents the width of the current chroma block.

[0095] According to the eleventh aspect or any of the above implementations of the eleventh aspect, in one possible implementation of the method, at most N available reference pixels are used to derive the linear model coefficients, where N is the sum of W and H, W represents the width of the current chroma block, and H represents the height of the current chroma block.

[0096] According to the eleventh aspect or any of the above-described implementations of the eleventh aspect, in one possible implementation of the method, the linear model coefficients α and β are calculated according to the following equation:

[0097]

[0098] β=y A -αx A

[0099] Where, x B This represents the maximum brightness value, y B x represents the first chromaticity value. A This represents the minimum brightness value, y A This represents the second chromaticity value.

[0100] According to the eleventh aspect or any of the above implementations of the eleventh aspect, in one possible implementation of the method, the predicted value of the current chroma block is obtained according to the following equation:

[0101] pred C (i,j)=α·rec L ′(i,j)+β

[0102] Among them, pred C (i,j) represents the chromaticity pixel, rec L (i,j) represents the corresponding reconstructed brightness pixel.

[0103] According to the eleventh aspect or any of the above implementations of the eleventh aspect, in one possible implementation of the method, the number of reference pixels is greater than or equal to the size of the current chroma block.

[0104] According to the eleventh aspect or any of the above implementations of the eleventh aspect, in one possible implementation of the method, the reference pixel is a downsampled luminance pixel.

[0105] According to the eleventh aspect or any of the above implementations of the eleventh aspect, in one possible implementation of the method, when the current chroma block is located at the upper boundary, the reference pixel is obtained by using only one row of adjacent reconstructed luminance pixels.

[0106] According to the eleventh aspect or any of the above implementations of the eleventh aspect, in one possible implementation of the method, the CCLM is a multi-directional linear model (MDLM), and the MDLM is obtained using the coefficients of the linear model.

[0107] According to the eleventh aspect or any of the above implementations of the eleventh aspect, in one possible implementation of the method, the method is called CCIP_T.

[0108] In a twelfth aspect, the present invention relates to an intra-frame prediction method using cross-component liner prediction mode (CCLM). The method includes:

[0109] Obtain the reference pixel of the current brightness block, wherein the reference pixel belongs only to the left template of the current brightness block;

[0110] Based on the reference pixel, the maximum and minimum brightness values ​​are obtained;

[0111] The first chromaticity value is obtained based on the pixel position of the pixel with the maximum brightness value;

[0112] The second chromaticity value is obtained based on the pixel position of the minimum brightness value;

[0113] Calculate the linear model coefficients based on the first chromaticity value, the second chromaticity value, the maximum luminance value, and the minimum luminance value;

[0114] Based on the linear model coefficients, the predicted value of the current chroma block is obtained, wherein the current chroma block corresponds to the current luminance block.

[0115] According to the twelfth aspect, in one possible implementation of the method, the number of reference pixels is greater than or equal to the height of the current chroma block.

[0116] According to the twelfth aspect or any of the above-described implementations of the twelfth aspect, in one possible implementation of the method, the reference pixel point is available.

[0117] According to the twelfth aspect or any of the above implementations of the twelfth aspect, in one possible implementation of the method, the method further includes:

[0118] The availability of the reference pixel is checked within a range, wherein the length of the range is 2×H or the length of the range is the sum of W and H, where W represents the width of the current chroma block and H represents the height of the current chroma block.

[0119] According to the twelfth aspect or any of the above implementations of the twelfth aspect, in one possible implementation of the method, up to 2×H available reference pixels are used to derive the linear model coefficients, where H represents the height of the current chroma block.

[0120] According to the twelfth aspect or any of the above implementations of the twelfth aspect, in one possible implementation of the method, at most N available reference pixels are used to derive the linear model coefficients, where N is the sum of W and H, W represents the width of the current chroma block, and H represents the height of the current chroma block.

[0121] According to the twelfth aspect or any of the above-described implementations of the twelfth aspect, in one possible implementation of the method, the linear model coefficients α and β are calculated according to the following equation:

[0122]

[0123] β=y A -αx A

[0124] Where, x B This represents the maximum brightness value, y B x represents the first chromaticity value. A This represents the minimum brightness value, y A This represents the second chromaticity value.

[0125] According to the twelfth aspect or any of the above-described implementations of the twelfth aspect, in one possible implementation of the method, the predicted value of the current chroma block is obtained according to the following equation:

[0126] pred C (i,j)=α·rec L ′(i,j)+β

[0127] Among them, pred C (i,j) represents the chromaticity pixel, rec L (i,j) represents the corresponding reconstructed brightness pixel.

[0128] According to the eleventh aspect or any of the above implementations of the eleventh aspect, in one possible implementation of the method, the number of reference pixels is greater than or equal to the size of the current chroma block.

[0129] According to the eleventh aspect or any of the above implementations of the eleventh aspect, in one possible implementation of the method, the reference pixel is a downsampled luminance pixel.

[0130] According to the twelfth aspect or any of the above implementations of the twelfth aspect, in one possible implementation of the method, when the current block of the current chroma block is located at the left boundary, the reference pixel is obtained using only one column of adjacent reconstructed luminance pixels.

[0131] According to the twelfth aspect or any of the above implementations of the twelfth aspect, in one possible implementation of the method, the CCLM is a multi-directional linear model (MDLM), and the MDLM is obtained using the coefficients of the linear model.

[0132] According to the twelfth aspect or any of the above-described implementations of the twelfth aspect, in one possible implementation of the method, the method is referred to as CCIP_L.

[0133] In a thirteenth aspect, the present invention relates to an intra-frame prediction method using cross-component liner prediction mode (CCLM). The method includes:

[0134] Obtain a reference pixel for the current brightness block, wherein the reference pixel belongs only to the upper template of the current brightness block or only to the left template of the current brightness block;

[0135] Obtain the chroma pixels in the current chroma block, wherein the current chroma block corresponds to the current luminance block;

[0136] Calculate the linear model coefficients based on the reference pixel and the chromaticity pixel;

[0137] The predicted value of the current chroma block is obtained based on the linear model coefficients.

[0138] According to the thirteenth aspect, in one possible implementation of the method, up to N reference pixels are used to derive the linear model coefficients, where N is the sum of W and H, W represents the width of the current chroma block, and H represents the height of the current chroma block.

[0139] According to the thirteenth aspect or any of the above implementations of the thirteenth aspect, in one possible implementation of the method, when the reference pixel belongs only to the upper template of the current luminance block, the number of the reference pixels is greater than or equal to the width of the current chroma block.

[0140] According to the thirteenth aspect or any of the above implementations of the thirteenth aspect, in one possible implementation of the method, when the reference pixel belongs only to the template above the current luminance block, a maximum of 2×W reference pixels are used to derive the linear model coefficients, where W represents the width of the current chroma block.

[0141] According to the thirteenth aspect or any of the above implementations of the thirteenth aspect, in one possible implementation of the method, when the reference pixel belongs only to the left template of the current luminance block, the number of the reference pixels is greater than or equal to the height of the current luminance block.

[0142] According to the thirteenth aspect or any of the above implementations of the thirteenth aspect, in one possible implementation of the method, when the reference pixel belongs only to the left template of the current luminance block, a maximum of 2×H reference pixels are used to derive the linear model coefficients, where W represents the height of the current chroma block.

[0143] According to the thirteenth aspect or any of the above-described implementations of the thirteenth aspect, in one possible implementation of the method, the number of reference pixels is greater than or equal to the size of the current chroma block.

[0144] According to the thirteenth aspect or any of the above-described implementations of the thirteenth aspect, in one possible implementation of the method, the reference pixel is a downsampled luminance pixel.

[0145] According to the thirteenth aspect or any of the above implementations of the thirteenth aspect, in one possible implementation of the method, when the reference pixel belongs only to the upper template of the current luminance block and the current block of the current chroma block is located at the upper boundary, the reference pixel is obtained by using only one row of adjacent reconstructed luminance pixels.

[0146] According to the thirteenth aspect or any of the foregoing implementations of the thirteenth aspect, in a possible implementation of the method, when the reference pixel belongs only to the current luminance block and the current block of the current chroma block is located at the left boundary, the reference pixel is obtained by using only one column of adjacent reconstructed luminance pixels.

[0147] According to the thirteenth aspect or any of the above-described implementations of the thirteenth aspect, in one possible implementation of the method, the CCLM is a multi-directional linear model (MDLM), and the MDLM is obtained using the coefficients of the linear model.

[0148] According to the thirteenth aspect or any of the above-described implementations of the thirteenth aspect, in one possible implementation of the method, the reference pixel point is available.

[0149] According to a fourteenth aspect, the present invention relates to a decoder. The decoder includes processing circuitry for performing the method according to the eleventh aspect or any of the above-described implementations of the eleventh aspect.

[0150] According to a fifteenth aspect, the present invention relates to a decoder. The decoder includes processing circuitry for performing the method according to any of the above-described implementations of the twelfth aspect or the twelfth aspect.

[0151] According to a sixteenth aspect, the present invention relates to a decoder. The decoder includes processing circuitry for performing the method according to the thirteenth aspect or any of the above-described implementations of the thirteenth aspect.

[0152] In a seventeenth aspect, the present invention relates to an intra-frame prediction method using cross-component liner prediction mode (CCLM). The method includes:

[0153] Obtain the reference pixel of the current brightness block, wherein the reference pixel belongs only to the template above the current brightness block;

[0154] Based on the reference pixel, the maximum and minimum brightness values ​​are obtained;

[0155] Based on the maximum brightness value and the minimum brightness value, a first chromaticity value and a second chromaticity value are obtained;

[0156] Calculate the linear model coefficients based on the first chromaticity value, the second chromaticity value, the maximum luminance value, and the minimum luminance value;

[0157] The predicted value of the current block is obtained based on the linear model coefficients.

[0158] According to the seventeenth aspect, in one possible implementation of the method, the number of reference pixels is greater than or equal to the width of the current chroma block.

[0159] According to the seventeenth aspect or any of the above-described implementations of the seventeenth aspect, in one possible implementation of the method, the reference pixel point is available.

[0160] According to the seventeenth aspect or any of the above implementations of the seventeenth aspect, in one possible implementation of the method, a maximum of 2×W reference pixels are used to derive the model coefficients.

[0161] According to the seventeenth aspect or any of the above-described implementations of the seventeenth aspect, in one possible implementation of the method, the method is referred to as CCIP_T.

[0162] In an eighteenth aspect, the present invention relates to an intra-frame prediction method using cross-component liner prediction mode (CCLM). The method includes:

[0163] Obtain the reference pixel of the current brightness block, wherein the reference pixel belongs only to the left template of the current brightness block;

[0164] Based on the reference pixel, the maximum and minimum brightness values ​​are obtained;

[0165] Based on the maximum brightness value and the minimum brightness value, a first chromaticity value and a second chromaticity value are obtained;

[0166] Calculate the linear model coefficients based on the first chromaticity value, the second chromaticity value, the maximum luminance value, and the minimum luminance value;

[0167] The predicted value of the current block is obtained based on the linear model coefficients.

[0168] According to the eighteenth aspect, in one possible implementation of the method, the number of reference pixels is greater than or equal to the height of the current chroma block.

[0169] According to the eighteenth aspect or any of the above-described implementations of the eighteenth aspect, in one possible implementation of the method, the reference pixel point is available.

[0170] According to the eighteenth aspect or any of the above-described implementations of the eighteenth aspect, in one possible implementation of the method, a maximum of 2×H reference pixels are used to derive the model coefficients.

[0171] According to the eighteenth aspect or any of the above-described implementations of the eighteenth aspect, in one possible implementation of the method, the method is referred to as CCIP_L.

[0172] According to the nineteenth aspect, the present invention relates to a decoder for performing the method described in accordance with the seventeenth aspect or any of the above-described implementations of the seventeenth aspect.

[0173] According to the twentieth aspect, the present invention relates to a decoder for performing the method described in accordance with the eighteenth aspect or any of the above-described implementations of the eighteenth aspect.

[0174] According to a twenty-first aspect, the present invention relates to a method for intra-frame prediction using a linear model. The method includes: obtaining reference pixels for a current luminance block; obtaining a maximum luminance value and a minimum luminance value based on the reference pixels; obtaining a first chromaticity value and a second chromaticity value based on the positions of the luminance pixels having the maximum luminance value and the minimum luminance value; calculating linear model coefficients based on the first chromaticity value, the second chromaticity value, the maximum luminance value, and the minimum luminance value; and obtaining a predicted value for the current block based on the linear model coefficients, wherein obtaining the reference pixels for the current luminance block includes: determining L available chromaticity template pixels for the current luminance block, wherein the reference pixels for the current luminance block are L luminance template pixels corresponding to the L available chromaticity template pixels, or determining L available adjacent chromaticity pixels for the current luminance block, wherein the reference pixels for the current luminance block are L adjacent luminance pixels corresponding to the L available adjacent chromaticity pixels, where L ≥ 1 and is a positive integer.

[0175] According to the twenty-first aspect, in one possible implementation of the method, the step of determining the L available chroma template pixels of the current chroma block includes:

[0176] Check the availability of the adjacent chroma pixels above the current chroma block;

[0177] If L adjacent chroma pixels above are available, then the reference pixels of the current luminance block are the L adjacent luminance pixels corresponding to the L adjacent chroma pixels above, where L≥1, L≤W2, W2 represents the range of the upper template pixels, and L and W2 are positive integers.

[0178] According to any of the above implementations of the twenty-first aspect or the twenty-first aspect, in one possible implementation of the method, the step of determining the L available chroma template pixels of the current chroma block includes:

[0179] Check the availability of the left-side adjacent chroma pixels of the current chroma block;

[0180] If L adjacent left-side chroma pixels are available, then the reference pixels of the current luminance block are the L adjacent luminance pixels corresponding to the L adjacent left-side chroma pixels, where L≥1, L≤H2, H2 represents the range of left-side template pixels, and L and H2 are positive integers.

[0181] According to any of the above implementations of the twenty-first aspect or the twenty-first aspect, in one possible implementation of the method, the step of determining the L available chroma template pixels of the current chroma block includes:

[0182] Check the availability of the adjacent chroma pixels above the current chroma block and the adjacent chroma pixels to the left of the current chroma block;

[0183] If L1 adjacent upper chroma pixels and L2 adjacent left chroma pixels are available, then the reference pixels of the current luminance block include or are the L1 adjacent luminance pixels corresponding to the L1 adjacent upper chroma pixels and the L2 adjacent luminance pixels corresponding to the L2 adjacent luminance pixels, where L2≥1, L2≤H2, H2 represents the range of left template pixels, and L and H2 are positive integers; L1≥1, L1≤W2, W2 represents the range of upper template pixels, L1 and W2 are positive integers, and L=L1+L2.

[0184] According to the 21st aspect or any of the above implementations of the 21st aspect, in one possible implementation of the method, if L adjacent chroma pixels are available within the template range, then the model coefficients are obtained using L template luminance pixels and L template chroma pixels.

[0185] According to the second eleventh aspect or any of the above-described implementations of the second eleventh aspect, in one possible implementation of the method, the reference pixel point is available.

[0186] According to the second eleventh aspect or any of the above-described implementations of the second eleventh aspect, in one possible implementation of the method, the linear model coefficients α and β are calculated according to the following equation:

[0187]

[0188] β=y A -αx A

[0189] Where, x B This represents the maximum brightness value, y B This represents the first chromaticity value, x. A This represents the minimum brightness value, y A This represents the second chromaticity value.

[0190] According to the second-first aspect or any of the above-described implementations of the second-first aspect, in one possible implementation of the method, the predicted value of the current chroma block is obtained according to the following equation:

[0191] pred C (i,j)=α·rec L ′(i,j)+β

[0192] Among them, pred C(i,j) represents the chromaticity pixel, rec L (i,j) represents the corresponding reconstructed brightness pixel.

[0193] According to the second eleventh aspect or any of the above-described implementations of the second eleventh aspect, in one possible implementation of the method, the number of reference pixels is greater than or equal to the size of the current brightness block.

[0194] According to the second eleventh aspect or any of the above implementations of the second eleventh aspect, in one possible implementation of the method, the reference pixel is a downsampled luminance pixel.

[0195] According to the 21st aspect or any of the above implementations of the 21st aspect, in one possible implementation of the method, when the current block of the current chroma block is located at the upper boundary, the reference pixel is obtained by using only one row of adjacent reconstructed luminance pixels.

[0196] According to the second eleventh aspect or any of the above-described implementations of the second eleventh aspect, in one possible implementation of the method, the linear model is a multi-directional linear model (MDLM), and the MDLM is obtained using the coefficients of the linear model.

[0197] According to the 21st aspect or any of the above-described implementations of the 21st aspect, in one possible implementation of the method, the method is referred to as CCIP_T, or the method is referred to as CCIP_L.

[0198] According to the 21st aspect or any of the above implementations of the 21st aspect, in one possible implementation of the method, the reference pixel belongs only to the upper template of the current brightness block or only to the left template of the current brightness block, or the reference pixel belongs to both the upper template of the current brightness block and the left template of the current brightness block.

[0199] According to a twenty-second aspect, the present invention relates to a decoder. The decoder includes processing circuitry for performing the method described according to the twenty-first aspect or any of the above-described implementations of the twenty-first aspect.

[0200] According to a twenty-third aspect, the present invention relates to an encoder. The encoder includes processing circuitry for performing the method described according to the twenty-first aspect or any of the above-described implementations of the twenty-first aspect.

[0201] In a twenty-fourth aspect, the present invention relates to a binarization method for chroma modes. The method includes:

[0202] Intra-frame prediction is performed using a linear model (e.g., a multi-directional linear model, MDLM).

[0203] Generate a bitstream comprising multiple syntax elements, wherein the multiple syntax elements indicate or include CCLM mode, CCIP_L mode or CCIP_T mode.

[0204] According to the twenty-four aspects, in one possible implementation of the method...

[0205] The first indicator (77) indicates CCLM mode, with an index of 4 for intra_chroma_pred_mode;

[0206] The second indicator (78) indicates the CCIP_L mode, with the index intra_chroma_pred_mode set to 5;

[0207] The third indicator (79) indicates the CCIP_T mode, with the index intra_chroma_pred_mode set to 6.

[0208] According to the twenty-fourth aspect or any of the above implementations of the twenty-fourth aspect, in one possible implementation of the method, when sps_cclm_enabled_flag is 1, IntraPredModeC[xCb][yCb] depends on intra_chroma_pred_mode[xCb][yCb] and IntraPredModeY[xCb][yCb].

[0209] According to a twenty-fifth aspect, the present invention relates to a decoding method implemented by a decoding device. The decoding method includes:

[0210] Parse multiple syntax elements from the bitstream, wherein the multiple syntax elements indicate or include CCLM mode, CCIP_L mode or CCIP_T mode;

[0211] Intra-frame prediction is performed using the linear model indicated.

[0212] According to the twenty-five aspects, in one possible implementation of the method...

[0213] The first indicator (77) indicates CCLM mode, with an index of 4 for intra_chroma_pred_mode;

[0214] The second indicator (78) indicates the CCIP_L mode, with the index intra_chroma_pred_mode set to 5;

[0215] The third indicator (79) indicates the CCIP_T mode, with the index intra_chroma_pred_mode set to 6.

[0216] According to the 25th aspect or any of the above implementations of the 25th aspect, in one possible implementation of the method, when sps_cclm_enabled_flag is 1, IntraPredModeC[xCb][yCb] depends on intra_chroma_pred_mode[xCb][yCb] and IntraPredModeY[xCb][yCb].

[0217] According to a twenty-sixth aspect, the present invention relates to a decoder. The decoder includes processing circuitry for performing the method described according to or any of the above-described implementations of the twenty-fourth and twenty-fifth aspects.

[0218] According to a twenty-seventh aspect, the present invention relates to an encoder. The encoder includes processing circuitry for performing the method described according to or any of the above-described implementations of the twenty-fourth and twenty-fifth aspects.

[0219] According to a twenty-eighth aspect, the present invention relates to a computer-readable medium storing instructions. When the instructions are executed on a processor, the processor causes the processor to perform the method according to the twenty-fourth and twenty-fifth aspects or any of the above-described implementations of the twenty-fourth and twenty-fifth aspects.

[0220] According to a twenty-eighth aspect, the present invention relates to a decoder. The decoder comprises:

[0221] One or more processors;

[0222] A non-transitory computer-readable storage medium coupled to and storing a program executed by the one or more processors, wherein, when the processor executes the program, the decoder is configured to execute the method according to the second fourteenth and second fifteenth aspects or any of the above-described implementations of the second fourteenth and second fifteenth aspects.

[0223] According to a twenty-eighth aspect, the present invention relates to an encoder. The encoder comprises:

[0224] One or more processors;

[0225] A non-transitory computer-readable storage medium coupled to and storing a program executed by the one or more processors, wherein, when the processor executes the program, the encoder is configured to perform the method according to the second fourteenth and second fifteenth aspects or any of the above-described implementations of the second fourteenth and second fifteenth aspects.

[0226] In a twenty-ninth aspect, the present invention relates to an intra-frame prediction method using cross componentliner prediction mode (CCLM). The method includes:

[0227] Obtain the reference pixel of the current brightness block;

[0228] Based on the reference pixel, the maximum and minimum brightness values ​​are obtained;

[0229] Based on the maximum brightness value and the minimum brightness value, a first chromaticity value and a second chromaticity value are obtained;

[0230] Calculate the linear model coefficients based on the first chromaticity value, the second chromaticity value, the maximum luminance value, and the minimum luminance value;

[0231] Based on the coefficients of the linear model, the predicted value of the current block is obtained.

[0232] The availability of template pixels is determined by checking adjacent chroma pixels.

[0233] According to the thirtieth aspect, the present invention relates to a decoder for performing the method of the twenty-eighth aspect.

[0234] According to the thirty-first aspect, the present invention relates to a decoder for performing the method of the twenty-ninth aspect.

[0235] According to the thirty-second aspect, the present invention relates to a decoder for performing the method of the twenty-eighth or twenty-ninth aspect.

[0236] According to the thirty-third aspect, an apparatus is provided. The apparatus includes modules / units / components / circuits to perform at least a portion of the steps of the method according to any of the foregoing aspects or any of the foregoing implementations of any of the foregoing aspects.

[0237] The apparatus according to aspect thirty-three can be extended to an implementation corresponding to an implementation of the method according to any of the above aspects. Therefore, one implementation of the apparatus includes the features of a corresponding implementation of the method according to any of the above aspects.

[0238] The advantages of the apparatus according to any of the above aspects are the same as the advantages of the corresponding implementation of the method according to any of the above aspects.

[0239] For clarity, any of the above examples may be combined with any or more of the other examples to create new examples within the scope of this invention.

[0240] These and other features will become clearer from the following detailed description taken in conjunction with the accompanying drawings and claims. Attached Figure Description

[0241] To gain a more complete understanding of the invention, reference is now made to the following brief description taken in conjunction with the accompanying drawings and detailed description, wherein similar reference numerals denote similar parts.

[0242] Figure 1A This is a block diagram of an exemplary decoding system that can implement an example of the present invention.

[0243] Figure 1B This is a block diagram of another exemplary decoding system that can implement the embodiments of the present invention.

[0244] Figure 2 This is a block diagram of an exemplary video encoder that can implement an example of the present invention.

[0245] Figure 3 This is a block diagram of an example video decoder that can implement the embodiments of the present invention.

[0246] Figure 4 This is a schematic diagram of a video decoding device.

[0247] Figure 5 A simplified block diagram of a device 500 provided for an exemplary embodiment, which can be used as... Figure 1A The source device 12 and the destination device 14 are either or both of them.

[0248] Figure 6A A conceptual diagram showing the nominal vertical and horizontal relative positions of luminance and chrominance pixels.

[0249] Figure 6B This is a conceptual diagram of an example of luminance and chrominance locations. These luminance and chrominance locations are used to downsample pixels in a luminance block to generate a prediction block.

[0250] Figure 6C Here is a conceptual diagram of another example of luminance and chrominance locations, which are used to downsample pixels in a luminance block to generate a prediction block.

[0251] Figure 6D The intra-prediction mode in H.265 / HEVC is shown.

[0252] Figure 7The reference pixel point of the current block is shown.

[0253] Figure 8 This shows the positions of the left and top reference pixels of the current luma block and the current chroma block involved in CCLM mode.

[0254] Figure 9 The line between the minimum and maximum brightness values ​​is shown.

[0255] Figure 10 The template (including reference pixels) is shown for the chroma block and the downsampled luminance block.

[0256] Figure 11 An example of a template that includes unavailable reference pixels is shown.

[0257] Figure 12 The reference pixel point used in CCLM_T mode is shown.

[0258] Figure 13 The reference pixel points used in CCLM_L mode are shown.

[0259] Figure 14 This is an illustration of an example of using an upper template or a left module to determine model coefficients, where the upper template is larger than the width of the downsampled luminance block of the current luminance block, the left template is larger than the height of the downsampled luminance block of the current luminance block, the upper template includes reference pixels located above the downsampled luminance block, and the left template includes reference pixels located to the left of the downsampled luminance block.

[0260] Figure 15 This is an illustration of an example of using the top template or the left module to determine model coefficients, where the size of the top template is the same as the width of the downsampled luminance block of the current luminance block, and the size of the left template is the same as the height of the downsampled luminance block of the current luminance block.

[0261] Figure 16 This is a schematic diagram of an example of using available reference pixels to determine model coefficients for intra-frame prediction.

[0262] Figure 17 This is a schematic diagram illustrating an example of downsampling luminance pixels using multiple rows or columns of adjacent luminance pixels.

[0263] Figure 18 This is a schematic diagram of another example of downsampling luminance pixels using multiple rows or columns of adjacent luminance pixels.

[0264] Figure 19This is a schematic diagram illustrating an example of downsampling using a single row of adjacent luminance pixels from a luminance block located on the upper boundary of the CTU.

[0265] Figure 20 A flowchart illustrating a method for performing intra-frame prediction using a linear model, provided for some aspects of the present invention.

[0266] Figure 21 A flowchart illustrating a method for performing intra-frame prediction using a linear model, provided for other aspects of the present invention.

[0267] Figure 22 A block diagram of an exemplary structure for an apparatus that performs intra-frame prediction using a linear model.

[0268] Figure 23 A flowchart illustrating a method for encoding chroma intra-frame decoding modes into a video signal bitstream, provided for some aspects of the present invention.

[0269] Figure 24 A flowchart illustrating a method for decoding chroma intra-frame decoding modes in a video signal bitstream, provided for some aspects of the present invention.

[0270] Figure 25 A block diagram of an exemplary structure of an apparatus for generating video streams.

[0271] Figure 26 This is a block diagram of an exemplary structure of an apparatus for decoding a video stream.

[0272] Figure 27 A block diagram of an exemplary structure for a content supply system that provides content distribution services.

[0273] Figure 28 A block diagram of an example structure for a terminal device. Detailed Implementation

[0274] First, it should be understood that although one or more illustrative implementations are provided below, the disclosed systems and / or methods can be implemented using any number of techniques, whether currently known or existing. The invention should not in any way be limited to the illustrative implementations, drawings, and techniques described below, including the exemplary designs and implementations illustrated and described herein, but can be modified within the full scope of the appended claims and their equivalents.

[0275] Figure 1A This is a block diagram of an exemplary decoding system 10 that can employ bidirectional prediction techniques. (See diagram for example.) Figure 1AAs shown, the decoding system 10 includes a source device 12, which provides encoded video data that is later decoded by a destination device 14. Specifically, the source device 12 can provide video data to the destination device 14 via a computer-readable medium 16. The source device 12 and the destination device 14 can include any of a variety of devices, including desktop computers, laptops, tablets, set-top boxes, mobile phones such as smartphones and tablets, televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some cases, the source device 12 and the destination device 14 can be used for wireless communication.

[0276] Destination device 14 can receive encoded video data to be decoded via computer-readable medium 16. Computer-readable medium 16 can include any type of medium or device capable of transmitting encoded video data from source device 12 to destination device 14. In one example, computer-readable medium 16 can include a communication medium enabling source device 12 to transmit encoded video data directly to destination device 14 in real time. The encoded video data can be modulated according to communication standards such as wireless communication protocols and transmitted to destination device 14. The communication medium can include any wireless or wired communication medium, such as radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium can form part of a packet network (e.g., a local area network, a wide area network, or a global network such as the Internet). The communication medium can include a router, switch, base station, or any other device that can be used for communication from source device 12 to destination device 14.

[0277] In some examples, encoded data can be output from output interface 22 to a storage device. Similarly, encoded data can be accessed from a storage device via an input interface. The storage device can include any of a variety of distributed or locally accessible data storage media, such as hard disk drives, Blu-ray discs, digital video discs (DVDs), compact disc read-only memory (CD-ROMs), flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In yet another example, the storage device can correspond to a file server or another intermediate storage device that can store the encoded video generated by source device 12. Destination device 14 can access the stored video data from the storage device via streaming or downloading. The file server can be any type of server capable of storing and sending encoded video data to destination device 14. Exemplary file servers include World Wide Web servers (e.g., for websites), file transfer protocol (FTP) servers, network attached storage (NAS) devices, or local disk drives. Destination device 14 can access encoded video data via any standard data connection, including an Internet connection. This standard data connection may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., a digital subscriber line (DSL), a cable modem, etc.), or a combination of both suitable for accessing encoded video data stored on a file server. Transfer of encoded video data from the storage device can be streaming, downloading, or a combination thereof.

[0278] The techniques in this invention are not necessarily limited to wireless applications or setups. These techniques can be applied to video decoding to support any of a variety of multimedia applications, such as over-the-air television broadcasting, cable television transmission, satellite television transmission, internet streaming video transmission (e.g., dynamic adaptive streaming over HTTP (DASH)), encoding digital video into data storage media, decoding digital video stored on data storage media, or other applications. In some examples, the decoding system 10 can be used to support one-way or two-way video transmission to support applications such as video streaming, video playback, video broadcasting, and / or video telephony.

[0279] exist Figure 1AIn one example, source device 12 includes a video source 18, a video encoder 20, and an output interface 22. Destination device 14 includes an input interface 28, a video decoder 30, and a display device 32. According to the invention, the video encoder 20 in source device 12 and / or the video decoder 30 in destination device 14 may use bidirectional prediction technology. In other examples, the source and destination devices may include other components or devices. For example, source device 12 may receive video data from an external video source (e.g., an external camera). Similarly, destination device 14 may be connected to an external display device, rather than including an integrated display device.

[0280] Figure 1A The decoding system 10 shown is merely an example. Bidirectional prediction techniques can be performed by any digital video encoding and / or decoding device. Although the techniques in this invention are typically performed by video encoding devices, they can also be performed by video encoders / decoders (often referred to as "codecs"). Furthermore, the techniques in this invention can also be performed by a video preprocessor. The video encoder and / or decoder can be a graphics processing unit (GPU) or a similar device.

[0281] Source device 12 and destination device 14 are merely examples of such decoding devices, where source device 12 generates decoded video data to be sent to destination device 14. In some examples, source device 12 and destination device 14 can operate substantially symmetrically, such that both source device 12 and destination device 14 include video encoding and decoding components. Therefore, decoding system 10 can support one-way or two-way video transmission between video devices 12 and 14, for example, for video streaming, video playback, video broadcasting, or video telephony.

[0282] The video source 18 in source device 12 may include a video capture device (e.g., a camera), a video archive containing previously captured video, and / or a video input interface for receiving video from a video content provider. Alternatively, the video source 18 may generate computer graphics-based data as source video, or as a combination of live video, archived video, and computer-generated video.

[0283] In some cases, when the video source 18 is a camera, the source device 12 and the destination device 14 can form a camera phone or video phone. However, as described above, the techniques described in this invention are generally applicable to video decoding and can be applied to wireless and / or wired applications. In each case, the captured, pre-captured, or computer-generated video can be encoded by the video encoder 20. The encoded video information can then be output to the computer-readable medium 16 via the output interface 22.

[0284] Computer-readable medium 16 may include transient media such as wireless broadcasting or wired network transmissions, and may also include storage media (i.e., non-transient storage media), such as hard disks, flash drives, optical discs, digital video discs, Blu-ray discs, or other computer-readable media. In some examples, a network server (not shown) may receive encoded video data from source device 12 and provide the encoded video data to destination device 14, for example, via network transmission. Similarly, a computing device in a media production facility (e.g., an optical disc stamping facility) may receive encoded video data from source device 12 and produce an optical disc including the encoded video data. Therefore, in various examples, computer-readable medium 16 can be understood to include one or more computer-readable media of various forms.

[0285] The input interface 28 in the destination device 14 receives information from the computer-readable medium 16. The information received by the computer-readable medium 16 may include syntax information defined by the video encoder 20, which is also used by the video decoder 30, including grammatical elements describing the features and / or processing methods of descriptive blocks and other decoded units (e.g., groups of pictures, GOPs). The display device 32 displays the decoded video data to the user and may include any of a variety of display devices, such as a cathode ray tube (CRT), liquid crystal display (LCD), plasma display, organic light emitting diode (OLED) display, or other types of display devices.

[0286] The video encoder 20 and video decoder 30 can operate according to video decoding standards (such as the Efficiency Video Coding (HEVC) standard currently under development) and can comply with the HEVC Test Model (HM). Alternatively, the video encoder 20 and video decoder 30 can operate according to other proprietary or industry standards, such as the International Telecommunication Union Telecommunication Standardization Sector (ITU-T) H.264 standard (or Motion Picture Expert Group (MPEG)-4 Part 10, Advanced Video Coding (AVC)), H.265 / HEVC, and extended versions of such standards. However, the technology in this invention is not limited to any particular decoding standard. Other examples of video decoding standards include MPEG-2 and ITU-T H.263. Although Figure 1A Not shown, but in some aspects, the video encoder 20 and video decoder 30 may be integrated with the audio encoder and audio decoder, respectively, and may include suitable multiplexer-demultiplexer (MUX-DEMUX) units or other hardware and software to handle the encoding of both audio and video in a common data stream or separate data streams. Where applicable, the MUX-DEMUX unit may comply with the ITU H.223 multiplexer protocol or other protocols such as the User Datagram Protocol (UDP).

[0287] Video encoder 20 and video decoder 30 can each be implemented as any of a variety of suitable encoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When the above techniques are partially implemented in software, a device can store software instructions in a suitable non-transitory computer-readable medium and use one or more processors to execute these instructions in hardware to perform the techniques of the present invention. Both video encoder 20 and video decoder 30 can be included in one or more encoders or decoders, either of which can be integrated into the respective device as part of a combined encoder / decoder (codec). A device including video encoder 20 and / or video decoder 30 can include integrated circuits, microprocessors, and / or wireless communication devices such as cellular phones.

[0288] Figure 1B A schematic diagram of an exemplary video decoding system 40 provided for an exemplary embodiment. The video decoding system 40 includes... Figure 2 encoder 200 and / or Figure 3 The decoder 300 is located in the system 40. The system 40 can implement the techniques in this application, such as fusion estimation in inter-frame prediction. In the illustrated implementation, the video decoding system 40 may include one or more imaging devices 41, a video encoder 200, a video decoder 300 (and / or a video decoder implemented by logic circuitry 47 in one or more processing units 46), an antenna 42, one or more processors 43, one or more memories 44, and / or a display device 45.

[0289] As shown in the figure, one or more imaging devices 41, antenna 42, one or more processing units 46, logic circuits 47, video encoder 200, video decoder 300, one or more processors 43, one or more memories 44, and / or display device 45 are capable of communicating with each other. As discussed, although the video decoding system 40 is shown to include a video encoder 200 and a video decoder 30, in various practical scenarios, the video decoding system 40 may include only the video encoder 200 or only the video decoder 300.

[0290] As shown in the figure, in some examples, the video decoding system 40 may include an antenna 42. For example, the antenna 42 may be used to transmit or receive encoded streams of video data. Furthermore, in some examples, the video decoding system 40 may include a display device 45. The display device 45 may be used to present the video data. As shown in the figure, in some examples, the logic circuitry 47 may be implemented by one or more processing units 46. The one or more processing units 46 may include application-specific integrated circuit (ASIC) logic, one or more graphics processors, or one or more general-purpose processors, etc. The video decoding system 40 may also include one or more optional processors 43, which may similarly include application-specific integrated circuit (ASIC) logic, one or more graphics processors, or one or more general-purpose processors, etc. In some examples, the logic circuitry 47 may be implemented by hardware or video decoding-specific hardware, and the one or more processors 43 may be implemented by general-purpose software or an operating system, etc. Additionally, one or more memories 44 can be any type of memory, such as volatile memory (e.g., Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), etc.) or non-volatile memory (e.g., flash memory), etc. In a non-limiting example, one or more memories 44 can be implemented using high-speed cache memory. In some examples, logic circuitry 47 can access one or more memories 44 (for implementing image buffers, etc.). In other examples, logic circuitry 47 and / or one or more processing units 46 may include memory (e.g., cache, etc.) for implementing image buffers, etc.

[0291] In some examples, the video encoder 200 implemented via logic circuitry may include an image buffer (e.g., implemented via one or more processing units 46 or one or more memories 44) and a graphics processing unit (e.g., implemented via one or more processing units 46). The graphics processing unit may be communicatively coupled to the image buffer. The graphics processing unit may include the video encoder 200 implemented via logic circuitry 47, as embodied in the reference... Figure 2 And / or any other encoder system or subsystem described herein, and the various modules discussed herein. Logic circuits may be used to perform the various operations discussed herein.

[0292] The video decoder 300 can be implemented in a similar manner via logic circuit 47 to reflect the reference. Figure 3The various modules discussed herein refer to the decoder 300 and / or any other decoder system or subsystem described herein. In some examples, the video decoder 300 implemented via logic circuitry may include (e.g., implemented via one or more processing units 46 or one or more memories 44) an image buffer and (e.g., implemented via one or more processing units 46) a graphics processing unit. The graphics processing unit may be communicatively coupled to the image buffer. The graphics processing unit may include the video decoder 300 implemented via logic circuitry 47, embodying the reference... Figure 3 And / or the various modules discussed in any other decoder system or subsystem described herein.

[0293] In some examples, antenna 42 in video decoding system 40 can be used to receive the encoded bitstream of video data. As discussed herein, the encoded bitstream may include data related to video frame encoding, indicators, index values, mode selection data, etc., such as data related to decoding segmentation (e.g., transform coefficients or quantization transform coefficients, optional indicators (as discussed), and / or data defining decoding segmentation). Video decoding system 40 may also include a video decoder 300 coupled to antenna 42 and used for decoding the encoded bitstream. Display device 45 is used to display video frames.

[0294] Figure 2 This is a block diagram of an exemplary video encoder 200 that can implement the technology of this application. The video encoder 200 can perform intra-frame decoding and inter-frame decoding on video blocks within a video stripe. Intra-frame decoding relies on spatial prediction to reduce or remove spatial redundancy in the video within a given video frame or image. Inter-frame decoding relies on temporal prediction to reduce or remove temporal redundancy in the video within adjacent frames or images of a video sequence. The intra-frame mode (I-mode) can be any of several spatially based decoding modes. The inter-frame mode (e.g., one-way prediction (P-mode) or two-way prediction (B-mode) can be any of several time-based decoding modes.

[0295] Figure 2 This is a schematic / conceptual block diagram of an exemplary video encoder 200 used to implement the technology in this invention. Figure 2In the example, the video encoder 200 includes a residual calculation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transform processing unit 212, a reconstruction unit 214, a buffer 216, a loop filter unit 220, a decoded picture buffer (DPB) 230, a prediction processing unit 260, and an entropy coding unit 270. The prediction processing unit 260 may include inter-frame estimation 242, inter-frame prediction unit 244, intra-frame estimation 252, intra-frame prediction unit 254, and a mode selection unit 262. The inter-frame prediction unit 244 may also include a motion compensation unit (not shown). Figure 2 The video encoder 200 shown can also be called a hybrid video encoder or a video encoder based on a hybrid video codec.

[0296] For example, the residual calculation unit 204, transform processing unit 206, quantization unit 208, prediction processing unit 260, and entropy coding unit 270 form the forward signal path of the encoder 200, while, for example, the inverse quantization unit 210, inverse transform processing unit 212, reconstruction unit 214, buffer 216, loop filter 220, decoded picture buffer (DPB) 230, and prediction processing unit 260 form the backward signal path of the encoder. The backward signal path of the encoder is connected to the decoder (see...). Figure 3 The signal path corresponds to the decoder 300 in the middle.

[0297] Encoder 200 is used to receive image 201 or image block 203 in image 201 via input terminal 202, etc. Image 201 may be an image in an image sequence that forms a video or video sequence. Image block 203 may also be referred to as the current image block or the image block to be decoded, and image 201 may be referred to as the current image or the image to be decoded (especially in video decoding when the current image is distinguished from other images (e.g., previously encoded and / or decoded images in the same video sequence (i.e., a video sequence that also includes the current image)).

[0298] segmentation

[0299] In an embodiment, encoder 200 may include a segmentation unit ( Figure 2 (Not shown in the image) is used to segment image 201 into multiple blocks (e.g., block 203, etc.), typically into multiple non-overlapping blocks. The segmentation unit can be used to apply the same block size and a corresponding grid defining the block size to all images in a video sequence, or to change the block size between images, subsets, or groups of images, segmenting each image into corresponding blocks.

[0300] In HEVC and other video decoding specifications, a set of coding tree units (CTUs) is generated to produce an encoded representation of an image. These CTUs can include a coding tree block composed of luma pixels (samples), two corresponding coding tree blocks composed of chroma pixels, and the syntax structure used to decode the pixels in these coding tree blocks. In a black and white image or an image containing three different color planes, a CTU can include a coding tree block and the syntax structure used to decode the pixels in that coding tree block. A coding tree block can be an N×N block of pixels. A CTU is also called a "tree block" or "largest coding unit (LCU)". In a broad sense, a CTU in HEVC can be similar to a macroblock in other standards such as H.264 / AVC. However, CTUs are not necessarily limited to a specific size and can include one or more coding units (CUs). A stripe can include an integer number of CTUs ordered consecutively in raster scan order.

[0301] In HEVC, a quadtree structure (represented as a decoding tree) is used to divide the CTU into CUs to accommodate different local features. At the CU level, a decision is made on whether to use inter-frame (temporal) prediction or intra-frame (spatial) prediction to decode the image region. A CU may include a decoding block consisting of luma pixels and two corresponding decoding blocks consisting of chroma pixels, as well as the syntax structure used to decode the pixels in these decoding blocks. The image includes an array of luma pixels, an array of Cb pixels, and an array of Cr pixels. In a black and white image or an image containing three different color planes, a CU may include a decoding block and the syntax structure used to decode the pixels in that decoding block. A decoding block is an N×N block of pixels. In some examples, the size of the CU is the same as the size of the CTU. Each CU uses a decoding mode for decoding, which may be, for example, an intra-frame decoding mode or an inter-frame decoding mode. Other decoding modes are also possible. Encoder 200 receives video data. Encoder 200 can encode each CTU into stripes of the video data image. As part of the encoding CTU, the prediction processing unit 260 or other processing units in the encoder 200 (including but not limited to) Figure 2 The unit in the encoder 200 shown can perform segmentation to divide the CTB in the CTU into progressively smaller blocks 203. These smaller blocks can be decoding blocks in the CU.

[0302] The size of a CTU can also be defined by the syntax data within the bitstream. A stripe consists of multiple consecutive CTUs arranged in decoding order. Video frames or images / pictures can be segmented into one or more stripes. As mentioned above, each tree block can be divided into coding units (CUs) according to a quadtree. Generally, a quadtree data structure includes one node per CU, where the root node corresponds to a tree block (e.g., a CTU). If a CU is divided into 4 sub-CUs, the node corresponding to that CU includes 4 child nodes, each child node corresponding to a sub-CU. Multiple nodes in a quadtree structure include leaf nodes and non-leaf nodes. Leaf nodes have no child nodes in the tree structure (i.e., leaf nodes are not further subdivided). Non-leaf nodes include the root node in the tree structure. Each non-root node in the multiple nodes corresponds to a sub-CU in the CU, which corresponds to the parent node in the tree structure where the non-root node resides. Each non-leaf node has one or more child nodes in the tree structure.

[0303] Each node in a quadtree data structure provides syntax data for its corresponding CU. For example, a node in a quadtree can include a partition flag indicating whether the CU corresponding to that node should be partitioned into subCUs. The syntax elements of a CU can be redefined and can be determined based on whether the CU is partitioned into subCUs. If a CU is not further partitioned, it is called a leaf CU. If a CU block is further partitioned, it is usually called a non-leaf CU. Each level of partitioning divides the quadtree into 4 subCUs. The black CU is an example of a leaf node (i.e., a block that is not further partitioned).

[0304] A CU serves a similar function to a macroblock in the H.264 standard, except that CUs are not size-distinct. For example, a tree block can be divided into four child nodes (also called child CUs), and each child node can also be a parent node and divided into another four child nodes. The final undivided child nodes (called leaf nodes of the quadtree) include the decoder nodes (also called leaf CUs). The syntax data associated with the decoded bitstream can define the maximum number of times the tree block can be divided, called the maximum CU depth, and can also define the minimum size of the decoder nodes. Correspondingly, the bitstream can also define the smallest coding unit (SCU). The term "block" is used to refer to any of the CUs, PUs, or TUs in the HEVC context or similar data structures in other standard contexts (such as macroblocks and their child blocks in H.264 / AVC).

[0305] In HEVC, each CU can be further divided into one, two, or four PUs based on the PU partitioning type. The same prediction process is performed within a PU, and relevant information is sent to the decoder on a PU-by-PU basis. After obtaining residual blocks through the prediction process based on the PU partitioning type, the CU can be partitioned into transform units (TUs) according to other quadtree structures similar to those used for the CU. One of the key features of the HEVC structure is the existence of multiple partitioning concepts such as CU, PU, ​​and TU. PUs can be partitioned into non-square shapes. The syntax data associated with a CU can also describe, for example, partitioning the CU into one or more PUs. TUs can be square or non-square (e.g., rectangular). The syntax data associated with a CU can describe, for example, partitioning the CU into one or more TUs according to a quadtree. The partitioning pattern may differ depending on whether the CU is encoded in skip mode or direct mode, intra-predictive mode, or inter-predictive mode.

[0306] However, Versatile Video Coding (VVC) does not distinguish between the concepts of PU and TU, and supports more CU segmentation shapes. The size of the CU corresponds to the size of the decoding node, and the shape can be square or non-square (e.g., rectangular). The size of the CU can range from 4×4 pixels (or 8×8 pixels) up to a maximum of 128×128 pixels or larger (e.g., 256×256 pixels) in block size.

[0307] After encoder 200 generates prediction blocks (e.g., luminance, Cb, and Cr prediction blocks) for the CU, encoder 200 can generate residual blocks for the CU. For example, encoder 100 can generate luminance residual blocks for the CU. Each pixel in the luminance residual block of the CU indicates the difference between a luminance pixel in the predicted luminance block of the CU and a corresponding pixel in the original luminance decoded block of the CU. Additionally, encoder 200 can generate Cb residual blocks for the CU. Each pixel in the Cb residual block of the CU can indicate the difference between a Cb pixel in the predicted Cb block of the CU and a corresponding pixel in the original Cb decoded block of the CU. Encoder 100 can also generate Cr residual blocks for the CU. Each pixel in the Cr residual block of the CU can indicate the difference between a Cr pixel in the predicted Cr block of the CU and a corresponding pixel in the original Cr decoded block of the CU.

[0308] In some examples, encoder 100 does not perform a transformation on the transform block. In such examples, encoder 200 may process the residual pixel values ​​in the same way as the transform coefficients. Therefore, in examples where encoder 100 does not perform a transformation, the following discussion of transform coefficients and coefficient blocks applies to transform blocks composed of residual pixels.

[0309] After generating coefficient blocks (e.g., luminance coefficient blocks, Cb coefficient blocks, or Cr coefficient blocks), encoder 200 can quantize the coefficient blocks to minimize the amount of data used to represent them, thereby enabling further compression if possible. Quantization generally refers to the process of compressing a series of values ​​into a single value. After quantizing the coefficient blocks, encoder 200 can entropy encode the syntax elements that indicate the quantized transform coefficients. For example, encoder 200 can perform context-adaptive binary arithmetic coding (CABAC) or other entropy decoding techniques on the syntax elements that indicate the quantized transform coefficients.

[0310] The encoder 200 can output a bitstream consisting of encoded image data 271, which includes bit sequences that constitute a representation of the decoded image and related data. Therefore, the bitstream includes an encoded representation of the video data.

[0311] In their September 2015 paper, "Block Partitioning Structure for Next Generation Video Coding," ITU-T COM16-C966 (hereinafter referred to as "VCEG Recommendation COM16-C966"), J. An et al. proposed using the quad-tree-binary-tree (QTBT) partitioning technique for future video decoding standards above HEVC. Simulation results show that the proposed QTBT structure is more efficient than the quad-tree structure used in HEVC. In HEVC, inter-frame prediction is restricted to small blocks to reduce memory access for motion compensation; thus, bidirectional prediction for 4×8 and 8×4 blocks is not supported, nor is inter-frame prediction for 4×4 blocks. JEM's QTBT does not have these limitations.

[0312] In QTBT, the CU (Code Unit) can be square or rectangular. For example, the coding tree unit (CTU) is first partitioned using a quadtree structure. The leaf nodes of the quadtree can then be further partitioned using a binary tree structure. There are two types of binary tree partitioning: symmetrical horizontal partitioning and symmetrical vertical partitioning. In each case, a node is partitioned horizontally or vertically from the middle. The leaf nodes of the binary tree are called coding units (CUs), and prediction and transformation processing is performed through this partitioning without any further division. That is, CUs, PUs, and TUs have the same block size in the QTBT decoding block structure. Sometimes, a CU includes coding blocks (CBs) composed of different color components; for example, in the case of P-stripes and B-stripes in a 4:2:0 chroma format, a CU includes one luma CB and two chroma CBs. Sometimes, a CU includes CBs composed of a single component; for example, in the case of I-stripes, a CU includes only one luma CB or only two chroma CBs.

[0313] The following parameters are defined for the QTBT segmentation scheme:

[0314] -CTU size: The size of the root node in a quadtree, the same concept as in HEVC.

[0315] -MinQTSize: Minimum allowed size of quadtree leaf nodes

[0316] -MaxBTSize: Maximum allowed size of the root node of the binary tree

[0317] -MaxBTDepth: Maximum allowed binary tree depth

[0318] -MinBTSize: Minimum allowed size of binary leaf nodes

[0319] In an example of a QTBT segmentation structure, the CTU size is set to 128×128 luma pixels and two corresponding blocks consisting of 64×64 chroma pixels. The MinQTSize is set to 16×16, the MaxBTSize to 64×64, the MinBTSize (width and height) to 4×4, and the MaxBTDepth to 4. First, the CTU is segmented into a quadtree, generating quadtree leaf nodes. The size of the quadtree leaf nodes ranges from 16×16 (i.e., MinQTSize) to 128×128 (i.e., the CTU size). When the size of a quadtree leaf node equals MinQTSize, further quadtree segmentation is not considered. If a quadtree leaf node is 128×128, since this size exceeds MaxBTSize (i.e., 64×64), further binary tree segmentation is not performed. Otherwise, the quadtree leaf nodes can be further segmented using a binary tree. Therefore, the quadtree leaf node is also the root node of the binary tree, with a binary tree depth of 0. When the binary tree depth reaches MaxBTDepth (i.e., 4), further partitioning is no longer considered. When the width of a binary tree node equals MinBTSize (i.e., 4), further horizontal partitioning is no longer considered. Similarly, when the height of a binary tree node equals MinBTSize, further vertical partitioning is no longer considered. Leaf nodes of the binary tree are further processed through prediction and transformation without any further partitioning. In JEM, the maximum CTU size is 256×256 luminance pixels. Leaf nodes (CUs) of the binary tree can undergo further processing (e.g., performing prediction and transformation processes) without any further partitioning.

[0320] Furthermore, the QTBT scheme supports different QTBT structures for luma and chroma. Currently, for P-strip and B-strip, the luma CTU and chroma CTB within a single CTU can use the same QTBT structure. However, for I-strip, the luma CTB is segmented into CUs using a QTBT structure, while the chroma CTB can be segmented into chroma CUs using other QTBT structures. That is, the CU in an I-strip includes a decoding block composed of luma components or two chroma components, while the CU in a P-strip or B-strip includes a decoding block composed of three color components.

[0321] Encoder 200 performs rate-distortion optimization (RDO) on the QTBT structure to determine block segmentation.

[0322] Additionally, U.S. patent application No. 20170208336 proposes a block partitioning structure called multi-type-tree (MTT) to replace CU structures based on QT, BT, and / or QTBT. The MTT partitioning structure is still a repeating tree structure. In MTT, multiple different partitioning structures (e.g., two or more partitioning structures) are used. For example, according to MTT technology, at each depth of the tree structure, each corresponding non-leaf node in the tree structure can use three or more different partitioning structures. The depth of a node in the tree structure can refer to the path length (e.g., the number of partitions) from that node to the root of the tree structure. A partitioning structure generally refers to how many different blocks a block can be divided into. A partitioning structure can be a quadtree partitioning structure that divides a block into 4 blocks; a binary tree partitioning structure that divides a block into two blocks; or a ternary tree partitioning structure that divides a block into three blocks. Furthermore, a ternary tree partitioning structure may not partition the block from the middle. A partitioning structure can have multiple different partitioning types. The segmentation type can further define how a block is divided, including symmetrical or asymmetrical segmentation, uniform or non-uniform segmentation, and / or horizontal or vertical segmentation.

[0323] In MTT, at each depth of the tree structure, encoder 200 can further divide the subtree using a specific segmentation type from one of three or more segmentation structures. For example, encoder 100 can be used to determine a specific segmentation type based on QT, BT, triple-tree (TT), and other segmentation structures. In one example, a QT segmentation structure may include a square quadtree or a rectangular quadtree segmentation type. Encoder 200 can use a square quadtree segmentation to segment a square block, that is, to divide the square block horizontally and vertically into four equal-sized square blocks. Similarly, encoder 200 can use a rectangular quadtree segmentation to segment a rectangular block (e.g., a non-square block), that is, to divide the rectangular block horizontally and vertically into four equal-sized rectangular blocks.

[0324] The BT partitioning structure can include at least one of the following partitioning types: horizontally symmetric binary tree, vertically symmetric binary tree, horizontally asymmetric binary tree, or vertically asymmetric binary tree. For the horizontally symmetric binary tree partitioning type, encoder 200 can be used to horizontally divide a block into two symmetric blocks of the same size from the middle. For the vertically symmetric binary tree partitioning type, encoder 200 can be used to vertically divide a block into two symmetric blocks of the same size from the middle. For the horizontally asymmetric binary tree partitioning type, encoder 100 can be used to horizontally divide a block into two blocks of different sizes. For example, one block can be 1 / 4 the size of the parent block, and the other block can be 3 / 4 the size of the parent block. This type is similar to the PART_2N×nU or PART_2N×nD partitioning types. For the vertically asymmetric binary tree partitioning type, encoder 100 can be used to vertically divide a block into two blocks of different sizes. For example, one block could be 1 / 4 the size of its parent block, and another block could be 3 / 4 the size of its parent block. This type is similar to the PART_nL×2N or PART_nR×2N partition types. In other examples, asymmetric binary tree partition types can divide a parent block into parts of different sizes. For example, one child block could be 3 / 8 the size of its parent block, and another child block could be 5 / 8 the size of its parent block. Of course, such partition types can be vertical or horizontal.

[0325] The TT partitioning structure differs from QT results or BT structures in that it does not divide blocks from the middle. The middle region of a block remains within the same sub-block. Unlike QT, which produces four blocks, or a binary tree, which produces two blocks, the TT partitioning structure produces three blocks. Exemplary partitioning types according to the TT partitioning structure include (horizontal and vertical) symmetrical partitioning types and (horizontal and vertical) asymmetrical partitioning types. Furthermore, the symmetrical partitioning type according to the TT partitioning structure can be a non-average / non-uniform partitioning type or an average / uniform partitioning type. The asymmetrical partitioning type according to the TT partitioning structure is a non-average / non-uniform partitioning type. In one example, the TT partitioning structure may include at least one of the following partitioning types: horizontal average / uniform symmetrical ternary tree, vertical average / uniform symmetrical ternary tree, horizontal non-average / non-uniform symmetrical ternary tree, vertical non-average / non-uniform symmetrical ternary tree, horizontal non-average / non-uniform asymmetrical ternary tree, or vertical non-average / non-uniform asymmetrical ternary tree partitioning type.

[0326] Generally, a non-uniform / non-symmetrical ternary tree partition is a partition type that is symmetrical about the center line of a block, but at least one of the resulting three blocks is not the same size as the other two. A preferred example is that the side blocks are 1 / 4 the size of the block, and the center block is 1 / 2 the size of the block. A uniform / uniform ternary tree partition is a partition type that is symmetrical about the center line of a block, and all resulting blocks are the same size. This partition can be used if the height or width of the blocks obtained by vertical or horizontal partitioning is a multiple of 3. A non-uniform / non-symmetrical ternary tree partition is a partition type that is asymmetrical about the center line of a block, and at least one of the resulting blocks is not the same size as the other two.

[0327] In an example of partitioning a block (e.g., at a subtree node) into an asymmetric ternary tree partition, encoder 200 and / or decoder 300 may restrict two of the three partitions to have the same size. This restriction may correspond to a constraint that encoder 200 must adhere to when encoding video data. Furthermore, in some examples, encoder 200 and decoder 300 may apply a restriction that, when partitioning according to the asymmetric ternary tree partition type, the sum of the areas of two partitions equals the area of ​​the remaining partition.

[0328] In some examples, encoder 200 can be used to select the segmentation type for QT, BT, and TT segmentation structures from all the segmentation types mentioned above, respectively. In other examples, encoder 200 can be used to determine the segmentation type only from a subset of the segmentation types mentioned above. For example, quadtree structures of certain block sizes or depths can use a subset of the segmentation types mentioned above (or other segmentation types). The supported subsets of segmentation types can be indicated in the bitstream for decoder 200 to use, or they can be predefined so that encoder 200 and decoder 300 can determine the subset without any indication.

[0329] In other examples, the number of supported segmentation types can be fixed for all depths across all CTUs. That is, encoder 200 and decoder 300 can be pre-configured to use the same number of segmentation types for any depth of the CTU. In other examples, the number of supported segmentation types can vary and may be determined based on depth, stripe type, or other previously decoded information. In one example, at depth 0 or depth 1 of the tree structure, only the QT segmentation structure is used. At depths greater than 1, each of the QT, BT, and TT segmentation structures can be used.

[0330] In some examples, encoder 200 and / or decoder 300 can impose pre-configured constraints on the supported segmentation types to avoid repeated segmentation of a region of the video image or a region of the CTU. In one example, when partitioning a block using an asymmetric segmentation type, encoder 200 and / or decoder 300 can be used to prevent further subdivision of the largest sub-block derived from the current block. For example, when partitioning a square block according to an asymmetric segmentation type (similar to the PART_2N×nU segmentation type), the largest sub-block among all sub-blocks (similar to the largest sub-block in the PART_2N×nU segmentation type) is the marked leaf node and cannot be further subdivided. However, smaller sub-blocks (similar to smaller sub-blocks in the PART_2N×nU segmentation type) can be further subdivided.

[0331] In another example where constraints can be imposed on supported segmentation types to avoid duplicate segmentation of a region, when a block is divided using an asymmetric segmentation type, the largest sub-block derived from the current block cannot be further divided in the same direction. For example, when a square block is divided using an asymmetric segmentation type (similar to the PART_2N×nU segmentation type), the encoder 200 and / or decoder 300 can be used to prevent the largest sub-block (similar to the largest sub-block in the PART_2N×nU segmentation type) from being divided in the horizontal direction.

[0332] In another example where constraints can be imposed on the supported segmentation types to avoid the difficulty of further segmentation, the encoder 200 and / or decoder 300 can be used to not segment the block horizontally or vertically when the width / height of the block is not a power of 2 (e.g., when the width / height is not 2, 4, 8, 16, etc.).

[0333] The above example describes how encoder 200 can be used to perform MTT segmentation. Decoder 300 can then perform the same MTT segmentation as encoder 200. In some examples, using the same set of predefined rules on the decoder 300 side, encoder 200 can determine how to segment images of video data. However, in many cases, encoder 200 can determine the specific segmentation structure and segmentation type to use for a specific image of the video data being decoded based on rate-distortion criteria. Therefore, in order for decoder 300 to determine the segmentation of a specific image, encoder 200 can indicate syntax elements in the encoded bitstream that indicate how to segment the image and the CTU of the image. Decoder 200 can parse such syntax elements and segment the image and CTU accordingly.

[0334] In one example, the prediction processing unit 260 in the video encoder 200 can be used to perform any combination of the segmentation techniques described above, especially motion estimation, which will be described in detail later.

[0335] Similar to image 201, block 203 is also, or can be considered as, a two-dimensional array or matrix of pixels with intensity values ​​(pixel values), but the size of block 203 is smaller than that of image 201. In other words, block 203 may include, for example, a single pixel array (e.g., a luminance array in the case of black and white image 201) or a three-pixel array (e.g., a luminance array and two chrominance arrays in the case of color image 201) or any other number and / or type of array depending on the color format employed. The number of pixels in block 203 in the horizontal and vertical directions (or axes) defines the size of block 203.

[0336] Figure 2 The encoder 200 shown is used to encode the image 201 block by block, for example, to perform encoding and prediction for each block 203.

[0337] Residual calculation

[0338] The residual calculation unit 204 is used to calculate the residual block 205 based on the image block 203 and the prediction block 265 (other details of the prediction block 265 are provided below), for example, by subtracting the pixel value of the prediction block 265 from the pixel value of the image block 203 pixel by pixel to obtain the residual block 205 in the pixel domain.

[0339] Transformation

[0340] The transformation processing unit 206 can be used to perform discrete cosine transform (DCT) or discrete sine transform (DST) on the pixel values ​​of the residual block 205 to obtain the transformation coefficients 207 in the transform domain. The transformation coefficients 207 can also be called transformation residual coefficients, which can represent the residual block 205 in the transform domain.

[0341] Transform processing unit 206 can be used to apply an integer approximation of DCT / DST, such as the transform specified for HEVC / H.265. This integer approximation is typically scaled by a factor compared to orthogonal DCT transforms. To maintain the norm of the residual block after both forward and inverse transforms, an additional scaling factor is used as part of the transform process. The scaling factor is typically chosen based on certain constraints, such as the power of 2 used for shift operations, the bit depth of the transform coefficients, and a trade-off between accuracy and implementation cost. For example, a specific scaling factor is specified for the inverse transform (and the corresponding inverse transform) on the decoder 300 side via inverse transform processing unit 212, etc.; correspondingly, a corresponding scaling factor can be specified for the forward transform on the encoder 200 side via transform processing unit 206, etc.

[0342] Quantification

[0343] Quantization unit 208 is used to quantize transform coefficients 207 using scalar quantization or vector quantization, etc., to obtain quantized transform coefficients 209. Quantized transform coefficients 209 can also be called quantization residual coefficients 209. The quantization process can reduce the bit depth associated with some or all of the transform coefficients 207. For example, n-bit transform coefficients can be rounded down to m-bit transform coefficients during quantization, where n is greater than m. The degree of quantization can be modified by adjusting the quantization parameter (QP). For example, for scalar quantization, different degrees of scaling can be applied to achieve finer or coarser quantization. Smaller quantization steps correspond to finer quantization, while larger quantization steps correspond to coarser quantization. The appropriate quantization step size can be indicated by the quantization parameter (QP). For example, the quantization parameter can be an index of a predefined set of appropriate quantization steps. For example, a smaller quantization parameter can correspond to fine quantization (smaller quantization step size), a larger quantization parameter can correspond to coarse quantization (larger quantization step size), and vice versa. Quantization can include division by the quantization step size, while the corresponding dequantization performed by dequantization 210, etc., can include multiplication by the quantization step size. According to embodiments of some standards such as HEVC, the quantization step size can be determined using quantization parameters. Generally, the quantization step size can be calculated from the quantization parameters using a fixed-point approximation of an equation involving division. Additional scaling factors can be introduced for quantization and dequantization to recover the norm of the residual block, which may be modified due to scaling used in the fixed-point approximation of the equations used for the quantization step size and quantization parameters. In one exemplary implementation, scaling for inverse transform and dequantization can be incorporated. Alternatively, a custom quantization table can be used, and the quantization table can be sent from the encoder to the decoder via a signal in the bitstream, etc. Quantization is a lossy operation, where the loss increases with the size of the quantization step size.

[0344] The dequantization unit 210 performs dequantization on the quantization coefficients as performed by the quantization unit 208, resulting in dequantized coefficients 211. For example, it performs a dequantization scheme that is the opposite of the quantization scheme performed by the quantization unit 208, using the same quantization step size as the quantization unit 208. The dequantization coefficients 211 can also be called dequantization residual coefficients 211, corresponding to the transform coefficients 207. However, due to the loss caused by quantization, the dequantization coefficients 211 are usually different from the transform coefficients.

[0345] The inverse transform processing unit 212 performs the inverse transform of the transform performed by the transform processing unit 206, such as the inverse discrete cosine transform (DCT) or the inverse discrete sine transform (DST), to obtain the inverse transform block 213 in the pixel domain. The inverse transform block 213 can also be called the inverse transform dequantization block 213 or the inverse transform residual block 213.

[0346] The reconstruction unit 214 (e.g., summer 214) is used to add the inverse transform block 213 (i.e., the reconstruction residual block 213) to the prediction block 265 by adding the pixel values ​​of the reconstruction residual block 213 and the pixel values ​​of the prediction block 265, thereby obtaining the reconstruction block 215 in the pixel domain.

[0347] Optionally, buffer unit 216 (or simply "buffer" 216) (e.g., column buffer 216) is used to buffer or store reconstructed block 215 and corresponding pixel values ​​for intra-frame prediction, etc. In other embodiments, the encoder can use the unfiltered reconstructed block and / or corresponding pixel values ​​stored in buffer unit 216 to perform any type of estimation and / or prediction, such as intra-frame prediction.

[0348] In an embodiment, encoder 200 may be configured such that, for example, buffer unit 216 is used not only to store reconstructed blocks 215 for intra-frame prediction 254, but also for loop filter unit 220. Figure 2 (not shown in the image), and / or such that, for example, buffer unit 216 and decoded image buffer 230 form a buffer. In other embodiments, encoder 200 may be used to filter block 221 and / or blocks or pixels from decoded image buffer 230 (not shown in the image), and / or such that buffer unit 216 and decoded image buffer 230 form a buffer. Figure 2 (Not shown in the image) is used as input or basis for intra-frame prediction 254.

[0349] Loop filter unit 220 (or simply "loop filter" 220) is used to filter the reconstructed block 215 to obtain the filtered block 221, thereby facilitating pixel transformation or improving video quality. Loop filter unit 220 represents one or more loop filters, such as deblocking filters, sample-adaptive offset (SAO) filters, or other filters such as bilateral filters, adaptive loop filters (ALF), sharpening or smoothing filters, or collaborative filters. Although loop filter unit 220... Figure 2 The loop filter unit 220 is shown as an in-loop filter, but in other configurations, it can be implemented as a post-loop filter. The filtered block 221 can also be called the filtered reconstruction block 221. After the loop filter unit 220 performs filtering on the reconstructed decoding block, the decoded image buffer 230 can store the reconstructed decoding block.

[0350] In an embodiment, encoder 200 (correspondingly, loop filter unit 220) can be used to output loop filter parameters (e.g., pixel adaptive offset information), for example, directly output or output after entropy encoding by entropy encoding unit 270 or any other entropy decoding unit, so that decoder 300 can receive and use the same loop filter parameters for decoding.

[0351] The decoded picture buffer (DPB) 230 can be a reference picture memory that stores reference picture data for use by the video encoder 20 when encoding video data. The DPB 230 can be composed of any of a variety of memory devices, such as dynamic random access memory (DRAM) (including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), and resistive RAM (RRAM)) or other types of memory devices. The DPB 230 and buffer 216 can be provided by the same memory device or separate memory devices. In one example, the decoded picture buffer (DPB) 230 is used to store filtered block 221. The decoded picture buffer 230 can also be used to store other previously filtered blocks (e.g., previously filtered reconstructed block 221) in the same current image or different images (e.g., previously reconstructed images), and can provide the complete previously reconstructed (i.e., decoded) image (and corresponding reference blocks and pixels) and / or partially reconstructed current image (and corresponding reference blocks and pixels) for inter-frame prediction, etc. In one example, if reconstructed block 215 can be reconstructed without in-loop filtering, the decoded picture buffer (DPB) 230 is used to store reconstructed block 215.

[0352] The prediction processing unit 260, also known as the block prediction processing unit 260, is used to receive or acquire block 203 (current block 203 in current image 201) and reconstruct image data, such as reference pixels from the same (current) image from buffer 216 and / or reference image data 231 from one or more previously decoded images from decoded image buffer 230, and to process such data for prediction, i.e., to provide prediction block 265 which can be an inter-frame prediction block 245 or an intra-frame prediction block 255.

[0353] The mode selection unit 262 can be used to select a prediction mode (e.g., intra-frame or inter-frame prediction mode) and / or the corresponding prediction block 245 or 255 used as prediction block 265 to calculate the residual block 205 and reconstruct the reconstruction block 215.

[0354] In an embodiment, the mode selection unit 262 can be used to select a prediction mode (e.g., from prediction modes supported by the prediction processing unit 260), which provides the best match or minimum residual (minimum residual refers to better compression in transmission or storage), or provides minimum signaling overhead (minimum signaling overhead refers to better compression in transmission or storage), or considers or balances both. The mode selection unit 262 can be used to determine the prediction mode based on rate distortion optimization (RDO), i.e., selecting the prediction mode that provides minimum rate distortion optimization, or selecting the prediction mode whose correlation rate distortion at least meets the prediction mode selection criteria.

[0355] The prediction processing (e.g., performed by prediction processing unit 260) and mode selection (e.g., performed by mode selection unit 262) performed by the exemplary encoder 200 will be described in detail below.

[0356] As described above, encoder 200 is used to determine or select the best or optimal prediction mode from a (predetermined) set of prediction modes. The set of prediction modes may include intra-frame prediction modes and / or inter-frame prediction modes, etc.

[0357] The intra-prediction mode set can include 35 different intra-prediction modes, such as non-directional modes like DC (or mean) mode and planar mode, or directional modes as defined in H.265, or it can include 67 different intra-prediction modes, such as non-directional modes like DC (or mean) mode and planar mode, or directional modes as defined in H.266 which is under development.

[0358] The set of (possible) inter-frame prediction modes depends on the available reference image (i.e., at least a portion of the decoded image stored in the aforementioned DPB 230, etc.) and other inter-frame prediction parameters, such as whether the entire reference image or only a portion of the reference image (e.g., a search window region surrounding the current block) is used to search for the best matching reference block, and / or, for example, whether pixel interpolation (e.g., half-pixel interpolation and / or quarter-pixel interpolation) is used.

[0359] In addition to the prediction modes mentioned above, skip mode and / or direct mode can also be used.

[0360] The prediction processing unit 260 can also be used to divide the block 203 into smaller block portions or sub-blocks, for example, by iteratively using quad-tree (QT) partitioning, binary-tree (BT) partitioning, or triple-tree (TT) partitioning, or any combination thereof; and to perform predictions on each of the block portions or sub-blocks, wherein mode selection includes selecting the tree structure for partitioning the block 203 and selecting the prediction mode used for each of the block portions or sub-blocks.

[0361] Inter-frame prediction unit 244 may include a motion estimation (ME) unit and a motion compensation (MC) unit (both in... Figure 2 (Not shown in the image). The motion estimation unit is used to receive or obtain image block 203 (current image block 203 in current image 201) and decoded image 331, or at least one or more previously reconstructed blocks (e.g., reconstructed blocks of one or more other / different previously decoded images 331) for motion estimation. For example, the video sequence may include the current image and the previous decoded image 331, or in other words, the current image and the previous decoded image 331 may be part of or form part of an image sequence in the video sequence. For example, the encoder 200 may be used to select a reference block from multiple reference blocks in the same or different images in multiple other images, and provide the motion estimation unit with the offset (spatial offset) between the position (x, y coordinates) of the reference image (or reference image index, etc.) and / or the position of the reference block and the position of the current block as an inter-frame prediction parameter. Figure 2 (Not shown in the image). This offset is also called a motion vector (MV). Fusion is an important motion estimation tool used in HEVC, and continues to be used in VVC. To perform fusion estimation, the first step is to construct a fusion candidate list, where each candidate includes all motion data, including information on whether one or two reference image lists are used, as well as the reference index and motion vector for each list. The fusion candidate list is constructed based on the following candidates: a) up to four spatial fusion candidates obtained from five spatially adjacent blocks; b) one temporal fusion candidate obtained from two temporally juxtaposed blocks; c) other fusion candidates, including combined bidirectional prediction candidates and zero motion vector candidates.

[0362] Intra-prediction unit 254 is further configured to determine intra-prediction block 255 based on intra-prediction parameters (e.g., the selected intra-prediction mode). In any case, after selecting an intra-prediction mode for a block, intra-prediction unit 254 is also configured to provide intra-prediction parameters to entropy coding unit 270, i.e., to provide information indicating the selected intra-prediction mode for that block. In one example, intra-prediction unit 254 may be used to perform any combination of intra-prediction techniques described below.

[0363] Entropy coding unit 270 is used to apply (or not apply) entropy coding algorithms or schemes (such as variable length coding (VLC), context adaptive VLC (CAVLC), arithmetic decoding, syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) decoding, or other entropy coding methods or techniques) to (or not apply) one or all of the quantization residual coefficients 209, inter-frame prediction parameters, intra-frame prediction parameters, and / or loop filter parameters, to obtain encoded image data 271 that can be output through output terminal 272 in the form of an encoded bitstream 21. The encoded bitstream 21 can be sent to video decoder 30, or archived for later transmission or retrieval by video decoder 30. Entropy coding unit 270 can also be used to entropy code other syntax elements of the current video strip being encoded.

[0364] Other structural variations of the video encoder 200 can be used to encode video streams. For example, a non-transform-based encoder 200 can directly quantize residual signals in certain blocks or frames where there is no transform processing unit 206. In another implementation, the encoder 200 may include a quantization unit 208 and an inverse quantization unit 210 combined into a single unit.

[0365] Figure 3 An exemplary video decoder 300 for implementing the technology of this application is shown. The video decoder 300 is used to receive, for example, encoded image data (e.g., an encoded bitstream) 271 encoded by encoder 200 to obtain a decoded image 331. During the decoding process, the video decoder 300 receives video data from the video encoder 200, such as an encoded video bitstream representing image blocks in an encoded video strip and associated syntax elements.

[0366] exist Figure 3In one example, decoder 300 includes an entropy decoding unit 304, an inverse quantization unit 310, an inverse transform processing unit 312, a reconstruction unit 314 (e.g., a summer 314), a buffer 316, a loop filter 320, a decoded image buffer 330, and a prediction processing unit 360. Prediction processing unit 360 may include an inter-frame prediction unit 344, an intra-frame prediction unit 354, and a mode selection unit 362. In some examples, video decoder 300 may perform substantially the same functions as the referenced unit. Figure 2 The video encoder 200 describes the encoding round as the inverse of the decoding round.

[0367] Entropy decoding unit 304 is used to perform entropy decoding on encoded image data 271 to obtain quantization coefficients 309 and / or decoding parameters. Figure 3 (Not shown in the image), such as any one or all of the (decoded) inter-frame prediction parameters, intra-frame prediction parameters, loop filter parameters, and / or other syntax elements. The entropy decoding unit 304 is also used to forward the inter-frame prediction parameters, intra-frame prediction parameters, and / or other syntax elements to the prediction processing unit 360. The video decoder 300 can receive syntax elements at the video stripe level and / or video block level.

[0368] The inverse quantization unit 310 can function the same as the inverse quantization unit 110, the inverse transform processing unit 312 can function the same as the inverse transform processing unit 112, the reconstruction unit 314 can function the same as the reconstruction unit 114, the buffer 316 can function the same as the buffer 116, the loop filter 320 can function the same as the loop filter 120, and the decoding image buffer 330 can function the same as the decoding image buffer 130.

[0369] The prediction processing unit 360 may include an inter-frame prediction unit 344 and an intra-frame prediction unit 354, wherein the inter-frame prediction unit 344 may be functionally similar to the inter-frame unit 144, and the intra-frame prediction unit 354 may be functionally similar to the intra-frame prediction unit 154. The prediction processing unit 360 is typically used to perform block prediction and / or obtain prediction blocks 365 from the encoded data 21, and to receive or obtain prediction-related parameters and / or information about the selected prediction mode from the entropy decoding unit 304, etc. (explicitly or implicitly).

[0370] When a video strip is decoded into an intra-decoded (I) strip, the intra-prediction unit 354 in the prediction processing unit 360 generates a prediction block 365 for the current video strip based on the indicated intra-prediction mode and data from previously decoded blocks in the current frame or image. When a video frame is decoded into an inter-decoded (i.e., B or P) strip, the inter-prediction unit 344 (e.g., a motion compensation unit) in the prediction processing unit 360 generates a prediction block 365 for the current video strip based on motion vectors and other syntax elements received from the entropy decoding unit 304. For inter-prediction, a prediction block can be generated based on one of the reference images in one of the reference image lists. The video decoder 300 can construct reference frame lists 0 and 1 using a default construction technique based on the reference images stored in the DPB 330.

[0371] The prediction processing unit 360 is used to determine the prediction information of video blocks in the current video strip by parsing motion vectors and other syntax elements, and to generate prediction blocks for the current video block being decoded using the prediction information. For example, the prediction processing unit 360 uses some received syntax elements to determine the prediction mode (e.g., intra-frame prediction or inter-frame prediction) for decoding video blocks in the video strip, the inter-frame prediction stripe type (e.g., B-strip, P-strip, or GPB-strip), the construction information for one or more reference image lists for the strip, the motion vectors for each inter-frame encoded video block in the strip, the inter-frame prediction state for each inter-frame decoded video block in the strip, and other information to decode video blocks in the current video strip.

[0372] The dequantization unit 310 is used to dequantize, or dequantize, the quantization transform coefficients provided in the bitstream and decoded by the entropy decoding unit 304. The dequantization process may include using the quantization parameters calculated by the video encoder 100 for each video block in the video strip to determine the degree of quantization to be performed, and also to determine the degree of dequantization to be performed.

[0373] The inverse transform processing unit 312 is used to perform an inverse transform on the transform coefficients (e.g., inverse DCT, inverse integer transform, or a conceptually similar inverse transform process) to generate a residual block in the pixel domain.

[0374] The reconstruction unit 314 (e.g., summer 314) is used to add the inverse transform block 313 (i.e., the reconstruction residual block 313) to the prediction block 365 in the following manner to obtain the reconstruction block 315 in the pixel domain: the pixel values ​​in the reconstruction residual block 313 are added to the pixel values ​​in the prediction block 365.

[0375] Loop filter unit 320 (in or after the decoding loop) is used to filter the reconstructed block 315 to obtain the filtered block 321, thereby facilitating pixel transformation or improving video quality. In one example, loop filter unit 320 can be used to perform any combination of filtering techniques described below. Loop filter unit 320 is used to represent one or more loop filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, or other filters such as a bilateral filter, an adaptive loop filter (ALF), a sharpening or smoothing filter, or a co-filter. Although loop filter unit 320 is in Figure 3 The loop filter unit 320 is shown as an in-loop filter, but in other configurations, it can be implemented as a post-loop filter.

[0376] The decoded video block 321 in the given frame or image is then stored in the decoded image buffer 330, which stores a reference image for subsequent motion compensation.

[0377] The decoder 300 is used, for example, to output the decoded image 311 through the output terminal 312 for presentation to a user or for the user to view.

[0378] Other variations of the video decoder 300 can be used to decode compressed bitstreams. For example, the decoder 300 can generate an output video stream without the loop filter unit 320. For example, the non-transform-based decoder 300 can directly dequantize the residual signal in certain blocks or frames without the inverse transform processing unit 312. In another implementation, the video decoder 300 may include a dequantization unit 310 and an inverse transform processing unit 312 combined into a single unit.

[0379] Figure 4 This is a schematic diagram of a network device 400 (e.g., a decoding device) provided according to one embodiment of the present invention. The network device 400 is suitable for implementing the disclosed embodiments described herein. In one embodiment, the network device 400 may be a decoder (e.g., a decoder...). Figure 1A The video decoder 300 or encoder (e.g., in the video decoder 300) or encoder (e. Figure 1A (Video encoder 200 in the middle). In one embodiment, the network device 400 can be as described above. Figure 1A Video decoder 300 or Figure 1A One or more components in the video encoder 200.

[0380] Network device 400 includes: an input port 410 and a receiving unit (Rx) 420 for receiving data; a processor, logic unit, or central processing unit (CPU) 430 for processing the data; a transmitting unit (Tx) 440 and an output port 450 for transmitting the data; and a memory 460 for storing the data. Network device 400 may also include optical-to-electrical (OE) components and electrical-to-optical (EO) components coupled to the input port 410, receiving unit 420, transmitting unit 440, and output port 450 for the entry or exit of optical or electrical signals.

[0381] Processor 430 is implemented in both hardware and software. Processor 430 can be implemented as one or more CPU chips, cores (e.g., multi-core processors), FPGAs, ASICs, and DSPs. Processor 430 communicates with ingress port 410, receiver unit 420, transmitter unit 440, egress port 450, and memory 460. Processor 430 includes a decoding module 470. Decoding module 470 implements the disclosed embodiments described above. For example, decoding module 470 performs, processes, prepares, or provides various decoding operations. Therefore, including decoding module 470 provides a substantial improvement to the functionality of network device 400 and affects the transitions of network device 400 to different states. Alternatively, decoding module 470 can be implemented with instructions stored in memory 460 and executed by processor 430.

[0382] Memory 460 includes one or more disks, tape drives, or solid-state drives and can be used as an overflow data storage device to store such programs when a program is selected for execution, or to store instructions and data read during program execution. Memory 460 can be volatile and / or non-volatile, and can be read-only memory (ROM), random-access memory (RAM), ternary content-addressable memory (TCAM), or static random-access memory (SRAM).

[0383] Figure 5 A simplified block diagram of the apparatus 500 provided for an exemplary embodiment. The apparatus 500 can be used as... Figure 1AThe device 500 can be any one or both of the source device 12 and the destination device 14. The device 500 can implement the technology of this application. The device 500 can be a computing system including multiple computing devices, or a single computing device, such as a mobile phone, tablet computer, laptop computer, desktop computer, etc.

[0384] The processor 502 in device 500 may be a central processing unit. Alternatively, processor 502 may be any other type of device or multiple devices, existing or to be developed in the future, capable of manipulating or processing information. While the disclosed implementation may be implemented using a single processor, such as processor 502 as shown, using multiple processors can improve speed and efficiency.

[0385] In one implementation, the memory 504 in device 500 may be a read-only memory (ROM) device or a random access memory (RAM) device. Any other suitable type of storage device may be used as memory 504. Memory 504 may include code and data 506 accessed by processor 502 via bus 512. Memory 504 may also include an operating system 508 and an application program 510, which includes at least one program that causes processor 502 to perform the methods described herein. For example, application program 510 may include applications 1 to N, which may also include a video decoding application that performs the methods described herein. Device 500 may also include additional memory in the form of auxiliary memory 514, which may be a memory card or the like used with a mobile computing device. Since video communication sessions may include a large amount of information, it may be stored, in whole or in part, in auxiliary memory 514 and loaded into memory 504 for processing as needed.

[0386] Device 500 may also include one or more output devices, such as display 518. In one example, display 518 may be a touch-sensitive display combining a display with a touch-sensitive element capable of sensing touch input. Display 518 may be coupled to processor 502 via bus 512. In addition to display 518, other output devices may be provided to allow a user to program or otherwise use device 500. When the output device is a display or includes a display, the display may be a liquid crystal display (LCD), a cathode ray tube (CRT) display, a plasma display, or a light-emitting diode (LED) display (e.g., an organic LED (OLED) display).

[0387] The device 500 may also include an image sensing device 520 or communicate with an image sensing device 520. The image sensing device 520 may be a camera, or any other existing or future image sensing device 520 capable of sensing images (e.g., images of a user operating the device 500). The image sensing device 520 may be positioned facing the user operating the device 500. In one example, the position and optical axis of the image sensing device 520 may be configured such that the field of view includes an area immediately adjacent to the display 518, from which the display 518 can be seen.

[0388] The device 500 may also include or communicate with a sound sensing device 522. The sound sensing device 522 may be a microphone, or any other existing or future sound sensing device capable of sensing sounds near the device 500. The sound sensing device 522 may be placed towards the user operating the device 500. The sound sensing device 522 may be used to receive voice or other sounds emitted by the user while operating the device 500.

[0389] Although Figure 5 The processor 502 and memory 504 in device 500 are described as being integrated into a single device, but other configurations can be used. The operation of processor 502 can be distributed across multiple machines (each machine including one or more processors), which can be directly coupled or coupled via a local area network or other network. Memory 504 can be distributed across multiple machines, such as network-based memory or memory across multiple machines performing the operations of device 500. Although bus 512 of device 500 is shown herein as a single bus, there can be multiple buses 512. Furthermore, auxiliary memory 514 can be directly coupled to other components in device 500 or accessed via a network, and can include a single integrated unit (e.g., a memory card) or multiple units (e.g., multiple memory cards). Therefore, device 500 can have a wide variety of configurations.

[0390] In 4:2:0 sampling, the height and width of both chroma arrays are half that of the luminance array. Figure 6A The nominal vertical and horizontal relative positions of the luminance and chrominance pixels in the image are shown.

[0391] Figure 8 This is a conceptual diagram of exemplary locations, from which scaling parameters are derived for scaling the downsampled reconstructed luma blocks. For example, Figure 8 An example of 4:2:0 sampling is shown, with scaling parameters α and β.

[0392] Generally, when using LM prediction mode, video encoder 20 and video decoder 30 can invoke the following steps: Video encoder 20 and video decoder 30 can downsample adjacent luma pixels. Video encoder 20 and video decoder 30 can derive linear parameters (i.e., α and β) (also called scaling parameters). Video encoder 20 and video decoder 30 can downsample the current luma block and derive a predicted value (e.g., a prediction block) based on the linear parameters and the downsampled luma block. There are many methods for performing downsampling.

[0393] Figure 6B This is a conceptual diagram illustrating an example of luminance and chrominance locations. These luminance and chrominance locations are used to downsample pixels within a luminance block to generate a predicted block for the chrominance block. (Example:) Figure 6B As shown, a [1,1] filter is used to predict the chroma pixels represented by a solid triangle, based on two luminance pixels represented by two solid (i.e., black solid) circles. This [1,1] filter is an example of a 2-tap filter.

[0394] Figure 6C This is a conceptual diagram illustrating another example of luminance and chrominance locations. These luminance and chrominance locations are used to downsample pixels within a luminance block to generate a prediction block. (See diagram for example.) Figure 6C As shown, a 6-tap filter is used to predict the chromaticity pixels represented by solid triangles, based on the six luminance pixels represented by six solid (i.e., black solid) circles.

[0395] In one or more examples, the described functionality can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, these functions can be stored or transmitted as one or more instructions or code in a computer-readable medium and executed by a hardware-based processing unit. A computer-readable medium can include a computer-readable storage medium, corresponding to a tangible medium such as a data storage medium or a communication medium comprising any medium, thereby facilitating the transfer of a computer program from one place to another according to a communication protocol, etc. In this way, a computer-readable medium can generally correspond to (1) a non-transitory tangible computer-readable storage medium, or (2) a communication medium such as a signal or carrier wave. A data storage medium can be any available medium accessible to one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementing the techniques described herein. A computer program product can include a computer-readable medium.

[0396] Video compression techniques (such as motion compensation, intra-frame prediction, and loop filtering) have proven effective, and therefore various video decoding standards (such as H.264 / AVC and H.265 / HEVC) employ video compression techniques. Intra-frame prediction can be used when no reference image is available in an I-frame or I-strip, or when the current block or image does not use inter-frame prediction decoding. Typically, the reference pixels used for intra-frame prediction are derived from previously decoded (or reconstructed) adjacent blocks in the same image. For example, both H.264 / AVC and H.265 / HEVC use the boundary pixels of adjacent blocks as reference pixels for intra-frame prediction. To accommodate different texture or structural features, various intra-frame prediction modes exist. Each mode uses a different prediction signal derivation method. For example, H.265 / HEVC supports a total of 35 intra-frame prediction modes, such as... Figure 6D As shown.

[0397] Description of Intra-Prediction Algorithm in H.265 / HEVC

[0398] Intra-frame prediction uses the decoding boundary pixels in adjacent blocks as reference pixels. The encoder selects the optimal luminance intra-frame prediction mode for each block from 35 options (i.e., the mode that provides the most accurate prediction value for the current block): 33 directional prediction modes, DC mode, and planar mode. Figure 6D This illustrates the mapping between intra-frame prediction directions and intra-frame prediction mode numbers. It should be noted that recent video decoding technologies (such as versatile video coding, VVC) have developed 65 or more intra-frame prediction modes, enabling the capture of arbitrary edge directions presented in natural video. Among these prediction modes, horizontal modes (e.g., Figure 6D Pattern 10 in the middle) is also called the "horizontal pattern", and the vertical pattern (e.g.) Figure 6D Mode 26 in the text is also known as "vertical mode".

[0399] Figure 7 This shows a reference pixel for a block. (Example) Figure 7 As shown, the block "CUR" is the current block to be predicted, and the dark pixels surrounding the boundary of the current block are the reference pixels used to predict the current block. These reference pixels are pixels in the reconstructed blocks (also called neighboring blocks) adjacent to the current block. The block "CUR" can be a luma block or a chroma block, depending on the type of block to be predicted. The predicted signal can be derived by mapping the reference pixels, depending on the specific method indicated by the intra-frame prediction mode.

[0400] Reference pixel replacement

[0401] For several reasons, some or all reference pixels may be unavailable for intra-frame prediction. For example, pixels outside of an image, strip, or block may be unavailable for prediction. Additionally, when performing constrained intra-frame prediction, reference pixels belonging to the PU (PU) that has undergone inter-frame prediction are omitted to avoid error propagation from previously received and reconstructed images that may be incorrectly received. As described herein, a reference pixel is available if it is within the current image, strip, or block, if it can be reconstructed before decoding the current block, and / or if it is not omitted due to a decoding decision made by the encoder. In HEVC, all prediction modes can be used after replacing unavailable reference pixels. In the extreme case where all reference pixels are unavailable, all reference pixels are replaced with a nominal average pixel value (e.g., 128 for 8-bit data) at a given positioning depth. If at least one reference pixel is marked as available for intra-frame prediction, the unavailable reference pixels are replaced with the available reference pixels. The process involves scanning reference pixels clockwise and replacing unavailable pixel values ​​with the latest available pixel values, thus replacing unavailable reference pixels. If the first pixel encountered during the clockwise scan is unavailable, it is replaced with the first available reference pixel encountered during the clockwise scan. Here, "replacement" can also be called filling, and the replaced pixel can be called the filled pixel.

[0402] Constrained Intra-Frame Prediction

[0403] Constrained intra-prediction is a tool used to avoid the propagation of spatial noise caused by using inconsistent reference pixels between the encoder and decoder for spatial intra-prediction. Packet loss occurs when transmitting stripes that have undergone inter-frame decoding, resulting in inconsistent reference pixels between the encoder and decoder. Inconsistent reference pixels also occur when using lossy decoder-side memory compression. When constrained intra-prediction is enabled, pixels that have undergone inter-frame prediction are marked as unusable for intra-prediction, and these unusable pixels can be filled using the padding methods disclosed above, thereby performing full intra-prediction estimation on the encoding side or intra-prediction on the decoding side.

[0404] Intercomponent linear model (CCLM) prediction

[0405] Cross-component linear model prediction (CCLM), also known as cross-component intra-prediction (CCIP), is a type of intra-prediction mode used to reduce inter-component redundancy in intra-prediction. Figure 8 (include Figure 8 A and Figure 8B) is a schematic diagram of an exemplary mechanism for performing CCLM intra-frame prediction. Figure 8 An example of 4:2:0 sampling is shown. Figure 8 This shows an example of the position of a pixel in the current block and its left and top adjacent pixels in CCLM mode. The white square represents a pixel in the current block, and the shaded circle represents a reconstructed pixel in an adjacent block. Figure 8 A shows an example of adjacent reconstructed pixels of a chroma block. Figure 8 B shows an example of adjacent reconstructed pixels of a juxtaposed luma block. If the video format is YUV4:2:0, there is one 16×16 luma block and two 8×8 chroma blocks.

[0406] Intra-frame estimation unit 254 in encoder 200 and / or intra-frame prediction unit 354 in decoder 300 can perform CCLM intra-frame prediction. CCLM intra-frame prediction is used to predict chroma pixel 803 in chroma block 801. Chroma pixel 803 appears at an integer position displayed as a square. CCLM intra-frame prediction is performed partially based on adjacent reference pixels displayed as black circles. Chroma pixel 803 is not entirely predicted based on adjacent chroma reference pixel 805. Chroma pixel 803 is also predicted based on luma reference pixel 813 and adjacent luma reference pixel 815. Specifically, a CU includes one luma block 811 and two chroma blocks 801. A model is generated that correlates chroma pixel 803 and luma reference pixel 813 within the same CU. The linearity coefficients of the model are determined by comparing adjacent luma reference pixel 815 with adjacent chroma reference pixel 805.

[0407] When reconstructing the luminance reference pixel 813, the luminance reference pixel 813 is represented as the reconstructed luminance pixel (Rec'L). When reconstructing the adjacent chrominance reference pixel 805, the adjacent chrominance reference pixel 805 is represented as the reconstructed adjacent chrominance pixel (Rec'C).

[0408] As shown in the figure, the number of pixels in luminance block 811 is four times that in chroma block 801. Figure 8In the example shown, chroma block 801 comprises N×N pixels, while luma block 811 comprises 2N×2N pixels. Therefore, the resolution of luma block 811 is four times that of chroma block 801. To predict luma reference pixel 813 and its adjacent luma reference pixel 815, luma reference pixel 813 and its adjacent luma reference pixel 815 are downsampled to allow for accurate comparison with adjacent chroma reference pixels 805 and chroma pixel 803. The downsampling process reduces the resolution of a set of pixels. For example, when using the YUV4:2:0 format, luma pixels can be downsampled by a factor of 4 (e.g., width divided by 2, height divided by 2). YUV is a color encoding system that uses a color space that includes a luma component Y and two chroma components U and V.

[0409] In CCLM prediction, based on the downsampled corresponding reconstructed luminance pixels (current luminance block), the following linear model is used to predict chrominance pixels:

[0410] pred C (i,j)=α·rec L ′(i,j)+ β (1)

[0411] Among them, pred C (i,j) represents the predicted chromaticity pixel, rec′ L (i,j) represents the corresponding reconstructed luminance pixel after downsampling. To minimize the regression error between the current luminance block and the adjacent reconstructed luminance and chrominance pixels around the current chrominance block, the parameters α and β can be derived as follows:

[0412]

[0413] Where L(n) represents the reconstructed luminance pixels adjacent to the top and left sides after downsampling, C(n) represents the reconstructed chrominance pixels adjacent to the top and left sides, and the value of N is equal to the number of pixels used to derive the two coefficients. For square decoding blocks, the above two equations are used directly. This regression error minimization calculation is also part of the decoding process, not just an encoder search operation, therefore no syntax is used to indicate the values ​​of α and β.

[0414] Besides using the regression error minimization method described above (also known as the least squares (LS) method), the maximum and minimum luminance pixel values ​​can also be used to derive the linear model coefficients α and β. This latter method is also called the MaxMin method. In the MaxMin method, after downsampling the reconstructed luminance pixels above and to the left, a one-to-one relationship is obtained between these adjacent reconstructed luminance pixels and their respective adjacent reconstructed chrominance pixels above and to the left. Therefore, the linear model coefficient parameters α and β can be derived using pairs of luminance and chrominance pixels based on this one-to-one relationship. First, the minimum and maximum values ​​of the downsampled reconstructed luminance pixels above and to the left are identified. Then, the corresponding pixels are identified from the reconstructed chrominance templates above and to the left, thus obtaining the pairs of luminance and chrominance pixels. The pairs of luminance and chrominance pixels are as follows: Figure 9 As shown in (A, B). The linear model parameters α and β are obtained according to the following equation:

[0415]

[0416] β=y A -αx A (5)

[0417] Among them, (x A y A )yes Figure 9 The coordinates of A in the middle, (x B y B )yes Figure 9 The coordinates of B in the middle.

[0418] Add CCLM luma to the chroma prediction mode as an additional intra-chroma prediction mode. On the encoder side, in order to select the intra-chroma prediction mode, add an additional rate-distortion (RD) cost check to the chroma components.

[0419] For simplicity, in this document, the term "template" is used to refer to adjacent reconstructed chroma pixels and downsampled adjacent reconstructed luma pixels. These adjacent reconstructed chroma pixels and downsampled adjacent reconstructed luma pixels are also referred to as reference pixels in the template. Figure 10 This shows templates for chroma blocks and their corresponding downsampled luma blocks. Figure 10In the example shown, luminance block '1020 is a downsampled luminance block of the current luminance block and has the same spatial resolution as chroma block 1040. That is, luminance block '1020 is a downsampled juxtaposed luminance block of chroma block 1040. The upper template 1002 includes the adjacent reconstructed chroma pixels above the current chroma block 1040 and the corresponding downsampled upper adjacent reconstructed luminance pixels of luminance block '1020. The downsampled upper adjacent reconstructed luminance pixels of luminance block '1020 are obtained based on the adjacent pixels above the luminance blocks. As described herein, the adjacent pixels above the luminance blocks may include adjacent pixels directly above the luminance blocks, adjacent pixels that are not adjacent to the luminance blocks, or both. The left template 1004 includes the left adjacent reconstructed chroma pixels and the corresponding downsampled left adjacent reconstructed luminance pixels. The upper adjacent reconstructed chroma pixels are also referred to as the "upper chroma template," such as upper chroma template 1006. The downsampled, adjacent reconstructed luminance pixels above are called the "upper luminance template," for example, upper luminance template 1008. The adjacent reconstructed chrominance pixels to the left are called the "left chrominance template," for example, left chrominance template 1010. The downsampled, adjacent reconstructed luminance pixels to the left are called the "left luminance template," for example, left luminance template 1012. The elements included in the template are called reference pixels in that template.

[0420] In existing CCLM applications, if a reference pixel in the upper or left template is marked as unavailable, the entire template will not be used. Figure 11 An example of a template including unavailable reference pixels is shown. Figure 11 In the example shown, for chroma block 1140, if there is an unusable reference pixel in the upper template, such as reference pixel A2 in 1102, the upper template is not used to derive the linear model coefficients. Similarly, if there is an unusable reference pixel in the left template, such as reference pixel B2 in 1104, the left template is not used to derive the linear model coefficients. This degrades the decoding performance of intra-frame prediction.

[0421] Multidirectional linear model

[0422] The reference pixels in the upper and left templates are used together to calculate the linear model coefficients, and can also be used in the other two CCLM modes (CCLM_T mode and CCLM_L mode). CCLM_T and CCLM_L can also be collectively referred to as multi-directional linear model (MDLM). Figure 12 The reference pixel used in CCLM_T mode is shown. Figure 13 The reference pixel points used in CCLM_L mode are shown. For example... Figure 12 As shown, in CCLM_T mode, only reference pixels in the template above (e.g., reference pixels 1202 and 1204) are used to calculate the linear model coefficients. Figure 13 As shown, in the CCLM_L mode, only reference pixels in the left template (e.g., reference pixels 1212 and 1214) are used to calculate the linear model coefficients. The number of reference pixels used in these modes is W+H, where W is the width of the chroma block and H is the height of the chroma block.

[0423] CCLM and MDLM modes (i.e., CCLM_T and CCLM_L modes) can be used simultaneously or selectively. For example, the codec can use only CCLM mode, only MDLM mode, or both CCLM and MDLM modes simultaneously. In the latter case, where both CCLM and MDLM modes are used, three additional modes (CCLM, CCLM_T, and CCLM_L) are added as three extra chroma intra-prediction modes. On the encoder side, three additional RD cost checks are added to the chroma components to select the chroma intra-prediction mode. In existing MDLM methods, the LS method is used to derive model parameters or model coefficients. If there are insufficient available reference pixels, padding is used to copy the farthest pixel value or extract pixel values ​​from available reference pixels.

[0424] However, obtaining the linear model coefficients of the MDLM pattern using the LS method leads to computational complexity. Furthermore, in existing MDLM patterns, some template pixels can be located far from the current block, especially for non-square blocks. For example, the rightmost reference pixel in the upper template and the lower reference pixel in the left template are both far from the current block. Therefore, these reference pixels have low correlation with the current block, resulting in low prediction efficiency for chroma blocks. The technique presented in this paper can reduce the complexity of MDLM and improve the correlation between template pixels and the current block.

[0425] In one example, when using the MaxMin method to determine model coefficients, besides using reference pixels from both the top and left templates simultaneously, it's also possible to use only reference pixels from either the left or top template. For instance, in the MaxMin method, only reference pixels from the top luminance template are examined to determine the maximum and minimum luminance values. Alternatively, only reference pixels from the left luminance template are examined to determine the maximum and minimum luminance values. After determining the pixel positions for the maximum and minimum luminance values, the corresponding chroma pixel values ​​can be obtained based on their positions.

[0426] Figure 14A schematic diagram illustrating an example of the reference pixels used to determine the maximum and minimum brightness values. Figure 14 In the example shown, the number of reference pixels in the upper luminance template (represented as W1) is greater than the width of the current chroma block (represented as W), and the number of reference pixels in the left luminance template (represented as H1) is greater than the height of the current chroma block (represented as H). Figure 15 A schematic diagram illustrating another example of reference pixels used to determine the maximum and minimum brightness values. Figure 15 In the example shown, the number of luminance reference pixels at the top is equal to the width W of the current chroma block, and the number of luminance reference pixels on the left is equal to the height H of the current chroma block.

[0427] In summary, besides the LS method, the MaxMin method can also be used in MDLM patterns. In other words, the MaxMin method can be used to derive the model coefficients for CCLM_T and CCLM_L patterns. Since the computational complexity of the MaxMin method is lower than that of the LS algorithm, the proposed method reduces computational complexity, thereby improving MDLM. Furthermore, existing MaxMin methods use both the upper and left templates for CCLM patterns. The proposed method uses either the upper or left template to derive the model coefficients, further reducing the computational complexity of this pattern.

[0428] Based on other examples of the techniques provided in this paper, reference pixels in the template are selected to increase the relevance between the reference pixels and the current block. Up to W2 reference pixels from the upper template are used. Up to H2 reference pixels from the left template are used. Thus, reference pixels that are far from the W2 reference pixels in the upper template or the H2 reference pixels in the left template have low relevance to the current block and are therefore not used.

[0429] Furthermore, when deriving model coefficients using the MaxMin method, only available reference pixels are used, and unavailable reference pixels are not replaced by padding. For example, only available pixels in the upper luminance template are checked to determine the maximum and minimum luminance values ​​in the CCLM_T mode. Because W2 reference pixels in the upper template are used, the number of available pixels (denoted as W3) is less than or equal to W2. Similarly, available pixels in the left luminance template are checked to determine the maximum and minimum values ​​in the CCLM_L mode, and the number of available pixels (denoted as H3) can be less than or equal to H2. The relationships between W2 and W3, and between H2 and H3, are as follows: Figure 16 As shown, where W3≤W2, H3≤H2. In one example, W2=2×W, H2=2×H.

[0430] Alternatively, both W2 and H2 can be equal to W+H. In other words, the model coefficients for the CCLM_T mode can be derived using up to (W+H) reference pixels in the upper luminance template, and the model coefficients for the CCLM_L mode can be derived using up to W+H reference pixels in the left luminance template. Among these reference pixels, only the available luminance template pixels within this range (i.e., W+H in both CCLM_T and CCLM_L) are examined to determine the maximum and minimum luminance values. In this example, the available pixels in the upper luminance template (less than or equal to W+H) are examined to determine the maximum and minimum values ​​in the CCLM_T mode. The available pixels in the left luminance template (less than or equal to W+H) are examined to determine the maximum and minimum values ​​in the CCLM_L mode.

[0431] Existing MDLM methods use exactly (W+H) reference pixels to derive the model coefficients for CCLM_T and CCLM_L modes. In contrast, the proposed method uses at most (2×W) or (W+H) reference pixels to derive the model coefficients for CCLM_T mode and at most (2×H) or (W+H) reference pixels to derive the model coefficients for CCLM_L mode. Furthermore, the maximum and minimum values ​​are determined using only available luminance reference pixels within an examined pixel range (2×W or W+H for CCLM_T, and 2×H or W+H for CCLM_L). In the proposed method, sampling reference pixels with a step size greater than 1 (e.g., 2, 4, or other values) speeds up the determination of the maximum and minimum luminance values ​​in the luminance template.

[0432] downsampling method

[0433] As mentioned above, because the spatial resolution of the luminance component in an image is greater than that of the chrominance component, in MDLM mode, the luminance component needs to be downsampled to the resolution of the chrominance component. For example, in YUV4:2:0 format, the luminance component needs to be downsampled by 4 (width divided by 2, height divided by 2) to match the resolution of the chrominance component. In MDLM mode, prediction of the chrominance block can be performed using a downsampled luminance block (with the same spatial resolution as the chrominance block) corresponding to the chrominance block. The size of the downsampled luminance block is equal to the size of the chrominance block (because the luminance block is downsampled to the size of the chrominance block).

[0434] The reference pixels in the luminance template also need to be downsampled to derive the linear model coefficients for the MDLM mode. In the CCLM_T mode, the adjacent reconstructed luminance pixels above are downsampled to generate reference pixels in the upper template. These reference pixels correspond to the reference pixels in the upper chrominance template (i.e., the adjacent reconstructed chrominance pixels above). Downsampling the adjacent reconstructed luminance pixels above typically involves multiple rows of adjacent reconstructed luminance pixels above. Figure 17 This is a schematic diagram illustrating an example of downsampling luminance pixels using multiple rows or columns of adjacent luminance pixels. For example... Figure 17 As shown, for a luminance block, two adjacent rows A1 and A2 above can be used during the downsampling process to obtain the downsampled adjacent row A. A[i] represents the i-th pixel in A, A1[i] represents the i-th pixel in A1, and A2[i] represents the i-th pixel in A2. A 6-tap downsampling filter can be used, as shown below:

[0435] A[i]=(A2[2i]×2+A2[2i–1]+A2[2i+1]+A1[2i]×2+A1[2i–1]+A1[2i+1]+4)>>3.

[0436] The number of adjacent pixels can also be greater than the size of the current block. For example, such as Figure 18 As shown, the number of adjacent pixels above the downsampled luminance block can be M, where M is greater than the width W of the downsampled luminance block.

[0437] In the existing downsampling method described above, multiple rows of upper adjacent reconstructed luminance pixels are used to generate downsampled upper adjacent reconstructed luminance pixels. Compared to conventional intra-frame mode prediction, this increases the size of the column buffer, thereby increasing memory overhead.

[0438] If the current block is located above the top boundary of the current coding tree unit (CTU) (i.e., the top row of the current chroma block overlaps with the top row of the current CTU), then in CCLM_T mode, only the top adjacent row of luminance pixels is used to reconstruct the luminance pixels. This reduces the memory footprint of downsampling. Figure 19 This is a schematic diagram illustrating an example of downsampling using a single row of adjacent luminance pixels of a luminance block located at the upper boundary of the CTU. (See diagram below.) Figure 19 As shown, only A1 (including a row of adjacent reconstructed luminance pixels) is used to generate the downsampled upper adjacent reconstructed luminance pixels in A.

[0439] It should be understood that while the above description focuses on the upper adjacent reconstructed luminance pixels in CCLM_T mode, a similar approach can be applied to the left adjacent reconstructed luminance pixels in CCLM_L mode. For example, if the current block is located on the left boundary of the CTU (i.e., the left column of the current chroma block overlaps with the left column of the CTU), instead of using multiple columns of left adjacent reconstructed luminance pixels during downsampling, a single column of left adjacent reconstructed luminance pixels is used to generate the reference pixel in the left template of the downsampled luminance block.

[0440] Determining the availability of reference pixels

[0441] In the example above, a luma template is used to determine the availability of reference pixels, thereby determining the maximum and minimum luma values. However, in some scenarios, available luma reference pixels do not have corresponding chroma reference pixels. For example, luma and chroma blocks may be decoded separately. Thus, when a luma block is reconstructed to be available, the corresponding chroma block remains unavailable. Therefore, the lack of corresponding available chroma reference pixels for available luma reference pixels leads to decoding errors.

[0442] The technique presented in this paper addresses this problem by determining the availability of reference pixels in a chroma template by examining the availability of reference pixels within that template. In some examples, a chroma reference pixel is available if it is within the current image, stripe, or block, and has already been reconstructed. In other examples, a chroma reference pixel is available if it is within the current image, stripe, or block, has already been reconstructed, and is not omitted based on encoding decisions, etc. Available reference pixels for the current chroma block can be available adjacent reconstructed pixels of the chroma block. The maximum and minimum luminance values ​​are determined using the luminance reference pixels corresponding to the available chroma reference pixels. For example, if L reference pixels in the chroma template are available, then the maximum and minimum luminance values ​​are determined using the reference pixels in the luminance template corresponding to the L available chroma reference pixels. By identifying luminance reference pixels in the luminance template that are located at the same position as chrominance reference pixels in the chrominance template (i.e., juxtaposed luminance reference pixels (position (x, y))), one or more luminance reference pixels corresponding to that chrominance reference pixel can be determined. For example, one or more luminance reference pixels corresponding to a chrominance reference pixel may include adjacent luminance reference pixels (position (x-1, y)), luminance reference pixels (position (x, y)), and adjacent luminance reference pixels (position (x+1, y)). Then, the model coefficients described above are determined using luminance reference pixels with maximum and minimum luminance values, as well as chrominance reference pixels corresponding to the luminance reference pixels (related to the maximum and minimum luminance values).

[0443] In one example, L available chroma reference pixels can be used to determine the model coefficients, for example, L = 4. In another example, a subset of the L available chroma reference pixels is used to determine the model coefficients. For example, a fixed number of available chroma reference pixels are selected from the L available chroma reference pixels. Luminance reference pixels corresponding to the selected chroma reference pixels are identified, and these luminance reference pixels and the selected chroma reference pixels are used to determine the model coefficients. For example, if the selected chroma reference pixels are 4 available chroma reference pixels, then 24 neighboring reconstructed luminance pixels corresponding to these 4 available chroma reference pixels are identified. The luminance reference pixels used to determine the model coefficients are obtained by downsampling these 24 neighboring reconstructed luminance pixels, where a 6-tap filter is used during the downsampling process.

[0444] exist Figure 16 In the example shown, the range of upper stencil pixels in the CCLM_T mode is W2. Then, the availability of reference pixels in the upper chroma stencil of the current chroma block is determined. If W3 chroma reference pixels are available (W3 ≤ W2), then W3 or fewer luminance reference pixels are obtained. These obtained luminance and chroma reference pixels are used to derive the model coefficients for the CCLM_T mode.

[0445] Similarly, again Figure 16 In the example shown, the left stencil pixel range in the CCLM_L mode is H2. The availability of reference pixels in the left chroma stencil of the current chroma block is checked. If H3 chroma reference pixels are available (H3 ≤ H2), then H3 or fewer luma reference pixels are obtained. These obtained luma and chroma reference pixels are used to derive the model coefficients for the CCLM_L mode. Similarly, the availability of reference pixels in the CCLM mode can be determined by determining the availability of reference pixels in the upper and left chroma stencils, and then finding the corresponding luma reference pixels to determine the model coefficients as described above.

[0446] Table 1 describes the details of the provided method in the format of the INTRA_CCLM, INTRA_CCLM_L, or INTRA_CCLM_T intra-prediction mode specification. Table 2 shows an alternative implementation of the method provided in this paper.

[0447] Table 1

[0448]

[0449]

[0450]

[0451]

[0452] Table 2

[0453]

[0454]

[0455] Binarization of MDLM mode

[0456] To encode MDLM modes into the video signal bitstream, the MDLM modes need to be binarized so that the selected MDLM mode can be encoded into the bitstream and the decoder can determine and decode the selected mode. Existing binarization methods do not include the two chroma modes of MDLM, namely CCLM_L and CCLM_T. A new chroma mode decoding method is presented here.

[0457] Tables 3 and 4 provide details on the binarization of these two chroma modes. In Table 3, 77 indicates CCLM mode with an index of intra_chroma_pred_mode of 4; 78 indicates CCLM_L mode with an index of intra_chroma_pred_mode of 5; and 79 indicates CCLM_T mode with an index of intra_chroma_pred_mode of 6. If the index intra_chroma_pred_mode is 7, the selected mode is DM mode. The remaining index values ​​0, 1, 2, and 3 represent planar mode, vertical mode, horizontal mode, and DC mode, respectively.

[0458] Table 3: Detailed description of IntraPredModeC[xCb][yCb] (depending on intra_chroma_pred_mode[xCb][yCb] and IntraPredModeY[xCb][yCb]) when sps_cclm_enabled_flag is 1

[0459] Table 4: An example of binarization of intra_chroma_pred_mode when sps_cclm_enabled_flag is 1

[0460]

[0461]

[0462] Table 4 shows an example of the bit string or syntax element used for each chroma intra-prediction mode. As shown in Table 4, the syntax element is 0 for DM mode (index 7), 10 for CCLM mode (index 4), 1110 for CCLM_L mode (index 5), 1111 for CCLM_T mode (index 6), 11000 for planar mode (index 0), 11001 for vertical mode (index 1), 11010 for horizontal mode (index 2), and 11011 for DC mode (index 3). Table 5 shows another example of the bit string or syntax element used for each chroma intra-prediction mode. The corresponding syntax element is carried in the encoded video bitstream according to the decoding mode selected by the encoder.

[0463] Table 5: Another example of binarization of intra_chroma_pred_mode when sps_cclm_enabled_flag is 1

[0464] The value of intra_chroma_pred_mode binary string 7 00 4 10 5 110 6 111 0 0100 1 0101 2 0110 3 0111

[0465] When the video encoder (e.g.) Figure 1A When the video encoder 20) performs intra-frame prediction on chroma blocks in the video signal according to the intra-frame chroma prediction mode, the video encoder selects the intra-frame chroma prediction mode and carries the syntax element indicating the selected intra-frame chroma prediction mode in the bitstream to generate the bitstream of the video signal. The video encoder can select the intra-frame chroma prediction mode from multiple sets of modes. For example, these modes may include a first set of modes, which includes derived mode (DM) or cross component linear model (CCLM) prediction mode, or both. These modes may also include a second set of modes, which includes at least one of CCLM_L mode or CCLM_T mode. These modes may also include a third set of modes, which may include at least one of vertical mode, horizontal mode, DC mode, or planar mode.

[0466] In some examples, the number of bits in the syntax element of the intra-chroma prediction mode selected from the first group of modes is less than the number of bits in the syntax element of the intra-chroma prediction mode selected from the second group of modes. Furthermore, the number of bits in the syntax element of the intra-chroma prediction mode when selected from the second group of modes is less than the number of bits in the syntax element of the intra-chroma prediction mode when selected from the third group of modes. In some examples, the syntax elements of various intra-chroma prediction modes are selected according to the examples shown in Table 4 or Table 5.

[0467] When the decoder (e.g.) Figure 1A When the video decoder 30 receives the encoded bitstream from the video signal to decode the video, the decoder parses syntax elements from the bitstream of the video signal and determines the intra-frame chroma prediction mode used by the chroma block based on a syntax element selected from the parsed syntax elements. Based on the determined intra-frame chroma prediction mode, the decoder performs intra-frame prediction on the current chroma block in the video signal.

[0468] Figure 20 This is a flowchart illustrating a method for performing intra-frame prediction using a linear model, provided for some aspects of the present invention. In step 2002, a luma block (e.g., luma block 811) corresponding to the current chroma block (e.g., chroma block 801) is determined.

[0469] In step 2004, a luminance reference pixel for the luminance block is obtained based on the L available chroma reference pixels of the current chroma block. The obtained luminance reference pixel for the luminance block is a downsampled luminance reference pixel. In some examples, the luminance reference pixel for the luminance block is a downsampled luminance reference pixel obtained by downsampling adjacent luminance pixels selected based on the L available chroma reference pixels (e.g., a subset or all of the L available chroma reference pixels). In other words, the luminance reference pixel for the luminance block is a downsampled luminance reference pixel obtained by downsampling adjacent luminance pixels corresponding to the available chroma reference pixels. In some examples, the obtained luminance reference pixel corresponds to the L available chroma reference pixels. In other examples, the obtained luminance reference pixel corresponds to a subset of the L available chroma reference pixels. It is understood that the correspondence between the obtained luminance reference pixels (i.e., the downsampled luminance reference pixels) and the L available chrominance reference pixels is not limited to a one-to-one correspondence; it can also be understood that the correspondence between the obtained luminance reference pixels (i.e., the downsampled luminance reference pixels) and the L available chrominance reference pixels can be an "M to N correspondence". For example, M = 4, N = 4; or, M = 4, N > 4.

[0470] In some examples, the chroma reference pixels of the current chroma block include the adjacent reconstructed pixels of the current chroma block. The L available chroma reference pixels are determined from the adjacent reconstructed pixels. Similarly, the adjacent pixels of the luma block are also the adjacent reconstructed pixels of the luma block. The luma reference pixels of the luma block are obtained by downsampling the adjacent reconstructed pixels selected based on the L available chroma reference pixels. For example, L = 4.

[0471] In some examples, a chroma reference pixel is available if it is within the current image, stripe, or block, and has already been reconstructed. In other examples, a chroma reference pixel is available if it is within the current image, stripe, or block, has already been reconstructed, and is not omitted according to a coding decision, etc. The available reference pixel for the current chroma block can be a available reconstructed neighboring pixel of the chroma block. A luminance reference pixel corresponding to the available chroma reference pixel is obtained.

[0472] In some examples, determining the L available chroma reference pixels includes: determining that the L upper adjacent chroma pixels of the current chroma block are available, where 1 ≤ L ≤ W2, and L and W2 are positive integers. W2 represents the range of upper reference pixels, and the L upper adjacent chroma pixels are used as the available chroma reference pixels. In some examples, W2 is equal to 2 × W or W + H. Here, W represents the width of the current chroma block, and H represents the height of the current chroma block.

[0473] In other examples, determining the L available chroma reference pixels includes: determining that the L left-adjacent chroma pixels of the current chroma block are available. Here, 1 ≤ L ≤ H2, where L and H2 are positive integers. H2 represents the range of left-adjacent reference pixels. The L left-adjacent chroma pixels are used as the available chroma reference pixels. In some examples, H2 is equal to 2 × H or W + H. W represents the width of the current chroma block, and H represents the height of the current chroma block.

[0474] In other examples, determining the L available chroma reference pixels includes: determining that L1 upper adjacent chroma pixels and L2 left adjacent chroma pixels of the current chroma block are available. Here, 1 ≤ L1 ≤ W2, 1 ≤ L2 ≤ H2. W2 represents the range of upper reference pixels, and H2 represents the range of left reference pixels. L1, L2, W2, and H2 are positive integers, and L1 + L2 = L. In these examples, the L1 upper adjacent chroma pixels and the L2 left adjacent chroma pixels are used as the available chroma reference pixels.

[0475] In one example, the luminance reference pixel is obtained by downsampling only the adjacent pixels above the luminance block selected according to the L available chroma reference pixels. In another example, the luminance reference pixel is obtained by downsampling only the adjacent pixels to the left of the luminance block selected according to the L available chroma reference pixels.

[0476] In the above example, a downsampled luminance block of the luminance block is obtained by downsampling the reconstructed luminance block of the luminance block corresponding to the current chroma block. In some cases, such as when the luminance reference pixel is obtained only based on the adjacent pixels above the luminance block, and when the top row of the current chroma block overlaps with the top row of the current coding tree unit (CTU), the luminance reference pixel is obtained using only one row of adjacent reconstructed luminance pixels of the reconstructed luminance block of the luminance block.

[0477] In step 2006, the linear model coefficients used for inter-component prediction are calculated based on the luminance reference pixel obtained in step 2004 and the chrominance reference pixel corresponding to the luminance reference pixel. In some examples, the chrominance reference pixel corresponding to the luminance reference pixel is a chrominance reference pixel juxtaposed with the luminance reference pixel.

[0478] In step 2008, the predicted value of the current chroma block is generated based on the calculated linear model coefficients and the value of the downsampled chroma block obtained by downsampling the chroma block (e.g., chroma block 811).

[0479] Figure 21 A flowchart of a method for predicting a cross-component linear model (CCLM) provided for other aspects of the present invention. In step 2102, a luminance block (e.g., luminance block 811) corresponding to the current chrominance block (e.g., chrominance block 801) is determined.

[0480] In step 2104, a luminance reference pixel for the luminance block is obtained by downsampling the adjacent pixels of the luminance block. In some examples, the luminance reference pixel only includes luminance reference pixels obtained based on the adjacent pixels above the luminance block. In other examples, the luminance reference pixel only includes luminance reference pixels obtained based on the adjacent pixels to the left of the luminance block.

[0481] In step 2106, the maximum brightness value and the minimum brightness value are determined based on the brightness reference pixel.

[0482] In step 2108, a first chromaticity value is obtained at least in part based on one or more locations of one or more luminance reference pixels associated with the maximum luminance value. A second chromaticity value is also obtained at least in part based on one or more locations of one or more luminance reference pixels associated with the minimum luminance value.

[0483] In step 2110, the linear model coefficients are calculated based on the first chromaticity value, the second chromaticity value, the maximum luminance value, and the minimum luminance value.

[0484] In step 2112, the predicted value of the current chroma block is generated based on the linear model coefficients and the downsampled chroma block value of the chroma block.

[0485] Figure 22 This is a block diagram illustrating an exemplary structure of an apparatus 2200 for performing intra-frame prediction using a linear model. The apparatus 2200 may include a determination unit 2202 and an intra-frame prediction processing unit 2204. In one example, the apparatus 2200 may be coupled with… Figure 2 This corresponds to the intra-frame prediction unit 254. In another example, the device 2200 can be associated with... Figure 3 The intra-prediction unit 354 in the frame corresponds to this.

[0486] The determining unit 2202 is used to determine the luminance block (e.g., block 811) corresponding to the current chroma block (e.g., chroma block 801). The determining unit 2202 is also used to obtain the luminance reference pixels of the luminance block based on determining L available chroma reference pixels in the current chroma block. The obtained luminance reference pixels of the luminance block are downsampled luminance reference pixels.

[0487] In some examples, the chroma reference pixel of the current chroma block includes the adjacent reconstructed pixels of the current chroma block. The L available chroma reference pixels are determined from the adjacent reconstructed pixels. Similarly, the adjacent pixels of the luminance block are also the adjacent reconstructed pixels of the luminance block. The luminance reference pixel of the luminance block is obtained by downsampling the adjacent reconstructed pixels of the luminance block. In some examples, the luminance reference pixel of the luminance block is a downsampled luminance reference pixel obtained by downsampling the adjacent reconstructed pixels of the luminance block selected according to the L available chroma reference pixels. In some examples, the luminance reference pixel of the luminance block is a downsampled luminance reference pixel obtained by downsampling the adjacent reconstructed pixels corresponding to the L available chroma reference pixels.

[0488] In some examples, a chroma reference pixel is available if it is within the current image, stripe, or block, has already been reconstructed, and is not omitted based on encoding decisions, etc. The available reference pixel for the current chroma block can be a available reconstructed neighboring pixel of that chroma block. The luminance reference pixel corresponding to the available chroma reference pixel is obtained.

[0489] In some examples, determining the L available chroma reference pixels includes: determining that the L upper adjacent chroma pixels of the current chroma block are available, where 1 ≤ L ≤ W2, and L and W2 are positive integers. W2 represents the range of upper reference pixels, and the L upper adjacent chroma pixels are used as the available chroma reference pixels. In some examples, W2 is equal to 2 × W or W + H. Here, W represents the width of the current chroma block, and H represents the height of the current chroma block.

[0490] In other examples, determining the L available chroma reference pixels includes: determining that the L left-adjacent chroma pixels of the current chroma block are available. Here, 1 ≤ L ≤ H2, where L and H2 are positive integers. H2 represents the range of left-adjacent reference pixels. The L left-adjacent chroma pixels are used as the available chroma reference pixels. In some examples, H2 is equal to 2 × H or W + H. W represents the width of the current chroma block, and H represents the height of the current chroma block.

[0491] In other examples, determining the L available chroma reference pixels includes: determining that L1 upper adjacent chroma pixels and L2 left adjacent chroma pixels of the current chroma block are available. Here, 1 ≤ L1 ≤ W2, 1 ≤ L2 ≤ H2. W2 represents the range of upper reference pixels, and H2 represents the range of left reference pixels. L1, L2, W2, and H2 are positive integers, and L1 + L2 = L. In these examples, the L1 upper adjacent chroma pixels and the L2 left adjacent chroma pixels are used as the available chroma reference pixels.

[0492] In one example, the luminance reference pixel is obtained by downsampling only the adjacent pixels selected from the L available chroma reference pixels above the luminance block. In another example, the luminance reference pixel is obtained by downsampling only the adjacent pixels selected from the L available chroma reference pixels to the left of the luminance block.

[0493] In the above example, a downsampled luminance block of the luminance block is obtained by downsampling the reconstructed luminance block of the luminance block corresponding to the current chroma block. In some cases, such as when the luminance reference pixel is obtained only based on the adjacent pixels above the luminance block, and when the top row of the current chroma block overlaps with the top row of the current coding tree unit (CTU), the luminance reference pixel is obtained using only one row of adjacent reconstructed luminance pixels of the reconstructed luminance block of the luminance block.

[0494] The intra-frame prediction processing unit 2204 is used to calculate linear model coefficients (e.g., α and β) based on the luminance reference pixel and the chrominance reference pixel corresponding to the luminance reference pixel. The intra-frame prediction processing unit 2204 is also used to obtain the predicted value of the current chrominance block based on the linear model coefficients and the downsampled value of the luminance block.

[0495] Figure 23 A flowchart illustrating a method for encoding chroma intra-frame decoding modes into a video signal bitstream, provided for some aspects of the present invention.

[0496] In step 2302, intra-frame prediction is performed on the chroma blocks in the video signal according to the intra-frame chroma prediction mode. The intra-frame chroma prediction mode can be selected from multiple modes. In some examples, the multiple modes include three groups: a first group of modes, including at least one of derived mode (DM) or cross component linear model (CCLM) prediction mode; a second group of modes, including at least one of CCLM_L mode or CCLM_T mode; or a third group of modes, including at least one of vertical mode, horizontal mode, DC mode, or planar mode.

[0497] In step 2304, the bitstream of the video signal is generated by carrying a syntax element indicating the intra-frame chroma prediction mode in the bitstream. In some examples, the number of bits in the syntax element when selecting the intra-frame chroma prediction mode from the first set of modes is less than the number of bits in the syntax element when selecting the intra-frame chroma prediction mode from the second set of modes. The number of bits in the syntax element when selecting the intra-frame chroma prediction mode from the second set of modes is less than the number of bits in the syntax element when selecting the intra-frame chroma prediction mode from the third set of modes.

[0498] In one example, the syntax element for the DM mode is 0. The syntax element for the CCLM mode is 10. The syntax element for the CCLM_L mode is 1110. The syntax element for the CCLM_T mode is 1111. The syntax element for the planar mode is 11000. The syntax element for the vertical mode is 11001. The syntax element for the horizontal mode is 11010. The syntax element for the DC mode is 11011.

[0499] In another example, the syntax element for the DM mode is 00. The syntax element for the CCLM mode is 10. The syntax element for the CCLM_L mode is 110. The syntax element for the CCLM_T mode is 111. The syntax element for the planar mode is 0100. The syntax element for the vertical mode is 0101. The syntax element for the horizontal mode is 0110. The syntax element for the DC mode is 0111.

[0500] Figure 24 A flowchart illustrating a method for decoding chroma intra-frame decoding modes in a video signal bitstream, provided for some aspects of the present invention.

[0501] In step 2402, multiple syntax elements are parsed from the video signal bitstream. In step 2404, the intra-frame chroma prediction mode is determined based on a syntax element among the multiple syntax elements that indicates an intra-frame chroma prediction mode. In some examples, the intra-frame chroma prediction mode is determined from multiple modes. For example, the multiple modes include three groups: a first group of modes including at least one of derived mode (DM) or cross-component linear model (CCLM) prediction modes; a second group of modes including at least one of CCLM_L mode or CCLM_T mode; or a third group of modes including at least one of vertical mode, horizontal mode, DC mode, or planar mode. In these groups of intra-frame chroma prediction modes, the number of bits of the syntax element when selecting the intra-frame chroma prediction mode from the first group of modes is less than the number of bits of the syntax element when selecting the intra-frame chroma prediction mode from the second group of modes. The number of bits of the syntax element when selecting the intra-frame chroma prediction mode from the second group of modes is less than the number of bits of the syntax element when selecting the intra-frame chroma prediction mode from the third group of modes.

[0502] In step 2406, intra-frame prediction is performed on the current chroma block in the video signal according to the intra-frame chroma prediction mode.

[0503] Figure 25 This is a block diagram of an exemplary structure of an apparatus 2500 for generating a video bitstream. The apparatus 2500 may include an intra-frame prediction processing unit 2502 and a binarization unit 2504. In one example, the intra-frame prediction processing unit 2502 may be coupled with... Figure 2 The intra-prediction unit 254 corresponds to this. In one example, the binarization unit 2504 can be associated with... Figure 2 The entropy coding unit 270 in the code corresponds to this.

[0504] The intra-frame prediction processing unit 2502 is configured to perform intra-frame prediction on chroma blocks in the video signal according to an intra-frame chroma prediction mode. The intra-frame chroma prediction mode is selected from a first set of modes, a second set of modes, or a third set of modes. The first set of modes includes at least one of derived mode (DM) or cross-component linear model (CCLM) prediction mode. The second set of modes includes at least one of CCLM_L mode and CCLM_T mode. The third set of modes includes at least one of vertical mode, horizontal mode, DC mode, and planar mode.

[0505] The binarization unit 2504 is used to generate a bitstream of the video signal by carrying syntax elements that indicate the intra-frame chroma prediction mode. The number of bits in the syntax element when selecting the intra-frame chroma prediction mode from the first set of modes is less than the number of bits in the syntax element when selecting the intra-frame chroma prediction mode from the second set of modes, and the number of bits in the syntax element when selecting the intra-frame chroma prediction mode from the second set of modes is less than the number of bits in the syntax element when selecting the intra-frame chroma prediction mode from the third set of modes.

[0506] In one example, the syntax element for the DM mode is 0. The syntax element for the CCLM mode is 10. The syntax element for the CCLM_L mode is 1110. The syntax element for the CCLM_T mode is 1111. The syntax element for the planar mode is 11000. The syntax element for the vertical mode is 11001. The syntax element for the horizontal mode is 11010. The syntax element for the DC mode is 11011.

[0507] In another example, the syntax element for the DM mode is 00. The syntax element for the CCLM mode is 10. The syntax element for the CCLM_L mode is 110. The syntax element for the CCLM_T mode is 111. The syntax element for the planar mode is 0100. The syntax element for the vertical mode is 0101. The syntax element for the horizontal mode is 0110. The syntax element for the DC mode is 0111.

[0508] Figure 26 This is a block diagram of an exemplary structure of an apparatus 2600 for decoding a video bitstream. The apparatus may include a parsing unit 2602, a determining unit 2604, and an intra-frame prediction processing unit 2606. In one example, the parsing unit 2602 may be combined with… Figure 3 This corresponds to the entropy coding unit 304 in the model. In one example, the determination unit 2604 and the intra-frame prediction processing unit 2606 can be associated with... Figure 3 The intra-prediction unit 354 in the frame corresponds to this.

[0509] Parsing unit 2602 is used to parse multiple syntax elements from the bitstream of the video signal. Determination unit 2604 is used to determine an intra-frame chroma prediction mode based on one of the multiple syntax elements. The intra-frame chroma prediction mode is determined from a first set of modes, a second set of modes, or a third set of modes. The first set of modes includes at least one of derived mode (DM) or cross-component linear model (CCLM) prediction mode. The second set of modes includes at least one of CCLM_L mode and CCLM_T mode. The third set of modes includes at least one of vertical mode, horizontal mode, DC mode, and planar mode.

[0510] The number of bits in the syntax element when selecting the intra-chroma prediction mode from the first group of modes is less than the number of bits in the syntax element when selecting the intra-chroma prediction mode from the second group of modes, and the number of bits in the syntax element when selecting the intra-chroma prediction mode from the second group of modes is less than the number of bits in the syntax element when selecting the intra-chroma prediction mode from the third group of modes.

[0511] In one example, the syntax element for the DM mode is 0. The syntax element for the CCLM mode is 10. The syntax element for the CCLM_L mode is 1110. The syntax element for the CCLM_T mode is 1111. The syntax element for the planar mode is 11000. The syntax element for the vertical mode is 11001. The syntax element for the horizontal mode is 11010. The syntax element for the DC mode is 11011.

[0512] In another example, the syntax element for the DM mode is 00. The syntax element for the CCLM mode is 10. The syntax element for the CCLM_L mode is 110. The syntax element for the CCLM_T mode is 111. The syntax element for the planar mode is 0100. The syntax element for the vertical mode is 0101. The syntax element for the horizontal mode is 0110. The syntax element for the DC mode is 0111.

[0513] The intra-frame prediction processing unit 2606 is used to perform intra-frame prediction on the current chroma block in the video signal according to the intra-frame chroma prediction mode.

[0514] To better understand this invention, the following references are incorporated herein by reference: JCTVC-H0544: Description of MDLM, JVET-G1001: Description of CCLM or LM in Section 2.2.4, JVET-K0204: Description of model coefficients derived from maximum and minimum values.

[0515] The following explains the application of the encoding and decoding methods shown in the above embodiments, as well as the systems that use these applications.

[0516] Figure 27 This is a block diagram of a content delivery system 3100 used to implement content provisioning services. The content delivery system 3100 includes a capture device 3102, a terminal device 3106, and (optionally) a display 3126. The capture device 3102 communicates with the terminal device 3106 via a communication link 3104. This communication link may include the communication channel 13 described above. The communication link 3104 includes, but is not limited to, Wi-Fi, Ethernet, wired, wireless (3G / 4G / 5G), USB, or any combination thereof.

[0517] The capture device 3102 generates data and can encode the data using the encoding method shown in the above embodiments. Alternatively, the capture device 3102 can distribute the data to a streaming media server (not shown), which encodes the data and transmits the encoded data to the terminal device 3106. The capture device 3102 includes, but is not limited to, cameras, smartphones or tablets, computers or laptops, video conferencing systems, PDAs, in-vehicle devices, or any combination thereof. For example, the capture device 3102 may include the source device 12 described above. When the data includes video, the video encoder 20 included in the capture device 3102 can actually perform video encoding processing. When the data includes audio (i.e., sound), the audio encoder included in the capture device 3102 can actually perform audio encoding processing. In some practical scenarios, the capture device 3102 distributes the encoded video data and the encoded audio data together. In other practical scenarios, such as in a video conferencing system, the encoded audio data and the encoded video data are not multiplexed. The capture device 3102 distributes the encoded audio data and the encoded video data to the terminal device 3106 respectively.

[0518] Terminal device 310 in content distribution system 3100 receives and reproduces encoded data. Terminal device 3106 can be a device capable of receiving and reproducing data, such as a smartphone or tablet 3108, a computer or laptop 3110, a network video recorder (NVR) / digital video recorder (DVR) 3112, a television 3114, a set-top box (STB) 3116, a video conferencing system 3118, a video surveillance system 3120, a personal digital assistant (PDA) 3122, an in-vehicle device 3124, or any combination thereof, or similar devices capable of decoding the aforementioned encoded data. For example, terminal device 3106 may include destination device 14 as described above. When the encoded data includes video, the video decoder 30 included in the terminal device preferentially performs video decoding. When the encoded data includes audio, the audio decoder included in the terminal device preferentially performs audio decoding processing.

[0519] For terminal devices with displays, such as smartphones or tablets 3108, computers or laptops 3110, network video recorders (NVRs) / digital video recorders (DVRs) 3112, televisions 3114, personal digital assistants (PDAs) 3122, or in-vehicle devices 3124, the terminal device can input decoded data to its display. For terminal devices without displays, such as STBs 3116, video conferencing systems 3118, or video surveillance systems 3120, an external display 3126 is connected to the terminal device to receive and display the decoded data.

[0520] When performing encoding or decoding, the various devices in this system can use the image encoding device or image decoding device shown in the above embodiments.

[0521] Figure 28This is a schematic diagram illustrating an example structure of terminal device 3106. After terminal device 3106 receives a stream from capture device 3102, protocol processing unit 3202 analyzes the transport protocol of the stream. This protocol includes, but is not limited to, Real-Time Streaming Protocol (RTSP), Hypertext Transfer Protocol (HTTP), HTTP Live Streaming Protocol (HLS), MPEG-DASH, Real-time Transport Protocol (RTP), Real-Time Messaging Protocol (RTMP), or any combination thereof.

[0522] After the protocol processing unit 3202 processes the stream, a stream file is generated. This stream file is then output to the demultiplexing unit 3204. The demultiplexing unit 3204 can separate the multiplexed data into encoded audio data and encoded video data. As mentioned above, in some practical scenarios, such as in a video conferencing system, the encoded audio data and encoded video data are not multiplexed. In this case, the encoded data is sent to the video decoder 3206 and audio decoder 3208 without going through the demultiplexing unit 3204.

[0523] Through demultiplexing, a video elementary stream (ES), an audio ES, and (optionally) subtitles are generated. Video decoder 3206 includes video decoder 30 as explained in the above embodiments, decodes the video ES to generate video frames using the decoding method shown in the above embodiments, and inputs this data to synchronization unit 3212. Audio decoder 3208 decodes the audio ES to generate audio frames and inputs this data to synchronization unit 3212. Alternatively, the video frames can be stored in a buffer (…). Figure 28 The audio frames are then input to the synchronization unit 3212 (not shown in the image). Similarly, audio frames can be stored in a buffer (not shown in the image). Figure 28 (not shown in the image) and then input to the synchronization unit 3212.

[0524] Synchronization unit 3212 synchronizes video and audio frames and provides the video / audio to video / audio display 3214. For example, synchronization unit 3212 enables the synchronous presentation of video and audio information. Information can be encoded into the syntax using timestamps associated with the presentation of decoded audio and visual data and timestamps associated with the distribution of the data stream.

[0525] If the bitstream includes subtitles, the subtitle decoder 3210 decodes the subtitles, synchronizes the subtitles with the video and audio frames, and provides the video / audio / subtitles to the video / audio / subtitle display 3216.

[0526] This invention is not limited to the system described above. The image encoding device or image decoding device in the above embodiments can be used in other systems such as automotive systems.

[0527] By way of example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, disk storage or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Furthermore, any connection may be appropriately referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but rather refer to non-transient tangible storage media. The disks and optical discs used herein include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), and Blu-ray discs, wherein disks typically reproduce data magnetically, while optical discs utilize lasers to reproduce data optically. Combinations of the above items should also be included within the scope of computer-readable media.

[0528] Instructions can be executed by one or more processors, such as digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, the term "processor" as used herein can refer to any of the above-described structures or any other structure suitable for implementing the techniques described herein. Furthermore, in some aspects, the various functions described herein can be provided within dedicated hardware and / or software modules for encoding and decoding, or incorporated into a combinational decoder. Moreover, these techniques can be fully implemented in one or more circuit or logic elements.

[0529] The techniques of this invention can be implemented in a variety of devices or apparatuses, including wireless handheld devices, integrated circuits (ICs), or a set of ICs (e.g., chipsets). This invention describes various components, modules, or units to emphasize functional aspects of the apparatus used to perform the disclosed techniques, but these do not necessarily need to be implemented by different hardware units. Rather, as described above, various units can be combined with suitable software and / or firmware within a decoder hardware unit, or provided as a collection of interoperable hardware units including one or more processors as described above.

[0530] While several embodiments have been provided in this invention, it should be understood that the disclosed systems and methods may be embodied in many other specific forms without departing from the spirit or scope of the invention. The examples of this invention should be considered illustrative rather than restrictive, and the invention is not limited to the details set forth herein. For example, various elements or components may be combined or integrated into another system, or certain features may be omitted or not implemented.

[0531] Furthermore, without departing from the scope of this invention, the technologies, systems, subsystems, and methods described and illustrated as discrete or separate in the various embodiments can be combined or integrated with other systems, modules, technologies, or methods. Other items shown or discussed as coupled or directly coupled or communicating with each other may also be indirectly coupled or communicating via an interface, device, or intermediate component in an electrical, mechanical, or other manner. Other variations, substitutions, and alterations can be determined by those skilled in the art without departing from the spirit and scope disclosed herein.

Claims

1. A method of encoding video data, the method comprising: The method is performed by an encoding device, and the method comprises: performing intra prediction on a current chroma block of video data according to an intra chroma prediction mode, wherein the intra chroma prediction mode comprises a CCLM_T mode, and wherein the performing intra prediction on the current chroma block comprises: determining a luma block corresponding to the current chroma block; determining L available chroma reference pixels of the current chroma block by checking availability of neighboring chroma pixels above the current chroma block; obtaining luma reference pixels of the luma block by determining the L available chroma reference pixels of the current chroma block, wherein the obtained luma reference pixels are down-sampled luma reference pixels, and wherein L is a positive integer; calculating linear model coefficients according to the luma reference pixels and chroma reference pixels corresponding to the luma reference pixels; obtaining a prediction value of the current chroma block according to the linear model coefficients and values of down-sampled luma blocks of the luma block; and encoding a syntax element indicating the intra chroma prediction mode into a bitstream of the video data to generate the bitstream of the video data.

2. The method of claim 1, wherein, The determining the L available chroma reference pixels comprises: determining that L neighboring chroma pixels above the current chroma block are available, wherein 1≤L≤W2, W2 represents an upper reference pixel range, and W2 is a positive integer, and wherein the L neighboring chroma pixels above the current chroma block are used as the L available chroma reference pixels.

3. The method of claim 2, wherein, W2 is equal to 2×W or W+H, W represents a width of the current chroma block, and H represents a height of the current chroma block.

4. The method according to any one of claims 1 to 3, characterized in that, The luma reference pixels are obtained by: down-sampling only neighboring pixels selected according to the L available chroma reference pixels above the luma block.

5. The method according to any one of claims 1 to 3, characterized in that, down-sampling a reconstructed luma block of the luma block corresponding to the current chroma block to obtain the down-sampled luma block of the luma block.

6. The method of claim 5, wherein, When the luma reference pixels are obtained only according to neighboring pixels above the luma block and when a top row of the current chroma block overlaps a top row of a current coding tree unit (CTU), only one row of neighboring reconstructed luma pixels of the reconstructed luma block of the luma block is used to obtain the luma reference pixels.

7. The method according to any one of claims 1 to 3, characterized in that, The calculating linear model coefficients according to the luma reference pixels and chroma reference pixels corresponding to the luma reference pixels comprises: determining a maximum luma value and a minimum luma value according to the luma reference pixels; obtaining a first chroma value at least partially according to a position of a luma reference pixel related to the maximum luma value; obtaining a second chroma value at least partially according to a position of a luma reference pixel related to the minimum luma value; calculating the linear model coefficients according to the first chroma value, the second chroma value, the maximum luma value, and the minimum luma value.

8. The method of claim 7, wherein, The obtaining the first chroma value according to at least in part the position of the luma reference sample associated with the maximum luma value comprises: obtaining the first chroma value according to at least in part one or more positions of one or more luma reference samples associated with the maximum luma value; The obtaining the second chroma value according to at least in part the position of the luma reference sample associated with the minimum luma value comprises: obtaining the second chroma value according to at least in part one or more positions of one or more luma reference samples associated with the minimum luma value.

9. A method of decoding video data, the method comprising: The method is performed by a decoding device, and the method comprises: parsing a syntax element from a bitstream, wherein the syntax element is used to indicate an intra chroma prediction mode; determining the intra chroma prediction mode according to the syntax element, wherein the intra chroma prediction mode comprises a CCLM_T mode; performing intra prediction on a current chroma block of video data according to the intra chroma prediction mode, wherein the performing intra prediction on the current chroma block comprises: determining a luma block corresponding to the current chroma block; determining L available chroma reference samples of the current chroma block by checking availability of neighboring chroma samples above the current chroma block; obtaining luma reference samples of the luma block by determining the L available chroma reference samples of the current chroma block, wherein the obtained luma reference samples are down-sampled luma reference samples, and L is a positive integer; calculating linear model coefficients according to the luma reference samples and chroma reference samples corresponding to the luma reference samples; obtaining a prediction value of the current chroma block according to the linear model coefficients and a value of a down-sampled luma block of the luma block.

10. The method of claim 9, wherein, The determining the L available chroma reference samples comprises: determining that L neighboring chroma samples above the current chroma block are available, wherein 1≤L≤W2, W2 represents an upper reference sample range, and W2 is a positive integer, and the L neighboring chroma samples above the current chroma block are used as the L available chroma reference samples.

11. The method of claim 9, wherein, W2 is equal to 2×W or W+H, W represents a width of the current chroma block, and H represents a height of the current chroma block.

12. The method according to any one of claims 9 to 11, characterized in that, The luma reference samples are obtained by: down-sampling only neighboring samples above the luma block selected according to the L available chroma reference samples.

13. The method according to any one of claims 9 to 11, characterized in that, down-sampling a reconstructed luma block of the luma block corresponding to the current chroma block to obtain the down-sampled luma block of the luma block.

14. The method of claim 13, wherein, When the luma reference samples are obtained only according to neighboring samples above the luma block and when a top row of the current chroma block overlaps a top row of a current coding tree unit (CTU), only one row of neighboring reconstructed luma samples of the reconstructed luma block of the luma block is used to obtain the luma reference samples.

15. The method of any one of claims 9-11, wherein, The calculating linear model coefficients according to the luma reference samples and chroma reference samples corresponding to the luma reference samples comprises: determining a maximum luma value and a minimum luma value according to the luma reference samples; obtaining a first chroma value at least partially according to a position of a luma reference sample point associated with the maximum luma value; obtaining a second chroma value at least partially according to a position of a luma reference sample point associated with the minimum luma value; calculating the linear model coefficient according to the first chroma value, the second chroma value, the maximum luma value and the minimum luma value.

16. The method of claim 15, wherein, The obtaining the first chroma value at least partially according to the position of the luma reference sample point associated with the maximum luma value comprises: obtaining the first chroma value at least partially according to one or more positions of one or more luma reference sample points associated with the maximum luma value. The obtaining the second chroma value at least partially according to the position of the luma reference sample point associated with the minimum luma value comprises: obtaining the second chroma value at least partially according to one or more positions of one or more luma reference sample points associated with the minimum luma value.

17. An encoder comprising: The encoder comprises processing circuitry configured to perform the method according to claim 1.

18. A decoder, comprising: The decoder comprises processing circuitry configured to perform the method according to claim 9.

19. A non-transitory computer readable medium comprising program instructions, wherein execution of the program instructions by one or more processors of a device configured to implement a video coding process causes the device to carry out the method of any of claims 1-18. The program instructions, when executed by a computer device or a processor, cause the computer device or the processor to perform the method according to claim 1 or 9.

20. A method of storing an encoded bitstream of video data, the method comprising: The method comprises: receiving the bitstream generated according to the method of claim 1; and, storing the bitstream into a storage medium.

21. A method of transmitting an encoded bitstream of video data, the method comprising: The method comprises: obtaining a bitstream from a storage medium, the bitstream being generated according to the method of claim 1 and stored in the storage medium; and, sending the bitstream.

22. A method of transmitting an encoded bitstream of video data, the method comprising: The method comprises: receiving a bitstream, the bitstream being generated according to the method of claim 1; and, sending the bitstream.

23. An electronic device, comprising: The electronic device comprises: one or more processors; a computer readable storage medium coupled to the processors and storing program instructions for execution by the processors, wherein the program instructions, when executed by the processors, cause the electronic device to perform the method according to any one of claims 20-22.

24. A computer-readable storage medium, characterized in that, The computer readable storage medium stores an encoded video bitstream, wherein the bitstream is generated by performing the following operations: performing intra prediction for a current chroma block of video data according to an intra chroma prediction mode, wherein the intra chroma prediction mode comprises a CCLM_T mode, and wherein the performing intra prediction for the current chroma block comprises: determining a luma block corresponding to the current chroma block; determining L available chroma reference sample points of the current chroma block by checking availability of neighboring chroma sample points above the current chroma block; obtaining luma reference sample points of the luma block by determining the L available chroma reference sample points of the current chroma block, wherein the obtained luma reference sample points are down-sampled luma reference sample points, and the L is a positive integer; calculating a linear model coefficient according to the luma reference sample points and chroma reference sample points corresponding to the luma reference sample points; According to the linear model coefficients and values of the down-sampled luma block of the luma block, obtain a prediction value of the current chroma block; encode a syntax element indicating an intra chroma prediction mode into a bitstream of the video data to generate the bitstream of the video data.

Images