Weighting factors for prediction sample filtering in intra mode
By adjusting the weighting factor and enabling/disabling PDPC strategies in video encoding and decoding, the problem of undefined weighting factor in intra prediction mode is solved, improving the efficiency and quality of video encoding and decoding, and adapting to different codec modes and block shapes.
Patent Information
- Application Number
- CN202411173625.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2019-08-14
- Filing Date
- 2020-08-14
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2040-08-14
AI Technical Summary
Existing video encoding and decoding techniques have problems with undefined weighting factors in intra prediction mode, especially in planar mode and DC mode, and the lack of adjacent sample selection and weight determination mechanisms when PDPC is enabled in CCLM mode, resulting in some Wide angle mode disables PDPC, which may not be reasonable.
In planar mode or DC mode, the weighting factor adjacent to the sample points depends only on the dimension or sample point position of the video block. By adjusting the weight formula to suit different modes and block shapes, PDPC is enabled or disabled, based on the shape of the block and the predicted direction. Decide whether to apply PDPC and use the predicted signal generated by the luminance block in CCLM mode to derive the chrominance block.
It improves the accuracy and flexibility of intra prediction, ensures the rationality of weighting factors, adapts to different encoding and decoding modes and block shapes, and improves the efficiency and quality of video encoding and decoding.
Smart Images

Figure CN118921457A_ABST
Abstract
Description
[0001] This application is a divisional application of Chinese invention patent application No. 202080057350.5 filed on August 14, 2020, which is an application entering the Chinese national phase of international patent application No. PCT / CN2020 / 109217 filed on August 14, 2020, and claims priority to international patent application No. PCT / CN2019 / 100615 filed on August 14, 2019. The entire disclosure of the above application is incorporated by reference as part of the disclosure of this application. Technical Field
[0002] This document deals with video and image encoding and decoding technologies. Background Art
[0003] Digital video accounts for the largest amount of bandwidth usage on the Internet and other digital communications networks. As the number of connected user devices capable of receiving and displaying video increases, bandwidth demand for digital video usage is expected to continue to grow. Summary of the invention
[0004] The disclosed techniques may be used by video or image decoder or encoder embodiments to perform video encoding or decoding using intra-prediction sample filtering.
[0005] In one example aspect, a video processing method is disclosed. The method includes: deriving weighting factors of neighboring samples of a sample of a current video block of a video according to a rule; and performing a conversion between the current video block and a codec representation of the video, and wherein the rule stipulates that when the current video block is coded using a planar mode or a DC mode, the weighting factors are determined from at least one of the dimensions of the current video block or the positions of the samples of the current video block, wherein the current video block uses a position dependent intra prediction (PDPC) method that combines the neighboring samples with a prediction signal of the current video block to generate a refined prediction signal of the current video block, and wherein the weighting factor of the prediction signal of the sample is determined based on the weighting factors of the corresponding neighboring samples of the sample.
[0006] In yet another example aspect, the above method may be implemented by a video encoder device including a processor.
[0007] In yet another example aspect, the above method may be implemented by a video decoder device comprising a processor.
[0008] In yet another example aspect, the methods may be embodied in the form of processor-executable instructions and stored on a computer-readable program medium.
[0009] This document further describes these and other aspects. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] Figure 1 An example of an encoder block diagram is shown.
[0011] Figure 2 Examples of 67 intra prediction modes are shown.
[0012] Figure 3A and Figure 3B An example of reference samples for wide angle intra prediction is shown.
[0013] Figure 4 Shows the discontinuity problem where the orientation exceeds 45 degrees.
[0014] FIG. 5A to FIG. 5D The definition of the reference sample points is shown.
[0015] Figure 6 Examples of partitioning into 4×8 blocks and 8×4 blocks are shown.
[0016] Figure 7 Examples of partitioning of all blocks except 4×8, 8×4, and 4×4 are shown.
[0017] Figure 8 An example of the location of the sample points used for the derivation of α and β is shown.
[0018] Fig. 9 An example of a "CR" position derived from the DM of the corresponding luminance region is shown.
[0019] Fig.10 An example of neighboring samples involved along the prediction direction is shown.
[0020] Fig.11A and Fig. 11B is a block diagram of an example of a hardware platform for implementing the techniques described in this document.
[0021] Fig.12 and Fig.13 is a flow chart of an example method of video processing. DETAILED DESCRIPTION
[0022] This document provides various techniques that can be used by decoders of image or video bitstreams to improve the quality of decompressing or decoding digital video or images. For brevity, the term "video" is used herein to include both sequences of pictures (conventionally referred to as video) and individual images. In addition, video encoders can also implement these techniques during the encoding process to reconstruct decoded frames for further encoding.
[0023] The section headings used in this document are intended to facilitate understanding, rather than to limit the embodiments and techniques to the corresponding sections. Thus, embodiments from one section can be combined with embodiments from other sections.
[0024] 1. Overview
[0025] This document relates to video coding technology. Specifically, this document relates to intra-frame prediction in image / video coding. It can be applied to existing video coding standards, such as HEVC, or to-be-determined standards (Multi-function Video Codec). It can also be applied to future video coding standards or video codecs. 2. Background technology
[0027] Video codec standards have evolved primarily through the development of the well-known ITU-T and ISO / IEC standards. ITU-T developed H.261 and H.263, ISO / IEC developed MPEG-1 and MPEG-4 Vision, and the two organizations jointly developed the H.262 / MPEG-2 Video, H.264 / MPEG-4 Advanced Video Coding (AVC), and H.265 / HEVC standards. Since H.262, video codec standards have been based on a hybrid video codec structure that uses temporal prediction plus transform codec. To explore future video codec technologies beyond HEVC, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET) in 2015. Since then, JVET has adopted many new methods and applied them to reference software called the Joint Exploration Model (JEM). In April 2018, the Joint Video Experts Team (JVET) was created between VCEG (Q6 / 16) and ISO / IEC JTC1SC29 / WG11 (MPEG) to work on the VVC standard with the goal of a 50% bit rate reduction compared to HEVC.
[0028] The latest version of the VVC draft, Versatile Video Codec (Draft 6), can be found at:
[0029] http: / / phenix.it-sudparis.eu / jvet / doc_end_user / documents / 15_Gothenburg / wg11 / JVET-O2001-v14.zip
[0030] The latest reference software for VVC, called VTM, can be found at:
[0031] https: / / vcgit.hhi.fraunhofer.de / jvet / VVCSoftware_VTM / tags / VTM-5.2
[0032] 2.1 Color Space and Chroma Subsampling
[0033] A color space, also known as a color model (or color system), is an abstract mathematical model that simply describes the range of colors as a tuple of numbers, usually 3 or 4 values or color components (e.g., RGB). Basically, a color space is a description of coordinate systems and subspaces.
[0034] For video compression, the most frequently used color spaces are YCbCr and RGB.
[0035] YCbCr, Y'CbCr or Y Pb / Cb Pr / Cr (also written as YCBCR or Y'CBCR) is a series of color spaces used as part of the color image pipeline in video and digital photography systems. Y' is the luminance component, CB and CR are the blue difference and red difference chrominance components. Y' (preferred) is different from Y as luminance, which means that the light intensity is non-linearly encoded based on the gamma-corrected RGB primaries.
[0036] Chroma subsampling is the practice of encoding an image at a lower resolution for chroma information than for luminance information, taking advantage of the fact that the human visual system is less sensitive to differences in color than to luminance. 2.1.1 4:4:4
[0038] Each of the three Y'CbCr components has the same sample rate, so there is no chroma subsampling. This scheme is sometimes used in high-end film scanners and film post-production. 2.1.2 4:2:2
[0040] The two chroma components are sampled at half the rate of luma: the horizontal chroma resolution is halved. This reduces the bandwidth of the uncompressed video signal by one-third with little visual difference. 2.1.3 4:2:0
[0042] In 4:2:0, the horizontal sampling is doubled compared to 4:1:1, but the vertical resolution is halved because the Cb and Cr channels are sampled only every other line in this scheme. Therefore, the data rate is the same. Each of Cb and Cr is subsampled by a factor of 2 in both the horizontal and vertical directions. There are three variations of the 4:2:0 scheme with different horizontal and vertical sitings.
[0043] In MPEG-2, Cb and Cr are co-located horizontally. Cb and Cr are addressed between pixels in the vertical direction (interstitial addressing).
[0044] In JPEG / JFIF, H.261, and MPEG-1, Cb and Cr are interleaved and addressed halfway between every other luma sample.
[0045] In 4:2:0 DV, Cb and Cr are co-located in the horizontal direction. In the vertical direction, they are co-located on alternate lines.
[0046] 2.2 Encoding and decoding process of typical video codecs
[0047] Figure 1 An example of an encoder block diagram for VVC is shown, which includes three loop filter blocks: deblocking filter (DF), sample adaptive offset (SAO), and ALF. Unlike DF, which uses a predefined filter, SAO and ALF use the original samples of the current picture to reduce the mean square error between the original and reconstructed samples by adding an offset and applying a finite impulse response (FIR) filter, respectively, with the offset and filter coefficients signaled by the codec side information. ALF is located at the last processing stage of each picture and can be seen as a tool that tries to capture and repair artifacts created by previous stages.
[0048] 2.3 Intra-mode codec with 67 intra-prediction modes
[0049] To capture arbitrary edge directions present in natural videos, the number of directional intra modes is extended from 33 (as used in HEVC) to 65. Figure 2 The additional directional modes are depicted as red dashed arrows in , and the planar and DC modes remain the same. These more intensive directional intra prediction modes apply to all block sizes and both luma and chroma intra prediction.
[0050] The conventional angular intra prediction direction is defined from 45 degrees to -135 degrees in the clockwise direction, such as Figure 2 As shown. In VTM2, for non-square blocks, several conventional angle intra prediction modes are adaptively replaced by wide angle intra prediction modes. The replaced modes are signaled using the original method, and after parsing, the replaced modes are remapped to the index of the wide angle mode. The total number of intra prediction modes is unchanged, i.e. 67, and the intra mode encoding and decoding is unchanged.
[0051] In HEVC, each intra-coded block has a square shape, and the length of each side is a power of 2. Therefore, in case of using DC mode, no split operation is required to generate intra prediction values. In VVV2, blocks can have a rectangular shape, which in general must use a split operation for each block. To avoid the split operation for DC prediction, only the longer sides are used to calculate the mean of non-square blocks.
[0052] Figure 2 Examples of 67 intra prediction modes are shown.
[0053] 2.4 Wide-angle intra prediction for non-square blocks
[0054] Conventional angular intra prediction directions are defined from 45 degrees to -135 degrees in a clockwise direction. In VTM2, for non-square blocks, several conventional angular intra prediction modes are adaptively replaced by wide-angle intra prediction modes. The replaced modes are signaled using the original method, and after parsing, the replaced modes are remapped to the index of the wide-angle mode. The total number of intra prediction modes for a block remains unchanged, i.e. 67, and the intra mode encoding and decoding is unchanged.
[0055] Figure 3A and Figure 3B An example of reference samples for wide angle intra prediction is shown.
[0056] To support these predictions, Figure 3A-3B As shown, a top reference having a length of 2W+1 and a left reference having a length of 2H+1 are defined.
[0057] The mode number of the replaced mode in the wide angle direction mode depends on the aspect ratio of the block. The replaced intra prediction modes are shown in Table 2-1.
[0058] Table 2-1 - Intra prediction modes replaced by wide angle modes
[0059] condition Alternative intra prediction modes W / H==2 Mode 2, 3, 4, 5, 6, 7 W / H>2 Mode 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 W / H==1 none H / W==1 / 2 Mode 61, 62, 63, 64, 65, 66 H / W<1 / 2 Modes 57, 58, 59, 60, 61, 62, 63, 64, 65, 66
[0060] Figure 4 Shows the discontinuity problem where the orientation exceeds 45 degrees.
[0061] like Figure 4 As shown in FIG. 1 , in the case of wide angle intra prediction, two vertically adjacent prediction samples can use two non-adjacent reference samples. Therefore, a low-pass reference sample filter and edge smoothing are applied to wide angle prediction to reduce the increased gap Δp α negative impact.
[0062] 2.5 Position-dependent intra prediction combination
[0063] In VTM2, the intra prediction results of the planar mode are further modified by the position dependent intraprediction combination (PDPC) method. PDPC is an intra prediction method that calls a combination of unfiltered boundary reference samples and HEVC style intra prediction using filtered boundary reference samples. PDPC is applied to the following intra modes without signaling: planar, DC, horizontal, vertical, lower left angle mode and its eight adjacent angle modes, and upper right angle mode and its eight adjacent angle modes.
[0064] The intra prediction sample pred(x,y) is predicted using the intra prediction mode (DC, planar, angular), and the linear combination of the reference samples can be further applied according to the equation to generate the final prediction sample pred(x,y) as follows:
[0065] pred(x,y)=(wL×R (-1,y) +wT×R (x,-1) –wTL×R (-1,-1) +(64–wL–wT+wTL)×pred(x,y)+32)>>
[0066] 6(2-1)
[0067] Among them, R (x,-1) , R (-1,y) represents the reference sample points located on the top and left of the current sample point (x, y), and R (-1,-1) Represents the reference sample located at the upper left corner of the current block.
[0068] If PDPC is applied to DC, planar, horizontal and vertical intra modes, no additional boundary filters are needed, whereas in the case of HEVC DC mode boundary filters or horizontal / vertical mode edge filters, these additional boundary filters are required.
[0069] Figure 5A-5D The reference sample points (R (x,-1) , R (-1,y) and R (-1,-1) ) is defined. The prediction sample pred(x',y') is located at (x',y') in the prediction block. The reference sample R is given by x = x' + y' + 1 (x,-1) The coordinate x of the reference point R is given by y=x'+y'+1. (-1,y) The y coordinate of .
[0070] Figure 5A-5D An example definition of samples used by PDPC applied to diagonal and adjacent angle intra modes is shown. Figure 5A A diagonal top right mode example is shown. Figure 5B A diagonal bottom left mode example is shown. Figure 5C An adjacent diagonal upper right pattern example is shown. Figure 5D An adjacent diagonal lower left pattern example is shown.
[0071] The PDPC weight depends on the prediction mode and is shown in Table 2-2.
[0072] Table 2-2 - Example of PDPC weights according to prediction mode
[0073] Prediction Model w wxya wxya Diagonal upper right 16>>((y'<<1)>>shift) 16>>((x'<<1)>>shift) 0 Diagonal bottom left 16>>((y'<<1)>>shift) 16>>((x'<<1)>>shift) 0 Adjacent diagonal upper right 32>>((y'<<1)>>shift) 0 0 Adjacent diagonal bottom left 0 32>>((x'<<1)>>shift) 0
[0074] A detailed description of PDPC in VVC Draft 6 is given below.
[0075] 8.4.5.2.5 General intra-frame sample prediction
[0076] The input to this process is:
[0077] – Sample point position (xTbCmp, yTbCmp), specifies the upper left sample point of the current transform block relative to the upper left sample point of the current picture,
[0078] – variable predModeIntra, specifies the intra prediction mode,
[0079] – variable nTbW, specifies the transform block width,
[0080] – variable nTbH, specifies the transform block height,
[0081] – variable nCbW, specifies the codec block width,
[0082] – variable nCbH, specifies the codec block height,
[0083] – The variable cIdx specifies the color component of the current block.
[0084] The output of this process is the predicted samples predSamples[x][y], where x=0..nTbW-1 and y=0..nTbH-1.
[0085] The variables refW and refH are derived as follows:
[0086] – If IntraSubPartitionsSplitType is equal to ISP_NO_SPLIT or cIdx is not equal to 0, then the following applies:
[0087] refW=nTbW*2(8-118)
[0088] refH=nTbH*2(8-119)
[0089] – Otherwise (IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT and cIdx is equal to 0), the following applies:
[0090] refW=nCbW+nTbW (8-120)
[0091] refH=nCbH+nTbH (8-121)
[0092] The variable refIdx that specifies the intra prediction reference line index is derived as follows:
[0093] refIdx=(cIdx==0)? IntraLumaRefLineIdx[xTbCmp][yTbCmp]:0(8-122)
[0094] The wide angle intra prediction mode mapping process as specified in clause 8.4.5.2.6 is called with predModeIntra, nTbW, nTbH and cIdx as input and the output is the modified predModeIntra.
[0095] The variable refFilterFlag is derived as follows:
[0096] – If predModeIntra is equal to one of the following values: 0, -14, -12, -10, -6, 2, 34, 66, 72, 76, 78, 80, then set refFilterFlag equal to 1.
[0097] – Otherwise, set refFilterFlag equal to 0.
[0098] For the generation of the reference sample p[x][y] (where x=-1-refIdx, y=-1-refIdx..refH-1, and x=-refIdx..refW-1, y=-1-refIdx), the following sequential steps apply:
[0099] 1. The reference sample availability marking process as specified in clause 8.4.5.2.7 is called with the sample position (xTbCmp, yTbCmp), the intra prediction reference line index refIdx, the reference sample width refW, the reference sample height refH, the color component index cIdx as input, and the output is the reference sample refUnfilt[x][y] (where x = -1-refIdx, y = -1-refIdx..refH-1 and x = -refIdx..refW-1, y = -1-refIdx).
[0100] 2. When at least one sample refUnfilt[x][y] (where x = -1-refIdx, y = -1-refIdx..refH-1 and x = -refIdx..refW-1, y = -1-refIdx) is marked as "not available for intra prediction", the intra prediction reference line index refIdx, reference sample width refW, reference sample height refH, reference sample refUnfilt[x][y] (where x = -1-refIdx, y = -1-refIdx..refW-1, y = -1-refIdx) is marked as "not available for intra prediction", the intra prediction reference line index refIdx, reference sample width refW, reference sample height refH, reference sample refUnfilt[x][y] (where x = -1-refIdx, y = -1- The reference sample replacement process as specified in clause 8.4.5.2.8 is called with the color component index cIdx as input, and the output is the modified reference sample refUnfilt[x][y] (where x=-1-refIdx, y=-1-refIdx..refH-1 and x=-refIdx..refW-1, y=-1-refIdx).
[0101] 3. The reference sample filtering process as specified in clause 8.4.5.2.9 is called with as input the intra prediction reference line index refIdx, the transform block width nTbW and height nTbH, the reference sample width refW, the reference sample height refH, the reference filter flag refFilterFlag, the unfiltered samples refUnfilt[x][y] (where x = -1-refIdx, y = -1-refIdx ..refH-1 and x = -refIdx ..refW-1, y = -1-refIdx) and the color component index cIdx, and the output is the reference sample p[x][y] (where x = -1-refIdx, y = -1-refIdx ..refH-1 and x = -refIdx ..refW-1, y = -1-refIdx).
[0102] The intra-frame sample prediction process according to predModeIntra is applied as follows:
[0103] – If predModeIntra is equal to INTRA_PLANAR, then the corresponding intra prediction mode process as specified in clause 8.4.5.2.10 is called with transform block width nTbW and transform block height nTbH and reference sample array p as input, and the output is the prediction sample array predSamples.
[0104] – Otherwise, if predModeIntra is equal to INTRA_DC, then the corresponding intra prediction mode process as specified in clause 8.4.5.2.11 is called with transform block width nTbW, transform block height nTbH, intra prediction reference line index refIdx and reference sample array p as input, and the output is the prediction sample array predSamples.
[0105] – Otherwise, if predModeIntra is equal to INTRA_LT_CCLM, INTRA_L_CCLM or INTRA_T_CCLM, then the corresponding intra prediction mode process as specified in clause 8.4.5.2.13 is called with intra prediction mode predModeIntra, sample position (xTbC, yTbC) set equal to (xTbCmp, yTbCmp), transform block width nTbW and height nTbH, color component index cIdx and reference sample array p as input, and the output is the prediction sample array predSamples.
[0106] – Otherwise, invoke the corresponding intra prediction mode process as specified in clause 8.4.5.2.12 with as input the intra prediction mode predModeIntra, the intra prediction reference line index refIdx, the transform block width nTbW, the transform block height nTbH, the reference sample width refW, the reference sample height refH, the codec block width nCbW and height nCbH, the reference filter flag refFilterFlag, the colour component index cIdx and the reference sample array p, and the output is the prediction sample array predSamples.
[0107] When all of the following conditions are true, the position-dependent prediction sample filtering process as specified in clause 8.4.5.2.14 is called with as input intra prediction mode predModeIntra, transform block width nTbW, transform block height nTbH, prediction samples predSamples[x][y] where x=0..nTbW-1, y=0..nTbH-1, reference sample width refW, reference sample height refH, reference samples p[x][y] where x=-1, y=-1..refH-1 and x=0..refW-1, y=-1, and color component index cIdx, and the output is the modified prediction sample array predSamples:
[0108] –nTbW is greater than or equal to 4 and nTbH is greater than or equal to 4, or cIdx is not equal to 0
[0109] –refIdx is equal to 0 or cIdx is not equal to 0
[0110] –BdpcmFlag[xTbCmp][xTbCmp] is equal to 0
[0111] – One of the following conditions is true:
[0112] –predModeIntra equals INTRA_PLANAR
[0113] –predModeIntra equals INTRA_DC
[0114] –predModeIntra is less than or equal to INTRA_ANGULAR18
[0115] –predModeIntra is less than or equal to INTRA_ANGULAR50
[0116] 8.4.5.2.14 Position-dependent intra-frame prediction sample filtering
[0117] The input to this process is:
[0118] – Intra prediction mode predModeIntra,
[0119] – variable nTbW that specifies the transform block width,
[0120] – variable nTbH that specifies the transform block height,
[0121] – variable refW that specifies the reference sample width,
[0122] – variable refH that specifies the reference sample point height,
[0123] – Prediction samples predSamples[x][y], where x = 0..nTbW-1, y = 0..nTbH-1,
[0124] – neighboring samples p[x][y], where x=-1, y=-1..refH-1 and x=0..refW-1, y=-1,
[0125] – The variable cIdx specifies the color component of the current block.
[0126] The output of this process is the modified prediction samples predSamples[x][y], where x=0..nTbW-1 and y=0..nTbH-1.
[0127] Depending on the value of cIdx, set the function clip1Cmp as follows:
[0128] – If cIdx is equal to 0, then set clip1Cmp equal to Clip1 Y .
[0129] – Otherwise, set clip1Cmp equal to Clip1 C .
[0130] The variable nScale is derived as follows:
[0131] – If predModeIntra is greater than INTRA_ANGULAR50, then nScale is set equal to Min(2,Log2(nTbH)-Floor(Log2(3*invAngle-2))+8) using invAngle as specified in clause 8.4.5.2.12.
[0132] – Otherwise, if predModeIntra is less than INTRA_ANGULAR18, then nScale is set equal to Min(2,Log2(nTbW)-Floor(Log2(3*invAngle-2))+8) using invAngle as specified in clause 8.4.5.2.12.
[0133] – Otherwise, set nSacle to ((Log2(nTbW)+Log2(nTbH)-2)>>2).
[0134] The reference sample arrays mainRef[x] and sideRef[y] (where x=0..refW-1 and y=0..refH-1) are derived as follows:
[0135] mainRef[x]=p[x][-1](8-244)
[0136] sideRef[y]=p[-1][y]
[0137] The variables refL[x][y], refT[x][y], wT[y], wL[x] and wTL[x][y] (where x=0..nTbW-1, y=0..nTbH-1) are derived as follows:
[0138] – If predModeIntra is equal to INTRA_PLANAR or INTRA_DC, then the following applies:
[0139] refL[x][y]=p[-1][y](8-245)refT[x][y]=p[x][-1](8-246)wT[y]=32>>((y<<1)>>nScale)
[0140] (8-247)
[0141] wL[x]=32>>((x<<1)>>nScale)
[0142] (8-248)
[0143] wTL[x][y] = 0 (8-249) - otherwise, if predModeIntra is equal to INTRA_ANGULAR18 or INTRA_ANGULAR50, then the following applies:
[0144] refL[x][y]=p[-1][y](8-250)refT[x][y]=p[x][-1](8-251)wT[y]=(predModeIntra==INTRA_ANGULAR18)?
[0145] 32>>((y<<1)>>nScale):0(8-252)
[0146] wL[x]=(predModeIntra==INTRA_ANGULAR50)?
[0147] 32>>((x<<1)>>nScale):0(8-253)
[0148] wTL[x][y]=(predModeIntra==INTRA_ANGULAR18)? wT[y]:
[0149] wL[x](8-254)
[0150] – Otherwise, if predModeIntra is less than INTRA_ANGULAR18 and nScale is equal to or greater than 0, then the following ordered steps apply:
[0151] 1. Using invAngle as specified in clause 8.4.5.2.12 according to intraPredMode, derive the variables dXInt[y] and dX[x][y] as follows:
[0152] dXInt[y]=((y+1)*invAngle+256)>>9
[0153] (8-255)
[0154] dX[x][y]=x+dXInt[y]
[0155] 2. Derive the variables refL[x][y], refT[x][y], wT[y], wL[x], and wTL[x][y] as follows:
[0156] refL[x][y]=0(8-256)refT[x][y]=(y<(3<<nScale))?mainRef[dX[x][y]]:0(8-257)wT[y]=32> >((y<<1)>>nScale)
[0157] (8-258)
[0158] wL[x]=0(8-259) wTL[x][y]=0(8-260) – Otherwise, if predModeIntra is greater than INTRA_ANGULAR50 and nScale is equal to or greater than 0, then the following ordered steps apply:
[0159] 1. Derived variables dYInt[x] and dY[x][y] according to intraPredMode using invAngle as specified in clause 8.4.5.2.12 as follows:
[0160] dYInt[x]=((x+1)*invAngle+256)>>9
[0161] (8-261)
[0162] dY[x][y]=y+dYInt[x]
[0163] 2. Derive the variables refL[x][y], refT[x][y], wT[y], wL[x], and wTL[x][y] as follows:
[0164] refL[x][y]=(y<(3<<nScale))?sideRef[dY[x][y]]:0(8-262)refT[x][y]=0(8-263)wT[y]=0(8-264)wL[x]=32> >((x<<1)>>nScale)
[0165] (8-265)
[0166] wTL[x][y]=0(8-266)—otherwise, refL[x][y], refT[x][y], wT[y], wL[x], and wTL[x][y] are all set equal to 0.
[0167] The values of the modified prediction samples predSamples[x][y] are derived as follows, where x=0..nTbW-1 and y=0..nTbH-1:
[0168] predSamples[x][y]=clip1Cmp((refL[x][y]*wL[x]+refT[x][y]*wT[y]-
[0169] p[-1][-1]*wTL[x][y]+
[0170] (8-267)(64-wL[x]-wT[y]+
[0171] wTL[x][y])*predSamples[x][y]+32)
[0172] >>6)
[0173] 2.6 Intra Subblock Partitioning (ISP)
[0174] In JVET-M0102, ISP is proposed, which divides the luma intra prediction block vertically or horizontally into 2 or 4 sub-partitions, depending on the block size dimension, as shown in Table 1. Figure 6 and Figure 7 Examples of two possibilities are shown. All sub-partitions satisfy the condition of having at least 16 samples.
[0175] Table 1: Number of subpartitions depending on block size
[0176] Block size Number of subpartitions 4×4 No division 4×8 and 8×4 2 All other cases 4
[0177] Figure 6 Examples of partitioning into 4×8 blocks and 8×4 blocks are shown.
[0178] Figure 7Examples of partitioning of all blocks except 4×8, 8×4, and 4×4 are shown.
[0179] For each of these sub-partitions, a residual signal is generated by entropy decoding the coefficients sent by the encoder, followed by inverse quantization and inverse transformation. After that, the sub-partition is intra-predicted, and finally the corresponding reconstructed samples are obtained by adding the residual signal to the prediction signal. Therefore, the reconstructed values of each sub-partition will be used to generate the prediction of the next sub-partition, and the process is repeated for the next sub-partition, and so on. All sub-partitions share the same intra mode.
[0180] Based on the intra mode and partitioning utilized, two different kinds of processing order are used, which are referred to as normal order and inverse order. In the normal order, the first sub-partition to be processed is the sub-partition containing the upper left sample of the CU, and then proceeds downwards (horizontal partitioning) or to the right (vertical partitioning). Therefore, the reference samples used to generate the sub-partition prediction signal are located only on the left and upper sides of the line. On the other hand, the inverse processing order either starts with the sub-partition containing the lower left sample of the CU and continues upwards, or starts with the sub-partition containing the upper right vertex of the CU and continues to the left.
[0181] 2.7 Quantized Residual Domain BDPCM
[0182] In JVET-N0413, quantized residual domain BDPCM (hereinafter referred to as RBDPCM) is proposed. Intra-frame prediction is performed on the entire block by copying samples in the prediction direction (horizontal or vertical prediction) similar to intra-frame prediction. The residual is quantized and the delta between the quantized residual and its predictor (horizontal or vertical) quantized value is encoded and decoded.
[0183] For a block of size M (rows) × N (columns), let r i,j , 0≤u≤M-1,0≤j≤N-1 is the prediction residual after performing intra prediction horizontally (copying the left neighbor pixel values line by line across the prediction block) or vertically (copying the top neighbor line to each line in the prediction block) using unfiltered samples from the upper or left block boundary samples. Let Q(r i,j ), 0≤u≤M-1,0≤j≤N-1 represents the residual r i,j The quantized version of , where the residual is the difference between the original block and the predicted block value. Then, block DPCM is applied to the quantized residual samples to obtain the modified M×N array Its elements are When signaling vertical BDPCM:
[0184]
[0185] For horizontal prediction, similar rules apply and the residual quantization samples are obtained by the following equation
[0186]
[0187] Quantize the residual samples Send to the decoder.
[0188] On the decoder side, the above calculation is reversed, yielding Q(r i,j ),0≤i≤M-1,0≤j≤N-1. For vertical prediction,
[0189]
[0190] For the horizontal case,
[0191]
[0192] The inverse quantized residual Q -1 QUR i,j )) is added to the intra block prediction value to generate the reconstructed sample value.
[0193] The main benefit of this scheme is that inverse DPCM can be done on the fly during coefficient analysis simply by adding the predictor when the coefficients are analyzed, or it can be performed after analysis.
[0194] Transform skipping is always used in quantized residual domain BDPCM.
[0195] 2.8 Cross-component linear model prediction (CCLM)
[0196] To reduce cross-component redundancy, a cross-component linear model (CCLM) prediction mode is used in VTM4, in which chrominance samples are predicted based on the reconstructed luma samples of the same CU using a linear model as follows:
[0197] pred C (i,j)=α·rec L '(i,j)+β
[0198] Among them, pred C (i, j) represents the predicted chroma sample in the CU, and rec L(i, j) represents the downsampled reconstructed luma sample in the same CU. The linear model parameters α and β are derived from the relationship between the luma value and the chroma value from two samples, which are the luma sample with the minimum sample value and the maximum sample value in the set of downsampled adjacent luma samples and their corresponding chroma samples. The linear model parameters α and β are obtained according to the following equations.
[0199]
[0200] β=Y b -α·X b
[0201] where Y a and X a represents the luminance value and chrominance value of the luminance sample with the maximum luminance sample value. b and Y b Represents the luma and chroma values of the luma sample with the minimum luma sample value, respectively. Figure 8 An example of the positions of the left and upper samples involved in the CCLM mode and the samples of the current block is shown.
[0202] Figure 8 An example of the location of the sample points used for the derivation of α and β is shown. The division operation for calculating the parameter α is implemented using a lookup table. In order to reduce the amount of memory required to store the table, the diff value (the difference between the maximum and minimum values) and the parameter α are expressed by exponential notation. For example, a 4-bit significant part and an exponent are used to approximate the diff. Therefore, the table of 1 / diff is simplified to 16 elements for the 16 values of the significant digit as follows:
[0203] DivTable[]={0,7,6,5,5,4,4,3,3,2,2,1,1,1,1,0}
[0204] The benefit of doing this is that it reduces both the complexity of the calculation and the size of the memory required to store the necessary tables.
[0205] In addition to being able to use the upper template and the left template together to calculate the linear model coefficients, it is also possible to selectively use the upper template and the left template in two other LM modes (called LM_A and LM_L).
[0206] In LM_A mode, only the upper template is used to calculate the linear model coefficients. To obtain more samples, the upper template is expanded to (W+H). In LM_L mode, only the left template is used to calculate the linear model coefficients. To obtain more samples, the left template is expanded to (H+W).
[0207] For non-square blocks, the upper template is extended to W+W and the left template is extended to H+H.
[0208] To match the chroma sample positions of a 4:2:0 video sequence, two types of downsampling filters are applied to the luma samples, resulting in a 2 to 1 downsampling ratio in both the horizontal and vertical directions. The selection of the downsampling filter is specified by the SPS level flag. The two downsampling filters are described below, corresponding to "Type 0" and "Type 2" content, respectively.
[0209]
[0210] Note that when the upper reference line is at a CTU boundary, only one luma line (normal line buffer in intra prediction) is used to generate the downsampled luma samples.
[0211] According to the current VVC design, PDPC is further applied to the prediction block generated in CCLM mode based on luma samples.
[0212] This parameter calculation is performed as part of the decoding process, rather than just as an encoder search operation. Therefore, no syntax is used to communicate the alpha and beta values to the decoder.
[0213] For chroma intra mode coding, a total of 8 intra modes are allowed for chroma intra mode coding. These modes include five traditional intra modes and three cross-component linear model modes (CCLM, LM_A and LM_L). Chroma mode coding and decoding depends directly on the intra prediction mode of the corresponding luminance block. Since the separate block partitioning structure for luminance component and chrominance component is enabled in I stripe, a chroma block can correspond to multiple luminance blocks. Therefore, for the chroma DM mode, the intra prediction mode of the corresponding luminance block covering the center position of the current chroma block is directly inherited.
[0214] 2.9 Chroma Intra Prediction Mode
[0215] For chroma intra mode codec, a total of 8 or 5 intra modes are allowed for chroma intra mode codec, depending on whether the cross component linear model (CCLM) is enabled. These modes include five traditional intra modes and three cross component linear model modes (where IntraPredModeC is set to 81, 82, and 83, respectively).
[0216] 2.9.1DM Mode
[0217] In chroma direct mode or derived mode (DM), the chroma intra prediction mode is derived using the prediction mode of the collocated luma block.
[0218] First, the intra prediction mode lumaIntraPredMode is derived.
[0219] If the collocated luma block is encoded or decoded in MIP mode, then lumaIntraPredMode
[0220] Set to equal Planar mode.
[0221] Otherwise, if the collocated luma block is encoded or decoded in IBC or palette mode, then lumaIntraPredMode is set equal to DC mode.
[0222] Otherwise, lumaIntraPredMode is set equal to the intra prediction mode of the collocated luma block corresponding to the luma sample covering the center of the chroma block. Fig. 9 An example is shown in FIG.
[0223] Next, the intra chroma prediction mode (denoted IntraPredModeC) is derived from lumaIntraPredMode, as highlighted in bold italics in the table below: Note that intra_chroma_pred_mode equal to 4 refers to DM mode.
[0224] Note that in VVC, due to the dual tree, a chroma block can correspond to a luma region covering multiple CUs. Fig. 9 In the example, the gray area of the chrominance block corresponds to the luminance area covering 5 CUs. When deriving the DM mode, only one of the five CUs is checked, which is as follows Fig. 9 The “CR” shown in .
[0225] Fig. 9 Examples of "CR" positions from corresponding luminance regions for DM derivation are shown.
[0226] Table 8-2 - depends on cclm_mode_flag, cclm_mode_idx,
[0227] Specification of IntraPredModeC[xCb][yCb] of intra_chroma_pred_mode and lumaIntraPredMode
[0228]
[0229] Finally, if the color format of the picture is 4:2:2, then for DM mode, further modify IntraPredModeC according to the following table.
[0230] Specification of the 4:2:2 mapping process from chroma intra prediction mode X to mode Y when chroma_format_idc is equal to 2
[0231]
[0232] 3. Examples of technical problems solved by the embodiments
[0233] PDPC has the following problems:
[0234] 1. In planar mode or DC mode, the weighting factors of neighboring samples (eg, wL, wT, and wLT) depend on the prediction angle which is not defined in planar and DC modes.
[0235] 2. PDPC is enabled for CCLM mode, however, it is not defined how to select neighboring samples and the weights applied to the neighboring samples and the chrominance prediction signal generated from the luminance block using a linear model.
[0236] 3. Enable PDPC when "predModeIntra is less than or equal to INTRA_ANGULAR50". For the upper right wide angle mode, PDPC will be disabled. Therefore, PDPC is disabled for some wide angle modes, which may not be reasonable.
[0237] 4. List of technologies and embodiments
[0238] The following list should be considered as an example to explain the general concept. These items should not be interpreted narrowly. In addition, these items can be combined in any way.
[0239] Assuming that the width and height of a block (CU / PU / TU / CB / PB / TB, etc.) are W and H, respectively, then predSamples(x,y) represents the predicted sample at position (x,y), where x=0...W–1 and y=0…H–1. (x,y) are the coordinates of the sample relative to the top left sample of the block, and x and y are the horizontal and vertical positions of the sample, respectively. R(x,y) represents a neighboring sample (e.g., a reconstructed neighboring sample or a reconstructed neighboring sample modified by some filtering process), where x=-1, y=-1…refH-1 and x=0…refW-1, y=-1, where refH and refW are the height and width of the reference neighboring sample. Let maxTbSize be the maximum transform block size, e.g., 32 or 64, and wL[x], wT[y] be the weighting factors for the left neighboring sample and the top neighboring sample, respectively. Let the function Log2(N) be the base 2 logarithm of N.
[0240] 1. It is proposed that in planar mode or / and DC mode, the weighting factors of neighboring samples in PDPC may depend only on the dimension(s) of the block (e.g., the width denoted as W or / and the height denoted as H) or / and the position of the sample to be filtered.
[0241] a. In one example, for a sample point at position (x, y), the upper neighboring sample point (e.g., R (x,-1) ) is defined as wT[y]=N1>>((y<<N2)> >nScale), where nScale=((Log2(W)+Log2(H)−N3)>>N4) and N1, N2, N3 and N4 are non-negative integers.
[0242] b. In one example, for a sample point at position (x, y), the left neighboring sample point (e.g., R (-1,y) ) is defined as wL[x]=N1>>((x<<N2)> >nScale), where nScale=((Log2(W)+Log2(H)–N3)>>N4).
[0243] c. In one example, for a sample point at position (x, y), the upper left neighboring sample point (e.g., R (-1,-1) ) is set equal to zero.
[0244] d. N1, N2, N3 and N4 are non-negative integers. For example, N1=32, N2=1, N3=2, and N4=2.
[0245] 2. Whether and / or how to apply PDPC may depend on whether in CCLM mode (such as LM,
[0246] The current block is encoded and decoded under LM-T, LM-L).
[0247] a. PDPC may not be applied in CCLM mode.
[0248] b. Alternatively, PDPC can be applied together with CCLM mode, where the prediction signal generated from the luma block and the chroma neighboring samples are used to derive the final predicted chroma block. The selection of neighboring chroma samples and / or weighting factors can be defined as follows:
[0249] i. In one example, the manner of selecting neighboring chroma samples and / or determining weighting factors of neighboring chroma samples may be the same as in a predefined mode (eg, planar mode).
[0250] ii. Alternatively, when the corresponding luminance block is encoded and decoded in an intra-frame prediction mode such as an angular prediction mode (for example, further including a wide-angle prediction mode or a vertical mode or a horizontal mode) or a planar mode or a DC mode, such an intra-frame prediction mode can be used to select neighboring chroma samples and determine weighting factors of the neighboring chroma samples.
[0251] 1. In one example, the corresponding luminance block is a corresponding luminance region (e.g., Fig. 9One of the codec units / prediction units / transform units covered by the gray area in .
[0252] a. In one example, the corresponding luma block is the one covering the center chroma sample (e.g., Fig. 9 The coding and decoding unit / prediction unit / transformation unit of the corresponding luminance sample point of CR in (in).
[0253] iii. Alternatively, when the corresponding luminance block is encoded and decoded in an intra-frame prediction mode such as an angular prediction mode (for example, further including a wide-angle prediction mode or a vertical mode or a horizontal mode) or a planar mode or a DC mode, such an intra-frame prediction mode can be used to select neighboring chroma samples and determine weighting factors of the neighboring chroma samples.
[0254] c. Alternatively, PDPC can be applied to CCLM codec blocks using a prediction signal generated from the luma block and the derived chroma neighboring samples instead of using reconstructed chroma neighboring samples.
[0255] i. In one example, neighboring chrominance samples may be derived from neighboring luma samples of a corresponding luma block by using a linear model derived in the CCLM process.
[0256] d. Alternatively, the corresponding reconstructed luma samples in the corresponding luma block may be filtered by PDPC before being used to predict the chroma samples.
[0257] i. In one example, when filtering the reconstructed luminance samples, the neighboring samples and
[0258] / or the way of determining the weighting factors of neighboring brightness samples can be consistent with the predefined mode
[0259] (e.g., planar mode).
[0260] ii. Alternatively, the manner of selecting neighboring samples and / or determining weighting factors for neighboring luma samples may depend on the intra prediction mode of the luma block.
[0261] 1. In one example, when the luminance block is encoded and decoded in an intra-frame prediction mode such as an angle prediction mode (including a wide angle prediction mode or a vertical mode or a horizontal mode) or a planar mode or a DC mode, such an intra-frame prediction mode can be used to select neighboring luminance samples and determine weighting factors of neighboring luminance samples.
[0262] 3. Whether and / or how to apply PDPC may depend on whether the current block is encoded in wide-angle intra prediction mode.
[0263] a. PDPC can be applied to blocks adopting wide-angle intra prediction mode.
[0264] i. In one example, PDPC may be applied to blocks that adopt certain (but not all) wide-angle intra prediction modes.
[0265] b. PDPC may not be applied to blocks adopting wide-angle intra prediction mode.
[0266] 4. Whether PDPC is applied to a block may depend on whether the upper neighboring sample and the left neighboring sample are “involved” along the intra prediction direction (e.g., R in equation (2-1) (-1,y) and R (x,-1) ) both. For example, in the intra prediction process, it is assumed that the sample is predicted from the left / upper neighboring sample (which can be located at a fractional position). With the starting point being the left / upper neighboring sample, a ray is drawn along the intra prediction direction. If the ray intersects the upper / left neighboring row / column, then for this sample, both the upper neighboring sample and the left neighboring sample are "involved" along the intra prediction direction. Note that R can always be (-1,-1) Considered as either the left neighbor or the top neighbor, but not both.
[0267] a. In one example, the intra prediction direction may be defined as an angular intra prediction mode.
[0268] i. In one example, the intra prediction direction may exclude the vertical prediction direction or / and the horizontal prediction direction.
[0269] ii. In one example, the intra prediction direction may exclude DC mode and / or planar mode.
[0270] iii. In one example, the intra prediction direction may exclude wide-angle intra prediction mode.
[0271] b. In one example, if both the upper neighboring samples and the left neighboring samples are “involved” along the intra prediction direction of at least N samples (e.g., N=1) used to predict the block, then PDPC can be applied. Fig.10 An example of enabling PDPC is shown in (a) and 10(b).
[0272] i. Alternatively, furthermore, if these two neighboring samples are not “involved” in the intra prediction process for predicting any sample of the block, then PDPC may not be applied. Fig.10 (c) and Fig.10 An example is shown in (d).
[0273] Fig.10 An example of neighboring samples involved along the prediction direction is shown.
[0274] 5. Whether PDPC is enabled or disabled may depend on the color component.
[0275] a. In one example, PDPC may be enabled for a luma color component (e.g., Y in a YCbCr color format; G in an RGB color format), but PDPC may be disabled for at least one chroma color component (e.g., Cb and / or Cr in a YCbCr color format; B and / or R in an RGB color format).
[0276] 6. Depending on how many samples are involved in the filtering process to derive a final prediction sample, multiple approaches to PDPC are allowed.
[0277] a. In one example, a final prediction sample can be derived based on one or more left neighboring samples (filtered or unfiltered) and an internal prediction value obtained by (for example) a normal intra-frame prediction process (e.g., pred(x, y) in Equation 2-1).
[0278] b. In one example, a final prediction sample may be derived based on one or more upper neighboring samples (filtered or unfiltered) and an internal prediction value obtained by, for example, a normal intra prediction process (e.g., pred(x, y) in Equation 2-1).
[0279] c. Which type of PDPC is applied to a block may depend on the codec information:
[0280] i. Block Dimension
[0281] ii. Block shape
[0282] iii. Ratio between block width and height
[0283] iv. Index or flag signaled in video unit
[0284] 7. Whether PDPC can be applied to a block may depend on the block dimensions and / or the block shape (square or non-square).
[0285] a. In one example, for non-square blocks, PDPC can be disabled.
[0286] b. In one example, whether PDPC is enabled or disabled may depend on the ratio between block width and block height.
[0287] c. PDPC may not be allowed when the block size is small.
[0288] i. In one example, when the width of a block is less than or equal to a threshold T (eg, T=2, 4), PDPC may not be allowed.
[0289] ii. In one example, when the height of a block is less than or equal to a threshold (eg, T=2, 4), PDPC may not be allowed.
[0290] iii. In one example, the number of luma samples in a block is less than or equal to a threshold
[0291] (e.g., 16, 32, 64), PDPC may not be allowed.
[0292] d. PDPC may not be allowed when the block size is large.
[0293] i. In one example, when the width of a block is greater than or equal to a threshold T (eg, T=32), PDPC may not be allowed.
[0294] ii. In one example, when the height of a block is greater than or equal to a threshold (eg, T=32), PDPC may not be allowed.
[0295] iii. In one example, the number of luma samples in a block is greater than or equal to a threshold
[0296] (e.g., 1024), PDPC may not be allowed.
[0297] e. You can decide to enable or disable PDPC independently for different color components.
[0298] 5. Examples
[0299] New additions are highlighted in bold italics, and deletions are marked with double brackets (eg, [[a]] indicates deletion of the letter "a").
[0300] 5.1 An Example
[0301] This is an example for bullet point 1.
[0302] 8.4.5.2.14 Position-dependent intra-frame prediction sample filtering
[0303] The input to this process is:
[0304] – Intra prediction mode predModeIntra,
[0305] – variable nTbW that specifies the transform block width,
[0306] – variable nTbH that specifies the transform block height,
[0307] – variable refW that specifies the reference sample width,
[0308] – variable refH that specifies the reference sample point height,
[0309] – Prediction samples predSamples[x][y], where x = 0..nTbW-1, y = 0..nTbH-1,
[0310] – neighboring samples p[x][y], where x=-1, y=-1..refH-1 and x=0..refW-1, y=-1,
[0311] – The variable cIdx specifies the color component of the current block.
[0312] The output of this process is the modified prediction samples predSamples[x][y], where x=0..nTbW-1 and y=0..nTbH-1.
[0313] Depending on the value of cIdx, set the function clip1Cmp as follows:
[0314] – If cIdx is equal to 0, then set clip1Cmp equal to Clip1 Y .
[0315] – Otherwise, set clip1Cmp equal to Clip1 C .
[0316] The variable nScale is derived as follows:
[0317] – If predModeIntra is greater than INTRA_ANGULAR50, then nScale is set equal to Min(2,Log2(nTbH)-Floor(Log2(3*invAngle-2))+8) using invAngle as specified in clause 8.4.5.2.12.
[0318] – Otherwise, if predModeIntra is less than INTRA_ANGULAR18 Then using invAngle as specified in clause 8.4.5.2.12, set nScale equal to Min(2,Log2(nTbW)-Floor(Log2(3*invAngle-2))+8).
[0319] – Otherwise, set nSacle to ((Log2(nTbW)+Log2(nTbH)-2)>>2).
[0320] The reference sample arrays mainRef[x] and sideRef[y] (where x=0..refW-1 and y=0..refH-1) are derived as follows:
[0321] mainRef[x]=p[x][-1](8-244)
[0322] sideRef[y]=p[-1][y]
[0323] The variables refL[x][y], refT[x][y], wT[y], wL[x] and wTL[x][y] (where x=0..nTbW-1, y=0..nTbH-1) are derived as follows:
[0324] – If predModeIntra is equal to INTRA_PLANAR or INTRA_DC, then the following applies:
[0325] refL[x][y]=p[-1][y] (8-245)
[0326] refT[x][y]=p[x][-1] (8-246)
[0327] wT[y]=32>>((y<<1)>>nScale) (8-247)
[0328] wL[x]=32>>((x<<1)>>nScale) (8-248)
[0329] wTL[x][y]=0 (8-249)
[0330] Otherwise, if predModeIntra is equal to INTRA_ANGULAR18 or INTRA_ANGULAR50, then the following applies:
[0331] refL[x][y]=p[-1][y] (8-250)
[0332] refT[x][y]=p[x][-1] (8-251)
[0333] wT[y]=(predModeIntra==INTRA_ANGULAR18)? 32>>((y<<1)>>nScale):
[0334] 0(8-252)
[0335] wL[x]=(predModeIntra==INTRA_ANGULAR50)? 32>>((x<<1)>>nScale):
[0336] 0(8-253)
[0337] wTL[x][y]=(predModeIntra==INTRA_ANGULAR18)? wT[y]:wL[x] (8-254)
[0338] Otherwise, if predModeIntra is less than INTRA_ANGULAR18 and nScale is equal to or greater than 0, then the following ordered steps apply:
[0339] 3. Using invAngle as specified in clause 8.4.5.2.12 according to intraPredMode, derive the variables dXInt[y] and dX[x][y] as follows:
[0340]
[0341] 4. Derive the variables refL[x][y], refT[x][y], wT[y], wL[x], and wTL[x][y] as follows:
[0342] refL[x][y]=0(8-256)
[0343] refT[x][y]=(y<(3<<nScale))? mainRef[dX[x][y]]: 0 (8-257)
[0344] wT[y]=32>>((y<<1)>>nScale)(8-258)
[0345] wL[x]=0(8-259)
[0346] wTL[x][y]=0 (8-260)
[0347] Otherwise, if predModeIntra is greater than INTRA_ANGULAR50 and nScale is equal to or greater than 0, then the following ordered steps apply:
[0348] 6. Using invAngle as specified in clause 8.4.5.2.12 according to intraPredMode, derive the variables dYInt[x] and dY[x][y] as follows:
[0349]
[0350] 7. Derived variables refL[x][y], refT[x][y], wT[y], wL[x], and wTL[x][y] are as follows:
[0351] refL[x][y]=(y<(3<<nScale))? sideRef[dY[x][y]]: 0 (8-262)
[0352] refT[x][y]=0 (8-263)
[0353] wT[y]=0 (8-264)
[0354] wL[x]=32>>((x<<1)>>nScale) (8-265)
[0355] wTL[x][y]=0 18-266)
[0356] - Otherwise, set refL[x][y], refT[x][y], wT[y], wL[x], and wTL[x][y] to
[0357] Set to 0.
[0358] The values of the modified prediction samples predSamples[x][y] are derived as follows, where x=0..nTbW-1 and y=0..nTbH-1:
[0359]
[0360] Alternatively, the above line can be replaced by the following:
[0361] The variable nScale is derived as follows:
[0362] - If predModeIntra is greater than INTRA_ANGULAR50, then nScale is set equal to Min(2, Log2(nTbH)-Floor(Log2(3*invAngle-2))+8) using invAngle as specified in clause 8.4.5.2.12.
[0363] - Otherwise, if predModeIntra is less than INTRA_ANGULAR18 and greater than INTRA_DC, then nScale is set equal to Min(2, Log2(nTbW)-Floor(Log2(3*invAngle-2))+8) using invAngle as specified in clause 8.4.5.2.12.
[0364] - Otherwise, if Then set nSacle to ((Log2(nTbW)+Log2(nTbH)-2)>>2).
[0365] 5.2 An Example
[0366] Here is an example of bullet points 2 and 3.
[0367] 8.4.5.2.5 General intra-frame sample prediction
[0368] The input to this process is:
[0369] - Sample point position (xTbCmp, yTbCmp), specifies the position of the upper left sample point of the current transform block relative to the upper left sample point of the current picture,
[0370] - variable predModeIntra, specifies the intra prediction mode,
[0371] - variable nTbW, specifies the transform block width,
[0372] - variable nTbH, specifies the transform block height,
[0373] - variable nCbW, specifies the codec block width,
[0374] - variable nCbH, specifies the codec block height,
[0375] - The variable cIdx specifies the color component of the current block.
[0376] The output of this process is the prediction samples predSamples[x][y], where x=0..nTbW-1 and y=0..nTbH-1.
[0377] The variables refW and refH are derived as follows:
[0378] - If IntraSubPartitionsSplitType is equal to ISP_NO_SPLIT or cIdx is not equal to 0, then the following applies:
[0379] refW=nTbW*2(8—118)
[0380] refH=nTbH*2 (8—119)
[0381] Otherwise (IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT and cIdx is equal to 0), the following applies:
[0382] refW=nCbW+nTbW(8—120)
[0383] refH=nCbH+nTbH (8-121)
[0384] The variable refIdx that specifies the intra prediction reference line index is derived as follows:
[0385] refIdx=(cIdx==0)? IntraLumaRefLineIdx[xTbCmp][yTb(mp): 0 (8—122)
[0386] The wide-angle intra prediction mode mapping process as specified in clause 8.4.5.2.6 is called with predModeIntra, nTbW, nTbH and cIdx as input and the modified predModeIntra as output.
[0387] The variable refFilterFlag is derived as follows:
[0388] - If predModeIntra is equal to one of the following values: 0, -14, -12, -10, -6, 2, 34, 66,
[0389] 72, 76, 78, 80, then set refFilterFlag equal to 1.
[0390] - Otherwise, set refFilterFlag equal to 0.
[0391] For the generation of the reference sample p[x][y] (where x=-1-refIdx, y=-1-refIdx..refH-1, and x=-refIdx..refW-1, y=-1-refIdx), the following sequential steps apply:
[0392] 4. Invoke the reference sample availability marking process as specified in clause 8.4.5.2.7 with the sample position (xTbCmp, yTbCmp), the intra prediction reference line index refIdx, the reference sample width refW, the reference sample height refH, the color component index cIdx as input, and with the reference sample refUnfilt[x][y] (where x = -1-refIdx, y = -1-refIdx..refH-1 and x = -refIdx..refW-1, y = -1-refIdx) as output.
[0393] 5. When at least one sample refUnfilt[x][y] (where x = -1-refIdx, y = -1-refIdx..refH-1 and x = -refIdx..refW-1, y = -1-refIdx) is marked as "not available for intra prediction", the intra prediction reference line index refIdx, the reference sample width refW, the reference sample height refH, the reference sample refUnfilt[x][y] (where x = -1-refIdx, y = -1- The reference sample replacement process as specified in clause 8.4.5.2.8 is invoked with the color component index cIdx as input and the modified reference sample refUnfilt[x][y] (where x=-1-refIdx, y=-1-refIdx..refH-1 and x=-refIdx..refW-1, y=-1-refIdx) as output.
[0394] 6. Invoke the reference sample filtering process as specified in clause 8.4.5.2.9 with as input the intra prediction reference line index refIdx, the transform block width nTbW and height nTbH, the reference sample width refW, the reference sample height refH, the reference filter flag refFilterFlag, the unfiltered samples refUnfilt[x][y] (where x = -1-refIdx, y = -1-refIdx ..refH-1 and x = -refIdx ..refW-1, y = -1-refIdx) and the color component index cIdx and with as output the reference sample p[x][y] (where x = -1-refIdx, y = -1-refIdx ..refH-1 and x = -refIdx ..refW-1, y = -1-refIdx).
[0395] The following application performs intra-frame sample prediction processing according to predModeIntra:
[0396] – If predModeIntra is equal to INTRA_PLANAR, then the corresponding intra prediction mode process as specified in clause 8.4.5.2.10 is called with transform block width nTbW and transform block height nTbH and reference sample array p as input, and the output is the prediction sample array predSamples.
[0397] – Otherwise, if predModeIntra is equal to INTRA_DC, then the corresponding intra prediction mode process as specified in clause 8.4.5.2.11 is called with transform block width nTbW, transform block height nTbH, intra prediction reference line index refIdx and reference sample array p as input, and the output is the prediction sample array predSamples.
[0398] – Otherwise, if predModeIntra is equal to INTRA_LT_CCLM, INTRA_L_CCLM or INTRA_T_CCLM, then the corresponding intra prediction mode process as specified in clause 8.4.5.2.13 is invoked with intra prediction mode predModeIntra, sample position (xTbC, yTbC) set equal to (xTbCmp, yTbCmp), transform block width nTbW and height nTbH, colour component index cIdx and reference sample array p as input, and the output is the prediction sample array predSamples.
[0399] – Otherwise, invoke the corresponding intra prediction mode process as specified in clause 8.4.5.2.12 with intra prediction mode predModeIntra, intra prediction reference line index refIdx, transform block width nTbW, transform block height nTbH, reference sample width refW, reference sample height refH, codec block width nCbW and height nCbH, reference filter flag refFilterFlag, colour component index cIdx and reference sample array p as input and with prediction sample array predSamples as output.
[0400] The position-dependent prediction sample filtering process as specified in clause 8.4.5.2.14 is invoked with as input intra prediction mode predModeIntra, transform block width nTbW, transform block height nTbH, prediction samples predSamples[x][y] where x=0..nTbW-1, y=0..nTbH-1, reference sample width refW, reference sample height refH, reference samples p[x][y] where x=-1, y=-1..refH-1 and x=0..refW-1, y=-1, and color component index cIdx, and output is the modified prediction sample array predSamples, when all of the following conditions are true:
[0401] –nTbW is greater than or equal to 4 and nTbH is greater than or equal to 4, or cIdx is not equal to 0
[0402] –refIdx is equal to 0 or cIdx is not equal to 0
[0403] –BdpcmFlag[xTbCmp][xTbCmp] is equal to 0
[0404] – One of the following conditions is true:
[0405] –predModeIntra equals INTRA_PLANAR
[0406] –predModeIntra equals INTRA_DC
[0407] –predModeIntra is less than or equal to INTRA_ANGULAR18
[0408] –predModeIntra [[less than]] greater than or equal to INTRA_ANGULAR50
[0409] 5.4 An Example
[0410] This is an example for bullet point 4.
[0411] 8.4.5.2.5 General intra-frame sample prediction
[0412] The input to this process is:
[0413] – Sample point position (xTbCmp, yTbCmp), specifies the position of the upper left sample point of the current transform block relative to the upper left sample point of the current picture,
[0414] – variable predModeIntra, specifies the intra prediction mode,
[0415] – variable nTbW, specifies the transform block width,
[0416] – variable nTbH, specifies the transform block height,
[0417] - variable nCbW, specifies the codec block width,
[0418] - variable nCbH, specifies the codec block height,
[0419] - The variable cIdx specifies the color component of the current block.
[0420] The output of this process is the prediction samples predSamples[x][y], where x=0..nTbW-1 and y=0..nTbH-1.
[0421] The variables refW and refH are derived as follows:
[0422] - If IntraSubPartitionsSplitType is equal to ISP_NO_SPLIT or cIdx is not equal to 0, then the following applies:
[0423] refW=nTbW*2(8—118)
[0424] refH=nTbH*2 (8-119)
[0425] Otherwise (IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT and cIdx is equal to 0), the following applies:
[0426] refW=nCbW+nTbW(8—120)
[0427] refH=nCbH+nTbH (8-121)
[0428] The variable refIdx that specifies the intra prediction reference line index is derived as follows:
[0429] refIdx=(cIdx==0)? IntraLumaRefLineIdx[xTbCmp][yTbCmp]: 0 (8-122)
[0430] The wide angle intra prediction mode mapping process as specified in clause 8.4.5.2.6 is called with predModeIntra, nTbW, nTbH and cIdx as input and the modified predModeIntra as output.
[0431] The variable refFilterFlag is derived as follows:
[0432] - If predModeIntra is equal to one of the following values: 0, -14, -12, -10, -6, 2, 34, 66,
[0433] 72, 76, 78, 80, then set refFilterFlag equal to 1.
[0434] - Otherwise, set refFilterFlag equal to 0.
[0435] For the generation of the reference sample p[x][y] (where x=-1-refIdx, y=-1-refIdx..refH-1, and x=-refIdx..refW-1, y=-1-refIdx), the following sequential steps apply:
[0436] 7. Invoke the reference sample availability marking process as specified in clause 8.4.5.2.7 with the sample position (xTbCmp, yTbCmp), the intra prediction reference line index refIdx, the reference sample width refW, the reference sample height refH, the color component index cIdx as input, and with the reference sample refUnfilt[x][y] (where x = -1-refIdx, y = -1-refIdx..refH-1 and x = -refIdx..refW-1, y = -1-refIdx) as output.
[0437] 8. When at least one sample refUnfilt[x][y] (where x = -1-refIdx, y = -1-refIdx..refH-1 and x = -refIdx..refW-1, y = -1-refIdx) is marked as "not available for intra prediction", the intra prediction reference line index refIdx, the reference sample width refW, the reference sample height refH, the reference sample refUnfilt[x][y] (where x = -1-refIdx, y = -1- The reference sample replacement process as specified in clause 8.4.5.2.8 is invoked with the color component index cIdx as input and the modified reference sample refUnfilt[x][y] (where x=-1-refIdx, y=-1-refIdx..refH-1 and x=-refIdx..refW-1, y=-1-refIdx) as output.
[0438] 9. Invoke the reference sample filtering process as specified in clause 8.4.5.2.9 with as input the intra prediction reference line index refIdx, the transform block width nTbW and height nTbH, the reference sample width refW, the reference sample height refH, the reference filter flag refFilterFlag, the unfiltered samples refUnfilt[x][y] (where x = -1-refIdx, y = -1-refIdx ..refH-1 and x = -refIdx ..refW-1, y = -1-refIdx) and the color component index cIdx and with as output the reference sample p[x][y] (where x = -1-refIdx, y = -1-refIdx ..refH-1 and x = -refIdx ..refW-1, y = -1-refIdx).
[0439] The following application performs intra-frame sample prediction processing according to predModeIntra:
[0440] – If predModeIntra is equal to INTRA_PLANAR, then the corresponding intra prediction mode process as specified in clause 8.4.5.2.10 is called with transform block width nTbW and transform block height nTbH and reference sample array p as input, and the output is the prediction sample array predSamples.
[0441] – Otherwise, if predModeIntra is equal to INTRA_DC, then the corresponding intra prediction mode process as specified in clause 8.4.5.2.11 is called with transform block width nTbW, transform block height nTbH, intra prediction reference line index refIdx and reference sample array p as input, and the output is the prediction sample array predSamples.
[0442] – Otherwise, if predModeIntra is equal to INTRA_LT_CCLM, INTRA_L_CCLM or INTRA_T_CCLM, then the corresponding intra prediction mode process as specified in clause 8.4.5.2.13 is invoked with intra prediction mode predModeIntra, sample position (xTbC, yTbC) set equal to (xTbCmp, yTbCmp), transform block width nTbW and height nTbH, colour component index cIdx and reference sample array p as input, and the output is the prediction sample array predSamples.
[0443] – Otherwise, invoke the corresponding intra prediction mode process as specified in clause 8.4.5.2.12 with intra prediction mode predModeIntra, intra prediction reference line index refIdx, transform block width nTbW, transform block height nTbH, reference sample width refW, reference sample height refH, codec block width nCbW and height nCbH, reference filter flag refFilterFlag, colour component index cIdx and reference sample array p as input and with prediction sample array predSamples as output.
[0444] When all of the following conditions are true, the position-dependent prediction sample filtering process as specified in clause 8.4.5.2.14 is invoked with as input intra prediction mode predModeIntra, transform block width nTbW, transform block height nTbH, prediction samples predSamples[x][y] where x=0..nTbW-1, y=0..nTbH-1, reference sample width refW, reference sample height refH, reference samples p[x][y] where x=-1, y=-1..refH-1 and x=0..refW-1, y=-1, and color component index cIdx, and the output is the modified prediction sample array predSamples:
[0445] –nTbW is greater than or equal to 4 and nTbH is greater than or equal to 4, or cIdx is not equal to 0
[0446] –refIdx is equal to 0 or cIdx is not equal to 0
[0447] –BdpcmFlag[xTbCmp][xTbCmp] is equal to 0
[0448] –predModeIntra is less than INTRA_LT_CCLM
[0449] [[One of the following conditions is true:
[0450] –predModeIntra equals INTRA_PLANAR
[0451] –predModeIntra equals INTRA_DC
[0452] –predModeIntra is less than or equal to INTRA_ANGULAR18
[0453] –predModeIntra is less than or equal to INTRA_ANGULAR50]]
[0454] 8.4.5.2.14 Position-dependent intra-frame prediction sample filtering
[0455] The input to this process is:
[0456] – Intra prediction mode predModeIntra,
[0457] – variable nTbW that specifies the transform block width,
[0458] – variable nTbH that specifies the transform block height,
[0459] – variable refW that specifies the reference sample width,
[0460] – variable refH that specifies the reference sample point height,
[0461] – predicted samples predSamples[x][y], where x=0..nTbW-1, y=0..nTbH-1, – neighboring samples p[x][y], where x=-1, y=-1..refH-1 and x=0..refW-1,
[0462] y=-1,
[0463] – The variable cIdx specifies the color component of the current block.
[0464] The output of this process is the modified prediction samples predSamples[x][y], where x=0..nTbW-1 and y=0..nTbH-1.
[0465] Depending on the value of cIdx, set the function clip1Cmp as follows:
[0466] – If cIdx is equal to 0, then set clip1Cmp equal to Clip1 Y .
[0467] – Otherwise, set clip1Cmp equal to Clip1 C .
[0468] The variable nScale is derived as follows:
[0469] –
[0470] –
[0471] –
[0472] –
[0473] –
[0474] – If predModeIntra is greater than
[0475] [[INTRA_ANGULAR50]], then using invAngle as specified in clause 8.4.5.2.12, set nScale equal to Min(2,Log2(nTbH)-Floor(Log2(3*invAngle-2))+8).
[0476] – Otherwise, [[if predModeIntra is greater than or less than INTRA_ANGULAR18,]] using invAngle as specified in clause 8.4.5.2.12, set nScale equal to Min(2,Log2(nTbW)-Floor(Log2(3*invAngle-2))+8).
[0477] –[[Otherwise, set nSacle to ((Log2(nTbW)+Log2(nTbH)-2)>>2). ]]
[0478]
[0479] The reference sample arrays mainRef[x] and sideRef[y] (where x=0..refW-1 and y=0..refH-1) are derived as follows:
[0480] mainRef[x]=p[x][-1](8-244)
[0481] sideRef[y]=p[-1][y]
[0482] The variables refL[x][y], refT[x][y], wT[y], wL[x] and wTL[x][y] (where x=0..nTbW-1, y=0..nTbH-1) are derived as follows:
[0483] – If predModeIntra is equal to INTRA_PLANAR or INTRA_DC, then the following applies:
[0484] refL[x][y]=p[-1][y] (8-245)
[0485] refT[x][y]=p[x][-1] (8-246)
[0486] wT[y]=32>>((y<<1)>>nScale) (8-247)
[0487] wL[x]=32>>((x<<1)>>nScale) (8-248)wTL[x][y]=0(8-249)
[0488] Otherwise, if predModeIntra is equal to INTRA_ANGULAR18 or INTRA_ANGULAR50, then the following applies:
[0489] refL[x][y]=p[-1][y] (8-250)
[0490] refT[x][y]=p[x][-1] (8-251)
[0491] wT[y]=(predModeIntra==INTRA_ANGULAR18)? 32>>((y<<1)>>nScale):
[0492] 0(8-252)
[0493] wL[x]=(predModeIntra==INTRA_ANGULAR50)? 32>>((x<<1)>>nScale):
[0494] 0(8-253)
[0495] wTL[x][y]=(predModeIntra==INTRA_ANGULAR18)? wT[y]:wL[x](8-254)
[0496] Otherwise, if predModeIntra is less than INTRA_ANGULAR18 and nScale is equal to or greater than 0, then the following ordered steps apply:
[0497] 5. Using invAngle as specified in clause 8.4.5.2.12 according to intraPredMode, derive the variables dXInt[y] and dX[x][y] as follows:
[0498]
[0499] 6. Derive the variables refL[x][y], refT[x][y], wT[y], wL[x], and wTL[x][y] as follows:
[0500] refL[x][y]=0 (8-256)
[0501] refT[x][y]=(y<(3<<nScale))? mainRef[dx[x][y]]: 0 (8-257)
[0502] wT[y]=32>>((y<<1)>>nScale) (8-258)
[0503] wL[x]=0 (8-259)
[0504] wTL[x][y]=0 (8-260)
[0505] Otherwise, if predModeIntra is greater than INTRA_ANGULAR50 and nScale is equal to or greater than 0, then the following ordered steps apply:
[0506] 7. Using invAngle as specified in clause 8.4.5.2.12 according to intraPredMode, derive the variables dYInt[x] and dY[x][y] as follows:
[0507]
[0508] 8. Derive the variables refL[x][y], refT[x][y], wT[y], wL[x], and wTL[x][y] as follows:
[0509] refL[x][y]=(y<(3<<nScale))? sideRef[dY[x][y]]: 0 (8-262)
[0510] refT[x][y]=0 (8-263)
[0511] wT[y]=0(8-264)
[0512] wL[x]=32>>((x<<1)>>nScale) (8-265)
[0513] wTL[x][y]=0 (8-266)
[0514] Otherwise, set refL[x][y], refT[x][y], wT[y], wL[x], and wTL[x][y] all equal to 0.
[0515] The values of the modified prediction samples predSamples[x][y] are derived as follows, where x=0..nTbW-1 and y=0..nTbH-1:
[0516]
[0517] Fig.11A1 is a block diagram of a video processing device 1100. Device 1100 may be used to implement one or more of the methods described herein. Device 1100 may be embodied in a smartphone, a tablet computer, a computer, an Internet of Things (IoT) receiver, etc. Device 1100 may include one or more processors 1102, one or more memories 1104, and video processing hardware 1106. (Multiple) processors 1102 may be configured to implement one or more methods described in this document. (Multiple) memories 1104 may be used to store data and code for implementing the methods and techniques described herein. Video processing hardware 1106 may be used to implement some of the techniques described in this document in hardware circuits. In some embodiments, hardware 1106 may be at least partially within processor 1102 (e.g., a graphics coprocessor).
[0518] Fig. 11B is another example of a block diagram of a video processing system in which the disclosed technology may be implemented. Fig. 11B 2400. The system 2400 may include an input 2402 for receiving video content. The video content may be received in a raw or uncompressed format, for example, 8-bit or 10-bit multi-component pixel values, or may have a compressed or encoded format. The input 2402 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces such as Ethernet, a passive optical network (PON), and the like, and wireless interfaces such as Wi-Fi or cellular interfaces.
[0519] System 2400 may include a codec component 2404, which may implement various codecs or coding methods described in this document. Codec component 2404 may reduce the average bit rate of the video from input 2402 to the output of codec component 2404 to generate the codec representation of the video. Therefore, codec technology is sometimes referred to as video compression or video transcoding technology. The output of codec component 2404 may be stored or may be transmitted by the connected communication, as represented by component 2406. The storage or communication bitstream (or coded) representation of the video received at input 2402 may be used by component 2408 to generate pixel values or to send to the displayable video of display interface 2410. The processing of generating a user-viewable video from a bitstream representation is sometimes referred to as video decompression. In addition, although some video processing operations are referred to as "codec" operations or tools, it should be understood that codec tools or operations are used at encoders, and the corresponding decoding tools or operations of the reverse codec results will be performed by decoders.
[0520] Examples of peripheral bus interfaces or display interfaces may include universal serial bus (USB) or high definition multimedia interface (HDMI) or Displayport, etc. Examples of storage interfaces include SATA (Serial Advanced Technology Attachment), PCI, IDE interface, etc. The technology described in this document may be embodied in various electronic devices, such as mobile phones, laptops, smart phones, or other devices capable of performing digital data processing and / or video display.
[0521] In some embodiments, the Fig.11A Or a device implemented on the hardware platform described in 11B to implement the video processing method discussed in this patent document.
[0522] Some embodiments of the disclosed technology include making a decision or determination to enable a video processing tool or mode. In one example, when a video processing tool or mode is enabled, the encoder will use or implement the tool or mode in the processing of a video block, but will not necessarily modify the resulting bitstream based on the use of the tool or mode. That is, when a video processing tool or mode is enabled based on a decision or decision, the conversion from a video block to a bitstream representation of a video will use the video processing tool or mode. In another example, when a video processing tool or mode is enabled, a decoder will process the bitstream knowing that the bitstream has been modified based on the video processing tool or mode. That is, the conversion from the bitstream representation of a video to a video block will be performed using a video processing tool or mode enabled based on a decision or decision.
[0523] Some embodiments of the disclosed technology include making a decision or determination to disable a video processing tool or mode. In one example, when a video processing tool or mode is disabled, the encoder will not use the tool or mode in converting video blocks to a bitstream representation of the video. In another example, when a video processing tool or mode is disabled, the decoder will process the bitstream knowing that no modifications have been made to the bitstream using the video processing tool or mode that was disabled based on the decision or determination.
[0524] As used herein, the term "video processing" may refer to video encoding, video decoding, video compression, or video decompression. For example, a video compression algorithm may be applied during conversion from a pixel representation of a video to a corresponding bitstream representation, or vice versa. The bitstream representation of the current video block may correspond, for example, to bits that are located in one place or dispersed in different places within the bitstream, as defined by the syntax. For example, a macroblock may be encoded in accordance with a transformed and encoded error residual value and also using bits in a header and other fields in the bitstream.
[0525] Various techniques and embodiments may be described using the following clause-based format: The first set of clauses describes certain features and aspects of the disclosed techniques in the previous sections.
[0526] The following items may be implemented with the additional techniques described in the items listed in the previous sections (eg, item 1).
[0527] 1. A video processing method (eg, Fig.12 The method 1200 shown in FIG. 1 comprises: determining (1202) a weighting factor to be applied to adjacent samples for conversion between a current video block of a video and a codec representation of the current video block, wherein the weighting factor is completely defined by a size of the current video block or a sample position to which the weighting factor is applied; and performing (1204) the conversion based on the determination, wherein the self-codec mode comprises a planar codec mode of a direct current (DC) codec mode.
[0528] 2. A method according to clause 1, wherein the sample point position is (x, y), and wherein the weighting factor of the neighboring sample points as the upper neighboring sample points is wT[y]=N1>>((y<<N2)> >nScale), where nScale=((Log2(W)+Log2(H)–N3)>>N4), where N1, N2, N3 and N4 are non-negative integers.
[0529] 3. A method according to clause 1, wherein the sample point position is (x, y), and wherein the weighting factor of the neighboring sample point as the left neighboring sample point is wL[x]=N1>>((x<<N2)> >nScale), where nScale=((Log2(W)+Log2(H)–N3)>>N4), where N1, N2, N3 and N4 are non-negative integers.
[0530] 4. The method of clause 1, wherein the sample point position is (x, y) and the weighting factor of the neighboring sample point in the upper left position is zero.
[0531] 5. A method according to any of clauses 1-4, wherein N1=32, N2=1, N3=2 or N4=2.
[0532] The following clauses may be implemented with the additional techniques described in the items listed in the previous sections (eg, items 2, 3, 4, and 7).
[0533] 6. A video processing method, comprising: during a conversion between a current video block of a video and a codec representation of the current video block, determining whether to apply a self-codec mode to the current video block based on a rule, the rule using the codec mode used for the conversion of the current video block; and performing the conversion based on the determination.
[0534] 7. The method of clause 6, wherein the coding mode comprises a cross-component linear model (CCLM).
[0535] 8. The method of clause 7, wherein the rule provides for disabling of self-coding mode when CCLM is used.
[0536] 9. The method of clause 7, wherein the rule provides for enabling self-coding mode when CCLM is used.
[0537] 10. A method according to clause 9, wherein the second rule further specifies the location of reference samples to be used during self-codec based conversion.
[0538] 11. A method according to clause 10, wherein the second rule specifies that the same positions of reference samples as used in planar coding mode are to be used.
[0539] 12. The method of clause 10, wherein the second rule specifies that the self-codec uses predicted luma samples or derived chroma neighboring samples during conversion of the current video block.
[0540] 13. The method of clause 6, wherein the coding mode comprises a wide angle intra prediction mode.
[0541] 14. The method of clause 6, wherein the rule provides for disabling the self-encoding mode when using wide angle intra prediction mode.
[0542] 15. The method of clause 6, wherein the rule specifies that self-coding is enabled due to use of wide angle intra prediction mode.
[0543] 16. A method according to clause 6, wherein the codec mode is defined according to whether above or left samples are involved during intra prediction of the current video block.
[0544] 17. A method according to clause 16, wherein the intra prediction comprises an angular intra prediction mode.
[0545] 18. The method of clause 6, wherein the codec mode corresponds to a height or a width of the current video block or a shape of the current video block.
[0546] 19. The method of clause 18, wherein the rule provides for disabling the self-codec mode due to the current video block having a non-square shape.
[0547] The following clauses may be implemented with the additional techniques described in the items listed in the previous sections (eg, item 5).
[0548] 20. A method of video processing, comprising: during a conversion between a current video block of a video and a codec representation of the current video block, determining whether to apply a self-codec mode to the current video block based on a rule, the rule using a component of the current video block; and performing the conversion based on the determination.
[0549] 21. The method of clause 20, wherein the rule specifies that the self-encoding mode is enabled because the current video block is a luma block.
[0550] 22. The method of clause 20, wherein the rule provides for disabling the self-encoding mode because the current video block is a chroma block.
[0551] The solutions described below may be implemented with the additional techniques described in the items listed in the previous sections (eg, item 6).
[0552] 23. A video processing method, comprising: during a conversion between a current video block of a video and a codec representation of the current video block, determining based on rules to apply a plurality of self-codec modes to the current video block; and performing the conversion using a result of applying the plurality of self-codec modes.
[0553] 24. The method of clause 23, wherein the plurality of self-coding modes comprises a coding mode based on filtered or unfiltered left neighboring samples.
[0554] 25. The method of clause 23, wherein the plurality of self-codec modes comprises a codec mode based on filtered or unfiltered upper neighboring samples.
[0555] 26. A method according to any of clauses 23-25, wherein the rule is based on codec information comprising the dimensions of the current video block or the shape of the current video block or the ratio of the height and width of the current video block or a flag signaled in the codec representation.
[0556] 27. A method as described in any of the preceding clauses, wherein the self-coding mode comprises a Position Dependent Intra Prediction Combination (PDPC) mode.
[0557] 28. A method as described in any of clauses 1 to 27, wherein the conversion includes encoding the video into a codec representation.
[0558] 29. A method as described in any of clauses 1 to 27, wherein the conversion includes decoding a codec representation to generate pixel values of the video.
[0559] 30. A video decoding apparatus comprising a processor configured to implement the method according to one or more of clauses 1 to 27.
[0560] 31. A video encoding apparatus comprising a processor configured to implement the method according to one or more of clauses 1 to 27.
[0561] 32. A computer program product having computer code stored thereon which, when executed by a processor, causes the processor to carry out the method according to any of clauses 1 to 27.
[0562] 33. The method, apparatus or system described in this document.
[0563] The second set of clauses describes certain features and aspects of the disclosed technology in the previous sections (eg, Example Implementation 1).
[0564] 1. A video processing method (eg, Fig.13 The method 1300 shown in FIG. 1 comprises: deriving (1302) weighting factors of neighboring samples of a sample of a current video block of a video according to a rule; and performing (1304) a conversion between the current video block and a coded representation of the video, and wherein the rule stipulates that when the current video block is coded and decoded using a planar mode or a DC mode, the weighting factors are determined by at least one of the dimensions of the current video block or the positions of the samples of the current video block, wherein the current video block uses a position-dependent intra prediction (PDPC) method, which combines the neighboring samples with a prediction signal of the current video block to generate a refined prediction signal of the current video block, and wherein the weighting factor of the prediction signal of the sample is determined based on the weighting factors of the corresponding neighboring samples of the sample.
[0565] 2. A method according to clause 1, wherein the rule stipulates that for a sample point position (x, y), the weighting factor of the upper neighboring sample point is wT[y]=N1>>((y<<N2)> >nScale), and wherein nScale=((Log2(W)+Log2(H)−N3)>>N4), and wherein N1, N2, N3 and N4 are non-negative integers.
[0566] 3. The method of clause 2, wherein the upper neighboring sample point comprises a position (x, -1).
[0567] 4. A method according to clause 1, wherein the rule stipulates that for a sample point position (x, y), the weighting factor of the left neighboring sample point is wL[x]=N1>>((x<<N2)> >nScale), where nScale=((Log2(W)+Log2(H)−N3)>>N4), and N1, N2, N3, and N4 are non-negative integers.
[0568] 5. The method of clause 4, wherein the left neighboring sample point comprises a position (-1, y).
[0569] 6. A method according to clause 1, wherein the rule provides that, for a sample point location (x, y), the weighting factor of the upper left neighboring sample point is zero.
[0570] 7. The method of clause 6, wherein the position of the upper left neighboring sample point is (-1, -1).
[0571] 8. A method according to any of clauses 2 to 5, wherein N1=32, N2=1, N3=2 or N4=2.
[0572] 9. A method according to any of clauses 1 to 8, wherein the performing of the conversion comprises generating a codec representation from the current video block.
[0573] 10. A method according to any of clauses 1 to 8, wherein the performing of the conversion comprises generating the current video block from a codec representation.
[0574] 11. A video processing apparatus comprising a processor configured to implement the method according to any one or more of clauses 1 to 10.
[0575] 12. A computer readable medium storing program code which, when executed, causes a processor to implement the method according to any one or more of clauses 1 to 10.
[0576] The disclosed and other solutions, examples, embodiments, modules and functional operations described in this document can be implemented in digital electronic circuits or computer software, firmware or hardware, including the structures disclosed in this document and their structural equivalents, or a combination of one or more thereof. The disclosed embodiments and other embodiments can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer-readable medium, for execution by a data processing device or for controlling the operation of the data processing device. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a material composition that affects a machine-readable propagation signal, or a combination of one or more thereof. The term "data processing device" covers all devices, equipment and machines for processing data, including (for example) a programmable processor, a computer or multiple processors or computers. In addition to hardware, the device may also include code that creates an execution environment for the computer program under consideration, for example, code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more thereof. The propagated signal is an artificially generated signal, for example, a machine-generated electrical, optical or electromagnetic signal, which is generated to encode information for transmission to a suitable receiver device.
[0577] A computer program (also referred to as a program, software, software application, script, or code) may be written in any form of programming language (including compiled or interpreted languages) and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or portions of code). A computer program may be deployed to be executed on one or more computers that are located at one site or distributed across multiple sites and interconnected by a communications network.
[0578] The processes and logic flows described in this specification can be performed by executing one or more computer programs by one or more programmable processors to perform functions by operating on input data and generating output. These processes and logic flows can also be performed by special purpose logic circuits, and the apparatus can also be implemented as special purpose logic circuits, such as FPGAs (field programmable gate arrays) or ASICs (application specific integrated circuits).
[0579] For example, processors suitable for executing computer programs include general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally speaking, the processor will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a processor that executes instructions and one or more storage devices that store instructions and data. Usually, a computer will also include one or more large-capacity storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or be operatively coupled to receive data from one or more large-capacity storage devices or transfer data to them, or both. However, a computer does not necessarily have such a device. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including (for example) semiconductor storage devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD ROM and DVDROM disks. The processor and memory can be supplemented by or incorporated into a dedicated logic circuit.
[0580] Although this patent document contains many details, they should not be interpreted as limitations on any subject matter or claim scope, but rather as descriptions of specific features of specific embodiments of specific technologies. Certain features described in this patent document in the context of each individual embodiment may also be implemented in combination in a single embodiment. Conversely, the various features described in the context of a single embodiment may also be implemented in multiple embodiments individually or in any suitable sub-combination. In addition, although certain features may be described above as working in certain combinations and even initially claimed for protection, one or more features from the claimed combination may be removed from the combination in some cases, and the claimed combination may involve sub-combinations or variations of sub-combinations.
[0581] Similarly, although operations are shown in a particular order in the drawings, this should not be understood as requiring such operations to be performed in a sequential order or the particular times shown, or requiring all of the operations shown to be performed to achieve the desired results. In addition, the division of various system components among the embodiments described in this patent document should not be understood as requiring such division in all embodiments.
[0582] Only a few implementations and examples are described, and other implementations, enhancements, and variations can be made based on what is described and illustrated in this patent document.
Claims
1. A video processing method, include: generating prediction samples of a current video block for a conversion between a current video block of a video and a bitstream of the video, generating modified prediction samples of the current video block using position-dependent intra prediction sample filtering, and performing the conversion based on the modified prediction samples, wherein, in the filtering process, at least one neighboring sample of the current video block and a first prediction sample are combined to generate a modified first prediction sample based on a weighting factor of the first prediction sample and at least one weighting factor of the at least one neighboring sample, and wherein, when the current video block is coded using one of INTRA_ANGULAR18, INTRA_ANGULAR50, planar mode, or DC mode, the width of the current video block is greater than or equal to 4, the height of the current video block is greater than or equal to 4, the reference line index is equal to 0, and the block-based differential pulse codec modulation (Bdpcm) flag is equal to 0, the filtering process is allowed for the current video block, and at least one weighting factor of the at least one neighboring sample is determined only from the dimension of the current video block and the position of the first prediction sample, wherein, when the current video block is encoded and decoded using an intra-frame prediction mode greater than INTRA_ANGULAR50 and less than INTRA_LT_CCLM mode, or when the current video block is encoded and decoded using an intra-frame prediction mode less than INTRA_ANGULAR18 and not equal to the planar mode nor the DC mode, the width of the current video block is greater than or equal to 4, the height of the current video block is greater than or equal to 4, the reference line index is equal to 0, and the block-based differential pulse codec modulation (Bdpcm) flag is equal to 0, the filtering process is allowed for the current video block, and the at least one weighting factor of the at least one neighboring sample is determined not only from the dimension of the current video block and the position of the first prediction sample, and Wherein, when the current video block is encoded and decoded using an intra-frame prediction mode greater than INTRA_ANGULAR18 and less than INTRA_ANGULAR50, the filtering process is not allowed to be performed on the current video block, In the case where the DC mode is used to encode and decode the current video block, for the position (x, y) of the first prediction sample point, the weighting factor of the upper left neighboring sample point is zero.
2. The method according to claim 1, in, The dimensions of the current video block include a width and a height of the current video block.
3. The method according to claim 1, in, In the case where the current video block is encoded using one of INTRA_ANGULAR18, planar mode or DC mode, for the position (x, y) of the first prediction sample, the weighting factor of the upper neighboring sample of the at least one neighboring sample is wT[y]=N1>>((y<<N2)> >nScale), Where nScale = ((Log2(W) + Log2(H) – N3) >> N4), Wherein, W represents the width of the current video block, Wherein, H represents the height of the current video block, and Among them, N1, N2, N3 and N4 are non-negative integers.
4. The method according to claim 3, in, The upper neighboring sample point is a sample point located at the position (x, -1).
5. The method according to claim 3, in, In the case of using one of INTRA_ANGULAR50, planar mode or DC mode to encode the current video block, for the position (x, y) of the first prediction sample, the weighting factor of the left neighboring sample of the at least one neighboring sample is wL[x]=N1>>((x<<N2)> >nScale).
6. The method according to claim 5, in, The left neighboring sample point is the sample point at the position (-1, y).
7. The method according to claim 5, in, N1=32, N2=1, N3=2, or N4=2.
8. The method according to claim 1, in, When the current video block is encoded and decoded using an intra-frame prediction mode greater than INTRA_ANGULAR50, or when the current video block is encoded and decoded using an intra-frame prediction mode less than INTRA_ANGULAR18 and not equal to the planar mode or the DC mode, at least one weighting factor of the at least one neighboring sample is further determined according to the invAngle variable, wherein the invAngle variable is determined based on the intra-frame prediction mode.
9. The method according to claim 8, in, In case the current video block is encoded using an intra prediction mode greater than INTRA_ANGULAR50, at least one weighting factor of the at least one neighboring sample is determined according to Min(2, Log2(H)-Floor(Log2(3*invAngle-2))+8), where Floor(x) specifies the largest integer less than or equal to x, and Wherein, H represents the height of the current video block.
10. The method according to claim 8, in, When the current video block is encoded using an intra prediction mode that is less than INTRA_ANGULAR18 and not equal to the planar mode nor the DC mode, at least one weighting factor for the at least one neighboring sample is determined according to Min(2, Log2(W)-Floor(Log2(3*invAngle-2))+8), where Floor(x) specifies the largest integer less than or equal to x, and Wherein, W represents the width of the current video block.
11. The method according to claim 1, in, The converting includes encoding the current video block into the bitstream.
12. The method according to claim 1, in, The converting includes decoding the current video block from the bitstream.
13. An apparatus for processing video data, comprising a processor and a non-transitory memory having instructions thereon, in, The instructions, when executed by the processor, cause the processor to: generating prediction samples of a current video block for a conversion between a current video block of a video and a bitstream of the video, generating modified prediction samples of the current video block using position-dependent intra prediction sample filtering, and performing the conversion based on the modified prediction samples, wherein, in the filtering process, at least one neighboring sample of the current video block and a first prediction sample are combined to generate a modified first prediction sample based on a weighting factor of the first prediction sample and at least one weighting factor of the at least one neighboring sample, and wherein, when the current video block is coded using one of INTRA_ANGULAR18, INTRA_ANGULAR50, planar mode, or DC mode, the width of the current video block is greater than or equal to 4, the height of the current video block is greater than or equal to 4, the reference line index is equal to 0, and the block-based differential pulse codec modulation (Bdpcm) flag is equal to 0, the filtering process is allowed for the current video block, and at least one weighting factor of the at least one neighboring sample is determined only from the dimension of the current video block and the position of the first prediction sample, wherein, when the current video block is encoded and decoded using an intra-frame prediction mode greater than INTRA_ANGULAR50 and less than INTRA_LT_CCLM mode, or when the current video block is encoded and decoded using an intra-frame prediction mode less than INTRA_ANGULAR18 and not equal to the planar mode nor the DC mode, the width of the current video block is greater than or equal to 4, the height of the current video block is greater than or equal to 4, the reference line index is equal to 0, and the block-based differential pulse codec modulation (Bdpcm) flag is equal to 0, the filtering process is allowed for the current video block, and the at least one weighting factor of the at least one neighboring sample is determined not only from the dimension of the current video block and the position of the first prediction sample, and Wherein, when the current video block is encoded and decoded using an intra-frame prediction mode greater than INTRA_ANGULAR18 and less than INTRA_ANGULAR50, the filtering process is not allowed to be performed on the current video block, In the case where the DC mode is used to encode and decode the current video block, for the position (x, y) of the first prediction sample point, the weighting factor of the upper left neighboring sample point is zero.
14. The device according to claim 13, in, The dimensions of the current video block include a width and a height of the current video block.
15. The device according to claim 13, in, In the case where the current video block is encoded using one of INTRA_ANGULAR18, planar mode or DC mode, for the position (x, y) of the first prediction sample, the weighting factor of the upper neighboring sample of the at least one neighboring sample is wT[y]=N1>>((y<<N2)> >nScale), Where nScale = ((Log2(W) + Log2(H) – N3) >> N4), Wherein, W represents the width of the current video block, Wherein, H represents the height of the current video block, and Among them, N1, N2, N3 and N4 are non-negative integers.
16. The device according to claim 15, in, The upper neighboring sample point is a sample point located at the position (x, -1).
17. The device according to claim 15, in, In the case of using one of INTRA_ANGULAR50, planar mode or DC mode to encode the current video block, for the position (x, y) of the first prediction sample, the weighting factor of the left neighboring sample of the at least one neighboring sample is wL[x]=N1>>((x<<N2)> >nScale).
18. The device according to claim 17, in, The left neighboring sample point is the sample point at the position (-1, y).
19. The device according to claim 17, in, N1=32, N2=1, N3=2, or N4=2.
20. A non-transitory computer-readable storage medium storing instructions that cause a processor to: generating prediction samples of a current video block for a conversion between a current video block of a video and a bitstream of the video, generating modified prediction samples of the current video block using position-dependent intra prediction sample filtering, and performing the conversion based on the modified prediction samples, in, In the filtering process, at least one neighboring sample of the current video block and a first prediction sample are combined to generate a modified first prediction sample based on a weighting factor of the first prediction sample and at least one weighting factor of the at least one neighboring sample, and wherein, when the current video block is coded using one of INTRA_ANGULAR18, INTRA_ANGULAR50, planar mode, or DC mode, the width of the current video block is greater than or equal to 4, the height of the current video block is greater than or equal to 4, the reference line index is equal to 0, and the block-based differential pulse codec modulation (Bdpcm) flag is equal to 0, the filtering process is allowed for the current video block, and at least one weighting factor of the at least one neighboring sample is determined only from the dimension of the current video block and the position of the first prediction sample, wherein, when the current video block is encoded and decoded using an intra-frame prediction mode greater than INTRA_ANGULAR50 and less than INTRA_LT_CCLM mode, or when the current video block is encoded and decoded using an intra-frame prediction mode less than INTRA_ANGULAR18 and not equal to the planar mode nor the DC mode, the width of the current video block is greater than or equal to 4, the height of the current video block is greater than or equal to 4, the reference line index is equal to 0, and the block-based differential pulse codec modulation (Bdpcm) flag is equal to 0, the filtering process is allowed for the current video block, and the at least one weighting factor of the at least one neighboring sample is determined not only from the dimension of the current video block and the position of the first prediction sample, and Wherein, when the current video block is encoded and decoded using an intra-frame prediction mode greater than INTRA_ANGULAR18 and less than INTRA_ANGULAR50, the filtering process is not allowed to be performed on the current video block, In the case where the DC mode is used to encode and decode the current video block, for the position (x, y) of the first prediction sample point, the weighting factor of the upper left neighboring sample point is zero.
21. A method for storing a bit stream of a video, include: generating predicted samples of a current video block of the video, Using position-dependent intra-frame prediction sample filtering to generate modified prediction samples of the current video block, generating the bitstream based on the modified prediction samples, and storing the bitstream in a non-transitory computer-readable recording medium, wherein, in the filtering process, at least one neighboring sample of the current video block and a first prediction sample are combined to generate a modified first prediction sample based on a weighting factor of the first prediction sample and at least one weighting factor of the at least one neighboring sample, and wherein, when the current video block is coded using one of INTRA_ANGULAR18, INTRA_ANGULAR50, planar mode, or DC mode, the width of the current video block is greater than or equal to 4, the height of the current video block is greater than or equal to 4, the reference line index is equal to 0, and the block-based differential pulse codec modulation (Bdpcm) flag is equal to 0, the filtering process is allowed for the current video block, and at least one weighting factor of the at least one neighboring sample is determined only from the dimension of the current video block and the position of the first prediction sample, wherein, when the current video block is encoded and decoded using an intra-frame prediction mode greater than INTRA_ANGULAR50 and less than INTRA_LT_CCLM mode, or when the current video block is encoded and decoded using an intra-frame prediction mode less than INTRA_ANGULAR18 and not equal to the planar mode nor the DC mode, the width of the current video block is greater than or equal to 4, the height of the current video block is greater than or equal to 4, the reference line index is equal to 0, and the block-based differential pulse codec modulation (Bdpcm) flag is equal to 0, the filtering process is allowed for the current video block, and the at least one weighting factor of the at least one neighboring sample is determined not only from the dimension of the current video block and the position of the first prediction sample, and Wherein, when the current video block is encoded and decoded using an intra-frame prediction mode greater than INTRA_ANGULAR18 and less than INTRA_ANGULAR50, the filtering process is not allowed to be performed on the current video block, In the case where the DC mode is used to encode and decode the current video block, for the position (x, y) of the first prediction sample point, the weighting factor of the upper left neighboring sample point is zero.
22. A video processing apparatus comprising a processor configured to implement the method according to any one of claims 8 to 12.
23. A computer readable medium storing program code which, when executed, causes a processor to implement the method according to any one of claims 2 to 12.
Citation Information
Patent Citations
Intra-frame prediction method
CN101820546A
Method and apparatus of video coding
CN108702515A
Intra-prediction encoding method, intra-prediction decoding method, intra-prediction encoding device, intra-prediction decoding device, program therefor, and program recorded recording medium
JP2014131162A
Intra prediction improvements for scalable video coding
US20140064360A1
