Transcoding method and apparatus for video coding

Abstract

A video encoding and decoding method and apparatus using a low frequency non-separable transform (LFNST) mode are disclosed. The input data at the encoder corresponds to the post-main transform data, and the input data at the decoder corresponds to the coded data of the current CU. The CU is partitioned into one or more transform blocks (TBs). A LFNST syntax is determined at the encoder side or at the decoder side if one or more conditions are satisfied. The LFNST syntax indicates whether the LFNST mode is applied in the current CU and / or which LFNST kernel is applied when the LFNST mode is applied. The conditions include that the CBF (coded block flag) for the target TB is indicated as false. The current CU is encoded or decoded according to the LFNST mode indicated by the LFNST syntax.

Description

[0001] Cross-reference

[0002] This application claims priority to U.S. Provisional Patent Application No. 62 / 981,066, filed February 25, 2020, and U.S. Provisional Patent Application No. 62 / 988,423, filed March 22, 2020, which are hereby incorporated by reference in their entirety. TECHNICAL FIELD

[0003] The present disclosure relates to video coding. In particular, the present disclosure discloses methods and apparatus for secondary transform signaling to improve performance. BACKGROUND

[0004] High Efficiency Video Coding (HEVC) is a new generation of international video coding standard developed by the Joint Collaborative Team on Video Coding (JCT-VC). HEVC is a hybrid block-based motion-compensated DCT-like coding architecture. The basic unit of compensation, called coding unit (CU), is a square block of 2Nx2N, and each CU can be recursively split into four smaller CUs until a predetermined minimum size is reached. Each CU contains one or more prediction units (PUs).

[0005] To improve the coding efficiency of the hybrid coding architecture, HEVC provides two types of prediction modes for each PU, i.e., intra prediction and inter prediction. For the intra prediction mode, directional prediction can be generated using spatial neighboring reconstructed pixels, and there are up to 35 directions in HEVC. For the inter prediction mode, motion-compensated prediction can be generated using temporal reconstructed reference frames. There are three different modes, including Skip, Merge, and Inter-Advanced Motion Vector Prediction (Inter-AMVP) modes.

[0006] Transform procedure

[0007] After prediction, the post-prediction residual for a CU is partitioned into transform units (TUs) and coded using transform and quantization. Like many other previous standards, HEVC employs Discrete Cosine Transform type II (DCT-II) as its core transform (primary transform) because of its strong "energy compaction" property. To improve the transform, for intra modes, Discrete Sine Transform (DST) is also introduced to be used in some cases to replace DCT. For inter-predicted residuals, DCT-II is the only transform currently used in HEVC. However, DCT-II is not the best transform for all cases. Discrete Sine Transform type VII (DST-VII) and Discrete Cosine Transform type VIII (DCT-VIII) are proposed to replace DCT-II in some cases. A multiple transform selection (MTS) scheme is also used for residual coding for both intra and inter coded blocks. It also exploits multiple selected transforms from the DCT / DST family (in addition to the current transform in HEVC). The newly introduced transform matrix is DCT-VIII. In VVC, the multiple transform selection (MTS) for core transform is described as follows.

[0008] In addition to DCT-II already used in HEVC, a multiple transform selection (MTS) scheme is used for residual coding for inter and / or intra coded blocks. It uses multiple selected transforms from DCT8 (DCT-VIII) / DST7 (DST-VII). The newly introduced transform matrices are DST-VII and DCT-VIII. The following table shows the basis functions of the selected DST / DCT.

[0009] Table 1: DCT-II / VIII and DST VII transform basis functions for N-point input

[0010]

[0011]

[0012] To maintain orthogonality, the transform matrices are quantized more accurately than in HEVC. To keep the intermediate values ​​of the transformed coefficients within the 16-bit range (after horizontal and vertical conversion), all coefficients are kept at 10 bits.

[0013] To control the MTS scheme, separate enabling flags are specified at the SPS level for intra and inter modes, respectively. When MTS is enabled in the SPS, a CU-level index is signaled to indicate the transform mode (indicating the type of transform applied to the horizontal and vertical directions of the current CU). Here, MTS is applied only to luma. This MTS CU-level index (i.e., mts_idx) can be signaled when both width and height are less than or equal to 32 and the CBF flag is equal to one.

[0014] If the MTS CU index is equal to 0, DCT2 is applied in both directions. However, if the MTS CU index is greater than zero, the transform types used for horizontal and vertical directions are specified in Table 2.

[0015] Table 2: Conversion and mailing mapping table

[0016]

[0017]

[0018] In some embodiments, to reduce the complexity of large-scale DST-7 and DCT-8, high-frequency transform coefficients are zeroed in DST-7 and DCT-8 blocks with size (width or height, or both) equal to 32. Only coefficients in the 16x16 low-frequency region are used.

[0019] Low-frequency non-separable transition (LFNST)

[0020] In VVC, such as Figure 1 As shown, a forward LFNST (Low Frequency Non-Separable Transform) 120 (referred to as a reduced secondary transform) is applied between the forward primary transform 110 and quantization 130 (at the encoder), while an inverse LFNST 150 is applied between the dequantization 140 and the inverse primary transform 160 (at the decoder side). In LFNST, a 4x4 non-separable transform or an 8x8 non-separable transform is applied depending on the size of the block. For example, a 4x4 LFNST is applied to smaller blocks (i.e., min(width, height) < 8), and an 8x8 LFNST is applied to larger blocks (i.e., min(width, height) > 4). Figure 1In this case, the input to the forward primary transform is the prediction residual, and the output of the inverse primary transform is the reconstructed residual.

[0021] The application of non-separable transform (used for LFNST) is described in the following examples. To apply 4x4 LFNST, a 4x4 input block X

[0022]

[0023] First, a vector is expressed as:

[0024]

[0025] This non-separable transform is computed as where indicates the transform coefficient vector, and T is a 16x16 transform matrix. Using the scan order (i.e., horizontal, vertical, or diagonal) for the block, the 16x1 coefficient vector is then reorganized into a 4x4 block. The coefficients with smaller indices are placed in the 4x4 coefficient block with smaller scan indices.

[0026] Reduced non-separable transform

[0027] LFNST (Low Frequency Non-Separable Transform) is to apply non-separable transform based on direct matrix multiplication approach, such that it is performed on a single pass without multiple iterations. However, the non-separable transform matrix dimension needs to be reduced to reduce the computational complexity and memory space to store the transform coefficients. Therefore, the reduced non-separable transform (or RST) approach is used in LFNST. The main concept of reduced non-separable transform is to map a N-dimensional vector to a R-dimensional vector in a different space, where N / R (R

[0028]

[0029] where the transformed R row is the R base of the N dimensional space. The inverse transform matrix for the RT is the transpose of its forward transform. For the 8x8 LFNST, a reduction factor of 4 is applied. In this case, a 64x64 direct matrix (which is normally used for an 8x8 non-separable transform matrix) is reduced to a 16x48 direct matrix. Therefore, a 48x16 inverse RST matrix is used at the decoder side to produce the core (primary) transform coefficients in the 8x8 top-left region. When the 16x48 matrix with the same transform group configuration is applied (instead of 16x64), each of them takes 48 input data from three 4x4 blocks (excluding the right-bottom 4x4 block) of the left-top 8x8 block.

[0030] With the help of the reduced dimension, the memory usage for storing all LFNST matrices is reduced from 10KB to 8KB with reasonable performance degradation. To reduce the complexity, the LFNST is restricted to be applied only when all the coefficients except the first coefficient group are non-significant. Therefore, when the LFNST is applied, all the primary-only transform coefficients are zeros. This allows a conditional LFNST index signaling depending on the last significant position. Therefore, this avoids the extra coefficient scanning in the current LFNST design which only needs to check the significant coefficients at certain positions. The worst case processing of the LFNST (in terms of multiplication per pixel) will limit the non-separable transform for 4x4 and 8x8 blocks to 8x16 and 8x48 transform respectively. In these cases, when the LFNST is applied to other sizes smaller than 16, the last significant scanning position must be smaller than 8. For blocks with 4xN and Nx4 (and N >= 4) shape, the proposed restriction implies that the LFNST is applied only once and only in the top-left 4x4 region. For blocks with 8xN and Nx8 (and N >= 8) shape, the proposed restriction implies that the LFNST is applied only once and only in the top-left 8x8 region. Since all the primary-only coefficients are zeroed when the LFNST is applied, the number of operations needed for the primary transform is reduced in these cases. From the encoder point of view, the quantization of the coefficients is considerably simplified when the LFNST transform is tested. A rate-distortion optimized quantization needs to be done at most on the first 8 or 16 coefficients in the scanning order, the rest of the coefficients are forced to be zero.

[0031] LFNST transform selection

[0032] In LFNST, there are 4 transform sets and 2 non-separable transform matrices (cores) per transform set in total. The mapping from the intra prediction mode to the transform set is defined as shown in the following table in advance. If one of the three CCLM (Cross-Component Linear Model) modes (i.e., INTRA LT CCLM, INTRA T CCLM or INTRA L CCLM indicated by 81 <= predModeIntra < 83) is used in the current block, the transform set 0 or the intra prediction mode for luma is selected for the current chroma block. For each transform set, the selected non-separable secondary transform candidate (or non-separable transform matrix) is further indicated by the explicit signaled LFNST index. For an intra CU, this LFNST index is signaled in the bitstream after the transform coefficients.

[0033] Table 3: Transform selection table

[0034] IntraPredMode Transform set index IntraPredMode < 0 1 0 <= IntraPredMode <= 1 0 2 <= IntraPredMode <= 12 1 13 <= IntraPredMode <= 23 2 24 <= IntraPredMode <= 44 3 45 <= IntraPredMode <= 55 2 56 <= IntraPredMode <= 80 1 81 <= IntraPredMode <= 83 0

[0035] LFNST index signaling and interaction with other tools

[0036] Since LFNST is only restricted to be applicable when all the coefficients outside the first coefficient sub-group are non-significant, in some embodiments, the LFNST index (CU level) coding depends on the position of the last significant coefficient. In addition, the LFNST index is context coded. However, the LFNST index does not depend on the intra prediction mode, and at least one bin is context coded. Furthermore, LFNST is applied to intra CUs in both intra and inter slices, and for both luma and / or chroma. If dual tree is enabled, the LFNST index for luma and chroma is signaled separately. For inter slices (i.e., dual tree is disabled), a single LFNST index is signaled and used for both luma and / or chroma.

[0037] Considering that large CUs larger than 64x64 are implicitly split (TU tiling) due to the existing maximum transform size restriction (i.e., 64x64 or set by configuration), the LFNST index search can increase the data buffering by four times for certain number of decoding pipeline stages. Therefore, the maximum size allowed by LFNST is restricted to 64x64 or the maximum transform size. Note that MTS is enabled only when LFNST is off.

[0038] As proposed in JVET-P0058 ("CE8-2.1: Transform Skip for Chroma with limiting maximum number of context-coded bin in TS residual coding" by T. Tsukuba et al., in International Telecommunication Union - Telecommunication Standardization Sector, Study Group 16, Working Party 3, 3rd Meeting: Geneva, CH, 1-11 October 2019, Document JVET-P0058), it introduces transform skip (TS) for chroma and applies TS residual coding to transform-skipped chroma blocks. For example, TS is enabled in chroma for all chroma sampling formats. In addition, since BDPCM (block-based delta pulse code modulation) uses TS, BDPCM can only be enabled when the TS enabling condition is satisfied. The TS enabling condition includes a size constraint, which means that when the block width is less than or equal to the maximum transform skip size (MaxTsSize) and the block height is less than or equal to MaxTsSize. If the condition is satisfied, TS can be enabled. MaxTsSize is a fixed integer or a variable equal to 1 « (log2_transform_skip_max_size_minus2 + 2), where log2_transform_skip_max_size_minus2 specifies the maximum block size for transform skip. log2_transform_skip_max_size_minus2 should be in the range of 0 to 3, and when not present, it is inferred to be equal to 0.

[0039] In VVC, the TS size constraint for luma is: if tbWidth <= MaxTsSize && tbHeight <= MaxTsSize, TS can be enabled.

[0040] In VVC, the TS size constraint for chroma is: if wC <= MaxTsSize && hC <= MaxTsSize, TS can be enabled.

[0041] In the above constraints, wC = tbWidth / SubWidthC and hC = tbHeight / SubHeightC. tbWidth is the block width for luma and tbHeight is the block height for luma. The variables SubWidthC and SubHeightC are specified in the following table, depending on the chroma format sampling structure (which is specified by chroma format idc and separate colour plane flag). Other values for chroma format idc, SubWidthC and SubHeightC can be specified in the future.

[0042] Table 4: Specification of variables SubWidthC and SubHeightC

[0043]

[0044] The detailed signalling conditions for the transform skip mode for each component are shown in the following table.

[0045] Table 5. Signalling conditions for the transform skip mode for each component

[0046]

[0047]

[0048]

[0049]

[0050] In the above syntax table, transform_skip_flag[ x0 ][ y0 ][ cldx ] specifies whether a transform is applied to the associated transform block. The array indices x0, y0 specify the position (x0, y0) of the top-left luma sample of the considered transform block relative to the top-left luma sample of the picture. The array index cldx specifies an indicator for the colour component; it is equal to 0 for Y, 1 for Cb and 2 for Cr. transform_skip_flag[ x0 ][ y0 ][ cldx ] equal to 1 specifies that no transform is applied to the associated transform block. transform_skip_flag[ x0 ][ y0 ][ cldx ] equal to 0 specifies that the decision whether a transform is applied to the associated transform block depends on other syntax elements.

[0051] When transform_skip_flag[ x0 ][ y0 ][ cldx ] is not present, the following can be inferred:

[0052] - If BdpcmFlag[ x0 ][ y0 ][ cldx ] is equal to 1, transform_skip_flag[ x0 ][ y0 ][ cldx ] is inferred to be equal to 1.

[0053] - Otherwise (BdpcmFlag[ x0 ][ y0 ][ cldx ] is equal to 0), transform_skip_flag[ x0 ][ y0 ][ cldx ] is inferred to be equal to 0.

[0054] In the above, BdpcmFlag[ x0 ][ y0 ][ cldx ] is a variable relative to an intra BDPCM flag applied to a luma component (i.e. cldx = 0) or a chroma component (i.e. cldx = 1 or 2).

[0055] BDPCM (Block DPCM)

[0056] The old BDPCM method was proposed in document JVET-M0057 (“CE8: BDPCM with horizontal / vertical predictor and independently decodable areas (test 8.3.1b)”, document JVET-M0057, in Joint Video Expert Team (JVET) of International Telecommunication Union - Telecommunication Standardization Sector, Study Group 16, Working Party 3, Meeting 3 (ITU-T SG16 WP3) and International Organization for Standardization / International Electrotechnical Commission, first Technical Committee, Sub-Committee 29, Working Group 11 (ISO / IEC JTC1 / SC29 / WG11), Marrakech, MA, January 2019), using reconstructed samples line by line to predict a row or a column. The signaled BDPCM direction indicates whether a vertical or a horizontal prediction is used. The used reference pixels are unfiltered samples. The prediction error is quantized in the spatial domain. The pixels are reconstructed by adding the dequantized prediction error to the prediction.

[0057] In JVET-N0413 (Marta Karczewicz, Muhammed Coban, "CE8-related: Quantized residual BDPCM," Joint Video Team (JVT) of International Telecommunication Union - Telecommunication Standardization Sector, Study Group 16, Working Party 3 (ITU-T SG16 WP3) and the 1st Technical Committee of International Organization for Standardization / International Electrotechnical Commission, Sub-Committee 29, Working Group 11 (ISO / IEC JTC1 / SC29 / WG11), 14th Meeting: Geneva, CH, 19-27 March 2019, Document JVET-N0413, as an alternative scheme to the old BDPCM, a quantized residual domain BDPCM (referred to as RDPCM or BDPCM) was proposed. The signaling and prediction directions used are the same as the old BDPCM scheme described in JVET-M0057 (F. Henry et al., "CE8: BDPCM with horizontal / vertical predictor and independently decodable areas (test8.3.1b)," Joint Video Team (JVT) of International Telecommunication Union - Telecommunication Standardization Sector, Study Group 16, Working Party 3 (ITU-T SG16 WP3) and the 1st Technical Committee of International Organization for Standardization / International Electrotechnical Commission, Sub-Committee 29, Working Group 11 (ISO / IEC JTC1 / SC29 / WG11), Marrakech, MA, USA, January 2019, Document JVET-M0057). Intra prediction for BDPCM is done by sample copy in the prediction direction (horizontal or vertical prediction) across the block similar to regular intra prediction. The residual is quantized and the delta between the quantized residual and its predictor (horizontal or vertical) is coded. This can be described as follows.

[0058] For an M(row) x N(col) block, let r i，j , 0≤i≤M-1, 0≤j≤N-1, be the prediction residual after using the unfiltered samples from the above or left block boundary samples for intra prediction vertically (i.e., copy left neighboring pixel values across the rows in the prediction block) or horizontally (i.e., copy the above neighboring row to each row in the prediction block). Let Q(r i，j ), 0≤i≤M-1, 0≤j≤N-1, represent the quantized residual r i，jquantized version of the residual, which is the difference between the original block and the predicted block value. Then, a block DPCM is applied to the quantized residual samples, resulting in a modified MxN array with elements DPCM(i,j) = Q(r(i,j)) - Q(r(i-1,j)) When vertical BDPCM is signaled, the quantized samples of the residual are obtained by:

[0059]

[0060] For horizontal prediction, similar rules apply and the quantized samples of the residual can be obtained by:

[0061]

[0062] The quantized samples of the residual are encoded and sent to the decoder.

[0063] At the decoder side, the above calculations are reversed to produce eQ(r i，j ), 0≤i≤M-1, 0≤j≤N-1. For the vertical prediction case,

[0064]

[0065] For the horizontal case,

[0066]

[0067] The inverse quantized quantized residual Q -1 (Q(r i，j ) is added to the intra block prediction value to produce the reconstructed sample value.

[0068] The syntax of RDPCM is signaled at the CU / CB level, when the CU / CB is a luma intra CU / CB and the CB width and / or height is smaller than or equal to a pre-defined threshold (e.g., 16, 32, 64, 128, 256, 512 or 1024), a flag (e.g., bdpcm_flag) is signaled to indicate whether to enable RDPCM. If bdpcm_flag is true, an additional flag (bdpcm_dir_flag) is signaled to indicate the prediction direction used in RDPCM. For example, if bdpcm_dir_flag is equal to 0, the horizontal direction is used; otherwise, if bdpcm_dir_flag is equal to 1, the vertical direction is used.

[0069] BDPCM can be applied to luma and chroma. Syntax tables for BDPCM are shown in the following tables. More details can be found in document JVET-N0413 (by M. Karczewicz et al., “CE8-related: Quantized residual BDPCM,” Joint Video Expert Team (JVET), 14th Meeting: Geneva, CH, 19-27 March 2019, document JVET-N0413, under the auspices of the International Telecommunication Union - Telecommunication Standardization Sector, Study Group 16, Working Party 3 (ITU-T SG 16 WP 3) and the International Organization for Standardization / International Electrotechnical Commission, first Technical Committee, Sub-Committee 29, Working Group 11 (ISO / IEC JTC 1 / SC 29 / WG 11).

[0070] Table 6. Syntax tables for BDPCM

[0071]

[0072]

[0073] VVC supports a tool of Joint Coding of Chroma Residual (JCCR), in which chroma residuals are jointly coded. The usage (enablement) of JCCR mode is indicated by a TU-level flag tu_joint_cbcr_residual_flag, and the selected mode is implicitly indicated by the chroma CBF. If one or both of the TU's chroma CBFs are equal to 1, the flag tu_joint_cbcr_residual_flag is present. In PPS and slice header, chroma QP offset values are signaled for JCCR mode, to be distinguished from the normal chroma QP offset values signaled for regular chroma residual coding mode. These chroma QP offset values are used to derive the chroma QP values for certain blocks coded using JCCR mode. JCCR mode has 3 sub-modes. When the corresponding JCCR sub-mode (sub-mode 2 in Table 7) is active in a TU, this chroma QP offset is added to the applied luma-derived chroma QP in the quantization and decoding process of that TU. Table 7 corresponds to Table 3-13 of document JVET-Q2002 (by (J. Chen et al., "Algorithm description for Versatile Video Coding and Test Model 8 (VTM 8)," Joint Video Team (JVT) of International Telecommunication Union - Telecommunication Standardization Sector (ITU-T) Sector of International Organization for Standardization / International Electrotechnical Commission (ISO / IEC) Joint Technical Committee 29th Sub-Committee 11th Working Group (ISO / IEC JTC1 / SC29 / WG11), 17th Meeting: Brussels, BE, 7-17 Jan 2020, document JVET-Q2002). For other JCCR sub-modes (sub-modes 1 and 3 in Table 7), chroma QP is derived in the same way as for a regular Cb or Cr block. The process of reconstructing chroma residuals (resCb and resCr) from the transmitted transform blocks is described in Table 7. When JCCR mode is enabled, a single joint chroma residual block (resJointC[x][y] in Table 7) is signaled, from which the residual block for Cb (resCb) and the residual block for Cr (resCr) are derived, taking into account information such as tu_cbf_cb, tu_cbf_cr, and CSign (a sign value indicated in slice header).

[0074] At the encoder side, the joint chroma components are derived as follows. Depending on the mode (as listed in the table above), resJointC{1,2} is generated by the encoder as follows:

[0075] - If mode is equal to 2 (single residual with reconstructed Cb = C, Cr = CSign * C), the joint residual is determined according to

[0076] resJointC[x][y] = (resCb[x][y] + CSign * resCr[x][y]) / 2,

[0077] - Else if mode is equal to 1 (single residual with reconstructed Cb = C, Cr = (CSign * C) / 2), the joint residual is determined according to

[0078] resJointC[x][y] = (4 * resCb[x][y] + 2 * CSign * resCr[x][y]) / 5,

[0079] - Else (mode is equal to 3, i.e. single residual with reconstructed Cr = C, Cb = (CSign * C) / 2), the joint residual is determined according to

[0080] resJointC[x][y] = (4 * resCr[x][y] + 2 * CSign * resCb[x][y]) / 5.

[0081] Reconstruction of chroma residual. CSign is a sign value (+1 or -1) that is signaled in the slice header, resJointC[][] is the transmitted residual.

[0082] Table 7. Syntax table for BDPCM

[0083]

[0084]

[0085] The three joint chroma coding sub-modes described in table 7 above are only supported in I slices. In P and B slices, only mode 2 is supported. Therefore, in P and B slices, the syntax element tu_joinLcbcr_residual_flag is present only when both chroma cbf are equal to 1.

[0086] The JCCR mode can be combined with a chroma transform skip (TS) mode. To speed up the encoder decision, the JCCR transform selection depends on whether the independent coding of Cb and Cr components selects DCT-2 or TS as the best transform, and whether there are non-zero coefficients in the independent chroma coding. Specifically, if one chroma component selects DCT-2 (or TS), and the other component is all zero, or both chroma components select DCT-2 (or TS), only DCT-2 (or TS) will be considered in JCCR coding. Otherwise, if one component selects DCT-2 and the other component selects TS, both DCT-2 and TS will be considered in JCCR coding. More details can be found in document JVET-N0054 (J. Lainema, “CE7: Joint coding of chrominance residuals (CE7-1),” Joint Video Expert Team (JVET) of the International Telecommunication Union - Telecommunication Standardization Sector, Study Group 16, Working Party 3 (ITU-T SG16 WP3) and the International Organization for Standardization / International Electrotechnical Commission, first Technical Committee, subcommittee 29, working group 11 (ISO / IEC JTC1 / SC29 / WG11), 14th Meeting: Geneva, CH, 19-27 March 2019, document JVET-N0054).

[0087] Intra sub-partition (ISP) in VVC

[0088] Intra sub partitioning (ISP) partitions a luma intra prediction block vertically or horizontally into 2 or 4 sub-partitions depending on the block size. For example, the minimum block size for a block partitioned by ISP is 4x8 (or 8x4). If the block size is larger than 4x8 (or 8x4), the corresponding block is partitioned into 4 sub-partitions. It has been noted that Mxl28 (where M<64) and Mxl28 (where N<64) ISP blocks can cause potential problems for 64x64 VDPU. For example, an Mxl28 CU in the single tree case has an Mxl28 luma TB and two corresponding (M / 2)x64 chroma TBs. If the CU uses ISP, the luma TB will be partitioned into four Mx32 TBs (only if horizontal partitioning is possible), each of which is smaller than a 64x64 block. However, in the current ISP design, the chroma blocks are not partitioned. Therefore, the two chroma components will have a size larger than a 32x32 block. Similarly, using ISP can create a similar situation for a 126xN CU. Therefore, these two cases are problematic for 64x64 decoder pipeline. For this reason, the CU size using ISP is limited to a maximum of 64x64. Figure 2A With Figure 2B Examples of the two possibilities are shown. All sub-partitions satisfy the condition of having at least 16 samples. Figure 2A The case of block size 4x8 or 8x4 is illustrated. In this case, block 210 is partitioned horizontally into two sub-blocks 220 or vertically into two sub-blocks 230. Figure 2B The case of block size other than 4x8 and 8x4 is illustrated. In this case, block 240 is partitioned horizontally into four sub-blocks 250 or vertically into four sub-blocks 260.

[0089] In ISP, lxN or 2xN sub-block prediction is not allowed to depend on reconstructed values of lxN or 2xN sub-blocks of previously decoded blocks of the coded block, so that the minimum width prediction for sub-blocks becomes four samples. For example, an 8xN (N>4) coded block coded using ISP with vertical partitioning is partitioned into two prediction regions (each size 4xN) and four transforms of size 2xN. Likewise, a 4xN coded block coded using ISP with vertical partitioning is predicted using the full 4xN block; four transforms (each of size lxN) are used. While lxN and 2xN transform sizes are allowed, we propose that the transforms of these blocks can be done in parallel in the 4xN region. For example, when a 4xN prediction region contains four lxN transforms, there are no transforms in the horizontal direction; the transforms in the vertical direction can be done in a single 4xN transform in the vertical direction. Similarly, when a 4xN prediction region contains two 2xN transform blocks, the transform operations of the two 2xN blocks in each direction (horizontal and vertical) can be done in parallel. Thus, there is no added delay in processing these smaller blocks compared to processing 4x4 regular coded intra blocks.

[0090] For each sub-partition, the reconstructed samples are obtained by adding the residual signal to the prediction signal. Here, the residual signal is generated by, for example, entropy decoding, inverse quantization, and inverse transform processing. Thus, the reconstructed sample values of each sub-partition can be used to generate the prediction for the next sub-partition, and each sub-partition is processed repeatedly. Furthermore, the first sub-partition to be processed is the one containing the top-left sample of the CU, and then it continues either down (horizontal partitioning) or to the right (vertical partitioning). As a result, the reference samples used to generate the sub-partition prediction signal are only located to the left and above the line. All sub-partitions share the same intra mode. The following is a summary of the interaction of ISP with other coding tools.

[0091] - Multiple reference lines (MRL): If a block has a MRL index that is not 0, the ISP coding mode is inferred to be 0, and thus ISP mode information is not sent to the decoder.

[0092] - Size of entropy coded coefficient groups: The size of entropy coded sub-blocks has been modified to have 16 samples in all possible cases, as shown in Table 8. Note that the new sizes only affect blocks produced by ISP where one of the dimensions is less than 4 samples. In all other cases, the coefficient groups remain 4x4 in size.

[0093] - CBF coding: It is assumed that at least one sub-partition has a non-zero CBF. Thus, if n is the number of sub-partitions and the first n-1 sub-partitions have resulted in a zero CBF, the CBF of the nth sub-partition is inferred to be 1.

[0094] - MPM usage: In blocks coded by ISP mode, the MPM flag is inferred to be 1 and the MPM list is modified to exclude the DC mode, and the horizontal intra mode is prioritized for ISP horizontal partitioning, and the vertical intra mode is prioritized for vertical partitioning.

[0095] - Transform size restriction: All ISP transforms of length greater than 16 points use DCT-II.

[0096] - PDPC: When a CU uses ISP coding mode, the PDPC filter will not be applied to the generated sub-partitions.

[0097] - MTS flag: If a CU uses ISP coding mode, the MTS CU flag will be set to 0 and will not be signaled to the decoder. Therefore, the encoder will not RD test different available transforms for each generated sub-partition. The transform selection for ISP mode will become fixed and selected according to the intra mode, processing order, and the block size used. Therefore, no signaling is needed. For example, let t H and t V be the selected horizontal and vertical transforms for w x h sub-partitions, where w is the width and h is the height. Then the transform is selected according to the following rules:

[0098] - If w = 1 or h = 1, then there is no horizontal or vertical transform, respectively.

[0099] - If w = 2 or w > 32, then t H = DCT-II

[0100] - If h = 2 or h > 32, then t V = DCT-II

[0101] - Otherwise, the transform is selected as in Table 9.

[0102] Table 8. Size of the entropy coded coefficient group

[0103] Block size Coefficient group size 1 x N, N >= 16 1×16 N x 1, N >= 16 16×1 2 x N, N >= 8 2×8 N x 2, N >= 8 8×2 All other possible M x N cases 4×4

[0104] Table 9. Transform selection depending on the intra mode

[0105]

[0106] In ISP mode, all 67 intra modes are allowed. PDPC is also applied if the corresponding width and height are at least 4 samples long. In addition, the condition for intra interpolation filter selection no longer exists, and the Cubic (DCT-IF) filter is always applied for fractional position interpolation in ISP mode. More details can be found in document JVET-M0102 (S. De-Luxán-Hernández et al., “CE3: Intra Sub-Partitions Coding Mode (Tests 1.1.1 and 1.1.2),” Joint Video Expert Team (JVET) of the International Telecommunication Union - Telecommunication Standardization Sector, Study Group 16, Working Party 3, and the first Technical Committee, Sub-Committee 29, Working Group 11 of the International Organization for Standardization / International Electrotechnical Commission (ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11), 13th Meeting: Marrakech, MA, 9-18 January 2019, document JVET-M0102.

[0107] The “block” in this proposal can be TB / TU / PU / PB / CB / CU. SUMMARY

[0108] Video encoding and decoding methods and apparatuses using a low frequency non-separable transform (LFNST) mode are disclosed. In a decoding procedure according to the present disclosure, input data regarding a current coding unit (CU) in a current picture is received, wherein the current CU is partitioned into one or more transform blocks (TBs), and the input data corresponds to coded data of the current CU. One or more coded block flag (CBF) indications are checked based on the coded data for one or more target TBs. If one or more conditions are satisfied, a LFNST syntax is parsed, wherein the LFNST syntax indicates whether the LFNST mode is applied in the current CU, and / or which LFNST kernel is applied if the LFNST mode is applied. The one or more conditions include that the one or more CBF indications for the one or more target TBs are false. The current CU is decoded according to the LFNST mode indicated by the LFNST syntax at a decoder side.

[0109] In an encoding process, input data for a current coding unit (CU) in a current picture is received, where the input data corresponds to primary transformed data. A LFNST process is applied based on a LFNST kernel to derive temporary output data. A LFNST syntax is determined and signaled if one or more conditions are satisfied. The conditions include one or more coded block flags (CBF) for one or more target transform blocks (TB) being false. The current CU is encoded according to a LFNST mode indicated by the determined LFNST syntax.

[0110] In an embodiment, the target TBs correspond to one or more target TBs having a transform skip (TS) flag not equal to 0.

[0111] In an embodiment, in a luma partition tree, the current CU corresponds to a luma coding block, and the one or more target TBs correspond to one or more luma TBs. In another embodiment, in a chroma partition tree, the current CU corresponds to one or more chroma coding blocks, and the one or more target TBs correspond to one or more chroma TBs. In yet another embodiment, in a single partition tree, the current CU corresponds to a luma coding block and one or more chroma coding blocks, and the one or more target TBs correspond to one or more luma TBs and one or more chroma TBs.

[0112] In an embodiment, a target TB corresponds to a predefined TB for each coding block in the current CU. For example, the predefined TB corresponds to a first TB for each coding block in the current CU.

[0113] In an embodiment, the LFNST mode is allowed for the current CU if all CBF indications for target TBs are false. In another embodiment, the LFNST mode is allowed for the current CU if all CBF indications for target TBs having a transform skip (TS) flag not equal to 0 are false. In yet another embodiment, the LFNST mode is not allowed for the current CU if any CBF indication for target TBs having a transform skip (TS) flag not equal to 0 is true. BRIEF DESCRIPTION OF DRAWINGS

[0114] Various embodiments of the disclosure will be presented in the following detailed description with reference to the figures, wherein like numbers refer to like elements throughout, and wherein:

[0115] Figure 1 An example of a LFNST (low frequency non-separable transform) process is illustrated.

[0116] Figure 2A with Figure 2B Examples of displaying intra sub-partition (ISP) are shown.

[0117] Figure 3 A flowchart of an exemplary decoding system embodying LFNST parsing is illustrated in accordance with an embodiment of the present application.

[0118] Figure 4 A flowchart of an exemplary encoding system embodying LFNST parsing is illustrated in accordance with an embodiment of the present application. DETAILED DESCRIPTION

[0119] It will be readily understood that the components of the present application, as generally described and illustrated in the Figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following more detailed description of the embodiments of the systems and methods of the present application, as represented in the Figures, is not intended to limit the scope of the application, as claimed, but is merely representative of selected embodiments of the application. The described features, structures, or characteristics can be combined in one or more embodiments. The phraseology and terminology utilized herein is for the purpose of description and not of limitation. Thus, the

[0120] Further, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. One skilled in the relevant art will recognize, however, that the application can be practiced without one or more of the specific details, or with other methods, components, etc. In other instances, well-known structures or operations are not shown or described in detail in order to avoid obscuring aspects of the application. The embodiments of the application illustrated will be best understood by reference to the drawings, wherein like reference numerals refer to like parts throughout the several views. The following description is intended only by way of example, and simply illustrates certain selected embodiments of devices and methods consistent with the application as claimed herein.

[0121] The combination of LFNST with transform skip should not be allowed, as transform skip is applied when no transform procedure (primary / core transform and / or secondary transform) should be used. In VVC Draft 7 (B. Bross, et. al., “Versatile Video Coding (Draft 7),” Joint Video Team (JVT) of the International Telecommunication Union - Telecommunication Standardization Sector (ITU-T) Study Group 16 (SG 16) Working Party 3 (WP 3) and the International Organization for Standardization / International Electrotechnical Commission (ISO / IEC) Joint Technical Committee 1 (JTC 1) Subcommittee 29 (SC 29) Working Group 11 (WG 11), 16th Meeting: Geneva, CH, 1-11 Oct 2019, JVET-P2001), the syntax for transform skip mode is signaled / parsed at the TB level. On the other hand, the syntax for LFNST is signaled / parsed at the CU level after all TUs / TBs in the CU / CB are signaled / parsed. Thus, in the current VVC Draft text (as shown in Table 6A), the signaling / parsing condition for LFNST considers the following transform skip flag for luma. As shown in the following syntax table, the existing condition includes a check for luma transform skip (i.e., transform_skip_flag[x0][y0][0] == 0) to prevent this combination. For this check, the VVC Test Model version 7 (VTM7) (J. Chen, et. al., “Algorithm description for Versatile Video Coding and Test Model 7 (VTM 7),” Joint Video Team (JVT) of the International Telecommunication Union - Telecommunication Standardization Sector (ITU-T) Study Group 16 (SG 16) Working Party 3 (WP 3) and the International Organization for Standardization / International Electrotechnical Commission (ISO / IEC) Joint Technical Committee 1 (JTC 1) Subcommittee 29 (SC 29) Working Group 11 (WG 11), 16th Meeting: Geneva, CH, 1-11 Oct 2019, JVET-P2002) code seems to match VVC Draft 7. The syntax table for residual coding according to JVET-P2001 is shown in Table 10B.

[0122] Table 10A. Signaling / parsing condition for LFNST in VVC Draft 7

[0123]

[0124]

[0125]

[0126] Table 10B. Syntax table for residual coding in VVC Draft 7

[0127]

[0128]

[0129] With the existing conditions for LFNST signaling / parsing, two problems are observed. One problem is that when luma and chroma use different partition trees, it cannot find the transform skip flag for luma (i.e., transform_skip_flag[x0][y0][0]) when the current CU is located in a chroma partition tree (i.e., the case of treeType == DUAL_TREE_CHROMA). The other problem is caused by extending transform skip to chroma as disclosed in JVET-P0058. The above check should be extended to include Cb and Cr checks. Some methods are proposed to handle these problems.

[0130] The proposed check takes into account the condition of the transform skip flag of the M TB(s) in the CU. For a TB with one or more transform coefficient levels not equal to 0, the transform skip flag for the TB is used to indicate whether the transform operation is applied to the TB, and the proposed check is used to prevent the combination of the TB with LFNST and transform skip. As mentioned before, in the corresponding partition tree (which can be a luma partition tree (DUAL_TREE_LUMA), a chroma partition tree (DUAL_TREE_CHROMA), or a single partition tree (SINGLE_TREE)), there is one or more TBs in the current CU. The M TB(s) correspond to a group of selected TBs, referred to as target TBs. The condition of the transform skip flag(s) for the target TB group is checked. By checking that the transform skip flag for all M TB(s) is false (i.e., the transform skip flag(s) for all M TBs is equal to 0); in other words, by checking that the target condition (corresponding to all target TBs in the target TB group having TS mode indication as false) is satisfied. In other words, the condition of the transform skip flag(s) is satisfied if none of the selected TB(s) uses the transform skip mode. After the check (i.e., the condition of the transform skip flag(s) is satisfied), the signaling / parsing condition for transform skip mode for LFNST is satisfied, and if other signaling / parsing conditions for LFNST are also satisfied, the syntax for LFNST can be signaled / parsed.

[0131] In one embodiment, the M TB(s) includes only the first component of each luma / chroma split tree. An example of a proposed syntax table is shown below.

[0132] Table 11. An example syntax table for conditional signaling / profile of LFNST according to an embodiment of the application

[0133]

[0134] In another embodiment, the M TB(s) includes one or more components for a single split tree for both luma and chroma components.

[0135] In one embodiment, the M TB(s) refers to a selected component. For example, the M TB(s) refers to the first component. In another example, the M TB(s) refers to Y (i.e., the luma component). An example syntax table according to an embodiment is shown below. In another example, the M TB(s) can be any one of the components in the split tree.

[0136] Table 12. An example syntax table for signaling / profile of LFNST according to an embodiment of the application

[0137]

[0138]

[0139] In another embodiment, when the split tree is not a chroma tree (i.e., the split tree contains the Y (i.e., luma) component), the M TB(s) refers to the Y (i.e., luma) TB(s). An example syntax table according to an embodiment is shown below.

[0140] Table 13. An example syntax table for signaling / profile of LFNST according to an embodiment of the application

[0141]

[0142] In another embodiment, for a chroma split tree, the M TB(s) includes all chroma components (e.g., Cb and Cr). If any transform skip flag for the chroma components is false (i.e., the transform skip flag is equal to 0), the check passes.

[0143] In another embodiment, for a luma split tree, the M TB(s) includes all components (e.g., Y). If any transform skip flag for the components is false (i.e., the transform skip flag is equal to 0), the check passes. An example syntax table according to this embodiment is shown in Table 14.

[0144] In another embodiment, the MTB(s) include all components (e.g., Y, Cb and Cr) for a single tree for luma and chroma components. If all transform skip flags for these components are false (i.e., transform skip flags equal to 0), the check passes. An exemplary syntax table according to this embodiment is shown in Table 14.

[0145] Table 14. Exemplary syntax table for signaling / profile of LFNST according to an embodiment of the application

[0146]

[0147] In another embodiment, the MTB(s) include all chroma components (e.g., Cb and Cr) for a chroma partition tree. If all transform skip flags for chroma components are false (i.e., all transform skip flags for chroma components equal to 0), the check passes.

[0148] In another embodiment, the MTB(s) include all luma components (e.g., Y) for a luma partition tree. If all transform skip flags for these components are false (i.e., all transform skip flags for these components equal to 0), the check passes. An exemplary syntax table according to this embodiment is shown in Table 15.

[0149] In another embodiment, the MTB(s) include all components (e.g., Y, Cb and Cr) for a single tree for luma and chroma components. If all transform skip flags for these components are false (i.e., transform skip flags equal to 0), the check passes. An exemplary syntax table according to this embodiment is shown in Table 15.

[0150] In yet another embodiment, two or more of the above three embodiments can be combined. For example, the combined embodiment can check only luma transform_skip_flag when the partition tree is not a chroma partition tree (e.g., not for DUAL_TREE_CHROMA), and check only chroma transform_skip_flag when the partition tree is not a luma partition tree (e.g., not for DUAL_TREE_LUMA). An exemplary syntax table according to this embodiment is shown in Table 15.

[0151] Table 15. Exemplary syntax table for signaling / profile of LFNST according to an embodiment of the application

[0152]

[0153] Table 16 shows another example syntax table for the combination of the above three embodiments.

[0154] Table 16. An example syntax table for signaling / profile of LFNST according to an embodiment of the present application

[0155]

[0156] In another embodiment, the check fails, and the syntax for LFNST (e.g., LFNST index) is not signaled / profiled.

[0157] In one embodiment, the syntax for LFNST (LFNST index) is inferred to be 0 (i.e., LFNST is not applied).

[0158] In another embodiment, if the check fails, a bitstream conformance is needed to handle this case. The bitstream conformance is as follows. The requirement of the bitstream conformance is that the value of lfnst_index shall not be greater than 0 when the check fails. Below, an example of the bitstream conformance is illustrated to check the case of "transform skip flag(s) for M TB(s) representing only the first component in each luma / chroma partition tree". The requirement of the bitstream conformance corresponds to that the value of the transform skip flag for the first component in each luma / chroma partition tree (e.g., transform_skip_flag[x0][y0][chType], where chType indicates 1 (i.e., Cb) if treeType == DUAL_TREE_CHROMA; otherwise, chType indicates 0 (i.e., Y)) shall not be greater than 1.

[0159] In another embodiment, a variable can be created in the draft text or software to record whether the syntax for LFNST is signaled / profiled or not. The value of this variable is updated according to one or more existing signaling / profile conditions for LFNST and / or one or more proposed checks in the present application. For example, this variable is initialized to 1, and if any existing signaling / profile conditions for LFNST and / or one or more proposed checks in the present application are not satisfied, this variable is changed to 0 and the syntax for LFNST is not signaled / profiled.

[0160] In another embodiment, a unified check mechanism is used for different partition trees for luma and chroma. For example, when luma and chroma use dual trees (i.e., separate partition trees), luma CUs are in a luma partition tree and chroma CUs are in a chroma partition tree. The unified mechanism is that LFNST is disabled if any of the transform skip flags for all components in the current CU use transform skip.

[0161] Due to the current size constraint of LFNST, LFNST can be applied when a CU / CB contains one TU / TB. The check can take into account the transform skip flag for one TU / TB (instead of multiple TU / TBs). When a CU / CB contains multiple TU / TBs, the proposed check is performed on one or more TU / TBs in the CU / CB. In one embodiment, the proposed check is performed on all TU / TBs in the CU / CB. In another embodiment, the check is performed on any TU / TB (e.g., the first TU / TB or the last TU / TB) in the CU / CB. For example, in a luma partition tree, the target TB set includes the first luma TB for the luma CB in the current CU. In another example, in a chroma partition tree, the target TB set includes the first Cb TB for the Cb CB in the current CU and the first Cr TB for the Cr CB in the current CU. In another example, in a single partition tree, the target TB set includes the first luma TB for the luma CB in the current CU, the first Cb TB for the Cb CB in the current CU, and the first Cr TB for the Cr CB in the current CU. In another embodiment, the check is performed on a subset of any TU / TB in the CU / CB.

[0162] Furthermore, the use of LFNST can be limited in certain cases. In the current design, LFNST is applied in intra and inter CUs for both luma and / or chroma. If a dual tree is enabled, the LFNST index for luma and chroma is signaled / parsed separately. For inter slices, when a dual tree is disabled, a single LFNST index is signaled / parsed and used for luma and / or chroma. In this disclosure, chroma LFNST is disabled in certain cases.

[0163] In one embodiment, chroma LFNST is disabled for a single tree.

[0164] In one embodiment, when chroma LFNST is disabled, the LFNST index is still signaled / parsed and can be used in luma.

[0165] In another embodiment, chroma LFNST is disabled.

[0166] In one embodiment, when chroma LFNST is disabled, the LFNST index is not signaled / parsed in the chroma dual tree.

[0167] In another embodiment, LFNST cannot be used in a TB even if the LFNST index of the CU containing the TB is greater than zero. A variable (applyLfnstFlag) is created to indicate whether LFNST can be used or not. If applyLfnstFlag is equal to 0, LFNST cannot be used. If applyLfnstFlag is equal to 1, LFNST can be used.

[0168] For example, for a single tree, chroma LFNST is disabled. The variable applyLfnstFlag is derived as follows: (where xTbY and yTbY represent the corresponding luma sample positions for the TB, cldx represents the component for the TB (e.g., cldx equal to 0 represents the luma component, cldx equal to 1 represents the Cb component, and cldx equal to 2 represents the Cr component), lfnst_idx is the LFNST index for the CU, and nTbW and nTbH represent the width and height of the TB), lfnst_idx is the LFNST index for the CU, and nTbW and nTbH represent the width and height of the TB.

[0169] - If (1) treeType is equal to SINGLE_TREE, (2) lfnst_idx is not equal to 0, (3) transform_skip_flag[ xTbY ][ yTbY ][ cldx ] is equal to 0, (4) cldx is equal to 0, and (5) both nTbW and nTbH are greater than or equal to 4, applyLfnstFlag is set equal to 1. (Any subset of (1) to (5) can be used in this condition)

[0170] - Otherwise, if (1) treeType is not equal to SINGLE_TREE, (2) lfnst_idx is not equal to 0, (3) transform_skip_flag[ xTbY ][ yTbY ][ cldx ] is equal to 0, and (4) both nTbW and nTbH are equal to greater than or equal to 4, applyLfnstFlag is set equal to 1. (Any subset of (1) to (4) can be used in this condition)

[0171] - Otherwise, applyLfnstFlag is set equal to 0.

[0172] For another example, the chroma LFNST is disabled. The variable applyLfnstFlag is derived as follows:

[0173] - If (1) lfnst_idx is not equal to 0, (2) transform_skip_flag[ xTbY ][ yTbY ][ cldx ] is equal to 0, (3) cldx is equal to 0, and (4) both nTbW and nTbH are greater than or equal to 4, applyLfnstFlag is set equal to 1. (Any subset of 1 to 4 can be used in this condition)

[0174] - Otherwise, applyLfnstFlag is set equal to 0.

[0175] In another embodiment, applyLfnstFlag can be used for one or more LFNST related sections. For example, the LFNST index is referenced in the corresponding section of the draft standard.

[0176] 8.7.4 Transform procedure for scaling the transform coefficients

[0177] … When applyLfnstFlag is equal to 1 / / lfnst_idx is not equal to 0 and transform_skip_flag[ xTbY ][ yTbY ][ cldx ] is equal to 0 and both nTbW and nTbH are greater than or equal to 4 / / , the following applies…

[0178] In the above modified text based on the draft standard, the text enclosed by a pair of " / / " indicates the deleted text.

[0179] 8.7.3 Scaling procedure for the transform coefficients

[0180] … Derivation of the scaled transform coefficients d[ x ][ y ] (with x = 0..nTbW-1, y = 0..nTbH-1) applies the following:

[0181] - The intermediate scaling factor m[ x ][ y ] is derived as follows:

[0182] - If one or more of the following conditions are true, m[ x ][ y ] is set equal to 16:

[0183] - sps_scaling_list_enabled_flag is equal to 0.

[0184] - pic_scaling_list_present_flag is equal to 0.

[0185] transform_skip_flag[ xTbY ][ yTbY ][ cldx ] is equal to 1.

[0186] scale_matrix_follnst_disabled_flag is equal to 1 and applyLfnstFlag is equal to 1 / / lfnst_idx[ xTbY ][ yTbY ] is not equal to 0 / / …

[0187] In the above modified text based on the standard draft, the text enclosed by a pair of " / / " indicates the deleted text.

[0188] In another embodiment, when chroma LFNST is disabled in some cases, LfnstDcOnly, which is initialized to 1 before parsing each TB in a CU and is changed to 0 if any TB in the CU has any significant coefficient (or last significant coefficient) bit at a position greater than the DC position, is not updated in non-LFNST TBs. For example, chroma LFNST is disabled in a single tree. Then, the non-LFNST TBs include chroma TBs for a single tree. An example of the corresponding changes in the syntax table is shown as follows.

[0189] Table 17. An exemplary syntax table for signaling / parsing of LFNST according to an embodiment of the present application

[0190]

[0191]

[0192] In another example, chroma LFNST is disabled, and the non-LFNST TBs include chroma TBs. An example of the corresponding changes in the syntax table is shown as follows.

[0193] Table 18. An exemplary syntax table for signaling / parsing of LFNST according to an embodiment of the present application

[0194]

[0195] When a TB has no cbf, no transform procedure needs to be applied. However, in this case, the LFNST index can still be signaled / parsed. For example, for a single tree, when the luma does not contain cbf, but the chroma satisfies the LFNST signaling / parsing condition (e.g., the chroma is not transform skipped and there is a coefficient at a non-DC position), the LFNST index can be signaled / parsed. In this case, the LFNST index is signaled / parsed and always has a value of 0 because LFNST is applied to the luma of the single tree and the index is redundant.

[0196] In an embodiment, LfnstDcOnlyFlag is only updated by Y (the LFNST TB to be applied) for a single tree.

[0197] In another embodiment, LfnstDcOnlyFlag is split into LfnstDCOnlyLumaFlag and lfnstDCOnlychromaFlag. LfnstDCOnlyLumaFlag is updated by Y TB and lfnstDCOnlychromaFlag is updated by Cb or Cr TB.

[0198] In an embodiment, for a single tree, only LfnstDCOnlyLumaFlag is considered for LFNST signaling / parsing.

[0199] In another embodiment, for a luma dual tree, only LfnstDCOnlyLumaFlag is considered for LFNST signaling / parsing.

[0200] In another embodiment, for a chroma dual tree, only lfnstDCOnlychromaFlag is considered for LFNST signaling / parsing.

[0201] In another embodiment, as follows, a check is added for LFNST signaling. If the luma does not have Cbf, the LFNST index is not signaled / parsed. An example of a syntax table is shown as follows.

[0202] Table 19. An example syntax table for LFNST signaling / parsing according to an embodiment of the application

[0203]

[0204]

[0205] In an embodiment, this check is added to the CU-level check.

[0206] In another embodiment, this check is added to the TB-level check.

[0207] In another embodiment, the check is performed on a single tree as shown in the following example.

[0208] Table 20. An example syntax table for signaling / profile of LFNST according to an embodiment of the application

[0209]

[0210] In another embodiment, the check is not limited to ISP mode. The reason is described as follows. When ISP is applied, one luma CB is split into multiple TBs (e.g. 4 TBs), and at least one significant coefficient is contained in one TB (the TB containing significant coefficient can be any one or more TBs within the CU). An example of syntax table is shown as follows.

[0211] Table 21A. An example syntax table for signaling / profile of LFNST according to an embodiment of the application

[0212]

[0213] Table 21B. An example syntax table for signaling / profile of LFNST according to an embodiment of the application

[0214]

[0215]

[0216] Table 21C. An example syntax table for signaling / profile of LFNST according to an embodiment of the application

[0217]

[0218] In the above tables 20 and 21A-C, lfnstNotTsFlag is a flag, where if lfnstNotTsFlag is equal to 1, LFNST mode can be applied (assuming other enabling conditions for LFNST are met). Otherwise, (i.e. lfnstNotTsFlag is equal to 0), LFNST mode cannot be applied (i.e. LFNST syntax is inferred to be disabled).

[0219] In another embodiment, the check is performed on luma dual tree.

[0220] In an embodiment, when lfnst index > 0 (i.e. lfnst is in progress), LFNST is applied to the first available component in a split tree. LFNST is not applied to other components.

[0221] In one embodiment, the first available component is the first component that contains a cbf. For example, for dual tree chroma, if cb contains a significant coefficient, the first available component is Cb; otherwise, if Cb cbf=0 and Cr contains a significant coefficient, the first available component is Cr; otherwise, if neither Cb nor Cr contains a significant coefficient, lfnst is not applied (i.e., the lfnst index is inferred to be 0 without signaling). In another example, for single tree, lfnst is applied to luma. In this case, if luma contains a significant coefficient, the first available component is luma; otherwise, lfnst is not applied.

[0222] In another embodiment, when JCCR mode is enabled, a single joint chroma residual block (resJointC[x][y] in Table 7) is signaled / parsed, and thus lfnst can be applied to this single block (i.e., LFNST affects both Cb and Cr).

[0223] In another embodiment, lfnst is signaled / parsed at the first available TB.

[0224] In one sub-embodiment, the first available TB is the first TB that contains a cbf. For example, for dual tree chroma, if cb contains a significant coefficient, the first available TB is Cb; otherwise, if Cb cbf=0 and Cr contains a significant coefficient, the first available TB is Cr; otherwise, if neither Cb nor Cr contains a significant coefficient, lfnst is not applied (i.e., the lfnst index is inferred to be 0 without signaling). In another example, lfnst is applied to luma. For single tree, if luma contains a significant coefficient, the first available component is luma; otherwise, the first available component is luma. Otherwise, lfnst is not signaled / parsed.

[0225] In another embodiment, when joint coding of chroma residuals (JCCR) is applied, for dual tree chroma, the first available TB is the chroma TB whose residual is used to derive the residual of the other chroma component. LFNST can be considered as applied to both chroma components. For example, for dual tree chroma, if tu_cbf_cb[x0][y0] > 0 and tu_cbf_cr[x0][y0] = 0 (the residual of Cr is derived from Cb and the residual related information of Cb is signaled), the first available TB is Cb. As another example, for dual tree chroma, if tu_cbf_cb[x0][y0] = 0 and tu_cbf_cr[x0][y0] > 0 (the residual of Cb is derived from Cr and the residual related information of Cr is signaled), the first available TB is Cr. As another example, for dual tree chroma, if tu_cbf_cb[x0][y0] > 0 and tu_cbf_cr[x0][y0] > 0 (the residual of Cr is derived from Cb and the residual related information of Cb is signaled), the first available TB is Cb.

[0226] In another embodiment, the lfnst index is signaled / parsed at the end of the first available TB. For example, after parsing the residual of each sub-block (coding group or 4x4 sub-block) in the TB, the lfnst index is parsed.

[0227] In another embodiment, the lfnst index is signaled / parsed after the signaling / parsing of the significant flag of the TB. The signaling / parsing condition for LFNST (e.g. LfnstDcOnlyFlag or LfnstZeroOutSigCoeffFlag) depends only on the information of the first available TB.

[0228] In another embodiment, for single tree, LFNST is only applied to luma. When luma cannot apply LFNST (e.g. no luma containing cbf = 1), even if chroma is available for signaling / parsing LFNST (e.g. chroma containing cbf or JCCR for chroma), the lfnst index is not signaled / parsed.

[0229] When a TB has no cbf, no transform procedure is needed. However, in this case, the transform skip (TS) flag can be equal to 1. For example, for a BDPCM block, the transform skip flag is inferred to be 1. However, this BDPCM block can not contain cbf. When considering LFNST signaling / parsing, a TB without cbf is considered as a TS block and since LFNST is not allowed for a TS block, the LFNST index cannot be signaled / parsed. Some methods are proposed to prevent the unexpected case.

[0230] In an embodiment, when considering signaling / parsing a LFNST index, a coded block flag (CBF) indication is used to prevent LFNST from being allowed in a CU containing one or more TBs with one or more TS flags not equal to 0 but actually not proceeded with TS procedure. First, a coded block flag (CBF) indication for the current CU is checked. The CBF indication for the current CU is denoted as cu_coded_flag (or cu_cbf).

[0231] cu_coded_flag equal to 1 indicates that the transform_tree( ) syntax structure is present in the current coding unit. cu_coded_flag equal to 0 indicates that the transform_tree( ) syntax structure is not present in the current coding unit. When cu_coded_flag is not present, it can be inferred as follows:

[0232] If cu_skip_flag[ x0 ][ y0 ] is equal to 1 or pred_mode_plt_flag is equal to 1, cu_coded_flag is inferred to be equal to 0.

[0233] Otherwise, cu_coded_flag is inferred to be equal to 1.

[0234] If the CBF indication for the current CU is true, the following checks are needed for one or more indications of one or more target TBs; otherwise, the LFNST syntax is inferred to be disabled.

[0235] In an embodiment, when considering signaling / parsing a LFNST index, in addition to checking the CBF indication for the current CU, tu_cbf is also checked. Thus, for a TB with TS flag equal to 1, if this TB does not contain cbf, it will not be considered as a TS block for LFNST signaling / parsing. lfnstNotTsFlag is updated as follows:

[0236] lfnstNotTsFlag = ( treeType == DUAL_TREE_CHROMA || ( transform_skip_flag[ x0 ][ y0 ][ 0 ] == 0 ||! tu_cbf_luma[ x0 ][ y0 ] ) ) && ( treeType == DUAL_TREE_LUMA || ( ( transform_skip_flag[ x0 ][ y0 ][ 1 ] == 0 ||! tu_cbf_cb[ x0 ][ y0 ] ) && ( transform_skip_flag[ x0 ][ y0 ][ 2 ] == 0 ||! tu_cbf_cr[ x0 ][ y0 ] ) ) )

[0237] The derivation of lfnstNotTsFlag shown above is based on at least one of two factors: TS mode indication is false and CBF indication is false. If each target TB satisfies at least one of factor 1 and factor 2, LFNST mode is allowed for the current CU. The allowance of LFNST mode for the current CU means that if other enabling conditions of LFNST are satisfied, LFNST syntax is signaled / parsed to indicate whether and / or which LFNST core is applied to the current CU when LFNST mode is applied. The TS mode indication depends on a transform skip (TS) flag. The CBF indication depends on a coded block flag (cbf) for a target TB. The Cbf for Y, Cb and Cr can be represented by tu_y_coded_flag (or tu_cbf_luma), tu_cb_coded_flag (or tu_cbf_cb) and tu_cr_coded_flag (or tu_cbf_cr).

[0238] tu_cb_coded_flag[ x0 ][ y0 ] equal to 1 specifies that the Cb transform block contains one or more non-zero transform coefficient levels. The array indices x0 and y0 specify the position (x0, y0) of the top-left luma sample of the considered transform block relative to the top-left luma sample of the picture.

[0239] When tu_cb_coded_flag[ x0 ][ y0 ] is not present, its value is inferred to be equal to 0.

[0240] tu_cr_coded_flag[ x0 ][ y0 ] equal to 1 specifies that the Cr transform block contains one or more non-zero transform coefficient levels. The array indices x0 and y0 specify the position (x0, y0) of the top-left luma sample of the considered transform block relative to the top-left luma sample of the picture.

[0241] When tu_cr_coded_flag[ x0 ][ y0 ] is not present, its value is inferred to be equal to 0.

[0242] tu_y_coded_flag[ x0 ][ y0 ] equal to 1 specifies that the luma transform block contains one or more non-zero transform coefficient levels. The array indices x0 and y0 specify the position (x0, y0) of the top-left luma sample of the considered transform block relative to the top-left luma sample of the picture.

[0243] When tu_y_coded_flag[ x0 ][ y0 ] is not present and treeType is not equal to DUAL_TREE_CHROMA, its value is inferred as follows:

[0244] - If cu_sbt_flag is equal to 1 and one of the following conditions is true, tu_y_coded_flag[ x0 ][ y0 ] is inferred to be equal to 0:

[0245] - subTuIndex is equal to 0 and cu_sbt_pos_flag is equal to 1 ;

[0246] - subTuIndex is equal to 1 and cu_sbt_pos_flag is equal to 0.

[0247] - Otherwise, tu_y_coded_flag[ x0 ][ y0 ] is inferred to be equal to 1.

[0248] For example, in a luma tree:

[0249] - If the TS flag for a target luma TB is equal to 0, lfnstNotTsFlag is set equal to 1.

[0250] - If the target luma TB does not contain significant coded data (CBF indication equal to false), lfnstNotTsFlag is set equal to 1.

[0251] - Otherwise: lfnstNotTsFlag is set equal to 0.

[0252] In another example, in a chroma tree:

[0253] - If the TS flag for all target Cb and Cr TBs is equal to 0, lfnstNotTsFlag is set equal to 1.

[0254] - If "all target TBs do not contain significant coded data", lfnstNotTsFlag is set equal to 1.

[0255] - If "each TS target TB (TS target TB = target TB with TS flag not equal to 0) does not contain significant coded data", lfnstNotTsFlag is set equal to 1.

[0256] - Otherwise: lfnstNotTsFlag is set equal to 0.

[0257] In another example, in a single tree:

[0258] - If the TS flag for all target luma, Cb, and Cr TBs is equal to 0, lfnstNotTsFlag is set equal to 1.

[0259] - If "all target TBs do not contain significant coded data", lfnstNotTsFlag is set equal to 1.

[0260] - If "every TS target TB (TS target TB = target TB with TS flag not equal to 0) does not contain valid coded data", lfnstNotTsFlag is set to 1.

[0261] - Otherwise, lfnstNotTsFlag is set to 0.

[0262] In an embodiment, the target TBs correspond to one or more TBs with a transform skip (TS) flag not equal to 0.

[0263] In an embodiment, in a luma partition tree, the current CU corresponds to a luma coding block, and the one or more target TBs correspond to one or more luma TBs. In another embodiment, in a chroma partition tree, the current CU corresponds to one or more chroma coding blocks, and the one or more target TBs correspond to one or more chroma TBs. For example, the chroma coding blocks are Cb and Cr coding blocks, and the chroma TBs are Cb and Cr TBs. In yet another embodiment, in a single partition tree, the current CU corresponds to a luma coding block and one or more chroma coding blocks, and the one or more target TBs correspond to one or more luma TBs and one or more chroma TBs. For example, the chroma coding blocks are Cb and Cr coding blocks, and the chroma TBs are Cb and Cr TBs.

[0264] In an embodiment, the target TBs correspond to a pre-defined TB for each coding block in the current CU. For example, the pre-defined TB corresponds to a first TB for each coding block in the current CU. The position of the first TB can be the position of the top-left luma sample of the considered transform block. The position of the first TB can be the position of the top-left luma sample of the considered coding block (CB in the current CU) corresponding to the top-left luma sample of the picture.

[0265] In an embodiment, the LFNST mode is allowed for the current CU (independent of the TS check) if all CBF indications for the target TBs are false. In another embodiment, the LFNST mode is allowed for the current CU if there is a target TB with a transform skip (TS) flag not equal to 0. In yet another embodiment, the LFNST mode is not allowed for the current CU if any of the CBF indications for the target TBs with a transform skip (TS) flag not equal to 0 is true.

[0266] In another embodiment, the TS flag will not be 1 when the block does not contain cbf (cbf equal to false). For example, if a BDPCM block does not have cbf, its TS flag will not be inferred to be 1. According to an embodiment of the disclosure, the following shows an example of modified semantics.

[0267] When transform_skip_flag[ x0 ][ y0 ][ cldx ] is not present, it can be inferred as follows:

[0268] - If BdpcmFlag[ x0 ][ y0 ][ cldx ] is equal to 1,

[0269] transform_skip_flag[ x0 ][ y0 ][ cldx ] is inferred as follows

[0270] - If cldx = 0 and tu_cbf_luma[ x0 ][ y0 ] = 1, transform_skip_flag[ x0 ][ y0 ][ cldx ] is inferred to be equal to 1.

[0271] - Else if cldx = 1 and tu_cbf_cb[ x0 ][ y0 ] = 1, transform_skip_flag[ x0 ][ y0 ][ cldx ] is inferred to be equal to 1.

[0272] - Else if cldx = 1 and tu_cbf_cr[ x0 ][ y0 ] = 1, transform_skip_flag[ x0 ][ y0 ][ cldx ] is inferred to be equal to 1.

[0273] - Else, transform_skip_flag[ x0 ][ y0 ][ cldx ] is inferred to be equal to 0.

[0274] In another embodiment, when chroma LFNST is disabled in certain cases, LfnstZeroOutSigCoeffFlag, which is initialized to 1 before parsing each TB in a CU and changed to 0 if any TB in the CU has any significant coefficient (or last significant coefficient) bit in the LFNST zero-out region, is not updated in non-LFNST TBs. For example, chroma LFNST is disabled in a single tree. Then, the non-LFNST TBs include chroma TBs for a single tree. An example of corresponding changes in the syntax table is shown below.

[0275] Table 22. An exemplary syntax table for signaling / parsing of LFNST according to an embodiment of the application

[0276]

[0277]

[0278] In another example, chroma LFNST is disabled, and non-LFNST TB includes chroma TB. An example of corresponding changes in the syntax table is shown as follows.

[0279] Table 23. An example of signaling / profile of LFNST according to an embodiment of the application

[0280]

[0281] Based on Table 15 (which refers to checking only luma transform_skip_flag when coding luma TU (e.g., not for DUAL_TREE_CHROMA), and checking only chroma transform_skip_flag when coding chroma TU (e.g., not for DUAL_TREE_LUMA)), chroma LFNST is disabled in some cases. For example, in a single tree, chroma LFNST is disabled. An example of the proposed syntax table is shown as follows.

[0282] Table 24. An example of signaling / profile of LFNST according to an embodiment of the application

[0283]

[0284] An example of the proposed syntax table based on Table 16 is also shown as follows.

[0285] Table 25. An example of signaling / profile of LFNST according to an embodiment of the application

[0286]

[0287] In another example, chroma LFNST is disabled. The proposed syntax table is shown as follows. An example of the proposed syntax table is shown as follows.

[0288] Table 26. An example of signaling / profile of LFNST according to an embodiment of the application

[0289]

[0290] In VVC Draft 7, LFNST and MTS indices are coded at the end of a CU, which introduces latency and buffer problem, a decoder needs to buffer all coefficients of all three color components before receiving MTS and LFNST indices. To reduce the latency and buffer problem, it is proposed to send MTS and / or LFNST enabling flag or index (e.g., 0 means disabled, 1 and 2 means enabled. 1 and 2 means different primary transform or different LFNST matrix is used) at the end of the first TB in a CU or at the end of one or more TBs of the first color component, or at the end of the first non-zero TB (and non-transform skip TB) in a CU or at the end of one or more TBs (and non-transform skip TB) of the first color component.

[0291] In another embodiment, it is proposed to send MTS and / or LFNST enabling flag or index at the end of the first non-zero TB (and non-transform skip TB) in a CU or at the end of one or more TBs of the first color component (and non-transform skip TB). In one example, it can only be applied to single tree. In a single tree, LFNST and MTS indices are signaled / parsed after the luma TB (or before the chroma TB). If single tree and ISP are applied, LFNST and MTS indices are signaled / parsed after the last luma TB (or before the chroma TB). For example, subTuIndex can be used. When subTuIndex is equal to NumIntraSubPartitions - 1, the current TB is luma TB and the current tree type is single tree, MTS and LFNST indices are signaled / parsed (if one or more conditions are met).

[0292] In another embodiment, it is proposed to send MTS and / or LFNST enabling flag or index at the following positions: in single tree case, at the end of one or more luma TBs in a CU (or before the chroma TB); while in luma dual tree, MTS and / or LFNST indices are signaled / parsed at the end of one or more luma TBs in a CU (or at the end of the CU); while in chroma dual tree, MTS and / or LFNST indices are signaled / parsed after the end of Cr TBs in a CU (or at the end of the CU). If single tree and ISP are applied, LFNST and MTS indices are signaled / parsed after the last luma TB (or before the chroma TB). For example, subTuIndex can be used. When subTuIndex is equal to NumIntraSubPartitions - 1, the current TB is luma TB and the current tree type is single tree, MTS and LFNST indices are signaled / parsed (if one or more conditions are met).

[0293] In another embodiment, when ISP mode is used, the MTS and / or LFNST enabling flag or index is signaled / parsed in the first TB (e.g., at the end of the first TB). The proposed method can only be applied to a single tree (e.g., in a luma dual tree or a chroma dual tree, MTS and / or LFNST is still signaled / parsed at the end of the CU).

[0294] In the above methods, the MTS index can be signaled / parsed after the LFNST index. If LFNST is used (e.g., LFNST index is not 0), the MTS index is inferred to be 0. In another embodiment, LFNST can be signaled / parsed after the MTS index is signaled / parsed. If MTS is used (e.g., MTS index is not 0), the LFNST index is inferred to be 0.

[0295] Any of the above proposed methods can be combined.

[0296] Any of the above variations can be implicitly determined by the block width or block height or block area, or explicitly determined by a flag signaled in the CU, CTU, slice, tile, tile group, SPS, PPS, or picture level. In this disclosure, "block" can refer to TU / TB / CU / CB / PU / PB.

[0297] Any of the above proposed methods can be implemented in an encoder and / or a decoder. For example, any of the proposed methods can be implemented in an inter / intra / transform coding module, a motion compensation module of an encoder, a merge candidate derivation module of a decoder. Any of the proposed methods can also alternatively be implemented as a circuit coupled to an inter / intra / transform coding module, and / or a motion compensation module of an encoder, a merge candidate derivation module of a decoder.

[0298] A decoding program embodying an embodiment of the disclosure can be understood based on the above description. For a decoding program for a system embodying LFNST, input data is coding data including a CU to be decoded. The decoding program then checks a CBF indication based on the coding data. A LFNST syntax is parsed according to the checking result. The CU is then decoded according to the LFNST syntax. Figure 3 A flowchart of an exemplary decoding system embodying LFNST (low frequency non-separable transform) parsing is shown according to an embodiment of the disclosure. The steps shown in the flowchart can be implemented as program code executable on one or more processors (e.g., one or more CPUs) at the encoder side. The steps shown in the flowchart can also be implemented based on hardware such as one or more electronic devices or processors arranged to perform the steps in the flowchart.

[0299] like Figure 3 As shown, in step 310, input data is received for a current codec unit (CU) in a current picture, wherein the current CU is partitioned into one or more transform blocks (TBs), and the input data corresponds to codec data for the current CU. In step 320, one or more codec block flag (CBF) indications are checked for one or more target TBs based on the codec data. In step 330, LFNST syntax is parsed if one or more conditions are met, wherein the LFNST syntax indicates whether an LFNST mode is applied to the current CU and / or, if so, which LFNST core is applied, and the one or more conditions include being false for the one or more target TBs. In step 340, the current CU is decoded according to the LFNST mode indicated by the LFNST syntax.

[0300] An encoding program embodying an embodiment of the present invention can be understood based on the above disclosure. For an encoding program embodying an LFNST system, the input data for the LFNST is primary converted data. The encoding program then applies LFNST based on the LFNST core to derive temporary output data. For example, if the check passes, the LFNST syntax is signaled. In another example, if the check fails (and the TS check also fails), the LFNST syntax is forced to zero. The LFNST syntax is determined and signaled based on the check result. The current CU is then encoded based on the determined LFNST syntax.

[0301] Figure 4 A flowchart of an exemplary encoding system embodying LFNST (Low Frequency Non-Separable Transform) analysis is shown according to an embodiment of the present invention. Figure 4 As shown, in step 410, input data for a current codec unit (CU) in a current picture is received, wherein the current CU is partitioned into one or more transform blocks (TBs), and the input data corresponds to primary transform data. In step 420, a low-frequency non-separable transform process is applied based on an LFNST kernel to derive temporary output data. In step 430, LFNST syntax is determined and signaled if one or more conditions are met, wherein the LFNST syntax indicates whether an LFNST mode is applied to the current CU and / or, if so, which LFNST kernel is applied, and the one or more conditions include one or more codec block flags (CBFs) indicating false for one or more target transform blocks (TBs). In step 440, the current CU is encoded according to the LFNST mode indicated by the determined LFNST syntax.

[0302] The flowchart shown is used to illustrate an example of video encoding / decoding according to the present application. Those skilled in the art having the benefit of the teachings herein can modify the depicted steps, reorganize the steps, separate one step into multiple steps, or combine steps while remaining within the spirit of the present application. In this disclosure, specific syntax and semantics have been used to illustrate examples of implementing embodiments of the present application. Those skilled in the art can implement the present application by replacing the syntax and semantics with equivalent syntax and semantics without departing from the spirit of the present application.

[0303] The above description enables those skilled in the art having the benefit of the teachings herein to implement the application in specific applications. Various modifications to the described embodiments will be apparent to those skilled in the art having the benefit of the teachings herein, and the general principles defined herein can be applied to other embodiments. Therefore, the present application is not to be limited to the specific embodiments shown and described, but is to be accorded the greatest scope consistent with the principles and novel features disclosed herein. In the above detailed description, various specific details are described in order to provide a thorough understanding of the present application. Nevertheless, it will be understood by those skilled in the art having the benefit of the teachings herein that the present application can be practiced

[0304] Embodiments of the application as described above can be implemented in various hardware, software, code, or combinations thereof. For example, embodiments of the application can be circuitry integrated within a video compression chip, or code integrated into video compression software to perform the processes described herein. An embodiment of the application can also be code executed on a Digital Signal Processor (DSP) to perform the processes described herein. The application can also include functions executed by a computer processor, a digital signal processor, a microprocessor, or a field programmable gate array (FPGA). According to the application, these processors can be configured to perform specific tasks according to the machine-readable software code or firmware code stored in a machine-readable medium that defines the particular machine- implemented process. The software code or firmware code can be developed in different programming languages and different formats or styles. The software code can also be compiled into different target platforms. However, different code formats, styles, and languages of the software code and other means of configuring code to perform the tasks of the application will not depart from the spirit and scope of the application. The embodiments of the application described above can be implemented in a video encoder and a video decoder. The components of the video encoder and the video decoder can be implemented by hardware components, one or more processors configured to execute program instructions stored in a memory, or a combination of hardware and processors. For example, the processors execute program instructions to control the reception of input data associated with a video sequence including a current block in a current picture. The processors are equipped with single or multiple processing cores. In some examples, the processors execute program instructions to perform functions in some components of the encoder and the decoder, and the memory electrically coupled to the processors is used to store the program instructions, information corresponding to reconstructed images of the blocks, and / or intermediate data in the encoding or decoding process. The memory in some embodiments includes a non-transitory computer-readable medium, such as a semiconductor or solid state memory, a random access memory (RAM), a read-only memory (ROM), a hard disk, an optical disk, or other suitable storage medium. The memory can also be a combination of two or more of the non-transitory computer-readable media listed above.

[0305] The application can be implemented in other specific forms without departing from the spirit or essential characteristics thereof. The described examples are to be considered in all respects only as illustrative and not restrictive. The scope of the application is, therefore, indicated by the appended claims, rather than by the foregoing description. All changes that come within the meaning of the claims are intended to be embraced therein as far as is permitted by the prior art.

Claims

1. A method of decoding a video sequence, wherein a low-frequency non-separable transform mode is supported, the method comprising: receiving input data regarding a current coding unit in a current picture, wherein the current coding unit is partitioned into one or more transform blocks, and the input data corresponds to coded data of the current coding unit; checking one or more coding block flag indications for one or more target transform blocks based on the coded data; if one or more conditions are satisfied, parsing a low-frequency non-separable transform syntax, wherein the low-frequency non-separable transform syntax indicates whether the low-frequency non-separable transform mode is applied in the current coding unit, and / or which low-frequency non-separable transform kernel is applied when the low-frequency non-separable transform mode is applied, and the one or more conditions comprise the one or more coding block flag indications for the one or more target transform blocks being false, wherein the one or more coding block flags of the one or more target transform blocks are coding block flags at a transform block level; and decoding the current coding unit according to the low-frequency non-separable transform mode indicated by the low-frequency non-separable transform syntax.

2. The method of decoding a video sequence according to claim 1, wherein, The one or more target transform blocks correspond to one or more target transform blocks having a transform skip flag not equal to 0.

3. The method of decoding a video sequence according to claim 1, wherein, In a luma partition tree, the current coding unit corresponds to a luma coding block, and the one or more target transform blocks correspond to one or more luma transform blocks.

4. The method of decoding a video sequence according to claim 1, wherein, In a chroma partition tree, the current coding unit corresponds to one or more chroma coding blocks, and the one or more target transform blocks correspond to one or more chroma transform blocks.

5. The method of decoding a video sequence according to claim 1, wherein, In a single partition tree, the current coding unit corresponds to a luma coding block and one or more chroma coding blocks, and the one or more target transform blocks correspond to one or more luma transform blocks and one or more chroma transform blocks.

6. The method of decoding a video sequence according to claim 1, wherein, The one or more target transform blocks correspond to a predefined transform block for each coding block in the current coding unit.

7. The method of decoding a video sequence according to claim 6, wherein, The predefined transform block corresponds to a first transform block for each coding block in the current coding unit.

8. The method of decoding a video sequence according to claim 1, wherein, The low-frequency non-separable transform mode is allowed in the current coding unit if all the one or more coding block flag indications for the one or more target transform blocks are false.

9. The method of decoding a video sequence according to claim 1, wherein, The low-frequency non-separable transform mode is allowed in the current coding unit if all the one or more coding block flag indications for the one or more target transform blocks are false, and the one or more target transform blocks indicate one or more transform blocks having a transform skip flag not equal to 0.

10. The method of Claim 1, wherein, The low-frequency non-separable transform mode is not allowed in the current coding unit if any of the one or more coding block flag indications for the one or more target TBs has a transform skip flag not equal to 0 is true.

11. A decoding apparatus of a video sequence, characterized by, A low-frequency non-separable transform mode is supported, the apparatus comprising one or more electronic circuits or processors arranged to: receiving input data regarding a current coding unit in a current picture, wherein the current coding unit is partitioned into one or more transform blocks, and the input data corresponds to coded data of the current coding unit; checking one or more coding block flag indications for one or more target transform blocks based on the coded data; if one or more conditions are satisfied, parsing a low-frequency non-separable transform syntax, wherein the low-frequency non-separable transform syntax indicates whether the low-frequency non-separable transform mode is applied to the current coding unit, and / or which low-frequency non-separable transform kernel is applied when the low-frequency non-separable transform mode is applied, and the one or more conditions include the one or more coding block flag indications for the one or more target transform blocks being false, wherein the one or more coding block flags of the one or more target transform blocks are coding block flags at a transform block level; and decoding the current coding unit according to the low-frequency non-separable transform mode indicated by the low-frequency non-separable transform syntax.

12. A method of encoding a video sequence, characterized by, A low-frequency non-separable transform mode is supported, the method comprising: receiving input data regarding a current coding unit in a current picture, wherein the current coding unit is partitioned into one or more transform blocks, and the input data corresponds to primary post-transform data; applying a low-frequency non-separable transform process based on a low-frequency non-separable transform kernel to derive temporary output data; if one or more conditions are satisfied, deciding and signaling a low-frequency non-separable transform syntax, wherein the low-frequency non-separable transform syntax indicates whether the low-frequency non-separable transform mode is applied to the current coding unit, and / or which low-frequency non-separable transform kernel is applied when the low-frequency non-separable transform mode is applied, and the one or more conditions include the one or more coding block flag indications for one or more target transform blocks being false, wherein the one or more coding block flags of the one or more target transform blocks are coding block flags at a transform block level; and encoding the current coding unit according to the low-frequency non-separable transform mode indicated by the decided low-frequency non-separable transform syntax.

13. An apparatus for encoding a video sequence, the apparatus comprising: A low-frequency non-separable transform mode and a transform skip mode are supported, the apparatus comprising one or more electronic circuits or processors arranged to: receive input data regarding a current coding unit in a current picture, wherein the current coding unit is partitioned into one or more transform blocks, and the input data corresponds to primary post-transform data; apply a low-frequency non-separable transform process based on a low-frequency non-separable transform kernel to derive temporary output data; determining to signal a low-frequency non-separable transform syntax if one or more conditions are met, wherein the low-frequency non-separable transform syntax indicates whether the low-frequency non-separable transform mode is applied in the current coding unit, and / or which low-frequency non-separable transform kernel is applied when the low-frequency non-separable transform mode is applied, and the one or more conditions comprise one or more coding block flags for one or more target transform blocks being false, wherein the one or more coding block flags for the one or more target transform blocks are coding block flags at a transform block level; and encoding the current coding unit according to the low-frequency non-separable transform mode indicated by the determined low-frequency non-separable transform syntax.

Images