Method and apparatus for conversion type assignment for intra partitions in video coding

Abstract

A method and apparatus for prediction in video coding are disclosed. According to the method, when an Intra Sub-Partition (ISP) mode is applied to a block, the block is divided horizontally or vertically into a plurality of sub-blocks. A target horizontal transform and a target vertical transform are determined for each of the plurality of sub-blocks from a candidate transform set according to a target setting belonging to a setting group including a unified setting and a block setting. Then, the selected target horizontal transform and target vertical transform are applied to each of the plurality of sub-blocks. According to another method, a target horizontal and vertical transform for a current block is determined from a candidate transform set without a transform index according to a unified setting, where the unified setting includes two or more Intra modes or Intra-related modes.

Description

[0001]

Cross - reference to related applications

[0002] This invention claims the priority of U.S. Provisional Patent Application No. 62 / 799,127 filed on January 31, 2019 and U.S. Provisional Patent Application 62 / 813,810 filed on March 5, 2019. The contents of the above U.S. Provisional Patent Applications are hereby incorporated by reference into this case.

Technical Field

[0003] This invention relates to the prediction of video coding and decoding using the ISP (Intra Sub - Partition) mode. In particular, this invention discloses the selection of transform types for the ISP mode to improve coding and decoding performance or simplify the coding and decoding process.

Background Art

[0004] High - Efficiency Video Coding (HEVC) is a new international video coding and decoding standard developed by the Joint Collaborative Team on Video Coding (JCT - VC). HEVC is based on a hybrid block - based motion - compensated DCT - like transform coding architecture. The basic unit of compression is called a coding unit (CU), which is a 2Nx2N square block. Each CU can be recursively divided into four smaller CUs until a predefined minimum size is reached. Each CU contains one or more prediction units (PUs).

[0005] To achieve the best coding and decoding efficiency of the hybrid coding architecture in HEVC, each PU has two prediction modes (intra - prediction and inter - prediction). For the intra - prediction mode, spatially adjacent reconstructed pixels can be used to generate directional predictions. There are up to 35 directions in HEVC. For the inter - prediction mode, temporally reconstructed reference frames can be used to generate motion - compensated predictions. There are three different modes, including skip, merge, and inter - frame advanced motion vector prediction (AMVP) mode.

[0006] When encoding or decoding a PU in the inter-frame AMVP mode, motion compensation prediction is performed using the transmitted motion vector difference (MVD), which can be used together with the motion vector predictor (MVP) to derive the motion vector (MV). To determine the MVP in the inter-frame AMVP mode, the advanced motion vector prediction (AMVP) scheme is used to select a motion vector predictor from an AMVP candidate set that includes two spatial MVPs and one temporal MVP. Therefore, in the AMVP mode, it is necessary to encode and transmit the MVP index of the MVP and the corresponding MVD. Additionally, in bidirectional prediction and unidirectional prediction for list 0 (i.e., L0) and list 1 (i.e., L1), the inter-frame prediction direction that specifies the inter-frame prediction direction should also be encoded and transmitted along with the reference frame index for each list.

[0007] When encoding or decoding a PU in the skip or merge mode, the skip and merge mode utilizes a motion inference method and does not transmit motion information except for the selected candidate merge index. Since the motion vector difference (MVD) is zero in the skip and merge modes, the MV of the skipped or merged decoded block is the same as the motion vector predictor (MVP) (i.e., MV = MVP + MVD = MVP). Therefore, the skipped or merged decoded block obtains motion information from spatially adjacent blocks (spatial candidates) or temporal blocks (temporal candidates) in the co-located picture at the same position. The co-located picture is the first reference picture in list 0 or list 1 and is signaled in the slice header. In the case of a skipped PU, the residual signal is also omitted. To determine the merge index for the skip and merge modes, a merge scheme is used to select a motion vector predictor from a merge candidate set that includes four spatial MVPs and one temporal MVP.

[0008] Figure 1 Adjacent PUs are shown for deriving both spatial and temporal MVPs for both the AMVP and the merge scheme. In the AMVP, the left MVP is the first available among A0, A1, the top MVP is the first available among B0, B1, B2, and the temporal MVP is T BR or T CTR the first available of (first use T BR , if T BR is not available, then use T CTR ). If the left MVP is not available and the top MVP is not a scaled MVP, a second top MVP can be derived when there is a scaled MVP among B0, B1, and B2. The list size of the MVP for the AMVP is 2. Therefore, after the derivation of the two spatial MVPs and one temporal MVP, only the first two MVPs can be included in the MVP list. If the number of available MVPs is less than two after removing redundancy, a zero vector candidate is added to the candidate list.

[0009] For skip and merge modes, as Figure 1 shown, up to four spatial merge indices are derived from A0, A1, B0, and B1, and one temporal merge index is derived from T BR or T CTR (first using T BR , and if T BR is not available, then using T CTR ). Note that if any of the four spatial merge indices is not available, then position B2 is then used to derive a merge index as a replacement. After the derivation of the four spatial merge indices and one temporal merge index, redundancy removal is applied to remove redundant merge indices. If the number of available merge indices is less than five after removing redundancy, then three types of additional candidates are derived and added to the candidate list.

[0010] Other bi-predictive merge candidates are created by using the original merge candidates. The other candidates are divided into three candidate types:

[0011] 1. Merged bi-predictive merge candidates (candidate type 1)

[0012] 2. Scaled bi-predictive merge candidates (candidate type 2)

[0013] 3. Zero vector merge / AMVP candidates (candidate type 3)

[0014] In candidate type 1, merged bi-predictive merge candidates are created by combining the original merge candidates. In particular, two candidates among the original candidates, having mvL0 (motion vector in list 0) and refIdxL0 (reference picture index in list 0) or mvL1 (motion vector in list 1) and refIdxL1 (reference picture index in list 1), are used to create bi-predictive merge candidates. Figure 2 An example of the derivation process for merged bi-predictive merge candidates is shown. Candidate list 210 corresponds to the original candidate list, which includes mvL0_A, ref0(231) in L0 and mvL1_B, ref(232) in L1. In candidate set 220, a bi-predictive MVP 233 can be formed by combining candidates in L0 and L1.

[0015] In candidate type 2, scaled bi-predictive merge candidates are created by scaling the original merge candidates. Specifically, one of the original candidates (with mvLX (motion vector in list X) and refIdxLX (reference picture index in list X), where X can be 0 or 1) is used to create the bi-predictive merge candidate. For example, a candidate A is a list 0 uni-predictive with mvL0_A and ref0. First, ref0 is copied to the reference index ref0' in list 1. Thereafter, mvL0'_A is calculated by scaling mvL0_A using ref0 and ref0'. Then, a bi-predictive merge candidate with mvL0_A and ref0 in list 0 and mvL0'_A and ref0' in list 1 is created and added to the merge candidate list. An example of the derivation process of the scaled bi-predictive merge candidate is shown in Figure 3 which shows that candidate list 310 corresponds to the original candidate list, and candidate list 320 corresponds to the extended candidate list including two generated bi-predictive MVPs.

[0016] In candidate type 3, zero vector merge / AMVP candidates are created by combining a zero vector and a reference index. Figure 4A An example for adding a zero vector merge candidate is shown, where candidate list 410 corresponds to the original merge candidate list, and candidate list 420 corresponds to the extended merge candidate list by adding zero candidates. Figure 4B An example for adding a zero vector AMVP candidate is shown, where candidate lists 430 (L0) and 432 (L1) correspond to the original AMVP candidate lists, and candidate lists 440 (L0) and 442 (L1) correspond to the extended AMVPs by adding zero candidates. If the zero vector candidates are not duplicates, they are added to the merge / AMVP candidate list.

[0017] When encoding or decoding a PU in intra mode, the intra prediction method generates a prediction for the current PU using only one reference layer (or line) adjacent to the current prediction unit (PU) and one of the intra prediction modes. The reference layer adjacent to the current prediction unit (PU) refers to the reference L-shaped reconstructed samples used for intra prediction. For intra prediction modes, spatially adjacent reconstructed pixels can be used to generate directional predictions. There are up to 35 directions in HEVC. Among all 35 intra prediction modes in HEVC, 3 modes are considered the most probable modes (MPM) for predicting the intra prediction mode in the current prediction block. Three modes are selected as the MPM set. For example, the MPM set includes the intra prediction modes used in the left prediction block and the upper prediction block. If the intra prediction modes in two adjacent blocks are the same and both are directional, or only one of the two adjacent blocks is available and is encoded or decoded in intra prediction, and at the same time this intra prediction mode is directional, then the two adjacent directions adjacent to this direction are also used in the MPM. The DC mode and the planar mode are also considered in the MPM set to fill the available slots in the MPM, especially when the upper or top adjacent block is not available or not encoded or decoded in intra prediction, or the intra prediction mode in the adjacent block is not directional. If the intra prediction mode used for the current prediction block is one of the modes in the MPM set, 1 or 2 bits (bins) are used to indicate which one. Otherwise, it is different from any entry in the MPM set and it will be encoded or decoded as a non-MPM mode. There are a total of 32 such non-MPM modes, and a (5-bit) fixed-length encoding or decoding method is used to signal this mode. 33 directions are as Figure 5 shown. In Figure 5 , there are a total of 33 directional modes, so H, V, The system can be extended to the general case where the horizontal and vertical modes are represented as H and V modes. For other directional modes, they can be represented as H + k or V + k modes, where k = ±1, ±2, etc. For example, if 65 directional modes are used, the range of k can be from ±1 to ±16.

[0018] Regarding signaling, in the Joint Exploration Test Model (JEM) (J. Chen et al., "Algorithm Description of Joint Exploration Test Model 7 (JEM 7)", ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, 7th meeting: Turin, Italy, July 2017, document: JVET-G1001), the following method is used to encode the intra prediction mode of the luminance component.

[0019] ·Encoding and decoding of the MPM mode: Generate a list of 6 Most Probable Modes (MPM) from adjacent modes. If the prediction mode is one of the modes in the MPM list, truncated unary coding is used.

[0020] ·Encoding and decoding of non-MPM modes: If the best prediction mode does not belong to the MPM list, a fixed-length code or a truncated binary method is used.

[0021] Generating the MPM list in the JEM test model

[0022] In JEM, the modes included in the MPM list are divided into three categories:

[0023] ·Intra mode of adjacent blocks (i.e., adjacent intra mode)

[0024] ·Derived intra mode

[0025] ·Default intra mode

[0026] Five adjacent intra prediction modes are used to form the MPM list. Figure 6 The positions of 5 adjacent modes are shown, namely left (L), above (A), bottom left (BL), top right (AR), and top left (AL). The list is formed by inserting 5 adjacent intra modes, the planar mode, and the DC mode into the MPM list. A pruning process is used to remove duplicate modes so that only unique modes can be included in the MPM list. The order of the modes in the MPM list is initialized as: left, above, planar, DC, bottom left, top right, and then top left.

[0027] If the MPM list is incomplete (i.e., there are fewer than 6 MPM candidates in the list), then a derived mode is added. The derived mode can be obtained by adding -1 or +1 to the angular modes already included in the MPM list. Such additional derived modes are not generated from non-angular modes (i.e., DC or planar modes).

[0028] Finally, if the MPM list is still incomplete, the default modes are added to the list in the following order: vertical, horizontal, mode 2, and diagonal mode. As a result of this process, a unique list of 6 MPM modes is generated.

[0029] The encoding and decoding of the 61 non-MPMs used to select the residuals are as follows. First, the 61 non-MPMs are divided into two groups: a selected mode set and an unselected mode set. The selected mode group contains 16 modes, and the remaining modes (i.e., 45 modes) are assigned to the unselected mode group. The mode set to which the current mode belongs is indicated by a flag in the bitstream. If the mode to be indicated is in the selected mode set, the selected mode is signaled using a 4-bit fixed-length code.

[0030] If the mode to be indicated is from the unselected set, the selected mode is signaled using a truncated binary code. The selected mode set is generated by subsampling the 61 non-MPM modes as follows:

[0031] Selected mode set = {{0, 4, 8, 12, 12, 20…60}, and

[0032] Unselected mode set = {{1, 2, 3, 5, 6, 7, 9, 10…59}.

[0033] To improve the transform, in JCTVC-B024 (C. Yeo et al., "Mode-Dependent Fast Separable KLT for Block-based Intra Coding", Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11, 2nd meeting: Geneva, Switzerland, July 21 - 28, 2010, Document: JCTVC-B024), JCTVC-C108 (A. Saxena et al., "Jointly optimal intra prediction and adaptive primary transform", Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11, 3rd meeting: Guangzhou, China, October 7 - 15, 2010, Document: JCTVC-C108), JCTVC-E125 (A. Saxena et al., "CE7: Mode-dependent DCT / DST without 4*4 full matrix multiplication for intra prediction", Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11, 5th meeting: Geneva, Switzerland, March 16 - 23, 2011, Document: JCTVC-E125), the discrete sine transform (DST) was introduced to be used interchangeably with the DCT for the oblique Intra mode. For the residual of inter prediction, DCT-II is the only transform used in current HEVC.

[0034] However, the DCT-II is not the best transform for all cases. In JCTVC-G281 (J. An et al., "Non-CE7: Boundary-Dependent Transform for Inter-Predicted Residue", Joint Collaborative Team on Video Coding (JCT-VC) of ITU-TSG16 WP3 and ISO / IEC JTC1 / SC29 / WG11, 7th meeting: Geneva, Switzerland, November 21-30, 2011, Document: JCTVC-G281), the Type-VII discrete sine transform (DST-VII) and the Type-IV discrete cosine transform (DCT-IV)) were proposed to replace the DCT-II in some cases. In JVET-D1001 (J. Chen et al., "Algorithm Description of Joint Exploration Test Model 4", Joint Video Exploration Team (JVET) of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11, 4th meeting: Chengdu, China, October 15-21, 2016, Document: JVET-D1001), an adaptive multi-transform (AMT) scheme was also proposed for residual coding of intra- and inter-coded blocks. In addition to the current transform in HEVC, it also utilizes multiple selected transforms from the DCT / DST family. The newly introduced transform matrices are DST-VII, DCT-VIII, DST-I, and / or DCT-V. For intra-residual coding, due to the different residual statistics of different intra-prediction modes, a mode-dependent transform candidate selection process is used. As shown in Table 1, three transform subsets are defined, and the transform subsets are selected according to the intra-prediction mode, as shown in Table 2; alternatively, only one transform subset in Table 1 is used.

[0035] Table 1: Three predefined transform candidate sets

[0036]

[0037] Table 2: (H) horizontal and (V) vertical transform sets selected for each intra-prediction mode

[0038]

[0039] Intra Sub-Partition (ISP) mode

[0040] In JVET-M0102 (S. De-Luxán-Hernández et al., "CE3: Intra Sub-Partitions Coding Mode (Tests 1.1.1 and 1.1.2)", ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11, 2nd meeting: Marrakech, MA, January 2019, Document: JVET-M0102), the Intra Sub-Partition (ISP) codec mode is an updated version of the Line-Based Intra (LIP) codec, which can correct hardware-related problems in the previous design of the algorithm. The ISP tool divides the blocks of the luminance intra prediction vertically or horizontally into 2 or 4 sub-partitions according to the block size, as shown in Table 3. Figure 7A and Figure 7B shows examples of these two possibilities. In Figure 7A , the HxW block 710 is divided into two H / 2xW blocks 720 (i.e., horizontal partition) or two HxW / 2 blocks 730 (i.e., vertical partition). For example, the block can be a 4×8 block or an 8×4 block. In Figure 7B , in addition to 4x8, 8x4, and 4x4 blocks, the HxW block 710 is divided into four H / 4xW blocks 740 (i.e., horizontal partition) or four HxW / 4 blocks 750 (i.e., vertical partition). All sub-partitions meet the condition of having at least 16 samples.

[0041] Table 3: Number of sub-partitions depending on the block size

[0042] Block size Number of sub - partitions 4×4 Not divided 4×8 and 8×4 2 Other cases 4

[0043] For each of these sub-partitions, a residual signal is generated by entropy decoding the coefficients sent by the codec, then inverse quantizing and inverse transforming them. Then, intra prediction is performed on the sub-partition, and finally the corresponding reconstructed samples are obtained by adding the residual signal and the prediction signal. Therefore, the reconstructed values of each sub-partition will be available for generating the prediction of the next partition, which will repeat the process, and so on. All sub-partitions share the same intra mode.

[0044] Based on the intra mode and the utilized partitioning, two different categories of processing orders are used, which are referred to as the normal order and the reverse order; or the normal order and the reverse order are fixedly used. In the normal order, the first sub-partition to be processed is the sub-partition containing the top-left sample of the CU, and then it continues downwards (horizontal split) or to the right (vertical split). As a result, the reference samples for generating the sub-partition prediction signal are only located to the left and above the line. On the other hand, the reverse processing order either starts from the sub-partition containing the bottom-left sample of the CU and then continues upwards, or starts from the sub-partition containing the top-right sample of the CU and then continues to the left.

[0045] A more detailed description of this algorithm can be found in JVET-M0102.

[0046] In the development of the VVC software, the transform process for the scaled transform coefficients can be described as follows.

[0047] The inputs to this process are:

[0048] – The luma position (xTbY, yTbY), specifying the top-left sample of the current luma transform block relative to the top-left luma sampling of the current picture.

[0049] – The variable nTbW, specifying the width of the current transform block.

[0050] – The variable nTbH, specifying the height of the current transform block.

[0051] – The variable cIdx, specifying the color component of the current block.

[0052] – The scaled transform coefficient (nTbW)x(nTbH) array d[x][y], where x = 0..nTbW-1 and y = 0..nTbH-1.

[0053] The output of this process is the (nTbW)x(nTbH) array r[x][y] of residual samples, where x = 0..nTbW-1 and y = 0..nTbH-1.

[0054] The derivation of the variable implicitMtsEnabled is as follows: (When multiple transform selection (denoted as MTS) is enabled, implicitMtsEnabled equal to 0 means that the transform type is determined by the transmitted transform index, while implicitMtsEnabled equal to 1 means that the transform type is determined using the implicit transform setting. In some cases, ImplicitMtsEnabled is set to 1. The following is an example.

[0055] · If sps_mts_enabled_flag is equal to 1 (which means multiple transform selection is enabled), and one of the following conditions is met, then implicitMtsEnabled is set to be equal to 1:

[0056] – IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT (which means ISP mode is used)

[0057] – cu_sbt_flag is equal to 1, and Max(nTbW, nTbH) is less than or equal to 32

[0058] – sps_explicit_mts_intra_enabled_flag and / or sps_explicit_mts_inter_enabled_flag is equal to 0, and CuPredMode[xTbY][yTbY] is equal to MODE_INTRA, and low frequency non-separable transform (LFNST) and / or matrixed-based intra prediction (MIP) are not used.

[0059] · Otherwise, implicitMtsEnabled is set to be equal to 0.

[0060] The conditions for implicitMtsEnable can be any subset of the above conditions.

[0061] The variable trTypeHor that specifies the horizontal transform kernel and the variable trTypeVer that specifies the vertical transform kernel are derived as follows:

[0062] A value of 0 for trTypeHor or trTypeVer indicates the default transform type, such as DCT-II. A value greater than 0 for trTypeHor or trTypeVer indicates the transform type selected from MTS. For example, a value of 1 for trTypeHor or trTypeVer indicates DST-VII, and a value of 2 for trTypeHor or trTypeVer indicates DCT-VIII.

[0063] · If cIdx is greater than 0 (the current TB is a chrominance TB), then trTypeHor and trTypeVer are set to be equal to 0.

[0064] · If IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT and lfnst_idx is not equal to 0, then trTypeHor and trTypeVer are set to be equal to 0.

[0065] · Otherwise, if implicitMtsEnabled is equal to 1, different methods can be used to determine trTypeHor or trTypeVer.

[0066] The following is an example.

[0067] – If IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT, then specify trTypeHor and trTypeVer according to intraPredMode in Table 6.

[0068] – Otherwise, if cu_sbt_flag is equal to 1, then specify trTypeHor and trTypeVer according to cu_sbt_horizontal_flag and cu_sbt_pos_flag in Table 5.

[0069] – Otherwise, (sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag are equal to 0), trTypeHor and trTypeVer are obtained as follows:

[0070] trTypeHor = (nTbW >= 4 && nTbW <= 16 && nTbW <= nTbH)? 1 : 0 (1)

[0071] trTypeVer = (nTbH >= 4 && nTbH <= 16 && nTbH <= nTbW)? 1 : 0 (2)

[0072] Otherwise, specify trTypeHor and trTypeVer according to tu_mts_idx[xTbY][yTbY] in Table 4.

[0073] Table 4 – Specification of trTypeHor and trTypeVer depends on tu_mts_idx[x][y]

[0074]

[0075] Table 5 - Specification of trTypeHor and trTypeVer depending on cu_sbt_horizontal_flag and cu_sbt_pos_flag

[0076] cu_sbt_horizontal_flag cu_sbt_pos_flag trTypeHor trTypeVer 0 0 2 1 0 1 1 1 1 0 1 2 1 1 1 1

[0077] Table 6 - Specification of trTypeHor and trTypeVer depending on predModeIntra

[0078]

[0079]

[0080] When developing new coding tools and applying them to transformation, multiple transformation types can be selected, such as DCT-II, DST-VII, or DST-VIII, to improve coding efficiency. For horizontal and vertical transformations, instead of setting DCT-II as the default transformation mode (which is indicated by the shortest codeword compared to other transformation modes), some criteria can be used to select the default transformation types for horizontal and vertical transformations for inter-frame or intra-frame blocks.

[0081] In VTM, when coding / decoding a CU in merge mode and if the CU contains at least 64 luma samples (i.e., the CU width times the CU height is equal to or greater than 64), a signaling additional flag is sent to indicate whether the combined Inter / Intra prediction (CIIP) mode is applied to the current CU. To form the CIIP prediction, first the intra prediction mode is derived from two additional syntax elements. Up to four intra prediction modes DC, planar, horizontal or vertical can be used. Then, the inter prediction and intra prediction signals are derived using the regular intra and inter decoding processes. Finally, the inter and intra prediction signals are weighted and averaged to obtain the CIIP prediction. A more detailed description of this algorithm can be found in JVET-L0100 (M.-S. Chiang et al., “CE10.1.1: Multi-hypothesis prediction for improving AMVP mode, skip or merge mode, and Intra mode”, ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, 12th meeting: Macau, China, October 2018, Document: JVET-L0100).

[0082] It should be noted that the description of the background art provided herein is for the purpose of generally presenting the context of the present invention. The subject matter discussed in the background art of the invention section should not be regarded as prior art merely because it is mentioned in the background art of the invention section. Similarly, the problems mentioned in the background art of the invention section, or problems related to the subject matter of the background of the invention section, should not be regarded as problems that have been previously recognized in the prior art.

Summary of the Invention

[0083] A method and apparatus for prediction in video coding and decoding are disclosed. According to this method, input data related to a current block in a current picture is received at a video encoder side, or a video bitstream corresponding to compressed data including the current block in the current picture is received at a video decoder side. If the current block uses an Intra Sub-Partition (ISP) mode, the following steps are performed: the current block is horizontally or vertically divided into a plurality of sub-blocks; according to a set of settings including unified settings or block settings, a target horizontal transform and a target vertical transform are determined for each of the plurality of sub-blocks from a candidate transform set, wherein the unified settings are applicable to a specific coding and decoding mode, a normal intra mode, or other intra-related modes, and the block settings are based on the block width, block height, block ratio, block size, or any combination thereof of the current block or sub-block; the target horizontal transform and the target vertical transform are applied to each of the plurality of sub-blocks. The current block in the present invention can be a Coding Unit (CU or CB), a Prediction Unit (PU or PB), or a Transform Unit (TU or TB). When the current block represents a coding block, the sub-blocks of the current block are transform blocks.

[0084] Another transform method and apparatus for video coding and decoding are also disclosed. According to this method, when it is not necessary to use / has been signaled a transform index to select a target horizontal transform and a target vertical transform, a target horizontal transform and a target vertical transform for the current block are determined from a candidate transform set according to unified settings, wherein the unified settings include two or more intra modes or intra-related modes. Then the target horizontal transform and the target vertical transform are applied to the current block.

[0085] In one embodiment, the target horizontal transform and the target vertical transform belong to a subset including any combination of DCT-II, DST-VII, DCT-VIII, and flipped DST-VII. For example, a pair representing (target horizontal transform, target vertical transform) corresponds to (DCT-II, DCT-II), (DST-VII, DCT-II), (DCT-II, DST-VII), or (DST-VII, DST-VII).

[0086] In one embodiment, the target horizontal transform and the target vertical transform are determined based on the block width, block height, block ratio, block size of the current block, or any combination thereof. For example, when the block width is less than or equal to the block height, the target horizontal transform is changed from DCT-II to another transform type. In another example, when the block width is in the range of [4, 16], the block width is less than or equal to the block height, or both block widths are in the range of [4, 16] and the block width is less than or equal to the block height, the target horizontal transform is changed from DCT-II to DST-VII. In yet another example, when the block height is within [4, 16], the block height is less than or equal to the block width, or both block heights are within [4, 16] and the block height is less than or equal to the block width, the target vertical transform is changed from DCT-II to DST-VII.

[0087] In one embodiment, the unified setting includes a normal intra mode. The candidate transform set may be the same as the normal transform set for the normal intra mode. For the target horizontal transform and the target vertical transform, the shortest codeword of the candidate transform in the candidate transform set may be set to DST-VII. In another embodiment, the ISP mode uses the same intra prediction mode list as the normal intra mode. In another embodiment, the ISP mode uses a subset of the normal intra prediction mode list as its normal intra mode list. In yet another embodiment, the ISP mode is disabled according to the block width, block height, or block size.

[0088] In one embodiment, the target setting is determined implicitly. In another embodiment, the target setting is signaled at the CU (Coding Unit), CTU (Coding Tree Unit), slice, tile, tile group, SPS (Sequence Parameter Set), or PPS (Picture Parameter Set) level.

[0089] In one embodiment, the setting group further includes a mode setting, and the mode setting is based on the intra prediction mode selected for the current block. For example, the selected intra prediction mode is a directional mode and should satisfy (directional mode % 2 == 1). In another example, the selected intra prediction mode is a directional mode and should satisfy (directional mode % 2 == 0).

Description of the Drawings

[0090] Figure 1 Adjacent PUs for deriving spatial and temporal MVPs for both the AMVP and merge schemes are shown.

[0091] Figure 2 An example of the derivation process for combined bidirectional prediction merge candidates is shown.

[0092] Figure 3An example of the derivation process of scaled bi - directional prediction merge candidates is shown, where the candidate list on the left corresponds to the original candidate list, and the candidate list on the right corresponds to the extended candidate list, which includes two generated bi - directional prediction MVPs.

[0093] Figure 4A An example of adding zero - vector merge candidates is shown, where the candidate list on the left corresponds to the original merge candidate list, and the candidate list on the right corresponds to the extended merge candidate list by adding zero candidates.

[0094] Figure 4B An example of adding zero - vector AMVP candidates is shown, where the candidate list at the top corresponds to the original AMVP candidate list (L0 on the left and L1 on the right), and the candidate list at the bottom corresponds to the extended AMVP candidate list by adding zero candidates (L0 on the left and L1 on the right).

[0095] Figure 5 Shows the 33 - direction intra - prediction mode used by the HEVC (High Efficiency Video Coding) standard.

[0096] Figure 6 An example of deriving the MPM (Most Probable Mode) list based on five adjacent intra - prediction modes is shown, where the positions of the five adjacent modes include left (L), above (A), bottom - left (BL), top - right - above (AR), and top - left - above (AL).

[0097] Figure 7A An example of ISP (Intra Sub - Partition) is shown, where an HxW block is divided into two H / 2xW blocks (i.e., horizontal partition) or two HxW / 2 blocks (i.e., vertical partition).

[0098] Figure 7B An example of ISP (Intra Sub - Partition) is shown, where an HxW block is divided into four H / 4xW blocks (i.e., horizontal partition) or four HxW / 4 blocks (i.e., vertical partition), except for 4x8, 8x4, and 4x4 blocks.

[0099] Figure 8 A flowchart of an exemplary prediction for video coding according to an embodiment of the present invention is shown, where the horizontal and vertical transforms of each sub - block divided by ISP (Intra Sub - Partition) are determined according to a target setting, and the target setting belongs to a setting group including a unified setting and / or a block setting.

[0100] Figure 9A flowchart of another exemplary prediction for video coding and decoding according to an embodiment of the present invention is shown. When a unified setting includes two or more intra modes or intra-related modes, and when it is not necessary to use / be signaled a transform index to determine a target horizontal transform and a target vertical transform, according to the unified setting, a target horizontal transform and a target vertical transform for a current block are determined from a candidate transform set.

Detailed implementation manners

[0101] The following description is the best conceived mode for implementing the present invention. This description is for explaining the general principles of the present invention and should not be considered restrictive. The scope of the present invention is best determined by referring to the appended claims.

[0102] In the present invention, different methods for improving or simplifying an intra mode including a normal intra mode and ISP, an intra-related mode including CIIP, or a triangular prediction unit mode (TPM) are disclosed.

[0103] The normal intra mode refers to generating an intra prediction of a current block by using one of DC, planar, or directional modes composed of 33 or 65 directions and referring to reconstructed samples in adjacent blocks adjacent or not adjacent to the current block.

[0104] ISP will apply intra prediction on a sub-partition based on the sub-partition. Intra prediction is performed on the sub-partition, and finally, the corresponding reconstructed samples are calculated by adding the residual signal and the prediction signal. In one embodiment, the candidate transform set for the ISP mode is the same as the candidate transform set for the normal intra mode. In another embodiment, the first candidate transform that can be represented by the shortest codeword in the candidate transform set is different from that of the normal intra mode. In another embodiment, for vertical and horizontal transform types, the first candidate transform that can be represented by the shortest codeword in the candidate transform set is set to DST-VII. In another embodiment, when ISP is applied, the intra prediction mode list is the same as the intra prediction mode list of the normal intra mode. In another embodiment, when ISP is applied, according to a predefined rule, the intra prediction mode list is a subset of the intra prediction mode list of the normal intra mode. For example, the predefined rule is to delete the DC or planar mode from the intra prediction mode list of the normal intra mode. In another example, the predefined rule is to reduce the number of intra prediction modes in the intra prediction mode list according to the block width, block height, or block size.

[0105] In another embodiment, the ISP mode can be disabled or restricted according to the block width, block height, or block size. When the ISP is disabled, ISP signaling is not required and the ISP signaling can be skipped. When the ISP is restricted, the combinations of the ISP mode will be reduced, and the encoding / decoding bit sub for signaling the ISP mode can also be reduced. For example, when the block size is greater than or less than a threshold (e.g., 64, 128, 256, 512, 1024, 2048, or the square of the minimum / maximum transform block size specified in the standard), the ISP mode will be disabled or restricted. In another example, when the block width or height is greater than or less than a threshold (e.g., 64, 128, 256, 512, 1024, or the minimum / maximum transform block size specified in the standard), the ISP mode will be disabled. In another example, when the width or height of the block is greater than or less than a threshold (e.g., 64, 128, 256, 512, 1024, or the minimum / maximum transform block size specified in the standard), the ISP mode with vertical or horizontal partitioning will be disabled. In another example, when the long side of the block is greater than the short side of the block multiplied by a threshold (e.g., 64, 128, 256, 512, or 1024), the ISP mode is disabled. In another example, when the long side of the block is greater than the short side of the block multiplied by a threshold (e.g., 64, 128, 256, 512, or 1024), the ISP mode with vertical or horizontal partitioning will be disabled.

[0106] When developing the emerging new coding standard called VVC (Versatile Video Coding), different settings are applied to determine the transform mode for a specific block aspect ratio or some new tools (such as ISP). In the present invention, more efficient rules for improving the coding efficiency are disclosed. In addition, an extended rule is disclosed by extending the existing rules to general cases to simplify the process of selecting the default transform type for horizontal and vertical transforms. The rule can depend on type settings, mode settings, block settings, unified settings, or any combination thereof. The meanings of these settings will be described in detail later. Without this rule, DCT-II is used to assign the default transform type for horizontal and vertical transforms.

[0107] The purpose of this rule is to allow more transform types to increase the diversity when applying the default transform mode. It can help accelerate the codec processing. For example, in some fast algorithms, the RD (Rate-Distortion) cost of applying the default transform mode can be used for early termination of the following mode decision for multiple transform modes. When the cost of using the default transform mode is improved, it is no longer necessary to try some other transform modes. In one embodiment, in addition to DCT-II, the transform types used in this rule have been supported or can be easily derived from the kernels of existing transform types. Therefore, no additional buffer is required to store the coefficients of the transform modes used in this rule.

[0108] Type setting refers to the type of transformation used in this rule. In one embodiment, the default transformation type can be selected from any subset of {DCT-II, DST-VII, DCT-VIII, flipped DST-VII}, and the transformation types of the horizontal transformation and the vertical transformation can be any combination within the selected subset. For example, the combination represented as (hor, ver) can be (DCT-II, DCT-II), (DST-VII, DCT-II), (DCT-II, DST-VII), or (DST-VII, DST-VII). In another embodiment, type setting is used when selecting the target horizontal or target vertical transformation without the need to send / already sent transformation indices.

[0109] Block setting means that this rule determines the transformation type of the default transformation according to the block width, block height, block ratio, or block size of the current block. In one embodiment, if the block width is less than or equal to the block height, the transformation type of the horizontal transformation is changed to another transformation type, such as DST-VII, DCT-VIII, or flipped DST-VII; otherwise, the transformation type of the horizontal transformation is set to DCT-II. In another embodiment, if the block height is less than or equal to the block width, the transformation type of the vertical transformation is changed to another transformation type, such as DST-VII, DCT-VIII, or flipped DST-VII; otherwise, the transformation type of the vertical transformation is set to DCT-II. In another embodiment, block setting is used when selecting the target horizontal or target vertical transformation without the need to send / already sent transformation indices.

[0110] In another embodiment, if the length of the block width is within a predetermined interval, the transformation type of the horizontal transformation is changed to another transformation type, such as DST-VII, DCT-VIII, or flipped DST-VII; otherwise, the transformation type of the horizontal transformation is set to DCT-II. In another embodiment, if the length of the block height is within a predetermined interval, the transformation type of the vertical transformation is changed to another transformation type, such as DST-VII, DCT-VIII, or flipped DST-VII; otherwise, the transformation type of the vertical transformation is set to DCT-II. The above proposed block setting methods can be combined arbitrarily. In another embodiment, block setting can be applied to intra-frame and / or inter-frame blocks. Below, two block setting examples are shown, which respectively show the block width or height within the interval [4, 16]. Note that [4, 16] can be changed to [4, 8] or [4, 32].

[0111] Example 1. When the block width is within the interval [4, 16] and / or the block width is less than or equal to the block height, change the transformation type used for the horizontal transformation to DST-VII.

[0112] Example 2. When the block height is within the interval [4, 16] and / or the block height is less than or equal to the block width, change the transform type of the vertical transform to DST-VII.

[0113] Mode setting means that the rule determines the default transform type according to the selected mode. In one embodiment, the rule may depend on the intra prediction mode. For example, in a predetermined directional mode interval, set {directional mode % 2 == 1} to a combination (representing a transform combination including the transform type of the horizontal transform and the transform type of the vertical transform), and set {directional mode % 2 == 0} to another combination. In another embodiment, the default transform mode for the intra prediction mode (non-angular intra prediction modes such as planar or DC intra prediction modes) can be fixed to a defined combination, such as (DCT-II, DCT-II), (DST-VII, DST-VII), (FDST-VII, FDST-VII), or (DCT-VIII, DCT-VIII). In another embodiment, other settings such as block settings can be used to switch the default transform mode for the intra prediction mode, which is not a directional mode such as planar or DC. An example is shown in Table 7.

[0114] In another example of CIIP, if a horizontal or horizontally related intra prediction mode is selected, when the block width is within [4, 16] and / or the block width is less than or equal to the block height, change the transform type of the horizontal transform to DST-VII; if a vertical or vertically related intra prediction mode is selected, when the block height is within [4, 16] and / or the block height is less than or equal to the block width, change the transform type of the vertical transform to DST-VII; if a non-angular intra prediction mode is selected, the transform type selection will be shown in the following examples.

[0115] Example 1. When the block width is within [4, 16] and / or the block width is less than or equal to the block height, change the transform type of the horizontal transform to DST-VII.

[0116] Example 2. When the block height is within [4, 16] and / or the block height is less than or equal to the block width, change the transform type of the vertical transform to DST-VII.

[0117] Table 7. Examples of default transform types for horizontal and vertical transforms.

[0118]

[0119] A unified setting applies the proposed transform setting to a specific mode or a general case. The general case may include intra blocks and / or intra-related blocks, such as CIIP blocks. In one embodiment, Table 7 currently used by the ISP can also be applied to other intra modes and replaces the existing transform settings of other intra modes. For example, when the current block is intra-coded, the default transform type is determined according to Table 7 instead of the existing transform settings. In another embodiment, any proposed transform setting (e.g., example (1) or (2) for block setting) can be applied to the ISP mode and replaces the existing transform settings of the ISP. For example, when the current block is coded with the ISP, the default transform type is determined according to any proposed transform setting (e.g., example (1) or (2) for block setting) instead of Table 7. In another embodiment, any proposed transform setting (e.g., example (1) or (2) for block setting) can also be applied to CIIP blocks. In another embodiment, two existing transform settings can be combined to establish a new rule for allocating the default transform mode. For example, block settings (e.g., example (1) or (2)) can be used in combination with mode settings (e.g., Table 7 and Table 8). This new rule can be applied to the ISP and / or other intra modes. In another embodiment, Table 7 can be used to determine the transform mode of CIIP blocks. In another embodiment, Table 8 can be used to determine the transform mode of CIIP blocks.

[0120] Table 8. Another example of the default transform types for horizontal and vertical transforms.

[0121]

[0122]

[0123] Any of the above methods or any combination can be determined according to implicit rules or explicit rules. Implicit rules may depend on the width, height, area, aspect ratio of the block size, color component, or picture type. Explicit rules can be indicated by signaling flags at levels such as CU, CTU, slice, tile, tile group, picture, SPS (Sequence Parameter Set), PPS (Picture Parameter Set), etc. Any of the above methods or any combination can be applicable to intra or inter or CIIP modes.

[0124] Any of the previously proposed methods can be implemented in an encoder and / or a decoder. For example, any of the proposed methods can be implemented in the transform module of the encoder and / or the transform module of the decoder. Alternatively, any of the proposed methods can be implemented as a circuit coupled to the transform module of the encoder and / or the transform module of the decoder to provide the information required by the transform module. For example, the proposed method can be implemented in the inter prediction module of the encoder and / or the inter prediction module of the decoder. For example, the proposed method can be implemented in the intra prediction module of the encoder and / or the intra prediction module of the decoder.

[0125] Any of the above combinations can be applied to any tool such as an intra mode, an intra correlation mode, CIIP, or TPM.

[0126] Figure 8 A flowchart of an exemplary prediction for video coding and decoding according to an embodiment of the present invention is shown, in which a target horizontal transform and a target vertical transform are determined for each of a plurality of sub-blocks from a candidate transform set according to a target setting. The steps shown in the flowchart and other subsequent flowcharts in the present disclosure can be implemented as program code executable on one or more processors (e.g., one or more CPUs) on the encoder side and / or the decoder side. The steps shown in the flowchart can also be implemented based on hardware, such as one or more electronic devices or processors arranged to execute the steps in the flowchart. According to this method, in step 810, input data related to a current block in a current picture is received on the video encoder side, or a video bitstream corresponding to compressed data including the current block in the current picture is received on the video decoder side. In step 820, it is determined whether the intra sub-partition (ISP) mode is used for the current block. If the ISP mode is used for the current block (i.e., the "yes" path from step 820), steps 830 to 850 are executed. Otherwise (i.e., the "no" path from step 820), steps 830-850 are skipped. In step 830, the current block is partitioned horizontally or vertically into a plurality of sub-blocks. In step 840, a target horizontal transform and a target vertical transform are determined for each of the plurality of sub-blocks from a candidate transform set according to a set of settings including a unified setting or a block setting, where the unified setting is applicable to a specific coding / decoding mode, a normal intra mode, or other intra-related modes, and the block setting corresponds to the block width, block height, block ratio, block size, or any combination thereof of the current block or sub-block. In step 850, the target horizontal transform and the target vertical transform are applied to each of the plurality of sub-blocks.

[0127] Figure 9A flowchart showing another exemplary prediction for video coding and decoding according to an embodiment of the present invention is presented. When unified settings include two or more intra modes or intra-related modes, and when there is no need to use the signaled / already-signaled transform index to select the target horizontal transform and the target vertical transform, according to the unified settings, the target horizontal transform and the target vertical transform for the current block are determined from a candidate transform set. According to this method, in step 910, input data related to the current block in the current picture is received at the video encoder side, or a video bitstream corresponding to the compressed data including the current block in the current picture is received at the video decoder side. In step 920, when there is no need to use the signaled / already-signaled transform index to select the target horizontal transform and the target vertical transform, according to the unified settings, the target horizontal transform and the target vertical transform for the current block are determined from a candidate transform set, where the unified settings include two or more intra modes or intra-related modes. In step 930, the target horizontal transform and the target vertical transform are applied to the current block.

[0128] The flowchart shown is intended to illustrate an example of video coding and decoding according to the present invention. Those skilled in the art can modify each step, rearrange steps, split steps, or combine steps to implement the present invention without departing from the spirit of the present invention. In this disclosure, specific syntax and semantics have been used to show examples for implementing embodiments of the present invention. Those skilled in the art can practice the present invention by replacing the syntax and semantics with equivalent syntax and semantics without departing from the spirit of the present invention.

[0129] The above description is presented to enable those skilled in the art to practice the present invention provided in the context of a particular application and its requirements. Various modifications to the described embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments. Therefore, the present invention is not intended to be limited to the particular embodiments shown and described, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein. In the above detailed description, various specific details are shown to provide a thorough understanding of the present invention. However, those skilled in the art will understand that the present invention can be practiced.

[0130] Embodiments of the present invention as described above can be implemented in various hardware, software codes, or a combination of both. For example, embodiments of the present invention can be one or more circuits integrated into a video compression chip or program codes integrated into video compression software to perform the processing described herein. Embodiments of the present invention can also be program codes that will be executed on a digital signal processor (DSP) to perform the processing described herein. The present invention also relates to many functions performed by a computer processor, a digital signal processor, a microprocessor, or a field programmable gate array (FPGA). By executing machine-readable software codes or firmware codes that define the specific methods embodied by the present invention, these processors can be configured to perform specific tasks according to the present invention. Software codes for decoding or firmware codes can be developed in different programming languages and different formats or styles. Software codes can also be compiled for different target platforms. However, different codec formats, styles and languages of software codes, and other means of configuring codes to perform tasks according to the present invention will not depart from the spirit and scope of the present invention.

[0131] Without departing from the spirit or essential characteristics of the present invention, the present invention can be embodied in other specific forms. The described examples are to be considered illustrative rather than restrictive in all respects. Therefore, the scope of the present invention is indicated by the appended claims rather than the foregoing description. All changes that fall within the equivalent meaning and scope of the claims shall be included within their scope.

Claims

1. A video coding and decoding prediction method, the method comprising: Receiving input data related to a current block in a current picture at a video encoder side, or receiving a video bitstream corresponding to compressed data including the current block in the current picture at a video decoder side; If the current block uses an intra sub-partition mode: Horizontally or vertically partitioning the current block into a plurality of sub-blocks; Determining a target horizontal transform and a target vertical transform for each of the plurality of sub-blocks from a candidate transform set according to a set group including a unified setting or a block setting, wherein the unified setting is applicable to a specific coding and decoding mode, a normal intra mode or other intra-related modes, and the block setting corresponds to a block width, a block height, a block ratio, a block size or any combination thereof of the current block or the sub-block; and Applying the target horizontal transform and the target vertical transform to each of the plurality of sub-blocks, wherein the intra sub-partition mode is disabled according to at least one threshold, and the disabling of the intra sub-partition mode is related to the block width, the block height or the block size.

2. The video codec prediction method according to claim 1, wherein, The target horizontal transform and the target vertical transform belong to a subset of any combination including DCT-II, DST-VII, DCT-VIII and flipped DST-VII.

3. The video codec prediction method according to claim 2, wherein, A pair representing (the target horizontal transform, the target vertical transform) corresponds to (DCT-II, DCT-II), (DST-VII, DCT-II), (DCT-II, DST-VII) or (DST-VII, DST-VII).

4. The video codec prediction method according to claim 1, wherein, The target horizontal transform and the target vertical transform are determined based on the block width, the block height, the block ratio, the block size or any combination thereof of the current block or the sub-block.

5. The video codec prediction method according to claim 4, wherein, When the block width is less than or equal to the block height, changing the target horizontal transform from DCT-II to another transform type.

6. The video codec prediction method according to claim 4, wherein, When the block width is within [4, 16], or the block width is less than or equal to the block height, or the block width is within [4, 16] and the block width is less than or equal to the block height, changing the target horizontal transform from DCT-II to DST-VII.

7. The video codec prediction method according to claim 4, wherein, When the block height is within [4, 16], or the block height is less than or equal to the block width, or the block height is within [4, 16] and the block height is less than or equal to the block width, changing the target vertical transform from DCT-II to DST-VII.

8. The video codec prediction method according to claim 1, wherein, The candidate transform set is the same as the transform set used for the normal intra mode.

9. The video coding and decoding prediction method according to claim 8, wherein for both the target horizontal transform and the target vertical transform, setting the shortest codeword for the candidate transform in the candidate transform set to DST-VII.

10. The video codec prediction method according to claim 1, wherein, The intra prediction mode list used by the intra sub-partition mode is the same as the intra prediction mode list used by the normal intra mode.

11. The video codec prediction method according to claim 10, wherein, The intra prediction mode list used by the intra sub-partition mode is a subset of the intra prediction mode list used by the normal intra mode.

12. The video codec prediction method according to claim 1, wherein, The unified setting or the block setting is implicitly determined.

13. The video codec prediction method according to claim 1, wherein, Signaling a decision for the unified setting or the block setting at the level of a codec unit, a codec tree unit, a slice, a tile, a tile group, a sequence parameter set, or a picture parameter set.

14. The video codec prediction method according to claim 1, wherein, The set of settings further includes a mode setting, and the mode setting is based on the intra prediction mode selected for the current block.

15. The video coding and decoding prediction method according to claim 14, wherein, The selected intra prediction mode is a directional mode and satisfies (directional mode % 2 == 1).

16. The video codec prediction method according to claim 14, wherein, The selected intra prediction mode is a directional mode and satisfies (directional mode % 2 == 0).

17. A video codec device, the device comprising one or more electronic circuits or processors for: Receiving input data related to a current block in a current picture at a video encoder side, or receiving a video bitstream corresponding to compressed data including the current block in the current picture at a video decoder side; If the current block uses an intra sub-partition mode: Horizontally or vertically partitioning the current block into a plurality of sub-blocks; Determine a target horizontal transformation and a target vertical transformation for each of the plurality of sub-blocks from a set of candidate transformations according to a set of settings including unified settings or block settings, wherein, The unified setting applies to a specific codec mode, a normal intra mode, or other intra-related modes, and the block setting corresponds to the block width, block height, block ratio, block size, or any combination thereof of the current block or sub-block; and Applying the target horizontal transform and the target vertical transform to each of the plurality of sub-blocks, wherein the intra sub-partition mode is disabled according to at least one threshold, and the disabling of the intra sub-partition mode is related to the block width, the block height, or the block size.

18. A prediction method for video coding and decoding, the method comprising: Receiving input data related to a current block in a current picture at a video encoder side, or receiving a video bitstream corresponding to compressed data including the current block in the current picture at a video decoder side; When a transform index that needs to be signaled / has been signaled is not required to select a target horizontal transform and a target vertical transform, determining, according to a unified setting, the target horizontal transform and the target vertical transform for the current block from a candidate transform set, wherein the unified setting includes two or more intra modes or intra-related modes; and Applying the target horizontal transform and the target vertical transform to the current block, wherein the intra sub-partition mode is disabled according to at least one threshold, and the disabling of the intra sub-partition mode is related to the block width, the block height, or the block size.

19. A video codec device, the device comprising one or more electronic circuits or processors, the electronic circuits or processors being arranged to: Receiving input data related to a current block in a current picture at a video encoder side, or receiving a video bitstream corresponding to compressed data including the current block in the current picture at a video decoder side; When a transform index that needs to be signaled / has been signaled is not required to select a target horizontal transform and a target vertical transform, determining, according to a unified setting, the target horizontal transform and the target vertical transform for the current block from a candidate transform set, wherein the unified setting includes two or more intra modes or intra-related modes; and Applying the target horizontal transform and the target vertical transform to the current block, wherein the intra sub-partition mode is disabled according to at least one threshold, and the disabling of the intra sub-partition mode is related to the block width, the block height, or the block size.

Images