Method and apparatus for generic transformation of intra block copy mode or intra template matching mode for video coding

Abstract

The invention provides a video encoding and decoding method and a related device. The video encoding and decoding method comprises: receiving input data related to a current block in a current picture, the input data comprising residual data of the current block for encoding at an encoder end or transform coefficients of the current block for decoding at a decoder end, the prediction data of the current block is generated by applying intra block copying or intra template matching prediction; applying a target transform pattern to the current block to derive a final transform coefficient at the encoder side or to derive reconstructed residual data at the decoder side, where the target transform pattern includes a sub-block transform or a partial transform, and wherein the sub-block transform divides the current block into a plurality of sub-blocks and applies transform processing to one or more sub-blocks that are target portions of the current block, or the partial transform applies transform processing only to the target portions of the current block; and providing the final transform coefficient at the encoder side or the reconstructed residual data at the decoder side. According to the video coding and decoding method and the related device, the coding and decoding efficiency can be improved.

Description

[0001] RELATED APPLICATIONS This application is a non-provisional of and claims priority to U.S. Provisional Patent Application No. 63 / 495,583, filed April 12, 2023. The entire contents of the U.S. Provisional Patent Application are incorporated herein by reference. TECHNICAL FIELD

[0002] The present invention relates to video coding systems using Intra Block Copy Mode or Intra Template Matching (IntraTMP for short) mode. In particular, the present invention relates to supporting more transform modes and / or down- / up-sampling for Intra Block Copy Mode or IntraTMP mode. BACKGROUND

[0003] Versatile Video Coding (VVC for short) is the latest international video coding standard jointly developed by the Joint Video Expert Team (JVET) of ITU-T Video Coding Experts Group (VCEG) and ISO / IEC Moving Picture Experts Group (MPEG). The standard has been published as an ISO standard: ISO / IEC 23090-3:2021, Information technology - Coded representation of immersive media - Part 3: Versatile Video Coding, published in February 2021. VVC is based on its predecessor HEVC (High Efficiency Video Coding) by adding more coding tools to improve coding efficiency. It can also handle various types of video sources, including 3-dimensional (3D) video signals.

[0004] Figure 1AAn exemplary adaptive Inter / Intra video coding system is illustrated that includes loop processing. For Intra prediction, prediction data is derived from previously coded video data in the current picture. For Inter prediction 112, Motion Estimation (ME) is performed at the encoder side and Motion Compensation (MC) is performed based on the results of ME to provide prediction data derived from other pictures and motion data. A switch 114 selects either Intra prediction 110 or Inter prediction 112 and the selected prediction data is provided to a summer 116 to form prediction error, also referred to as residual. The prediction error is then processed by a transform (T) 118 and subsequent quantization (Q) 120. The transformed and quantized residual is then encoded by an entropy encoder 122 to be included in a video bitstream corresponding to compressed video data. The bitstream associated with the transform coefficients is then packed with side information, such as motion and coding modes associated with Intra and Inter prediction, as well as other information, such as parameters associated with loop filters applied to underlying image areas. Side information associated with Intra prediction 110, Inter prediction 112, and in-loop filter 130 is provided to entropy encoder 122, as shown. Figure 1A When Inter prediction mode is used, one or more reference pictures must also be reconstructed at the encoder end. Thus, the transformed and quantized residual is processed by dequantization (IQ) 124 and inverse transform (IT) 126 to recover the residual. The residual is then added back to prediction data 136 at reconstruction (REC) 128 to reconstruct the video data. The reconstructed video data can be stored in a reference picture buffer 134 and used for predicting other frames.

[0005] As Figure 1AAs shown, the input video data undergoes a series of processing steps in the encoding system. Due to these processing steps, the reconstructed video data from REC 128 may suffer various forms of degradation. Therefore, loop filters 130 are frequently applied to the reconstructed video data before it is stored in the reference picture buffer 134 to improve video quality. For example, deblocking filters (DF), Sample Adaptive Offset (SAO), and Adaptive Loop Filters (ALF) can be used. It may be necessary to incorporate loop filter information into the bitstream so that the decoder can correctly recover the required information. Therefore, loop filter information is also provided to entropy encoder 122 for incorporation into the bitstream. Figure 1A In the process, the loop filter 130 is applied to the reconstructed video before the reconstructed samples are stored in the reference image buffer 134. Figure 1A The system described herein is intended to illustrate an exemplary architecture of a typical video encoder. It may correspond to a High Efficiency Video Codec (HEVC) system, VP8, VP9, ​​H.264, or VVC.

[0006] like Figure 1B As shown, the decoder can use similar or identical function blocks to the encoder, except for transform 118 and quantization 120, because the decoder only needs to dequantize 124 and inverse transform 126. Instead of entropy encoder 122, the decoder uses entropy decoder 140 to decode the video bitstream into quantized transform coefficients and the required encoding / decoding information (e.g., ILPF information, intra-frame prediction information, and inter-frame prediction information). Intra-frame prediction 150 on the decoder side does not require mode search. Instead, the decoder only needs to generate intra-frame predictions based on the intra-frame prediction information received from entropy decoder 140. Furthermore, for inter-frame prediction, the decoder only needs to perform motion compensation (MC 152) 140 based on the inter-frame prediction information received from entropy decoder without motion estimation.

[0007] This invention discloses methods and apparatus for improving the IBC / IntraTMP mode. For example, a transform coding method for the IBC / IntraTMP mode is proposed. For example, a downsampling / upsampling method for the IBC / IntraTMP mode is proposed. Summary of the Invention

[0008] This invention provides a video encoding / decoding method and related apparatus.

[0009] A method of video coding includes receiving input data related to a current block in a current picture, wherein the input data includes residual data for the current block for encoding at an encoder side or transform coefficients for the current block for decoding at a decoder side, and prediction data for the current block is generated by applying intra block copy or intra template matching prediction; applying a target transform mode to the current block to derive final transform coefficients at the encoder side or reconstructed residual data at the decoder side, wherein the target transform mode includes sub-block transform or partial transform, and wherein the sub-block transform partitions the current block into a plurality of sub-blocks and applies transform processing to one or more sub-blocks as a target portion of the current block, or the partial transform applies transform processing only to a target portion of the current block; and providing the final transform coefficients at the encoder side or the reconstructed residual data at the decoder side.

[0010] The present disclosure also provides an apparatus for video coding, the apparatus comprising one or more electronic components or processors configured to: receive input data related to a current block in a current picture, wherein the input data includes residual data for the current block for encoding at an encoder side or transform coefficients for the current block for decoding at a decoder side, and prediction data for the current block is generated by applying intra block copy or intra template matching prediction; apply a target transform mode to the current block to derive final transform coefficients at the encoder side or reconstructed residual data at the decoder side, wherein the target transform mode includes sub-block transform or partial transform, and wherein the sub-block transform partitions the current block into a plurality of sub-blocks and applies transform processing to one or more sub-blocks as a target portion of the current block, or the partial transform applies transform processing only to a target portion of the current block; and provide the final transform coefficients at the encoder side or the reconstructed residual data at the decoder side.

[0011] These and other objects of the present application will no doubt become obvious to one of ordinary skill in the art after reading the following detailed description of the preferred embodiment, which is illustrated in various figures and diagrams. BRIEF DESCRIPTION OF DRAWINGS

[0012] Figure 1A An exemplary adaptive Inter / Intra video coding system incorporating loop processing is illustrated.

[0013] Figure 1B A corresponding decoder of the encoder in Figure 1A is shown.

[0014] Figure 2 An example of IBC compensation is illustrated.

[0015] Figure 3A search region for intra template matching is shown.

[0016] Figure 4 An example of using IntraTMP block vectors for IBC blocks is shown.

[0017] Figure 5 An example of low frequency non-separable transform (LFNST) processing is shown.

[0018] Figure 6 A transform core selection is shown that depends on the location of SBT.

[0019] Figure 7A An example of BV adjustment for horizontal flipping is shown.

[0020] Figure 7B An example of BV adjustment for vertical flipping is shown.

[0021] Figure 8 An example of splitting a current deblocking into 4 transform blocks by quad-tree partitioning is shown, where the target portion is indicated as a region in the current deblocking (indicated as a shaded region).

[0022] Figure 9 An example of down-sampling IBC mode is shown.

[0023] Figure 10 A flowchart of an exemplary video coding system using multiple transforms for IBC / IntraTMP modes according to embodiments of the present invention is shown. DETAILED DESCRIPTION

[0024] It will be readily understood that the components of the present invention, as generally described and illustrated in the Figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following more detailed description of the embodiments of the systems and methods of the present invention, as represented in the Figures, is not intended to limit the scope of the application, but is merely representative of selected embodiments of the application. Reference throughout this specification to "one embodiment," "an embodiment," or similar language means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Accordingly, appearances of the phrase "in one embodiment" or "in an embodiment" throughout this specification are not necessarily all referring to the same embodiment.

[0025] Moreover, the described features, structures, or characteristics can be combined in one or more embodiments. However, the skilled artisan will appreciate that the application can be practiced without one or more of the described features, structures, or characteristics, or with other methods, components, etc. In other instances, well-known structures or operations have not been shown or described in detail in order to avoid obscuring aspects of the application. Reference will now be made to the drawings to describe the illustrated embodiments of the application in detail. Like reference numerals can be used to refer to like parts throughout the description. The following description is merely exemplary in nature and is not intended to limit the equipment and methods consistent with the present application as claimed.

[0026] Intra block copy (IBC) mode Motion compensation is one of the key techniques in hybrid video coding, which explores the correlation of pixels between neighboring pictures. It is generally assumed that a pattern corresponding to an object or a background in a video sequence is displaced in one frame to form a corresponding object in the subsequent frame, or is related to other patterns within the current frame. By estimating such displacement (e.g., using block matching techniques), the pattern can be approximately reproduced without the need to re-encode it. Similarly, block matching and copying are also attempted to allow a reference block to be selected from the same picture as the current block. When this concept is applied to a film captured by a camera, one finds that the efficiency is very low. Part of the reason is that a text pattern in a spatially neighboring region can be similar to the current encoding block, but there are usually some gradual changes in space. It is difficult to find an exact match for a block in the same picture in a film captured by a camera. Therefore, the improvement of coding performance is limited.

[0027] However, for screen content, the situation of spatial correlation between pixels within the same picture is different. For a typical video with text and graphics, there are usually repeated patterns within the same picture. Therefore, it is observed that intra (picture) block compensation is very effective. A new prediction mode, i.e., intra block copy (IBC) mode or called current picture referencing (CPR) has been introduced for screen content coding to exploit this property. In the CPR mode, a prediction unit (PU) is predicted from a previously reconstructed block in the same picture. In addition, a displacement vector (called block vector or BV) is used to indicate the relative displacement from the position of the current block to the position of the reference block. The prediction error is then coded using transform, quantization, and entropy coding. An example of IBC compensation is shown in FIG. 1. In this example, the current block 100 is predicted from a previously reconstructed block 102 in the same picture. The displacement vector 104 is used to indicate the relative displacement from the position of the current block 100 to the position of the reference block 102. The prediction error 106 is then coded using transform, quantization, and entropy coding. Figure 2As shown, where block 212 is the corresponding block of block 210, and block 222 is the corresponding block of block 220. In this art, a reference sample corresponds to a reconstructed sample of the current decoded picture prior to the in-loop filter operation, which includes the deblocking effect and the sample adaptive offset (SAO) filter in HEVC.

[0028] The first version of IBC was developed as a range extension (RExt) to HEVC in JCTVC-M0350 (Budagavi et al., AHG8: Video coding using Intra motion compensation, ITU-T SG16 WP3 and ISO / IEC JTC 1 / SC 29 / WG11 Joint Collaborative Team on Video Coding (JCT-VC) 13th Meeting: Jeju Island, KR, 18-26 April 2013, document: JCTVC-M0350). In this version, the IBC compensation is limited to a small local region, only 1-D block vectors, and only for 2Nx2N block size. Later, a more advanced IBC design was developed in the standardization process of HEVC Screen Content Coding (SCC).

[0029] Intra Template Matching Prediction Intra Template Matching Prediction (IntraTMP) is a special intra prediction mode that copies the best prediction block from the reconstructed part of the current frame, whose L-shaped template matches the current template. For a predetermined search range, the encoder searches for the template match that is most similar to the current template in the reconstructed part of the current frame, and uses the corresponding block as the prediction block. The encoder then signals the use of this mode and performs the same prediction operation at the decoder side.

[0030] The prediction signal is generated by matching the L-shaped causal neighbour of the current block with Figure 3 another block in a predetermined search region, which includes: R1 : all or any subset of the current CTU; R2: the top-left CTU; R3: the above CTU; R4: the left CTU; InFigure 3 In this case, the current block 310 in R1 matches the corresponding block 312 in R2. The templates of the current block and the matching block are shown as dark L-shaped regions. Region 322 corresponds to the reconstructed region in the current picture 320. The sum of absolute differences (SAD) is used as the cost function. Within each region, the decoder searches for the template with the minimum SAD to the current template and uses its corresponding block as the predicted block.

[0031] The size of all regions (SearchRange_w, SearchRange_h) is set to be proportional to the block size (BlkW, BlkH) with a fixed number of SAD comparisons per pixel. That is: SearchRange_w = a * BlkW, SearchRange_h = a * BlkH, where 'a' is a constant that controls the gain / complexity trade-off. In practice, 'a' is equal to 5.

[0032] To speed up the template matching process, the search range of all search regions is subsampled by a factor of 2. This results in a 4x reduction in the template matching search. After the best match is found, a refinement process is performed. Refinement is done by performing a second template matching search around the reduced range of the best match. The reduced range is defined as min(BlkW, BlkH) / 2.

[0033] The intra template matching tool is enabled for CUs with width and height less than or equal to 64. The maximum CU size for intra template matching is configurable.

[0034] When decoder-side intra mode derivation (DIMD) is not used for the current CU, the intra template matching prediction mode is signaled at the CU level by a dedicated flag.

[0035] Intra TMP derived IBC block vector candidates In this approach, the block vectors (BV) derived from intra template matching prediction (IntraTMP) are used for intra block copy (IBC). The stored IntraTMP BVs of neighboring blocks as well as IBC BVs are used as spatial BV candidates in the IBC candidate list construction.

[0036] Intra-TMP block vectors are stored in the IBC block vector buffer, and the current IBC block can use the IBC BVs and Intra-TMP BVs of neighboring blocks as BV candidates of the IBC BV candidate list, as shown in Figure 4

[0037] In Figure 4 , block 410 corresponds to the current block, and block 412 corresponds to a neighboring Intra-TMP block. The Intra-TMP BV 416 is used to locate the best matching block 422 according to the matching cost between the template 424 and the template 414. The region 432 corresponds to the reconstructed region in the current picture 430. The Intra-TMP block vector is added to the IBC block vector candidate list as a spatial candidate.

[0038] The Intra-TMP block vector is added to the IBC block vector candidate list as a spatial candidate.

[0039] Multiple Transform Selection (MTS) of core transform.

[0040] In addition to DCT-II already used in HEVC, the MTS scheme is used for residual coding of intra and inter coded blocks. It uses multiple transforms selected from DCT8 / DST7. The newly introduced transform matrices are DST-VII and DCT-VIII. Table 1 shows the basis functions of the selected DST / DCT.

[0041]

[0042] To maintain the orthogonality of the transform matrices, the transform matrices are quantized more accurately than in HEVC. To keep the mid value of the transform coefficients in the 16-bit range, all coefficients have 10-bit precision after the horizontal and vertical transform.

[0043] To control the MTS scheme, separate enabling flags are specified at the SPS (Sequence Parameter Set) level for intra and inter, respectively. When MTS is enabled at the SPS, a CU level flag is signaled to indicate whether MTS is applied or not. Here, MTS is only applied to luma. The MTS signal will be skipped when one of the following conditions is applied: The position of the last significant coefficient of the luma TB is smaller than 1 (i.e., only DC), The last significant coefficient of the luma TB is located in the MTS zero-out region.

[0044] ​If the MTS CU flag is equal to zero, DCT2 is applied in both directions. However, if the MTS CU flag is equal to one, two additional flags are additionally signaled to indicate the transform type in horizontal and vertical directions, respectively. The transform and signal mapping table is shown in Table 2.

[0045]

[0046] To reduce the complexity of large size DST-7 and DCT-8, for DST-7 and DCT-8 blocks with size (width or height, or both width and height) equal to 32, the high frequency transform coefficients are zeroed. Only the coefficients within the 16x16 low frequency region are kept.

[0047] As in HEVC, the transform skip mode can be utilized to code the residual of a block. To avoid the redundancy of syntax coding, the transform skip flag is not signaled when the CU level MTS CU flag is not equal to 0. Note that the implicit MTS transform is set to DCT2 when LFNST. In addition, implicit MTS can still be enabled when MTS is enabled for inter coded blocks.

[0048] Low-frequency non-separable transform (LFNST) In VVC, as shown in Figure 5 LFNST is applied between the forward primary transform and quantization (at the encoder side) and between dequantization and inverse primary transform (at the decoder side). As shown in Figure 5As shown, after the forward primary transform 510, a forward low-frequency non-separable transform (Low-Frequency Non-Separable Transform) LFNST 520 is applied to the top-left region 522 of the output of the forward primary transform, e.g., 16 coefficients for a 4x4 forward LFNST and / or 64 coefficients for an 8x8 forward LFNST. In the LFNST, either a 4x4 non-separable transform or an 8x8 non-separable transform is applied depending on the block size. For example, a 4x4 LFNST is applied for small blocks (i.e., min (width, height)<8) and an 8x8 LFNST is applied for larger blocks (i.e., min (width, height)>4). After the LFNST, the transform coefficients are quantized by quantization 530. To reconstruct the input signal, the quantized transform coefficients are dequantized using dequantization 540 to obtain dequantized transform coefficients. An inverse LFNST 550 is applied to the top-left region 552 (8 coefficients for a 4x4 inverse LFNST or 16 coefficients for an 8x8 inverse LFNST). After the inverse LFNST, an inverse primary transform 560 is applied to recover the input signal.

[0049] The application of the non-separable transform used in the LFNST is described below using an input as an example. To apply a 4x4 LFNST, a 4x4 input block X , is first represented as a vector : , The non-separable transform is computed as where is the transform coefficient vector and T is a 16x16 transform matrix. The 16x1 coefficient vector is then reorganized into a 4x4 block using the scan order (horizontal, vertical, or diagonal) of the block. The coefficients with smaller indices will be placed in the 4x4 coefficient block with smaller scan indices.

[0050] Simplified non-separable transform ​LFNST (low frequency non-separable transform) applies non-separable transform based on direct matrix multiplication method, which is implemented in a single pass without multiple iterations. However, the dimension of the non-separable transform matrix needs to be reduced to minimize the computational complexity and the storage space for storing the transform coefficients. Therefore, a reduced non-separable transform (RST) method is used in LFNST. The main idea of the reduced non-separable transform is to map an N (N is usually equal to 64 for 8x8 NSST) dimensional vector to an R dimensional vector in a different space, where N / R (R<N) is the reduction factor. Therefore, the RST matrix is no longer an NxN matrix, but becomes an RxN matrix, as follows:

[0051] where the R rows of the transform are the R basis of the N dimensional space.

[0052] The inverse transform matrix of RST is the transpose of its forward transform. For 8x8 LFNST, a reduction factor of 4 is applied and the 64x64 direct matrix (traditional 8x8 non-separable transform matrix size) is reduced to a 16x48 direct matrix. Therefore, the 48x16 inverse RST matrix is used at the decoder side to produce the core (primary) transform coefficients in the 8x8 top-left region. When a 16x48 matrix is applied instead of a 16x64 with the same transform set configuration, each matrix takes 48 input data from three 4x4 blocks in the top-left 8x8 block (excluding the bottom-right 4x4 block). With the help of dimension reduction, the memory usage to store all LFNST matrices is reduced from 10KB to 8KB with reasonable performance degradation. To reduce complexity, LFNST is restricted to be applied only when all coefficients except the first coefficient subgroup are non-significant. Therefore, when LFNST is applied, all primary-only transform coefficients must be zero. This allows the LFNST index signal to be adjusted on the last significant position, thus avoiding the extra coefficient scanning in the current LFNST design, which only needs to check the significant coefficients at a specific position. The worst-case processing of LFNST (in terms of multiplication per pixel) limits the non-separable transform of 4x4 and 8x8 blocks to 8x16 and 8x48 transforms, respectively. In these cases, when LFNST is applied, the last significant scan position must be less than 8 for other sizes smaller than 16. For blocks with shape 4xN and Nx4 and N>8, the suggested restriction means that LFNST is now applied only once and only to the 4x4 region in the top-left corner. Since all primary-only coefficients are zero when LFNST is applied, the number of operations required for the primary transform is reduced in this case. From the encoder's perspective, the quantization of coefficients is significantly simplified when testing the LFNST transform. The first 16 coefficients (in scan order) must be maximum rate-distortion optimized quantized, and the rest of the coefficients are forced to zero.

[0053] LFNST transform selection There are 4 transform sets in total in LFNST, each of which uses 2 non-separable transform matrices (kernels). The mapping from the intra prediction mode to the transform set is pre-defined as shown in Table 3. If one of the three CCLM modes (INTRA LT CCLM, INTRA T CCLM or INTRA L CCLM) is used for the current block (81 <= predModeIntra <= 83), transform set 0 is selected for the current chroma block. For each transform set, the selected non-separable secondary transform candidate is further specified by an explicitly signaled LFNST index. After the transform coefficients, the index is signaled once in the bitstream for each intra CU.

[0054]

[0055] LFNST index signaling and interaction with other tools Since LFNST is restricted to be applied only when all coefficients except the first coefficient sub-group are non-significant, LFNST index coding depends on the position of the last significant coefficient. Furthermore, LFNST index is context coded but does not depend on the intra prediction mode and only the first bin is context coded. Moreover, LFNST is applied to intra CUs of intra slices and inter slices, and can be used for both luma and chroma. If dual tree is enabled, the LFNST index is signaled separately for luma and chroma. For inter slices (dual tree is disabled), a single LFNST index is signaled and used for luma and chroma.

[0056] Considering that large CUs larger than 64x64 are implicitly partitioned (TU tiling) due to the existing maximum transform size limit (64x64), for a certain number of decode pipeline stages, LFNST index search can increase data buffering by a factor of four. Therefore, the maximum size allowed for LFNST is limited to 64x64. Note that LFNST is only enabled in DCT2. LFNST index signaling is placed before MTS index signaling.

[0057] The use of scaling matrices for perceptual quantization does not obviously indicate that the scaling matrices specified for the primary matrices can be useful for LFNST coefficients. Therefore, the use of scaling matrices for LFNST coefficients is not allowed. For single tree partition modes, chroma LFNST is not applied.

[0058] Subblock Transform (SBT) In the VVC Test Model (VTM) software, subblock transform is introduced for inter predicted CUs. In this transform mode, for a CU, only subparts of the residual block are coded. When the cu_cbf of an inter predicted CU is equal to 1, cu_sbt_flag can be signaled to indicate whether the whole residual block or subparts of the residual block are coded. In the former case, the inter MTS information is further parsed to determine the transform type of the CU. In the latter case, one part of the residual block is coded with an inferred adaptive transform, while the other part of the residual block is zeroed out.

[0059] When SBT is used for inter coded CUs, the SBT type and SBT position information are signaled in the bitstream. There are two SBT types and two SBT positions. For SBT-V (or SBT-H), the TU width (or height) can be equal to half or 1 / 4 of the CU width (or height), resulting in 2:2 split or 1:3 / 3:1 split. The 2:2 split is similar to binary tree (BT) split, while the 1:3 / 3:1 split is similar to asymmetric binary tree (ABT) split. In ABT split, only the small region contains non-zero residual. If the CU has one dimension of 8 in luma samples, 1:3 / 3:1 split along that dimension is not allowed. A CU has at most 8 SBT modes.

[0060] Position-dependent transform core selection is applied to luma transform blocks in SBT-V and SBT-H (chroma TBs always use DCT-2). The two positions of SBT-H (630 and 640) and SBT-V (610 and 620) are associated with different core transforms as shown in Figure 6 More specifically, the horizontal and vertical transforms for each SBT position are specified in Figure 6 For example, the horizontal and vertical transforms for SBT-V position 0 are DCT-8 and DST-7, respectively. When one side of a residual TU is larger than 32, the transform in both dimensions (or directions) is set to DCT-2. Thus, subblock transform jointly specifies the TU tiling, cbf, and horizontal and vertical core transform types for a residual block.

[0061] SBT is not applied to CUs coded with combined inter-intra mode.

[0062] Block differential pulse coded modulation (BDPCM) VVC supports block differential pulse coded modulation (BDPCM) for screen content coding. At sequence level, a BDPCM enabling flag is signaled in SPS; the flag is only signaled when transform skip mode is enabled in SPS (as occurs in several sections or embodiments).

[0063] When BDPCM is enabled, if the CU size is smaller than or equal to MaxTsSize by MaxTsSize in luma samples, and if the CU is intra coded, a flag is transmitted at CU level, where MaxTsSize is the maximum block size that allows transform skip mode. This flag indicates whether regular intra coding or BDPCM is used. If BDPCM is used, a BDPCM prediction direction flag is sent to indicate whether the prediction is horizontal or vertical. Then, regular horizontal or vertical intra prediction process is used to predict the block with unfiltered reference samples. The residual is quantized and the difference between each quantized residual and its prediction value (i.e., the previously coded residual at the horizontally or vertically (depending on the BDPCM prediction direction) neighboring position) is coded.

[0064] For a block of size M (height) x N (width), let be the prediction residual. Let denote the quantized version of the residual . BDPCM is applied to the quantized residual values, resulting in a modified M x N array with elements , where is predicted from its neighboring quantized residual values. For vertical BDPCM prediction mode, for 0≤j≤N-1, is derived using the following equation: , For horizontal BDPCM prediction mode, for , is derived using the following equation: .

[0065] At the decoder side, the above process is reversed to compute , , as follows: , if vertical BDPCM is used, ​​ If horizontal BDPCM is used, The dequantized residual is added to the intra block prediction value to produce the reconstructed sample value.

[0066] The predicted quantized residual value is transmitted to the decoder using the same residual coding process as in transform skip mode residual coding. For lossless coding, if slice_ts_residual_coding_disabled_flag is set to 1, the quantized residual value is transmitted to the decoder using regular transform residual coding. As for MPM modes used for future intra mode coding, if the BDPCM prediction direction is horizontal or vertical, the CU coded with BDPCM stores the horizontal or vertical prediction mode, respectively. For deblocking, if both blocks on either side of a block boundary are coded with BDPCM, that particular block boundary is not deblocked.

[0067] Reconstruction-Reordered IBC (RR-IBC) IBC coded blocks allow the use of the Reconstruction-Reordered IBC (RR-IBC) mode. When RR-IBC is applied, the samples in the reconstructed block are flipped according to the flip type of the current block. At the encoder side, the original block is flipped before motion search and residual calculation, while the prediction block is derived without flipping. At the decoder side, the reconstructed block is flipped back to recover the original block.

[0068] RR-IBC coded blocks support two flipping methods: horizontal flipping and vertical flipping. First, a syntax flag is signaled for IBC AMVP coded blocks to indicate whether the reconstruction is flipped, and if flipped, another flag is further signaled to specify the flip type. For IBC merge, the flip type is inherited from the neighboring block without syntax signaling. Given the horizontal or vertical symmetry, the current block and the reference block are usually horizontally or vertically aligned. Therefore, when horizontal flipping is applied, the vertical component of the BV is not signaled and inferred to be equal to 0. Similarly, when vertical flipping is applied, the horizontal component of the BV is not signaled and inferred to be equal to 0.

[0069] To better exploit the symmetry property, a flip-aware BV refinement method is applied to refine the block vector candidates. For example, as Figure 7A and Figure 7BAs shown, (xnbr, ynbr) and (xcur, ycur) represent the coordinates of the center samples of the neighboring block and the current block, respectively, and BVnbr and BVcur represent the BVs of the neighboring block and the current block, respectively. The horizontal component of BVcur is not directly inherited from the neighboring block BV, but is calculated by adding a motion offset on the horizontal component of BVnbr (denoted as BVnbrh) to prevent the neighboring block from being coded using horizontal flipping (as shown in Figure 7A ). That is, BVcurh = 2(xnbr - xcur) + BVnbrh. Similarly, in the case that the neighboring block is coded using vertical flipping (as shown in Figure 7B ), the vertical component of BVcur is calculated by adding a motion offset on the vertical component of BVnbr (denoted as BVnbrv), that is, BVcurv = 2(ynbr - ycur) + BVnbrv.

[0070] IBC merge mode with block vector differences (IBC-MBVD) Affine-MMVD and GPM-MMVD have been adopted by ECM as extensions of regular MMVD mode. It is natural to extend MMVD mode to IBC merge mode.

[0071] In IBC-MBVD, the distance set is {1-pixel, 2-pixel, 4-pixel, 8-pixel, 12-pixel, 16-pixel, 24-pixel, 32-pixel, 40-pixel, 48-pixel, 56-pixel, 64-pixel, 72-pixel, 80-pixel, 88-pixel, 96-pixel, 104-pixel, 112-pixel, 120-pixel, 128-pixel}, and the BVD directions are two horizontal and two vertical directions.

[0072] The base candidate is selected from the top five candidates in the reordered IBC merge list. And all possible MBVD refinement positions (20x4) for each base candidate are reordered based on the SAD cost between the template (one row above and one column left of the current block) and its each refinement position. Finally, the top 8 refinement positions with the lowest template SAD cost are kept as available positions, which are used for MBVD index coding. The MBVD index is binarized by Rice code with a parameter equal to 1.

[0073] The blocks coded by IBC-MBVD do not inherit the flipping type from the neighboring blocks coded by RR-IBC.

[0074] IBC-LIC In JVET-AC0112 (Yang Wang et al., “EE2-3.6: IBC-CIIP, IBC-GPM and IBC-LIC,” Joint Video Team (JVT) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29, 29th Meeting, Teleconference, Jan. 11-20, 2023, Doc.: JVET-AC0112), ECM (Enhanced Compression Model) adopted IBC-LIC. Intra block copy with local illumination compensation (IBC-LIC) is a coding tool that compensates for the local illumination changes within the picture between a CU coded using IBC and its prediction block through linear equations. The derivation of the parameters of the linear equations is the same as for LIC for inter prediction, except that the block vector used in IBC-LIC produces the reference template. IBC-LIC can be applied to IBC AMVP mode and IBC merge mode. IBC with advanced motion vector prediction mode (IBC AMVP) corresponds to the IBC mode that directly signals the motion vector predictor index and, if necessary, the motion vector difference. The name IBC AMVP is used in this disclosure. For IBC AMVP mode, an IBC-LIC flag is signaled to indicate the use of IBC-LIC. For IBC merge mode, the IBC-LIC flag is inferred from the merge candidate.

[0075] IBC-GPM In JVET-AC0112, IBC-GPM was adopted in ECM. Intra block copy with geometry partitioning mode (IBC-GPM) is a coding tool that geometrically partitions a CU into two sub-partitions. The prediction signals of the two sub-partitions are generated using IBC and intra prediction. IBC-GPM can be applied to regular IBC merge mode or IBC TM merge mode. The intra prediction mode (IPM) candidate list is constructed in the same way as in GPM using inter and intra prediction, with the IPM candidate list size pre-defined to 3. There are 48 geometric partitioning modes in total, which are grouped into the following two geometric partitioning mode groups:

[0076] When IBC-GPM is used, an IBC-GPM geometric partition mode set flag is signaled to indicate whether the first or second geometric partition mode set is selected, followed by a geometric partition mode index. An IBC-GPM intra flag is signaled to indicate whether intra prediction is used for the first sub-partition. When intra prediction is used for the sub-partition, an intra prediction mode index is signaled. When IBC is used for the sub-partition, a merge index is signaled.

[0077] IBC-CIIP In JVET-AC0112, ECM adopted IBC-CIIP. The combined intra block copy and intra prediction (IBC-CIIP) is a coding tool for a CU that uses IBC prediction (e.g., IBC prediction from IBC with merge mode) and intra prediction (e.g., intra prediction from planar or DC mode) to derive two prediction signals, weights the two prediction signals, and produces a final prediction as follows: , where and denote the IBC prediction signal and the intra prediction signal, respectively. For IBC merge mode and IBC AMVP mode, are set equal to (13, 4) and (1, 1), respectively.

[0078] To improve the coding performance of IBC / IntraTMP mode, various transform methods and / or down- / up-sampling for IBC / IntaTMP coded blocks are disclosed.

[0079] In the present invention, when a scheme of Intra Block Copy is used to generate the prediction value of the current block, various transform methods, such as various transform modes, are proposed to improve the coding efficiency. The scheme of Intra Block Copy means that a set of pre-defined reconstructed samples in the current picture is indicated (referenced) by the block vector used for the current block and / or template matching. For example, the current block refers to a coding unit containing only luma coding blocks, containing only chroma coding blocks (including Cb coding blocks and Cr coding blocks), and / or containing both luma and chroma coding blocks. When the current block is coded using IBC or Intra TMP, the scheme of Intra Block Copy is used and the prediction value of the current block is generated by copying and / or referring to a set of reconstructed samples in the current picture that have already been reconstructed, where the set can have the same size as the current block. At the encoder, the residual of the current block is calculated as the difference between the original source of the current block and the prediction value of the current block. In conventional coding systems, only the transform mode DCT-II is used to process the residual from IBC and / or Intra TMP mode.

[0080] According to embodiments of the present application, one or more proposed transform modes are applied to the residual or to the temporarily transformed coefficients related to the residual to obtain improved transform coefficients. The improved transform coefficients can be the final transform coefficients used in the subsequent residual coding process and encoded in the bitstream to the decoder (after quantization). Or, the improved transform coefficients can be another temporarily transformed coefficients, which will be further transformed by a subsequent transform process. At the decoder, the final transform coefficients are parsed and recovered from the bitstream (after dequantization). If any subsequent transform process is applied after the proposed transform mode at the encoder, the final transform coefficients are first inverse transformed by the subsequent transform process (to obtain the improved transform coefficients) and then inverse transformed by the proposed transform mode. Then, after all inverse transform processes are completed, the residual of the current block is obtained and can be used to reconstruct the current block by adding the predicted value of the current block. In one embodiment, the proposed transform mode refers to a target transform mode that can be selected from a transform set. For example, the transform set includes two or more transform modes. For another example, the transform set and / or the transform selection is used by or unified with other non-block vector coding tools, such as inter coding tools using motion information for prediction and / or intra coding tools using spatial reference samples using DC, planar or angular prediction modes for prediction. For example, the transform set includes transform skip, primary transform, secondary transform or a combination of the above. For an example of transform skip mode, no transform operation and / or no transform kernel is needed. For an example of primary transform, it is applied to the residual at the encoder and / or it is applied to the final transform coefficients at the decoder (if the final transform coefficients are generated using only the primary transform) or to the temporarily transformed coefficients (if the final transform coefficients are first inverse transformed by other transform processes). The primary transform can be separable or non-separable. An example of separable primary transform is MTS, which explicitly and / or implicitly determines the transform type. Explicit transform selection can use MTS index to select the transform type in horizontal and / or vertical direction. Implicit transform selection can use position-dependent transform kernel selection. For an example of secondary transform, it is applied to the temporarily transformed coefficients at the encoder and / or it is applied to the final transform coefficients at the decoder (if the final transform coefficients are generated using the primary transform and the secondary transform). The secondary transform can be separable or non-separable. An example of non-separable secondary transform is LFNST or any RST variant. In another embodiment, the proposed transform mode is used as an additional transform option indicated by explicit syntax.For the example of MTS type transform mode using a prediction with a used block vector for the current block, the proposed transform option is applied and / or referred to using MTS type transform mode for the current block. The proposed transform option is a complement to the conventional option of using only DCT-II transform mode.

[0081] In one embodiment, the proposed transform mode refers to partial transform setting. The following is an example of applying the proposed method to the residual (at the encoder). Note that the proposed method can be applied to the temporary transform coefficients at the encoder and / or the corresponding operation performed at the decoder. The partial transform setting at the encoder means that (1) only the target part of the residual in the current block is kept, (2) the remaining residual in the current block (excluding the predefined part) is cleared to zero, and / or (3) the target part of the residual in the current block is transformed. The partial transform setting at the decoder means that (1) the transform coefficients of the current block (related to the target part of the residual) are received, and / or (2) the received transform coefficients are inverse transformed and the residual of the current block is retrieved. For (1), the coefficients in the non-target part of the current block can be or should be zero.

[0082] In one sub-embodiment, to define the target part, the current block is partitioned into multiple transform blocks following a partitioning rule, and the target part refers to one of the transform blocks. For example, the partitioning rule is to apply quad-tree partitioning to the current block, as shown in Figure 8

[0083] ​The current block is then split into 4 transform blocks. The target portion is indicated as one of the 4 transform blocks by implicit definition or explicit signaling. That is, the target portion is indicated as a region in the current block (which is indicated as a shaded region and / or can be considered as a transform block of the current block) by implicit definition or explicit signaling. In the case of a quad-tree, there are four possible locations for the target portion. The first, second, third, and fourth locations (as shown in 810, 820, 830, and 840) are the target portion of the top-left, top-right, bottom-left, and bottom-right regions of the current block, respectively. As another example, the splitting rule is to apply a 2:2 binary tree split to the current block. The current block is then split into 2 equal size transform blocks by vertical split or horizontal split. The target portion is indicated as one of the 2 transform blocks. The split direction (vertical or horizontal) and / or portion indication depends on implicit definition or explicit signaling. As another example, the splitting rule is to apply a 1:3 binary tree split to the current block. The current block is then split into 2 transform blocks of size 1:3 by vertical split or horizontal split. The target portion is indicated as the smaller one of the 2 transform blocks. If vertical split is used, the size ratio of the 2 transform blocks (left and right) is 1:3 or 3:1. If horizontal split is used, the size ratio of the 2 transform blocks (top and bottom) is 1:3 or 3:1. The split direction (vertical or horizontal) and / or size ratio (1:3 or 3:1, with 1 in the size ratio referred to as the target portion) depends on implicit definition or explicit signaling. The transform type of the transform block of the target portion is decided according to the split direction and / or portion indication (any pre-defined MTS type). The transform type can be selected from DCT-II and / or any MTS type and / or transform skip mode. For the case of 2 transform blocks, an example of the indication is shown in Table 6.

[0084]

[0085] For another example of 4 transform blocks, different portion indications refer to different combinations of transform modes. For any other combination of {horizontal, vertical} or DCT-VIII and DST-VII, the portion indication can refer to {DCT-VIII, DCT-VIII}, {DST-VII, DCT-VIII}, {DCT-VIII, DST-VII}, {DST-VII, DST-VII}.

[0086] In another sub-embodiment, to define the target portion, the residuals around the split line are scanned and put into the target portion. At the decoder, the coefficients associated with the target portion are scanned. For example, the split line is used to divide the current block into multiple (e.g., 2) prediction blocks, and different block vectors are used to generate the prediction values for each prediction block, and the prediction values from the multiple prediction blocks are combined as the prediction value around the split line. The residuals of the prediction value calculation close to the split line are scanned and put into the square transform block in the scan order. At the decoder, the coefficients associated with the square transform block as the target portion are scanned in the scan order.

[0087] In another sub-embodiment, the current block is not divided into multiple transform blocks by the proposed partial transform setting. The target portion does not refer to a separate transform block, but refers to a subset of the residuals of the current block. For example, the target portion refers to the first N residuals or coefficients after diagonal scan in the current block. For another example, the target portion refers to the first half of the current block by vertical or horizontal split. Specifically, the target portion refers to the first half residuals or coefficients of the current block after vertical or horizontal split.

[0088] In another sub-embodiment, the proposed method under the same root embodiment can be an optional mode. For example, a flag is signaled to indicate whether the partial transform setting of the current block is used. If the flag indicates that it is used, an index is signaled to indicate which sub-setting (in each sub-embodiment under the partial transform setting) is used for the current block.

[0089] In another embodiment, whether the function of the proposed transform mode is supported depends on a high-level syntax. For example, the syntax at tile, slice, picture, SPS or PPS level. In one case, the syntax is shared with other non-block vector coding tools. In another case, the syntax is independent of other non-block vector coding tools.

[0090] In another embodiment, the transform skip mode can be one of the transform modes of the current block, and the current block uses the scheme of intra block copy to generate the prediction value of the current block. Here the maximum size of the transform skip mode can be indicated by a high-level syntax. For example, the syntax at tile, slice, picture, SPS or PPS level. In one case, the syntax is shared with other non-block vector coding tools. In another case, the syntax is independent of other non-block vector coding tools.

[0091] In one sub-embodiment, the high-level syntax for indicating the maximum size of transform skip mode is independent of other high-level syntax. For example, the high-level syntax for indicating the maximum size of transform skip mode for the mode type inter, the mode type intra, and / or the mode type IBC (the third mode type) is separate.

[0092] In another embodiment, when IBC uses vertical or horizontal BV (BV_x = 0 or BV_y = 0) or the reference sample is horizontal-oriented or vertical-oriented, the BDPCM is applied to the quantized residual values, where is predicted according to its neighboring quantized residual values. For the vertical IBC BDPCM prediction mode, for 0≤j≤(N-1), the following equation is used to derive : .

[0093] In another embodiment, the proposed transform mode refers to the inheritance transform setting. The inheritance transform setting means that the inheritance transform candidate of the current block is supported. An example of the IBC case is shown as follows. One or more inherited transform candidates are inherited from the IBC reference block. The most popular K inherited transform candidates can be used for the current block, where K is 1, 2, or any pre-defined positive integer. One is selected from the K inherited candidates as the transform mode of the current block. In one embodiment, several proposed transform candidates with the inheritance transform setting are included in the transform set and / or the transform set provides the transform mode of the current block.

[0094] In one sub-embodiment, for IBC merge / inter mode (e.g., IBC merge and / or IBC AMVP), the reference block refers to the block that provides the block vector information for the current block.

[0095] In another sub-embodiment, the reference block is indicated by the block vector or the template matching of (the IBC block’s) (the intra-TMP block’s). In addition to deriving the prediction value of the current block, the reference block also provides the transform information of the current block.

[0096] In another sub-embodiment, the setting of the inheritance transform depends on the coding information of the RR-IBC. For example, when the current block uses RR-IBC, the transform type is flipped. For another example, when the current block uses RR-IBC, the transform type of the horizontal and vertical directions are exchanged. For another example, when the current block uses RR-IBC, the transform mode selection is different from other coding modes.

[0097] In another embodiment, the proposed transform mode supports all or any subset of JCCR (Joint Coding of Cb and Cr residual / coefficient) mode. The mapping between tu_cbf_cb / tu_cbf_cr and the reconstruction of Cb and Cr residuals is exemplified in Table 7.

[0098]

[0099] In another embodiment, whether the proposed transform mode is applied to the current block depends on a block-level syntax.

[0100] In another embodiment, a down / up-sampling IBC mode is proposed as follows. When the current block adopts the down / up-sampling IBC mode, the reference block indicated by the block vector is larger / smaller than the current block, the reference samples are down / sampled, and then used to predict the current block. Figure 9 An example of the down-sampling IBC mode is illustrated, in which the current block 920 uses the reference block 930 in the current picture 910 to generate the IBC prediction value. The current block size is (W, H), and the reference block size is (kW, kH). Therefore, the reference block is down-sampled by k:1 before being used as the prediction value.

[0101] The proposed method in this disclosure can be according to implicit rules (e.g., block width, height, or area) or according to explicit rules (e.g., block level or high level such as tile, slice, picture, SPS, or PPS level). For example, the proposed method is applied when the block area is smaller / larger than a threshold. For the example of the current block using MTS type transform mode using block vector to generate prediction and / or using block vector coding tool for coding, the proposed transform mode refers to using MTS type transform mode for the current block using only a sub-mode of the block vector coding tool. The sub-mode of the block vector coding tool can be IBC-CIIP, IBC-GPM, IBC-LIC, IBC-MBVD, RR-IBC, IBC merge, IBC AMVP (or referred to as IBC inter), and / or any sub-mode of IBC or intraTMP. The sub-mode of the block vector coding tool (e.g., IBC, IBC merge, IBC inter, or intraTMP as the root mode) can be any optional mode that can be used / signaled if the root mode is enabled / applied. For the example of the current block using MTS type transform mode using block vector to generate prediction and / or using block vector coding tool for coding, the proposed transform mode refers to allowing MTS type transform mode for the current block only when the width, height, and / or area of the current block is smaller than a predefined threshold.

[0102] The term "block" in this disclosure can refer to a TU / TB, a CU / CB, a PU / PB, a predefined region, or a CTU / CTB.

[0103] Any combination of the methods proposed in this disclosure can be applied.

[0104] Any of the proposed methods can be implemented in an encoder and / or a decoder. For example, any of the proposed methods can be implemented in an inter / intra / IBC / prediction / transform module (e.g., intra prediction 110 / inter prediction 112 / transform 118 / inverse transform 126) of an encoder, and / or an inter / intra / IBC / prediction / inverse transform module (e.g., intra prediction 150 / MC 152 / inverse transform 126) of a decoder. Alternatively, any of the proposed methods can be implemented as a circuit coupled to an inter / intra / IBC / prediction / transform module of an encoder and / or an inter / intra / IBC / prediction / (inverse) transform module of a decoder, thereby providing information required by the inter / intra / IBC / prediction / transform module (e.g., a transform module or an inverse transform module). Although the modules (e.g., units 110 / 112 / 118 / 126 in Figure 1A Figure 1B Figure 1A Figure 1B Although the modules (e.g., units 110 / 112 / 118 / 126 in

[0105] Figure 10 ​​​A flowchart of an exemplary video coding system using multiple transforms for IBC / IntraTMP modes according to embodiments of the application is shown. The steps shown in the flowchart can be implemented as program code executable on one or more processors (e.g., one or more CPUs) on the encoder side. The steps shown in the flowchart can also be implemented based on hardware, e.g., one or more electronic devices or processors arranged to perform the steps in the flowchart. According to the method, input data associated with a current block in a current picture is received in step 1010, where the input data includes residual data of the current block to be encoded on the encoder side or transform coefficients of the current block to be decoded on the decoder side, and prediction data of the current block is generated by applying intra block copy or intra template matching prediction. A target transform mode is applied to the current block in step 1020 to derive final transform coefficients on the encoder side or reconstructed residual data on the decoder side, where the target transform mode includes subblock transform or partial transform, and where the subblock transform partitions the current block into a plurality of subblocks and applies transform processing to one or more subblocks as a target portion of the current block, or the partial transform applies transform processing only to a target portion of the current block. The final transform coefficients on the encoder side or the reconstructed residual data on the decoder side are provided in step 1030.

[0106] The flowchart shown is intended to illustrate an example of video coding according to the present application. Those having ordinary skill in the art can modify each step, rearrange steps, split steps, or combine steps to implement the present application without departing from the spirit of the present application. In this disclosure, specific syntax and semantics have been used to illustrate examples of implementing embodiments of the present application. Those having ordinary skill in the art can practice the present application by substituting equivalent syntax and semantics without departing from the spirit of the present application.

[0107] The above description is given to enable those skilled in the art to practice the present application as provided in the particular context. Various modifications to the described embodiments will be readily apparent to those skilled in the art and the generic principles defined herein can be applied to other embodiments. Therefore, the present application is not intended to be limited to the particular embodiments described, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein. In the above detailed description, various specific details are presented to provide a thorough understanding of the present application. However, those skilled in the art will recognize that the present application can be practiced without some or all of these details.

[0108] Embodiments of the present application as described above can be implemented in various hardware, software program codes, or a combination of both. For example, embodiments of the present application can be one or more circuits integrated into a video compression chip or program codes integrated into a video compression software to perform the processes described herein. Embodiments of the present application can also be program codes executed on a digital signal processor (DSP) to perform the processes described herein. The present application can also relate to a variety of functions executed by a computer processor, a digital signal processor, a microprocessor, or a field programmable gate array (FPGA). These processors can be configured to perform specific tasks according to the present application by executing machine-readable software program codes or firmware programs embodied in processor- readable media that define the particular methods embodied by the present application. The software program codes or firmware programs can be developed in different programming languages and different formats or styles. The software program codes can also be compiled into different program languages, styles, or formats for implementation. However, different program codes, different formats, or different styles of software program codes, and other means of configuring program codes to perform the tasks according to the present application will not depart from the spirit and scope of the present application.

[0109] The present application can be carried out in other specific ways without departing from the spirit or essential characteristics of the application. The described embodiments are to be considered in all respects only as illustrative and not restrictive. Accordingly, the scope of the application is indicated by the appended claims rather than by the foregoing description, and all changes that come within the meaning and range of equivalents are intended to be embraced therein.

Claims

1. A video encoding / decoding method, the method comprising: Receive input data related to the current block in the current image, wherein the input data includes residual data for encoding the current block at the encoder end or transform coefficients for decoding the current block at the decoder end, and the prediction data of the current block is generated by applying intra-block copying or intra-template matching prediction. A target transform mode is applied to the current block to derive the final transform coefficients at the encoder or to derive the reconstructed residual data at the decoder. The target transform mode includes a sub-block transform or a partial transform. The sub-block transform divides the current block into multiple sub-blocks and applies transform processing to one or more sub-blocks that are the target part of the current block, or the partial transform applies transform processing only to the target part of the current block. as well as The final transform coefficients are provided at the encoder end, or the reconstructed residual data is provided at the decoder end.

2. The video encoding and decoding method according to claim 1, characterized in that, When the target transformation mode corresponds to the sub-block transformation, the target portion corresponding to one of the multiple sub-blocks is selected as one of the transformation blocks, and the transformation process is applied.

3. The video encoding and decoding method according to claim 1, characterized in that, Use a quadtree to divide the current block into 4 sub-blocks.

4. The video encoding and decoding method according to claim 1, characterized in that, The target section can be selected using either implicit definition or explicit sending.

5. The video encoding and decoding method according to claim 1, characterized in that, Use a binary tree to divide the current block into two sub-blocks.

6. The video encoding and decoding method according to claim 5, characterized in that, The two sub-blocks may be of equal or unequal size.

7. The video encoding and decoding method according to claim 6, characterized in that, The target portion is indicated by the smaller of the unequal-sized sub-blocks.

8. The video encoding and decoding method according to claim 1, characterized in that, The direction of the segmentation, the target portion, or both are indicated by implicit definition or explicit signaling.

9. The video encoding and decoding method according to claim 1, characterized in that, When the target transformation mode corresponds to the sub-block transformation, the residual of the non-target part of the current block is cleared to zero or the coefficient of the non-target part of the current block is zero.

10. The video encoding and decoding method according to claim 1, characterized in that, When the target transformation mode corresponds to the partial transformation, the target part follows a predetermined dividing line.

11. The video encoding and decoding method according to claim 10, characterized in that, The residual around the predetermined dividing line is scanned and placed into the target portion at the encoder end, or the coefficients associated with the target portion are scanned.

12. The video encoding and decoding method according to claim 1, characterized in that, When the target transformation mode corresponds to the partial transformation, the target part corresponds to a transformation block.

13. The video encoding and decoding method according to claim 1, characterized in that, When the target transformation mode corresponds to the partial transformation, the residual of the non-target part of the current block is cleared to zero, or the coefficient of the non-target part of the current block is zero.

14. The video encoding and decoding method according to claim 1, characterized in that, The target transformation mode is indicated as an additional transformation option by explicitly sending a message.

15. The video encoding and decoding method according to claim 1, characterized in that, The target transformation mode is enabled if one or more enable conditions are met.

16. The video encoding and decoding method according to claim 15, characterized in that, The one or more enabling conditions include one or more sub-modes of the target transform mode that can be used by the target block vector encoding / decoding tool.

17. The video encoding and decoding method according to claim 16, characterized in that, The target block vector encoding / decoding tool corresponds to either IBC merge mode or IBC inter-frame mode.

18. The video encoding and decoding method according to claim 15, characterized in that, The one or more enabling conditions include that the target transformation mode is available for blocks that meet the size conditions.

19. The video encoding and decoding method according to claim 15, characterized in that, The one or more enabling conditions, including whether the target transformation mode is supported, depend on the high-level syntax.

20. A video encoding / decoding apparatus, the apparatus comprising one or more electronic components or a processor, configured to: Receive input data related to the current block in the current image, wherein the input data includes residual data for encoding the current block at the encoder end or transform coefficients for decoding the current block at the decoder end, and the prediction data of the current block is generated by applying intra-block copying or intra-template matching prediction. A target transform mode is applied to the current block to derive the final transform coefficients at the encoder or to derive the reconstructed residual data at the decoder. The target transform mode includes a sub-block transform or a partial transform. The sub-block transform divides the current block into multiple sub-blocks and applies transform processing to one or more sub-blocks that are the target part of the current block, or the partial transform applies transform processing only to the target part of the current block. as well as The final transform coefficients are provided at the encoder end, or the reconstructed residual data is provided at the decoder end.

Images