Coefficient reordering in video encoding and decoding

Abstract

Methods, systems, and apparatus for video processing are described. An example method for processing video data includes performing a conversion between the current block of video and the video bitstream according to rules. These rules specify the use of coefficient reordering based on the codec information of the current block, by which the coefficients of the current block parsed from the bitstream are reordered before applying dequantization, inverse transform, or reconstruction processes.

Description

[0001] Cross-reference of related applications

[0002] This application claims priority and benefit to International Patent Application No. PCT / CN2020 / 110427, filed on August 21, 2020. The entire disclosure of the aforementioned application is incorporated herein by reference in part. Technical Field

[0003] This patent document relates to image encoding and decoding as well as video encoding and decoding. Background Technology

[0004] Digital video accounts for the largest share of bandwidth usage on the Internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, the bandwidth demand for digital video is expected to continue to grow. Summary of the Invention

[0005] This document discloses techniques that can be used by video encoders and decoders to process the encoded and decoded representation of video using control information useful for decoding the encoded and decoded representation.

[0006] In one example, a method for processing video data includes performing a transformation between the current block of the video and the video bitstream according to rules. The rules specify whether to enable a transform skip mode based on the codec information of the current block. The transform skip mode is a codec mode that skips the predictive residual transform of a video block.

[0007] In another example, a method for processing video data includes performing a transformation between the current block of the video and the video bitstream according to rules. The rules specify that the representativeness coefficients of one or more representative blocks, where at least one representative block is different from the current block, determine the transform skip mode to be used on the current block. The one or more representative blocks include the last N blocks preceding the current block in decoding order, having the same prediction mode, where N is an integer greater than 1.

[0008] In another example, a method for processing video data includes performing a transformation between the current block of the video and the video bitstream according to rules. The rules specify representativeness coefficients of one or more representative blocks of the video to determine the transformation of the current video block using a transform skip mode path. The representativeness coefficients include coefficients relative to the representative blocks at predefined positions (xPos, yPos).

[0009] In another example, a method for processing video data includes performing a conversion between the current block of the video and the bitstream according to rules. The rules specify that a transform skip mode is applied in response to an identity transform (IT) being used to encode and decode the current block. The current block is encoded and decoded in either an inter-frame encoding / decoding mode or a direct encoding / decoding mode.

[0010] In another example, a method for processing video data includes performing a conversion between the current block of the video and the video bitstream according to rules. The rules specify the use of coefficient reordering based on the codec information of the current block. Through coefficient reordering, the coefficients of the current block parsed from the bitstream are reordered before applying dequantization, inverse transform, or reconstruction processes.

[0011] In another example, a method for processing video data includes performing a conversion between a current block of video and a video bitstream according to rules. The rules specify that the current block should be divided into at least two parts in response to the current block having a specific pattern. At least one of the at least two parts does not have a non-zero residual indicated in the bitstream.

[0012] In another example, a video processing method is disclosed. The method includes: converting video blocks of a video to a codec representation of the video; determining, based on a rule, whether a horizontal or vertical identity transformation is applied to the video block; and performing the transformation based on the determination. The rule specifies the relationship between the determination and the representative coefficients of the decoding coefficients from one or more representative blocks of the video.

[0013] In another example, a different video processing method is disclosed. This method includes: converting video blocks to a codec representation of the video; determining, based on a rule, whether a horizontal or vertical identity transformation is applied to the video block; and performing the conversion based on that determination. The rule specifies the relationship between the determination and the decoded luminance coefficients of the video block.

[0014] In another example, a different video processing method is disclosed. This method includes: converting video blocks of a video to their codec representations; determining, based on a rule, whether a horizontal or vertical identity transformation is applied to the video blocks; and performing the transformation based on that determination. The rule specifies the relationship between the determination and a value V, which is associated with decoding coefficients or representative coefficients of a representative block.

[0015] In another example, another video processing method is disclosed. This method includes determining that one or more syntax fields exist in the codec representation of a video, wherein the video contains one or more video blocks; and based on the one or more syntax fields, determining whether a horizontal identity transformation or a vertical identity transformation is enabled on the video blocks in the video.

[0016] In another example, another video processing method is disclosed. This method includes making a first determination regarding whether to enable the use of an identity transformation for the conversion between video blocks and the codec representation of the video; making a second determination regarding whether to enable a zeroing operation during the conversion; and performing the conversion based on the first and second determinations.

[0017] In another example, another video processing method is disclosed. This method includes performing a conversion between video blocks and a codec representation of the video; wherein the video blocks are represented as codec blocks in the codec representation, wherein the non-zero coefficients of the codec blocks are restricted to one or more sub-regions; and wherein an identity transformation is applied to generate the codec blocks.

[0018] In another example, a different video processing method is disclosed. This method includes performing a conversion between a video comprising one or more video regions and a codec representation of the video, wherein the codec representation conforms to a format rule; wherein the format rule specifies that the coefficients of the video regions are reordered according to the mapping after being parsed from the codec representation.

[0019] In another example, a different video processing method is disclosed. The method includes: converting video blocks and a codec representation of the video; determining whether the video blocks satisfy the condition for signaling notification of a partial residual block in the codec representation; and performing the conversion based on the determination; wherein the partial residual block is divided into at least two parts, wherein at least one part does not have a non-zero residual signaled in the codec representation.

[0020] In yet another example, a video encoder apparatus is disclosed. The video encoder includes a processor configured to implement the methods described above.

[0021] In yet another example, a video decoder apparatus is disclosed. The video decoder includes a processor configured to implement the methods described above.

[0022] In yet another example, a computer-readable medium on which code is stored is disclosed. This code embodies one of the methods described herein in the form of processor-executable code.

[0023] These and other features are described in this document. Attached Figure Description

[0024] Figure 1 A block diagram of an example video encoder is shown.

[0025] Figure 2 Examples of 67 intra-frame prediction modes are shown.

[0026] Figure 3A An example of a reference sample for wide-angle intra-frame prediction is shown.

[0027] Figure 3B Another example of a reference sample for wide-angle intra-frame prediction is shown.

[0028] Figure 4 The discontinuity problem is shown when the orientation exceeds 45 degrees.

[0029] Figure 5A An example definition of the sample points used by the PDPC applied to diagonal intra-frame mode and adjacent angle intra-frame mode is shown.

[0030] Figure 5B Another example definition of the sample points used by the PDPC applied to diagonal intra-frame mode and adjacent angle intra-frame mode is shown.

[0031] Figure 5C Another example definition of the sample points used by the PDPC applied to diagonal intra-frame mode and adjacent angle intra-frame mode is shown.

[0032] Figure 5D This shows yet another example definition of the samples used by the PDPC applied to diagonal intra-frame mode and adjacent angle intra-frame mode.

[0033] Figure 6 Examples of 4×8 and 8×4 block partitioning are shown.

[0034] Figure 7 Examples of block partitioning are shown for all blocks except 4×8, 8×4, and 4×4.

[0035] Figure 8 An example of a quadratic transformation in JEM is shown.

[0036] Figure 9 An example of the simplified quadratic transformation LFNST is shown.

[0037] Figure 10A An example of positive simplification transformation is shown.

[0038] Figure 10B An example of the inverse reduction transformation is shown.

[0039] Figure 11 An example of a positive LFNST8×8 process with a 16×48 matrix is ​​shown.

[0040] Figure 12 Examples of scan positions 17 to 64 for non-zero elements are shown.

[0041] Figure 13 Examples of subblock transformation modes SBT-V and SBT-H are shown.

[0042] Figure 14AAn example of Scan Region Based Coefficient Coding (SRCC) is shown.

[0043] Figure 14B Another example of scan region-based coefficient encoding and decoding (SRCC) is shown.

[0044] Figure 15A Example constraints of IST based on the position of non-zero coefficients are shown.

[0045] Figure 15B Another example constraint of IST based on the position of non-zero coefficients is shown.

[0046] Figure 16A An example of a zeroed-type TS codec block is shown.

[0047] Figure 16B Another example of a zeroed-type TS codec block is shown.

[0048] Figure 16C Another example of a zeroed-type TS codec block is shown.

[0049] Figure 16D Another zero-type TS codec block is shown.

[0050] Figure 17 This is a block diagram of an example video processing system.

[0051] Figure 18 This is a block diagram illustrating a video encoding / decoding system according to some embodiments of the present disclosure.

[0052] Figure 19 This is a block diagram illustrating an encoder according to some embodiments of the present disclosure.

[0053] Figure 20 This is a block diagram illustrating a decoder according to some embodiments of the present disclosure.

[0054] Figure 21 This is a block diagram of a video processing device.

[0055] Figure 22 This is a flowchart of an example method for video processing.

[0056] Figure 23 An example of L-shaped segmentation of a partial residual block is shown.

[0057] Figure 24 This is a flowchart representation of a method for processing video data according to this technology.

[0058] Figure 25This is a flowchart representation of another method for processing video data according to this technology.

[0059] Figure 26 This is a flowchart representation of another method for processing video data according to this technology.

[0060] Figure 27 This is a flowchart representation of another method for processing video data according to this technology.

[0061] Figure 28 This is a flowchart representation of another method for processing video data according to this technology.

[0062] Figure 29 This is a flowchart representation of yet another method for processing video data based on this technology. Detailed Implementation

[0063] The use of section headings in this document is for ease of understanding and does not limit the application of the technologies and embodiments disclosed in each section to that section only. Furthermore, the use of H.266 terminology in some specifications is merely for ease of understanding and not to limit the scope of the disclosed technologies. Thus, the technologies described herein are also applicable to other video codec protocols and designs.

[0064] 1. Overview

[0065] This document relates to video codec technology. Specifically, it covers transform skipping modes and transform types (e.g., identity transform) in video codecs. It can be applied to existing video codec standards (e.g., HEVC) or upcoming standards (General Video Codec). It can also be applied to future video codec standards or video codecs.

[0066] 2. Preliminary Discussion

[0067] Video codec standards have primarily evolved through the development of well-known ITU-T and ISO / IEC standards. ITU-T developed H.261 and H.263, while ISO / IEC developed MPEG-1 and MPEG-4. These two organizations jointly developed the H.262 / MPEG-2 video and H.264 / MPEG-4 Advanced Video Coding (AVC) standards, as well as the H.265 / HEVC standard. Since H.262, video codec standards have been based on a hybrid video codec architecture, utilizing temporal prediction plus transform coding. To explore future video codec technologies beyond HEVC, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET) in 2015. Since then, JVET has adopted many new methods and incorporated them into reference software called the Joint Exploration Model (JEM). In April 2018, the Joint Video Experts Group (JVET) between VCEG (Q6 / 16) and ISO / IEC JTC1 SC29 / WG11 (MPEG) was established to work on the VVC (Versatile Video Coding) standard, with the goal of reducing the bit rate by 50% compared to HEVC.

[0068] 2.1. Encoding and decoding process of a typical video codec

[0069] Figure 1 An example of a VVC encoder block diagram is shown, comprising three in-loop filtering blocks: Deblocking Filter (DF), Sample Adaptive Offset (SAO), and ALF. Unlike DF, which uses predefined filters, SAO and ALF utilize the raw samples of the current image, signaling the offset and filter coefficients with encoding / decoding side information. They reduce the mean square error between the raw and reconstructed samples by adding an offset and by applying a Finite Impulse Response (FIR) filter, respectively. ALF is the last processing stage for each image and can be viewed as a tool attempting to capture and repair artifacts created in previous stages.

[0070] 2.2. Intra-mode encoding and decoding with 67 intra-prediction modes

[0071] To capture arbitrary edge directions presented in natural video, the number of intra-frame directional modes has been expanded from the 33 used in HEVC to 65. Additional directional modes include... Figure 2 The dashed arrows in the diagram depict this, and the planar and DC modes remain unchanged. These dense directional intra-prediction modes are applicable to all block sizes as well as luma and chroma intra-prediction.

[0072] Traditional intra-frame prediction direction is defined as ranging from 45 degrees to -135 degrees in a clockwise direction, such as... Figure 2 As shown. In VTM2, for non-square blocks, several traditional angular intra-prediction modes are adaptively replaced with wide-angle intra-prediction modes. The replaced modes are signaled using the original method and remapped to the wide-angle mode index after parsing. The total number of intra-prediction modes remains unchanged, for example, 67, and the intra-mode encoding and decoding remain unchanged.

[0073] In HEVC, each intra-codec block has a square shape, with each side's length being a power of 2. Therefore, division is unnecessary for generating intra-prediction values ​​using DC mode. In VVV2, blocks can have rectangular shapes, which typically requires division for each block. To avoid division for DC prediction, only the longer sides are used to calculate the average of non-square blocks.

[0074] 2.3. Wide-angle intra-frame prediction for non-rectangular blocks

[0075] Traditional angular intra-prediction directions are defined clockwise from 45 degrees to -135 degrees. In VTM2, for non-square blocks, several traditional angular intra-prediction modes are adaptively replaced with wide-angle intra-prediction modes. The replaced modes are communicated using the original method signaling and remapped to the wide-angle mode index after resolution. The total number of intra-prediction modes for a given block remains unchanged, for example, 67, and the intra-mode encoding and decoding remain unchanged.

[0076] To support these predicted directions, a top reference of length 2W+1 and a left reference of length 2H+1 are defined as follows: Figures 3A to 3B As shown.

[0077] The number of replacement modes in wide-angle directional mode depends on the aspect ratio of the block. Table 1 shows the intra-prediction modes for replacement.

[0078] Table 1: Intra-prediction modes replaced by wide-angle mode

[0079]

[0080] like Figure 4 As shown, in the case of wide-angle intra-frame prediction, two vertically adjacent prediction samples can use two non-adjacent reference samples. Therefore, low-pass reference sample filtering and side smoothing are applied to wide-angle prediction to reduce the increased gap Δp. α The negative impact.

[0081] 2.4. Location-dependent intra-frame prediction combination

[0082] In VTM2, the intra-prediction results for planar modes are further modified using the position-dependent intraprediction combination (PDPC) method. PDPC is an intra-prediction method that combines unfiltered boundary reference samples with HEVC-style intra-prediction using filtered boundary reference samples. PDPC is applied to the following intra-modal modes without signaling notification: planar, DC, horizontal, vertical, lower left angle mode and its eight adjacent angle modes, and upper right angle mode and its eight adjacent angle modes.

[0083] Using a linear combination of intra-frame prediction modes (DC, plane, angle) and reference samples, the prediction sample pred(x,y) is predicted according to the following equation:

[0084] pred(x,y)=(wL×R -1,y +wT×R x,-1 –wTL×R -1,-1 +(64–wL–wT+wTL)×pred(x,y)+32)>>6

[0085] Where R x,-1 R -1,y R represents the reference sample points located at the top and left of the current sample point (x,y), respectively. -1,-1 This represents the reference sample point located at the top left corner of the current block.

[0086] If PDPC is applied to DC intra-frame mode, planar intra-frame mode, horizontal intra-frame mode, and vertical intra-frame mode, no additional boundary filtering is required, but it is required in the case of HEVC DC mode boundary filtering or horizontal / vertical mode edge filtering.

[0087] Figures 5A to 5D Reference samples (R) of PDPC applied to various prediction modes are shown. x,-1 ,R -1,y and R -1,-1 The definition of ). The predicted sample point pred(x',y') is located at (x',y') within the prediction block. The reference sample point R. x,-1 The coordinates x are given by the following formula: x = x' + y' + 1, with reference sample point R. -1,y The coordinates y are similarly given by the following formula: y = x' + y' + 1. Figure 5A The top-right diagonal pattern is shown. Figure 5B The bottom left diagonal pattern is shown. Figure 5C The adjacent diagonal top right pattern is shown. Figure 5D This shows an adjacent diagonal bottom left pattern.

[0088] The PDPC weights depend on the prediction pattern, as shown in Table 2.

[0089] Table 2: Examples of PDPC weights based on prediction patterns

[0090] Predictive patterns wT wL wTL diagonal top right 16>>((y’<<1)>>shift) 16>>((x’<<1)>>shift) 0 diagonal bottom left 16>>((y’<<1)>>shift) 16>>((x’<<1)>>shift) 0 adjacent diagonal top right 32>>((y’<<1)>>shift) 0 0 The adjacent diagonal bottom left 0 32>>((x’<<1)>>shift) 0

[0091] 2.5. Intra-frame sub-block segmentation (ISP)

[0092] In some embodiments, an ISP is proposed, which divides the luminance intra-frame prediction block vertically or horizontally into 2 sub-segments or 4 sub-segments based on the block size dimension, as shown in Table 3. Figure 6 and Figure 7 Examples of two possibilities are shown. All sub-segments satisfy the condition of having at least 16 samples.

[0093] Table 3: The number of sub-segments depends on the block size.

[0094] Block size Number of sub-segments 4×4 Undivided 4×8 and 8×4 2 All other cases 4

[0095] For each of these sub-segments, a residual signal is generated by entropy decoding of the coefficients transmitted by the encoder, followed by inverse quantization and inverse transform. The sub-segment is then intra-predicted, and the corresponding reconstructed samples are finally obtained by adding the residual signal to the predicted signal. Thus, the reconstructed values ​​of each sub-segment can be used to generate the next prediction, and this process is repeated. All sub-segments share the same intra-frame mode.

[0096] Based on the intra-frame mode and the partitions used, two different types of processing orders are employed, referred to as the normal order and the reverse order. In the normal order, the first sub-segment to be processed is the one containing the top-left sample of the CU, then it continues downwards (horizontal partitioning) or to the right (vertical partitioning). As a result, the reference samples used to generate the sub-segment prediction signal are located only to the left and above these lines. On the other hand, the reverse processing order starts with the sub-segment containing the bottom-left sample of the CU and continues upwards, or starts with the sub-segment containing the top-right sample of the CU and continues to the left.

[0097] 2.6. Multiple Transform Set (MTS)

[0098] In addition to DCT-II, which is already used in HEVC, the Multiple Transform Selection (MTS) scheme is used for residual coding and decoding of both inter-frame and intra-frame codec blocks. It uses multiple transforms selected from DCT8 / DST7. The newly introduced transform matrices are DST-VII and DCT-VIII. Table 4 shows the basis functions of the selected DST / DCTs.

[0099] Table 4: Transformation Types and Basis Functions

[0100]

[0101] There are two ways to enable MTS: explicit MTS and implicit MTS.

[0102] 2.6.1. Implicit MTS

[0103] Implicit MTS is a new tool in VVC. The derivation of the variable implicitMtsEnabled is as follows:

[0104] Whether implicit MTS is enabled depends on the value of the variable implicitMtsEnabled. The derivation of the variable implicitMtsEnabled is as follows:

[0105] – If sps_mts_enabled_flag equals 1, and one or more of the following conditions are true, then implicitMtsEnabled is set to equal to 1:

[0106] –IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT (i.e., ISP is enabled).

[0107] –cu_sbt_flag equals 1 (i.e., ISP is enabled), and Max(nTbW, nTbH) is less than or equal to 32.

[0108] –sps_explicit_mts_intra_enabled_flag equals 0 (i.e., explicit MTS is disabled), CuPredMode[0][xTbY][yTbY] equals MODE_INTR A, and lfnst_idx[x0][y0] equals 0, and intra_mip_flag[x0][y0] equals 0.

[0109] Otherwise, implicitMtsEnabled is set to 0.

[0110] The derivation of the variable trTypeHor, which defines the horizontal transform kernel, and the variable trTypeVer, which defines the vertical transform kernel, is as follows:

[0111] – Set trTypeHor and trTypeVer to 0 if one or more of the following conditions are true (e.g., DCT2).

[0112] –cIdx is greater than 0 (i.e., for the chromaticity component).

[0113] –IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT, lfnst_idx is not equal to 0

[0114] Otherwise, if implicitMtsEnabled equals 1, the following applies:

[0115] – If cu_sbt_flag equals 1, then trTypeHor and trTypeVer are specified in Table 40 according to cu_sbt_horizontal_flag and cu_sbt_pos_flag.

[0116] Otherwise (cu_sbt_flag equals 0), the derivation of trTypeHor and trTypeVer is as follows:

[0117] trTypeHor=(nTbW>=4&&nTbW<=16)? 1:0 (1188)

[0118] trTypeVer=(nTbH>=4&&nTbH<=16)? 1:0 (1189)

[0119] Otherwise, trTypeHor and trTypeVer are specified in Table 39 according to mts_idx.

[0120] The derivation of variables nonZeroW and nonZeroH is as follows:

[0121] – If ApplyLfnstFlag equals 1, nTbW is greater than or equal to 4, and nTbH is greater than or equal to 4, then the following conditions apply:

[0122] nonZeroW=(nTbW==4||nTbH==4)? 4:8 (1190)

[0123] nonZeroH=(nTbW==4||nTbH==4)? 4:8 (1191)

[0124] Otherwise, the following applies:

[0125] nonZeroW=Min(nTbW,(trTypeHor>0)?16:32) (1192)

[0126] nonZeroH=Min(nTbH,(trTypeVer>0)?16:32) (1193)

[0127] 2.6.2. Explicit MTS

[0128] To control the MTS scheme, a flag is used to specify whether explicit MTS exists in the bitstream for intra / inter-frame use. Additionally, two separate enable flags are specified at the SPS level for intra and inter-frame use to indicate whether explicit MTS is enabled. When MTS is enabled at SPS, the CU-level transform index can be signaled to indicate whether MTS should be applied. Here, MTS applies only to luma. The MTS CU-level index (represented by mts_idx) is signaled when the following conditions are met.

[0129] - Width and height are both less than or equal to 32

[0130] -CBF brightness mark equals one

[0131] -Non-TS

[0132] -Non-ISP

[0133] -Non-SBT

[0134] -LFNST is disabled

[0135] - There exists a non-zero coefficient that is not at the DC position (top left of the block).

[0136] - There are no non-zero coefficients outside the 16×16 region at the top left.

[0137] If the first bit of `mts_idx` is zero, DCT2 applies in both directions. However, if the first bit of `mts_idx` is one, additional signaling informs the other two bits to indicate the transform type in the horizontal and vertical directions, respectively. The transform and signaling mapping table is shown in Table 5. For transform matrix precision, an 8-bit master transform kernel is used. Therefore, all transform kernels used in HEVC remain unchanged, including 4-point DCT-2 and DST-7, 8-point DCT-2, 16-point DCT-2, and 32-point DCT-2. Furthermore, other transform kernels, including 64-point DCT-2, 4-point DCT-8, 8-point, 16-point, 32-point DST-7, and DCT-8, use an 8-bit master transform kernel.

[0138] Table 5: Signaling Notifications of MTS

[0139]

[0140] To reduce the complexity of large-sized DST-7 and DCT-8 blocks, the high-frequency transform coefficients are set to zero for DST-7 and DCT-8 blocks with a size (width or height, or both) equal to 32. Only the coefficients in the 16×16 low-frequency region are retained.

[0141] In HEVC, for example, block residuals can be encoded and decoded using transform skip mode. To avoid redundancy in syntax encoding and decoding, the transform skip flag is not signaled when the CU level MTS_CU_flag is not equal to zero. The block size limit for transform skip is the same as the block size limit for MTS in JEM4, which indicates that transform skip applies to the CU when both the block width and block height are equal to or less than 32.

[0142] 2.6.3. Zeroing in MTS

[0143] In VTM8, large block size transforms up to 64×64 are enabled, primarily for higher resolution video, such as 1080p and 4K sequences. For transform blocks with a size (width or height, or both) of 64 or more, the high-frequency transform coefficients of the block to which DCT2 transform is applied are set to zero, thus retaining only the low-frequency coefficients, and all other coefficients are forced to zero without signaling notification. For example, for an M×N transform block, where M is the block width and N is the block height, when M is not less than 64, only the left 32 columns of transform coefficients are retained. Similarly, when N is not less than 64, only the first 32 rows of transform coefficients are retained.

[0144] For transform blocks with dimensions (width or height, or both) not less than 32, the high-frequency transform coefficients of blocks to which DCT8 or DST7 transform is applied are set to zero, thus retaining only the low-frequency coefficients, while all other coefficients are forced to zero without being notified. For example, for an M×N transform block, where M is the block width and N is the block height, when M is not less than 32, only the left 16 columns of transform coefficients are retained. Similarly, when N is not less than 32, only the first 16 rows of transform coefficients are retained.

[0145] 2.7. Low-frequency non-separable secondary transform (LFNST)

[0146] 2.7.1. JEM Non-Separable Secondary Transform (NSST)

[0147] In JEM, a quadratic transform is applied between the forward master transform and quantization (at the encoder) and between the dequantization and inverse master transform (at the decoder). For example... Figure 8 As shown, a 4×4 (or 8×8) quadratic transformation is performed based on the block size. For example, for each 8×8 block, the 4×4 quadratic transformation is applied to the smaller block (e.g., min(width, height) < 8), and the 8×8 quadratic transformation is applied to the larger block (e.g., min(width, height) > 4).

[0148] The application of the non-separable transform is described below using an input as an example. To apply the non-separable transform, a 4x4 input block X

[0149]

[0150] is first represented as a vector

[0151]

[0152] The non-separable transform is computed as where denotes the transform coefficient vector, and T is a 16x16 transform matrix. Subsequently, the 16x1 coefficient vector is reorganized into a 4x4 block using the scan order (horizontal, vertical, or diagonal) of the block. Coefficients with smaller indices are placed in the 4x4 coefficient block together with smaller scan indices. There are a total of 35 transform sets, and each transform set uses 3 non-separable transform matrices (kernels). The mapping from the intra prediction mode to the transform set is predefined. For each transform set, the selected non-separable quadratic transform candidate is further specified by a quadratic transform index signaled explicitly. After the transform coefficients, this index is signaled once per frame per CU in the bitstream.

[0153] 2.7.2. Reduced Secondary Transform (LFNST)

[0154] In some embodiments, LFNST is introduced and a mapping using 4 transform sets (instead of 35 transform sets) is used. In some implementations, 16×64 (which can be further reduced to 16×48) matrices and 16×16 matrices are used for 8×8 blocks and 4×4 blocks respectively. For ease of notation, the 16×64 (which can be further reduced to 16×48) transform is denoted as LFNST8×8 and the 16×16 transform is denoted as LFNST4×4. Figure 9 An example of LFNST is shown.

[0155] LFNST calculation

[0156] The main idea of the reduced transform (RT) is to map an N-dimensional vector to an R-dimensional vector in a different space, where R / N (R < N) is the reduction factor.

[0157] The RT matrix is an R×N matrix as follows:

[0158]

[0159] Here, the R rows of the transformation form R bases in N-dimensional space. The inverse transformation matrix of RT is the transpose of its forward transformation. The forward and inverse RT are as follows: Figure 10A and Figure 10B The description.

[0160] In this proposal, a reduction factor of 4 (1 / 4 size) is applied to LFNST 8×8. Therefore, instead of 64×64, a 16×64 direct matrix is ​​used, which is the traditional size of an 8×8 inseparable transform matrix. In other words, a 64×16 inverse LFNST matrix is ​​used on the decoder side to generate the core (first) transform coefficients in the top-left region of the 8×8. Positive LFNST 8×8 uses a 16×64 (or 8×64 for 8×8 blocks) matrix such that it produces non-zero coefficients only in the top-left 4×4 region of a given 8×8 region. In other words, if LFNST is applied, the 8×8 region outside the top-left 4×4 region will only have zero coefficients. For LFNST 4×4, 16×16 (or 8×16 for 4×4 blocks) direct matrix multiplication is applied.

[0161] The inverse LFNST is conditionally applied when the following two conditions are met:

[0162] a. Block size is greater than or equal to a given threshold (W>=4 && H>=4)

[0163] b. The transition skip mode flag is equal to zero.

[0164] If both the width (W) and height (H) of the transform coefficient block are greater than 4, then LFNST 8x8 is applied to the top-left 8×8 region of the transform coefficient block. Otherwise, LFNST 4x4 is applied to the top-left min(8,W)×min(8,H) region of the transform coefficient block.

[0165] If the LFNST index is equal to 0, then LFNST is not applied. Otherwise, LFNST is applied, and its core is selected along with the LFNST index. The LFNST selection method and the encoding / decoding of the LFNST index will be explained later.

[0166] In addition, LFNST is applied to intra-frame CUs in intra-frame and inter-frame stripes, as well as luma and chroma. If dual-tree is enabled, the LFNST indexes for luma and chroma are signaled separately. For inter-frame stripes (where dual-tree is disabled), a single LFNST index is signaled and used for luma and chroma.

[0167] At the 13th JVET conference, Intra-Frame Sub-Segmentation (ISP) was adopted as a new intra-frame prediction mode. When ISP mode is selected, LFNST is disabled, and the LFNST index is not signaled because the performance improvement is limited even if LFNST is applied to every feasible segmentation block. Furthermore, disabling LFNST on the residuals of ISP predictions reduces coding complexity.

[0168] LFNST selection

[0169] The LFNST matrix is ​​selected from four transform sets, each consisting of two transforms. Which transform set is applied is determined by the intra-prediction mode, as follows:

[0170] 1) If one of the three CCLM modes is indicated, then select transform set 0.

[0171] 2) Otherwise, perform the transformation set selection according to Table 6.

[0172] Table 6: Transform Set Selection Table

[0173] IntraPredMode Tr. set index IntraPredMode<0 1 0<=IntraPredMode<=1 0 2<=IntraPredMode<=12 1 13<=IntraPredMode<=23 2 24<=IntraPredMode<=44 3 45<=IntraPredMode<=55 2 56<=IntraPredMode 1

[0174] The index of the access table, denoted as IntraPredMode, ranges from [-14, 83], and is the transform mode index used for wide-angle intra-frame prediction.

[0175] reduce LFNST matrix of dimension

[0176] For further simplification, a 16×48 matrix is ​​used instead of a 16×64 matrix with the same transformation set configuration. Each matrix obtains 48 input data points from three 4×4 blocks, excluding the bottom right 4×4 block from the top left 8×8 block. Figure 11 ).

[0177] LFNST signaling

[0178] A positive LFNST 8×8 with R=16 uses a 16×64 matrix, therefore it produces non-zero coefficients only in the top-left 4×4 region of a given 8×8 region. In other words, if LFNST is applied, the 8×8 region produces only zero coefficients except for the top-left 4×4 region. Therefore, when the top-left 4×4 region is excluded (e.g., ... Figure 12 When any non-zero element is detected in an 8×8 block region outside of the one shown in the diagram, the LFNST index is not encoded or decoded because this means that no LFNST has been applied. In this case, the LFNST index is inferred to be zero.

[0179] Zeroing range

[0180] Normally, any coefficient in a 4×4 subblock can be nonzero before applying the inverse LFNST to it. However, in some cases, there are constraints that some coefficients in the 4×4 subblock must be zero before applying the inverse LFNST to it.

[0181] Let nonZeroSize be a variable. Any coefficient with an index not less than nonZeroSize must be zero when rearranging it into a 1-D array before inverting LFNST.

[0182] When nonZeroSize equals 16, the coefficients in the top left 4×4 sub-block have no zeroing constraint.

[0183] In some examples, nonZeroSize is set to 8 when the current block size is 4×4 or 8×8. For other block sizes, nonZeroSize is set to 16.

[0184] 2.8. Affine linear weighted intra prediction (ALWIP, also known as matrix-based intra prediction)

[0185] In some embodiments, alpha-based weighted intra prediction (ALWIP, also known as matrix-based intra prediction (MIP)) is used.

[0186] In some embodiments, two tests are performed. In Test 1, ALWIP is designed with an 8KB memory limit and a maximum of 4 multiplications per sample. Test 2 is similar to Test 1, but with a further simplified design in terms of memory requirements and model architecture.

[0187] • A single set of matrices and offset vectors for all block shapes.

[0188] • The number of patterns for all block shapes has been reduced to 19.

[0189] • Reduce memory requirements to 5760 10-bit values, or 7.20 kilobytes.

[0190] • Linear interpolation of the predicted samples is performed in a single step in each direction, instead of iterative interpolation in the first test.

[0191] 2.9. Sub-block Transformation

[0192] For an inter-frame prediction CU with a cu_cbf equal to 1, the cu_sbt_flag can be signaled to indicate whether to decode the entire residual block or a sub-part of the residual block. In the former case, the inter-frame MTS information is further parsed to determine the transform type of the CU. In the latter case, a portion of the residual block is encoded and decoded using the inferred adaptive transform, and the other portion of the residual block is set to zero. SBT is not applied to combined inter-frame and intra-frame modes.

[0193] In the sub-block transformation, a position-dependent transformation is applied to the luma transform blocks in SBT-V and SBT-H (chroma TB always uses DCT-2). The two positions in SBT-H and SBT-V are associated with different kernel transforms. More specifically, the horizontal and vertical transforms at each SBT position are... Figure 13 The rules specify that, for example, the horizontal and vertical transformations for SBT-V position 0 are DCT-8 and DST-7, respectively. When one side of the residual TU is greater than 32, the corresponding transformation is set to DCT-2. Therefore, the sub-block transformation joint specifies TU tiling, cbf, and the horizontal and vertical transformations of the residual block, which can be considered a syntax shortcut for cases where the main residual of the block is on one side of the block.

[0194] 2.10. Scan Region-Based Coefficient Encoding / Decoding (SRCC)

[0195] SRCC has been adopted by AVS-3. Regarding SRCC, such as... Figures 14A to 14B The lower right position (SRx, SRy) shown in the diagram is signaled, and only the coefficients within the rectangle with its four corners (0, 0), (SRx, 0), (0, SRy), and (SRx, SRy) are scanned and signaled. All coefficients outside the rectangle are zero.

[0196] 2.11. Implicit Selection of Transform (IST)

[0197] As disclosed in PCT / CN2019 / 090261 (included herein by reference), an implicit choice of the transformation solution is given, wherein the choice of the transformation matrix (DCT2 for horizontal and vertical transformations, or DST7 for both) is determined by the parity of the non-zero coefficients in the transformation block.

[0198] The proposed method is applied to the luminance component of intra-frame encoded blocks, excluding those encoded using DT, and allows block sizes from 4×4 to 32×32. The transform type is hidden in the transform coefficients. Specifically, the parity of the number of valid coefficients (e.g., non-zero coefficients) in a block is used to indicate the transform type. Odd numbers indicate the application of DST-VII, and even numbers indicate the application of DCT-II.

[0199] To eliminate the 32-point DST-7 introduced by IST, it is proposed that the use of IST be limited based on the remaining scan area when using SRCC. For example... Figures 15A to 15B As shown, IST is not allowed when the x-coordinate or y-coordinate of the lower right position in the remaining scan area is not less than 16. That is to say, in this case, DCT-II is applied directly.

[0200] In another scenario, when using run-length coefficient encoding / decoding, each non-zero coefficient needs to be checked. IST is not allowed when the x-coordinate or y-coordinate of a non-zero coefficient position is not less than 16.

[0201] The corresponding grammatical changes are indicated by bold, italic, and underlined text, as shown below:

[0202]

[0203]

[0204]

[0205] 2.12. SBT in AVS3

[0206] The corresponding syntax changes are highlighted below (bold and italic):

[0207]

[0208]

[0209] 7.2.6 Encoding / Decoding Unit

[0210] Sub-block transformation flag sbt_cu_flag

[0211] A binary variable. See section 8.3 for the analysis process. The value of SbtCuFlag is equal to the value of sbt_cu_flag. If sbt_cu_flag does not exist in the bitstream, the value of SbtCuFlag is 0.

[0212] Sub-block conversion quarter-size flag sbt_quad_flag

[0213] A binary variable. See section 8.3 for the analysis process. The value of SbtQuadFlag is equal to the value of sbt_quad_flag. If sbt_quad_flag does not exist in the bitstream, the value of SbtQuadFlag is 0.

[0214] Sub-block transformation direction flag sbt_dir_flag

[0215] Binary variables. See section 8.3 for the analysis process. The value of SbtDirFlag is equal to the value of sbt_dir_flag. If sbt_dir_flag does not exist in the bitstream, when SbtQuadFlag is 1, the value of SbtDirFlag is equal to the value of SbtHorQuad; when SbtQuadFlag is 0, the value of SbtDirFlag is equal to the value of SbtHorHalf.

[0216] Sub-block transformation position flag sbt_pos_flag

[0217] A binary variable. See section 8.3 for the analysis process. The value of SbtPosFlag is equal to the value of sbt_pos_flag. If sbt_pos_flag does not exist in the bitstream, the value of SbtPosFlag is 0.

[0218] 3. Examples of technical problems solved by publicly available technical solutions

[0219] The current designs of IST and MTS have the following problems:

[0220] 1. In VVC, the TS mode is signaled at the block level. However, while DCT2 and DST7 work well for residual blocks in camera-captured sequences, the Transition Skip (TS) mode is used more frequently for video with screen content compared to DST7. Further research is needed on how to more effectively determine the use of the TS mode.

[0221] 2. In VVC, the maximum allowed TS block size is set to 32×32. How to support large TS blocks requires further investigation.

[0222] 4. Example technologies and implementation examples

[0223] The items listed below should be considered as examples for explaining general concepts. These items should not be interpreted in a narrow way. Furthermore, these items can be combined in any way.

[0224] min(x, y) yields the smaller of x and y.

[0225] Implicit determination of transformation skip mode / identity transformation

[0226] A method is proposed to determine whether to apply a horizontal and / or vertical identity transform (IT) (e.g., transform skip mode) to the current first block based on the decoding coefficients of one or more representative blocks. This method is called "implicit determination of IT". When both the horizontal and vertical transforms are IT, the transform skip (TS) mode is applied to the current first block.

[0227] A “block” can be a transform unit (TU) / prediction unit (PU) / encoder / decoder unit (CU) / transform block (TB) / prediction block (PB) / encoder / decoder block (CB). A TU / PU / CU can include one or more color components, such as a luma-only component for a two-tree segment, where the currently encoded color component is luma; and two chroma components for a two-tree segment, where the currently encoded color component is chroma; or three color components for a single-tree case.

[0228] 1. Decoding coefficients can be associated with one or more representative blocks of the same or different color components of the current first block.

[0229] a. In one example, the representative block is the first block, and the decoding coefficients associated with the first block are used to determine the use of IT in the first block.

[0230] b. In one example, the determination of which IT is used for the first block may depend on the decoding coefficients of multiple blocks, including at least one block different from the first block.

[0231] i. In one example, multiple blocks may include the first block.

[0232] ii. In one example, multiple blocks may include one or more blocks that are adjacent to the first block.

[0233] iii. In one example, multiple blocks may include one or more blocks having the same block dimension as the first block.

[0234] iv. In one example, multiple blocks may include the last N decoded blocks that precede the first block in decoding order and satisfy certain conditions (such as having the same prediction mode as the current block, e.g., all intra-frame codecs or IBC codecs, or having the same dimensions as the current block). N is an integer greater than 1.

[0235] 1) The same prediction mode can be an inter-frame encoding / decoding mode.

[0236] 2) The same prediction mode can be the direct encoding / decoding mode.

[0237] v. In one example, multiple blocks may include one or more blocks that have a different color component than the first block.

[0238] 1) In one example, the first block can be in the luminance component. Multiple blocks can include blocks in the chrominance components (e.g., a second block in the Cb / B component and a third block in the Cr / R component).

[0239] a) In one example, the three blocks are in the same codec unit.

[0240] b) In addition, optionally, implicit MTS is applied only to the luma block and not to the chroma block.

[0241] 2) In one example, the first block in the first color component and the multiple blocks included in the multiple blocks that are not in the first color component can be in corresponding or juxtaposed positions in the image.

[0242] 2. The decoding coefficients used to determine the use of IT are called representative coefficients.

[0243] a. In one example, the representative coefficients only include coefficients that are not equal to zero (referred to as effective coefficients).

[0244] b. In one example, the representativeness coefficient can be modified before it is used to determine the use of IT.

[0245] i. For example, representativeness coefficients can be calibrated before being used to derive the transform.

[0246] ii. For example, representativeness coefficients can be scaled before being used to derive the transformation.

[0247] iii. For example, the representativeness coefficient can be offset before it is used to derive the transformation.

[0248] iv. For example, representativeness coefficients can be filtered before being used to derive the transform.

[0249] v. For example, coefficients or representative coefficients can be mapped to other values ​​(e.g., by lookup tables or dequantization) before being used to derive a transformation.

[0250] c. In one example, the representative coefficients are all the significant coefficients in the representative block.

[0251] d. Alternatively, the representativeness coefficient is a portion of the effective coefficients in the representative block.

[0252] i. In one example, the representative coefficients are those odd-numbered valid decoding coefficients.

[0253] 1) Optionally, the representative coefficients are those even-numbered valid decoding coefficients.

[0254] ii. In one example, the representative coefficients are those valid decoding coefficients that are greater than or not less than the threshold.

[0255] 1) Optionally, representative coefficients are those effective decoding coefficients whose amplitude is greater than or not less than a threshold.

[0256] iii. In one example, the representative coefficients are those valid decoding coefficients that are less than or no greater than the threshold.

[0257] 1) Optionally, representative coefficients are those effective decoding coefficients whose amplitude is less than or not greater than the threshold.

[0258] iv. In one example, the representative coefficients are the first K (K>=1) valid decoded coefficients in the decoding order.

[0259] v. In one example, the representative coefficients are the last K (K>=1) valid decoded coefficients in the decoding order.

[0260] vi. In one example, the representativeness coefficient can be the coefficient at a predefined location within the block.

[0261] 1) In one example, the representativeness coefficient may include only one coefficient relative to the representative block at the coordinates (xPos, yPos). For example, xPos = yPos = 0.

[0262] 2) In one example, the representativeness coefficient may include only one coefficient relative to the representative block at coordinates (xPos, yPos). And xpo and / or ypo satisfy the following condition:

[0263] a) In one example, xPos is not greater than the threshold Tx (e.g., 31) and / or yPos is not greater than the threshold Ty (e.g., 31).

[0264] b) In one example, xPos is not less than the threshold Tx (e.g., 32) and / or yPos is not less than the threshold Ty (e.g., 32).

[0265] 3) For example, the location can depend on the dimensions of the block.

[0266] 4) In one example, the representativeness coefficients may include only the coefficients located at (xPos, yPos)(multiple) coordinates relative to the representative block. And xPos and / or yPos satisfy the following condition:

[0267] a) In one example, xPos is not greater than the threshold Tx (e.g., 31) and / or yPos is not greater than the threshold Ty (e.g., 31).

[0268] b) In one example, xPos is not less than the threshold Tx (e.g., 32) and / or yPos is not less than the threshold Ty (e.g., 32).

[0269] 5) In one example, the representative coefficients may include only the coefficients located at (xPos, yPos)(multiple) coordinates relative to the representative block. And xPos and / or yPos can be derived from the syntax elements indicating the allowed sub-blocks with non-zero coefficients.

[0270] a) In one example, the syntax element is the same as the syntax element used to indicate the pattern of subblock transformation (e.g., SBT, such as SbtCuFlag, SbtQuadFlag, SbtDirFlag, SbtPosFlag, or position-based transformation (pbt)).

[0271] vii. In one example, representative coefficients can be those coefficients at predefined positions in the coefficient scan order.

[0272] e. Alternatively, the representativeness coefficient may also include those with zero coefficients.

[0273] f. Alternatively, the representative coefficients may be coefficients derived from the decoded coefficients, such as by limiting to a range, or by quantization.

[0274] g. In one example, the representative coefficient can be the coefficient preceding the last effective coefficient (which may include the last effective coefficient).

[0275] 3. The determination of whether to use IT for the first block may depend on the decoding luminance coefficient of the first block.

[0276] a. In addition, optionally, the use of a specific IT is applied only to the luminance component of the first block, while DCT2 is always used for the chrominance component of the first block.

[0277] b. Alternatively, the determined IT is applied to all color components of the first block. That is, the same transformation matrix is ​​applied to all color components of the first block.

[0278] 4. The determination of the use of IT can depend on a function of representativeness coefficients, such as a function that uses representativeness coefficients as input and value V as output.

[0279] a. In one example, V is derived as the number of representative coefficients.

[0280] i. Optionally, V is derived as the sum of representative coefficients.

[0281] 1) Optionally, V is derived as the sum of the levels (or absolute values) of the representative coefficients.

[0282] 2) Optionally, V can be derived as a level (or absolute value) of a representative coefficient (such as the last one).

[0283] 3) Optionally, V can be derived as the number of representative coefficients of even levels.

[0284] 4) Optionally, V can be derived as the number of representative coefficients of odd level.

[0285] 5) In addition, optionally, the sum can be limited to derive V.

[0286] ii. Alternatively, V is derived as the output of a function, where the function defines the residual energy distribution.

[0287] 1) In one example, the function returns the ratio of the sum of the absolute values ​​of the partial representative coefficients to the absolute values ​​of all representative coefficients.

[0288] 2) In one example, the function returns the ratio of the sum of squares of the absolute values ​​of the partial representative coefficients to the sum of squares of the absolute values ​​of all representative coefficients.

[0289] 3) In one example, the function returns whether the energy of the top K representative coefficients multiplied by the scaling factor is greater than the energy of the top M (M>K) or all representative coefficients.

[0290] 4) In one example, the function returns whether the energy of the representative coefficient in the first subregion of the representative block multiplied by the scaling factor is greater than the energy of the representative coefficient in the second subregion that contains the first subregion and is greater than the first subregion.

[0291] a) Alternatively, the function returns whether the energy of the representative coefficient in the first subregion of the representative block multiplied by the scaling factor is greater than the energy of the representative coefficient in the second subregion that does not overlap with the first subregion.

[0292] b) In one example, the first sub-region is the top-left MxN sub-region (i.e., M=N=1, only DC).

[0293] i. Alternatively, the first sub-region is the second sub-region excluding the top-left MxK sub-region (i.e., excluding DC).

[0294] c) In one example, the first sub-region is the top left 4x4 sub-region.

[0295] 5) In the above example, energy is defined as the sum of absolute values ​​or the sum of squares of values.

[0296] iii. Alternatively, V is derived as whether at least one representative coefficient is located outside a subregion of the representative block.

[0297] 1) In one example, a subregion is defined as the top left subregion of the representative block, for example, the top left quarter of the representative block.

[0298] b. In one example, the determination of the use of IT may depend on the parity of V.

[0299] i. For example, if V is even, then IT is used; but if V is odd, then IT is not used.

[0300] 1) Optionally, if V is even, use IT; if V is odd, do not use IT.

[0301] ii. In one example, if V is less than threshold T1, then IT is used; but if V is greater than threshold T2, then IT is not used.

[0302] 1) Optionally, if V is greater than threshold T1, then IT is used; if V is less than threshold T2, then IT is not used.

[0303] iii. For example, the threshold can depend on encoding / decoding information, such as block dimension and prediction mode.

[0304] iv. For example, the threshold can depend on QP.

[0305] c. In one example, the determination of the use of IT may depend on a combination of V and other codec information (e.g., prediction mode, strip type / picture type, block dimension).

[0306] 5. The determination of the use of IT can further depend on the encoding and decoding information of the current block.

[0307] a. In one example, the determination may also depend on mode information (e.g., inter-frame, intra-frame, or IBC).

[0308] i. In one example, the mode could be an inter-frame encoding / decoding mode.

[0309] ii. In one example, the mode could be direct encoding / decoding mode.

[0310] iii. In one example, the current block can be encoded or decoded using SBT mode (e.g., sbt_cu_flag equals 1).

[0311] 1) In addition, whether to use the current transform set (e.g., DCT2 / DST7 / DCT8) or use IT may depend on the representativeness coefficient.

[0312] a) In one example, if it is determined that IT is not used, DCT2 / DST7 / DCT8 can be used and it is determined which transformation to use can follow prior techniques (e.g., described in sections 2.12, 2.9).

[0313] iv. In one example, the determination of the use of IT may depend on the syntax elements of the signaling notification at a higher level, such as in the picture header and / or sequence header and / or SPS / PPS / VPS / strip header.

[0314] b. In one example, whether IT is applied may depend on the representativeness coefficient of the inter-frame codec block, which is not direct and does not use SBT.

[0315] c. In one example, if SBT is used, IT can be applied to inter-frame codec blocks.

[0316] d. The first message (denoted as ph_ts_inter_enable_flag) is signaled in the picture header. Inter-frame codec blocks can only apply IT if ph_ts_inter_enable_flag is true. Otherwise, the default conversion (e.g., DCT2) will be used.

[0317] i. The second message (denoted as ts_inter_enable_flag) is signaled in the sequence header. ph_ts_inter_enable_flag is signaled only if ts_inter_enable_flag is true. Otherwise, ph_ts_inter_enable_flag is not signaled and is inferred to be false.

[0318] e. In one example, the transformation determination may depend on the scan area, which is the smallest rectangle covering all valid coefficients (e.g., as depicted in Figure 14).

[0319] i. In one example, if the size of the scan region associated with the current block (e.g., width multiplied by height) is greater than a given threshold, a default transformation (such as DCT-2) can be used, including both horizontal and vertical transformations. Otherwise, rules such as those defined in bullet point 3 can be used (e.g., IT when V is even and DCT-2 when V is odd).

[0320] ii. In one example, if the width of the scan region associated with the current block is greater than (or less than) a given maximum width (e.g., 16), then a default horizontal transformation (such as DCT-2) can be used. Otherwise, rules such as those defined in bullet point 3 can be used.

[0321] iii. In one example, if the height of the scan region associated with the current block is greater than (or less than) a given maximum height (e.g., 16), a default vertical transformation (such as DCT-2) can be utilized. Otherwise, rules such as those defined in bullet point 3 can be used.

[0322] iv. In one example, the given dimensions are L×K, where L and K are integers, such as 16.

[0323] v. In one example, the default transformation matrix can be either DCT-2 or DST-7.

[0324] f. In one example, the determination of IT usage can only be invoked if the conditions (e.g., pattern information) are met; otherwise, other transformations (excluding identity transformations) can be used instead.

[0325] 6. One or more of the methods disclosed in bullet points 1 through 5 can only be applied to a specific block.

[0326] a. For example, one or more of the methods disclosed in bullets 1 to 5 may only be applied to blocks of IBC encoding and / or intra-frame encoding and decoding other than DT.

[0327] i. In one example, one or more methods disclosed in bullets 1 through 5 may be applied to inter-frame codec blocks.

[0328] ii. In one example, one or more methods disclosed in bullets 1 through 5 may be applied to a direct codec block.

[0329] b. For example, one or more of the methods disclosed in bullet points 1 through 5 can only be applied to blocks with specific constraints on the coefficients.

[0330] i. A rectangle with four corners (0, 0), (CRx, 0), (0, CRy), and (CRx, CRy) is defined as a constrained rectangle, as in the SRCC method, for example. In one example, one or more of the methods disclosed in bullets 1 through 5 may be applied only if all coefficients outside the constrained rectangle are zero. For example, CRx = CRy = 16.

[0331] 1) For example, CRx = SRx and CRy = SRy, where (SRx, SRy) is defined in SRCC as described in Section 2.14.

[0332] 2) Alternatively, the above method may be applied only when the block width or block height is greater than K.

[0333] a) In one example, K equals 16.

[0334] b) In one example, the above method is applied only when the block width is greater than K1 and K1 equals CRx; or when the block height is greater than K2 and K2 equals CRy.

[0335] ii. One or more of the methods may be applied only if the last non-zero coefficients (in the forward scan order) meet certain conditions, such as when the horizontal / vertical coordinates are not greater than a threshold (e.g., 16 / 32).

[0336] 7. When it is determined that IT is not to be used, default transformations such as DCT-2 or DST-7 can be used instead.

[0337] a. Optionally, when it is determined that IT is not to be used, one can choose from several default transformations such as DCT-2 or DST-7.

[0338] 8. To determine the transformations from the transformation set to be applied to the block encoded in prediction mode A, representative coefficients (e.g., bullet 2) in one or more representative blocks (e.g., bullet 1) are used, and the transformation set may depend on prediction mode A and / or one or more syntax elements and / or other encoding and decoding information (e.g., the use of DT, block dimensions).

[0339] a. In one example, the transform set is {DCT2}.

[0340] b. In one example, the transformation set is {IT}.

[0341] c. In one example, the transform set includes {DCT2, IT}.

[0342] d. In one example, the transform set includes {DCT2, DST7}.

[0343] e. In one example, DCT2 is used when the number of representative coefficients is even. Otherwise (when the number of representative coefficients is odd), DST7 or IT is used, which can be determined by prediction mode A and / or one or more syntax elements and / or other codec information.

[0344] i. In one example, if the prediction pattern A is IBC, IT is always used when the number of representative coefficients is odd.

[0345] ii. In one example, if the prediction mode A is intra-frame and DT is applied, DCT2 is always used when the number of representative coefficients is odd.

[0346] iii. In one example, if prediction mode A is intra-frame and DT is not applied, and one or more syntax elements indicate the implicit determination of IT or the implicit determination of transform skip mode, IT is always used when the number of representative coefficients is odd.

[0347] 9. Whether and / or how the methods disclosed above can be applied to signaling notification at the video region level (such as sequence level / picture level / strip level / group level / piece level / subpicture level).

[0348] a. In one example, signaling notifications (e.g., flags) can be found in the sequence header / picture header / SPS / VPS / DCI / DPS / PPS / APS / strip header / piece group header.

[0349] i. Additionally, alternatively, one or more syntax elements (e.g., one or more flags) may be signaled to specify whether the implicit determination method of IT is enabled or the use of implicit determination of the transformation skip mode is enabled.

[0350] 1) In one example, a signaling notification first flag can be used to control the use of a method for implicitly determining the IT of an IBC codec block at the video region level.

[0351] a) Additionally, a signaling notification flag may be added if IBC is checked to see if it is enabled.

[0352] b) Alternatively, whether the use of an implicit determination method for the IT of an IBC codec block is enabled can be controlled by the same flag used to control the use of the implicit selection (IST) mode for the transformation of intra-codec blocks in Excluded Derivative Tree (DT) mode.

[0353] 2) In one example, a signaling notification of a second flag may be used to control the implicit determination of the IT of intra-frame codec blocks at the video region level (e.g., blocks with derivation tree (DT) mode may be excluded).

[0354] 3) In one example, a signaling notification third flag can be used to control the use of an implicit method for determining the IT of inter-frame codec blocks at the video region level.

[0355] 4) In one example, signaling notification flags can be used to control the implicit determination of the IT of intra-frame codec blocks (e.g., blocks with DT mode can be excluded) and inter-frame codec blocks at the video region level.

[0356] 5) In one example, signaling notification flags can be used to control the implicit determination of the IT of IBC codec blocks and intra-frame codec blocks at the video region level (e.g., blocks with DT mode can be excluded).

[0357] ii. Additionally, optionally, when an implicit method for determining the video region is enabled (e.g., a flag is true), the following can be further applied:

[0358] 1) In one example, for IBC codec blocks, if IT is used for the block, TS mode is applied; otherwise, DCT2 is used.

[0359] 2) In one example, for intra-frame encoded blocks (e.g., blocks with DT mode can be excluded), if IT is used for the block, then TS mode is applied; otherwise, DCT2 is used.

[0360] 3) In one example, for inter-frame codec blocks, if IT is used for the block, then TS mode is applied; otherwise, the following may apply:

[0361] a) In one example, DCT2 is used.

[0362] b) In one example, DCT2 / DST7 / DCT8 are used depending on the use of SBT.

[0363] 4) In one example, for direct codec blocks, if IT is used for the block, then TS mode is applied; otherwise, the following may apply.

[0364] a) In one example, DCT2 is used.

[0365] b) In one example, DCT2 / DST7 / DCT8 are used depending on the use of SBT.

[0366] iii. Additionally, alternatively, when the implicit determination method of IT is disabled for a video region (e.g., the flag is set to false), the following can be further applied;

[0367] 1) In one example, DCT-2 is used for IBC and / or intra-frame codec blocks.

[0368] 2) In one example, for intra-frame encoded blocks (e.g., excluding blocks with DT mode), DCT-2 or DST-7 can be determined on the fly, such as by IST.

[0369] b. Multi-level control that enables / applies IT methods can signal notifications at multiple video unit levels (e.g., sequence level, picture level, stripe level).

[0370] i. In one example, the first video unit level is defined as the sequence level.

[0371] 1) Additionally, optionally, the first syntax element (e.g., a flag) is signaled in the sequence header / SPS / to indicate the use of IT.

[0372] a) In one example, a first syntax element equal to 0 indicates that the Implicit Selection Transform Skip (ISTS) method cannot be used in a sequence. Otherwise (a first syntax element equal to 1) indicates that the Implicit Selection Transform Skip (ISTS) method can be used in a sequence.

[0373] b) Alternatively, the first syntax element may be conditionally signaled, such as according to "Implicit selection of transformation (IST) is enabled / ist_enable_flag equals 1".

[0374] c) Additionally, optionally, when the first syntax element is not present, it is inferred to be a default value. For example, IT is inferred to be disabled for the first video unit level.

[0375] ii. In one example, the second video unit level is defined as the image / strip level.

[0376] 1) Additionally, optionally, a second syntax element (e.g., a flag) is signaled in the picture header (e.g., intra-frame picture header and / or inter-frame picture header) / strip header to indicate the use of IT.

[0377] a) Additionally, optionally, the second syntax element may be conditionally signaled, such as according to “Implicit Selection of Transformation (IST) is enabled” or “ist_enable_flag equals 1” and / or “IT method is enabled at the first video unit level (e.g., sequence)”.

[0378] b) Additionally, or, if the second syntax element is not present, it is inferred to be the default value. For example, IT is inferred to be disabled for the second video unit level.

[0379] c. Syntax elements indicating the allowed transform sets can be signaled at the video unit level (e.g., picture), and can be conditionally signaled based on the implicit selection (IST) of whether transforms are enabled.

[0380] i. In one example, syntax elements (e.g., flags or indexes) are signaled in the picture header (e.g., intra-frame picture header and / or inter-frame picture header) / strip header.

[0381] 1) In addition, or, support N different allowed transformation sets, the choice of which depends on the syntax element.

[0382] a) In one example, N is set to 2.

[0383] b) In one example, the choice of transform set may depend on block information, such as the encoding / decoding mode of the CU, or whether the CU is encoded / decoded in (derivative tree) DT mode.

[0384] i. For example, DCT2 is always used for CUs with DT mode.

[0385] ii. For example, DCT2 is always used for chroma blocks.

[0386] c) In one example, the two sets are {DCT2,DST7} and {DCT2,IT}.

[0387] i. Alternatively, they are used for intra-frame codec blocks.

[0388] ii. Alternatively, they are used for intra-frame codec blocks that are not DT-coded.

[0389] d) In one example, the two sets are {DCT2} and {DCT2,IT}.

[0390] i. Alternatively, they are used for intra-block copy (IBC) codec blocks.

[0391] ii. Alternatively, they are used for intra-block copy (IBC) codec blocks that are not DT-coded.

[0392] e) In one example, the two sets are {DCT2} and {DCT2,IT}.

[0393] i. In addition, or, they are used for inter-frame codec blocks.

[0394] ii. Alternatively, they may be used for inter-frame codec blocks that are not DT-coded.

[0395] f) In one example, the two sets are {DCT2} and {DCT2,IT}.

[0396] i. In addition, or, they are used for direct encoding and decoding blocks.

[0397] ii. Alternatively, they are used for direct codec blocks that are not DT codecs.

[0398] 2) Alternatively, if it does not exist, it is inferred to be the default value.

[0399] For example, the allowed set of transformations is inferred to be that only one type of transformation is allowed (e.g., DCT2).

[0400] ii. In one example, syntax elements (e.g., flags or indexes) are used to control the selection of transformations from the allowed set of transformations for blocks with a particular encoding / decoding mode.

[0401] 1) In one example, it controls the selection of transforms from the allowed transform set for intra-frame codec blocks / intra-frame codec blocks, excluding blocks with applied DT and blocks with Pulse Codec Modulation (PCM).

[0402] The block of the pattern.

[0403] 2) Alternatively, for blocks with other encoding / decoding modes, the allowed set of transformations may be independent of the syntax elements.

[0404] a) In one example, for an inter-frame codec block, the allowed transform set is {DCT2}.

[0405] b) In one example, for an IBC codec block, the allowed transform set is {DCT2} or {DCT2, IT}, which may depend on whether IST or IBC is enabled for the current picture.

[0406] c) In one example, for an inter-frame codec block, the allowed transform set is {DCT2,IT}.

[0407] d) In one example, for a direct codec block, the allowed transform set is {DCT2, IT}.

[0408] e) In one example, how to select the set of allowed transformations may depend on, for example, whether IST / SBT is enabled.

[0409] d. Whether and / or how the methods disclosed in this document are applied can be determined by considering one or more messages of signaling notification at the video region level (e.g., sequence level / picture level / strip level / group level / piece level / subpicture level) and picture / strip type and / or CTU / CU / PU / TU mode.

[0410] i. For example, for the first CU mode (e.g., IBC or inter-frame), TS is applied if the signaling notification message at the video region level (e.g., picture level) indicates that TS is enabled. For the second CU mode (e.g., intra-frame), TS is applied if the signaling notification message at the video region level indicates that TS should be used and additional conditions are met (e.g., the parity of the coefficients meets certain requirements set forth in this document).

[0411] ii. For example, for a first mode (e.g., CU / TU / PU mode, e.g., DT codec), TS is applied if the signaling notification message at the video region level (e.g., picture level) indicates that TS is enabled. For a second mode (e.g., non-DT codec), TS is applied if the signaling notification message at the video region level indicates that TS should be used and additional conditions are met (e.g., the parity of the coefficients meets certain requirements set forth in this document).

[0412] iii. For example, for the first mode (e.g., CU / TU / TU mode) (e.g., SBT codec), TS is applied if the signaling notification message at the video region level (e.g., picture level) indicates that TS is enabled. For the second mode (e.g., non-SBT codec), TS is applied if the signaling notification message at the video region level indicates that TS should be used and additional conditions are met (e.g., the parity of the coefficients meets certain requirements set forth in this document).

[0413] 10. At the video region level, such as sequence level / picture level / strip level / group level / piece level / subpicture level, signaling indicates whether to apply a zeroing instruction to transform blocks (including identity transforms).

[0414] a. In one example, the indication (e.g., a flag) can be signaled in the sequence header / picture header / SPS / VPS / DCI / DPS / PPS / APS / strip header / piece group header.

[0415] b. In one example, when the instruction specifies that zeroing is enabled, only IT transformations are allowed.

[0416] c. In one example, when the instruction specifies that zeroing is disabled, only non-IT transformations are allowed.

[0417] d. Additionally, optionally, the allowed range of binary / context modeling / last valid coefficients / bottom-right position (e.g., the maximum X / Y coordinates relative to the top-left position of the block) in the SRCC can depend on this indication.

[0418] 11. The first rule (e.g., in bullet points 1 to 7 above) can be used to determine the use of IT in the first block, and the second rule can be used to determine the transformation type that does not include IT.

[0419] a. In one example, the first rule can be defined as the residual energy distribution.

[0420] b. In one example, the second rule can be defined as the parity of the representative coefficients.

[0421] Transformation skip

[0422] 12. Apply zeroing to IT (e.g., TS) codec blocks, where non-zero coefficients are restricted to specific sub-regions of the block.

[0423] a. In one example, the zeroing range of an IT (e.g., TS) codec block is set to the upper right K*L sub-region of the block, where K is set to min(T1, W) and L is set to min(T2, H), where W and H are the block width / block height respectively, and T1 / T2 are two thresholds.

[0424] i. In one example, T1 and / or T2 can be set to 32 or 16.

[0425] ii. Alternatively, the last non-zero coefficient should be located within the K*L subregion.

[0426] iii. Alternatively, the lower right position (SRx, SRy) in the SRCC method should be located within the K*L subregion.

[0427] 13. Multiple zeroing types are defined for IT (e.g., TS) codec blocks, where each type corresponds to a sub-region of the block, where non-zero coefficients exist only in that sub-region.

[0428] a. In one example, non-zero coefficients exist only in the top-left K0*L0 subregion of the block.

[0429] b. In one example, non-zero coefficients exist only in the upper right K1*L1 subregion of the block.

[0430] i. Alternatively, signaling may be used to indicate the lower left position of a sub-region with a non-zero coefficient.

[0431] c. In one example, non-zero coefficients exist only in the lower left K2*L2 subregion of the block.

[0432] i. Alternatively, signaling may be used to indicate the upper right position of a sub-region with a non-zero coefficient.

[0433] d. In one example, non-zero coefficients exist only in the lower right K3*L3 subregion of the block.

[0434] i. Alternatively, signaling may be used to indicate the upper left position of a sub-region with a non-zero coefficient.

[0435] e. In addition, alternatively, explicit signaling notifications or immediate export of IT zeroing type instructions may be provided.

[0436] 14. When at least one valid coefficient is outside the zeroing region defined by IT (e.g., TS), such as outside the top-left K0*L0 sub-region of the block, IT (e.g., TS) is not used in the block.

[0437] a. Alternatively, in this case, the default transformation can be used.

[0438] 15. Use IT (e.g., TS) in a block when at least one valid coefficient is outside the zeroing region defined by another transformation matrix (e.g., DST7 / DCT2 / DCT8), such as outside the top-left K0*L0 sub-region of the block.

[0439] a. Alternatively, in this case, the TS mode may be used for inference.

[0440] Figures 16A to 16D The various zeroing types of TS codec blocks are shown. Figure 16A The top-left K0*L0 sub-region is shown. Figure 16B The upper right K1*L1 sub-region is shown. Figure 16CThe lower left K2*L2 sub-region is shown. Figure 16D The lower right K3*L3 sub-region is shown.

[0441] General

[0442] 16. The transformation matrix can be determined at the CU / CB level or the TU level.

[0443] a. In one example, the decision is made at the CU level, where all TUs share the same transformation matrix.

[0444] i. Alternatively, when a CU is divided into multiple TUs, the coefficients in one TU (e.g., the first TU or the last TU) or some or all of the TUs can be used to determine the transformation matrix.

[0445] b. Whether to use a CU-level solution or a TU-level solution may depend on the block size and / or VPDU size and / or maximum CTU size and / or encoding / decoding information of a block.

[0446] i. In one example, when the block size is larger than the VPDU size, the CU level determination method can be applied.

[0447] 17. Whether and / or how the disclosed methods are applied may depend on encoding / decoding information, which may include:

[0448] a. Block dimension.

[0449] ii. In one example, the above method can be applied to blocks whose width and / or height are not greater than a threshold (e.g., 32).

[0450] iii. In one example, the above method can be applied to blocks whose width and / or height are not less than a threshold (e.g., 4).

[0451] iv. In one example, the above method can be applied to blocks whose width and / or height are less than a threshold (e.g., 64).

[0452] b.QP

[0453] c. Image or strip type (such as I-frame or P / B frame, I-strip or P / B strip)

[0454] i. In one example, the proposed method can be enabled for I-frames but disabled for P / B frames.

[0455] d. Structural segmentation methods (single-tree or dual-tree)

[0456] i. In one example, the above method can be applied to strips / images / tiles / pieces for which single-tree segmentation is applied.

[0457] e. Encoding / decoding modes (such as inter-frame mode / intra-frame mode / IBC mode, etc.).

[0458] i. In one example, the above method can be applied to intra-frame codec blocks.

[0459] ii. In one example, the above method can be applied to intra-frame codec blocks (excluding blocks with DT applied and blocks without PCM mode).

[0460] iii. In one example, the above method can be applied to intra-frame codec blocks (excluding blocks with DT applied and blocks without PCM mode) and IBC codec blocks.

[0461] f. Encoding and decoding methods (such as intra-frame sub-block segmentation, Derived Tree (DT) methods, etc.).

[0462] i. In one example, the above method can be disabled for intra-frame codec blocks that have applied DT.

[0463] ii. In one example, the above method can be disabled for intra-frame codec blocks that have applied ISP.

[0464] iii. Encoding and decoding methods may include Subblock Transform (SBT).

[0465] iv. Encoding and decoding methods may include position-based transform (PBT).

[0466] v. In one example, the above method can be applied to blocks that use SBT for IBC encoding / decoding or inter-frame encoding / decoding.

[0467] g. Color components

[0468] i. In one example, the above method can be applied to the luma block, but not to the chroma block.

[0469] h. Intra-frame prediction modes (such as DC, vertical, horizontal, etc.).

[0470] i. Motion information (such as MV and reference index).

[0471] j. Standard grade / level / hierarchy.

[0472] Coefficient Reordering

[0473] On the decoder side, coefficient reordering is defined as how to reorder or map the resolved coefficients before dequantization, or before inverse transform, or before reconstruction. These coefficients can be used to reconstruct samples with or without inverse transform.

[0474] 18. Coefficient reordering can be done at the CU level, TU level, or PU level.

[0475] 19. Whether / how coefficient reordering is applied may depend on the encoding / decoding information.

[0476] a. In one example, the encoding / decoding information may include the TU position of a TU-level partial sub-block.

[0477] b. In one example, the encoding / decoding information may include the transformation type.

[0478] i. In one example, the transformation type can be derived based on the coefficients presented in bullet points 1 through 17.

[0479] ii. Transformation types may include DCT-II, DST-VII, DCT-VIII, and TS (also known as IT).

[0480] iii. In one example, how the coefficients are reordered may depend on whether TS (also known as IT) is used.

[0481] 1) In one example, the analyzed coefficients are rearranged in reverse order before dequantization.

[0482] 2) In one example, the analyzed coefficients are rearranged in reverse order before being used for reconstruction.

[0483] 3) In one example, the analytic coefficients C(i,j) of i = 0, 1, ..., M-1 and j = 0, 1, ..., N-1 are rearranged into C'(i,j) in the reverse order of C'(i,j).

[0484] 20. Whether to determine the coefficient reordering based on the transformation type can depend on the strip / image / slice type.

[0485] a. In one example, the coefficient reordering is determined based on the transformation type of a specific strip / picture / slice type, such as I-strip or I-picture.

[0486] b. In one example, the coefficient reordering is not determined based on the transformation type of a specific strip / picture / slice type, such as P strips and / or B strips or P pictures and / or B pictures.

[0487] 21. Whether coefficient reordering is determined based on the transformation type can depend on the CTU / CU / block type.

[0488] a. In one example, the coefficient reordering is determined based on the transform type of a specific CTU / CU / block type (e.g., intra-frame codec block).

[0489] b. In one example, the coefficient reordering is not determined based on the transformation type of a specific CTU / CU / block type, or CTU / CU / block type (e.g., inter-frame codec block or IBC codec block).

[0490] c. In one example, the coefficient reordering is not determined based on the transform type of the DT codec block.

[0491] 22. Whether to determine the coefficient reordering based on the transformation type may depend on the message that can be signaled in VPS / SPS / PPS / sequence header / picture header / strip header / CTU / CU / block.

[0492] a. The message can be a flag.

[0493] b. The message can be conditionally signaled.

[0494] i. For example, if the CU-level block is not a zero residual block, only signaling notification messages can be used.

[0495] ii. For example, if a TU-level block or a portion of a sub-block is not a zero residual block, then only signaling notification messages are allowed.

[0496] iii. If the message is not signaled, it is assumed to be a default value, such as 0 or 1.

[0497] iv. For example, if TS is enabled, only signaling notification messages can be sent.

[0498] c. For example, the first message (denoted as ph_rts_enable_flag) is a signaling notification in the image header. The TS codec block can only reorder the coefficients if ph_rts_enable_flag is true.

[0499] i. The second message (denoted as rts_enable_flag) is signaled in the sequence header. ph_rts_enable_flag will only be signaled if rts_enable_flag is true. Otherwise, ph_rts_enable_flag will not be signaled and will be inferred as false.

[0500] 23. How coefficients are reordered may depend on the signaling notification messages that can be included in VPS / SPS / PPS / sequence header / picture header / strip header / CTU / CU / block.

[0501] a. The message can be a flag.

[0502] b. The message can be conditionally signaled.

[0503] i. If the message is not signaled, it is assumed to be a default value, such as 0 or 1.

[0504] ii. For example, if TS is enabled, only signaling notification messages can be sent.

[0505] iii. For example, signaling notification messages can only be sent for specific CTU / CU / block types, or CTU / CU / block types (e.g., inter-frame codec blocks or IBC codec blocks).

[0506] Partial residual blocks

[0507] 24. If the current block is IBC encoded, there may be no signaling notification indicating whether a position-based transformation message (represented as pbt_cu_flag) should be applied.

[0508] 25. A method was proposed to divide a block with a specific pattern into at least two parts, and at least one of the parts has no non-zero residuals (in other words, the residuals are set to zero without signaling notification). Such a block is called a Partial Residual Block (PRB).

[0509] a. In one example, the specific mode is IBC.

[0510] i. In one example, IT or DCT2 can be applied to PRB.

[0511] ii. In one example, IT is applied to PRB.

[0512] b. In one example, if the current block is IBC encoded / decoded, it indicates whether to use...

[0513] SBT messages (represented as sbt_cu_flag) can be signaled.

[0514] i. If sbt_cu_flag is true for an IBC codec block, then the block uses PRB codec. Furthermore, the segmentation method is the same as defined in the messages describing SBT segmentation (such as sbt_quad_flag, sbt_dir_flag, sbt_pos_flag).

[0515] c. In one example, the specific mode is inter-frame, and only IT (or one of IT / DCT2) is applied to the block.

[0516] d. In one example, the M×N block is divided into two parts:

[0517] i. The two parts can be (M / k)×N blocks A and (MM / k)×N blocks B, where k is an integer greater than 1, such as 2, 4, 8, 16.

[0518] ii. The two parts can be M×(N / k) blocks A and M×(NN / k) blocks B, where k is an integer greater than 1, such as 2, 4, 8, 16.

[0519] iii. The two parts can be (M / k)×(N / k) blocks A, and the other part is an "L"-shaped region B, where k is an integer greater than 1, such as 2, 4, 8, or 16. Block A can be located at the top left, top right, bottom left, or bottom right corner of the M×N block, as shown below. Figure 23 As shown.

[0520] iv. In one example, block A may have no residuals, while block B may have non-zero residuals.

[0521] v. In one example, block B may have no residuals, while block A may have non-zero residuals.

[0522] vi. In one example, a message (such as a CBF) used to indicate whether a block has a non-zero residual may not be signaled and may be inferred as one for the portion of the PRB to be determined to have a non-zero residual.

[0523] e. In one example, signaling can be used in the bitstream to notify portions of the data that have non-zero coefficients (or have all zero coefficients).

[0524] i. In one example, the instruction for this part can be signaled, such as an instruction for SBT.

[0525] ii. In one example, the allowed set of segments (e.g., as described in the bullet points above) may be predefined and / or indicated in the bitstream based on decoding information (e.g., mode).

[0526] f. In one example, transformation skipping can be applied to a PRB with dimensional (width and / or height) constraints.

[0527] i. In one example, a transform skip can only be applied if (W <= T1) && (H <= T1) && (W > T2 || H > T2), where W and H are the width and height of the codec block, and T1 and T2 are integers, for example, T1 = 64, T2 = 4.

[0528] ii. In one example, the transformation skip can only be applied if (W<=T1)&&(H<=T2), where T1 and T2 are integers, such as 32 or 64, and W and H are the width and height of the non-zero portion of the PRB or codec block.

[0529] iii. In one example, the transform skip can only be applied if (W>T1)||(H>T2), where T1 and T2 are integers, such as 4 or 8, and W and H are the width and height of the non-zero portion of the PRB or codec block.

[0530] g. In one example, a transformation or transformation skip can be applied to the portion of the PRB that has non-zero residues.

[0531] i. All bullet points and examples in this document can be applied to this section to determine whether and / or how to use transformations.

[0532] h. In one example, whether a block is a PRB depends on the signaling notification messages that can be found in VPS / SPS / PPS / Sequence Header / Picture Header / Strip Header / CTU / CU / Block.

[0533] i. The message can be a flag.

[0534] ii. The message can be conditionally signaled.

[0535] 4) If the message is not signaled, it is assumed to be a default value, such as 0 or 1.

[0536] iii. For example, signaling notification messages can only be sent for specific CTU / CU / block types, or CTU / CU / block types (e.g., IBC codec blocks).

[0537] iv. If the block is a PRB, a signaling notification message will be sent to indicate how the PRB should be split.

[0538] i. In one example, multi-level signaling can be used to determine whether to use PRB.

[0539] i. In one example, for the first level, such as the sequence / image level, a signaling indication can be given as to whether the PRB is enabled.

[0540] 5) In addition, optionally, the instruction can be conditionally signaled, for example, based on whether IBC / screen content is used.

[0541] ii. In one example, for the second level, such as at the block level, signaling can be used to indicate whether a PRB should be applied.

[0542] 6) Alternatively, the instruction may be conditionally signaled, for example, based on the instruction in the first level and / or other decoding information (e.g., block dimension / mode).

[0543] j. Whether PRB mode is allowed may depend on the information provided by the signaling notification or may be determined on the fly based on the decoding information (e.g., color components).

[0544] k. For example, the first message (denoted as ph_pts_enable_flag) is a signaling notification in the image header. The IBC codec block can only apply the PRB if ph_pts_enable_flag is true.

[0545] i. The second message (denoted as pts_enable_flag) is signaled in the sequence header. ph_pts_enable_flag is only signaled if pts_enable_flag is true. Otherwise, ph_pts_enable_flag is not signaled and is inferred to be false.

[0546] 5. Example Implementation

[0547] The following are some example embodiments of this disclosure summarized in Section 4 above, which can be applied to the VVC specification. Most of the relevant parts that have been added or modified are based on... Bold Italic Underline the text, and use [[]] to indicate any deleted parts.

[0548] 5.1 Example #1

[0549] This section presents an example of a solution for Implicit Selection of Transform Skip Modes (ISTS). Essentially, it follows the design principles of Implicit Selection of Transforms (ISTS) already adopted by AVS3. A high-level flag in the image header signaling indicates that ITS is enabled. If enabled, the allowed transform set is set to {DCT-II TS}, and the determination of the TS mode is based on the parity of the number of non-zero coefficients in the block. Simulation results report that, compared to HPM 6.0, the proposed ITS achieves a 15.86% and 12.79% bitrate reduction in screen content encoding / decoding, respectively, under AI and RA configurations. The increase in encoder and decoder complexity is claimed to be negligible.

[0550] 5.1.1 Introduction

[0551] In the current design of AVS3, only DCT-II is allowed for encoding and decoding residual blocks in IBC mode. For intra-frame codec blocks that do not include DT, applying IST allows the block to choose between DCT-II or DST-VII based on the parity of the number of non-zero coefficients. However, DST-VII is much less efficient for screen content encoding and decoding. Transform Skip (TS) mode is an efficient encoding and decoding method for screen content encoding and decoding. It is necessary to investigate how to allow the codec to support TS without explicitly signaling the block.

[0552] The method proposed in 5.1.2

[0553] In some embodiments, implicit selection of transformation skip mode (ISTS) can be used. A high-level flag is signaled in the picture header to indicate whether ISTS is enabled.

[0554] When ISTS is enabled, the allowed transform set is set to {DCT-II TS}, and the TS mode is determined based on the parity of the number of non-zero coefficients in the block, following the same design principles as IST. Odd indicators apply TS, while even indicators apply DCT-II. ISTS is applicable to CUs of sizes from 4×4 to 32×32 for intra-frame or IBC codecs (excluding CUs applying DT or PCM).

[0555] When ITS is disabled and IST is enabled, the allowed transform set is set to {DCT-II DST-VII}, which is the same as the current AVS3 design.

[0556] 5.1.3 Suggested changes to the syntax table, semantics, and decoding process

[0557] 7.1.2.2 Sequence Header

[0558] Table 14 Sequence Header Definitions

[0559]

[0560] 7.1.3.1 Image header of image I

[0561] Table 27 Intra-frame Prediction Header Definitions

[0562]

[0563] 7.1.3.2 Inter-frame Prediction Header Definition

[0564] Table 28 Inter-frame Prediction Header Definitions

[0565]

[0566]

[0567] 7.2.2.2 Sequence Header

[0568]

[0569] 7.2.3.1 Intra-frame prediction header

[0570]

[0571] 9.6.3 Inverse Transformation

[0572] This paper defines the process of converting the M1×M2 transformation coefficient matrix CoeffMatrix into the residual sample matrix ResidueMatrix.

[0573] If the intra-prediction mode is neither 'Intra_Luma_PCM' nor 'Intra_Chroma_PCM'

[0574]

[0575] like The current transform block is a lumen intra-frame prediction residual block. The values ​​of M1 and M2 are both less than 64 and the value of IstTuFlag is equal to 1. Then, the residual sample matrix ResidueMatrix is ​​derived according to the method defined by 0.

[0576]

[0577] Otherwise, derive the residual sample matrix ResidueMatrix according to the method defined in 9.6.3.1.

[0578] Otherwise (intra-prediction mode is 'Intra_Luma_PCM' or 'Intra_Chroma_PCM'), derive the residual sample matrix ResidueMatrix according to the method defined in 9.6.3.3.

[0579]

[0580] 5.2 Example #2

[0581] 5.2.1 Suggested changes to the syntax table, semantics, and decoding process 7.1.2.2 Sequence header definition

[0582] Table 14 Sequence Header Definitions

[0583]

[0584] 7.1.3.2 Inter-frame Prediction Header Definition

[0585]

[0586] 7.1.6 Encoding / Decoding Unit Definition

[0587]

[0588] 7.1.7 Transform Block Definition

[0589]

[0590]

[0591] 7.2.2.2 Sequence Header

[0592]

[0593] 7.2.3.2 Inter-frame Prediction Image Header

[0594]

[0595] 9.6.3 Inverse Transformation

[0596] If the intra-prediction mode is neither 'Intra_Luma_PCM' nor 'Intra_Chroma_PCM', or if the current block uses the block copy intra-prediction mode, then:

[0597] —If PhIstsEnableFlag is 0 and the current transform block is a lumen intra-frame prediction residual block, the values ​​of M1 and M2 are both less than 64 and the value of IstTuFlag is equal to 1, then the residual sample matrix ResidueMatrix is ​​derived according to the method defined in 9.6.3.2.

[0598] Otherwise, if PhIstsEnableFlag is 1, and the current transform block is a luma intra-prediction residual block or a luma block copy of an intra-prediction residual block. If the values ​​of M1 and M2 are both less than 64 and the value of IstTuFlag is equal to 1, then the residual sample matrix ResidueMatrix is ​​derived according to the method defined in 9.6.3.4.

[0599] Otherwise, derive the residual sample matrix ResidueMatrix according to the method defined in 9.6.3.1.

[0600] Otherwise (intra-prediction mode is 'Intra_Luma_PCM' or 'Intra_Chroma_PCM'), derive the residual sample matrix ResidueMatrix according to the method defined in 9.6.3.3.

[0601] 5.3. Example #3

[0602] 7.1.2.2 Sequence Header Definition

[0603]

[0604] 7.1.3.2 Inter-frame Prediction Header Definition

[0605]

[0606] 7.1.6 Encoding / Decoding Unit Definition

[0607]

[0608]

[0609] 7.1.7 Transform Block Definition

[0610]

[0611]

[0612] 7.2.2.2 Sequence Header

[0613]

[0614] 7.2.3.2 Inter-frame Prediction Image Header

[0615]

[0616]

[0617] 9.6.3 Inverse Transformation

[0618] If the intra-prediction mode is neither 'Intra_Luma_PCM' nor 'Intra_Chroma_PCM', or if the current block uses the block copy intra-prediction mode, then:

[0619] —If PhIstsEnableFlag is 0 and the current transform block is a lumen intra-frame prediction residual block, the values ​​of M1 and M2 are both less than 64 and the value of IstTuFlag is equal to 1, then the residual sample matrix ResidueMatrix is ​​derived according to the method defined in 9.6.3.2.

[0620] Otherwise, if PhIstsEnableFlag is 1, and the current transform block is a luma intra-prediction residual block or a luma block copy of an intra-prediction residual block. If the values ​​of M1 and M2 are both less than 64 and the value of IstTuFlag is equal to 1, then the residual sample matrix ResidueMatrix is ​​derived according to the method defined in 9.6.3.4.

[0621] Otherwise, derive the residual sample matrix ResidueMatrix according to the method defined in 9.6.3.1.

[0622] Otherwise (intra-prediction mode is 'Intra_Luma_PCM' or 'Intra_Chroma_PCM'), derive the residual sample matrix ResidueMatrix according to the method defined in 9.6.3.3.

[0623] 5.4 Example #4

[0624] 7.1.2.2 Sequence Header Definition

[0625]

[0626] 7.1.3.2 Inter-frame Prediction Header Definition

[0627]

[0628]

[0629] 7.1.6 Encoding / Decoding Unit Definition

[0630]

[0631]

[0632] 7.1.7 Transform Block Definition

[0633]

[0634] 7.2.2.2 Sequence Header

[0635]

[0636]

[0637] 7.2.3.2 Inter-frame Prediction Header Definition

[0638]

[0639]

[0640] 9.6.3 Inverse Transformation

[0641] If the intra-prediction mode is neither 'Intra_Luma_PCM' nor 'Intra_Chroma_PCM', or if the current block uses the block copy intra-prediction mode, then:

[0642] —If PhIstsEnableFlag is 0 and the current transform block is a lumen intra-frame prediction residual block, the values ​​of M1 and M2 are both less than 64 and the value of IstTuFlag is equal to 1, then the residual sample matrix ResidueMatrix is ​​derived according to the method defined in 9.6.3.2.

[0643] Otherwise, if PhIstsEnableFlag is 1, and the current transform block is a luma intra-prediction residual block or a luma block copy of an intra-prediction residual block. If the values ​​of M1 and M2 are both less than 64 and the value of IstTuFlag is equal to 1, then the residual sample matrix ResidueMatrix is ​​derived according to the method defined in 9.6.3.4.

[0644] Otherwise, derive the residual sample matrix ResidueMatrix according to the method defined in 9.6.3.1.

[0645] Otherwise (intra-prediction mode is 'Intra_Luma_PCM' or 'Intra_Chroma_PCM'), derive the residual sample matrix ResidueMatrix according to the method defined in 9.6.3.3.

[0646] 9.6.3.4 Implicit Selective Inverse Transform Skip Method

[0647] The implicit selective inverse transform skipping method is as follows:

[0648] a) Calculate the offset shift, which is equal to 15 – BitDepth – ((logM1 + logM2) >> 1).

[0649]

[0650] otherwise:

[0651] m = i, n = j

[0652] [[b)]] If the shift is greater than or equal to 0, then the carry factor rnd_factor is equal to 1 << (shift – 1).

[0653] The matrix W is obtained from the transformation coefficient matrix:

[0654]

[0655] [[c)]] If the shift is less than 0, let shift = -shift. Obtain matrix W from the transformation matrix:

[0656]

[0657] [[d)]] The matrix W is directly used as the residual sample matrix ResidueMatrix, and the implicit inverse transformation skip operation is terminated.

[0658] Figure 17This is a block diagram illustrating an example video processing system 1700, in which various techniques disclosed herein can be implemented. Various implementations may include some or all of the components of system 1700. System 1700 may include an input 1702 for receiving video content. The video content may be received in a raw or uncompressed format (e.g., 8-bit or 10-bit multi-component pixel values), or in a compressed or encoded format. Input 1702 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces (e.g., Ethernet, Passive Optical Networking (PON), etc.) and wireless interfaces (e.g., Wi-Fi or cellular interfaces).

[0659] System 1700 may include codec component 1704, which can implement the various codec or encoding methods described in this document. Codec component 1704 can reduce the average bit rate of video from input 1702 to the output of codec component 1704 to produce a codec representation of the video. Therefore, codec techniques are sometimes referred to as video compression or video transcoding techniques. The output of codec component 1704 can be stored or transmitted via communication through the connection represented by component 1706. The stored or transmitted bitstream (or codec) representation of the video received at input 1702 can be used by component 1708 to generate pixel values ​​or displayable video to be sent to display interface 1710. The process of generating a user-viewable video from the bitstream is sometimes referred to as video decompression. Furthermore, although some video processing operations are referred to as “codec” operations or tools, it will be understood that codec tools or operations are used at the encoder, and the corresponding decoding tools or operations that inversely represent the codec results will be performed by the decoder.

[0660] Examples of peripheral bus interfaces or display interfaces may include Universal Serial Bus (USB), High Definition Multimedia Interface (HDMI), or DisplayPort. Examples of storage interfaces include SATA (Serial Advanced Technology Accessory), PCI, IDE, etc. The technologies described in this document can be found in a variety of electronic devices, such as mobile phones, laptops, smartphones, or other devices capable of performing digital data processing and / or video display.

[0661] Figure 21This is a block diagram of a video processing apparatus 3600. Apparatus 3600 can be used to implement one or more methods described herein. Apparatus 3600 can be embodied in smartphones, tablets, computers, Internet of Things (IoT) receivers, etc. Apparatus 3600 may include one or more processors 3602, one or more memories 3604, and video processing circuitry 3606. The processors(multiple) 3602 can be configured to implement one or more methods described in this document. The one or more memories 3604 can be used to store data and code for implementing the methods and techniques described herein. The video processing circuitry 3606 can be used to implement some of the techniques described in this document in hardware circuitry.

[0662] Figure 18 This is a block diagram illustrating an example video codec system 100 that can utilize the techniques disclosed herein.

[0663] like Figure 18 As shown, the video encoding / decoding system 100 may include a source device 110 and a destination device 120. The source device 110 generates encoded video data, which may be referred to as a video encoding device. The destination device 120 can decode the encoded video data generated by the source device 110, which may be referred to as a video decoding device.

[0664] The source device 110 may include a video source 112, a video encoder 114, and an input / output (I / O) interface 116.

[0665] Video source 112 may include sources such as video capture devices, interfaces for receiving video data from video content providers, and / or computer graphics systems for generating video data, or combinations of these sources. Video data may include one or more images. Video encoder 114 encodes the video data from video source 112 to generate a bitstream. The bitstream may include a sequence of bits forming a codec representation of the video data. The bitstream may include codec images and associated data. A codec image is a codec representation of an image. Associated data may include sequence parameter sets, image parameter sets, and other syntax structures. I / O interface 116 may include a modulator / demodulator (modem) and / or a transmitter. Encoded video data can be transmitted directly to destination device 120 via network 130a through I / O interface 116. Encoded video data may also be stored on storage medium / server 130b for access by destination device 120.

[0666] Destination device 120 may include I / O interface 126, video decoder 124 and display device 122.

[0667] I / O interface 126 may include a receiver and / or a modem. I / O interface 126 may acquire encoded video data from source device 110 or storage medium / server 130b. Video decoder 124 may decode the encoded video data. Display device 122 may display the decoded video data to a user. Display device 122 may be integrated with destination device 120, or may be located external to destination device 120, which is configured to interact with an external display device.

[0668] The video encoder 114 and the video decoder 124 can operate according to video compression standards such as the High Efficiency Video Codec (HEVC) standard, the Universal Video Codec (VVM) standard, and other current and / or further standards.

[0669] Figure 19 This is a block diagram illustrating an example of a video encoder 200, which can be... Figure 18 The video encoder 114 in the system 100 shown.

[0670] The video encoder 200 can be configured to perform any or all of the techniques disclosed herein. Figure 19 In the example, the video encoder 200 includes multiple functional components. The techniques described in this disclosure can be shared among the various components of the video encoder 200. In some examples, the processor can be configured to perform any or all of the techniques described in this disclosure.

[0671] The functional components of the video encoder 200 may include a segmentation unit 201, a prediction unit 202 (which may include a mode selection unit 203), a motion estimation unit 204, a motion compensation unit 205, an intra-frame prediction unit 206, a residual generation unit 207, a transform unit 208, a quantization unit 209, an inverse quantization unit 210, an inverse transform unit 211, a reconstruction unit 212, a buffer 213, and an entropy coding unit 214.

[0672] In other examples, the video encoder 200 may include more, fewer, or different functional components. In one example, the prediction unit 202 may include an intra-block copy (IBC) unit. The IBC unit can perform prediction in IBC mode, where at least one reference picture is the picture containing the current video block.

[0673] Furthermore, some components (such as motion estimation unit 204 and motion compensation unit 205) may be highly aggregated, but for illustrative purposes, in Figure 11 The examples represent the examples respectively.

[0674] The segmentation unit 201 can segment an image into one or more video blocks. The video encoder 200 and the video decoder 300 can support various video block sizes.

[0675] The mode selection unit 203 can, for example, select one of the encoding / decoding modes (intra-frame or inter-frame) based on the error result, and provide the resulting intra-frame or inter-frame encoded / decoded block to the residual generation unit 207 to generate residual block data, and to the reconstruction unit 212 to reconstruct the coded block for use as a reference picture. In some examples, the mode selection unit 203 can select a combination of intra-frame prediction and inter-frame prediction (CIIP) modes, where the prediction is based on the inter-frame prediction signal and the intra-frame prediction signal. The mode selection unit 203 can also select the resolution of the motion vector for the block (e.g., sub-pixel precision or integer pixel precision) in the case of inter-frame prediction.

[0676] To perform inter-frame prediction on the current video block, motion estimation unit 204 can generate motion information for the current video block by comparing one or more reference frames from buffer 213 with the current video block. Motion compensation unit 205 can determine the predicted video block for the current video block based on motion information and decoded samples from images other than those associated with the current video block from buffer 213.

[0677] For example, motion estimation unit 204 and motion compensation unit 205 can perform different operations on the current video block, depending on whether the current video block is in an I-band, P-band, or B-band.

[0678] In some examples, motion estimation unit 204 can perform unidirectional prediction on the current video block, and can search for a reference video block for the current video block in the reference images of list 0 or list 1. Then, motion estimation unit 204 can generate a reference index indicating the reference image in list 0 or list 1 containing the reference video block, and a motion vector indicating the spatial displacement between the current video block and the reference video block. Motion estimation unit 204 can output the reference index, prediction direction indicator, and motion vector as motion information for the current video block. Motion compensation unit 205 can generate a predicted video block for the current block based on the reference video block indicated by the motion information of the current video block.

[0679] In other examples, motion estimation unit 204 can perform bidirectional prediction on the current video block. Motion estimation unit 204 can search for a reference video block for the current video block in the reference images in list 0, and also search for another reference video block for the current video block in the reference images in list 1. Then, motion estimation unit 204 can generate reference indices indicating the reference images in lists 0 and 1 containing the reference video blocks, and motion vectors indicating the spatial displacement between the reference video blocks and the current video block. Motion estimation unit 204 can output the reference index and motion vector of the current video block as motion information for the current video block. Motion compensation unit 205 can generate a predicted video block for the current video block based on the reference video blocks indicated by the motion information of the current video block.

[0680] In some examples, the motion estimation unit 204 can output a complete set of motion information for the decoder's decoding processing.

[0681] In some examples, motion estimation unit 204 may not output the complete set of motion information for the current video. Instead, motion estimation unit 204 can signal the motion information of the current video block by referencing the motion information of another video block. For example, motion estimation unit 204 may determine that the motion information of the current video block is sufficiently similar to the motion information of neighboring video blocks.

[0682] In one example, the motion estimation unit 204 may instruct the video decoder 300, within the syntax structure associated with the current video block, to indicate that the current video block has the same motion information as another video block.

[0683] In another example, motion estimation unit 204 can identify another video block and motion vector difference (MVD) within the syntactic structure associated with the current video block. The motion vector difference indicates the difference between the motion vector of the current video block and the motion vector of the indicated video block. Video decoder 300 can use the motion vector of the indicated video block and the motion vector difference to determine the motion vector of the current video block.

[0684] As discussed above, the video encoder 200 can predictively signal motion vectors. Two examples of predictive signaling techniques that can be implemented by the video encoder 200 include Advanced Motion Vector Prediction (AMVP) and merge pattern signaling.

[0685] Intra-prediction unit 206 can perform intra-prediction on the current video block. When intra-prediction unit 206 performs intra-prediction on the current video block, it can generate prediction data for the current video block based on decoded samples from other video blocks in the same frame. The prediction data for the current video block may include the predicted video block and various syntax elements.

[0686] The residual generation unit 207 can generate residual data for the current video block by subtracting (e.g., indicated by a minus sign) the predicted video block from the current video block. The residual data for the current video block can include residual video blocks corresponding to different sample components in the current video block.

[0687] In other examples, there may be no residual data for the current video block. For example, in skip mode, the residual generation unit 207 may not perform the subtraction operation.

[0688] The transform processing unit 208 can generate one or more transform coefficient video blocks for the current video block by applying one or more transforms to the residual video blocks associated with the current video block.

[0689] After the transform processing unit 208 generates a transform coefficient video block associated with the current video block, the quantization unit 209 can quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values ​​associated with the current video block.

[0690] The inverse quantization unit 210 and the inverse transform unit 211 can apply inverse quantization and inverse transform to the transform coefficient video block, respectively, to reconstruct the residual video block based on the transform coefficient video block. The reconstruction unit 212 can add the reconstructed residual video block to the corresponding samples of one or more predicted video blocks generated by the prediction unit 202 to produce a reconstructed video block associated with the current block, which is then stored in the buffer 213.

[0691] After the video block is reconstructed by reconstruction unit 212, a loop filtering operation can be performed to reduce video block artifacts in the video block.

[0692] Entropy encoding unit 214 can receive data from other functional components of video encoder 200. When entropy encoding unit 214 receives data, it can perform one or more entropy encoding operations to generate entropy encoded data and output a bit stream including the entropy encoded data.

[0693] Figure 20 This is a block diagram illustrating an example of a video decoder 300, which can be... Figure 18 The video decoder 114 in the system 100 shown.

[0694] The video decoder 300 can be configured to perform any or all of the technologies disclosed herein. Figure 20 In the example, the video decoder 300 includes multiple functional components. The techniques described in this disclosure can be shared among the various components of the video decoder 300. In some examples, the processor can be configured to perform any or all of the techniques described in this disclosure.

[0695] exist Figure 20 In the example, the video decoder 300 includes an entropy decoding unit 301, a motion compensation unit 302, an intra-frame prediction unit 303, an inverse quantization unit 304, an inverse transform unit 305, a reconstruction unit 306, and a buffer 307. In some examples, the video decoder 300 can perform encoding passes typically described with respect to the video encoder 200. Figure 19 The opposite decoding iteration.

[0696] The entropy decoding unit 301 can retrieve the encoded bitstream. The encoded bitstream may include entropy-coded video data (e.g., encoded blocks of video data). The entropy decoding unit 301 can decode the entropy-coded video data, and the motion compensation unit 302 can determine motion information based on the entropy-decoded video data. This motion information includes motion vectors, motion vector precision, reference image list index, and other motion information. For example, the motion compensation unit 302 can determine this information by executing AMVP and merge modes.

[0697] The motion compensation unit 302 can generate motion compensation blocks, possibly performing interpolation based on an interpolation filter. The syntax elements may include identifiers for the interpolation filter used at sub-pixel precision.

[0698] The motion compensation unit 302 can use the interpolation filter used by the video encoder 200 during video block encoding to calculate the interpolation of sub-integer pixels of the reference block. The motion compensation unit 302 can determine the interpolation filter used by the video encoder 200 based on the received syntax information, and use the interpolation filter to generate the prediction block.

[0699] The motion compensation unit 302 may use some syntax information to determine the size of the blocks used to encode (multiple) frames and / or (multiple) stripes of the encoded video sequence, segmentation information describing how each macroblock of the image of the encoded video sequence is segmented, a mode indicating how each partition is encoded, one or more reference frames (and a list of reference frames) for each inter-frame coded block, and other information for decoding the encoded video sequence.

[0700] Intra-prediction unit 303 can use, for example, an intra-prediction mode received in the bitstream to form prediction blocks based on spatially adjacent blocks. Dequantization unit 303 dequantizes (e.g., dequantizes) the quantized video block coefficients provided in the bitstream and decoded by entropy decoding unit 301. Inverse transform unit 303 applies an inverse transform.

[0701] The reconstruction unit 306 can add the residual block to the corresponding prediction block generated by the motion compensation unit 202 or the intra-frame prediction unit 303 to form a decoded block. If necessary, a deblocking filter can also be applied to filter the decoded block to remove block artifacts. The decoded video block is then stored in a buffer 307, which provides a reference block for subsequent motion compensation / intra-frame prediction and also generates decoded video for presentation on a display device.

[0702] Below is a list of preferred solutions for some embodiments.

[0703] The following solutions illustrate example embodiments of the techniques discussed in the previous section (e.g., Project 1).

[0704] 1. A video processing method (e.g., Figure 22 The method described in the document (2200) includes: converting between video blocks of a video and a codec representation of the video; determining, based on a rule, whether to apply a horizontal identity transformation or a vertical identity transformation to the video blocks (2202); and performing the conversion based on the determination (2204), wherein the rule specifies the relationship between the determination and the representative coefficients of the decoding coefficients from one or more representative blocks of the video.

[0705] 2. The method of Solution 1, wherein one or more representative blocks belong to the color component to which the video block belongs.

[0706] 3. The method of Solution 1, wherein one or more representative blocks belong to a color component that is different from the color component of the video block.

[0707] 4. The method of any one of solutions 1-3, wherein the one or more representative blocks correspond to video blocks.

[0708] 5. The method of any one of solutions 1-3, wherein the one or more representative blocks do not include video blocks.

[0709] The following solutions illustrate example embodiments of the techniques discussed in the previous section (e.g., bullet points 1 and 2).

[0710] 6. The method of any one of solutions 1-5, wherein the representative coefficients include decoding coefficients with non-zero values.

[0711] 7. The method of any one of solutions 1-6, wherein the relationship specifies the use of the representativeness coefficient based on a modified coefficient determined by modifying the representativeness coefficient.

[0712] 8. The method of any one of solutions 1-7, wherein the representative coefficient corresponds to the effective coefficient of the decoding coefficient.

[0713] The following solutions illustrate example embodiments of the techniques discussed in the previous section (e.g., Project 3).

[0714] 9. A video processing method, comprising: converting between video blocks of a video and a encoded / decoded representation of the video; determining, based on a rule, whether to apply a horizontal identity transformation or a vertical identity transformation to the video blocks; and performing the conversion based on the determination, wherein the rule specifies a relationship between the determination and the decoded luminance coefficients of the video blocks.

[0715] 10. The method of Solution 1, wherein performing the transformation includes applying a horizontal or vertical isomorphic transformation of the luminance component of the video block and applying DCT2 to the chrominance component of the video block.

[0716] The following solutions illustrate example embodiments of the techniques discussed in the previous section (e.g., Items 1 and 4).

[0717] 11. A video processing method comprising: converting between video blocks of a video and a codec representation of the video; determining, based on a rule, whether to apply a horizontal identity transformation or a vertical identity transformation to the video blocks; and performing the conversion based on the determination, wherein the rule specifies a relationship between the determination and a value V associated with a decoding coefficient or a representative coefficient of a representative block.

[0718] 12. The method of Solution 11, where V equals the number of representative coefficients.

[0719] 13. The method of Solution 11, where V equals the sum of the values ​​of the representative coefficients.

[0720] 14. The method of Solution 11, where V is a function of the residual energy distribution of the representative coefficient.

[0721] 15. The method of any one of solutions 11-14, wherein the relation is defined with respect to the parity of the value V.

[0722] The following solutions illustrate example embodiments of the techniques discussed in the previous section (e.g., Project 5).

[0723] 16. The method of any of the above solutions, wherein the rule specifies that the relationship further depends on the encoding and decoding information of the video block.

[0724] 17. The method of Solution 16, wherein the encoding / decoding information is the encoding / decoding mode of the video block.

[0725] 18. The method of Solution 16, wherein the encoding / decoding information comprises a minimum rectangular region covering all valid coefficients of the video block.

[0726] The following solutions illustrate example embodiments of the techniques discussed in the previous section (e.g., Item 6).

[0727] 19. The method of any of the above solutions, wherein the determination is performed because the video block has a pattern or a constraint on the coefficients.

[0728] 20. The method of Solution 19, wherein the type corresponds to the Intra-Block Copy (IBC) mode.

[0729] 21. The method of Solution 19, wherein the constraint on the coefficients makes the coefficients outside the rectangle of the current block zero.

[0730] The following solutions illustrate example embodiments of the techniques discussed in the previous section (e.g., item 7).

[0731] 22. The method of any one of solutions 1-21, wherein, in the case that horizontal and vertical identity transformations are not used, the transformation is performed using DCT-2 transformation or DST-7 transformation.

[0732] The following solutions illustrate example embodiments of the techniques discussed in the previous section (e.g., Item 9).

[0733] 23. The method of any one of solutions 1-22, wherein one or more syntax fields in the codec representation indicate whether the method is enabled for video blocks.

[0734] 24. The method of Solution 23, wherein the one or more syntax fields are included at the sequence level, picture level, strip level, slice group level, slice level, or sub-picture level.

[0735] 25. The method of any one of solutions 23-24, wherein the one or more syntax fields are included in a strip header or an image header.

[0736] The following solutions illustrate example embodiments of the techniques discussed in the previous section (e.g., items 1 and 8).

[0737] 26. A video processing method, comprising: determining that one or more syntax fields exist in the codec representation of a video, wherein the video contains one or more video blocks; and determining, based on the one or more syntax fields, whether to enable a horizontal identity transformation or a vertical identity transformation on the video blocks in the video.

[0738] 27. The method of Solution 1, wherein, in response to the implicit determination of the transform skip mode indicated by the one or more syntax fields being enabled, a transformation between a first video block of a video and the codec representation of the video is performed, and a rule is used to determine whether to apply a horizontal identity transformation or a vertical identity transformation to the video block; and a transformation is performed based on the determination, wherein the rule specifies the relationship between the determination and the representative coefficients of the decoding coefficients from one or more representative blocks of the video.

[0739] 28. The method of Solution 27, the first video block is encoded and decoded in intra-block copy mode.

[0740] 29. The method of Solution 27, the first video block is encoded and decoded in intra-frame mode.

[0741] 30. Solution 27's approach involves encoding and decoding the first video block using intra-frame mode instead of derivative tree (DT) mode.

[0742] 31. The method of Solution 27, wherein the parity is determined based on the number of non-zero coefficients in the first video block.

[0743] 32. The method of Solution 27 applies the horizontal and vertical identity transformations to the first video block when the parity of the number of non-zero coefficients in the first video block is even.

[0744] 33. The method of Solution 27, where the horizontal and vertical identity transformations are not applied to the first video block when the parity of the number of non-zero coefficients in the first video block is even.

[0745] 34. Solution 33 applies DCT-2 to the first video block.

[0746] 35. The method of Solution 32 further includes: in response to one or more syntax fields indicating that the implicit determination of the transform skip mode is disabled, the horizontal identity transformation and the vertical identity transformation are not applied to the first video block.

[0747] 36. The method of solution 32, wherein DCT-2 is applied to the first video block.

[0748] The following solutions illustrate example embodiments of the techniques discussed in the previous section (e.g., items 9, 10).

[0749] 37. A video processing method, comprising: making a first determination regarding whether to enable the use of an identity transformation for a conversion between video blocks of a video and a codec representation of a video; making a second determination regarding whether to enable a zeroing operation during the conversion; and performing the conversion based on the first determination and the second determination.

[0750] 38. The method of solution 37, wherein one or more syntax fields of the first level in the codec representation indicate a first determination.

[0751] 39. The method of any one of solutions 37-38, wherein one or more syntax fields of the second level in the codec representation indicate a second determination.

[0752] 40. The method of any one of solutions 38-39, wherein the first level and the second level correspond to the header field at the sequence or image level or the parameter set at the sequence or image level or the adaptive parameter set.

[0753] 41. The method of any one of solutions 37-40, wherein the transformation uses an identity transformation or a zeroing operation, but not both.

[0754] The following solutions illustrate example embodiments of the techniques discussed in the previous section (e.g., items 12 and 13).

[0755] 42. A video processing method, comprising: performing a conversion between video blocks of a video and a codec representation of the video; wherein the video blocks are represented as codec blocks in the codec representation, wherein the non-zero coefficients of the codec blocks are restricted to one or more sub-regions; and wherein an identity transformation is applied to generate the codec blocks.

[0756] 43. The method of Solution 1, wherein the one or more sub-regions include an upper right sub-region of a video block with dimensions K×L, where K and L are integers, K is min(T1, W), L is min(T2, H), where W and H are the width and height of the video block, respectively, and T1 and T2 are thresholds.

[0757] 44. The method of any one of solutions 42-43, wherein the encoding / decoding representation indicates the one or more sub-regions.

[0758] The following solutions illustrate example embodiments of the techniques discussed in the previous section (items 16 and 17).

[0759] 45. The method of any one of solutions 1-44, wherein the video region includes a video encoding / decoding unit.

[0760] 46. ​​The methods of solutions 1-45, wherein the video region is a prediction unit or a transform unit.

[0761] 47. The method of any one of solutions 1-46, wherein the video blocks satisfy a specific dimension condition.

[0762] The following solutions show example implementations of the techniques discussed in the previous chapter (e.g., items 18-23).

[0763] 49. A video processing method comprising: performing a conversion between a video comprising one or more video regions and a video codec representation, wherein the codec representation conforms to a format rule; wherein the format rule specifies that the coefficients of the video regions are reordered according to a mapping after being parsed from the codec representation.

[0764] 50. The method according to solution 49, wherein the coefficients are reordered before dequantization, before inverse transformation, or before reconstruction.

[0765] 51. The method according to any one of solutions 49-50, wherein the format rules specify that the video region corresponds to the encoding / decoding unit, the transform unit, or the prediction unit.

[0766] 52. The method according to any one of solutions 49-51, wherein the format rules specify how the codec information of the codec representation determines how the coefficients are reordered after parsing.

[0767] 53. The method according to any one of solutions 49-52, wherein the format rules specify that the mapping is determined by the transformation type or slice type of the video region used in the strip or picture.

[0768] 54. The method according to any one of solutions 49-52, wherein the format rules specify that the mapping is determined by the transform type used in the codec tree unit, the codec unit of the video region, or the block type.

[0769] 55. The method according to any one of solutions 49-52, wherein the format rules specify that the mapping is determined by the syntax fields included in the parameter set or the header fields in the codec representation.

[0770] The following solutions show example implementations of the techniques discussed in the previous chapter (e.g., items 24-25).

[0771] 56. A video processing method, comprising: converting a video block of a video to a codec representation of the video; determining whether the video block satisfies the condition of signaling notification of a portion of residual blocks in the codec representation; and performing the conversion based on the determination; wherein the portion of residual blocks is divided into at least two parts, wherein at least one part does not have non-zero residuals signaled in the codec representation.

[0772] 57. The method according to solution 56, wherein the conditions are based on a mode used to encode and decode video blocks into a codec representation.

[0773] 58. The method according to solutions 56-57, wherein the signaling notification in the codec representation comprises at least one or more of two parts according to format rules.

[0774] 59. The method according to any one of solutions 1 to 58, wherein the video region includes video images.

[0775] 60. The method of any one of solutions 1 to 59, wherein the conversion includes encoding the video into a codec representation.

[0776] 61. The method of any one of solutions 1 to 59, wherein the conversion includes decoding the encoding / decoding representation to generate pixel values ​​of a video.

[0777] 62. A video decoding apparatus, comprising a processor configured to implement the method described in one or more of solutions 1 to 61.

[0778] 63. A video encoding / decoding apparatus, comprising a processor configured to implement the method described in one or more of solutions 1 to 61.

[0779] 64. A computer program product having computer code stored thereon, which, when executed by a processor, causes the processor to implement the method described in any one of solutions 1 to 61.

[0780] 65. The methods, apparatus or systems described in this document.

[0781] Figure 24 This is a flowchart representation of a video processing method according to the present technology. Method 2400 includes, in operation 2410, performing a conversion between the current block of video and the video bitstream according to rules. The rules specify whether to enable a transform skip mode based on the codec information of the current block. The transform skip mode is a codec mode that skips the predictive residual transform of a video block.

[0782] In some embodiments, the codec information includes whether subblock transform is applicable to the current block. In some embodiments, the determination is further based on syntax elements included in the sequence parameter set. In some embodiments, the determination is also based on syntax elements included in the picture header, sequence header, picture parameter set, video parameter set, or stripe header. In some embodiments, the codec information includes whether the current block is encoded in inter-frame mode. In some embodiments, the codec information includes whether the current block is encoded in direct mode. In some embodiments, the codec information includes whether the current block is encoded in intra-block copy (IBC) mode. In some embodiments, the codec information includes whether the current block uses subblock transform encoding / decoding. In some embodiments, the codec information includes whether the current block uses position-based transform (PBT) encoding / decoding.

[0783] In some embodiments, the transform skip mode is determined based on the representativeness coefficient of the current block. In some embodiments, the current block is encoded in an inter-frame codec mode. Direct mode and sub-block transform are not applicable to the current block. In some embodiments, the transform skip mode is determined based on the representativeness coefficient of the current block encoded in an inter-frame codec mode, wherein the sub-block transform is applicable to the current block. In some embodiments, the transform skip mode is applied to the current block in response to the current block being encoded in an inter-frame codec mode and the sub-block transform being applicable to the current block. In some embodiments, the transform skip mode is applied to the current block in response to the current block being encoded in an intra-frame copy mode and the sub-block transform being applicable to the current block. In intra-frame copy mode, prediction samples are derived from blocks of sample values ​​from the same decoded video region. In some embodiments, determination is invoked in response to the current block's codec information satisfying a condition.

[0784] Figure 25 This is a flowchart representation of a method for processing video data according to the present technology. Method 2500 includes, in operation 2510, performing a conversion between the current block of the video and the video bitstream according to rules. The rules specify that the representativeness coefficients of one or more representative blocks, in which at least one representative block is different from the current block, determine the transformation skip mode to be used on the current block, wherein the one or more representative blocks include the last N blocks with the same prediction mode that precede the current block in decoding order, where N is an integer greater than 1.

[0785] In some embodiments, the same prediction mode includes an inter-frame coding / decoding mode. In some embodiments, the same prediction mode includes a direct coding / decoding mode.

[0786] Figure 26 This is a flowchart representation of a video processing method according to the present technology. Method 2600 includes, in operation 2610, performing a conversion between the current block of video and the bitstream of video according to rules. The rules specify that the representativeness coefficients of one or more representative blocks of video determine the conversion of the current video block using a transform skip mode, wherein the representativeness coefficients include coefficients relative to the representative blocks at predefined positions (xPos, yPos).

[0787] In some embodiments, xPos is equal to or less than a threshold Tx and / or yPos is equal to or less than a threshold Ty. In some embodiments, Tx is 31 and / or Ty is 31. In some embodiments, xPos is equal to or greater than a threshold Tx and / or yPos is equal to or greater than a threshold Ty. In some embodiments, Tx is 32 and / or Ty is 32. In some embodiments, xPos and / or yPos are determined based on one or more syntax elements indicating allowed sub-blocks with non-zero coefficients. In some embodiments, one or more syntax elements also indicate the mode of sub-block transformation. In some embodiments, the method is applicable to video blocks encoded and decoded in an inter-frame encoding / decoding mode. In some embodiments, the method is applicable to video blocks encoded and decoded in a direct encoding / decoding mode.

[0788] Figure 27 This is a flowchart representation of a method for processing video data according to the present technology. Method 2700 includes, in operation 2710, performing a conversion between the current block of video and the bitstream according to rules. The rules specify that a transform skip mode is applied in response to an identity transform (IT) being used to encode and decode the current block. The current block is encoded and decoded in either an inter-frame encoding / decoding mode or a direct encoding / decoding mode.

[0789] In some embodiments, at least one of DCT2, DST7, or DCT8 is used when transform skipping mode is not applicable. In some embodiments, DCT2, DST7, or DCT8 is used depending on the subblock transform used. In some embodiments, the two allowed transform sets include {DCT2} and {DCT2, IT}. In some embodiments, the two allowed transform sets are used for non-DT codec blocks. In some embodiments, the allowed transform sets include {DCT2, IT}. In some embodiments, the allowed transform sets are determined based on whether implicit transform selection (IST) or subblock transform is enabled.

[0790] In some embodiments, the applicability of a transform skip mode is determined based on a first syntax element in the image header, where the first syntax element indicates that transform skipping is enabled. In some embodiments, in response to the first syntax element indicating that transform skipping is enabled, a second syntax element in the sequence header is included in the bitstream, and in response to the first syntax element being omitted in the bitstream, the second syntax element in the sequence header is omitted from the bitstream. In some embodiments, the applicability of the transform skip mode is further determined based on whether additional conditions are met, including whether the parity of the representative coefficients of the current block meets predetermined requirements. In some embodiments, the current block is encoded and decoded in either an inter-frame encoding / decoding mode or an intra-frame block copying mode.

[0791] Figure 28This is a flowchart representation of a method for processing video data according to the present technology. Method 2800 includes, in operation 2810, performing a conversion between the current block of video and the video bitstream according to rules. The rules specify the use of coefficient reordering based on the encoding and decoding information of the current block. By reordering the coefficients, the coefficients of the current block parsed from the bitstream are reordered before applying a dequantization process, an inverse transform process, or a reconstruction process.

[0792] In some embodiments, the current block is a codec unit, transform unit, or prediction unit, and coefficient reordering is performed at the codec unit level. In some embodiments, the codec information includes a transform type suitable for the transformation. In some embodiments, the transform type is determined based on the coefficients. In some embodiments, the transform type includes at least one of DCI-2 mode, DST-7 mode, DCT-8 mode, or transform skip mode, in which the coefficients of the video block are encoded and decoded without applying a non-identical transform. In some embodiments, the use of coefficient reordering is based on whether a transform skip mode is used. In some embodiments, the resolved coefficients are rearranged in reverse order before the dequantization process, inverse transform process, or reconstruction process.

[0793] In some embodiments, the rule specifies that the use of coefficient reordering is further based on codec tree unit type, codec unit type, or block type. In some embodiments, the use of coefficient reordering is based on the transform type of a block having a specific type. In some embodiments, the use of coefficient reordering is not based on the transform type of the block not being a specific type. In some embodiments, blocks having a specific type include intra-frame encoded blocks. In some embodiments, the rule specifies that the use of coefficient reordering is further based on stripe type, picture type, or slice type. In some embodiments, the use of coefficient reordering is based on the transform type of an I-strip or I-picture. In some embodiments, the use of coefficient reordering is not based on the transform type of a P-strip, B-strip, P-picture, or B-picture.

[0794] In some embodiments, the codec information includes the transform unit position of a transform unit-level sub-block. In some embodiments, the use of coefficient reordering is indicated by syntax flags in a video parameter set, sequence parameter set, picture parameter set, sequence header, picture header, stripe header, codec tree unit, codec unit, or codec block, based on the transform type. In some embodiments, syntax flags are conditionally included in the bitstream in response to a block not being a zero residual block, wherein the block is at the codec unit level, transform unit level, or partial sub-block level. In some embodiments, syntax flags are conditionally included in the bitstream in response to a transform type indicating that a transform skip mode is enabled. In some embodiments, a first syntax element is included in the sequence header indicating the transform type, and a second syntax element is conditionally included in the picture header based on the first syntax element.

[0795] In some embodiments, the use of coefficient reordering is indicated in a video parameter set, sequence parameter set, picture parameter set, sequence header, picture header, strip header, codec tree unit, codec unit, or codec block.

[0796] Figure 29 This is a flowchart representation of a method for processing video data according to the present technology. Method 2900 includes, in operation 2910, performing a conversion between a current block of video and a video bitstream according to rules. The rules specify that the current block is divided into at least two parts in response to the current block having a specific pattern. At least one of the at least two parts does not have a non-zero residual indicated in the bitstream.

[0797] In some embodiments, a specific mode includes an intra-block copy mode. In some embodiments, syntax elements indicating whether a position-based transform is applied are omitted from the bitstream. In some embodiments, an identity transform and / or a DCT2 transform are applied to the current block. In some embodiments, syntax elements indicating whether a sub-block transform is used are included in the bitstream. In some embodiments, a specific mode includes an inter-frame encoding / decoding mode. In some embodiments, only one of the identity transform or the DCT2 transform is applied to the current block.

[0798] In some embodiments, the current block has a size of M×N, and at least two parts comprise a first part of (M / k)×N and a second part of (MM / k)×N, where k is an integer greater than 1. In some embodiments, the current block has a size of M×N, and at least two parts comprise a first part of M×(N / k) and a second part of M×(NN / k), where k is an integer greater than 1. In some embodiments, the current block has a size of M×N, and at least two parts comprise a first part of (M / k)×(N / k) and a second part having an L-shape, where k is an integer greater than 1. In some embodiments, k is 2, 4, 8, or 16.

[0799] In some embodiments, a syntax element indicating whether a portion has a non-zero residual is conditionally included in the bitstream. In some embodiments, portions indicating non-zero residuals are included in the bitstream. In some embodiments, a transform skip mode that encodes the coefficients of a video block without applying a non-identity transform applies to the current block in response to constraints satisfying the dimensions of the current block. In some embodiments, the constraints specify (W <= T1 and H <= T2), where W is the width of the current block, H is the height of the current block, and T1 and T2 are integers. In some embodiments, T1 and T2 are equal to 32 or 64. In some embodiments, the constraints specify (W > T3 or H > T4), where W is the width of the current block, H is the height of the current block, and T3 and T4 are integers. In some embodiments, T3 and T4 are equal to 4 or 8.

[0800] In some embodiments, a transform skip mode is applied to portions with non-zero residuals. In some embodiments, the current block is indicated in a video parameter set, sequence parameter set, picture parameter set, sequence header, picture header, stripe header, codec tree unit, codec unit, or block as to whether the current block is a PRB and / or how the current block is segmented. In some embodiments, the bitstream includes multi-level signaling notifications to indicate whether and / or how the current block is segmented. In some embodiments, a first indication in a first level indicates whether a PRB is enabled, and a second indication in a second level indicates whether a PRB is applied. In some embodiments, the first level includes a sequence level or a picture level, and the second level includes a block level. In some embodiments, the first level includes a picture header, and the second level includes a sequence header. In some embodiments, it is determined during the transformation whether the current block is a PRB and / or how the current block is segmented.

[0801] In some embodiments, the conversion includes encoding the video into a bitstream. In some embodiments, the conversion includes decoding the bitstream to generate the video.

[0802] In this document, the term "video processing" can refer to video encoding, video decoding, video compression, or video decompression. For example, a video compression algorithm can be applied during the conversion from the pixel representation of a video to the corresponding bitstream, and vice versa. As defined in the syntax, the bitstream of the current video block can, for example, correspond to bits juxtaposed or scattered at different positions within the bitstream. For example, a macroblock can be encoded based on the residual error values ​​after transformation and encoding / decoding, and also using bits from the header and other fields in the bitstream. Furthermore, during the conversion, the decoder can parse the bitstream based on determinations as described in the solutions above, knowing that some fields may or may not be present. Similarly, the encoder can determine whether certain syntax fields are included or excluded, and generate the codec representation accordingly by including or excluding syntax fields from the codec representation.

[0803] The solutions and other solutions, examples, embodiments, modules, and functional operations disclosed in this document can be implemented in digital electronic circuits, or computer software, firmware, or hardware, including the structures disclosed in this document and their structural equivalents, or combinations of one or more of the foregoing. The disclosed embodiments and other embodiments can be implemented as one or more computer program products, such as one or more modules of computer program instructions encoded on a computer-readable medium for execution by or control of the operation of a data processing apparatus. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of substances affecting machine-readable propagation signals, or one or more combinations thereof. The term "data processing apparatus" includes all means, devices, and machines for processing data, such as programmable processors, computers, or multiple processors or computers. In addition to hardware, the apparatus may also include code that creates an execution environment for the computer program in question, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination thereof. Propagation signals are artificially generated signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode information for transmission to a suitable receiver device.

[0804] Computer programs (also known as programs, software, software applications, scripts, or code) can be written in any programming language, including compiled or interpreted languages, and can be deployed in any form, including as standalone programs or modules, components, subroutines, or other units suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a file portion that holds other programs or data (e.g., one or more scripts stored in a markup language document), a single file dedicated to a related program, or multiple coordination files (e.g., a file storing one or more modules, subroutines, or code portions). A computer program can be deployed to execute on a single computer, or on multiple computers located at one site or distributed across multiple sites and interconnected through a communication network.

[0805] The processes and logic flows described in this document can be executed by one or more programmable processors, which execute one or more computer programs to perform functions by manipulating input data and generating output. The processes and logic flows can also be executed by dedicated logic circuitry, and the devices can be implemented as dedicated logic circuitry, such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits).

[0806] For example, processors suitable for executing computer programs include general-purpose and special-purpose microprocessors, as well as any one or more processors in any type of digital computer. Typically, a processor receives instructions and data from read-only memory or random access memory, or both. The basic components of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include, or be operatively coupled to, receiving or transferring data to one or more mass storage devices (e.g., magnetic disks, magneto-optical disks, or optical disks) for storing data. However, a computer does not require such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM optical disks. The processor and memory may be supplemented by or incorporated into special-purpose logic circuitry.

[0807] Although this patent document contains numerous details, these details should not be construed as limiting the scope of any subject matter or claimed content, but rather as descriptions of features characteristic of specific embodiments of a particular technology. Certain features described in the context of individual embodiments in this patent document may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. Furthermore, although the foregoing features may be described as functioning in a particular combination, or even initially claimed to be so, in some cases one or more features may be removed from the claimed combination, and the claimed combination may refer to a sub-combination or a variation of a sub-combination.

[0808] Similarly, although these operations are described in a specific order in the accompanying drawings, this should not be construed as requiring that such operations be performed in the specific order or sequence shown, or requiring that all shown operations be performed to obtain the desired result. Furthermore, the separation of various system components in the embodiments described in this patent document should not be construed as requiring such separation in all embodiments.

[0809] Only some implementations and examples are described, and other implementations, enhancements and variations may be made based on the content described and illustrated in this patent document.

Claims

1. A method for processing video data, comprising: Perform the conversion between the current block of the video and the video bitstream according to the rules. The rule specifies the use of the first encoding / decoding information of the current block to determine the coefficient reordering. Through the coefficient reordering, the coefficients of the current block parsed from the bitstream are reordered before the application of the inverse quantization process, inverse transform process, or reconstruction process. The rule specifies whether to use a transform skip mode for the current block based on the second codec information of the current block, wherein the transform skip mode is a codec mode for skipping the predictive residual transform of the video block, and The second encoding / decoding information includes whether the Sub-Block Transform (SBT) is applicable to the encoding / decoding unit including the current block, whether the current block is encoded / decoded in inter-frame mode, and whether the width and height of the current block are less than a threshold. The transform skip mode is allowed to be used for the current block if the following conditions are met: the value of the higher-level enable syntax element indicates that the transform skip mode is enabled, the width and height of the current block are less than the threshold, the current block is encoded and decoded in inter-frame mode, and the SBT is applied to the current block.

2. The method of claim 1, wherein, The first encoding / decoding information includes the transformation type.

3. The method of claim 2, wherein, The use of the coefficient reordering is based on whether the transformation skip mode is applied to the current block.

4. The method of claim 1, wherein, The first encoding / decoding information includes whether the current block is an intra-frame encoding / decoding block.

5. The method of claim 2, wherein, The coefficients are reordered at least based on the fact that the transform type is a transform skip mode and the current block is an intra-frame codec block.

6. The method of claim 1, wherein, Whether the transform skip mode is used for the current block is also determined based on the higher-level enable syntax element included in the image header, sequence header, image parameter set, video parameter set, or stripe header.

7. The method of claim 1, wherein, The threshold is equal to 64.

8. The method of claim 1, wherein, The current block is a transform block.

9. The method of claim 1, wherein, The conversion includes encoding the video into the bitstream.

10. The method of claim 1, wherein, The conversion includes decoding the video from the bitstream.

11. The method according to claim 1, further comprising: Perform the conversion between the first block of the video and the bitstream of the video according to the rules. The rule specifies that, in response to the first block having a specific pattern, the first block is divided into at least two parts, wherein at least one of the at least two parts does not have a non-zero residual indicated in the bitstream.

12. The method according to claim 11, wherein, The specific modes include intra-frame block copy mode.

13. The method according to claim 12, wherein, Syntax elements indicating whether position-based transformations are applied are omitted from the bitstream.

14. The method according to claim 12, wherein, Identity transformations and / or DCT2 transformations are applied to the first block.

15. The method according to claim 12, wherein, Syntax elements indicating whether subblock transformations are used are included in the bitstream.

16. The method according to claim 11, wherein, The specific modes include inter-frame encoding / decoding modes.

17. The method according to claim 16, wherein, Only one of the identity transformation or the DCT2 transformation is applied to the first block.

18. The method according to claim 11, wherein, The first block has a size of M×N, and wherein the at least two parts comprise a first part of (M / k)×N and a second part of (MM / k)×N, where k is an integer greater than 1.

19. The method according to claim 11, wherein, The first block has a size of M×N, and wherein the at least two parts comprise a first part of M×(N / k) and a second part of M×(NN / k), where k is an integer greater than 1.

20. The method according to claim 11, wherein, The first block has a size of M×N, and wherein the at least two parts comprise a first part of (M / k)×(N / k) and a second part of an L shape, where k is an integer greater than 1.

21. The method according to any one of claims 18 to 20, wherein, k can be 2, 4, 8, or 16.

22. The method according to claim 11, wherein, Syntax elements indicating whether a portion has a non-zero residual are conditionally included in the bitstream.

23. The method according to claim 11, wherein, The portion of the bitstream that has a non-zero residual is indicated.

24. The method according to claim 11, wherein, In response to the constraint that the dimensions of the first block satisfy the requirement, a transform skip mode for encoding and decoding the coefficients of the video block without applying a non-identity transformation is applied to the first block.

25. The method according to claim 24, wherein, The constraints are specified as (W <= T1 and H <= T2), where W is the width of the first block, H is the height of the first block, and T1 and T2 are integers.

26. The method according to claim 25, wherein, T1 and T2 are equal to 32 or 64.

27. The method according to any one of claims 24-26, wherein, The constraint specifies (W>T3 or H>T4), where W is the width of the first block, H is the height of the first block, and T3 and T4 are integers.

28. The method according to claim 27, wherein, T3 and T4 are equal to 4 or 8.

29. The method according to claim 11, wherein, The transform skip mode applies to sections with non-zero residuals.

30. The method according to claim 11, wherein, The video parameter set, sequence parameter set, image parameter set, sequence header, image header, strip header, codec tree unit, codec unit, or block indicates whether the first block is a PRB and / or how the first block is segmented.

31. The method according to claim 30, wherein, The bitstream includes multi-level signaling notifications to indicate whether and / or how the first block is segmented.

32. The method according to claim 31, wherein, The first indication in the first level indicates whether the PRB is enabled, and the second indication in the second level indicates whether the PRB is applied.

33. The method according to claim 32, wherein, The first level includes either the sequence level or the image level, and the second level includes the block level.

34. The method according to claim 32, wherein, The first level includes an image header, and the second level includes a sequence header.

35. The method according to claim 11, wherein, During the conversion, it is determined whether the first block is a PRB and / or how the first block is segmented.

36. The method according to any one of claims 11 to 20, 22-26, 29-35, wherein, The conversion includes encoding the video into the bitstream.

37. The method according to any one of claims 11 to 20, 22-26, 29-35, wherein, The conversion includes decoding the video from the bitstream.

38. An apparatus for processing video data, comprising a processor and a non-transitory memory having instructions thereon, wherein, When the instruction is executed by the processor, the processor: Perform the conversion between the current block of the video and the bitstream of the video according to the rules. The rule specifies the use of the first encoding / decoding information of the current block to determine the coefficient reordering. Through the coefficient reordering, the coefficients of the current block parsed from the bitstream are reordered before the application of the inverse quantization process, inverse transform process, or reconstruction process. The rule specifies whether to use a transform skip mode for the current block based on the second codec information of the current block, wherein the transform skip mode is a codec mode for skipping the predictive residual transform of the video block, and The second encoding / decoding information includes whether the Sub-Block Transform (SBT) is applicable to the encoding / decoding unit including the current block, whether the current block is encoded / decoded in inter-frame mode, and whether the width and height of the current block are less than a threshold. The transform skip mode is allowed to be used for the current block if the following conditions are met: the value of the higher-level enable syntax element indicates that the transform skip mode is enabled, the width and height of the current block are less than the threshold, the current block is encoded and decoded in inter-frame mode, and the SBT is applied to the current block.

39. The apparatus according to claim 38, wherein, The first encoding / decoding information includes the transformation type. The use of the coefficient reordering is based on whether the transform skip mode is applied to the current block, and The coefficients are reordered at least based on the fact that the transform type is a transform skip mode and the current block is an intra-frame codec block.

40. The apparatus according to claim 38, wherein, The first encoding / decoding information includes whether the current block is an intra-frame encoding / decoding block.

41. A non-transitory computer-readable storage medium for storing instructions that cause a processor to perform the following operations: Perform the conversion between the current block of the video and the bitstream of the video according to the rules. in, The rule specifies the use of the first encoding / decoding information of the current block to determine the coefficient reordering. Through the coefficient reordering, the coefficients of the current block parsed from the bit stream are reordered before the application of the inverse quantization process, inverse transform process, or reconstruction process. The rule specifies whether to use a transform skip mode for the current block based on the second codec information of the current block, wherein the transform skip mode is a codec mode for skipping the predictive residual transform of the video block, and The second encoding / decoding information includes whether the Sub-Block Transform (SBT) is applicable to the encoding / decoding unit including the current block, whether the current block is encoded / decoded in inter-frame mode, and whether the width and height of the current block are less than a threshold. The transform skip mode is allowed to be used for the current block if the following conditions are met: the value of the higher-level enable syntax element indicates that the transform skip mode is enabled, the width and height of the current block are less than the threshold, the current block is encoded and decoded in inter-frame mode, and the SBT is applied to the current block.

42. The non-transitory computer-readable storage medium according to claim 41, wherein, The first encoding / decoding information includes the transformation type. The use of the coefficient reordering is based on whether the transform skip mode is applied to the current block, and The coefficients are reordered at least based on the fact that the transform type is a transform skip mode and the current block is an intra-frame codec block.

43. A non-transitory computer-readable recording medium having a computer program stored thereon, said computer program, when executed by a processor, performing the following method to generate a bitstream of video, wherein, The method includes: For the current block of the video, generate the bitstream of the video according to the rules. The rule specifies the use of the first encoding / decoding information of the current block to determine the coefficient reordering. Through the coefficient reordering, the coefficients of the current block parsed from the bit stream are reordered before the application of the inverse quantization process, inverse transform process, or reconstruction process. The rule specifies whether to use a transform skip mode for the current block based on the second codec information of the current block, wherein the transform skip mode is a codec mode for skipping the predictive residual transform of the video block, and The second encoding / decoding information includes whether the Sub-Block Transform (SBT) is applicable to the encoding / decoding unit including the current block, whether the current block is encoded / decoded in inter-frame mode, and whether the width and height of the current block are less than a threshold. The transform skip mode is allowed to be used for the current block if the following conditions are met: the value of the higher-level enable syntax element indicates that the transform skip mode is enabled, the width and height of the current block are less than the threshold, the current block is encoded and decoded in inter-frame mode, and the SBT is applied to the current block.

44. The non-transitory computer-readable recording medium according to claim 43, wherein, The first encoding / decoding information includes the transformation type. The use of the coefficient reordering is based on whether the transform skip mode is applied to the current block, and The coefficients are reordered at least based on the fact that the transform type is a transform skip mode and the current block is an intra-frame codec block.

45. A method for storing a video bitstream, comprising: The bitstream is generated by performing the video processing method described below; as well as The bitstream is stored in a non-transitory computer-readable recording medium. The video processing method includes: For the current block of the video, generate the bitstream of the video according to the rules. The rule specifies the use of the first encoding / decoding information of the current block to determine the coefficient reordering. Through the coefficient reordering, the coefficients of the current block parsed from the bit stream are reordered before the application of the inverse quantization process, inverse transform process, or reconstruction process. The rule specifies whether to use a transform skip mode for the current block based on the second codec information of the current block, wherein the transform skip mode is a codec mode for skipping the predictive residual transform of the video block, and The second encoding / decoding information includes whether the Sub-Block Transform (SBT) is applicable to the encoding / decoding unit including the current block, whether the current block is encoded / decoded in inter-frame mode, and whether the width and height of the current block are less than a threshold. The transform skip mode is allowed to be used for the current block if the following conditions are met: the value of the higher-level enable syntax element indicates that the transform skip mode is enabled, the width and height of the current block are less than the threshold, the current block is encoded and decoded in inter-frame mode, and the SBT is applied to the current block.

46. ​​A video encoding apparatus comprising a processor configured to implement the method of any one of claims 1 to 37.

47. A computer-readable storage medium having stored thereon computationally readable instructions that, when executed by a processor, implement the method of any one of claims 1 to 37.

Images