Coefficient-based transform and modal signaling

Abstract

Techniques are described for explicit and implicit signaling for transform modality selection in video coding. Information derived from coefficient samples in a given transform unit (TU) or prediction unit (PU) may constrain or modify signaling of certain syntax elements at coding block (CB), TU, or PU levels. For example, various syntax elements, such as transform types and related flags / indices or quadratic transform modalities / flag indices, residual coding modalities, intra prediction modalities and inter prediction modalities, and scan order, may be disabled or constrained based on spatial locations of decoded coefficients, spatial patterns of coefficients, or correlations with coefficients in neighboring blocks. In another case, if the coefficient samples match the desired spatial mode or have other desired properties, a default transform type, a default quadratic transform type, a default intra prediction modality, and an inter prediction modality, or other block level modality, may be inferred at the decoder side. Similar ideas may be extended to infer intra block copy (IBC) flags / modalities / indices from coefficient samples, or to infer flags and modalities related to predictive coding techniques, such as block-by-block differential pulse code modulation (BDPCM), i.e., horizontal and vertical BDPCM modalities, palette modality syntax, and other block level flags / indices.

Description

[0001] Priority claim

[0002] This application benefits from the priority granted by U.S. non-provisional patent application No. 18 / 312,077, entitled “Coefficient-Based Transform And Mode Signaling,” filed on May 4, 2023, and U.S. provisional patent application No. 63 / 392,943, entitled “Coefficient-Based Transform and Mode Signaling,” filed on July 28, 2022, the disclosures of which are incorporated herein in their entirety. Background Art

[0003] The present disclosure relates to encoding and decoding of image and video data.

[0004] Conventional image / video codecs consist of an encoder and a decoder, and can compress image and video data for transmission and storage. Some examples of standardized coding techniques include H.264 (AVC), H.265 (HEVC), H.266 (VVC), and AV1. A new video encoding and decoding software called the AOM Video Model (AVM) is currently being developed by AOMedia, with the intention that the resulting specification will be the successor to the AV1 specification. Conventional codecs are block-based, and they first partition an image or video frame into smaller image regions, often referred to as "coding blocks." This partitioning is a multi-stage process in which the frame is first split into smaller coding tree units (CTUs) or super blocks (SBs). A CTU or SB can be further divided into smaller coding blocks (CBs). In Figure 1 In the , an exemplary illustration of the H.266 coding standard on the encoder side is provided. Most video coding specifications including HEVC and AV1 follow the Figure 1 Following this illustration, each input frame is first segmented into CBs.

[0005] After the partitioning stage, the video encoder can predict the pixel samples of the current block from the neighboring blocks by using intra-frame prediction. Alternatively, the codec can also use pixel information from different time frames and blocks from other frames by using inter-frame prediction techniques. Some of these prediction techniques may include the use of motion compensated prediction, time interpolation prediction, weighted prediction, or even a combination of inter-frame prediction and intra-frame prediction. The prediction stage usually reduces the spatial and / or temporal redundant information in the coded block from adjacent samples or frames. The block obtained after subtracting the predicted value (e.g., using intra-frame prediction or inter-frame prediction) from the block of interest is usually called a "residual block". The encoder can further apply a transform to the residual block using a discrete cosine transform (DCT), a discrete sine transform (DST), or a variant of other available transforms. The block on which the transform is applied is usually called a "transform unit" (TU).

[0006] The transform stage provides energy compression in the residual block by mapping the residual values ​​from the pixel domain to an alternative Euclidean space. This stage effectively reduces the number of bits required to transmit the energy compression coefficients. The video codec can also skip the transform stage. Typically, if the codec determines that it is not beneficial to perform a transform on the residual block, such as when the residual signal after prediction is already compact enough and a DCT-like transform does not provide additional compression benefits, then it is skipped.

[0007] After this transform stage, the resulting coefficients are passed through a quantizer, which reduces the number of bits required to represent the transform coefficients. In addition, optimization techniques (such as grid-based quantization or discard optimization) can be used to tune the quantized coefficients based on a rate-distortion criterion. The quantization stage can result in a significant loss of information, especially at low bitrate targets. In such cases, quantization may result in visible distortion or information loss in the image / video. The trade-off between rate (the number of bits sent in a certain period of time) and distortion is typically controlled with a quantization parameter (QP). In the entropy coding stage, the quantized transform coefficients, which typically constitute a large part of the final output bitstream, are signaled to the decoder using lossless entropy coding methods such as multi-symbol arithmetic coding (MS-AC) in AV1 / AVM and context-adaptive binary arithmetic coding (CABAC) in VVC and HEVC.

[0008] In addition to the quantized coefficients, certain encoder decisions are signaled to the decoder as side information. Some of this information may include partition type, intra-frame and inter-frame prediction modes (e.g., weighted intra-frame prediction, multiple reference line modes, etc.), transform type applied to the transform block, and / or other flags / indexes about tools such as secondary transforms. Compared with the quantized transform coefficients, this side information usually accounts for a smaller portion of the final bitstream. The decoder uses this side information to perform an inverse transform on the dequantized coefficients and reconstruct pixel samples. Additional tools including recovery, deblocking, and loop filters can also be applied to the reconstructed pixel samples to enhance the quality of the reconstructed image.

[0009]

[0010]

[0011] Table 1. Transform types in AV1 and AVM

[0012] In both the AV1 reference software and the AVM reference software, several transform candidates can be used for intra-frame coded blocks and inter-frame coded blocks. These transforms summarized in Table 1 consist of a combination of: 1) discrete cosine transform (DCT), 2) asymmetric discrete sine transform (ADST), 3) flipped MDST, and 4) identity transform (IDTX). The transform can be applied horizontally or vertically as a 1D transform, or can be applied horizontally and vertically with a separable 2D transform. IDTX skips triangular transforms vertically and horizontally and is considered a transform skip case. Once the encoder selects the appropriate transform type, the transform type is signaled to the decoder using different transform sets. Some of these transform sets are shown in Table 2. For example, the discrete triangular transform set (DTT4) in AVM contains 4 possible transform types, in which a combination of DCT and ADST can be used. When the minimum value of the height or width of the block is less than 8, the DTT4 set can be selected for the intra-frame coded block. Other transform sets containing DTT9 are also shown in Table 2, which can be used for larger inter-frame coded blocks. The use of these sets can reduce the signaling overhead for different block types and sizes when signaling of transform types is required.

[0013]

[0014] Table 2 (left hand side) shows some examples of transform set groupings used in AV1 and AVM.

[0015] Table 3 (right hand side) shows which transform types are used when signaling intra and inter blocks.

[0016] Transform Set. The set of transforms signaled depends on the minimum block width and height.

[0017] In the AVM, a secondary transform called "intra-secondary transform" (IST) can also be performed. IST is applied on top of the main transform coefficients to further compress these transform coefficients, but in contrast to the DCT-like transform, IST is data-driven using a trained inseparable kernel. The IST kernel can be selected based on the intra-frame mode, or they can be determined by the encoder based on multiple criteria (such as rate-distortion or rate-distortion-complexity criteria) and signaled to the decoder side. In the AVM, IST is only used when the main transform type is 2D DCT or 2D ADST. In addition to the transform type signaling, the encoder can also signal additional IST-related modes to the decoder to indicate its specific kernel for the current block.

[0018] Coefficient encoding in AV1 / AVM

[0019] Regardless of the transform type selected by the encoder, the resulting coefficients or prediction residuals from the transform stage are signaled to the decoder. Coefficient coding in AVM can be summarized into three parts: 1) encoding of the all-zero flag and transform type, 2) signaling of the last coefficient position or end of block (EOB) syntax, and 3) coefficient coding for transmitting the absolute value and sign of each coefficient sample.

[0020] All_Zero Flags and Transform Types

[0021] AV1 and AVM first determine the position of the last significant coefficient in the TU for a given scanning order. When all coefficients in a coded block are determined to be significant, the last coefficient position of the coded block can be assigned as the end-of-block (EOB) position. In this document, for simplicity, the term "EOB" refers to the last significant coefficient position of a coded block. If the EOB value is 0, the current TU does not have any significant coefficients and no encoding is required for the current TU. Therefore, the coefficient encoding process can be terminated for the current TU. In this case, only the TU skip flag (all-zero syntax in AV1) is signaled to indicate whether the EOB is 0. Figure 2 This is also shown in FIG, which illustrates the signaling order of syntax elements related to coefficient coding. Figure 2 In , if the EOB value is non-zero (eob>0) for a given TU, the transform type is encoded only for luma blocks. The transform type is not encoded for chroma blocks, but is inferred by the decoder from the co-located luma block or intra mode, depending on whether the TU is an intra-coded block or an inter-coded block. In addition, the IST flag and kernel type (stx_type) can be signaled based on the main transform type.

[0022] Last coefficient position or end of block (EOB) syntax

[0023] The last coefficient position or EOB syntax is explicitly encoded after all zero syntax. This EOB value determines which coefficient indices to skip during coefficient encoding and decoding. To provide an example, Figure 3 In the example for a 4×4 TU is provided in the upper left corner. If EOB=5, only the coefficients at indices 0, 1, 2, 3, and 4 are accessed and need to be decoded. Other coefficient indices (>5) are not considered during the coefficient encoding stage. The EOB value can be signaled using a multi-symbol syntax element after binarizing the EOB index value. If the value is too large, bypass coding (non-arithmetic) can be further used. In codecs such as VVC and HEVC, CABAC can signal the row and column indices associated with the EOB value (e.g., last_x and last_y) in a given TU after binarizing the x and y positions of the last significant coefficient position.

[0024] Coefficient Coding

[0025] If the coefficient index is less than the EOB value, then the coefficient index is accessed during the coefficient encoding stage. Coefficients are encoded in multiple passes using loops. These loops access each coefficient position based on a given scan order (such as zigzag, row, column, or diagonal scan). Each coefficient in a TU is first converted into a "level" value by taking its absolute value. For square blocks with 2D transforms, inverse zigzag scanning is used in AV1 and AVM to encode the level information. Figure 3 As shown, this scan starts from the lower right side of the TU in the coding loop, from coefficient position 15 until the scan index hits position 0 ( Figure 3 The numerical value in represents the order in which the coefficient positions in the coding block are scanned according to the indicated scanning direction). In case the EOB value is less than 15, the level encoding starts from the EOB value and loops until the scan index hits 0. The level value is signaled to the decoder in multiple rounds as follows:

[0026] Base range (BR): This range covers level values ​​0, 1, 2 and 3. If the level value is less than 3, then the level encoding loop is terminated here and coefficient encoding does not access the low / high ranges, as discussed next. A value of 3 indicates that for the BR round, the level value can be equal to or greater than 3. The level value is context-coded depending on neighboring level values ​​and other parameters such as transform size, plane type, etc.

[0027] Low Range (LR): This range covers level values ​​between [3-14]. The level value is contextually encoded based on neighboring level values ​​and other parameters such as transform size, plane type, etc.

[0028] High Range (HR): This range corresponds to level values ​​greater than 15. Level information above 15 is encoded with Exp-Golomb code without using context.

[0029] After encoding the level value in reverse scan order, the sign information is encoded separately using a forward scan round to the significant coefficients. The sign flag is bypass coded with 1 bit per coefficient without using a probability model. The motivation for this bypass coding is to simplify entropy coding because DCT coefficients usually have random signs. In AVM, there is newer context coding logic for special cases (e.g., when the transform type is IDTX or across different UV color planes).

[0030] In AV1, multi-symbol arithmetic coding is used to encode level information with an appropriate choice of context or probability model. These contexts can be selected based on various parameters such as transform size, color plane (luminance or chrominance) information, and the sum of previously encoded level values ​​in the spatial neighborhood. Figure 3 Several examples of how to derive context based on neighboring level values ​​are illustrated. For basic range encoding with zigzag scan, the level value of scan index #4 can be encoded by using the level values ​​in the green neighborhood (7,8,10,11,12). The level values ​​in this neighborhood are summed together to select the appropriate probability mode or context index for arithmetic coding. The green block has been decoded because the level information is decoded in reverse scan order. Similarly, the 1D transform has access to only 3 previously decoded neighboring samples. Low range encoding constrains the context derivation neighborhood for 2D transforms to within a 2x2 area. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 Encoding operations that may be used for H.266-based video encoding are illustrated.

[0032] Figure 2 The signaling order of syntax elements related to coefficient coding in AV1 and AVM based video coding is illustrated.

[0033] Figure 3 Different coefficient scanning processes for AV1 and AVM based video coding are illustrated.

[0034] Figure 4 A video encoding / decoding system suitable for applying the embodiments of the present disclosure is illustrated.

[0035] Figure 5 An exemplary transformation unit to which the embodiments of the present disclosure can be applied is illustrated. Specifically, Figure 5 (a) illustrates an 8×8 transform unit with block conditional termination at coefficient position 0, Figure 5(b) illustrates an 8×8 transform unit with block conditional termination at coefficient position 2, and Figure 5 (c) illustrates an 8x8 transform unit with block conditional termination at coefficient position 62.

[0036] Figure 6 A method for inferring a transform type based on a block end syntax according to an embodiment of the present disclosure is illustrated.

[0037] Figure 7 The application of range partitioning to an exemplary transform unit according to an embodiment of the present disclosure is illustrated.

[0038] Figure 8 The application of pattern matching to an exemplary transformation unit according to an embodiment of the present disclosure is illustrated. Specifically, Figure 8 (a) illustrates pattern matching leading to the inference of a ID vertical DCT transform to be used for encoding and decoding. Figure 8 (b) illustrates pattern matching leading to the inference of a ID horizontal DCT transform to be used for encoding and decoding.

[0039] Fig. 9 An encoding method according to an embodiment of the present disclosure is illustrated.

[0040] Fig.10 is a functional block diagram of an encoding system according to an embodiment of the present disclosure.

[0041] Fig.11 is a functional block diagram of a decoding system according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0042] In the present disclosure, a new inference and signaling method is described. This method uses information derived from coefficient samples in a given transform unit TU or prediction unit PU in order to constrain or modify the signaling of certain syntax elements at the CB, TU or PU level. For example, various syntax elements such as transform type and related flags / indexes or secondary transform modes / flag indices, residual coding modes, intra-frame prediction modes and inter-frame prediction modes, and scanning order can be disabled or constrained based on the spatial position of the decoded coefficients, the spatial pattern of the coefficients, or the correlation with the coefficients in the neighboring blocks. In another case, if the coefficient samples match the required spatial pattern or have other required properties, then the default transform type, default secondary transform type, default intra-frame prediction mode and inter-frame prediction mode or other block-level modes can be inferred on the decoder side. Similar ideas can be extended to infer intra block copy (IBC) flags / modes / indexes from coefficient samples, or to infer flags and modes related to predictive coding techniques, such as block-by-block differential pulse code modulation (BDPCM), i.e., horizontal and vertical BDPCM modes, palette mode syntax, and other block-level flags / indexes. The scheme proposed here can be used for new image and video coding specifications and their specific implementations, such as extensions of HEVC (H.265) and VVC (H.266) from MPEG / ITU-T, or extensions of AV1 of the Alliance for Open Media (AOM), such as its successor development model AVM (AOM Video Model).

[0043] The principles of the present disclosure are useful in Figure 4 The video encoding and decoding system 400 shown in FIG. Figure 4 As illustrated, a pair of terminal devices 410, 420 may be provided that communicate with each other via a network 430. The terminals 410, 420 may exchange coded video unidirectionally or bidirectionally via the network 430. For unidirectional video exchange, the first terminal 410 may have a video encoder 410 that encodes an input video into a coded representation that is bandwidth compressed compared to the input video. The first terminal 410 may transmit the coded video to the second terminal 420 via the network 430. The second terminal 420 may have a video decoder 422 that reverses the coding operation applied by the video encoder 412 and generates a decoded video stream therefrom. As discussed, the encoding and decoding operations may be lossy processes, and thus the decoded video may represent such an input video that is derived from the input video but with some processing errors.

[0044] For bidirectional video exchange, the encoding / decoding process may be repeated for video exchange in the opposite direction from terminal 420 to terminal 410. In such a specific implementation, terminal 420 may have its own video encoder 424. The video encoder may encode the second input video into an encoded representation that is bandwidth compressed compared to the input video. The second terminal 420 may transmit the second encoded video to the first terminal 410 via the network 430. The first terminal 410 may have a video decoder 414 that reverses the encoding operation applied by the second video encoder 424 and generates a second decoded video stream therefrom. Again, the encoding and decoding operations of the second video encoder 424 and the second video decoder 414 may be lossy processes that may result in loss of information if the second decoded video is compared to the second input video.

[0045] The processing operations performed by the first video encoder 414 and the first video decoder 422 may be performed independently of the processing operations performed by the second video encoder 424 and the second video decoder 414 .

[0046] This disclosure describes a new inference and signaling method that uses information derived from coefficient samples in each TU or PU in order to constrain or modify the signaling of certain syntax elements at the CB, TU or PU level. For example, syntax elements such as transform type and any related flags / indexes or secondary transform modes / flag indices, etc., residual coding modes, intra-frame prediction modes and inter-frame prediction modes, and scanning order may be disabled or constrained based on the spatial position of the coded coefficients, the spatial pattern of the coefficients, or the correlation with coefficients in neighboring blocks.

[0047] In one embodiment, the last position index or end of block identifier (again, for convenience, EOB) may be signaled to the decoder side before transform syntax signaling. Such an EOB identifier may indicate the position of the last significant coded coefficient within a TU. Based on the signaled EOB value, transform signaling may be skipped and the decoder may infer the transform type to be a default transform type, such as a 2D DCT transform. In general, the default transform type may be any transform with some examples listed in Table 1.

[0048] Figure 5 A transform unit with several exemplary EOB positions is illustrated. Figure 4 In , the number at each coefficient position represents the scanning order of the coefficient by TU. Figure 5 An exemplary zigzag scanning application is shown, but the encoder may be used as in this article Figure 3 , Figure 7 and Figure 8 The scanning orders are those shown in .

[0049] exist Figure 5 In (a), an 8×8 TU is illustrated, where the last significant coefficient position is shown at coefficient index “0”. Figure 5 (a) Example, since there is at least one non-zero coefficient (marked with an x) at coefficient position "0", this situation corresponds to EOB=1. Note that EOB=0 can be reserved to indicate that a TU has no significant coefficients (e.g., the all-zero syntax explained in Section 3.2.1). Based on the decoded EOB=1, decoder 422 ( Figure 4 ) can be inferred that for this DC-only block, the transform type is the default transform type. This logic Figure 6 , where the encoder 412 may signal a last position index indicating the position of the last non-zero coefficient in the block (step 610), and the decoder 422 may infer the transform type or transform set applied to the block based on the EOB syntax element (step 620). Alternatively, if the codec uses multiple transform sets to constrain signaling overhead, then EOB=1 may further mean that the transform set is selected by the decoder 422 to include only the 2D DCT transform in the derived set of only DC blocks.

[0050] In one embodiment, the last position index or end of block (EOB) syntax element may be signaled to the decoder side prior to transform type signaling. Based on the signaled EOB value, encoder 412 ( Figure 4 ) can skip explicit transform type signaling, and the decoder 422 can infer the transform type to be a default transform type. The default transform type can be derived or inferred based on various CB-level or TU-level syntax elements, such as intra / inter prediction modes, multiple reference line (MRL) flags / indexes, and angle increment (AD) offsets in intra prediction. In one example, if the intra prediction mode is DC prediction and EOB=1, the decoder 422 can infer that the transform type is DCT_DCT. In another case, if the intra prediction mode is set to an intra prediction mode corresponding to an angle of 157 degrees and EOB=1, the decoder 422 can infer that the transform type is equal to ADST_ADST.

[0051] In one embodiment, in VVC (H.266) and other MPEG specifications, the EOB syntax may be encoded with the x- and y-coordinates of the last coefficient position, rather than binarizing and encoding the EOB value directly according to the scan order. In this case, if both the x- and y-coordinates are signaled to the decoder 422 as 0, the current TU may be a DC-only block. In this case, the decoder 422 may further infer that the transform type is a default transform type, such as a type 2DCT applied in 2D for such a DC-only TU.

[0052] In one embodiment, if EOB < T1, where T1 is a predetermined threshold, then decoder 422 ( Figure 4 ) can infer that the transform type is the default transform. For example, if T1 = 4 and the actual value of EOB = 3, this means that the current TU can only have at most 3 valid coefficients at indices 0, 1, and 2. This is shown in Figure 5 (b), where EOB = 3 and the position of the last valid coefficient is marked at coefficient index 2 for zigzag scan. Since the EOB value is less than T1, encoder 412 ( Figure 4 ) can then skip signaling of a particular transform and use the default transform. In this case, decoder 422 can first decode the EOB value and check if it lies in the interval 0 <= EOB < T1 to also infer the default transform. The default transform can be any transform type, such as a 2D DCT.

[0053] In one embodiment, if EOB > T2, where T2 is any threshold, then decoders 422, 414 ( Figure 4 ) can infer that the transform type is the default transform. For example, for an 8×8 TU, there are at most 64 coefficient samples. This is shown in Figure 5 (c). If the threshold T2 of the TU = 60 and the actual decoded EOB value is 63, then this means that the TU has significant coefficients at very large coefficient indices closer to satisfying T2 < EOB <= 64. In this case, decoder 422 ( Figure 4 ) can infer that the transform type is a predetermined transform, such as an IDTX transform, and can skip transform signaling for the current TU. The default transform can be any transform type.

[0054] In one embodiment, if the EOB value is decoded to satisfy EOB < T1 or EOB > T2, then decoder 422 ( Figure 4 ) can further infer that only a subset of the transform needs to be used. In this case, encoder 412 no longer disables transform type signaling. However, encoder 412 can signal the transform type from a subset of the available transform candidates. For example, in Table 2, some transform sets are defined for AV1 and AVM, including DTT4 or DTT9. These transform sets are currently used in AV1. DTT4 uses 4 symbol syntax elements and DTT9 uses 9 symbol syntax elements to encode the transform type in the bitstream. If the above EOB threshold conditions or the above rules are satisfied, then encoder 412 ( Figure 4)A binary symbol can be signaled to indicate between two transform types such as: 2D DCT or 2D ADST. In one example, a new DTT2 = {DCT_DCT, ADST_ADST} set can be defined, and the encoder / decoder can select from 2 alternative transform types based on this set. The binary symbol corresponding to DTT2 can be signaled to the decoder 422( Figure 4 ) to indicate which transform type from the DTT2 set is applied. For example, the value 0 can indicate DCT_DCT (2D DCT), and the value 1 can indicate ADST_ADST (2D MDST).

[0055] In one embodiment, if EOB < T1, where T1 = 2, it means that the TU has only one coded coefficient at the DC term position (i.e., EOB = 1). This case is shown in Figure 5 (a). In this case, the decoder 422( Figure 4 ) can use the DTT2 set to decode a specific transform type from the context-coded binary symbol.

[0056] Generally, an arbitrary DTT{N} set can be defined, where this set contains N possible transform candidates. If the EOB condition described in the present disclosure is satisfied, then the encoder 412( Figure 4 ) can use this set to signal the transform type. Note that the DTT{N} set can include the main transform types indicated in Table 1 and other transform types including secondary transform modes / flags or indices.

[0057] In one embodiment, given an MxN TU, there are M×N coefficient samples residing inside the TU. Several ranges 0 <= R1 < R2 < …… < Ri < …… < (M×N) can be defined to partition the coefficient indices from 0 to M×N. The ranges Ri can be determined based on the scan order or based on the spatial position of the coefficients within the TU. If the EOB value resides in the range Ri, then the encoder 412( Figure 4 ) can skip the transform type signaling for that TU, and the default transform type (e.g., DCT_DCT, ADST_ADST, IDTX, ……) can be inferred by the decoder 422( Figure 4 ). Alternatively, different DTT{N}i sets, where i = 0, 1,......, N can be defined for different ranges Ri, as shown in Figure 7 . Depending on which region the EOB value corresponds to, the available transform types in the DTT{N}i set can be used to decode the transform type.

[0058] In one embodiment, for an 8×8 TU below, in Figure 7Range partitioning 0 <= R0 < R1 < R2 < (M × N = 64) is shown. These ranges are defined according to the index values of the zigzag scan, as Figure 7 shown. In this example, scan indices 0, 1, and 2 are shown to correspond to range R0, scan indices 3 to 35 are shown to correspond to range R1, and scan indices 36 to 63 are shown to correspond to range R2. If the EOB value corresponds to a specific region Ri, then the decoder 422( Figure 4 ) can use the available transform types in the DTT{N}i set to identify the applicable transform type.

[0059] In one embodiment, if the EOB condition explained above is satisfied, then the secondary transform syntax signaling, such as intra-secondary transform (IST) in AVM or low-frequency non-separable transform (LFNST) in VVC, can be disabled. The decoder 422( Figure 4 ) can first decode the EOB value, and based on the satisfied EOB condition, the decoder 422 can further infer that the secondary transform for the current TU is disabled and set the flags and indices related to the secondary transform (i.e., lfnst_idx for LFNST, stx_flag for IST) equal to 0.

[0060] In one embodiment, based on the EOB condition explained above, the secondary transform can always be applied to the TU, but its syntax signaling is disabled. For example, the decoder 422 can determine to apply the secondary transform after the primary transform. However, only a specific kernel of the secondary transform is used, and the modal index associated with it is not signaled. In one case, the transform type is not signaled, but instead can be inferred as 2D DCT, and additionally, secondary transforms such as LFNST and IST are applied with the default kernel and no signaling is performed. To provide some examples, in VVC, lfnst_idx = 1 can be inferred at the decoder 422 associated with mode 1. For IST, any default kernel can be used in AVM.

[0061] In one embodiment, the DTT2 transform set may be formed as follows: DTT2 = {DCT_DCT, DCT_DCT+IST or LFNST enabled (kernel 1)}. The EOB value may first be decoded and compared to a threshold or rule, as explained above. If the EOB value satisfies these conditions, a syntax is signaled to indicate whether the current block is transformed using 1) a 2D DCT transform (if the flag value is 0) or 2) a 2D DCT main transform (if the flag value is 1) with an additional secondary transform (i.e., IST or LFNST) applied. If the flag value decoded with DTT2 is equal to 1, it may be inferred that the kernel used for the secondary transform is the first available kernel (the default kernel), which will avoid the signaling overhead associated with the secondary transform kernel selection. Alternatively, DTT3 = {DCT_DCT, DCT_DCT+IST or LFNST enabled (kernel 1), DCT_DCT+IST or LFNST enabled (kernel 2)} may be formed to include more candidates for transform selection.

[0062] In one implementation, if based on the EOB conditions explained above, the Multiple Transform Set Selection (MTS) in the VVC may be modified to skip transform signaling and signaling of the transform skip flag. Alternatively, the MTS may be modified to signal only a subset of available transforms.

[0063] In one embodiment, decoder 422 ( Figure 4 ) may first decode the coefficient samples. Decoder 422 may then use these coefficient samples or a pattern of these samples to infer a transform type or other flag index. For example, if the coefficient pattern (or coefficient matrix C) matches pattern Pi, decoder 422 may infer a default transform type and set it for the current TU, which avoids the need for the encoder 412 ( Figure 4 ) provides transformation-related signaling. Figure 8 An example is shown in . Figure 8 In (a), the coefficient level values ​​(absolute coefficient values) of an 8×8 TU for a vertical transform (such as a column-by-column DCT) are shown. Since the DCT kernel is applied to each column individually, it is likely that each column has a DC term at row=0 for all columns due to the energy compression nature of the DCT transform. This is shown as region R1. The decoder 422 can view Figure 8 (a) and infers the use of a 1D vertical DCT transform and skips decoding of the transform type flag. A similar example appears in Figure 8(b) where the horizontal transform is applied independently to all rows of the 8×8 TU. The resulting coefficient pattern is different in this case, and the decoder can inspect the decoded coefficients to infer that a ID horizontal transform was applied. In such cases where the decoder 422 can infer the encoding parameters, the encoder 412 need not provide such encoding parameters in the encoded video data bitstream.

[0064] In one embodiment, the encoding process is Fig. 9 . The coefficient samples in the TU may be used to form a matrix C910. For example, this matrix C may hold the absolute values ​​of the decoded coefficients. In method (1), the matrix C may be analyzed to check whether it matches a specific pattern Pi (step 920). Fig. 9 This mode is also illustrated in . If C and Pi match under a correlation, similarity, or distance metric defined for the matrix (condition 1.1), then the decoder can infer a combination of transform types, transform sets, secondary transform sets, and modalities (step 930). If multiple modes exist (e.g., Pi and Pj match C) (condition 1.2), then a transform set such as DTT2 can be formed to signal only the transform types associated with Pi and Pj, where the remaining transform types are excluded from arithmetic coding (step 940).

[0065] In one embodiment, the pattern Pi may be manually tuned or determined empirically. In one example, Pi may consist of a weight (eg, 0, 0.5, 1) at each coefficient position. Fig. 9 The Pi illustrated in is an example of a manually tuned mask having a weight of 1.0 in the first row, a weight of 0.5 in the second row, and a weight of 0 elsewhere. In this case, a correlation, similarity score, or distance metric between C and Pi can be calculated. Such a metric can be arbitrary and will reveal a quantitative score of the similarity between the two matrices. One such example is the Frobenius norm of the difference ||C-Pi||F, which can be calculated for i=0, 1, ..., N across different coefficient pattern matrices Pi. In this case, the minimum norm can correspond to the closest pattern match C. Alternatively, C can be masked with Pi via element-by-element multiplication of the matrix C and Pi, and the function F can map the masked matrix into a numerical score. Decoder 422 ( Figure 4 ) can determine which Pi more closely represents C and infer relevant modality decisions, such as transform type, intra / inter mode, scanning order, or other block-level modes.

[0066] In an alternative embodiment, the similarity between the rows (or columns) of C and Pi can be considered instead of calculating the metric on the full matrix. For example, the correlation coefficient, cosine similarity, or other vector-by-vector metric calculated between the rows of C and Pi will reveal a per-row score of similarity (step 950). This can be used to find the closest mode of C to infer relevant modality decisions, such as transform type, intra / inter mode, scan order, or other block-level modalities (step 960).

[0067] In one embodiment, patterns of different Pi may be determined based on the data. For example, several modal decisions (such as transform type, intra-frame or inter-frame prediction mode, or other block-level decisions) may be fixed, and coefficient-level information may be collected based on these decisions to form a data set. Patterns Pi associated with these modal decisions may be determined from the collected data using unsupervised algorithms such as singular value decomposition (SVD) or principal component analysis (PCA) to reveal the dominant pattern that explains the largest variance. The idea here is to minimize the KL divergence between the identified Pi and the coefficient-level data associated with the same modal decision. Other data-driven approaches may also be used, such as neural networks or supervised algorithms. In this case, the decoder 422 ( Figure 4 ) calculates the correlation, similarity or distance metric between C and Pi to infer block-level modes, such as transform type, DPCM mode, residual coding method, and scanning order.

[0068] In one embodiment, a high-level flag or index at the sequence, frame, picture or tile level and / or alternatively at a lower level syntax structure (such as a coding tree unit (CTU) level or a super block (SB) level) may be added to indicate enabling or disabling the transform type inference rules discussed above. This may be based on the EOB value and / or based on the coefficient pattern. In an example, if the high-level flag value is 1, then this may mean that a default transform type or a subset of transforms is inferred based on the rules explained above. Alternatively, if the high-level flag value is 0, then the inference rules are disabled for the underlying blocks.

[0069] In one embodiment, a high-level mode may be defined to instruct the decoder 422 ( Figure 4 ) Infer the transform type or transform subset based on the high-level index signaled to the decoder 422. For example, the mode signaled with the index may indicate:

[0070] 0: There is no inference rule related to the transformation type.

[0071] 1: the transform type or transform set inference rule always exists in the current coding block under the higher-level unit, or

[0072] 2: Switchable inference rules that apply to specific coding blocks with additional lower-level signaling or other inference rules and without signaling.

[0073] In one embodiment, the inference rules described above may be signaled at a lower level, including at a prediction unit (PU), coding unit (CU), or TU level. For example, encoder 412 ( Figure 4 ) can signal a low-level flag to instruct the decoder 422 to infer a default transform type or a smaller subset of available transform candidates. In addition to this type of low-level signaling, additional high-level flags as described in the previous embodiments can also be used to further turn on and off low-level signaling of transform type related inference rules.

[0074] In an alternative embodiment, a high level or low level flag may be signaled to the decoder 422 ( Figure 4 ) to switch between alternative inference schemes. For example, a flag / index value of 1 signaled to decoder 422 may indicate that Fig. 9 In the illustrated inference scheme, in this case the decoder 422 may infer a default transform type or a subset of a transform set based on the distribution or pattern of coefficients within the PU / TU. Another index value of 2 may indicate that only the EOB value is used to infer the default transform type or a subset of a transform set, as described above. In general, different indices may be used to switch between inference schemes.

[0075] In a preferred embodiment, a high level or block level flag / index or syntax (tx_inference_index) may be signaled to the decoder 422 ( Figure 4 ). The index value tx_inference_index=1 may indicate that the decoder 422 always infers a default transform type (such as 2D DCT) for all blocks or a subset of blocks, alternatively the index value tx_inference_index=2 may indicate that the decoder 422 will derive the transform type based on other side information including intra prediction mode or available transform sets. For example, if the encoder 412 ( Figure 4 ) uses multiple transform sets to signal the transform type to the decoder, then tx_inference_index=2 will indicate that the decoder 422 will always use the nth transform type in the available transform sets.

[0076] In another embodiment, the signaling of the transform type based on the DC coefficient or coefficient pattern may limit the signaling of the angular transform (such as the Haar transform). For example, the angular transform T(θ)=[cos(θ)sin(θ);-sin(θ)cos(θ)] may further compress the samples in the two chrominance channels U and V jointly into two new orthogonal channels (C1, C2). The angle or index (or multiple indices) associated with such a transform may be signaled to the decoder side. If the samples before the angular transform T(θ) contain only DC terms or contain specific coefficient / residual patterns as mentioned herein, such signaling may be limited. In this case, the decoder 422( Figure 4 ) can infer that the transform is turned off or disabled.

[0077] In general, any number of color channels, such as chroma U and chroma V, may contain residual samples or transform coefficients. These residual or coefficient samples may be projected or transformed into alternative spaces C1 and C2, CN, or into a single channel C via an arbitrary transform T. This projection may be angular, (U, V) -> T(θ) -> (C1, ... CN), and the signaling of this angle index and other flags and indices associated with the transform may be constrained based on the U, V samples.

[0078] In another embodiment, in a video codec such as VVC (H.266), a block level flag (tu_union_cbcr_residual_flag) may be sent from the encoder 412 ( Figure 4 ) signals to the decoder 422 to indicate that the U, V, or (Cb or Cr) channels are jointly encoded using a transform such as T. This block level flag may be limited or disabled based on coefficient mode or if the Cb and Cr channels contain only DC terms. In general, rules may be defined that limit signaling of such a flag if only the Cb channel contains a DC term or a specific mode. Alternatively, the rules may check both Cb and Cr modes to determine if signaling should be limited.

[0079] Fig.10 1 is a functional block diagram of an encoding system 1000 according to one aspect of the present disclosure. The system 1000 may be used as a video encoder 412, 424 ( Figure 4) application. System 1000 may include a coding block encoder 1010, a coding block decoder 1020, a frame buffer 1030, a loop filter system 1040, a reference picture buffer 1050, a virtual reference picture generator 1060, a predictor 1070, a controller 1080, and a syntax unit 1090. The coding system 1000 may differentially encode the input coding block according to the prediction technique. Therefore, the video frame to be encoded can be parsed into coding blocks, and the coding block encoder 1010 processes the coding blocks on a coding block by coding block basis. The coding block encoder 1010 can present the encoded coding block data to the syntax unit 1090, which formats the encoded coding block data into a transmission syntax that conforms to the management coding protocol.

[0080] The coding block decoder 1020 may decode the coded coding block data, thereby generating decoded coding block data. The frame buffer 1030 may generate reconstructed frame data according to the decoded coding block data. The loop filter 1040 may perform one or more filtering operations on the reconstructed frame. For example, the loop filter 1040 may perform deblocking filtering, sample adaptive offset (SAO) filtering, adaptive loop filtering (ALF), maximum likelihood (ML) based filtering scheme, de-ringing, de-banding, sharpening, resolution scaling, etc. The filtered frame may be stored in the reference picture buffer 1050, where the filtered frame may be used as a prediction source for a coding block received later.

[0081] The coding block encoder 1010 may include a subtractor 1012, a transform unit 1010, a quantizer 1016, and an entropy encoder 1010. The coding block encoder 1010 may accept a coding block of input data at the subtractor 1012. The subtractor 1012 may receive a predicted coding block from the predictor 1070 and thereby generate a pixel residual array representing the difference between the input coding block and the predicted coding block. The transform unit 1010 may apply a transform to the sample data output from the subtractor 1012 to convert the data from the pixel domain to the transform coefficient domain. In some scenarios before the transform unit 1010 and / or the subtractor 1012 (e.g., when operating in a high dynamic range), the input may be reshaped or an adaptive scheme may be applied to adjust the content delivery characteristics. This adaptation may be a simple scaling based on a remapping function or a more complex pixel manipulation technique. The quantizer 1016 may perform quantization on the transform coefficients output by the transform unit 1010 according to a quantization parameter qp. The quantizer 1016 may apply a uniform or non-uniform quantization parameter; a non-uniform quantization parameter may vary across predetermined positions of the coefficient block output from the transform unit 1010. The entropy encoder 1010 may reduce the output bandwidth of the coefficient quantizer by, for example, encoding the output with a variable length codeword or using a context-adaptive binary arithmetic encoder.

[0082] The transform unit 1010 may operate in a plurality of transform modes determined by the controller 1080. The controller 1080 may select one of the transforms described above based on the controller's determination of the coding efficiency that will be obtained from the selected transform. Once a transform is selected for encoding, the controller 1080 may determine whether it is necessary to signal its selection of the transform, and if so, how to signal such selection using the techniques described above.

[0083] Quantizer 1016 may operate according to a quantization parameter qp determined by controller 1080. Techniques for generating quantization parameters are discussed below. Controller 1080 may provide syntax unit 1090 with data representing its quantization parameter selection.

[0084] The entropy encoder 1010, as its name implies, may perform entropy encoding on the data output from the quantizer 1016. For example, the entropy encoder 1010 may perform run length encoding, Huffman encoding, Golomb encoding, context-adaptive binary arithmetic coding, etc. After entropy encoding, the encoder may determine the EOB for determining whether and how to signal the transform type, as discussed above.

[0085] The coding block decoder 1020 may reverse the encoding operation of the coding block encoder 1010. For example, the coding block decoder 1020 may include a dequantizer 1022, an inverse transform unit 1024, and an adder 1026. In some scenarios after the inverse transform unit 1024 and / or the adder 1026 (e.g., when operating in a high dynamic range), the input may be reversely reshaped or remapped, typically according to the function and content characteristics applied at the encoder. The coding block decoder 1020 may obtain its input data from the output of the quantizer 1016. Although allowed, the coding block decoder 1020 does not have to perform entropy decoding on the entropy encoded data because entropy encoding is a lossless event. The dequantizer 1022 may reverse the operation of the quantizer 1016 of the coding block encoder 1010. The dequantizer 1022 may perform uniform or non-uniform dequantization as specified by the quantization parameter data qp. Similarly, the inverse transform unit 1024 may reverse the operation of the transform unit 1010. The dequantizer 1022 and the inverse transform unit 1024 may use the same quantization parameter qp and transform mode as their counterparts in the coded block encoder 1010. The quantization operation will likely truncate the data at various points, and therefore, the data recovered by the dequantizer 1022 will likely have coding errors when compared to the data presented to the quantizer 1016 in the coded block encoder 1010.

[0086] The adder 1026 may reverse the operation performed by the subtractor 1012. It may receive the same predicted coded block from the predictor 1070 as used by the subtractor 1012 in generating the residual signal. The adder 1026 may add the predicted coded block to the reconstructed residual value output by the inverse transform unit 1024 and may output the reconstructed coded block data.

[0087] As described, frame buffer 1030 may utilize the output of coding block decoder 1020 to assemble a reconstructed frame. Loop filter 1040 may perform various filtering operations on the recovered coding block data. For example, loop filter 1040 may include a deblocking filter, a sample adaptive offset ("SAO") filter, and / or other types of loop filters (not shown). Reference picture buffer 1050 may store the filtered frame data output by loop filter 1040 for use in subsequent prediction of other coding blocks.

[0088] The virtual reference picture generator 1060 may generate a virtual reference frame. The virtual reference picture generator 1060 may include a virtual reference picture predictor 1062 and a virtual reference picture buffer 1064. The virtual reference picture predictor 1062 may generate a virtual reference frame based on a reference frame stored in the reference picture buffer 1050. The virtual reference picture buffer 1064 may store the virtual reference frame thus generated for use in prediction by the predictor 1070.

[0089] Different types of prediction data are made available to the predictor 1070 for different prediction modalities. For example, for an input coding block, intra prediction obtains a prediction reference from decoded data of the same frame in which the input coding block is located. Therefore, the reference frame repository 1050 can store decoded coding block data for each frame at the time of encoding. For the same input coding block, inter prediction can obtain a prediction reference from a previously encoded and decoded frame designated as a reference frame. Therefore, the reference frame repository 1050 can store these decoded reference frames.

[0090] The predictor 1070 may supply the prediction block to the coding block encoder 1010 for use in generating the residual. The predictor 1070 may perform a prediction search operation according to intra-frame modality coding and single prediction, dual prediction and / or multi-hypothesis inter-frame modality coding. For intra-frame modality coding, the predictor 1070 may search from the coding block data in the same frame as the coding block being encoded, which provides the closest match to the input coding block. For inter-frame modality coding, the predictor 1070 may search from the coding block data of other previously encoded frames stored in the reference picture buffer 1050, which provides a match to the input coding block. From the predictions generated according to the various modes, the predictor 1070 may select the mode that achieves the lowest distortion when decoding the video at a given target bit rate. Exceptions may occur when the coding mode is selected to meet other strategies followed by the coding system 1000, such as meeting specific channel behaviors or supporting random access or data refresh strategies.

[0091] When performing a prediction search within a virtual reference frame stored in virtual reference frame buffer 1064 , predictor 1070 may perform direct motion projection and temporal motion projection as described above.

[0092] Controller 1080 may control the overall operation of encoding system 1000. Controller 1080 may select operating parameters for encoding block encoder 1010 and predictor 1070 based on analysis of input encoding blocks and also external constraints such as encoding bit rate targets and other operating parameters. Controller 1080 may determine how to represent those selections in the encoded video data output from system 1000. Controller 1080 may also select between different operating modes that the system may use to generate reference images, and may include metadata identifying the mode selected for each portion of the encoded data.

[0093] During operation, the controller 1080 may modify operating parameters of the quantizer 1016 and the transform unit 1015 at different granularities of the image data, either on a per coding block basis or on a larger granularity (e.g., per frame, per slice, per largest coding unit ("LCU") or coding tree unit (CTU), or another region). In one aspect, the quantization parameters may be modified on a per-pixel basis within an encoded frame.

[0094] Additionally, as discussed, controller 1080 may control the operation of loop filter 1050 and prediction unit 1070. For prediction unit 1070, such control may include mode selection (λ, mode to test, search window, distortion strategy, etc.), and for loop filter 1050, such control may include selection of filter parameters, reordering parameters, weighted prediction, etc.

[0095] Fig.111 is a functional block diagram of a decoding system 1100 according to one aspect of the present disclosure. The decoding system 1100 may be used as a video decoder 422, 414 ( Figure 4 The decoding system 1100 may include a syntax unit 1110, a coding block decoder 1120, a frame buffer 1130, a loop filter 1140, a reference picture buffer 1150, a virtual reference picture generator 1160, a predictor 1170, and a controller 1180.

[0096] The syntax unit 1110 may receive an encoded video data stream and may parse the encoded data into its component parts. Data representing encoding parameters may be provided to the controller 1180, while data representing the encoded residual ( Fig.10 The data output by the coded block encoder 1010) may be provided to the coded block decoder 1120. The predictor 1170 may generate a prediction block determined by the encoding parameter data provided in the coded video according to the reference frame data available in the reference picture buffer 1150 or the virtual reference picture generator 1160. The predictor 1170 may provide the prediction block to the coded block decoder 1120.

[0097] The coded block decoder 1120 may reverse the encoding operations applied by the coded block encoder 1010 ( Fig.10 ). The frame buffer 1130 may create a reconstructed frame from the decoded coding block output by the coding block decoder 1120. The loop filter 1140 may filter the reconstructed frame data. The filtered frame may be output from the decoding system 1100. The filtered frame designated as a reference frame may also be stored in the reference picture buffer 1150.

[0098] The coding block decoder 1120 may include an entropy decoder 1122, a dequantizer 1124, an inverse transform unit 1126, and an adder 1128. The entropy decoder 1122 may perform entropy decoding to reverse the process performed by the entropy encoder 1010 ( Fig.10 ). The dequantizer 1124 may reverse the operation of the quantizer 1016 of the coding block encoder 1010 ( Fig.10 Similarly, the inverse transform unit 1126 may reverse the operation of the transform unit 1010 ( Fig.10 ). They may use quantization parameters and transform modes explicitly or implicitly identified by the encoder. Since quantization may truncate data, the coded block recovered by the dequantizer 1124 will likely have coding errors ( Fig.10 ).

[0099] Adder 1128 can reverse the operation performed by subtractor 1010 ( Fig.10). It may receive a predicted coding block from the predictor 1170, which is determined by a prediction reference in the encoded video data stream. The adder 1128 may add the predicted coding block to the reconstructed residual value output by the inverse transform unit 1126, and may output the reconstructed coding block data.

[0100] As described, frame buffer 1130 may utilize the output of coded block decoder 1120 to assemble a reconstructed frame. Loop filter 1140 may perform various filtering operations on the recovered coded block data identified by the coded video data. For example, loop filter 1140 may include a deblocking filter, a sample adaptive offset (“SAO”) filter, and / or other types of loop filters. In this manner, the operation of frame buffer 1130 and loop filter 1140 mimics the operation of the corresponding frame buffer 1030 and loop filter 1040 of encoder 1000 ( Fig.10 ).

[0101] The reference picture buffer 1150 may store filtered frame data for use in subsequent prediction of other coding blocks. The reference picture buffer 1150 may store decoded frames as they are encoded for use in intra-frame prediction. The reference picture buffer 1150 may also store decoded reference frames.

[0102] The virtual reference picture generator 1160 may generate a virtual reference frame. The virtual reference picture generator 1160 may include a virtual reference picture predictor 1162 and a virtual reference picture buffer 1164. The virtual reference picture predictor 1162 may generate a virtual reference frame based on a reference frame stored in the reference picture buffer 1150. The virtual reference picture buffer 1164 may store the virtual reference frame thus generated for use in prediction by the predictor 1170.

[0103] When performing a prediction search within a virtual reference frame stored in the virtual reference frame buffer 1164 , the predictor 1170 may perform direct motion projection and temporal motion projection as described above.

[0104] As discussed, the predictor 1170 may supply the prediction block to the coded block decoder 1120 according to the coding mode identified in the coded video data. The predictor 1170 may provide predicted coded block data determined by prediction reference indicators provided in the coded video data stream.

[0105] The controller 1180 may control the overall operation of the encoding system 1100. The controller 1180 may set operating parameters of the coding block decoder 1120 and the predictor 1170 based on parameters received in the encoded video data stream. Relevant to the present discussion, these operating parameters may include quantization parameters of the dequantizer 1124 and transform modes of the inverse transform unit 1110. As discussed, the received parameters may be set at various granularities of the image data, for example, on a per coding block, per frame, per slice, per LCU / CTU basis, or based on other types of regions defined for the input image.

[0106] The foregoing discussion has described the operation of various aspects of the present disclosure in the context of video encoders and decoders. These components are often provided as electronic devices. Video decoders and / or controllers can be embedded in integrated circuits, such as application specific integrated circuits, field programmable gate arrays, and / or digital signal processors. Alternatively, they can be embedded in computer programs executed on camera devices, personal computers, laptops, tablet computers, smart phones, or computer servers. Such computer programs are typically stored in physical storage media such as electronic, magnetic, and / or optical-based storage devices, where they are read to the processor and executed. Decoders are often packaged in consumer electronic devices, such as smart phones, tablet computers, game systems, DVD players, portable media players, etc.; and they can also be packaged in consumer software applications, such as video games, media players, media editors, etc. And, of course, these components can be provided as a hybrid system that distributes functions on dedicated hardware components and programmed general-purpose processors as needed.

[0107] Several embodiments of the present invention are specifically illustrated and described herein. However, it will be appreciated that modifications and variations of the present invention are covered by the above teachings and are within the purview of the appended claims without departing from the spirit and intended scope of the invention.

Claims

1. A video encoding method, the video encoding method comprising: encoding a transform unit of a video frame by transforming the content of the transform unit from a pixel domain to a transform domain and quantizing the transform coefficients thus obtained by a quantization parameter; Determining a position of a last valid quantized transform coefficient of the transform unit; The coded data of the transform unit are transmitted to a channel according to a coding protocol in which the identifier of the position of the last significant quantized transform coefficient precedes the identification of the transform type in signaling order.

2. The method according to claim 1, further comprising: comparing a transform type applied during said transform with a default transform type defined by said encoding protocol, and When a transform type applied during the transform is the same as the default transform type defined by the encoding protocol, an identifier of the transform type is omitted from the encoded data transmitted to the channel.

3. The method according to claim 1, wherein: When the position is less than a threshold position, the transmitted coded data of the transform unit omits an identification of a transform type applied during the transform. The method of claim 1 , wherein the position is identified as its scan position within the transform unit. The method of claim 1 , wherein the location is identified by its x-coordinate, y-coordinate within the transform unit.

6. A video encoder, comprising: Processing equipment, A memory storing program instructions, which, when executed by the processing device, cause the processing device to encode the input video in the following manner: encoding a transform unit of a video frame by transforming the content of the transform unit from a pixel domain to a transform domain and quantizing the transform coefficients thus obtained by a quantization parameter; Determining a position of a last valid quantized transform coefficient of the transform unit; The coded data of the transform unit are transmitted to a channel according to a coding protocol in which the identifier of the position of the last significant quantized transform coefficient precedes the identification of the transform type in signaling order.

7. A video encoding method, the video encoding method comprising: Encoding a transform unit of a video frame by transforming the content of the transform unit from a pixel domain to a transform domain and quantizing the obtained transform coefficients by a quantization parameter; determining a mode of quantizing transform coefficients of the transform unit based on coefficient values ​​of the transform unit; When the determined mode matches a predefined mode, determining whether a transform type used in encoding matches a default transform type associated with the predefined mode; as well as When the determined mode matches a predetermined mode, the transmitted coded data of the transform unit omits an identification of a transform type applied during the transform.

8. The method according to claim 7, wherein: When the determined pattern matches the predetermined pattern, the transmitted coded data of the transform unit identifies an array of candidate transform types, the array being smaller than all transform types supported by the coding protocol.

9. The method of claim 8, wherein the array of candidate transform types comprises a two-dimensional transform.

10. The method of claim 8, wherein the array of candidate transform types comprises a one-dimensional transform. The method of claim 8 , wherein the array of candidate transform types comprises angular transforms.

12. The method of claim 8, wherein the array of candidate transform types comprises an identity transform.

13. The method according to claim 7, wherein: The encoding further includes transforming the coefficients from a first color space to a second color space.

14. The method according to claim 7, wherein: When transform coefficients of transform units from multiple color channels are jointly transformed by a common transform, a flag identifying the application of the joint transform is provided in the transmitted coded data.

15. A video decoding method, the video decoding method comprising: receiving coded data of a transform unit from a channel according to a coding protocol in which an identifier of a position of a last significant quantized transform coefficient of the transform unit precedes an identifier of a transform type in signaling order; comparing the position of the last valid quantized transform to a threshold; assigning a transform type to the transform unit based on the comparing; dequantizing the received transform coefficients received from the channel; The dequantized transform coefficients are transformed according to the default transform type.

16. The method of claim 15, wherein the position is identified by its scan position within the transform unit.

17. The method of claim 15, wherein the location is identified by its x-coordinate, y-coordinate within the transform unit.

18. The method of claim 15, wherein when the comparison indicates that an implicit signaling condition is satisfied, the assigned transform type is a default transform type determined by the coding protocol.

19. The method of claim 15, wherein the assigned transform type is identified in a coding unit level of the encoding protocol.

20. The method of claim 15, wherein the assigned transform type is inferred from a prediction modality of the transform unit.

21. The method of claim 15, wherein the assigned transform type is inferred from multiple reference line data received from the channel.

22. The method of claim 15, wherein the assigned transform type is inferred from angular delta offset data received from the channel.

23. The method according to claim 15, further comprising: determining whether a transform type identifier representing the coded transform unit is present in the received data, In case the transform type identifier does not exist in the received data of the encoded transform unit, one position range among a plurality of position ranges to which the position identifier belongs is determined, and the transform type of the encoded transform unit is determined from the determined range.

24. The method of claim 15, wherein the comparing determines that the position is less than the threshold, and if the position is less than the threshold, assigning the transform type as a two-dimensional transform.

25. The method of claim 15, wherein the comparing determines that the position is less than the threshold, and if the position is less than the threshold, assigning the transform type from a DTT2 set binary symbol received from a channel.

26. The method of claim 15, wherein the comparing determines that the position is greater than the threshold, and if the position is greater than the threshold, assigning the transform type as an identity transform.

27. The method of claim 15, wherein the comparison determining whether the position is between a first threshold and a second threshold, and In case the position is between a first threshold and a second threshold, comparing the transform type received in the channel with an array of candidate transform types, the array being smaller than all transform types supported by the encoding protocol, and One of the candidate transform types is selected based on the transform type comparison.

28. A video decoding method according to claim 15, wherein the encoding protocol supports encoding of transform units in a quadtree representation, and identifies the assigned transform type in a transform unit level at a higher level of the quadtree level representation than the transform unit level at which the position data is received.

29. The method of claim 15, further comprising deriving a secondary transform type from a processing context of the encoded transform unit.

30. A video encoding method, the video encoding method comprising: receiving coded data of a transform unit from a channel according to a coding protocol in which an identifier of the position of a last significant quantized transform coefficient of the transform unit precedes an identifier of a transform type in signaling order; determining a mode for encoding transform coefficients of the transform unit; When the determined mode matches a predefined mode, assigning a transform type to the coded transform coefficients referenced by the mode, dequantizing transform coefficients received from the channel; The dequantized transform coefficients are transformed according to the assigned transform type.

31. A video decoder, the video decoder comprising: Processing equipment, A memory storing program instructions, which, when executed by the processing device, cause the processing device to decode the encoded video in the following manner: receiving coded data of a transform unit from a channel according to a coding protocol in which an identifier of a position of a last significant quantized transform coefficient of the transform unit precedes an identifier of a transform type in signaling order; comparing the position of the last valid quantized transform to a threshold; assigning a transform type to the transform unit based on the comparing; dequantizing the received transform coefficients received from the channel; The dequantized transform coefficients are transformed according to the default transform type.

Images