Joint component secondary transform

Abstract

To provide a method for decoding an encoded video bitstream using at least one processor.SOLUTION: A method for decoding an encoded video bitstream includes: obtaining an encoded video bitstream including encoded color components; entropy-parsing the encoded color components; dequantizing the color components and obtaining transform coefficients of the color components; applying a joint component secondary transform (JCST) on the transform coefficients of the color components, thereby generating JCST outputs; performing a backward transform on the JCST outputs, thereby obtaining residual components of the color components; and decoding the encoded video bitstream on the basis of the residual components of the color components.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of priority to U.S. Provisional Patent Application No. 63 / 020,280, filed May 5, 2020, and U.S. Patent Application No. 17 / 072,606, filed October 16, 2020, the entire contents of which are incorporated by reference herein.

[0002] The present disclosure relates generally to the field of data processing, and more particularly to video encoding and decoding. Even more particularly, embodiments of the present disclosure relate to a joint components secondary transform (JCST) technique proposal for encoding residuals from multiple color components, such as residuals from two chroma components. [Background technology]

[0003] AOMedia Video 1 (AV1) is an open video coding format designed for video transmission over the Internet. AV1 was founded in 2015 as a successor to VP9 and developed by the Alliance for Open Media (AOMedia), a group of semiconductor companies, video-on-demand providers, video content producers, software developers, and web browser vendors. Many components of the AV1 project were sourced from previous research efforts by AOMedia members, and individual contributors have been initiating experimental technology platforms for years. For example, Xiph's / Mozilla's Daala released its code in 2010. Google's experimental VP9 evolution project VP10 was announced on September 12, 2014. Cisco's Thor was released on August 11, 2015.

[0004] Building on the VP9 code base, AV1 incorporates other technologies, some of which were developed in these experimental forms. The first version (0.1.0) of the AV1 reference codec was released on April 7, 2016. AOMedia announced the release of the AV1 bitstream specification on March 28, 2018, along with a reference software-based encoder and decoder. On June 25, 2018, the validated version 1.0.0 of the AV1 specification was released. On January 8, 2019, the validated version 1.0.0 of the AV1 specification was released with Errata 1. The AV1 bitstream specification includes a reference video codec. Summary of the Invention [Means for solving the problem]

[0005] In AV1, the prediction residual signals generated for the chroma channels, e.g., Cb and Cr, are highly correlated with each other, so it is expected that reducing the statistical redundancy between the Cb and Cr prediction residuals can further improve the residual coding.

[0006] The embodiments of the present disclosure provide solutions to the above problems.

[0007] For example, a method for decoding an encoded video bitstream using at least one processor includes the steps of obtaining an encoded video bitstream including encoded color components, entropy analyzing the encoded color components, dequantizing the color components and obtaining transform coefficients for the color components, applying a joint component quadratic transform (JCST) to the transform coefficients for the color components to generate a JCST output, determining residual components for the color components by performing an inverse transform on the JCST output, and decoding the encoded video bitstream based on the residual components for the color components. [Brief description of the drawings]

[0008] [Figure 1] FIG. 2 is a schematic diagram of coding coefficients covered by a local template; [Diagram 2] FIG. 1 is a block diagram of a communication system according to an embodiment. [Diagram 3] FIG. 1 illustrates a diagram showing a placement scheme of graph-based point cloud compression (G-PCC), a G-PCC compressor, and a G-PCC decompressor in an environment according to an embodiment. [Figure 4] FIG. 2 is a schematic diagram of an encoder / decoder scheme according to an embodiment; [Diagram 5] FIG. 2 is a schematic diagram of an encoder / decoder scheme according to an embodiment; [Figure 6] FIG. 2 is a schematic diagram of pairs of Cb and Cr transform coefficients originating from two 4×2 sized blocks according to an embodiment. [Figure 7] FIG. 13 is a schematic diagram of JCST applied to two 4×2 sized Cb and Cr blocks according to an embodiment. [Figure 8] FIG. 1 is a schematic diagram of JCST utilizing four-point transform according to an embodiment. [Figure 9] 4 is a flowchart illustrating a decoding method according to an embodiment. [Figure 10] FIG. 1 illustrates a computer system suitable for implementing embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] The embodiments described herein provide methods and apparatus for encoding and / or decoding image data.

[0010] Residual coding with AV1

[0011] For a given transform unit, the AV1 coefficient encoder starts with a skip symbol, followed by the transform kernel type and the end-of-block (EOB) location of all non-zero coefficients if transform coding is not skipped. Then, each coefficient value can be mapped to multiple hierarchical layers and symbols, where a symbol layer covers the symbols and three layers of coefficients, and each coefficient value corresponds to a different hierarchical range of coefficients, namely, lower layer, middle layer, and upper layer. The lower layer corresponds to the range 0-2, the middle layer corresponds to the range 3-14, and the upper layer corresponds to the range 15 and above.

[0012] After the EOB position is coded, the lower layer, which indicates whether the coefficient magnitude is between 0 and 2, and the middle layer, which indicates whether its magnitude is between 3 and 14, are coded together in reverse scan order, then the symbol layer and the upper layer, which indicates residual values ​​with magnitude 14 or greater, are coded together in forward scan order. The remainder is entropy coded with an Exp-Golomb code. AV1 employs the traditional zigzag scan order.

[0013] Such separation allows for the allocation of rich context models to the lower layers, including transform directions such as bidirectional, horizontal, and vertical, transform sizes, and up to five adjacent coefficients for improved compression efficiency at moderate context model sizes. In the middle layers, similar context models are used as in the lower layers, in which the number of context adjacent coefficients is reduced from five to two. In the higher layers, Exp-Golomb codes are used for coding without using context models. In the symbol layer, symbols except for the DC symbol are coded using the DC symbols of adjacent transform units as context information. Other symbol bits are coded as they are without using a context model.

[0014] In Versatile Video Coding (VVC), a coding block is first divided into sub-blocks of size 4x4, and the sub-blocks in the coding block and the transform coefficients in the sub-blocks are coded according to a predefined scan order. For a sub-block that has at least one non-zero transform coefficient, the coding of the transform coefficients is divided into four scan paths.

[0015] For example, it is assumed that absLevel is the absolute value of the current transform coefficient. In the first pass, the syntax elements sig_coeff_flag (indicating absLevel greater than 0), par_level_flag (indicating the parity of absLevel), and rem_abs_gt1_flag (indicating (absLevel-1)>>1 is greater than 0) are coded; in the second pass, the syntax element rem_abs_gt2_flag (indicating absLevel greater than 4) is coded; in the third pass, the coefficient level remaining value (called abs_remainder) is coded; and, if necessary, in the fourth pass, symbol information is coded.

[0016] In order to exploit the correlation between transform coefficients, the coded coefficients covered by the local template shown in Fig. 1 are used for the context selection of the current coefficient. In the figure, the position (101) shown in black indicates the position of the current transform coefficient. The position (102) shown in shaded area indicates its five neighboring coefficients. Here, absLevel1[x][y] represents the partially reconstructed absolute level of the coefficient at position (x, y) after the first pass, d represents the diagonal position (d=x+y) of the current coefficient, numSig represents the number of non-zero coefficients in the local template, and sumAbs1 represents the sum of the partially reconstructed absolute levels absLevel1[x][y] of the coefficients covered by the local template.

[0017] When coding the current coefficient sig_coeff_flag, the context model index is selected according to sumAbs1 and the diagonal position d. More specifically, for the luma component, the context model index is determined according to the following Equation 1: ctxSig=18*max(0,state-1)+min(sumAbs1,5+(d<2?12:(d<5?6:0)), which is equivalent to the following Equation 2 and Equation 3: ctxSig=ctxIdSigTable[min(sumAbs1,5)]+ctxIdBase.

[0018] For Chroma, the context model index is determined according to equation 4: ctxSig=12*max(0,state-1+min(sumAbs1,5+(d<2?6:0). This is equivalent to equations 5 and 6: ctxIdBase=12*max(0,state-1+(d<2?6:0). Equation 6: ctxSig=ctxIdSigTable[min(sumAbs1,5)]+ctxIdBase.

[0019] Here, a scalar quantizer is used when dependent quantization is utilized and the states are derived using a state transformation process. The table ctxIdSigTable stores the context model index offsets, ctxIdSigTable[0~5] = {0, 1, 2, 3, 4, 5}.

[0020] When coding the current coefficient par_level_flag, the context model index is selected according to sumAbs1, numSig and the diagonal position d. More specifically, for the Luma component, the context model index is determined according to the following Equation 7: ctxPar=1+min(sumAbs1-numSig, 4)+(d==0?15:(d<3?10:(d<10?5:0))), which is equivalent to the following Equations 8 and 9: Equation 8: ctxIdBase=(d==0?15:(d<3?10:(d<10?5:0))). Equation 9: ctxPar=1+ctxIdTable[min(sumAbs1-numSig,4]+ctxIdBase. For chroma, the context model index is determined according to equation 10: ctxPar=1+min(sumAbs1-numSig,4)+(d==0?5:0). This is equivalent to equations 11 and 12: Equation 11: ctxIdBase=(d==0?5:0). Equation 12: ctxPar=1+ctxIdTable[min(sumAbs1-numSig,4)]+ctxIdBase.

[0021] Here, the table ctxIdTable stores the context model index offsets, ctxIdTable[0~4]={0,1,2,3,4}. When encoding the current coefficients rem_abs_gt1_flag and rem_abs_gt2_flag, the context model indexes are determined in the same way as for par_level_flag: ctxGt1=ctxPar and ctxGt2=ctxPar (Equation 13).

[0022] A different set of context models are used for rem_abs_gt1_flag and rem_abs_gt2_flag, so even if ctxGt1 is equal to ctxGt2, the context model used for rem_abs_gt1_flag is different from the context model for rem_abs_gt2_flag.

[0023] Residual Coding for Transform Skip Mode (TSM) and Differential Pulse-Code Modulation (DPCM)

[0024] In order to adapt the residual coding to the statistics and signal characteristics of the residual level of transform skip and Block Differential Pulse-Code Modulation (BDPCM) representing the quantized prediction residual (spatial domain), in addition to the residual coding scheme described in the residual coding part of AV1 above, the following modifications of the residual coding process are proposed and applied to TSM and BDPCM modes:

[0025] In the following, three coding passes are described. In the first coding pass, we first code sig_coeff_flag, coeff_sign_flag, abs_level_gt1_flag, par_level_flag in one pass. In the second coding pass, we code abs_level_gtX_flag, where X can be 3, 5, 7, .... In the third pass, we code the remaining coefficient levels. The coding passes operate at the Coefficient Group (CG) level, i.e. for each CG, three coding passes are performed.

[0026] There is no valid scan position. Because the residual signal reflects the spatial residual after prediction and no transform energy compaction is performed on the TS, there is no indication of a high probability of trailing zeros or insignificant levels occurring in the bottom right corner of the transform block. Therefore, the last valid scan position signaling is omitted in this case. Instead, the first sub-block to be processed is the sub-block in the bottom right corner of the transform block.

[0027] Next, the sub-block coding block flag (CBF) will be described. If there is no last valid scan position signaling, the sub-block CBF signaling with coded_sub_block_flag in the TS must be changed as follows.

[0028] Due to quantization, the aforementioned meaningless sequences may occur partially within a transform block, so the last valid scan position is deleted as described above and coded_sub_block_flag is coded for all sub-blocks.

[0029] The coded_sub_block_flag of the sub-block covering the DC frequency position (the top left sub-block) is a special case. In VVC draft 3, the coded_sub_block_flag of this sub-block is not signaled and is inferred to be equal to 1. If the last valid scan position is in another sub-block, there is at least one valid level other than the DC sub-block. Therefore, the coded_sub_block_flag of this sub-block is inferred to be equal to 1, but the DC sub-block may only contain zero / not valid levels. Since there is no last scan position information in the TS, the coded_sub_block_flag of each sub-block is signaled. This includes the coded_sub_block_flag of the DC sub-block, unless all other coded_sub_block_flag syntax elements are already equal to 0. In this case, the DC coded_sub_block_flag is inferred to be equal to 1 (inferDcSbCbf=1). Since there is at least one valid level in this DC sub-block, the sig_coeff_flag syntax element in the first position at (0,0) is not signaled and is derived to be equal to 1 (inferSbDcSigCoeffFlag=1) if all other sig_coeff_flag syntax elements in that DC sub-block are equal to 0.

[0030] The context modeling of the coded_sub_block_flag can be changed: the context model index can be calculated as the sum of the coded_sub_block_flag on the right and the coded_sub_block_flag below the current sub-block, but not the OR of both.

[0031] In the following, we will explain the sig_coeff_flag context modeling. The local template in the sig_coeff_flag context modeling can be modified to include only the right neighbor (NB0) of the current scan position and its lower neighbor (NB1). The offset of the context model represents the number of valid adjacent positions sig_coeff_flag[NB0]+sig_coeff_flag[NB1]. Thus, the selection of different contexts is set according to the diagonal d in the current transform block (d is removed). This creates three context models and a single context model set for coding the sig_coeff_flag flag.

[0032] The following describes the abs_level_gt1_flag and par_level_flag context modeling: A single context model is adopted for abs_level_gt1_flag and par_level_flag.

[0033] In the following, we explain the abs_remainder coding. The empirical distribution of the residual absolute levels of transform skips usually fits a Laplacian or geometric distribution, but there may be a larger instability in the absolute levels of the transform coefficients. In particular, the variance within a window of successive realizations is high for the residual absolute levels. This allows us to modify the binarization and context modeling of the abs_remainder syntax as follows:

[0034] Using a higher cutoff value in binarization, i.e. the transition points from coding with sig_coeff_flag, abs_level_gt1_flag, par_level_flag, and abs_level_gt3_flag to Rice code in abs_remainder, and using a dedicated context model for each bin position, results in higher compression efficiency. Increasing the cutoff will generate more "greater than X" flags, such as introducing abs_level_gt5_flag, abs_level_gt7_flag, etc., until the cutoff is reached. The cutoff itself is fixed at 5 (numGtFlags=5).

[0035] Similar to the local template in the sig_coeff_flag modeling context, the template for the Rice parameter derivation can be modified, i.e. only the neighbors to the left and below the current scan position are considered.

[0036] In the following, we explain the coeff_sign_flag context modeling. Due to the internal instability of the code sequence and the fact that the prediction residuals are often biased, a context model can be used to encode a symbol even if the global empirical distribution is approximately uniformly distributed. A single dedicated context model can be used to encode a symbol, and after sig_coeff_flag, the symbol can be parsed to put together all the binary numbers encoded in the context.

[0037] In the following, we explain the limitation of the context-coded binary number. The total number of context-coded binary numbers for each transform unit (TU) is limited to 2 times the TU region size. For example, the maximum number of context-coded binary numbers for a 16x8 TU is 16x8x2=256. The context-coded binary number budget is consumed at the TU level. That is, instead of an individual budget of context-coded binary numbers per CG, all CGs in the current TU share one budget of context-coded binary numbers.

[0038] Joint coding of chroma residuals

[0039] VVC draft 6 supports modes for jointly coding chroma residual. The use (activation) of a joint chroma coding mode is indicated by a TU-level flag tu_joint_cbcr_residual_flag, and the selected mode is implicitly indicated by the chroma CBF. The flag tu_joint_cbcr_residual_flag is considered present if any or both of the TU's chroma CBFs are equal to 1. In the PPS and slice header, chroma quantization parameter (QP) offset values ​​are signaled for the joint chroma residual coding mode to distinguish them from the chroma QP offset values ​​signaled for the normal chroma residual coding mode. These chroma QP offset values ​​are used to derive the chroma QP value for blocks coded by the joint chroma residual coding mode. If the corresponding joint chroma coding mode (mode 2 in Table 1) is active in a TU, during quantization and decoding of that TU, the chroma QP offset is added to the applied luma to obtain the chroma QP. For other modes (modes 1 and 3 in Table 1), the chroma QP is derived in the same way as for conventional Cb or Cr blocks. The reconstruction process of chroma residuals (resCb and resCr) from transmitted transform blocks is shown in Table 1. When this mode is activated, one single joint chroma residual block (resJointC[x][y] Table 1) is signaled, and the residual block of Cb (resCb) and the residual block of Cr (resCr) are derived taking into account information such as tu_cbf_cb, tu_cbf_cr, and CSign (signal value specified in the slice header).

[0040] The above three joint chroma coding modes are only supported for intra-coded Coding Units (CUs). For inter-coded CUs, only mode 2 is supported. Therefore, for inter-coded CUs, the syntax element tu_joint_cbcr_residual_flag is present only if both chroma cbfs are equal to 1.

[0041] [Table 1]

[0042] where the value CSign is the sign value (+1 or -1) specified in the slice header, and resJointC[][] is the transmitted residual.

[0043] Referring now to Figure 2, Figure 2 is a block diagram of a communication system 200 according to an embodiment. The communication system 200 may include at least two terminals 210 and 220 interconnected via a network 250. When transmitting data unidirectionally, the first terminal 210 may encode data at a local location for transmission to the second terminal 220 via the network 250. The second terminal 220 may receive the encoded data of the first terminal 210 from the network 250, decode the encoded data, and display the decoded data. The unidirectional transmission of data may be common in media serving applications, etc.

[0044] 2 further illustrates terminals 230 and 240 provided to support bidirectional transmission of encoded data, such as may occur during a video conference. For bidirectional transmission of data, each terminal 230 or 240 can encode data captured at a local location and transmit it to the other terminal over network 250. Each terminal 230 or 240 can also receive encoded data transmitted by the other terminal, decode the encoded data, and display the decoded data on a local display device.

[0045] In FIG. 2, the terminals 210-240 may be depicted as servers, personal computers, and smartphones, although the principles of the embodiments are not so limited. The embodiments are suitable for laptop computers, tablet computers, media players, and / or dedicated video conferencing devices. The network 250 represents any number of networks that transmit encoded data between the terminals 210-240, including, for example, wired and / or wireless communication networks. The communication network 250 may exchange data over circuit-switched and / or packet-switched channels. Representative networks include telecommunications networks, local area networks, wide area networks, and / or the Internet. For purposes of this discussion, the architecture and topology of the network 250 may not be important to the operation of the embodiments, unless otherwise described herein below.

[0046] 3 is a diagram illustrating a deployment scheme of a G-PCC compressor 303 and a G-PCC decompressor 310 in an environment according to an embodiment. The disclosed subject matter is equally applicable to other on-board applications such as, for example, video conferencing, digital TV, and storage of compressed data on digital media such as CDs, DVDs, memory sticks, etc.

[0047] The streaming system 300 may include a source 310, e.g., a digital camera, and may include a capture subsystem 313 that may create, e.g., uncompressed data 302. The larger amount of data 302 may be processed by a G-PCC compressor 303 coupled to the source 301. The G-PCC compressor 303 may include hardware, software, or a combination thereof, such that aspects of the disclosed subject matter may be realized or practiced as described in more detail below. The smaller amount of encoded data 304 may be stored on a streaming server 305 for subsequent use. One or more streaming clients 306 and 308 may access the streaming server 305 to retrieve copies 307 and 309 of the encoded data 304. The client terminal 306 may include a G-PCC decompressor 310 that decodes the incoming copy 307 of the encoded data and creates outgoing data 311 that may be displayed on a display 312 or other display device (not shown). In some streaming systems, the encoded data 304, 307, and 309 may be encoded according to a video coding / compression standard, examples of which include those standards developed by MPEG for G-PCC.

[0048] The embodiments of the present disclosure can jointly apply a quadratic transform to transform coefficients of multiple color components. The joint transform scheme proposed here can be called joint component quadratic transform (JCST). In FIG. 4, an encoder scheme for applying JCST to two color components is shown, where JCST is performed after the forward transform and before quantization.

[0049] An embodiment of the present disclosure may perform JCST after the inverse quantization transform and before the inverse (reverse) transform, as shown in FIG.

[0050] 9, LRRX is in the first block 901. The method 900 includes obtaining an encoded video bitstream that includes encoded color components.

[0051] In a second block 902, the method 900 includes entropy analyzing the encoded color components.

[0052] In a third block 903, the method 900 includes dequantizing the color components to obtain transform coefficients for the color components.

[0053] In a fourth block 904, the method 900 includes generating a joint component quadratic transform (JCST) output by applying a JCST to the transform coefficients of the color components.

[0054] According to an embodiment, a fifth block 905 may be provided. In the fifth block 905, the method 900 may include determining residual components of the color components by performing an inverse transform on the JCST output.

[0055] According to an embodiment, a sixth block 906 may be provided. In the sixth block 906, the method 900 may include decoding the encoded video bitstream based on the residual components of the color components.

[0056] According to an embodiment, the method may be reversed as an encoding method. Indeed, although this specification may refer to a particular encoding or decoding scheme, these descriptions are not limited to the particular encoding or decoding scheme, i.e., are equally applicable to both encoding and decoding schemes.

[0057] In one embodiment, the inputs of the JCST may be the Cb and Cr transform coefficients.

[0058] In another embodiment, the inputs of the JCST may be Y, Cb, and Cr transform coefficients.

[0059] In one embodiment, the JCST may be performed element-wise, such that the JCST is performed for each pair of Cb and Cr transform coefficients located at the same coordinates. Figure 6 illustrates pairs of Cb and Cr transform coefficients from two 4x2 size blocks.

[0060] In one embodiment, the JCST may be a two-point transform, and the input may be a pair of Cb and Cr transform coefficients located at the same coordinates.

[0061] In one embodiment, the JCST may be a two-point transform and the output may be a pair of secondary transform coefficients that replace the pair of Cb and Cr transform coefficients.

[0062] In one embodiment, the output pairs of Cb and Cr transform coefficients may be co-located with the Cb and Cr transform coefficient pairs used as inputs to the JCST. Figure 7 illustrates the JCST applied to two 4x2 sized Cb and Cr blocks. The output of the JCST constructing two more 4x2 sized Cb and Cr blocks is further quantized / dequantized and entropy coded / analyzed in the encoder / decoder.

[0063] In one embodiment, the output of the JCST may be less than the input: for example, the input may be a pair of Cb and Cr coefficients, while the output may be only one secondary transform coefficient.

[0064] In one embodiment, the transforms applied to the JCST may include, but are not necessarily limited to, the Hadamard transform, the discrete cosine / sine transform, and data-driven transforms such as the KLT and LGT (Line Graph Transform).

[0065] In one embodiment, the input of the JCST can come from one or more pairs (eg, three pairs) of different color components located at different coordinates.

[0066] In one embodiment, the JCST may be a four-point transform, and the inputs may be two pairs of Cb and Cr transform coefficients located at the same coordinates. An example is shown in FIG.

[0067] In one embodiment, the output may be one or more pairs of secondary transform coefficients that replace one or more pairs of Cb and Cr transform coefficients.

[0068] In one embodiment, the output of the JCST may be less than the input: for example, the input may be one or more pairs of Cb and Cr coefficients, while the output may be only two secondary transform coefficients.

[0069] In one embodiment, the transforms applied to the JCST may include, but are not necessarily limited to, the Hadamard transform, the discrete cosine / sine transform, and data-driven transforms such as the KLT and LGT (Line Graph Transform).

[0070] In one embodiment, JCST may be applied to a finite range of block sizes.

[0071] In one example, JCST may be applied to block sizes below a certain threshold, where block size may refer to block width, block height, block region size, block width and height, and maximum (or minimum) values ​​of block width and height.

[0072] In one example, JCST may be applied to block sizes equal to or greater than a certain threshold, where block size may refer to block width, block height, block region size, block width and height, and maximum (or minimum) values ​​of block width and height.

[0073] In one embodiment, whether JCST is applied may be signaled by a JCST flag at the transform block level.

[0074] In one embodiment, the JCST flag may be signaled after the transform coefficient.

[0075] In one embodiment, the JCST flag may be signaled only if at least one color component to which JCST is being applied has at least one non-zero coefficient.

[0076] In one embodiment, the JCST flag is signaled only if each color component to which JCST is applied has at least one non-zero coefficient.

[0077] In one embodiment, the JCST flag is signaled only if the total number of non-zero coefficients of a color component to which JCST is applied is greater than a predefined threshold, eg, 0, 1, 2, 3, or 4.

[0078] In one embodiment, the JCST flag is signaled only if the last non-zero coefficient of a color component to which JCST is applied is located at a point along the scan order greater than a predetermined threshold, for example 0, 1, 2, 3, or 4.

[0079] In one embodiment, whether the JCST applies is signaled by a JCST flag at the CU level (or CB level).

[0080] In one embodiment, the applicability of JCST is signaled via high level syntax, including but not limited to Video Parameter Set (VPS), Sequence Parameter Setting (SPS), Picture Parameter Set (PPS), Slice Header, Picture Header, Tile, Tile Group, or Coding Tree Unit (CTU) level.

[0081] In one embodiment, when JCST is applied, the primary transformation is of a predefined transformation type.

[0082] In one example, the predetermined transform type is DCT-2. In another example, the predetermined transform type is DST-7.

[0083] In one embodiment, the selection of the transform applied to the JCST is determined by coding information, including but not limited to the primary transform type, intra prediction mode such as intra prediction direction / angle, inter prediction mode, whether intra block copy (IntraBC) is applied, whether palette mode is applied, whether DPCM mode is applied, motion vector information (direction, magnitude), whether sub-block motion is applied, whether warped motion (affine motion) is applied.

[0084] In one embodiment, the context used for entropy coding of the flag indicating whether JCST is applied depends on neighboring block information from neighboring blocks, including but not limited to the information listed above.

[0085] The above techniques may be realized by a video encoder and / or decoder adapted for compression / decompression. The encoder and / or decoder may be realized in hardware, software, or any combination thereof, and the software, if any, may be stored on one or more non-transitory computer-readable media. For example, each of the methods (or embodiments), the encoder, and the decoder may be implemented by a processing circuit system (e.g., one or more processors or one or more integrated circuits). In one example, the one or more processors execute a program stored on the non-transitory computer-readable medium.

[0086] The above techniques may be implemented as computer software using computer readable instructions and physically stored on one or more computer readable media. For example, FIG. 10 illustrates a computer system 900 suitable for implementing some embodiments of the present disclosure.

[0087] Computer software can be coded using any suitable machine code or computer language, and can be assembled, compiled, linked, or like mechanisms to produce code containing instructions that can be executed directly by a computer central processing unit (CPU), graphics processing unit (GPU), or the like, or via interpretation, microcode execution, or the like.

[0088] The instructions may be executed in various types of computers or components thereof including, for example, personal computers, tablet computers, servers, smart phones, gaming devices, Internet of Things devices, and the like.

[0089] 10 is illustrative and is not intended to limit the scope of use or functionality of the computer software implementing the embodiments of the present disclosure, nor should the arrangement of components be interpreted as having any dependency or requirement relating to any one or combination of components illustrated in the non-limiting embodiment of computer system 900.

[0090] The computer system 900 may include certain human interface input devices. Such human interface input devices may, for example, respond to input by one or more human users through tactile input (e.g., keystrokes, swipes, data glove movements, etc.), audio input (e.g., voice, clapping, etc.), visual input (e.g., gestures, etc.), olfactory input (not shown). Human interface devices may also be used to capture some media that is not necessarily directly related to conscious human input, such as audio (voice, music, ambient sounds, etc.), images (scanned images, photographic images captured from still image cameras, etc.), video (two-dimensional video, three-dimensional video including stereoscopic video, etc.).

[0091] The input human interface devices may include one or more of a keyboard 901, a mouse 902, a trackpad 903, a touch screen 910, a data glove, a joystick 905, a microphone 906, a scanner 907, a camera 908 (only one is depicted for each item).

[0092] The computer system 900 may also include certain human interface output devices. Such human interface output devices may stimulate one or more of the human user's senses, for example, through tactile output, sound, light, and smell / taste. Such human interface output devices may include haptic output devices (e.g., there may be haptic feedback via a touch screen 910, data gloves, or joystick 905, although there may also be haptic feedback devices that do not function as input devices). For example, such devices may be audio output devices (such as speakers 909, headphones (not shown)), visual output devices (such as screens 910 including cathode ray tube (CRT) screens, liquid crystal display (LCD) screens, plasma screens, organic light-emitting diode (OLED) screens, each with or without touch screen input capabilities, each with or without haptic feedback capabilities, some of which may be capable of outputting two-dimensional visual output or three-dimensional or higher output in the form of stereographic output), virtual reality glasses (not shown), holographic displays, and smoke tanks (not shown)), and printers (not shown).

[0093] The computer system 900 may also include human accessible storage devices and their associated media, such as optical media including ROM / RW 920 having media 921 such as CD / DVDs, thumb drives 922, removable hard drives or solid state drives 923, legacy magnetic media such as tapes and floppy disks (not shown), dedicated ROM / ASIC / PLD based devices such as security dongles (not shown), etc.

[0094] Additionally, those skilled in the art should understand that the term "computer-readable medium" as used in connection with the presently disclosed subject matter does not include transmission media, carrier waves, or other transitory signals.

[0095] The computer system 900 may also include interfaces to one or more communication networks. The networks may be, for example, wireless, wired, optical. The networks may further be local, wide area, metropolitan, vehicular and industrial, real-time, delay tolerant, etc. Examples of networks include local area networks such as Ethernet, wireless LANs, cellular networks including GSM, 3G, 4G, 5G, LTE, etc., television wired or wireless wide area digital networks including cable television, satellite television, terrestrial broadcast television, vehicular and industrial including CANBus, etc. A particular network typically requires an external network interface adapter connected to a particular general-purpose data port or peripheral bus 949 (e.g., a USB port of the computer system 900, etc.). Other networks are generally integrated into the core of the computer system 900 by connecting to a system bus (e.g., an Ethernet interface to a PC computer system or a cellular network interface) as described below. Using any of these networks, the computer system 900 may communicate with other entities. Such communications may be one-way receive only (e.g., broadcast television), one-way transmit only (e.g., to the CANbus of a particular CANbus device), or two-way (e.g., to other computer systems using local or wide area digital networks). Such communications include communications to cloud computing environment 955. Specific protocols and protocol stacks may be used for each such network or network interface.

[0096] The aforementioned human interface devices, human accessible storage devices, and network interface 954 can be connected to core 940 of computer system 900 .

[0097] The core 940 may include one or more central processing units (CPUs) 941, graphics processing units (GPUs) 942, dedicated program processing units in the form of field programmable gate areas (FPGAs) 943, hardware accelerators for specific tasks 944, etc. These devices may be connected via a system bus 948 along with read only memory (ROM) 945, random access memory 946, internal mass storage 947 such as an internal hard drive, SSD, etc. that is not accessible to the user. In some computer systems, the system bus 948 may be accessed in the form of one or more physical plugs to allow expansion with additional CPUs, GPUs, etc. Peripherals may be connected to the core's system bus 948 directly or via a peripheral bus 949. Peripheral bus architectures include PCI, USB, etc. A graphics adapter 950 may be included in the core 940.

[0098] The CPU 941, GPU 942, FPGA 943, and accelerator 944 can execute certain instructions that, in combination, can constitute the aforementioned computer code. This computer code can be stored in ROM 945 or RAM 946. Transient data can also be stored in RAM 946. And permanent data can be stored, for example, in internal mass storage device 947. Cache memory, which can be closely associated with one or more of the CPU 941, GPU 942, mass storage 947, ROM 945, RAM 946, etc., can be used to enable fast storage and retrieval from any memory device.

[0099] The computer readable medium may bear computer code for performing various computer-implemented operations. The medium and computer code may be those specially designed and constructed for the purposes of the present disclosure, or they may be of the kind well known and available to those skilled in the computer software arts.

[0100] As an example, and not by way of limitation, computer system 900 having the architecture, and in particular core 940, may provide functionality as a result of a processor (including CPU, GPU, FPGA, accelerator, etc.) executing software embodied in one or more tangible computer-readable media. Such computer-readable media may be mass storage accessible to a user, as introduced above, as well as media associated with a particular storage device having a non-transitory nature, such as core 940, such as mass storage device 947 or ROM 945 internal to the core. Software implementing various embodiments of the present disclosure may be stored in such devices and executed by core 940. Computer-readable media may include one or more storage devices or chips, depending on the particular needs. The software may cause core 940, and in particular the processor therein (including CPU, GPU, FPGA, etc.) to perform a particular process or a particular part of a particular process described herein, but includes limitations to data structures stored in RAM 946 and modifications to these data structures in accordance with the process limited by the software. Additionally or alternatively, the computer system may provide functionality as a result of hardwired logic or logic embedded in circuitry (e.g., accelerator 944), which may operate in place of or in conjunction with software to perform particular operations or particular portions of particular operations described herein. Where appropriate, such software may encompass logic, and conversely, such logic may encompass software. Where appropriate, such computer-readable media may encompass circuitry (such as an integrated circuit (IC)) that stores software for execution, circuitry embodying logic for execution, or both. The present disclosure includes any suitable combination of hardware and software.

[0101] While this disclosure describes several non-limiting embodiments, there are modifications, permutations, and various substitute equivalents that fall within the scope of this disclosure. It should therefore be understood that those skilled in the art will be able to devise numerous systems and methods that embody the principles of the disclosure and are therefore within its spirit and scope, although not explicitly shown or described herein.

[0102] ALF: Adaptive Loop Filter APS: Adaptive Parameter Set AV1:AOMedia Video 1 AV2:AOMedia Video 2 CB: Coding block CC-ALF: Cross-Component Adaptive Loop Filter CDEF: Constrained Directivity Enhancement Filter CU: coding unit, CTU: Coding Tree Unit DPCM: Differential Pulse Coded Modulation DPS: Decoding Parameter Set HDR: High Dynamic Range HEVC: High Efficiency Video Coding ISP: Intra Subpartition JCCT: Joint Chroma Component Conversion JVET: Joint Video Exploration Team LR: Loop recovery filter PDPC: Position-dependent prediction combination PPS: Image Parameter Set PU: Prediction unit SDR: Standard Dynamic Range SPS: Sequence Parameter Settings TSM:Conversion Skip Mode TU: conversion unit VVC: General Purpose Video Coding WAIP: Wide-angle Intra Prediction VPS: Video Parameter Set [Explanation of symbols]

[0103] 944 Accelerator 948 System Bus 950 Graphics Adapter 954 Network Interface

Claims

1. 1. A method for encoding a video bitstream using at least one processor, comprising: obtaining video data comprising a plurality of video frames, including a current video frame including Cb and Cr color components; obtaining a residual block for each of the Cb and Cr color components; performing a linear transform on each of the residual blocks to obtain Cb and Cr transform blocks; applying a joint components secondary transform (JCST) element-wise to the Cb and Cr transform block to generate a JCST output, the JCST being a two-point transform performed on each pair of Cb and Cr transform values ​​that are located at the same coordinate in the Cb and Cr transform block, such that for each pair of Cb and Cr transform values ​​that are located at the same coordinate, when the Cr transform value is in the Cr transform block, the Cb transform value is located at the same coordinate as the Cb transform block; performing quantization on the JCST output to obtain transform coefficients; encoding the transform coefficients; transmitting the encoded transform coefficients in a video bitstream.

2. The method of claim 1, wherein the current video frame includes Y, Cb and Cr color components.

3. The method of claim 1, wherein the JCST is applied to block sizes within a limited range.

4. The method of claim 1, further comprising the step of signaling a JCST flag indicating whether the JCST should be applied to the Cb and Cr transform blocks.

5. The method described in claim 4, wherein the JCST flag is signaled at a transform block level, a coding unit level, or a coding block level.

6. The method of claim 4, wherein the JCST flags are signaled in a high-level syntax.

7. The method of claim 1, wherein the JCST includes a second transformation determined via encoding information.

8. The method of claim 1, wherein each of the Cb and Cr transform blocks is a 4x2 block.

9. The method of claim 1, wherein the JCST includes at least one of a Hadamard transform, a discrete cosine transform, a discrete sine transform, and a data-driven transform.

10. The method of claim 1, wherein the JCST is conditionally applied based on the number of non-zero values ​​in the Cb transform block and / or the Cr transform block.

11. A computing system configured to perform a method according to any one of claims 1 to 10.

12. A computer program product for causing at least one processor to carry out a method according to any one of claims 1 to 10.

Images