Joint component secondary transform

Abstract

To provide a method for decoding an encoded video bitstream using at least one processor.SOLUTION: A method for decoding an encoded video bitstream includes: obtaining an encoded video bitstream including encoded color components; entropy-parsing the encoded color components; dequantizing the color components and obtaining transform coefficients of the color components; applying a joint component secondary transform (JCST) on the transform coefficients of the color components, thereby generating JCST outputs; performing a backward transform on the JCST outputs, thereby obtaining residual components of the color components; and decoding the encoded video bitstream on the basis of the residual components of the color components.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of priority to U.S. Provisional Patent Application No. 63 / 020,280, filed May 5, 2020, and U.S. Patent Application No. 17 / 072,606, filed October 16, 2020, the entire contents of which are incorporated herein by reference.

[0002] The present disclosure relates generally to the field of data processing, and more particularly to video encoding and decoding. Even more particularly, embodiments of the present disclosure relate to a joint components secondary transform (JCST) technique proposal for encoding residuals from multiple color components, such as residuals from two chroma components. [Background technology]

[0003] AOMedia Video 1 (AV1) is an open video coding format designed for video transmission over the Internet. AV1, the successor to VP9, ​​was established in 2015 and developed by the Alliance for Open Media (AOMedia), a group of semiconductor companies, video-on-demand providers, video content producers, software developers, and web browser vendors. Many components of the AV1 project were sourced from previous research efforts by AOMedia members, and individual contributors have initiated experimental technology platforms for years. For example, Xiph's / Mozilla's Daala released its code in 2010. Google's experimental VP9 evolution project, VP10, was announced on September 12, 2014. Cisco's Thor was released on August 11, 2015.

[0004] Building on the VP9 codebase, AV1 incorporates other technologies, some of which were developed in experimental form. The first version (0.1.0) of the AV1 reference codec was released on April 7, 2016. AOMedia announced the release of the AV1 bitstream specification on March 28, 2018, along with a reference software-based encoder and decoder. The validated version 1.0.0 of the AV1 specification was released on June 25, 2018. The validated version 1.0.0 of the AV1 specification was released on January 8, 2019, along with Errata 1. The AV1 bitstream specification includes a reference video codec. Summary of the Invention [Means for solving the problem]

[0005] In AV1, the prediction residual signals generated for the chroma channels, e.g., Cb and Cr, are highly correlated with each other, so it is expected that reducing the statistical redundancy between the Cb and Cr prediction residuals will further improve residual coding.

[0006] The embodiments of the present disclosure provide a solution to the above problems.

[0007] For example, a method for decoding a coded video bitstream using at least one processor includes the steps of obtaining a coded video bitstream including coded color components, entropy analyzing the coded color components, dequantizing the color components to obtain transform coefficients for the color components, applying a joint component quadratic transform (JCST) to the transform coefficients for the color components to generate JCST outputs, determining residual components for the color components by performing an inverse transform on the JCST outputs, and decoding the coded video bitstream based on the residual components for the color components. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 10 is a schematic diagram of coding coefficients covered by a local template. [Figure 2] 1 is a block diagram of a communication system according to an embodiment. [Figure 3] FIG. 1 illustrates a deployment scheme of graph-based point cloud compression (G-PCC), a G-PCC compressor, and a G-PCC decompressor in an environment, according to an embodiment. [Figure 4] 1 is a schematic diagram of an encoder / decoder scheme according to an embodiment; [Figure 5] 1 is a schematic diagram of an encoder / decoder scheme according to an embodiment; [Figure 6] FIG. 10 is a schematic diagram of pairs of Cb and Cr transform coefficients originating from two 4×2 sized blocks, according to an embodiment. [Figure 7] FIG. 10 is a schematic diagram of JCST applied to two 4×2 sized Cb and Cr blocks, according to an embodiment. [Figure 8] FIG. 1 is a schematic diagram of a JCST utilizing a four-point transform according to an embodiment. [Figure 9] 1 is a flowchart illustrating a decoding method according to an embodiment. [Figure 10] FIG. 1 illustrates a computer system suitable for implementing embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0009] The embodiments described herein provide methods and apparatus for encoding and / or decoding image data.

[0010] Residual coding with AV1

[0011] For a given transform unit, the AV1 coefficient encoder begins with a skip symbol, followed by the transform kernel type and, if transform coding is not skipped, the end-of-block (EOB) location of all non-zero coefficients. Each coefficient value can then be mapped to multiple layered symbols. Here, a symbol layer covers the symbol and three layers of coefficients, and each coefficient value corresponds to a different layered range of coefficients: lower, middle, and upper. The lower layer corresponds to the range 0-2, the middle layer corresponds to the range 3-14, and the upper layer corresponds to the range 15 and above.

[0012] After the EOB position is coded, the lower layer, which indicates whether the coefficient magnitude is between 0 and 2, and the middle layer, which indicates whether the magnitude is between 3 and 14, are coded together in backward scan order, and then the symbol layer and the upper layer, which indicates residual values ​​with magnitude greater than or equal to 14, are coded together in forward scan order. The remainder is entropy coded with an Exp-Golomb code. AV1 employs the conventional zigzag scan order.

[0013] This separation allows for the allocation of rich context models to lower layers, taking into account transform directions (bidirectional, horizontal, and vertical), transform sizes, and up to five neighboring coefficients for improved compression efficiency with moderate context model sizes. Middle layers use similar context models to the lower layers, but reduce the number of context neighboring coefficients from five to two. Upper layers use Exp-Golomb coding without using context models. In the symbol layer, symbols other than the DC symbol are coded using the DC symbol of neighboring transform units as context information. Other symbol bits are coded directly without using a context model.

[0014] In Versatile Video Coding (VVC), a coding block is first divided into 4x4 sub-blocks, and the sub-blocks within the coding block and the transform coefficients within the sub-blocks are coded according to a predefined scan order. For a sub-block that has at least one non-zero transform coefficient, the coding of the transform coefficients is divided into four scan paths.

[0015] For example, it is assumed that absLevel is the absolute value of the current transform coefficient. In the first pass, the syntax elements sig_coeff_flag (indicating absLevel greater than 0), par_level_flag (indicating the parity of absLevel), and rem_abs_gt1_flag ((absLevel-1)>>1 indicates greater than 0) are coded; in the second pass, the syntax element rem_abs_gt2_flag (indicating absLevel greater than 4) is coded; in the third pass, the remaining value of the coefficient level (called abs_remainder) is coded; and, if necessary, in the fourth pass, symbol information is coded.

[0016] To take advantage of the correlation between transform coefficients, the coded coefficients covered by the local template shown in FIG. 1 are used for context selection of the current coefficient. In the figure, the black position (101) indicates the position of the current transform coefficient. The shaded position (102) indicates its five neighboring coefficients. Here, absLevel1[x][y] represents the partially reconstructed absolute level of the coefficient at position (x, y) after the first pass, d represents the diagonal position (d=x+y) of the current coefficient, numSig represents the number of non-zero coefficients in the local template, and sumAbs1 represents the sum of the partially reconstructed absolute levels absLevel1[x][y] of the coefficients covered by the local template.

[0017] When encoding the current coefficient sig_coeff_flag, the context model index is selected according to sumAbs1 and the diagonal position d. More specifically, for the luma component, the context model index is determined according to the following equation 1: ctxSig=18*max(0,state-1)+min(sumAbs1,5+(d<2?12:(d<5?6:0)). This is equivalent to the following equations 2 and 3: Equation 2: ctxIdBase=18*max(0,state-1+(d<2?12:(d<5?6:0)). Equation 3: ctxSig=ctxIdSigTable[min(sumAbs1,5)]+ctxIdBase.

[0018] In the case of Chroma, the context model index is determined according to the following formula 4: ctxSig = 12 * max(0,state-1 + min(sumAbs1,5 + (d<2?6:0)). This is equivalent to the following formulas 5 and 6: Formula 5: ctxIdBase = 12 * max(0,state-1 + (d<2?6:0). Formula 6: ctxSig = ctxIdSigTable[min(sumAbs1,5)] + ctxIdBase.

[0019] Here, when dependency quantization is utilized and the states are derived using a state transformation process, a scalar quantizer is used. The table ctxIdSigTable stores the context model index offsets, ctxIdSigTable[0~5] = {0, 1, 2, 3, 4, 5}.

[0020] When encoding the current coefficient par_level_flag, the context model index is selected according to sumAbs1, numSig, and the diagonal position d. More specifically, for the luma component, the context model index is determined according to the following equation 7: ctxPar=1+min(sumAbs1-numSig, 4)+(d==0?15:(d<3?10:(d<10?5:0))). This is equivalent to the following equations 8 and 9: Equation 8: ctxIdBase=(d==0?15:(d<3?10:(d<10?5:0))). Equation 9: ctxPar = 1 + ctxIdTable[min(sumAbs1-numSig,4)] + ctxIdBase. For chroma, the context model index is determined according to the following equation 10: ctxPar = 1 + min(sumAbs1-numSig,4) + (d == 0?5:0). This is equivalent to the following equations 11 and 12: Equation 11: ctxIdBase = (d == 0?5:0). Equation 12: ctxPar = 1 + ctxIdTable[min(sumAbs1-numSig,4)] + ctxIdBase.

[0021] Here, the table ctxIdTable stores the context model index offsets, ctxIdTable[0~4]={0,1,2,3,4}. When encoding the current coefficients rem_abs_gt1_flag and rem_abs_gt2_flag, the context model indexes are determined in the same way as for par_level_flag: ctxGt1=ctxPar and ctxGt2=ctxPar (Equation 13).

[0022] Different context model sets are used for rem_abs_gt1_flag and rem_abs_gt2_flag, so even if ctxGt1 is equal to ctxGt2, the context model used for rem_abs_gt1_flag is different from the context model for rem_abs_gt2_flag.

[0023] Residual Coding for Transform Skip Mode (TSM) and Differential Pulse-Code Modulation (DPCM)

[0024] In order to adapt the residual coding to the statistics and signal characteristics of the residual levels of transform skip and Block Differential Pulse-Code Modulation (BDPCM) representing the quantized prediction residual (spatial domain), in addition to the residual coding scheme described in the residual coding part of AV1 above, the following modifications of the residual coding process are proposed and applied to TSM and BDPCM modes:

[0025] Three coding passes are described below. In the first coding pass, sig_coeff_flag, coeff_sign_flag, abs_level_gt1_flag, and par_level_flag are coded in one pass. In the second coding pass, abs_level_gtX_flag is coded, where X can be 3, 5, 7, etc. In the third pass, the remaining coefficient levels are coded. The coding passes operate at the coefficient group (CG) level, i.e., three coding passes are performed for each CG.

[0026] There is no valid scan position. Because the residual signal reflects the spatial residual after prediction and no transform energy compaction is performed on the TS, there is no indication of a high probability of trailing zeros or insignificant levels occurring in the bottom right corner of the transform block. Therefore, in this case, the last valid scan position signaling is omitted. Instead, the first sub-block to be processed is the sub-block in the bottom right corner of the transform block.

[0027] Next, the sub-block coding block flag (CBF) will be explained. If there is no last valid scan position signaling, the sub-block CBF signaling with coded_sub_block_flag in the TS must be changed as follows:

[0028] Due to quantization, the aforementioned meaningless sequences may occur partially within a transform block, so the last valid scan position is deleted as described above, and coded_sub_block_flag is coded for all sub-blocks.

[0029] The coded_sub_block_flag of the sub-block covering the DC frequency position (the top-left sub-block) is a special case. In VVC Draft 3, the coded_sub_block_flag of this sub-block is not signaled and is inferred to be equal to 1. If the last valid scan position is in another sub-block, there is at least one valid level other than the DC sub-block. Therefore, the coded_sub_block_flag of this sub-block is inferred to be equal to 1, but the DC sub-block may contain only zero / invalid levels. Since the TS does not have the last scan position information, the coded_sub_block_flag of each sub-block is signaled. This includes the coded_sub_block_flag of the DC sub-block, unless all other coded_sub_block_flag syntax elements are already equal to 0. In this case, the DC coded_sub_block_flag is inferred to be equal to 1 (inferDcSbCbf=1). Since this DC sub-block has at least one valid level, the sig_coeff_flag syntax element in the first position at (0,0) is not signaled and is derived to be equal to 1 (inferSbDcSigCoeffFlag=1) if all other sig_coeff_flag syntax elements in that DC sub-block are equal to 0.

[0030] The context modeling of coded_sub_block_flag can be changed: the context model index can be calculated as the sum of the coded_sub_block_flag on the right and the coded_sub_block_flag below the current sub-block, but not the OR of both.

[0031] The following describes the sig_coeff_flag context modeling. The local template in the sig_coeff_flag context modeling can be modified to include only the right neighbor (NB0) and its lower neighbor (NB1) of the current scan position. The offset of the context model represents the number of valid adjacent positions: sig_coeff_flag[NB0] + sig_coeff_flag[NB1]. Therefore, the selection of different contexts is set according to the diagonal d within the current transform block (d is removed). This creates three context models and a single context model set for coding the sig_coeff_flag flag.

[0032] The abs_level_gt1_flag and par_level_flag context modeling is described below: A single context model is adopted for abs_level_gt1_flag and par_level_flag.

[0033] Below we explain abs_remainder coding. The empirical distribution of the absolute residual levels of transform skips usually fits a Laplacian or geometric distribution, but there may be greater instability than in the absolute levels of the transform coefficients. In particular, the variance within a window of successive realizations is high for the absolute residual levels. This allows us to modify the binarization and context modeling of the abs_remainder syntax as follows:

[0034] Using a higher cutoff value for binarization, i.e., the transition point from coding using sig_coeff_flag, abs_level_gt1_flag, par_level_flag, and abs_level_gt3_flag to Rice coding in abs_remainder, and using dedicated context models for each bin position, results in higher compression efficiency. Increasing the cutoff generates more "greater than X" flags, such as introducing abs_level_gt5_flag, abs_level_gt7_flag, etc., until the cutoff is reached. The cutoff itself is fixed at 5 (numGtFlags=5).

[0035] Similar to the local template in the sig_coeff_flag modeling context, the template for the rice parameter derivation can be modified, i.e. only the neighbors to the left and below the current scan position are considered.

[0036] Below we explain the coeff_sign_flag context modeling. Due to the internal instability of the code sequence and the fact that the prediction residuals are often biased, a context model can be used to encode a symbol even when the global empirical distribution is approximately uniform. A single dedicated context model can be used to encode a symbol, and after sig_coeff_flag, the symbol can be parsed to compile all binary numbers coded in the context.

[0037] The following describes the limitations on the number of context-coded binary numbers. The total number of context-coded binary numbers for each transform unit (TU) is limited to 2 times the TU region size. For example, the maximum number of context-coded binary numbers for a 16x8 TU is 16x8x2=256. The budget for context-coded binary numbers is consumed at the TU level. That is, instead of an individual budget for context-coded binary numbers per CG, all CGs in the current TU share one budget for context-coded binary numbers.

[0038] Joint coding of chroma residuals

[0039] VVC Draft 6 supports a mode for jointly coding chroma residual. The use (activation) of the joint chroma coding mode is indicated by the TU-level flag tu_joint_cbcr_residual_flag, and the selected mode is implicitly indicated by the chroma CBF. The flag tu_joint_cbcr_residual_flag is considered present if either or both of the chroma CBFs of the TU are equal to 1. In the PPS and slice header, chroma quantization parameter (QP) offset values ​​are signaled for the joint chroma residual coding mode to distinguish them from the chroma QP offset values ​​signaled for the regular chroma residual coding mode. These chroma QP offset values ​​are used to derive the chroma QP value for blocks coded by the joint chroma residual coding mode. When the corresponding joint chroma coding mode (mode 2 in Table 1) is active in a TU, the chroma QP offset is added to the applied luma to obtain the chroma QP during quantization and decoding of that TU. For other modes (modes 1 and 3 in Table 1), the chroma QP is derived in the same way as for conventional Cb or Cr blocks. The reconstruction process of chroma residuals (resCb and resCr) from the transmitted transform blocks is shown in Table 1. When this mode is activated, one single joint chroma residual block (resJointC[x][y] in Table 1) is signaled, and the Cb residual block (resCb) and the Cr residual block (resCr) are derived taking into account information such as tu_cbf_cb, tu_cbf_cr, and CSign (the sign value specified in the slice header).

[0040] The above three joint chroma coding modes are only supported for intra-coded Coding Units (CUs). For inter-coded CUs, only mode 2 is supported. Therefore, for inter-coded CUs, the syntax element tu_joint_cbcr_residual_flag is present only if both chroma cbfs are equal to 1.

[0041] [Table 1]

[0042] where the value CSign is the sign value (+1 or -1) specified in the slice header, and resJointC[][] is the transmitted residual.

[0043] Referring now to FIG. 2, FIG. 2 is a block diagram of a communication system 200 according to an embodiment. The communication system 200 may include at least two terminals 210 and 220 interconnected via a network 250. When transmitting data unidirectionally, the first terminal 210 may encode data at a local location for transmission to the second terminal 220 via the network 250. The second terminal 220 may receive the encoded data of the first terminal 210 from the network 250, decode the encoded data, and display the decoded data. Unidirectional transmission of data may be common in media serving applications, etc.

[0044] 2 further illustrates terminals 230 and 240 provided to support bidirectional transmission of coded data, such as may occur during a video conference. For bidirectional transmission of data, each terminal 230 or 240 can encode data captured at its local location and transmit it to the other terminal over network 250. Each terminal 230 or 240 can also receive coded data transmitted by the other terminal, decode the coded data, and display the decoded data on a local display device.

[0045] In FIG. 2 , terminals 210-240 may be depicted as servers, personal computers, and smartphones, although the principles of the embodiments are not limited thereto. Embodiments are suitable for laptop computers, tablet computers, media players, and / or dedicated videoconferencing devices. Network 250 represents any number of networks that transmit coded data between terminals 210-240, including, for example, wired and / or wireless communication networks. Communication network 250 may exchange data over circuit-switched and / or packet-switched channels. Exemplary networks include telecommunications networks, local area networks, wide area networks, and / or the Internet. For purposes of this discussion, the architecture and topology of network 250 may not be important to the operation of the embodiments, unless otherwise described herein.

[0046] 3 illustrates a deployment scheme of a G-PCC compressor 303 and a G-PCC decompressor 310 in an environment, according to an embodiment. The disclosed subject matter is equally applicable to other on-board applications, such as, for example, video conferencing, digital TV, and storing compressed data on digital media such as CDs, DVDs, memory sticks, etc.

[0047] The streaming system 300 may include a source 310, such as a digital camera, and may include a capture subsystem 313 that can create uncompressed data 302. The larger amount of data 302 may be processed by a G-PCC compressor 303 coupled to the source 301. The G-PCC compressor 303 may include hardware, software, or a combination thereof, such that aspects of the disclosed subject matter, described in more detail below, can be realized or implemented. The smaller amount of encoded data 304 can be stored on a streaming server 305 for subsequent use. One or more streaming clients 306 and 308 can access the streaming server 305 to retrieve copies 307 and 309 of the encoded data 304. The client terminal 306 may include a G-PCC decompressor 310 that decodes the incoming copy 307 of the encoded data and creates outgoing data 311 that can be displayed on a display 312 or other display device (not shown). In some streaming systems, the encoded data 304, 307, and 309 may be encoded according to a video coding / compression standard, examples of which include those developed by MPEG for G-PCC.

[0048] Embodiments of the present disclosure can jointly apply a quadratic transform to transform coefficients of multiple color components. The joint transform scheme proposed here can be called a joint component quadratic transform (JCST). Figure 4 shows an encoder scheme that applies a JCST to two color components. Here, the JCST is performed after the forward transform and before quantization.

[0049] An embodiment of the present disclosure may perform a JCST after the inverse quantization transform and before the inverse (reverse) transform, as shown in FIG.

[0050] Referring to Figure 9, the LRRX is in the first block 901. The method 900 includes obtaining an encoded video bitstream that includes encoded color components.

[0051] In a second block 902, the method 900 includes entropy analyzing the coded color components.

[0052] In a third block 903, the method 900 includes dequantizing the color components to obtain transform coefficients for the color components.

[0053] In a fourth block 904, the method 900 includes generating a joint component quadratic transform (JCST) output by applying a JCST to the transform coefficients of the color components.

[0054] According to an embodiment, a fifth block 905 may be provided. In the fifth block 905, the method 900 may include determining residual components of the color components by performing an inverse transform on the JCST output.

[0055] According to an embodiment, a sixth block 906 may be provided. In the sixth block 906, the method 900 may include decoding the encoded video bitstream based on the residual components of the color components.

[0056] According to embodiments, this method may be reversed as an encoding method. Indeed, although this specification may refer to a particular encoding or decoding scheme, these descriptions are not limited to the particular encoding or decoding scheme, i.e., are equally applicable to both encoding and decoding schemes.

[0057] In one embodiment, the inputs of the JCST may be Cb and Cr transform coefficients.

[0058] In another embodiment, the inputs of the JCST may be Y, Cb, and Cr transform coefficients.

[0059] In one embodiment, the JCST may be performed element-wise, such that the JCST is performed for each pair of Cb and Cr transform coefficients located at the same coordinates. Figure 6 illustrates pairs of Cb and Cr transform coefficients from two 4x2 sized blocks.

[0060] In one embodiment, the JCST may be a two-point transform, and the input may be a pair of Cb and Cr transform coefficients located at the same coordinates.

[0061] In one embodiment, the JCST may be a two-point transform, and the output may be a pair of secondary transform coefficients that replace the pair of Cb and Cr transform coefficients.

[0062] In one embodiment, the output pairs of Cb and Cr transform coefficients may be co-located with the Cb and Cr transform coefficient pairs used as inputs to the JCST. Figure 7 illustrates the JCST applied to two 4x2 size Cb and Cr blocks. The output of the JCST, which constructs two more 4x2 size Cb and Cr blocks, is further quantized / dequantized and entropy coded / analyzed in the encoder / decoder.

[0063] In one embodiment, the output of the JCST may be less than the input: for example, the input may be a pair of Cb and Cr coefficients, while the output may be only one secondary transform coefficient.

[0064] In one embodiment, the transforms applied to the JCST may include, but are not necessarily limited to, the Hadamard transform, the discrete cosine / sine transform, and data-driven transforms such as KLT and LGT (Line Graph Transform).

[0065] In one embodiment, the input of the JCST can come from one or more pairs (eg, three pairs) of different color components located at different coordinates.

[0066] In one embodiment, the JCST may be a four-point transform, and the input may be two pairs of Cb and Cr transform coefficients located at the same coordinates. An example is shown in FIG.

[0067] In one embodiment, the output may be one or more pairs of secondary transform coefficients that replace one or more pairs of Cb and Cr transform coefficients.

[0068] In one embodiment, the output of the JCST may be less than the input: for example, the input may be one or more pairs of Cb and Cr coefficients, while the output may be only two secondary transform coefficients.

[0069] In one embodiment, the transforms applied to the JCST may include, but are not necessarily limited to, the Hadamard transform, the discrete cosine / sine transform, and data-driven transforms such as KLT and LGT (Line Graph Transform).

[0070] In one embodiment, JCST can be applied to a finite range of block sizes.

[0071] In one example, JCST may be applied to block sizes below a predetermined threshold, where block size may refer to block width, block height, block region size, block width and height, and maximum (or minimum) values ​​of block width and height.

[0072] In one example, JCST may be applied to block sizes equal to or greater than a predetermined threshold, where block size may refer to block width, block height, block region size, block width and height, and maximum (or minimum) values ​​of block width and height.

[0073] In one embodiment, whether JCST is applied may be signaled by a JCST flag at the transform block level.

[0074] In one embodiment, the JCST flag may be signaled after the transform coefficient.

[0075] In one embodiment, the JCST flag may be signaled only if at least one color component to which JCST is being applied has at least one non-zero coefficient.

[0076] In one embodiment, the JCST flag is signaled only if each color component to which JCST is applied has at least one non-zero coefficient.

[0077] In one embodiment, the JCST flag is signaled only if the total number of non-zero coefficients of the color components to which JCST is applied is greater than a predetermined threshold, eg, 0, 1, 2, 3, or 4.

[0078] In one embodiment, the JCST flag is signaled only if the last non-zero coefficient of the color component to which JCST is applied is located at a point along the scan order greater than a predetermined threshold, for example, 0, 1, 2, 3, or 4.

[0079] In one embodiment, whether JCST applies is signaled by a JCST flag at the CU level (or CB level).

[0080] In one embodiment, the applicability of JCST is signaled via high-level syntax, including but not limited to the Video Parameter Set (VPS), Sequence Parameter Setting (SPS), Picture Parameter Set (PPS), Slice Header, Picture Header, Tile, Tile Group, or Coding Tree Unit (CTU) level.

[0081] In one embodiment, when JCST is applied, the primary transformation is of a predetermined transformation type.

[0082] In one example, the predetermined transform type is DCT-2. In another example, the predetermined transform type is DST-7.

[0083] In one embodiment, the selection of the transform to be applied to the JCST is determined by coding information, including but not limited to the primary transform type, intra prediction mode such as intra prediction direction / angle, inter prediction mode, whether intra block copy (IntraBC) is applied, whether palette mode is applied, whether DPCM mode is applied, motion vector information (direction, magnitude), whether sub-block motion is applied, and whether warped motion (affine motion) is applied.

[0084] In one embodiment, the context used for entropy coding of the flag indicating whether JCST is applied depends on neighboring block information from neighboring blocks, including but not limited to the information listed above.

[0085] The above techniques may be realized by a video encoder and / or decoder adapted for compression / decompression. The encoder and / or decoder may be realized in hardware, software, or any combination thereof, and the software, if any, may be stored on one or more non-transitory computer-readable media. For example, each of the method (or embodiment), the encoder, and the decoder may be implemented by a processing circuit system (e.g., one or more processors or one or more integrated circuits). In one example, the one or more processors execute a program stored on a non-transitory computer-readable medium.

[0086] The techniques described above may be implemented as computer software using computer-readable instructions and physically stored on one or more computer-readable media. For example, Figure 10 illustrates a computer system 900 suitable for implementing some embodiments of the present disclosure.

[0087] Computer software can be coded using any suitable machine code or computer language, and can be assembled, compiled, linked, and the like mechanisms to produce code containing instructions that can be executed directly by a computer central processing unit (CPU), graphics processing unit (GPU), and the like, or executed via interpretation, microcode execution, and the like.

[0088] The instructions may be executed in various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, Internet of Things devices, and the like.

[0089] 10 is illustrative and is not intended to limit the scope of use or functionality of computer software implementing embodiments of the present disclosure. Nor should the arrangement of components be interpreted as having any dependency or requirement relating to any one or combination of components shown in the non-limiting embodiment of computer system 900.

[0090] The computer system 900 may include certain human interface input devices. Such human interface input devices may, for example, respond to input by one or more human users through tactile input (e.g., keystrokes, swipes, data glove movements, etc.), audio input (e.g., voice, clapping, etc.), visual input (e.g., gestures, etc.), and olfactory input (not shown). Human interface devices may also be used to capture some media that is not necessarily directly related to conscious human input, such as audio (voice, music, ambient sounds, etc.), images (scanned images, photographic images captured from a still image camera, etc.), and video (two-dimensional video, three-dimensional video including stereoscopic video, etc.).

[0091] The input human interface devices may include one or more of the following (only one of each item is depicted): a keyboard 901, a mouse 902, a trackpad 903, a touchscreen 910, a data glove, a joystick 905, a microphone 906, a scanner 907, and a camera 908.

[0092] The computer system 900 may also include certain human interface output devices. Such human interface output devices may stimulate one or more of the human user's senses, for example, through tactile output, sound, light, and smell / taste. Such human interface output devices may include haptic output devices (e.g., there may be haptic feedback via a touchscreen 910, data gloves, or joystick 905, although there may also be haptic feedback devices that do not function as input devices). For example, such devices may be audio output devices (such as speakers 909 and headphones (not shown)), visual output devices (such as screens 910 including cathode ray tube (CRT) screens, liquid crystal display (LCD) screens, plasma screens, and organic light-emitting diode (OLED) screens, each with or without touchscreen input capabilities, each with or without haptic feedback capabilities, some of which may be capable of outputting two-dimensional visual output or three-dimensional or higher-dimensional output in the form of stereographic output), virtual reality glasses (not shown), holographic displays, and smoke tanks (not shown)), and printers (not shown).

[0093] The computer system 900 may also include human-accessible storage devices and their associated media, such as optical media including ROM / RW 920 with media 921 such as CD / DVD, thumb drive 922, removable hard drive or solid state drive 923, legacy magnetic media such as tape or floppy disk (not shown), dedicated ROM / ASIC / PLD based devices such as security dongles (not shown), etc.

[0094] Additionally, those skilled in the art should understand that the term "computer-readable medium" as used in connection with the presently disclosed subject matter does not include transmission media, carrier waves, or other transitory signals.

[0095] The computer system 900 may also include interfaces to one or more communication networks. The networks may be, for example, wireless, wired, or optical. The networks may further be local, wide area, metropolitan, vehicular and industrial, real-time, delay-tolerant, and the like. Examples of networks include local area networks such as Ethernet; wireless LANs; cellular networks including GSM, 3G, 4G, 5G, LTE, and the like; television wired or wireless wide area digital networks including cable television, satellite television, and terrestrial broadcast television; and vehicular and industrial networks including CANBus and the like. Particular networks typically require an external network interface adapter connected to a particular general-purpose data port or peripheral bus 949 (e.g., a USB port on the computer system 900). Other networks are generally integrated into the core of the computer system 900 by connecting to a system bus (e.g., an Ethernet interface or a cellular network interface to a PC computer system) as described below. Using any of these networks, the computer system 900 can communicate with other entities. Such communications may be one-way receive only (e.g., broadcast television), one-way transmit only (e.g., to the CANbus of a particular CANbus device), or two-way (e.g., to other computer systems using local or wide area digital networks). Such communications include communications to cloud computing environment 955. Specific protocols and protocol stacks may be used for each such network or network interface.

[0096] The aforementioned human interface devices, human-accessible storage devices, and network interface 954 may be connected to core 940 of computer system 900 .

[0097] The core 940 may include one or more central processing units (CPUs) 941, graphics processing units (GPUs) 942, dedicated program processing devices in the form of field programmable gate arrays (FPGAs) 943, task-specific hardware accelerators 944, etc. These devices may be connected via a system bus 948, along with read-only memory (ROM) 945, random access memory 946, and internal mass storage 947, such as an internal hard drive or SSD, that is not user accessible. In some computer systems, the system bus 948 may be accessed in the form of one or more physical plugs to allow expansion with additional CPUs, GPUs, etc. Peripheral devices may be connected to the core's system bus 948 directly or via a peripheral bus 949. Peripheral bus architectures include PCI, USB, etc. A graphics adapter 950 may be included in the core 940.

[0098] The CPU 941, GPU 942, FPGA 943, and accelerator 944 can execute specific instructions that, in combination, can constitute the aforementioned computer code. This computer code can be stored in ROM 945 or RAM 946. Transient data can also be stored in RAM 946, and permanent data can be stored, for example, in internal mass storage device 947. Cache memory, which can be closely associated with one or more of the CPU 941, GPU 942, mass storage 947, ROM 945, RAM 946, etc., can be used to enable fast storage and retrieval from any memory device.

[0099] The computer-readable medium may bear computer code for performing various computer-implemented operations, and the medium and computer code may be those specially designed and constructed for the purposes of the present disclosure, or they may be of the kind well known and available to those skilled in the computer software arts.

[0100] By way of example, and not limitation, computer system 900 having the architecture, and particularly core 940, may provide functionality as a result of a processor (including a CPU, GPU, FPGA, accelerator, etc.) executing software embodied in one or more tangible computer-readable media. Such computer-readable media may be user-accessible mass storage devices, as introduced above, as well as media associated with specific storage devices of a non-transitory nature, such as core 940, such as core-internal mass storage device 947 or ROM 945. Software implementing various embodiments of the present disclosure may be stored on such devices and executed by core 940. Computer-readable media may include one or more storage devices or chips, depending on specific needs. The software may cause core 940, and particularly the processors therein (including a CPU, GPU, FPGA, etc.), to perform specific processes or portions of specific processes described herein, including limitations to data structures stored in RAM 946 and modifications to these data structures pursuant to software-defined operations. Additionally or alternatively, a computer system may provide functionality as a result of hardwired logic or logic embedded in circuitry (e.g., accelerator 944), which may operate in place of or in conjunction with software to perform particular operations or portions of particular operations described herein. Where appropriate, such software may encompass logic, and vice versa. Where appropriate, such computer-readable media may encompass circuitry (such as an integrated circuit (IC)) that stores software for execution, circuitry embodying logic for execution, or both. The present disclosure includes any suitable combination of hardware and software.

[0101] While this disclosure has described several non-limiting embodiments, there are alterations, permutations, and various substitute equivalents that fall within the scope of this disclosure. It should therefore be understood that those skilled in the art will be able to devise many systems and methods that, although not explicitly shown or described herein, embody the principles of the disclosure and are therefore within its spirit and scope.

[0102] ALF: Adaptive Loop Filter APS: Adaptive Parameter Set AV1:AOMedia Video 1 AV2:AOMedia Video 2 CB: coding block CC-ALF: Cross-Component Adaptive Loop Filter CDEF: Constrained Directional Enhancement Filter CU: coding unit, CTU: Coding Tree Unit DPCM: Differential Pulse Coded Modulation DPS: Decoding Parameter Set HDR: High Dynamic Range HEVC: High Efficiency Video Coding ISP: Intra-subpartition JCCT: Joint Chroma Component Conversion JVET: Joint Video Exploration Team LR: Loop Recovery Filter PDPC: Position-dependent prediction combination PPS: Image Parameter Set PU: Prediction Unit SDR: Standard Dynamic Range SPS: Sequence parameter settings TSM:Conversion Skip Mode TU: Conversion unit VVC: General Purpose Video Coding WAIP: Wide-angle Intra Prediction VPS: Video Parameter Set [Explanation of symbols]

[0103] 944 Accelerator 948 System Bus 950 graphics adapter 954 network interface

Claims

1. A method for decoding a video bitstream using at least one processor, comprising: obtaining an encoded video bitstream comprising encoded Cb and Cr color components; performing an entropy analysis on the coded Cb and Cr color components; dequantizing the Cb and Cr color components to obtain transform coefficients of the Cb and Cr color components, the transform coefficients of the Cb and Cr color components including Cb and Cr transform coefficients located at the same coordinates in the Cb and Cr transform blocks, respectively; applying, element-by-element, a joint component quadratic transform (JCST) to the Cb and Cr color component transform coefficients for each pair of Cb and Cr transform coefficients located at the same coordinate in the Cb and Cr transform block, respectively, to generate a JCST output, the JCST being a two-point transform performed on each pair of Cb and Cr transform values ​​located at the same coordinate in the Cb and Cr transform block, such that for each pair of Cb and Cr transform values ​​located at the same coordinate, when the Cr transform value is in the Cr transform block, the Cb transform value is located at the same coordinate in the Cb transform block; performing an inverse transform on the JCST output to obtain residual components of the Cb and Cr color components; and decoding the encoded video bitstream based on residual components of the Cb and Cr color components.

2. The method described in claim 1, wherein the JCST is applied to block sizes within a limited range.

3. A step of obtaining a coded video sequence (CVS) from the coded video bitstream, the CVS including image units corresponding to coded images; obtaining a picture header (PH) network abstraction layer (NAL) unit contained in said picture unit; obtaining at least one video coding layer (VCL) NAL unit included in the image unit; and analyzing a JCST flag at a transform block level that signals from when the JCST is applied.

4. A step of obtaining a coded video sequence (CVS) from the coded video bitstream, the CVS including image units corresponding to coded images; obtaining a picture header (PH) network abstraction layer (NAL) unit contained in said picture unit; obtaining at least one video coding layer (VCL) NAL unit included in the image unit; and parsing a JCST flag at a CU or CB level that signals from when the JCST applies.

5. A method for encoding a video signal comprising: obtaining a coded video sequence (CVS) from the coded video bitstream, the coded video sequence comprising image units corresponding to coded images; obtaining a picture header (PH) network abstraction layer (NAL) unit contained in said picture unit; obtaining at least one video coding layer (VCL) NAL unit included in the image unit; and parsing a JCST flag that signals when the JCST is to be applied via a high-level syntax.

6. The method described in claim 1, wherein the JCST includes a second transformation determined by encoding information.

7. An apparatus for decoding an encoded video bitstream, comprising: An apparatus for performing the method for decoding an encoded video bitstream according to any one of claims 1 to 6.

8. A computer program stored on a non-transitory computer-readable storage medium, comprising: A computer program product which, when executed by at least one processor, implements the method for decoding an encoded video bitstream according to any one of claims 1 to 6.

9. A method for storing a video bitstream using at least one processor, comprising: generating an encoded video bitstream, obtaining video data comprising a plurality of video frames, including a current video frame comprising a Cb color component and a Cr color component; obtaining a residual block for each of the Cb color component and the Cr color component; obtaining Cb and Cr transform blocks by performing a linear transform on each of the residual blocks, respectively; applying a joint components secondary transform (JCST) element-wise to the Cb and Cr transform block to generate a JCST output, the JCST being a two-point transform performed on each pair of Cb and Cr transform values ​​located at the same coordinate in the Cb and Cr transform block, such that for each pair of Cb and Cr transform values ​​located at the same coordinate, when the Cr transform value is in the Cr transform block, the Cb transform value is located at the same coordinate as the Cb transform block; performing quantization on the JCST output to obtain transform coefficients; encoding the transform coefficients; transmitting the coded transform coefficients in a video bitstream. Steps and storing the encoded video bitstream on a non-transitory computer-readable medium.

Images