Video decoding method and related apparatus

Abstract

A video decoding method includes: before residual decoding of a coding unit is completed, referring to available information to determine whether to decode information required for inverse transform circuit (IT) to inverse transform a transform block of the coding unit and generate a judgment result; and controlling transmission of coefficients of the coding unit to the IT circuit according to the judgment result.

Description

[0001] CROSS-REFERENCE

[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 257,627, filed October 20, 2021, which is hereby incorporated by reference in its entirety. TECHNICAL FIELD

[0003] The present disclosure relates to video processing, and more particularly, to a method and apparatus for controlling coefficient transmission by early determining whether information needed for inverse transform is decoded. BACKGROUND

[0004] Conventional video coding standards typically employ block-based coding techniques to exploit spatial and temporal redundancies. For example, a basic approach is to divide an entire source picture into blocks, perform intra / inter prediction for each block, perform transform on the residual of each block, and perform quantization and entropy coding. In addition, a reconstructed picture is generated in the coding loop to provide reference pixel data for coding subsequent blocks. For certain video coding standards, an in-loop filter can be used to enhance the picture quality of the reconstructed frame. A video decoder is used to perform inverse operations of the video encoding operations performed by a video encoder. For example, a video decoder can include an entropy decoding circuit, an inverse scanning circuit, an inverse transform circuit, etc. The operation of the inverse transform circuit can be configured by values of certain syntax elements. According to the syntax decoding process specified by the Versatile Video Coding (VVC) standard (also known as the H.266 standard), the information needed by the inverse transform circuit to perform inverse transform on a transform block of a coding unit can be decoded by the following: the entropy decoding circuit of the coding unit residual decoding is completely done by the entropy decoding circuit. As a result, the inverse transform circuit does not start applying inverse transform on the transform block of the coding unit until the information needed by the inverse transform circuit is determined. In addition, before the information needed by the inverse transform circuit is determined, the inverse transform circuit needs a large buffer to hold the result of the residual decoding of the coding unit.

[0005] Therefore, there is a need for an innovative video decoder design that can improve the performance of the inverse transform circuit and / or reduce the buffer requirement of the inverse scanning circuit. SUMMARY

[0006] It is an object of the claimed invention to provide a method and apparatus for controlling coefficient transmission by early determining whether information needed for inverse transform (e.g., syntax elements lfnst_idx and mts_idx) is decoded.

[0007] According to a first aspect of the present application, an example video decoding method is disclosed. The example video decoding method comprises: before residual decoding of a coding unit is completed, referring to available information to determine whether information required for an inverse transform (IT) circuit to apply inverse transform to a transform block of the coding unit is decoded, and generating a determination result; and controlling coding unit coefficient transmission to the IT circuit according to the determination result.

[0008] According to a second aspect of the present application, an example video decoding apparatus is disclosed. The example video decoding apparatus comprises an entropy decoding circuit, an inverse scanning (IS) circuit and an inverse transform (IT) circuit. The entropy decoding circuit is configured to perform residual decoding of a coding unit. The IT circuit is arranged to apply inverse transform to a transform block of the coding unit. The IS circuit is coupled between the entropy decoding circuit and the IT circuit to rearrange coefficients of the coding unit. Before the residual decoding of the coding unit is completed, at least one of the entropy decoding circuit and the IS circuit is configured to determine whether information required for the IT circuit to apply inverse transform to the transform block of the coding unit is decoded by the entropy decoding circuit to generate a determination result, and to control coding unit coefficient transmission to the IT circuit according to the determination result.

[0009] These and other objects of the present application will no doubt become apparent to those of ordinary skill in the art after reading the following detailed description of the preferred embodiments, the appended claims and the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0010] FIG. 1 is a block diagram illustrating a video decoding apparatus according to an embodiment of the present application.

[0011] FIG. 2 is a diagram illustrating a part of a syntax decoding process of a coding unit as specified in the VVC standard.

[0012] FIG. 3 is a diagram illustrating a video decoding method according to an embodiment of the present application.

[0013] FIG. 4 is a diagram illustrating a 32x32 transform block according to an embodiment of the present application.

[0014] FIG. 5 is a diagram illustrating a performance comparison between an original syntax decoding process of the VVC standard and a proposed syntax decoding process of the present application.

[0015] FIG. 6 is a diagram illustrating another performance comparison between an original syntax decoding process of the VVC standard and a proposed syntax decoding process of the present application. DETAILED DESCRIPTION

[0016] Certain terms used in the following description and claims refer to specific components. As will be understood by those skilled in the art, electronic device manufacturers may use different names to refer to a component. This document is not intended to distinguish between components with different names but different functions. In the following description and claims, the terms “comprising” and “including” are used in an open-ended manner and should therefore be interpreted as meaning “including, but not limited to…”. Furthermore, the term “coupled” is intended to indicate an indirect or direct electrical connection. Thus, if one device is coupled to another device, the connection can be a direct electrical connection or an indirect electrical connection via other devices and connections.

[0017] FIG. 1 This is a block diagram illustrating a video decoding apparatus according to an embodiment of the present invention. The video decoding apparatus 100 is a hardware circuit for decompressing encoded video data to generate decompressed video data. As shown in the figure... FIG. 1 As shown, the video decoding device 100 may include an entropy decoding circuit (e.g., a variable-length decoding circuit 102, labeled "VLD"), an inverse scan circuit (labeled "IS") 104, an inverse quantization circuit (labeled "IQ") 106, an inverse transform circuit (labeled "IT") 108, a reconstruction circuit 110, a loop filter (e.g., a deblocking filter 112, labeled "DBF"), a decoded image buffer (DPB) 114, an intra-frame prediction circuit (labeled "IP") 116, a motion vector calculation circuit (labeled "MV calculation") 118, a motion compensation circuit (labeled "MC") 120, and an inter-frame / intra-frame selection circuit (labeled "inter-frame / intra-frame selection") 122. The variable-length decoding (VLD) circuit 102 is used for entropy decoding of the coded bitstream BS. For example, the VLD circuit 102 is responsible for syntax parsing, residual decoding, etc. The inverse scan (IS) circuit 104 is used to reorder the residuals (coefficients) derived from the residual decoding at the VLD circuit 102. The IT circuit 108 is used to apply an inverse transform to each transform block to transform the residuals from the frequency domain to the spatial domain. Since this invention focuses on the coefficient transfer from the IS circuit 104 to the IT circuit 108, those skilled in the art should readily understand the principles of the remaining functional blocks in the video decoding device 100. For the sake of brevity, a further description of the video decoding device 100 is omitted here.

[0018] It is important to note that FIG. 1The illustrated video decoder architecture is for illustration purposes only and is not meant to limit the present application. For example, the architecture of the video decoding device 100 can be adapted according to the coding standard. That is, the video decoding apparatus 100 can add additional coding tools, pre-processing and / or post-processing. For better understanding of the technical features of the present application, the following assumes that the video decoding apparatus 100 is a VVC / H.266 decoder. In practice, any video decoding apparatus that uses the proposed techniques to enhance the performance of the inverse transform circuit and / or reduce the buffer requirement of the inverse scan circuit falls within the scope of the present application. Compared to a typical VVC / H.266 decoder design, the video decoding apparatus 100 supports an early determination scheme and / or a reduced buffer size. Further details are provided below with reference to the accompanying drawings.

[0019] A picture can be divided into multiple coding units (CU), each of which can be divided into multiple transform units (TU), and each of which can be composed of multiple transform blocks (TB). For example, a TU can be composed of one luma TB and two chroma TBs. According to the original syntax decoding procedure specified by the VVC standard, the information required by the IT circuit 108 for the inverse transform of the blocks of the coding unit is decoded by the VLD circuit 102 after the residual decoding of the coding unit by the VLD circuit 102 is completely finished. FIG. 2 is a diagram illustrating a part of the syntax decoding procedure of a coding unit as specified by the VVC standard. According to the VVC standard, the information required by the IT circuit 108 includes the syntax element lfnst_idx and the syntax element mts_idx. With respect to the low-frequency non-separable transform (LFNST), the syntax element lfnst_idx specifies whether to use and which one of two low-frequency non-separable transform kernels in the selected transform set is used. lfnst_idx equal to 0 indicates that no low-frequency non-separable transform is used in the current coding unit. When lfnst_idx is not present, it is inferred to be equal to 0 (i.e. default value = 0). With respect to the multiple transform selection (MTS), the syntax element mts_idx specifies which transform kernels are applied along the horizontal and vertical directions of the associated luma transform block in the current coding unit. When mts_idx is not present, it is inferred to be equal to 0 (i.e. default value = 0). When all the conditions specified in the code segment 202 are met, the syntax element lfnst_idx appears in the coded bitstream BS. Therefore, the VLD circuit 102 needs to parse the syntax element lfnst_idx from the coded bitstream BS when all the conditions specified in the code segment 202 are met. When all the conditions specified in the code segment 204 are met, the syntax element mts_idx appears in the coded bitstream BS. Therefore, the VLD circuit 102 needs to parse the syntax element mts_idx from the coded bitstream BS when all the conditions specified in the code segment 204 are met.

[0020] Code section 202 includes partial code sections 212, 214 and 216. Code section 204 includes partial code sections 218, 220, 222 and 224. The high-level information (which can be obtained before the residual decoding of the current coding unit) is examined by partial code sections 212, 218 and 220. The other four variables LfnstDcOnly, LfnstZeroOutSigCoeffFlag, MtsZeroOutSigCoeffFlag, MtsDcOnly (which are not available before the residual decoding of the current coding unit starts, and can be determined during the residual decoding of the current coding unit) are examined by partial code sections 214, 216, 222 and 224. Since all the high-level information and the other four variables are available before the residual decoding of the current coding unit is completely finished by VLD circuit 102, the information required by IT circuit 108 (e.g. syntax elements lfnst_idx and mts_idx) whether to decode can be determined in advance, which allows the time to start transferring the coefficients to IT circuit 108 to be earlier than the end time of the residual decoding of the current coding unit.

[0021] Based on the above observation, the present application proposes a VLD circuit equipped with an early determination function and an IS circuit equipped with an early determination function. As shown in FIG. 1, the VLD circuit 102 includes a high-level information extractor 202, a residual decoder 204, a coefficient extractor 206 and a coefficient buffer 208. The high-level information extractor 202 is configured to extract the high-level information (e.g. syntax elements lfnst_idx and mts_idx) from the bitstream. The residual decoder 204 is configured to decode the residual of the current coding unit based on the high-level information. The coefficient extractor 206 is configured to extract the coefficients of the current coding unit from the bitstream based on the high-level information and the residual decoding result of the current coding unit. The coefficient buffer 208 is configured to buffer the extracted coefficients. The IS circuit 108 includes a coefficient buffer 110, a coefficient decoder 112, a coefficient outputter 114 and a coefficient output buffer 116. The coefficient buffer 110 is configured to buffer the coefficients of the current coding unit. The coefficient decoder 112 is configured to decode the coefficients of the current coding unit based on the high-level information and the residual decoding result of the current coding unit. The coefficient outputter 114 is configured to output the decoded coefficients of the current coding unit. The coefficient output buffer 116 is configured to buffer the decoded coefficients of the current coding unit. FIG. 1As shown, the VLD circuit 102 includes an early determination circuit (labeled "ED") 124, and the IS circuit 104 includes an IS buffer 126 and an early determination circuit (labeled "ED") 128. Before the residual decoding of the current coding unit is completely finished by the VLD circuit 102, at least one of the VLD circuit 102 and the IS circuit 104 (specifically, at least one of the early determination circuits 124 and 128) is configured to determine whether the VLD circuit 102 needs the information required by the IT circuit 108 to inverse transform the transform block of the current coding unit to generate a determination result, and to control the transmission of the coefficients of the current coding unit to the IT circuit 108 according to the determination result. Specifically, since the high-level information can be obtained before the residual decoding of the current coding unit is started, the VLD circuit 102 (particularly, the early determination circuit 124 of the VLD circuit 102) can refer to the available high-level information to determine whether the syntax element lfnst_idx is equal to a default value (e.g., 0) in advance before the residual decoding of the current coding unit is completely finished. When the syntax element lfnst_idx can be determined in advance to be equal to the default value (e.g., 0), the syntax parsing process of the syntax element lfnst_idx is not needed after the residual decoding of the current coding unit is completely finished. In addition, since the coefficients obtained by the residual decoding of the current coding unit are stored in the IS buffer 126, the IS circuit 104 (particularly, the early determination circuit 128 of the IS circuit 104) can refer to the available information buffered in the IS buffer 126 to determine whether the syntax element mts_idx can be determined in advance to be equal to a default value (e.g., 0) before the residual decoding of the current coding unit is completely finished. When the syntax element mts_idx can be determined in advance to be equal to the default value (e.g., 0), the syntax parsing process of the syntax element mts_idx is not needed after the residual decoding of the current coding unit is completely finished. In the case where both the syntax elements lfnst_idx and mts_idx can be determined in advance by the proposed early determination scheme, the transmission of the coefficients from the IS buffer 126 to the IT circuit 108 is allowed to start at a time point earlier than the end time of the residual decoding of the current coding unit.

[0022] FIG. 3 is a flowchart illustrating a video decoding method according to an embodiment of the present application. If the results are substantially the same, the syntax parsing process of the syntax element lfnst_idx is not needed. FIG. 3The exact order of performing these steps is shown. After starting the syntax decoding process of the current coding unit in step 301, the early determination circuit 124 checks the high level information (available before starting the residual decoding of the current coding unit) to determine whether there is at least one condition violated at least one of the conditions specified by the partial code section 212 for decoding the syntax element lfnst_idx and at least one of the conditions specified by the partial code sections 218, 220 for decoding the syntax element mts_idx (step 302). If a violation occurs, it is determined that the information (lfnst_idx & mts_idx) needed by the IT circuit 108 does not need to be decoded / parsed from the coded bitstream BS, and the process proceeds to step 306. If no violation occurs, it is determined that the information (lfnst_idx & mts_idx) needed by the IT circuit 108 can need to be decoded / parsed from the coded bitstream BS by the VLD circuit 102, and the process continues to step 304 for another check. In step 304, the early determination circuit 128 checks the other four variables (not available before starting the residual decoding of the current coding unit and can be determined during the residual decoding of the current coding unit) to determine whether there is at least one violated at least one of the conditions specified by the partial code sections 214, 216 for decoding the syntax element lfnst_idx and at least one of the conditions specified by the partial code sections 218, 220 for decoding the syntax element mts_idx. If a violation occurs, it is determined that the information (lfnst_idx & mts_idx) needed by the IT circuit 108 does not need to be decoded / parsed from the coded bitstream BS, and the process proceeds to step 306. If no violation occurs, it is confirmed that the information (lfnst_idx & mts_idx) needed by the IT circuit 108 needs to be decoded / parsed from the coded bitstream BS by the VLD circuit 102, and the process proceeds to step 310. In step 310, the VLD circuit 102 will follow the normal syntax decoding process to determine the value of the syntax element lfnst_idx and the value of the residual decoded syntax element mts_idx after the decoding of the current coding unit is completely finished. After the residual decoding of the coding unit is finished, the information needed by the IT

[0023] It is assumed that all TBs in a CU are 32x32 TBs, as FIG. 4As shown, taking the variable LfnstZeroOutSigCoeffFlag as an example, it is used to indicate whether the residuals (non-zero coefficients) in all TBs in the CU occupy only the top-left 4x4 region 402. When the residuals (non-zero coefficients) in each 32x32 TB occupy only the top-left 4x4 region 402, LfnstZeroOutSigCoeffFlag == 1. However, when the residuals (non-zero coefficients) in at least one 32x32 TB do not occupy only the top-left 4x4 region 402, it can be determined in advance that LfnstZeroOutSigCoeffFlag == 0, which means that the VLD circuit 102 will not parse the syntax element lfnst_idx from the encoded bitstream BS (i.e., it is not necessary to decode the syntax element lfnst_idx required by the IT circuit 108). It should be noted that if the TB size is 4x4 or 8x8, the residuals (non-zero coefficients) can occupy scan_pos < 8 in the top-left 4x4 region.

[0024] Assuming all luminance TBs in this CU are 32x32 luminance TBs, such as FIG. 4 As shown. Taking the variable MtsZeroOutSigCoeffFlag as an example, it is used to indicate whether the residuals (non-zero coefficients) in all luminance TBs in the CU occupy only the top-left 16x16 region 404. When the residuals (non-zero coefficients) in each 32x32 luminance TB occupy only the top-left 16x16 region 404, MtsZeroOutSigCoeffFlag == 1. However, when the residuals (non-zero coefficients) in at least one 32x32 luminance TB do not occupy only the top-left 16x16 region 404, it can be determined in advance that MtsZeroOutSigCoeffFlag == 0, which means that the VLD circuit 102 will not parse the syntax element mts_idx from the encoded bitstream BS (i.e., it does not need the syntax element mts_idx required by the decoding IT circuit 108).

[0025] When the early determination circuit 128 determines in advance that MtsZeroOutSigCoeffFlag == 0 and LfnstZeroOutSigCoeffFlag == 0 based on the coefficients stored in the IS buffer 126 (step 304), the process proceeds to step 306.

[0026] At step 306, the VLD circuit 102 directly sets the information (lfnst_idx and mts_idx) required by the IT circuit 108 to default values without decoding / resolving the information (lfnst_idx and mts_idx). For example, the syntax element lfnst_idx is inferred to a default value (e.g., 0), and the syntax element mts_idx is inferred to a default value (e.g., 0). At step 308, the IS circuit 104 starts the coefficient transmission in response to the early determination of the syntax values of lfnst_idx and mts_idx. Since the syntax values of lfnst_idx and mts_idx can be determined early before the VLD circuit 102 completely finishes the residual decoding of the current coding unit, the coefficient transmission from the IS circuit 104 (in particular, the IS buffer 126 of the IS circuit 104), through the IQ circuit 106 to the IT circuit 108 can be started early before the VLD circuit 102 completely finishes the residual decoding of the current coding unit. This can greatly improve the performance of the IT circuit 108.

[0027] FIG. 5is a diagram illustrating a performance comparison between the original syntax decoding process of the VVC standard and the proposed syntax decoding process of the present invention. Assume that in the single tree case, a CU has 3 TBs. According to the original syntax decoding process specified by the VVC standard, a typical VLD circuit performs residual decoding for a luma TB (abbreviated as Luma TB) in the time period between time points T0 and T1. Since the residual decoding of the current CU has not been completed at this time, the IT-related syntax elements lfnst idx and mts idx have not been determined, and the residual decoding results (i.e., transform coefficients) of the luma TB need to be buffered in the IS buffer in a typical manner. Next, the typical VLD circuit performs residual decoding for a chroma TB (e.g., Cb TB) in the time period between time points T1 and T2. Since the IT-related syntax elements lfnst idx and mts idx have not been determined, the residual decoding results (i.e., transform coefficients) of the chroma TB (e.g., Cb TB) have to be buffered in the typical IS buffer. Next, the original VLD circuit performs residual decoding for the other chroma TB (e.g., Cr TB) in the time period between time points T2 and T3. Since the IT-related syntax elements lfnst idx and mts idx have not been determined, the residual decoding results (i.e., transform coefficients) of the chroma TB (e.g., Cr TB) have to be buffered in the typical IS buffer. Since the residual decoding of the current CU is completed at time point T3, the typical VLD circuit determines the IT-related syntax elements lfnst idx and mts idx of the current CU. For example, the IT-related syntax elements lfnst idx and mts idx can be decoded from the coded bitstream. For another example, the IT-related syntax elements lfnst idx and mts idx can not be present in the coded bitstream and can be inferred as default values. After the IT-related syntax elements lfnst idx and mts idx are determined at time point T3, the coefficient transfer from the IS buffer to the IT circuit starts. For example, the IS buffer outputs the residual decoding results (i.e., transform coefficients) of the luma TB to the IT circuit through the IQ circuit in the time period between time points T3 and T4. The residual decoding results (i.e., transform coefficients) of the chroma TB (e.g., Cb TB) are transferred to the IT circuit through the IQ circuit in the time period between time points T4 and T5, and the residual decoding results (i.e., transform coefficients) of the other chroma TB (e.g., Cr TB) are output to the IT circuit through the IQ circuit in the time period between time points T5 and T6.

[0028] According to the proposed syntax decoding process with early determination scheme, the early determination circuit 124 can refer to the high-level information to determine whether the VLD circuit 102 can still need to decode the IT-related syntax elements lfnst idx and mts idx from the coded bitstream BS. In addition, the VLD circuit 102 performs residual decoding on the luma TB in the time period between the time points T0 and T1, and stores the residual decoding results (i.e., transform coefficients) of the luma TB into the IS buffer 126. At the time point T1, the early determination circuit 128 can find that the luma TB has residual (non-zero coefficients) occupying outside the top-left 16x16 region, which means that both the MTS and LFNST conditions are violated. Therefore, at the time point T1, it is determined in advance that the IT-related syntax elements lfnst idx and mts idx do not need to be decoded from the coded bitstream BS, and the IT-related syntax elements lfnst idx and mts idx are inferred as default values. Since the IT-related syntax elements lfnst idx and mts idx are determined in advance at the time point T1, the IS buffer 126 outputs the residual decoding results (i.e., transform coefficients) of the luma TB to the IT circuit 108 through the IQ circuit 106 in the time period between the time points T1 and T2. In addition, the VLD circuit 102 performs residual decoding on one chroma TB (e.g., Cb TB) in the same time period between the time points T1 and T2, and stores the residual decoding results (i.e., transform coefficients) of the chroma TB (e.g., Cb TB) in the time period between the time points T2 and T3. The IS buffer 126 outputs the residual decoding results (i.e., transform coefficients) of the chroma TB (e.g., Cb TB) to the IT circuit 108 through the IQ circuit 106, the VLD circuit 102 performs residual decoding on the other chroma TB (e.g., Cr TB), and stores the residual decoding results (i.e., transform coefficients) of the chroma TB (e.g., Cr TB) into the IS buffer 126. In T3 and T4, the IS buffer 126 outputs the residual decoding results (i.e., transform coefficients) of the chroma TB (e.g., Cr TB) to the IT circuit 108 through the IQ circuit 106. Compared with the original syntax decoding process specified by VVC, the proposed syntax decoding process makes the coefficients transmitted to the IT circuit earlier, which greatly improves the video decoding performance.

[0029] FIG. 6is another performance comparison between the original syntax decoding process of the VVC standard and the proposed syntax decoding process of the present invention. It is assumed that a CU has 2 TBs in the case of dual chroma tree. Since there is no luma TB, only the LFNST condition needs to be checked. According to the original syntax decoding process specified by the VVC standard, a typical VLD circuit performs residual decoding on one chroma TB (e.g., Cb TB) in the time period between time points T0 and T1. Since the residual decoding of the current CU has not been completed at this time, the IT-related syntax element lfnst idx has not been determined, and the residual decoding result (i.e., transform coefficients) of the chroma TB (e.g., Cb TB) needs to be buffered in a typical IS buffer. Next, the typical VLD circuit performs residual decoding on the other chroma TB (e.g., Cr TB) in the time period between time points T1 and T2. Since the IT-related syntax element lfnst idx has not been determined, the residual decoding result (i.e., transform coefficients) of the chroma TB (e.g., Cr TB) has to be buffered in a typical IS buffer. Since the residual decoding of the current CU is completed at time point T2, the typical VLD circuit determines the IT-related syntax element lfnst idx of the current CU. For example, the IT-related syntax element lfnst idx can be decoded from the coded bitstream. For another example, the IT-related syntax element lfnst idx can not exist in the coded bitstream and can be inferred as a default value. After the IT-related syntax element lfnst idx is determined at time point T2, the coefficient transfer from the IS buffer to the IT circuit starts. For example, the IS buffer outputs the residual decoding result (i.e., transform coefficients) of one chroma TB (e.g., Cb TB) to the IT circuit through the IQ circuit in the time period between time points T2 and T3, and outputs the residual decoding result (i.e., transform coefficients) of the other chroma TB (e.g., Cr TB) to the IT circuit through the IQ circuit in the time period between time points T3 and T4.

[0030] According to the proposed syntax decoding procedure with early determination scheme, the early determination circuit 124 can refer to the high-level information to determine whether the VLD circuit 102 still needs to decode the coded IT-related syntax element lfnst idx from the bitstream BS. In addition, the VLD circuit 102 performs residual decoding on one chroma TB (e.g., Cb TB) in the time period between time points T0 and T1, and stores the residual decoding result (i.e., transform coefficients) of the chroma TB (e.g., Cb TB) to the IS buffer 126. At time point T1, the early determination circuit 128 can find that the chroma TB has residual (non-zero coefficients) occupying outside the left-top 4x4 region, which means the LFNST condition is violated. Therefore, at time point T1, it is determined in advance that the IT-related syntax element lfnst idx does not need to be decoded from the coded bitstream BS, and the IT-related syntax element lfnst idx is inferred as the default value. Since the IT-related syntax element lfnst idx is determined in advance at time point T1, the IS buffer 126 outputs the residual decoding result (i.e., transform coefficients) of the chroma TB (e.g., Cb TB) to the IT circuit 108 through the IQ circuit 106 in the time period between time points T1 and T2. In addition, the VLD circuit 102 performs residual decoding on other chroma TBs (e.g., Cr TB) in the same time period between time points T1 and T2, and stores the residual decoding result (i.e., transform coefficients) of the chroma TBs (e.g., Cr TB) to the IS buffer 126. In the time period between time points T2 and T3, the IS buffer 126 outputs the residual decoding result (i.e., transform coefficients) of the chroma TBs (e.g., Cr TB) to the IT circuit 108 through the IQ circuit 106. Compared with the original syntax decoding procedure specified in the VVC standard, the proposed syntax decoding procedure can transmit the coefficients to the IT circuit earlier, thereby greatly improving the video decoding performance.

[0031] As to FIG. 2 As to the code segment 202 shown, the coding block size constraint Max(cbWidth, cbHeight) <= MaxTbSizeY is involved in determining whether to decode the syntax element lfnst idx. As to FIG. 2 As to the code segment 218 shown, the coding block size constraint Max(cbWidth, cbHeight) <= 32 is involved in determining whether to decode the syntax element mts idx. These coding block size constraints mean that the residual decoding of a CU requires a smaller IS buffer size. If these coding block size constraints are not considered when setting the IS buffer size, a typical IS buffer can be configured to have a buffer size large enough to accommodate a maximum CU size of 128x128 in the case of a shared tree.

[0032] Since the aforementioned coding block size constraint implies that residual decoding of one CU requires a smaller IS buffer size, the present application can take advantage of the coding block size constraint to achieve the goal of reducing the buffer requirement of the IS circuit 104. If the transform block size is not larger than 32, the VLD circuit 102 can have the opportunity to decode the IT related syntax elements lfnst_idx and mts_idx from the coded bitstream BS. In the case of the proposed syntax decoding process with the early determination scheme, the IS buffer can be required to have a sufficiently large buffer size to accommodate 16x16+2x(16x16) residues (coefficients) in the worst case, where the CU size is 32x32 in the case of shared tree, the luma residues (coefficients) only occupy the top-left 16x16 region, and the chroma residues (coefficients) occupy the entire 16x16 region. For the normal case, the IS buffer can be required to have a sufficiently large buffer size to accommodate 32x32 residues (coefficients). Therefore, the buffer size of the IS buffer 126 can be set to 32x32, i.e., max(16x16+2x(16x16), 32x32). In short, the IS buffer 126 can be designed to have a buffer size smaller than the maximum amount of data possessed by the coefficients of one CU with the maximum CU size (e.g., 128x128). In this way, the buffer requirement of the IS circuit 104 can be relaxed.

[0033] Those skilled in the art will readily observe that numerous modifications and alterations of the device and method can be made without departing from the teachings of the present application. Accordingly, the above disclosure is intended to be illustrative only and not limiting. The scope of the present application is limited only by the following claims.

Claims

1. A method of video decoding, comprising: determining, before the residual decoding of a coding unit is completed, whether information required by an inverse transform (IT) circuit for inverse transforming a transform block of the coding unit is to be decoded and generating a decision result; and controlling the transmission of coefficients of the coding unit to the IT circuit according to the decision result; wherein the information required by the IT circuit comprises a syntax element lfnst_idx and a syntax element mts_idx, and the syntax element lfnst_idx and the syntax element mts_idx are determined before the end time of the residual decoding of the coding unit in response to the decision result indicating that the information required by the IT circuit is not to be decoded.

2. The video decoding method of claim 1, wherein, determining whether the information required by the IT circuit is to be decoded with reference to available information comprises: obtaining the available information before the residual decoding of the coding unit is started, wherein the available information comprises high level information and variables LfnstDcOnly, LfnstZeroOutSigCoeffFlag, MtsZeroOutSigCoeffFlag, MtsDcOnly.

3. The video decoding method of claim 1, wherein, determining whether the information required by the IT circuit is to be decoded with reference to available information comprises: obtaining the available information during the residual decoding of the coding unit, wherein the available information comprises high level information and variables LfnstDcOnly, LfnstZeroOutSigCoeffFlag, MtsZeroOutSigCoeffFlag, MtsDcOnly.

4. The video decoding method of claim 1, wherein, controlling the transmission of coefficients of the coding unit to the IT circuit according to the decision result comprises: starting the transmission of coefficients of the coding unit before the residual decoding of the coding unit is completed in response to the decision result indicating that the information required by the IT circuit is not to be decoded.

5. The video decoding method of claim 4, wherein, further comprising: setting the information required by the IT circuit to a default value in response to the decision result indicating that the information required by the IT circuit is not to be decoded.

6. The video decoding method of claim 1, wherein, further comprising: storing each coefficient of the coding unit into a buffer during the residual decoding of the coding unit, wherein a buffer size of the buffer is smaller than a maximum data amount of coefficients of a CU having a maximum CU size. 7.A video decoding apparatus, comprising: an entropy decoding circuit configured to perform residual decoding on a coding unit; an inverse transform (IT) circuit configured to apply inverse transform to a transform block of the coding unit; and an inverse scanning (IS) circuit coupled between the entropy decoding circuit and the IT circuit and configured to reorder coefficients of the coding unit; wherein at least one of the entropy decoding circuit and the IS circuit is configured to determine whether information required by the IT circuit for inverse transforming the transform block of the coding unit is to be decoded by the entropy decoding circuit before the residual decoding of the coding unit is completed to generate a decision result, and to control the transmission of the coefficients of the coding unit to the IT circuit according to the decision result. ​ The information required by the IT circuit includes syntax element lfnst_idx and syntax element mts_idx, and the syntax element lfnst_idx and the syntax element mts_idx are determined in response to a determination result indicating that the information required by the IT circuit does not need to be decoded before the end time of the residual decoding of the coding unit.

8. The video decoding apparatus of claim 7, wherein, The entropy decoding circuit acquires available information before starting the residual decoding of the coding unit, and determines whether the information required by the IT circuit needs to be decoded according to the available information, wherein the available information includes high-level information and variables LfnstDcOnly, LfnstZeroOutSigCoeffFlag, MtsZeroOutSigCoeffFlag and MtsDcOnly.

9. The video decoding apparatus of claim 7, wherein, The IS circuit acquires available information during the residual decoding of the coding unit by the entropy decoding circuit, and determines whether the information required by the IT circuit needs to be decoded by the entropy decoding circuit according to the available information, wherein the available information includes high-level information and variables LfnstDcOnly, LfnstZeroOutSigCoeffFlag, MtsZeroOutSigCoeffFlag and MtsDcOnly.

10. The video decoding apparatus of claim 7, wherein, When the determination result is that the information required by the IT circuit does not need to be decoded, the IS circuit starts the coefficient transmission of the coding unit before the residual decoding of the coding unit by the entropy decoding circuit.

11. The video decoding apparatus of claim 10, wherein, When the determination result is that the information required by the IT circuit does not need to be decoded, the information required by the IT circuit is set to a default value.

12. The video decoding apparatus of claim 7, wherein, The IS circuit includes a buffer for storing each coefficient of the coding unit generated from the residual decoding of the coding unit, and the buffer size of the buffer is less than the maximum data amount of the coefficients possessed by one CU of the maximum CU size.

Images