Video residual decoding apparatus and associated methods

Abstract

A video residual decoding apparatus includes a residual decoding circuit and a neighboring storage device. The residual decoding circuit performs residual decoding on a transform block, which is divided into M sub-blocks, M being a positive integer, in a diagonal scanning order. The neighboring storage device stores neighboring data belonging to neighboring sub-blocks and referenced by the residual decoding of a current sub-block, wherein the neighboring data belonging to a sub-block is derived from a result of the residual decoding of the sub-block, and a storage size of the neighboring storage device is not greater than a maximum amount of neighboring data derived from results of residual decoding of N sub-blocks, N being a positive integer and N being less than M.

Description

[0001] RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Application No. 62 / 225,972, filed on July 27, 2021. The entire contents of the above provisional application are incorporated herein by reference. TECHNICAL FIELD

[0003] The present application relates to video decoder design, and more particularly, to a video residual decoding apparatus using a neighboring storage device with a smaller storage size to store neighboring data for context selection and its related method. BACKGROUND

[0004] The Versatile Video Coding (VVC) standard (also known as H.266 standard) is the latest video coding standard. Compared with the previous High Efficiency Video Coding (HEVC) standard (also known as H.265 standard), the main goal of the new VVC standard is to significantly improve the compression capability. At the same time, the design features of VVC make it suitable for a wide range of video applications. Like the HEVC standard, the VVC standard also uses context-adaptive binary arithmetic coding (CABAC) technology to entropy encode the residual syntax elements of the coefficients in a transform block. Therefore, when the residual decoding of the current coefficient is performed at the decoder side, the neighboring data derived from the residual decoding results of the neighboring coefficients can be used for context selection. In the traditional VVC decoder design, a large-size neighboring storage device is used to cache the neighboring data derived from the residual decoding results of all the coefficients in a transform block. Specifically, assuming that the maximum amount of neighboring data belonging to a coefficient is A and the number of coefficients contained in 1 TB is B, the existing neighboring storage device is configured to have B storage units, and the storage size of each storage unit is A.

[0005] Therefore, there is a need for an innovative neighboring storage mechanism to enable the neighboring storage device to have a smaller size in video residual decoding. SUMMARY

[0006] One of the purposes of the present application is to provide a video residual decoding apparatus that uses a neighboring storage device with a smaller storage capacity to store neighboring data for context selection.

[0007] According to a first aspect of the present disclosure, an exemplary video residual decoding apparatus is disclosed. The exemplary video residual decoding apparatus comprises a residual decoding circuit and a neighboring storage apparatus. The residual decoding circuit is arranged to apply residual decoding to a transform block, wherein the transform block is divided into M sub-blocks, M being a positive integer, and the M sub-blocks are processed in a diagonal scan order. The neighboring storage apparatus is arranged to store neighboring data belonging to neighboring sub-blocks and referenced by the residual decoding of a current sub-block, wherein the neighboring data belonging to a sub-block is derived from a result of the residual decoding of the sub-block, and a storage size of the neighboring storage apparatus is no larger than a maximum amount of neighboring data derived from residual decoding results of N sub-blocks, N being a positive integer and N being smaller than M.

[0008] According to a second aspect of the present disclosure, an exemplary video residual decoding method is disclosed. The exemplary video residual decoding method comprises: applying residual decoding to a transform block, wherein the transform block is divided into M sub-blocks, M being a positive integer, and the M sub-blocks are processed in a diagonal scan order; and storing, by a neighboring storage apparatus, neighboring data belonging to neighboring sub-blocks and referenced by the residual decoding of a current sub-block, wherein the neighboring data belonging to a sub-block is derived from a result of the residual decoding of the current sub-block, and a storage size of the neighboring storage apparatus is no larger than a maximum amount of neighboring data derived from residual decoding results of N sub-blocks, N being a positive integer and N being smaller than M.

[0009] These and other objects of the present application will no doubt become apparent to those of ordinary skill in the art after reading the following detailed description of the preferred embodiments that are illustrated in the various drawing figures. BRIEF DESCRIPTION OF DRAWINGS

[0010] Figure 1 A diagram showing the reverse diagonal scan order employed by the regular mode residual decoding according to embodiments of the present disclosure.

[0011] Figure 2 A diagram showing the forward diagonal scan order employed by the transform skip (TS) mode residual decoding according to embodiments of the present disclosure.

[0012] Figure 3 A diagram showing different diagonal coordinates within a transform block and different diagonal coordinates within a sub-block according to embodiments of the present disclosure.

[0013] Figure 4 A diagram showing the neighboring data dependency for residual syntax decoding of a current coefficient in the regular mode according to embodiments of the present disclosure.

[0014] Figure 5A diagram showing the neighboring data dependencies for residual syntax decoding of a current coefficient in TS mode according to an embodiment of the application.

[0015] Figure 6 A diagram showing a video residual decoding apparatus according to an embodiment of the application.

[0016] Figure 7 A diagram showing the case where neighboring storage devices are used to provide the neighboring data required for context selection during regular mode decoding of one maximum transform block.

[0017] Figure 8 A diagram showing the number of storage units allocated in a first level (LI) diagonal neighboring buffer for storing the required neighboring data belonging to one sub-block of a previously decoded diagonal line according to an embodiment of the application.

[0018] Figure 9 A diagram showing the number of storage units allocated in a LI diagonal neighboring buffer for storing the required neighboring data belonging to a head sub-block of a previously decoded diagonal line according to an embodiment of the application.

[0019] Figure 10 A diagram showing the number of storage units allocated in a LI diagonal neighboring buffer for storing the required neighboring data belonging to a tail sub-block of a previously decoded diagonal line according to an embodiment of the application.

[0020] Figure 11 A diagram showing the number of storage units allocated in a second level (L2) diagonal neighboring buffer for storing the required neighboring data belonging to one sub-block of another previously decoded diagonal line according to an embodiment of the application.

[0021] Figure 12 A diagram showing a first arrangement of neighboring storage devices for residual decoding of a transform block according to an embodiment of the application.

[0022] Figure 13 A flowchart showing a first regular mode video residual decoding method according to an embodiment of the application.

[0023] Figure 14 A flowchart showing a first alternative regular mode video residual decoding method according to an embodiment of the application.

[0024] Figure 15 A diagram showing a second arrangement of neighboring storage devices for residual decoding of a transform block according to an embodiment of the application.

[0025] Figure 16 A flowchart showing a first TS mode video residual decoding method according to an embodiment of the application.

[0026] Figure 17 A flowchart showing a first alternative TS mode video residual decoding method according to an embodiment of the application.

[0027] Figure 18 A diagram showing a third arrangement of neighboring storage for residual decoding of a transform block mode according to an embodiment of the application.

[0028] Figure 19 A flowchart showing a second regular mode video residual decoding method according to an embodiment of the application.

[0029] Figure 20 A flowchart showing a second alternative regular mode video residual decoding method according to an embodiment of the application.

[0030] Figure 21 A diagram showing a fourth arrangement of neighboring storage for residual decoding of a transform block mode according to an embodiment of the application.

[0031] Figure 22 A flowchart showing a second TS mode video residual decoding method according to an embodiment of the application.

[0032] Figure 23 A flowchart showing a second alternative TS mode video residual decoding method according to an embodiment of the application.

[0033] Figure 24 A diagram showing a first design embodiment according to the application.

[0034] Figure 25 A diagram showing a second design embodiment according to the application.

[0035] Figure 26 A diagram showing a third design embodiment according to the application.

[0036] Figure 27 A diagram showing a fourth design embodiment according to the application.

[0037] Figure 28 A diagram showing an embodiment of the application. Figure 6 A diagram showing an embodiment of the application.

[0038] Figure 29 A diagram showing an alternative embodiment of the application. Figure 6 A diagram showing an alternative embodiment of the application. DETAILED DESCRIPTION

[0039] Throughout the following description and claims, specific terminology will be used to refer to certain elements as they are used. As one skilled in the art will appreciate, electronic device manufacturers can refer to the same elements by different names. The present document does not intend to distinguish among names differing in sign but functioning identically. In the following description and claims, the terms "comprise" and "include" are used in an open-ended fashion, and thus should be interpreted as "including, but not limited to...". Also, the term "coupled" is intended to mean either an indirect or direct electrical connection. Thus, if one device is coupled to another device, that connection can be through a direct electrical connection, or through other devices and connections.

[0040] In embodiments of the present disclosure, an entropy decoder of a video decoder (e.g., a VVC / H.266 decoder) can perform residual decoding on a transform block according to a diagonal scan order. In the following, the term "transform block" can refer to a block of quantization indices (also referred to as transform coefficient levels) at different coefficient positions, whether they are generated by a transform and quantization at the encoder side or only by quantization at the encoder side. One transform block (TB) can be divided into multiple sub-blocks (SBs), and each SB can include multiple coefficients. The SB size can be 4x4 or 2x2, depending on the TB size.

[0041] In the case of residual decoding in the regular mode, a reverse diagonal scan order from bottom-right to top-left is used to decode the sub-blocks within a transform block and the coefficients within each sub-block of the transform block. Figure 1 A diagram showing the reverse diagonal scan order employed by the regular mode residual decoding according to embodiments of the present disclosure is shown. Assume that the TB size is 16x16 and the SB size is 4x4. For a transform block, the sub-blocks indexed by 0-15 are decoded in the reverse diagonal scan order (i.e., 15→14→13...→2→1→0). For each sub-block within the same transform block, the coefficients indexed by 0-15 are also decoded in the reverse diagonal scan order (i.e., 15→14→13...→2→1→0).

[0042] In another case of residual coding in the transform skip (TS) mode, a forward diagonal scan order from top-left to bottom-right is used to decode the sub-blocks within a transform block and the coefficients within each sub-block of the transform block. Figure 2A diagram showing the forward diagonal scan order employed by TS mode residual decoding according to embodiments of the application. Assume the TB size is 16x16 and the SB size is 4x4. For one transform block, the SBs indexed by 0-15 are decoded in the forward diagonal scan order (i.e. 0→1→2…→13→14→15). For each sub-block within the same transform block, the coefficients indexed by 0-15 are also decoded in the forward diagonal scan order (i.e. 0→1→2…→13→14→15).

[0043] The backward diagonal scan order can be defined by the descending order of the diagonal coordinate D, and the forward diagonal scan order can be defined by the ascending order of the diagonal coordinate D. Figure 3 A diagram showing different diagonal coordinates D={0, 1, 2, 3, 4, 5, 6} within one transform block and different diagonal coordinates D={0, 1, 2, 3, 4, 5, 6} within one sub-block according to embodiments of the application. Assume the TB size is 16x16 and the SB size is 4x4. The top-left position (X, Y) is set by (0, 0). Along the X direction, the X coordinate is incremented. Along the Y direction, the Y coordinate is incremented. The diagonal coordinate D is defined by the sum of X and Y. Therefore, along the same diagonal, the diagonal coordinate D is the same. The backward diagonal scan order employed by residual decoding in the regular mode is from D=6 to D=0. The forward diagonal scan order employed by residual decoding in the TS mode is from D=0 to D=6.

[0044] In each of the regular mode and the TS mode, the coefficient decoding loop can include three passes, where the first pass (pass1) decoding can rely on the CABAC decoding process. Therefore, neighboring data belonging to neighboring coefficients can be referenced by the residual syntax decoding of the current coefficient for context selection. Figure 4 A diagram showing the neighboring data dependency for residual syntax decoding of the current coefficient in the regular mode according to embodiments of the application. The current coefficient is located at (Xc, Yc). According to the regular mode, the neighboring data for the residual syntax decoding of the current coefficient can include the following: Figure 1The illustrated reverse diagonal scan order, five neighboring coefficients located at (Xc+1, Yc), (Xc+2, Yc), (Xc, Yc+1), (Xc+1, Yc+1), (Xc, Yc+2) are decoded before residual syntax decoding of the current coefficient located at (Xc, Yc). Thus, neighboring data belonging to the neighboring coefficients can be referenced by the residual syntax decoding of the current coefficient for context selection. For example, the AbsLevelPassl value of each neighboring coefficient located at (Xc+1, Yc), (Xc+2, Yc), (Xc, Yc+1), (Xc+1, Yc+1), (Xc, Yc+2) is calculated, where AbsLevelPassl = sig_coeff_flag + abs_level_gt1_flag + par_level_flag + 2*abs_level_gt3_flag; the locSumAbsLevelPassl value can be calculated by summing the AbsLevelPassl values obtained from the neighboring coefficients located at (Xc+1, Yc), (Xc+2, Yc), (Xc, Yc+1), (Xc+1, Yc+1), (Xc, Yc+2); the locNumSig value can be calculated by the number of non-zero absolute values of the neighboring parameters located at (Xc+1, Yc), (Xc+2, Yc), (Xc, Yc+1), (Xc+1, Yc+1), (Xc, Yc+2); and the locSumAbs value can be calculated by passing the sum of absolute values of the neighboring coefficients located at (Xc+1, Yc), (Xc+2, Yc), (Xc, Yc+1), (Xc+1, Yc), (Xc, Yc+2) to a clipping function, where locSumAbs = Clip3(0, 31, locSumAbs - baseLevel*5).

[0045] Figure 5 A diagram showing the neighboring data dependencies for residual syntax decoding of a current coefficient in TS mode according to an embodiment of the application. The current coefficient is located at (Xc, Yc). According to the reverse diagonal scan order, five neighboring coefficients located at (Xc+1, Yc), (Xc+2, Yc), (Xc, Yc+1), (Xc+1, Yc+1), (Xc, Yc+2) are decoded before residual syntax decoding of the current coefficient located at (Xc, Yc). Thus, neighboring data belonging to the neighboring coefficients can be referenced by the residual syntax decoding of the current coefficient for context selection. For example, the AbsLevelPassl value of each neighboring coefficient located at (Xc+1, Yc), (Xc+2, Yc), (Xc, Yc+1), (Xc+1, Yc+1), (Xc, Yc+2) is calculated, where AbsLevelPassl = sig_coeff_flag + abs_level_gt1_flag + par_level_flag + 2*abs_level_gt3_flag; the locSumAbsLevelPassl value can be calculated by summing the AbsLevelPassl values obtained from the neighboring coefficients located at (Xc+1, Yc), (Xc+2, Yc), (Xc, Yc+1), (Xc+1, Yc+1), (Xc, Yc+2); the locNumSig value can be calculated by the number of non-zero absolute values of the neighboring parameters located at (Xc+1, Yc), (Xc+2, Yc), (Xc, Yc+1), (Xc+1, Yc+1), (Xc, Yc+2); and the locSumAbs value can be calculated by passing the sum of absolute values of the neighboring coefficients located at (Xc+1, Yc), (Xc+2, Yc), (Xc, Yc+1), (Xc+1, Yc), (Xc, Yc+2) to a clipping function, where locSumAbs = Clip3(0, 31, locSumAbs - baseLevel*5). Figure 1The forward diagonal scan order as shown, two neighboring coefficients located at (Xc, Yc-1), (Xc-1, Yc) are decoded before the decoding of the current coefficient located at (Xc, Yc). Therefore, neighboring data belonging to the neighboring coefficients can be referenced by the residual decoding of the current coefficient for context selection. For example, the locNumSig value can be calculated by the number of non-zero absolute values of the neighboring coefficients located at (Xc, Yc-1), (Xc-1, Yc); the leftSign value can be derived from the sign of the neighboring coefficient located at (Xc-1, Yc); and the aboveSign value can be derived from the sign of the neighboring coefficient located at (Xc, Yc-1).

[0046] As mentioned above, conventional decoder designs employ large-sized neighboring storage devices for video residual decoding, leading to high hardware cost. To solve this problem, the present application proposes a neighboring storage mechanism that enables neighboring storage devices to have a smaller size in video residual decoding.

[0047] Figure 6 A diagram showing a video residual decoding apparatus according to an embodiment of the present application. The video residual decoding apparatus 600 can be part of a video decoder such as a VVC / H.266 decoder. The video residual decoding apparatus 600 can include a residual decoding circuit 602 and a neighboring storage device 604. The residual decoding circuit 602 is arranged to apply residual decoding to a transform block (TB), wherein the transform block is divided into M sub-blocks (SBs), M being a positive integer, and the M sub-blocks are processed in a diagonal scan order for residual decoding. For example, when performing residual decoding of the transform block in the regular mode, the diagonal scan order can be the reverse diagonal scan order as shown Figure 1 . For another example, when performing residual decoding of the transform block in the TS mode, the diagonal scan order can be the forward diagonal scan order as shown Figure 2 .

[0048] The neighboring storage 604 can be accessed by the residual decoding circuit 602 and can provide neighboring data required by the CABAC decoding process at the residual decoding circuit 602. For example, the neighboring storage 604 can store neighboring data belonging to neighboring sub-blocks and referenced by the residual decoding of the current sub-block. The neighboring data belonging to a sub-block is derived from the residual decoding result of the sub-block, which includes the decoded syntax elements of the coefficients in the sub-block. In this embodiment, the storage size of the neighboring storage 604 is not greater than the maximum amount of neighboring data derived from the residual decoding results of N sub-blocks, where N is a positive integer and N is less than M. For example, assuming the maximum amount of neighboring data belonging to one coefficient is A and the number of coefficients in a transform block is B, the storage size of the neighboring storage 604 is allowed to be less than A*B.

[0049] In this embodiment, the neighboring storage 604 can be configured with two first level (L1) diagonal neighboring buffers 606_1, 606_2, one second level (L2) diagonal neighboring buffer 608, and one optional current sub-block neighboring buffer 610. The cache size of the L1 diagonal neighboring buffers 606_1 / 606_2 can depend on the neighboring data buffering requirement of the diagonal line with the maximum length in the transform block. For example, the cache size of the L1 diagonal neighboring buffers 606_1 / 606_2 can be equal to the amount of neighboring data belonging to only a portion of the coefficients in a sub-block located on the diagonal line with the maximum length in the transform block. The cache size of the L2 diagonal neighboring buffer 608 can depend on the neighboring data buffering requirement of the diagonal line with the second maximum length in the transform block. For example, the cache size of the L2 diagonal neighboring buffer 608 can be equal to the amount of neighboring data belonging to only a portion of the coefficients in a sub-block located on the diagonal line with the second maximum length in the transform block. The cache size of the current sub-block neighboring buffer 610 can depend on the neighboring data buffering requirement of a single sub-block. For example, the cache size of the current sub-block neighboring buffer 610 can be not greater than the maximum amount of neighboring data belonging to all the coefficients in a sub-block.

[0050] In some embodiments of the present application, the neighboring storage 604 can be designed to meet the worst case residual decoding requirement. In other words, the L1 diagonal neighboring buffers 606_1, 606_2, the L2 diagonal neighboring buffer 608, and the current sub-block neighboring buffer 610 should be large enough to accommodate the neighboring data required by the residual decoding of a diagonal line with the maximum amount of neighboring data required for the residual decoding of a diagonal line in the worst case in the transform block. For example, in the case that a transform block with the maximum TB size is decoded in the regular mode, the worst case can occur in which the maximum amount of neighboring data is required to decode the sub-blocks on a diagonal line. Figure 7This diagram illustrates a scenario where adjacent storage device 604 is used to provide the adjacent data required for context selection during normal mode decoding of a maximum transform block. In this example, the maximum TB size of the transform block is 32x32. Furthermore, the transform block is divided into 64 sub-blocks, each with an SB size of 4x4. Figure 4 As shown, the residual syntax decoding of the current coefficient at (Xc, Yc) requires data belonging to the neighboring coefficients located at (Xc+1, Yc), (Xc+2, Yc), (Xc, Yc+1), and (Xc, Yc+2). When the current coefficient at (Xc, Yc) is included in the data of the neighboring coefficients located at (Xc+1, Yc),... Figure 7 When a sub-block is located at the current decoding diagonal D (D=6), it is possible that one or more adjacent coefficients located at (Xc+1, Yc), (Xc+2, Yc), (Xc, Yc+1), (Xc+1, Yc+1), (Xc, Yc+2) are included in the sub-block located at (Xc, Yc+2). Figure 7 In one or more sub-blocks of the previously decoded diagonal (D+1) shown (or in Figure 7 The previously decoded diagonal (D+1) sub-blocks shown are neutralized. Figure 7 (As shown in a sub-block of the previously decoded diagonal (D+2). Regarding the decoding of the sub-block at the current decoding diagonal D, the required adjacent data belonging to the sub-block at the previously decoded diagonal (D+1) is stored in an L1 diagonal adjacent buffer 606_1 / 606_2, and the adjacent data belonging to the sub-block at the previously decoded diagonal (D+2) is stored in an L2 diagonal adjacent buffer 608.

[0051] Figure 8 A diagram illustrating the number of storage units allocated in the L1 diagonal adjacent buffers 606_1 / 606_2 according to an embodiment of the present invention, for storing the required adjacent data belonging to a sub-block at the previously decoded diagonal (D+1). Figure 8As shown, the neighboring data derived from the residual decoding results of the coefficients located at the coefficient positions within the diagonal region needs to be stored in the LI diagonal neighboring buffer 606_1 / 606_2. Therefore, for one subblock, S K1-bit storage units in the LI diagonal neighboring buffer 606_1 / 606_2 are allocated for storing the required neighboring data, where S = 12. In one embodiment, each K1-bit storage unit is configured to store the sig_coeff_flag, the AbsLevelPassl, and the clipped AbsLevel (i.e., IocSumAbs) derived from the residual decoding results of one coefficient, and can have a storage size of at least 10 bits (i.e., K1 = 1 + 3 + 6 = 10). In another embodiment, since AbsLevelPassl == 0 means sig_coeff_flag = 0 and AbsLevelPassl == 1 means sig_coeff_flag = 1, each K1-bit storage unit can be configured to store the AbsLevelPassl and the clipped AbsLevel (i.e., IocSumAbs) derived from the residual decoding results of one coefficient, and can have a storage size of at least 9 bits (i.e., K1 = 3 + 6 = 9). In yet another embodiment, each K1-bit storage unit is configured to store more than 10 bits (i.e., K1 > 10). The buffer size of the LI diagonal neighboring buffer 606_1 / 606_2 can be set to M x N, where M = S x K1 bits, and N represents the number of subblocks located on the diagonal line of maximum length in the transform block. For a transform block with a maximum TB size of 32 x 32, N is equal to 8, as shown. Figure 7

[0052] In some embodiments of the present application, the number of storage units allocated in the LI diagonal neighboring buffer 606_1 / 606_2 for storing the required neighboring data belonging to the head subblock at the previously decoded diagonal line (D + 1) can be reduced to less than 12 (i.e., S < 12 for the neighboring data buffering of the head subblock), and / or the number of storage units allocated in the LI diagonal neighboring buffer 606_1 / 606_2 for storing the required neighboring data belonging to the tail subblock at the previously decoded diagonal line (D + 1) can be reduced to less than 12 (i.e., S < 12 for the neighboring data buffering of the tail subblock). Figure 9 A diagram showing the number of storage units allocated in the LI diagonal neighboring buffer 606_1 / 606_2 according to an embodiment of the present application for storing the required neighboring data belonging to the head subblock at the previously decoded diagonal line (D + 1) is shown. Figure 10 ​A diagram showing the number of storage units allocated in the L1 diagonal neighboring buffer 606_1 / 606_2 according to an embodiment of the present application for storing the required neighboring data belonging to the tail subblock at the previously decoded diagonal (D+1) is shown. Due to the inherent nature of data dependency, only the neighboring data derived from the residual decoding results of the eight coefficients located in the diagonal region of the head subblock need to be stored in the L1 diagonal neighboring buffer 606_1 / 606_2, and only the neighboring data derived from the residual decoding results of the eight coefficients located in the diagonal region of the tail subblock need to be stored in the L1 diagonal neighboring buffer 606_1 / 606_2. Therefore, the hardware cost of the L1 diagonal neighboring buffer 606_1 / 606_2 can be further reduced.

[0053] Figure 11 A diagram showing the number of storage units allocated in the L2 diagonal neighboring buffer 608 according to an embodiment of the present application for storing the required neighboring data belonging to one subblock at another previously decoded diagonal (D+2) is shown. As Figure 11 indicated, the neighboring data derived from the residual decoding results of the single coefficient located in the diagonal region need to be stored in the L2 diagonal neighboring buffer 608. Therefore, for one coefficient, one K1-bit storage unit can be allocated in the L2 diagonal neighboring buffer 608 for storing the required neighboring data. In one embodiment, each K1-bit storage unit can be configured to store the sig_coeff_flag, the AbsLevelPassl and the clipped AbsLevel (i.e. IocSumAbs) derived from the residual decoding results of one coefficient, and can have a storage size of at least 10 bits (i.e. K1 = 1 + 3 + 6 = 10). In another embodiment, since AbsLevelPassl == 0 implies sig_coeff_flag = 0 and AbsLevelPassl == 1 implies sig_coeff_flag = 1, each K1-bit storage unit can be configured to store the AbsLevelPassl and the clipped AbsLevel (i.e. IocSumAbs) derived from the residual decoding results of one coefficient, and can have a storage size of at least 9 bits (i.e. K1 = 3 + 6 = 9). In yet another embodiment, each K1-bit storage unit can be configured to store more than 10 bits (i.e. K1 > 10). The cache size of the L2 diagonal neighboring buffer 608 can be set by G x K1 bits, where G represents the number of subblocks in the transform block located in the diagonal line having the second largest length. For a transform block with the maximum TB size of 32x32, G is equal to 7 as Figure 7 indicated.

[0054] In decoding the coefficients located in theFigure 7 During the current decoding of the current subblock with diagonal D (D = 6) for residual decoding, the current subblock neighboring buffer 610 can be used to store neighboring data belonging to the current subblock. For example, the neighboring data belonging to the current subblock can include neighboring data belonging to all coefficients within the current subblock. Thus, the neighboring data belonging to the current subblock can include neighboring data belonging to one coefficient of the current subblock that is referred to for residual syntax decoding of another coefficient within the same subblock, and can also include neighboring data belonging to one coefficient of the current subblock that is referred to for residual syntax decoding of another coefficient in a different subblock (i.e., a subblock neighboring the current subblock). The buffer size of the current subblock neighboring buffer 610 can be set by L x K1 bits, where L represents the number of coefficients within the current subblock. For a subblock with SB size of 4 x 4, L is equal to 16. In an embodiment, each K1-bit storage unit can be configured to store sig_coeff_flag, AbsLevelPassl, and clipped AbsLevel (i.e., IocSumAbs) derived from the result of residual decoding of one coefficient, and can have a storage size of at least 10 bits (i.e., K1 = 1 + 3 + 6 = 10). In another embodiment, since AbsLevelPassl == 0 implies sig_coeff_flag = 0 and AbsLevelPassl == 1 implies sig_coeff_flag = 1, each K1-bit storage unit can be configured to store AbsLevelPassl and clipped AbsLevel (i.e., IocSumAbs) derived from the result of residual decoding of one coefficient, and can have a storage size of at least 9 bits (i.e., K1 = 3 + 6 = 9). In yet another embodiment, each K1-bit storage unit can be configured to store more than 10 bits (i.e., K > 10).

[0055] In the present embodiment, the video residual decoding apparatus 600 can be configured to support a first neighboring storage mechanism and a second neighboring storage mechanism, where the first neighboring storage mechanism employs a ping-pong buffering technique, and the second neighboring storage mechanism employs a buffer sharing technique. Compared with the first neighboring storage mechanism, the second neighboring storage mechanism can have a lower buffering requirement.

[0056] According to the first neighboring storage mechanism in video residual decoding, the video residual decoding apparatus 600 can perform residual decoding of one transform block in the regular mode by using two L1-diagonal neighboring buffers 606_1 and 606_2, an L2-diagonal neighboring buffer 608, and a current subblock neighboring buffer 610. Figure 12A diagram showing a first arrangement of neighboring storage devices 604 for residual decoding of a transform block according to an embodiment of the application. When regular mode residual decoding is enabled for the first neighboring storage mechanism, the Ll diagonal neighboring buffers 606_1 and 606_2 operate like ping-pong buffers. Specifically, when one of the Ll diagonal neighboring buffers 606_1 and 606_2 is used to buffer neighboring data belonging to subblocks located at a current diagonal D, the other of the Ll diagonal neighboring buffers 606_1 and 606_2 is used to buffer neighboring data belonging to subblocks located at a previous diagonal (D+1). In Figure 12 In the illustrated embodiment, the Ll diagonal neighboring buffer 606_1 is currently used to buffer neighboring data belonging to subblocks located at a current diagonal D and referenced by residual decoding of subblocks located at a next diagonal (D-1). According to the reverse diagonal scan order, the residual decoding of subblocks located at the current diagonal D is followed by the residual decoding of subblocks located at the next diagonal (D-1). As for the other Ll diagonal neighboring buffer 606_2, it is currently used to buffer neighboring data belonging to subblocks located at a previous diagonal (D+1) and referenced by residual decoding of subblocks located at the current diagonal D. According to the reverse diagonal scan order, the residual decoding of subblocks located at the previous diagonal (D+1) is followed by the residual decoding of subblocks located at the current diagonal D.

[0057] At the end of the residual decoding of subblocks located at the current diagonal D, the Ll diagonal neighboring buffer 606_2 is arranged to start buffering neighboring data belonging to subblocks located at a next diagonal (D-1). That is, at the end of the residual decoding of subblocks located at the current diagonal D, the next diagonal (D-1) becomes the current diagonal to be decoded. During the residual decoding of subblocks located at the current diagonal (D-1), the Ll diagonal neighboring buffer 606_1 buffers neighboring data belonging to subblocks located at a previous diagonal D and referenced by residual decoding of subblocks located at the current diagonal (D-1).

[0058] During the residual decoding of the sub-block located at the current diagonal D, the L2 diagonal neighboring buffer 608 is arranged to buffer neighboring data belonging to the sub-block located at another previous diagonal (D+2) and referenced by the residual decoding of the sub-block located at the current diagonal D. The residual decoding of the sub-block located at the previous diagonal (D+2) is followed by the residual decoding of the sub-block located at the previous diagonal (D+1) in the reverse diagonal scanning order. When the residual decoding of the sub-block located at the current diagonal D is finished, the next diagonal (D-1) becomes the current diagonal to be decoded and the partial data stored in the Ll diagonal neighboring buffer 606_2 is pushed to the L2 diagonal neighboring buffer 608. During the residual decoding of the sub-block located at the current diagonal (D-1), the L2 diagonal neighboring buffer 608 buffers neighboring data belonging to the sub-block located at the previous diagonal (D+1) and referenced by the residual decoding of the sub-block located at the current diagonal (D-1).

[0059] The current sub-block neighboring buffer 610 is arranged to buffer neighboring data belonging to the current sub-block located at the current diagonal D during the residual decoding of the current sub-block. Consider the case that the residual decoding of the current sub-block indicates that the current sub-block has at least one non-zero coefficient. When the residual decoding of the current sub-block is finished, the neighboring data belonging to the current sub-block stored in the current sub-block neighboring buffer 610 is pushed to the allocated storage area 606_1 in the Ll diagonal neighboring buffer. Thus, the neighboring data belonging to the current sub-block stored in the Ll diagonal neighboring buffer 606_1 is later referenced by the residual decoding of the sub-block at the diagonal (D-1).

[0060] Consider another case that the residual decoding of the current sub-block indicates that the current sub-block has all-zero coefficients. The allocated storage area in the Ll diagonal neighboring buffer 606_1 for storing the neighboring data belonging to the current sub-block is emptied, equivalent to pushing the neighboring data (zero coefficients) belonging to the current sub-block to the Ll diagonal neighboring buffer 606_1. Similarly, the neighboring data (zero coefficients) belonging to the current sub-block stored in the Ll diagonal neighboring buffer 606_1 is later referenced by the residual decoding of the sub-block at the diagonal (D-1).

[0061] Figure 13 A flowchart of the first regular mode video residual decoding method according to an embodiment of the present application is shown. These steps do not need to be performed in the exact order shown if the results are substantially the same. Figure 13 The video residual decoding method can be used for Figure 6The video residual decoding apparatus 600 is shown. In step 1302, the residual decoding circuit 602 begins residual decoding of the transform block. In step 1304, the residual decoding circuit 602 performs residual decoding on the current sub-block selected from the transform block according to the diagonal scan order. In step 1306, the residual decoding circuit 602 stores adjacent data belonging to the current sub-block in the current sub-block's adjacent buffer 610 during the residual decoding of the current block. In step 1308, the residual decoding circuit 602 checks whether the last sub-block located on the current diagonal line has been decoded. If yes, the process proceeds to step 1310. If not, the process proceeds to step 1316. Therefore, if the current sub-block is the last sub-block located on the current diagonal line and the decoding of the current sub-block is complete, the process proceeds to step 1310. If the current sub-block is not the last sub-block located on the current diagonal line and the decoding of the current sub-block is complete, the process proceeds to step 1316.

[0062] In each of steps 1310 and 1316, adjacent data belonging to the current sub-block is stored in an L1 diagonal adjacent buffer 606_1 (or 606_2). For example, when adjacent data includes one or more non-zero coefficients, the residual decoding circuit 602 can push the adjacent data in the current sub-block adjacent buffer 610 to the allocated storage space in the L1 diagonal adjacent buffer 606_1 (or 606_2). As another example, when adjacent data includes all zero coefficients, the residual decoding circuit 602 can simply instruct the L1 diagonal adjacent buffer 606_1 (or 606_2) to clear the storage space allocated in the L1 diagonal adjacent buffer 606_1 (or 606_2). In step 1312, the residual decoding circuit 602 pushes a portion of the data in the L1 diagonal adjacent buffer 606_1 (or 606_2) to the L2 diagonal adjacent buffer 608. In step 1314, the residual decoding circuit 602 enables a ping-pong change between the L1 diagonally adjacent buffers 606_1 and 606_2. In step 1318, the residual decoding circuit 602 checks whether the last sub-block of the transform block has been decoded. If yes, the video residual decoding process ends. If no, the process proceeds to step 1304, where the next sub-block selected from the transform block according to the diagonal scan order becomes the current sub-block to be decoded.

[0063] according to Figure 13 As shown in the flowchart, steps 1310, 1312, and 1314 are executed when it is determined that the last sub-block of the transform block has been decoded (step 1308). Since video residual decoding uses a transform block as a decoding unit, the video residual decoding of a transform block ends after redundantly executing steps 1310, 1312, and 1314 in response to the decoding of the last sub-block of the transform block. Figure 14A flowchart showing a first alternative regular mode video residual decoding method according to an embodiment of the present application is shown. If the results are substantially the same, these steps need not be performed in the exact order shown. Figure 14 Figure 13 Figure 14 The main difference between the flowcharts shown in FIGS. 13 and 14 is that step 1306 is followed by step 1318. In this way, the video residual decoding of a transform block ends when it is determined that the last subblock of the transform block is decoded. That is, the video residual decoding of a transform block ends when the current subblock is the last subblock of the transform block and the decoding of the current subblock is completed, without the need for redundant processing of neighboring data.

[0064] According to the first neighboring storage mechanism in video residual decoding, the video residual decoding device 600 can perform residual decoding on a transform block in TS mode by using two Ll diagonal neighboring buffers 606_1, 606_2 and the current subblock neighboring buffer 610 (without the need for L2 diagonal neighboring buffer 608). Figure 15 A diagram showing a second arrangement of the neighboring storage device 604 for residual decoding on a transform block according to an embodiment of the present application is shown. Figure 12 Figure 15 The main difference between the arrangements of the neighboring storage device 604 shown in FIGS. 12 and 14 is that the L2 diagonal neighboring buffer 608 is not needed in TS mode. Similarly, when the first neighboring storage mechanism is enabled for TS mode residual decoding, the Ll diagonal neighboring buffers 606_1 and 606_2 operate like ping-pong buffers. Specifically, when one of the Ll diagonal neighboring buffers 606_1 and 606_2 is used to buffer neighboring data belonging to a subblock located at the current diagonal D, the other of the Ll diagonal neighboring buffers 606_1 and 606_2 is used to buffer neighboring data belonging to a subblock located at the previous diagonal (D-1). In Figure 15 In the embodiment shown in FIG. 14, the Ll diagonal neighboring buffer 606_1 is currently used to buffer neighboring data belonging to a subblock located at the current diagonal D and referenced by the residual decoding of a subblock located at the next diagonal (D+1). According to the forward diagonal scan order, the residual decoding of a subblock located at the current diagonal D is followed by the residual decoding of a subblock located at the next diagonal (D+1). As for the Ll diagonal neighboring buffer 606_2, it is currently used to buffer neighboring data belonging to a subblock located at the previous diagonal (D-1) and referenced by the residual decoding of a subblock located at the current diagonal. According to the forward diagonal scan order, the residual decoding of a subblock located at the previous diagonal (D-1) is followed by the residual decoding of a subblock located at the current diagonal D.

[0065] ​​​At the end of the residual decoding of the sub-block located at the current diagonal D, the Ll diagonal neighboring buffer 606_2 is arranged to start buffering neighboring data belonging to the sub-block located at the next diagonal (D+1). That is, at the end of the residual decoding of the sub-block located at the current diagonal D, the next diagonal (D+1) becomes the current diagonal to be decoded. During the residual decoding of the sub-block located at the current diagonal (D+1), the Ll diagonal neighboring buffer 606_1 buffers neighboring data belonging to the sub-block located at the previous diagonal D and referenced by the residual decoding of the sub-block located at the current diagonal (D+1).

[0066] The current sub-block neighboring buffer 610 is arranged to buffer neighboring data belonging to the current sub-block located at the current diagonal D during the residual decoding of the current sub-block. Consider the case where the residual decoding of the current sub-block indicates that the current sub-block has at least one non-zero coefficient. At the end of the residual decoding of the current sub-block, the neighboring data belonging to the current sub-block stored in the current sub-block neighboring buffer 610 is pushed to the allocated storage area in the Ll diagonal neighboring buffer 606_1. Thus, the neighboring data belonging to the current sub-block stored in the Ll diagonal neighboring buffer 606_1 is later referenced by the residual decoding of the sub-block at the diagonal (D+1).

[0067] Consider another case where the residual decoding of the current sub-block indicates that the current sub-block has all-zero coefficients. The allocated storage area in the Ll diagonal neighboring buffer for storing neighboring data belonging to the current sub-block is emptied, equivalent to pushing the neighboring data (zero coefficients) belonging to the current sub-block to the Ll diagonal neighboring buffer 606_1. Similarly, the neighboring data (zero coefficients) belonging to the current sub-block stored in the Ll diagonal neighboring buffer 606_1 is later referenced by the residual decoding of the sub-block at the diagonal (D+1).

[0068] Figure 16 A flowchart of the first TS mode video residual decoding method according to an embodiment of the present application is shown. These steps do not need to be performed in the exact order shown if the results are substantially the same. Figure 16 The video residual decoding method can be used in the video residual decoding apparatus 600 shown in Figure 6 Figure 13 and Figure 16 The main difference between the video residual decoding method in Figure 16 and the video residual decoding method in Figure 13 is that the video residual decoding method in Figure 16 omits step 1312. Since the details of the steps in are readily apparent to those skilled in the art after reading the above paragraphs on the video residual decoding method in

[0069] , further description is omitted here for brevity. Figure 17A flowchart illustrating a first alternative TS mode video residual decoding method according to an embodiment of the present application is shown. If the results are substantially the same, these steps do not need to be performed in the exact order shown. This video residual decoding method can be used in the video residual decoding apparatus 600 shown. Figure 17 The main difference between the video residual decoding method in Figure 6 and the video residual decoding method in Figure 14 is that the video residual decoding method in Figure 17 omits step 1312. Since those skilled in the art can easily know the details of the steps in Figure 17 after reading the above paragraphs of Figure 14 , further description is omitted here for brevity. Figure 17

[0070] According to a second neighboring storage mechanism in video residual decoding, the video residual decoding apparatus 600 can perform residual decoding on a transform block in regular mode by using one Ll diagonal neighboring buffer 606_1 / 606_2, an L2 diagonal neighboring buffer 608, and a current sub-block neighboring buffer 610. Figure 18 A diagram illustrating a third arrangement of the neighboring storage device 604 for performing residual decoding on a transform block according to an embodiment of the present application is shown. When the second neighboring storage mechanism is enabled for regular mode residual decoding, only one of the Ll diagonal neighboring buffers 606_1 and 606_2 is selected as a shared buffer for buffering neighboring data belonging to sub-blocks located at a current diagonal D and referenced by residual decoding of sub-blocks located at a next diagonal (D-1), and for buffering neighboring data belonging to sub-blocks located at a previous diagonal (D+1) and referenced by residual decoding of sub-blocks located at the current diagonal D. In reverse diagonal scan order, residual decoding of sub-blocks located at the current diagonal D is followed by residual decoding of sub-blocks located at the next diagonal (D-1), and residual decoding of sub-blocks located at the previous diagonal (D+1) is followed by residual decoding of sub-blocks located at the current diagonal D.

[0071] Since a single Ll diagonal neighboring buffer 606_1 / 606_2 is shared for buffering neighboring data belonging to sub-blocks located at neighboring diagonals (including the current diagonal D and the previous diagonal (D+1)), data overwrite can occur when residual decoding sub-blocks located at the current diagonal D. For example, when residual decoding sub-blocks located at the current diagonal D, neighboring data belonging to sub-blocks located at the previous diagonal (D+1) can be overwritten by neighboring data belonging to sub-blocks located at the current diagonal D. As shown in Figure 18 ​As shown, the LI diagonal neighboring buffer 606_1 / 606_2 can be considered to have a current line part PI and a previous line part P2 when performing residual decoding on the sub-blocks located at the current diagonal D, where the current line part PI includes storage units for storing neighboring data belonging to the sub-blocks located at the current diagonal D, and the previous line part P2 includes storage units for storing neighboring data belonging to the sub-blocks located at the previous diagonal (D+1).

[0072] During the residual decoding on the sub-blocks located at the current diagonal D, the L2 diagonal neighboring buffer 608 is arranged to buffer neighboring data belonging to the sub-blocks located at another previous diagonal (D+2) and referenced by the residual decoding of the sub-blocks located at the current diagonal D. In the reverse diagonal scan order, the residual decoding of the sub-blocks located at the previous diagonal (D+2) is followed by the residual decoding of the sub-blocks located at the previous diagonal (D+1). It is noted that the portion of data stored in the LI diagonal neighboring buffer 606_1 / 606_2 belonging to the first sub-block located at the previous diagonal (D+1) should be pushed to the L2 diagonal neighboring buffer 608 before the neighboring data belonging to the first sub-block located at the previous diagonal (D+1) is overwritten by the neighboring data belonging to the second sub-block located at the current diagonal D. For example, the residual decoding circuit 602 can move the portion of neighboring data belonging to one sub-block from the LI diagonal neighboring buffer 606_1 / 606_2 to the L2 diagonal neighboring buffer 608 whenever the residual decoding of one sub-block of the current diagonal is completed.

[0073] The current sub-block neighboring buffer 610 is arranged to buffer neighboring data belonging to the current sub-block located at the current diagonal D during the residual decoding of the current sub-block. Consider the case where the residual decoding of the current sub-block indicates that the current sub-block has at least one non-zero coefficient. At the end of the residual decoding of the current sub-block, the neighboring data belonging to the current sub-block stored in the current sub-block neighboring buffer 610 is pushed to the allocated storage region 606_1 in the LI diagonal neighboring buffer. For example, the neighboring data belonging to the current sub-block stored in the current sub-block neighboring buffer 610 is pushed to one storage unit of the previous line part P2 such that the storage unit of the previous line part P2 is overwritten and becomes one storage unit of the current line part PI. Thus, the neighboring data belonging to the current sub-block stored in the LI diagonal neighboring buffer 606_1 is later referenced by the residual decoding of the sub-block at the diagonal (D-1).

[0074] Another case is considered where the residual decoding of the current subblock indicates that the current subblock has all-zero coefficients. The storage area allocated in the Ll diagonal neighboring buffer for storing the neighboring data belonging to the current subblock is emptied. For example, one storage unit of the previous row portion P2 is emptied and then becomes one storage unit of the current row portion P1. This is equivalent to pushing the neighboring data (zero coefficients) belonging to the current subblock to the Ll diagonal neighboring buffer 606_1. Similarly, the neighboring data (which are zero coefficients) belonging to the current subblock stored in the Ll diagonal neighboring buffer 606_1 are later referenced by the residual decoding of the subblock at the diagonal (D-1).

[0075] Figure 19 A flowchart of a second regular mode video residual decoding method according to an embodiment of the present application is shown. These steps do not need to be performed in the exact order shown if the results are substantially the same. Figure 19 The video residual decoding method can be used in the video residual decoding apparatus 600 shown. Figure 6 The video residual decoding method can be used in the video residual decoding apparatus 600 shown. At step 1902, the residual decoding circuit 602 starts the residual decoding of the transform block. At step 1904, the residual decoding circuit 602 performs the residual decoding of the current subblock selected from the transform block according to the diagonal scan order. At step 1906, during the residual decoding of the current block, the residual decoding circuit 602 pushes the neighboring data belonging to the current subblock into the current subblock neighboring buffer 610. At step 1908, the residual decoding circuit 602 pushes the partial data in the Ll diagonal neighboring buffer 606_1 / 606_2 into the L2 diagonal neighboring buffer 608. At step 1910, the neighboring data belonging to the current subblock are stored in the Ll diagonal neighboring buffer 606_1 / 606_2. For example, when the neighboring data include one or more non-zero coefficients, the residual decoding circuit 602 pushes the neighboring data in the current subblock neighboring buffer 610 into the allocated storage space in the Ll diagonal neighboring buffer 606_1 / 606_2. For another example, when the neighboring data include all-zero coefficients, the residual decoding circuit 602 empties the allocated storage space in the Ll diagonal neighboring buffer 606_1 / 606_2. At step 1912, the residual decoding circuit 602 checks whether the last subblock of the transform block is decoded. If yes, the video residual decoding flow ends. If no, the flow proceeds to step 1904, at which the next subblock selected from the transform block according to the diagonal scan order becomes the current subblock to be decoded.

[0076] According to Figure 19The illustrated flow, when the last sub-block of a transform block is decoded, steps 1908 and 1910 are performed. Since video residual decoding uses a transform block as a decoding unit, after steps 1908 and 1910 are performed redundantly in response to the decoding of the last sub-block of a transform block, video residual decoding ends. Figure 20 The flowchart illustrates a second alternative regular mode video residual decoding method according to an embodiment of the present application. These steps do not need to be performed in the exact order illustrated if the results are substantially the same. Figure 20 The illustrated flow, when the last sub-block of a transform block is decoded, steps 1908 and 1910 are performed. Since video residual decoding uses a transform block as a decoding unit, after steps 1908 and 1910 are performed redundantly in response to the decoding of the last sub-block of a transform block, video residual decoding ends. Figure 19 The illustrated flow, when the last sub-block of a transform block is decoded, steps 1908 and 1910 are performed. Since video residual decoding uses a transform block as a decoding unit, after steps 1908 and 1910 are performed redundantly in response to the decoding of the last sub-block of a transform block, video residual decoding ends. Figure 20 The main difference between the illustrated flows in Figure 20 In the illustrated flow, step 1906 is followed by step 1912. In this way, when it is determined that the last sub-block of a transform block is decoded, video residual decoding of a transform block ends. That is, when the current sub-block is the last sub-block of a transform block and the decoding of the current sub-block is completed, video residual decoding of a transform block ends without the need for redundant processing of neighboring data.

[0077] According to the second neighboring storage mechanism in video residual decoding, the video residual decoding apparatus 600 can perform residual decoding on a transform block in TS mode by using one Ll diagonal neighboring buffer 606_1 / 606_2 and the current sub-block neighboring buffer 610 (without the need for L2 diagonal neighboring buffer 608). Figure 21 The illustrated flow, when the last sub-block of a transform block is decoded, steps 1908 and 1910 are performed. Since video residual decoding uses a transform block as a decoding unit, after steps 1908 and 1910 are performed redundantly in response to the decoding of the last sub-block of a transform block, video residual decoding ends. Figure 18 The main difference between the illustrated flows in Figure 21 The main difference between the illustrated flows in

[0078] Since the single LI diagonal neighboring buffer 606_1 / 606_2 is shared for buffering neighboring data belonging to subblocks located in the neighboring diagonal including the current diagonal D and the previous diagonal (D-1), data overwrite occurs when residual decoding of a subblock located in the current diagonal D. For example, when residual decoding of a subblock located in the current diagonal D, the neighboring data lines belonging to a subblock located in the previous diagonal (D-1) are overwritten by neighboring data belonging to a subblock located in the current diagonal D. As shown in FIG. 6B, when residual decoding of a subblock located in the current diagonal D, the LI diagonal neighboring buffer 606_1 / 606_2 can be considered to have a current row portion PI and a previous row portion P2, where the current row portion PI includes storage locations for storing neighboring data belonging to a subblock located in the current diagonal D, and the previous row portion P2 includes storage locations for storing neighboring data belonging to a subblock located in the previous diagonal (D-1). Figure 21

[0079] The current subblock neighboring buffer 610 is arranged to buffer neighboring data belonging to a current subblock located in the current diagonal D during residual decoding of the current subblock. Consider the case where residual decoding of the current subblock indicates that the current subblock has at least one non-zero coefficient. At the end of residual decoding of the current subblock, the neighboring data belonging to the current subblock stored in the current subblock neighboring buffer 610 is pushed to the allocated storage area in the LI diagonal neighboring buffer 606_1 / 606_2. For example, the neighboring data belonging to the current subblock stored in the current subblock neighboring buffer 610 is pushed to one storage location of the previous row portion P2, such that the storage location of the previous row portion P2 is overwritten and then becomes one storage location of the current row portion PI. Thus, the neighboring data belonging to the current subblock stored in the LI diagonal neighboring buffer 606_1 / 606_2 is later referenced by residual decoding of a subblock at diagonal (D+1).

[0080] Consider another case where residual decoding of the current subblock indicates that the current subblock has all-zero coefficients. The allocated storage area in the LI diagonal neighboring buffer 606_1 / 606_2 for storing neighboring data belonging to the current subblock is emptied. For example, one storage location of the previous row portion P2 is emptied and then becomes one storage location of the current row portion PI. This is equivalent to pushing neighboring data (zero coefficients) belonging to the current subblock to the LI diagonal neighboring buffer 606_1 / 606_2. Similarly, the neighboring data (zero coefficients) belonging to the current subblock stored in the LI diagonal neighboring buffer 606_1 is later referenced by residual decoding of a subblock at diagonal (D+1).

[0081] Figure 22 ​A flow chart illustrating a second TS mode video residual decoding method according to an embodiment of the present application is shown. If the results are substantially the same, these steps need not be performed in the exact order shown. Figure 22 The video residual decoding method can be employed by the video residual decoding apparatus 600 shown in Figure 6 Figure 19 The main difference between the video residual decoding methods in Figure 22 is that the video residual decoding method in Figure 22 omits step 1908. Since those skilled in the art can easily know the details of the steps in Figure 19 after reading the above paragraphs about Figure 22 , further description is omitted here for brevity.

[0082] Figure 23 A flow chart illustrating a second optional TS mode video residual decoding method according to an embodiment of the present application is shown. If the results are substantially the same, these steps need not be performed in the exact order shown. Figure 23 The video residual decoding method can be employed by the video residual decoding apparatus 600 shown in Figure 6 Figure 20 The main difference between the video residual decoding methods in Figure 23 is that the video residual decoding method in Figure 23 omits step 1908. Since those skilled in the art can easily know the details of the steps in Figure 20 after reading the above paragraphs about Figure 23 , further description is omitted here for brevity.

[0083] In order to better understand the technical features of the present application, several examples are provided. Figure 24 A diagram illustrating a first design embodiment of the present application is shown. Assume that the TB size is 16x16, the SB size is 4x4, and the current decoding diagonal is D=2. When the first neighboring storage mechanism is enabled for regular mode video residual decoding of a transform block, one L1 diagonal neighboring buffer can store the neighboring data belonging to the sub-block at the current decoding diagonal D=2 (indicated by the diagonal area located within the sub-block at the current decoding diagonal D=2), another L1 diagonal neighboring buffer can store the neighboring data belonging to the sub-block at the previous decoding diagonal D+1=3 (indicated by the diagonal area located within the sub-block at the previous diagonal D+1=3), and an L2 diagonal neighboring buffer can store the neighboring data belonging to the sub-block at the previous decoding diagonal D+2=4 (indicated by the diagonal area located within the sub-block at the previous diagonal D+2=4).

[0084] ​​When the second neighboring storage mechanism is enabled for regular mode video residual decoding of a transform block, one Ll diagonal neighboring buffer is shared to store neighboring data belonging to the sub-block located at the current decoding diagonal D=2 (indicated by the hatched area within the sub-block located at the current decoding diagonal D=2) and for storing neighboring data belonging to the sub-block located at the previous decoding diagonal D+1=3 (indicated by the hatched area within the sub-block located at the previous decoding diagonal D+1=3). In addition, the L2 diagonal neighboring buffer can store neighboring data belonging to the sub-block located at the previous decoding diagonal D+2=4 (indicated by the hatched area within the sub-block located at the previous decoding diagonal D+2=4).

[0085] As mentioned above, in order to reduce hardware cost, the number of storage units allocated in the Ll diagonal neighboring buffer for storing the required neighboring data belonging to the head sub-block and / or the tail sub-block of a diagonal line can be reduced. Figure 25 A diagram showing a second design embodiment of the present invention. Compared to the first design embodiment, the second design embodiment can relax the buffering requirement.

[0086] Figure 26 A diagram showing a third design embodiment of the present invention. Assume that the TB size is 16x16, the SB size is 4x4, and the current decoding diagonal is D=4. When the first neighboring storage mechanism is enabled for TS mode video residual decoding of a transform block, one Ll diagonal neighboring buffer can store neighboring data belonging to the sub-block located at the current decoding diagonal D=4 (indicated by the hatched area within the sub-block located at the current decoding diagonal D=4), and another Ll diagonal neighboring buffer can store neighboring data belonging to the sub-block located at the previous decoding diagonal D-1=3 (indicated by the hatched area within the sub-block located at the previous decoding diagonal D-1=3).

[0087] When the second neighboring storage mechanism is enabled for TS mode video residual decoding of a transform block, one Ll diagonal neighboring buffer is shared to store neighboring data belonging to the sub-block located at the current decoding diagonal D=4 (indicated by the hatched area within the sub-block located at the current decoding diagonal D=4) and for storing neighboring data belonging to the sub-block located at the previous decoding diagonal D-1=3 (indicated by the hatched area within the sub-block located at the previous decoding diagonal D-1=3).

[0088] As mentioned above, in order to reduce hardware cost, the number of storage units allocated in the Ll diagonal neighboring buffer for storing the required neighboring data belonging to the head sub-block and / or the tail sub-block of a diagonal line can be reduced. Figure 27 A diagram showing a fourth design embodiment of the present invention. Compared to the third design embodiment, the fourth design embodiment can relax the buffering requirement.

[0089] In one example buffer design, the diagonal adjacent buffers can be implemented by a plurality of shift registers. Figure 28 FIG. 1 shows a diagram of an implementation of the adjacent memory device 604 according to one embodiment of the present disclosure. Figure 6 FIG. 1 shows a diagram of an implementation of the adjacent memory device 604 according to one embodiment of the present disclosure.

[0090] In another example buffer design, the diagonal adjacent buffers can be implemented by an index pointer and a plurality of cells. Figure 29 FIG. 1 shows a diagram of an implementation of the adjacent memory device 604 according to one embodiment of the present disclosure. Figure 6 FIG. 1 shows a diagram of an implementation of the adjacent memory device 604 according to one embodiment of the present disclosure.

[0091] Many modifications and variations of this application can be made without departing from its spirit and scope, which is defined by the appended claims.

Claims

1. A video residual decoding apparatus comprising: a residual decoding circuit arranged to perform residual decoding on a transform block, wherein the transform block is divided into M sub-blocks, M being a positive integer, and the M sub-blocks are performed residual decoding processing in a diagonal scan order; and a neighboring storage device arranged to store neighboring data belonging to a plurality of neighboring sub-blocks and referenced by the residual decoding of a current sub-block, wherein the neighboring data belonging to a sub-block is derived from a residual decoding result of the sub-block, and a storage size of the neighboring storage device is not greater than a maximum amount of neighboring data derived from a plurality of residual decoding results of N sub-blocks, N being a positive integer, N being smaller than M.

2. The video residual decoding apparatus of claim 1, wherein, the neighboring storage device comprises: a first diagonal neighboring buffer arranged to buffer neighboring data belonging to a plurality of sub-blocks located at a current diagonal and referenced by residual decoding of a plurality of sub-blocks located at a next diagonal, wherein the residual decoding of the plurality of sub-blocks located at the current diagonal is followed by the residual decoding of the plurality of sub-blocks located at the next diagonal according to the diagonal scan order; and a second diagonal neighboring buffer arranged to buffer neighboring data belonging to a plurality of sub-blocks located at a previous diagonal and referenced by the residual decoding of the plurality of sub-blocks located at the current diagonal, wherein the residual decoding of the plurality of sub-blocks located at the previous diagonal is followed by the residual decoding of the plurality of sub-blocks located at the current diagonal according to the diagonal scan order.

3. The video residual decoding apparatus of claim 2, wherein, a buffer size of the first diagonal neighboring buffer is equal to an amount of neighboring data belonging to only part of coefficients of the plurality of sub-blocks located at diagonals in the transform block, the diagonals having a maximum length.

4. The video residual decoding apparatus of claim 2, wherein, a buffer size of the second diagonal neighboring buffer is equal to an amount of neighboring data belonging to only part of coefficients of the plurality of sub-blocks located at diagonals in the transform block, the diagonals having a maximum length.

5. The video residual decoding apparatus of claim 2, wherein, at an end of the residual decoding of the plurality of sub-blocks located at the current diagonal, the second diagonal neighboring buffer is arranged to start buffering neighboring data belonging to the plurality of sub-blocks located at a next diagonal.

6. The video residual decoding apparatus of claim 2, wherein, the neighboring storage device further comprises: a third diagonal neighboring buffer arranged to buffer neighboring data belonging to a plurality of sub-blocks located at another previous diagonal and referenced by the residual decoding of the plurality of sub-blocks located at the current diagonal, wherein the residual decoding of the plurality of sub-blocks located at the another previous diagonal is followed by the residual decoding of the plurality of sub-blocks located at the previous diagonal according to the diagonal scan order.

7. The video residual decoding apparatus of claim 6, wherein, a buffer size of the third diagonal neighboring buffer is equal to an amount of neighboring data belonging to only part of coefficients of the plurality of sub-blocks located at diagonals in the transform block, the diagonals having a second maximum length.

8. The video residual decoding apparatus of claim 6, wherein, at an end of the residual decoding of the plurality of sub-blocks located at the current diagonal, part of data stored in the second diagonal neighboring buffer is pushed into the third diagonal neighboring buffer.

9. The video residual decoding apparatus of claim 2, wherein, The neighboring storage device further comprises: a current sub-block neighboring buffer arranged to cache neighboring data belonging to a current sub-block located on the current diagonal line during the residual decoding of the current sub-block.

10. The video residual decoding apparatus of claim 9, wherein, The residual decoding of the current sub-block indicates that the current sub-block has at least one non-zero coefficient, and at the end of the residual decoding of the current sub-block, the neighboring data belonging to the current sub-block stored in the current sub-block neighboring buffer is pushed to the first diagonal line neighboring buffer.

11. The video residual decoding apparatus of claim 9, wherein, The residual decoding of the current sub-block indicates that the current sub-block has all-zero coefficients, and the storage area allocated in the first diagonal line neighboring buffer for storing the neighboring data belonging to the current sub-block is emptied.

12. The video residual decoding apparatus of claim 1, wherein, The neighboring storage device comprises: a first diagonal line neighboring buffer arranged as a shared buffer for caching neighboring data belonging to a plurality of sub-blocks located on a current diagonal line and referenced by the residual decoding of a plurality of sub-blocks located on a next diagonal line, and for caching neighboring data belonging to a plurality of sub-blocks located on a previous diagonal line and referenced by the residual decoding of the plurality of sub-blocks located on the current diagonal line; wherein according to the diagonal scan order, the residual decoding of the plurality of sub-blocks located on the previous diagonal line is followed by the residual decoding of the plurality of sub-blocks located on the current diagonal line, and the residual decoding of the plurality of sub-blocks located on the current diagonal line is followed by the residual decoding of the plurality of sub-blocks located on the next diagonal line; wherein during the residual decoding of the plurality of sub-blocks located on the current diagonal line, the neighboring data belonging to a first sub-block located on the previous diagonal line is overwritten by the neighboring data belonging to a second sub-block located on the current diagonal line.

13. The video residual decoding apparatus of claim 12, wherein, The neighboring storage device further comprises: a second diagonal line neighboring buffer arranged to cache neighboring data belonging to a plurality of sub-blocks located on another previous diagonal line and referenced by the residual decoding of the plurality of sub-blocks located on the current diagonal line, wherein according to the diagonal scan order, the residual decoding of the plurality of sub-blocks located on the another previous diagonal line is followed by the residual decoding of the plurality of sub-blocks located on the previous diagonal line.

14. The video residual decoding apparatus of claim 13, wherein, Before the neighboring data belonging to the first sub-block located on the previous diagonal line is overwritten by the neighboring data belonging to the second sub-block located on the current diagonal line, part of data of the first sub-block located on the previous diagonal line is pushed to the second diagonal line neighboring buffer.

15. The video residual decoding apparatus of claim 12, wherein, The neighboring storage device further comprises: a current sub-block neighboring buffer arranged to cache neighboring data belonging to a current sub-block located on the current diagonal line during the residual decoding of the current sub-block.

16. The video residual decoding apparatus of claim 15, wherein, The residual decoding of the current sub-block indicates that the current sub-block has at least one non-zero coefficient, and at the end of the residual decoding of the current sub-block, the neighboring data belonging to the current sub-block stored in the current sub-block neighboring buffer is pushed to the first diagonal line neighboring buffer.

17. The video residual decoding apparatus of claim 15, wherein, The residual decoding of the current sub-block indicates that the current sub-block has all-zero coefficients, and a storage region allocated in the first diagonal-diagonal line neighboring buffer for storing neighboring data belonging to the current sub-block is emptied.

18. The video residual decoding apparatus of claim 1, wherein, The diagonal scan order is from a lower right sub-block of the transform block to an upper left sub-block of the transform block.

19. The video residual decoding apparatus of claim 1, wherein, The diagonal scan order is from an upper left sub-block of the transform block to a lower right sub-block of the transform block.

20. A method of video residual decoding, comprising: performing residual decoding on a transform block, wherein the transform block is divided into M sub-blocks, M being a positive integer, and the M sub-blocks are performed residual decoding processing in a diagonal scan order; and storing neighboring data belonging to multiple neighboring sub-blocks and referenced by the residual decoding of a current sub-block via a neighboring storage device, wherein the neighboring data belonging to a sub-block is derived from a residual decoding result of the sub-block, and a storage size of the neighboring storage device is no more than a maximum data amount of neighboring data derived from multiple residual decoding results of N sub-blocks, N being a positive integer and N being less than M.

Images