Spatially changing transformations with adaptive transformation types
Adaptive SVT-V and SVT-H transformations with DST-7 and DCT-8 improve video decoding efficiency and quality by optimizing block positions and sizes, addressing signaling overhead and encoder complexity issues.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- HUAWEI TECH CO LTD
- Filing Date
- 2024-10-30
- Publication Date
- 2026-06-11
AI Technical Summary
Existing spatially varying transform (SVT) schemes in video coding face issues with heavy signaling overhead and increased encoder complexity due to the number of candidate positions, and the transformation blocks are often too small to effectively cover major residuals.
Adaptive use of spatially varying transformations (SVT) with types SVT-V and SVT-H, employing DST-7 and DCT-8 transformations based on block position and size, reducing candidate positions to improve decoding efficiency and quality.
Enhances decoding quality and efficiency by adaptively using multiple transformation types, simplifying implementation with limited algorithms and reduced positional information overhead.
Smart Images
Figure 0007873279000007 
Figure 0007873279000008 
Figure 0007873279000009
Abstract
Description
Technical Field
[0001] The present disclosure relates to video decoding techniques, specifically, a video decoding method and related apparatus that employ a spatially varying transform having an adaptive transform type.
Background Art
[0002] Video coding such as H.265 is based on a framework of prediction and transformation. In an encoder, an image block (including multiple pixels) can be decomposed into a prediction block and a residual block, and prediction information (e.g., prediction mode and motion vector information) and residual information (e.g., transform mode, transform coefficients, and quantization parameters) are coded into a bitstream. In a decoder, the prediction information and residual information are parsed. According to the prediction information, intra or inter prediction is performed to generate prediction samples. According to the residual information, inverse quantization and inverse transformation are sequentially performed to generate residual samples. The prediction samples and the residual samples are added together to obtain reconstructed samples.
[0003] A spatially varying transform (SVT) was developed to improve video coding efficiency. For a rectangular residual block of width w and height h (i.e., w×h), a transform block smaller than the residual block is used to transform a part of the residual block, and the remaining part of the residual block is not coded. The logical basis behind SVT is that the residuals may not be equally distributed in the residual block. Using smaller transform blocks with adaptive positions can capture the major residuals in the residual block, and thus achieve better coding efficiency than transforming all the residuals in the residual block.
[0004] When SVT is applied to residual blocks of size w×h, the size and position information of the transformed blocks are coded into the video bitstream, so that the decoder can reconstruct the transformed blocks and place them in the appropriate positions of the prediction blocks associated with the residual blocks.
[0005] In one example, three types of SVT blocks may be used for residual blocks, as shown in Figure 1. 1) SVT-I: w_t=w / 2, h_t=h / 2. In this case, w_t and h_t represent the width and height of the transformation block, respectively, and w and h represent the width and height of the residual block. In other words, the width and height of both the transformation block and the residual block are both half of those of the residual block. 2) SVT-II: w_t=w / 4, h_t=h. 3) SVT-III: w_t=w, h_t=h / 4.
[0006] The type information of the SVT block is coded into the bitstream.
[0007] The position of a transform block is represented by a positional offset (x,y) to the upper-left corner of the residual block, where x represents the horizontal distance in pixels between the upper-left corner of the transform block and that of the residual block, and y represents the vertical distance in pixels between the upper-left corner of the transform block and that of the residual block. Each position that places the transform block within the residual block is a candidate position. For a residual block, the number of candidate positions is (w-w_t+1)×(h-h_t+1) for the type of SVT. More specifically, for a 16×16 residual block, there are 81 candidate positions when SVT-I is used, and 13 candidate positions when SVT-II or SVT-III is used. The values of x and y are coded into the bitstream. To reduce the complexity of SVT-I, a subset of 32 positions is selected from the 81 candidate positions as allowed candidate positions for SVT-I.
[0008] One drawback of the SVT scheme is the heavy signaling overhead of positional information. Moreover, encoder complexity can increase significantly with the number of positions tested in rate-distortion optimization (RDO). As the number of candidate positions increases with the size of the residual block, the overhead can become even greater for larger residual blocks, such as 32x32 or 64x128.
[0009] Another drawback of the SVT scheme is that the size of the transformation block is only one-quarter the size of the residual block. The transformation block is not large enough to cover the major residuals in the residual block with high certainty.
[0010] Therefore, a simple SVT has been developed, and as shown in Figure 2, two types of SVT blocks, denoted as SVT-H and SVT-V, are used for residual coding. 1) SVT-V: w_t=w / 2 and h_t=t. 2) SVT-H: w_t=w and h_t=h / 2.
[0011] SVT-V is similar to SVT-II, and SVT-H is similar to SVT-III. Compared to SVT-II and SVT-III, the transformation block in SVT-V and SVT-H is extended to half the residual block, allowing it to cover further residuals in the residual block.
[0012] Candidate positions are determined by the Candidate Position Step Size (CPSS). Therefore, candidate positions are spaced equally apart as specified by the CPSS. The number of candidate positions is reduced to five or less to mitigate the overhead of positional information, in addition to the encoder complexity required to determine the best transform block position. [Overview of the Initiative]
[0013] This disclosure provides a video decoding method and related apparatus that employs spatially varying transformations having adaptive transformation types to improve decoding quality.
[0014] The above and other objectives are achieved by the subject matter of the independent claims. Further embodiments are evident from the dependent claims, specification and drawings.
[0015] In accordance with a first aspect, the present disclosure relates to a video decoding method. The method is performed by a video decoding device. The method includes determining the use of a spatially varying transform (SVT) for a residual block, determining the SVT type for the residual block when the SVT is used for the residual block, wherein the SVT type for the residual block is either SVT-V or SVT-H, determining the transform block position of the transform block of the residual block, determining the transform type of the transform block, wherein the transform type indicates a horizontal transform and a vertical transform of the transform block, wherein at least one of the horizontal and vertical transforms is DST-7, and reconstructing the residual block based on the transform type, the transform block position, and the transform coefficients of the transform block.
[0016] In a possible embodiment of the method according to the first aspect, when the SVT type for the residual block is SVT-V and the conversion block position of the conversion block is such that it covers the upper left corner of the residual block, the horizontal conversion is DCT-8 and the vertical conversion is DST-7.
[0017] In a possible embodiment of the method according to the first aspect, when the SVT type for the residual block is SVT-V and the conversion block position of the conversion block is such that it covers the lower right corner of the residual block, the horizontal conversion is DST-7 and the vertical conversion is DST-7.
[0018] In a possible embodiment of the method according to the first aspect, when the SVT type for the residual block is SVT-H and the conversion block position of the conversion block is such that it covers the upper left corner of the residual block, the horizontal conversion is DST-7 and the vertical conversion is DCT-8.
[0019] In a possible embodiment of the method according to the first aspect, when the SVT type for the residual block is SVT-H and the conversion block position of the conversion block is such that it covers the lower right corner of the residual block, the horizontal conversion is DST-7 and the vertical conversion is DST-7.
[0020] In a second aspect, the present disclosure relates to a video decoding device having: a unit configured to determine the use of a spatially varying transformation (SVT) for a residual block; a unit configured to determine the SVT type for a residual block when an SVT is used for a residual block, wherein the SVT type for the residual block is either SVT-V or SVT-H; a unit configured to determine the transformation block position of the transformation block of the residual block; a unit configured to determine the transformation type of the transformation block, wherein the transformation type indicates a horizontal transformation and a vertical transformation of the transformation block, wherein at least one of the horizontal and vertical transformations is DST-7; and a unit configured to reconstruct the residual block based on the transformation type, the transformation block position, and the transformation coefficients of the transformation block.
[0021] In a possible embodiment of the apparatus according to the second aspect, when the SVT type for the residual block is SVT-V and the conversion block position of the conversion block is such that it covers the upper left corner of the residual block, the horizontal conversion is DCT-8 and the vertical conversion is DST-7.
[0022] In a possible embodiment of the apparatus according to the second aspect, when the SVT type for the residual block is SVT-V and the conversion block position of the conversion block is such that it covers the lower right corner of the residual block, the horizontal conversion is DST-7 and the vertical conversion is DST-7.
[0023] In a possible embodiment of the apparatus according to the second aspect, when the SVT type for the residual block is SVT-H and the conversion block position of the conversion block is such that it covers the upper left corner of the residual block, the horizontal conversion is DST-7 and the vertical conversion is DCT-8.
[0024] In a possible embodiment of the apparatus according to the second aspect, when the SVT type for the residual block is SVT-H and the conversion block position of the conversion block covers the lower right corner of the residual block, the horizontal conversion is DST-7 and the vertical conversion is DST-7.
[0025] According to a third aspect, the present disclosure relates to a video decoding apparatus having one or more processors and a non-transitory computer-readable storage medium coupled to the processors and storing programming for execution by the processors, the programming configuring the video decoding apparatus to perform the method according to the first aspect when executed by the processors.
[0026] According to a fourth aspect, the present disclosure relates to a non-volatile computer-readable storage medium storing computer instructions that cause one or more processors to perform the steps of the method according to the first aspect when executed by the one or more processors.
[0027] It can be seen that the present disclosure adaptively uses multiple conversion types of conversion blocks based on the SVT type and position information, and thus can improve the decoding quality and decoding efficiency. Furthermore, the number of conversion algorithms is limited in some embodiments, and thus the implementation of the decoding apparatus can be simplified. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] [Figure 1] Explanatory diagrams of SVT-I, SVT-II, and SVT-III. [Figure 2] Explanatory diagrams of SVT-V and SVT-H. [Figure 3] Explanatory diagrams of candidate positions of SVT-V and SVT-H blocks. <null> [Figure 4] Explanatory diagrams of SVT-V and SVT-H having three candidate positions. [Figure 5] A flowchart of a video decoding method according to an embodiment of the present disclosure. [Figure 6] This is a schematic diagram of a video decoding device according to an embodiment of the present disclosure. [Modes for carrying out the invention]
[0029] This disclosure introduces an improved SVT scheme. The improvement is that the type of horizontal and vertical transformation of an SVT block is determined based on the SVT type and the SVT block location. The horizontal transformation may differ from the vertical transformation.
[0030] The first embodiment describes a process for decoding residual blocks. A bitstream containing at least one picture of video data is decoded. The picture is divided into multiple rectangular image regions, each region corresponding to a coding tree unit (CTU). The CTU is partitioned into multiple blocks, similar to coding units in HEVC, according to block partitioning information contained in the bitstream. The coding information of the blocks is parsed from the bitstream, and the pixels of the blocks are reconstructed based on the coding information.
[0031] In one embodiment, the SVT is limited to being used for inter-predicted blocks. In other embodiments, the SVT may also be used for intra-predicted blocks.
[0032] In one example, SVT may be permitted for blocks using a specific interpretation method (e.g., motion compensation based on a translation model), but not for blocks using any other interpretation method (e.g., motion compensation based on an affine model). In another example, SVT may be permitted for prediction blocks using merge mode or AMVP (advanced motion vector prediction) mode with 1 / 4 pixel motion vector difference accuracy, but not for prediction blocks using affine merge mode, affine intermode, or AMVP mode with 1 pixel or 4 pixel motion vector difference accuracy. In yet another example, SVT may be permitted for prediction blocks using merge mode with a merge index less than 2, but not for prediction blocks using merge mode with a merge index not less than 2. Merge mode and AMVP mode may be referenced in the H.265 / HEVC standard. Affine merge mode and affine intermode may be referenced in the JEM (Joint Exploration Model) codec from JVET (Joint Video Exploration Team).
[0033] In one example, a block may refer to a coding unit, which may contain one prediction block and one residual block. The prediction block may contain all the prediction samples of the coding unit, and the residual block may contain all the residual samples of the coding unit, and the prediction block may have the same size as the residual block. In another example, a block may refer to a coding unit, which may contain two prediction blocks and one residual block, each prediction block may contain a portion of the prediction samples of the coding unit, and the residual block may contain all the residual samples of the coding unit. In yet another example, a block may refer to a coding unit, which may contain two prediction blocks and four residual blocks. The partition pattern of residual blocks in a coding unit, such as a residual quadtree (RQT) in HEVC, may be conveyed in the bitstream.
[0034] A block may contain only the Y component (luma) of an image sample (or pixel), or it may contain the Y, U (chrominance), and V (chrominance) components of an image sample.
[0035] Size w×h residual block R O This can be reconstructed by the following steps.
[0036] Step 1. Residual Block R O Determine the conversion block size.
[0037] Step 1.1. Determine the use of SVT according to the syntax elements. For residual blocks that are permitted to use SVT, a flag (i.e., svt_flag) is parsed from the bitstream if the residual block has a non-zero conversion coefficient for the Y component (or if it has a non-zero conversion coefficient for any color component). The flag indicates whether the residual block is coded with a conversion block of the same size as the residual block (e.g., svt_flag=0) or whether the residual block is coded with a conversion block smaller than the size of the residual block (e.g., svt_flag=1). Whether a block has a non-zero conversion coefficient for a color component may be indicated by a color component coded block flag (cbf), as used in HEVC. Whether a block has a non-zero conversion coefficient for any color component may be indicated by a root coded block flag (root cbf), as used in HEVC.
[0038] In one example, a block is permitted to use SVT if the following conditions are met: 1) Blocks are predicted using interpretation. 2) Either the block width or the block height falls within a predetermined range [a1, a2], for example, a1=16 and a2=64, or a1=8 and a2=64, or a1=16 and a2=128. The values of a1 and a2 can be fixed. The values can also be derived from the Sequence Parameter Set (SPS) or the slice header.
[0039] In other examples, a block is permitted to use SVT if the following conditions are met: 1) Blocks are predicted using a merge mode with a merge index smaller than a threshold (e.g., 1, 2, or 3), or using an AMVP mode with 1 / 4 pixel motion vector difference accuracy. 2) One dimension of the block falls within a given range [a1, a2], and the other dimension of the block is not greater than the threshold a3, for example, a1=8, a2=32, and a3=32. Parameter a1 may be set to twice the minimum transformation size, and a2 and a3 may both be set to the maximum transformation size. The values of a1, a2, and a3 can be fixed values. The values can also be derived from a sequence parameter set (SPS) or slice header.
[0040] If the block does not use SVT, the converted block size is set to w×h. Otherwise, step 1.2 is applied to determine the converted size.
[0041] Step 1.2. Determine the type of SVT according to the syntax elements and derive the transformed block size according to the SVT type. The permitted SVT types for residual blocks are determined based on the width and height of the residual block. SVT-V is permitted if w is within the range [a1, a2] and h is not greater than a3, and SVT-H is permitted if h is within the range [a1, a2] and w is not greater than a3. The SVT may be used only for the Y component, or it may be used for all three components, i.e., the Y component, U component and V component. When the SVT is used only for the Y component, the Y component residual is transformed by the SVT, and the U and V components are transformed according to the size of the residual block.
[0042] If both SVT-V and SVT-H are permitted, one flag (i.e., svt_type_flag) is parsed from the bitstream to indicate whether SVT-V is used for the residual block (e.g., svt_type_flag=0) or SVT-H is used (e.g., svt_type_flag=1), and the conversion block size is set according to the SVT type indicated by the signal (i.e., for SVT-V, W_t=w / 2 and h_t=h, and for SVT-H, w_t=w and h_t=h / 2). If only SVT-V or only SVT-H is permitted, svt_type_flag is not parsed from the bitstream, and the conversion block size is set according to the permitted SVT type.
[0043] Step 2. Determine the transformation block position according to the syntax elements, and determine the transformation type of the transformation block based on the SVT type and transformation block position information.
[0044] Step 2.1: Determine the transformation block position according to the syntax element.
[0045] The position index P is parsed from the bitstream, and the position offset Z of the upper-left corner of the transformation block relative to the upper-left corner of the residual block is determined as Z = s × P, where s is the candidate position step size (CPSS). The value of P is one of 0, 1, ..., (w - w_t) / s when SVT-V is used, or one of 0, 1, ..., (h - h_t) / s when SVT-H is used. More specifically, if (0,0) represents the coordinates of the upper-left corner of the residual block, then the coordinates of the upper-left corner of the transformation block are (Z,0) for SVT-V or (0,Z) for SVT-H.
[0046] In one example, CPSS is calculated as s = w / M1 for SVT-V, or s = h / M2 for SVT-H, where w and h are the width and height of the residual block, respectively, and M1 and M2 are predetermined integers in the range of 2 to 8. More candidate positions are allowed with larger M1 or M2 values. In this example, both M1 and M2 are set to 8. Therefore, the value of P is one between 0 and 4. The candidate positions are shown in Figure 3.
[0047] In other examples, CPSS is calculated as s=max(w / M1,Th1) for SVT-V or s=max(h / M2,Th2) for SVT-H, where Th1 and Th2 are predefined integers specifying the minimum step size. Th1 and Th2 are integers greater than or equal to 2. In this example, Th1 and Th2 are set to 4, and M1 and M2 are set to 8. In this example, different block sizes may have a different number of candidate positions. For example, when w=8, two candidate positions (represented by Figures 3(a) and 3(e)) are available for selection; when w=16, three candidate positions (represented by Figures 3(a), 3(c), and 3(e)) are available for selection; and when w>16, five positions are available for selection.
[0048] In another example, CPSS is calculated as s=w / M1 for SVT-V or s=h / M2 for SVT-H, where M1 and M2 are set to 4. Therefore, three candidate positions are allowed.
[0049] In other examples, CPSS is calculated as s=w / M1 for SVT-V or s=h / M2 for SVT-H, where M1 and M2 are set to 2. Therefore, two candidate positions are allowed.
[0050] In another example, CPSS is calculated as s = max(w / M1, Th1) for SVT-V or s = max(h / M2, Th2) for SVT-H, where Th1 and Th2 are set as 2, M1 is set as 8 when w ≥ h or 4 when w < h, and M2 is set as 8 when h ≥ w or 4 when h < w. In this case, the number of candidate positions for SVT-H or SVT-V may further depend on the aspect ratio of the residual block.
[0051] In another example, CPSS is calculated as s = max(w / M1, Th1) for SVT-V or s = max(h / M2, Th2) for SVT-H, where the values of M1, M2, Th1, and Th2 are derived from a high-level syntax structure (e.g., sequence parameter set) in the bitstream. M1 and M2 may share the same value parsed from syntax elements, and Th1 and Th2 may share the same value parsed from other syntax elements.
[0052] The position index P may be binarized into one or more bins using a truncated unary code. For example, when the P value is within the range of 0 to 4, the P values 0, 4, 2, 3, and 1 are binarized as 0, 01, 001, 0001, and 0000 respectively, and when the P value is within the range of 0 to 1, the P values 0 and 1 are binarized as 0 and 1 respectively.
[0053] The position index P may be binarized into one or more bins using one most likely position and some remaining positions. When the left and upper neighbors are available, the most likely position may be set as the position covering the bottom right corner of the residual block. In one example, when the P value is within the range of 0 to 4 and position 4 is set as the most likely position, the P values 4, 0, 1, 2, and 3 are binarized as 1, 000, 001, 010, and 011 respectively, and when the P value is within the range of 0 to 2 and position 2 is set as the most likely position, the P values 2, 0, and 1 are binarized as 1, 01, and 00 respectively.
[0054] Step 2.2: Determine the transformation type of the transformation block based on the SVT type and transformation block position information. The transformation type includes horizontal and vertical transformations of 2D separable transformations.
[0055] As shown in Figure 4, consider the case where, for example, three candidate positions are allowed. Position 0 covers the upper left corner, position 2 covers the lower right corner, and position 1 is in the middle of the residual block. As shown in Figure 4, there are three positions for both SVT-V and SVT-H.
[0056] In other examples, two candidate positions are allowed. Position 0 covers the upper left corner, and position 1 covers the upper right corner (the same as position 2 in Figure 4). That is, there are two positions for both SVT-V and SVT-H.
[0057] A two-dimensional transformation may be separable into a one-dimensional horizontal transformation and a one-dimensional vertical transformation. A forward two-dimensional transformation, which converts residuals into transformation coefficients, can be achieved by first applying a horizontal transformation to the residual block to generate block TA, and then applying a vertical transformation to block TA to generate a transformation coefficient block, as is done in the JEM codec. Thus, an inverse two-dimensional transformation, which converts transformation coefficients back to residuals, can be achieved by first applying an inverse vertical transformation to the transformation coefficient block to generate block TB, and then applying an inverse horizontal transformation to block TB to generate a residual block, as is done in the JEM codec.
[0058] In one example, as listed in Table I, the horizontal and vertical transformers for SVT-V position 0 are DCT-8 and DST-7, the horizontal and vertical transformers for SVT-V position 1 are DST-1 and DST-7, the horizontal and vertical transformers for SVT-V position 2 are DST-7 and DST-7, the horizontal and vertical transformers for SVT-H position 0 are DST-7 and DCT-8, the horizontal and vertical transformers for SVT-H position 1 are DST-7 and DST-1, and the horizontal and vertical transformers for SVT-H position 2 are DST-7 and DST7. In this example, the vertical transformer for SVT-V and the horizontal transformer for SVT-H are set as DST-7, and the other transformers are based on the SVT position. [Table 1]
[0059] In other examples, horizontal and vertical transformations for different SVT types and locations are listed in Table II. In this example, the vertical transformation for SVT-V and the horizontal transformation for SVT-H are set as DCT-2, while other transformations are based on the SVT location. [Table 2]
[0060] In other examples, horizontal and vertical transformations for different SVT types and locations are listed in Table III. In this example, horizontal and vertical transformations are determined solely by the SVT location. [Table 3]
[0061] Other examples of horizontal and vertical transformations for different SVT types and locations are listed in Table IV. [Table 4]
[0062] In other examples, horizontal and vertical transformations for different SVT types and locations are listed in Table V. [Table 5]
[0063] In other examples, horizontal and vertical transformations for different SVT types and locations are listed in Table VI. [Table 6]
[0064] Multiple position-dependent transformations may be applied only to the lumern transformation block, and the corresponding chromern transformation block always uses inverse DCT-2 in the inverse transformation process.
[0065] Step 3. Parse the conversion coefficients of the conversion blocks based on the conversion block size.
[0066] This is a commonly used process in video decoding, such as conversion factor parsing in HEVC or H.264 / AVC. The conversion factors may be coded using run-length coding or more elegantly as a set of conversion factor groups (CGs).
[0067] Step 3 may be performed before Step 2.
[0068] Step 4. Residual block R based on the conversion coefficient, conversion block position, and inverse conversion type. O Reconstruct it.
[0069] Inverse quantization and inverse transform of size w_t × h_t are applied to the transform coefficients to recover the residual sample. The size of the residual sample is w_t × h_t, which is the same as the transform block size. The inverse transform is a two-dimensional separable transform. The inversely quantized transform coefficient block is first transformed by an inverse vertical transform to produce a block TC, and then the block TC is transformed by an inverse horizontal transform, where the inverse horizontal and inverse vertical transforms are determined in step 2.2 based on the transform block position, or based on both the transform block position and the SVT type of the transform block.
[0070] The residual sample is placed in residual block R according to the transformation block position. O The samples are assigned to the corresponding regions within the residual block, and the remaining samples within the residual block are set to zero. For example, if SVT-V is used, the number of candidate positions is 5, and the position index is 4, the reconstructed residual samples are assigned to region A in Figure 3(e), and the region to the left of region A, with size (w / 2) × h, has zero residuals.
[0071] After steps 1 through 4 are performed, the reconstructed residual block may be composed of prediction blocks to generate the reconstructed sample in the coding unit. Filtering processes, such as deblocking filters and sample adaptive offset (SAO) processing in HEVC, may be applied to the reconstructed sample afterward.
[0072] Unlike existing solutions, this disclosure adaptively uses multiple transformation types of transformation blocks based on SVT type and location information.
[0073] Figure 5 is a flowchart of an exemplary method of video decoding that employs spatially varying transformations with adaptive transformation types. The method may begin when the decoder receives a bitstream. The method uses the bitstream to determine the predicted blocks and the transformed residual blocks. The method may also determine the transformed blocks used to determine the residual blocks. The residual blocks and predicted blocks are then used to reconstruct the image blocks. The method is described from the decoder's perspective, but it should be noted that a similar method may be used to encode video by using SVT (for example, in reverse). Here, the method includes:
[0074] Step 501: Decide on the use of SVT for residual block. The specific process for this decision is the same as in Step 1.1.
[0075] Step 502, when the SVT is used for the residual block, determine the SVT type for the residual block. In this case, the SVT type for the residual block is either SVT-V type or SVT-H type. The SVT-V type indicates that the width of the conversion block for the residual block is half the width of the residual block and the height of the conversion block is the same as the height of the residual block (shown in Figure 4). The SVT-H type indicates that the width of the conversion block is the same as the width of the residual block and the height of the conversion block is half the height of the residual block (shown in Figure 4). The specific process for determination is the same as in Step 1.2.
[0076] Step 503: Derive the conversion block size of the conversion block according to the SVT type. The specific derivation process is the same as in Step 1.2.
[0077] Step 504: Determine the position of the transformation block. The specific process for this determination may be similar to that in Step 2.1.
[0078] Alternatively, if there are two candidate positions for the SVT type, a 1-bit flag may be used to indicate the conversion block position of the conversion block in the residual block. For example, if only positions 0 and 2 in Figure 4 are used for SVT-V, the 1-bit flag is sufficient to indicate whether the conversion block position is position 0 or position 2. If only positions 0 and 2 in Figure 4 are used for SVT-H, the 1-bit flag is sufficient to indicate whether the conversion block position is position 0 or position 2.
[0079] Step 505, the conversion type of the conversion block is determined according to the SVT type and the conversion block position of the conversion block, in which case the conversion type indicates horizontal and vertical conversion of the conversion block, and at least one of the horizontal and vertical conversions is DST-7. The specific process of determination may be the same as in Step 2.2.
[0080] The specific conversion type may be any one of the conversion types from Tables I, IV, and V described above, or any one of the conversion types from Tables II, III, and VI described above that includes DST-7.
[0081] For example, if the SVT type for the residual block is SVT-V type, and the position of the transform block is such that it covers the upper left corner of the residual block (i.e., position 0 in Figure 4), then the horizontal transform is DCT-8 and the vertical transform is DST-7.
[0082] For example, if the SVT type for the residual block is SVT-V type, and the position of the transform block is such that it covers the lower right corner of the residual block (i.e., position 2 in Figure 4), then the horizontal transform is DST-7 and the vertical transform is DST-7.
[0083] For example, if the SVT type for the residual block is SVT-H type, and the position of the transform block is such that it covers the upper left corner of the residual block (i.e., position 0 in Figure 4), then the horizontal transform is DST-7 and the vertical transform is DCT-8.
[0084] For example, if the SVT type for the residual block is SVT-H type, and the position of the transformation block is such that it covers the lower right corner of the residual block (i.e., position 2 in Figure 4), then the horizontal transformation is DST-7 and the vertical transformation is DST-7.
[0085] Step 506: Parse the conversion coefficients of the conversion block according to the conversion block size. The specific process of parsing may be similar to that in Step 3.
[0086] Step 507: Reconstruct the residual blocks based on the transformation type, the location of the transformed blocks, and the transformation coefficients of the transformed blocks. The specific reconstruction process may be similar to that of Step 4.
[0087] This disclosure shows that multiple transformation types of transformation blocks can be adaptively used based on the SVT type and location information, thereby improving decoding quality and decoding efficiency. Furthermore, the number of transformation algorithms is limited in some embodiments, thus simplifying the implementation of the decoding device.
[0088] This disclosure relates to a video decoding device configured to implement the technology of the present application: A unit configured to determine the use of spatially varying transformations (SVTs) for residual blocks. The specific process for this determination is the same as in step 1.1. A unit configured to determine the SVT type for a residual block when the SVT is used for a residual block, wherein the SVT type for the residual block is either SVT-V type or SVT-H type, where SVT-V type indicates that the width of the conversion block of the residual block is half the width of the residual block and the height of the conversion block is the same as the height of the residual block, and SVT-H type indicates that the width of the conversion block is the same as the width of the residual block and the height of the conversion block is half the height of the residual block. The specific process of determination is the same as in step 1.2. A unit configured to derive the conversion block size of a conversion block according to the SVT type. The specific derivation process is the same as in step 1.2. A unit configured to determine the conversion block position of a conversion block. The specific process of determination may be the same as in step 2.1. Alternatively, if there are 2 candidate positions for the SVT type, a 1-bit flag may be used to indicate the conversion block position of the residual block's conversion block. For example, if only positions 0 and 2 in Figure 4 are used for SVT-V, the 1-bit flag is sufficient to indicate whether the conversion block position is position 0 or position 2. If only positions 0 and 2 in Figure 4 are used for SVT-H, the 1-bit flag is sufficient to indicate whether the conversion block position is position 0 or position 2. A unit configured to determine the conversion type of a conversion block according to the SVT type and the conversion block position of the conversion block, wherein the conversion type indicates horizontal and vertical conversion of the conversion block, and at least one of the horizontal and vertical conversions is DST-7. The specific process of determination may be the same as in step 2.2. A unit configured to parse the conversion coefficients of a conversion block according to the conversion block size. The specific process of parse can be similar to that in step 3. A unit configured to reconstruct residual blocks based on the transformation type, the location of the transformation block, and the transformation coefficient of the transformation block. The specific process of reconstruction may be similar to that in step 4. A video decoding device including the following is disclosed.
[0089] The specific conversion type may be any one of the conversion types from Tables I, IV, and V described above, or any one of the conversion types from Tables II, III, and VI described above that includes DST-7.
[0090] For example, if the SVT type for the residual block is SVT-V type, and the position of the transform block is such that it covers the upper left corner of the residual block (i.e., position 0 in Figure 4), then the horizontal transform is DCT-8 and the vertical transform is DST-7.
[0091] For example, if the SVT type for the residual block is SVT-V type, and the position of the transform block is such that it covers the lower right corner of the residual block (i.e., position 2 in Figure 4), then the horizontal transform is DST-7 and the vertical transform is DST-7.
[0092] For example, if the SVT type for the residual block is SVT-H type, and the position of the transform block is such that it covers the upper left corner of the residual block (i.e., position 0 in Figure 4), then the horizontal transform is DST-7 and the vertical transform is DCT-8.
[0093] For example, if the SVT type for the residual block is SVT-H type, and the position of the transformation block is such that it covers the lower right corner of the residual block (i.e., position 2 in Figure 4), then the horizontal transformation is DST-7 and the vertical transformation is DST-7.
[0094] This disclosure also discloses another video decoding device configured to implement the technology of the present application, comprising one or more processors and a non-volatile computer-readable storage medium coupled to the processors and storing a program for execution by the processors, wherein the video decoding device is configured to process any of the above methods when the program is executed by the processors.
[0095] This disclosure discloses a non-volatile computer-readable storage medium that stores computer instructions causing one or more processors to perform any of the steps of the above method when executed by one or more processors.
[0096] Figure 6 is a schematic diagram of a coding device 900 according to an embodiment of the present disclosure. The coding device 900 is suitable for carrying out embodiments of the disclosure described herein. The coding device 900 has an inlet port 910 and a receiver unit (Rx) 920 for receiving data, a processor, logic unit, or central processing unit (CPU) 930 for processing the data, a transmitter unit (Tx) 940 and an exit port 950 for transmitting data, and a memory 960 for storing data. The coding device 900 may also have optical-electrical (OE) components and electrical-optical (EO) components coupled to the inlet port 910, receiver unit 920, transmitter unit 940, and exit port 950 for the input and output of optical or electrical signals.
[0097] The processor 930 is implemented by hardware and software. The processor 930 may be implemented as one or more CPU chips, cores (e.g., as a multi-core processor), field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), and digital signal processors (DSPs). The processor 930 communicates with an inlet port 910, a receiver unit 920, a transmitter unit 940, an exit port 950, and memory 960. The processor 930 has a coding module 970. The coding module 970 implements the disclosed embodiments described above. For example, the coding module 970 performs, processes, parses, prepares, or provides various graphics processing and calculations. Thus, the inclusion of the coding module 970 results in a substantial improvement to the functionality of device 900 and results in the transformation of device 900 to different states. Alternatively, the coding module 970 is implemented as instructions stored in memory 960 and executed by the processor 930.
[0098] Memory 960 may have one or more disks, tape drives, and solid-state drives and may be used as an overflow data storage device to store such programs when they are selected for execution, and to store instructions and data read during program execution. Memory 960 may be volatile and / or non-volatile and may be read-only memory (ROM), random-access memory (RAM), tri-level associative memory (TCAM), and / or static random-access memory (SRAM).
[0099] The following references are incorporated into this application by reference as if their entire texts were being reproduced.
[0100] While several embodiments have been provided in this disclosure, it should be understood that the disclosed systems and methods can be embodied in many other specific forms without departing from the spirit or scope of this disclosure. These examples should be interpreted as examples, not as limitations, and are not intended to limit the details given herein. For example, various elements or components may be combined or integrated in other systems, or certain features may be omitted or not implemented at all.
[0101] Furthermore, the technologies, systems, subsystems, and methods described and illustrated individually or separately in various embodiments may be combined or integrated with other systems, modules, technologies, or methods without departing from the scope of this disclosure. Other items illustrated or discussed as being combined with, directly combined with, or communicating with one another may be indirectly combined or communicated through any interface, device, or intermediate component, whether electrically or mechanically or otherwise. Other examples of changes, substitutions, and modifications can be seen by those skilled in the art and may be made without departing from the spirit and scope of what is disclosed herein.
[0102] This application claims priority to U.S. Provisional Patent Application No. 62 / 678738, filed on 31 May 2018. The preceding U.S. application is incorporated herein by reference in its entirety.
Claims
1. A video decoding method, Receiving a bitstream containing a first flag, a second flag, syntax elements, and prediction information, According to the first flag, it is determined that a spatially varying transformation (SVT) is used for the residual block of the image block, The SVT type for the residual block is determined according to the second flag, wherein the SVT type is either an SVT-vertical (SVT-V) type or an SVT-horizontal (SVT-H) type, where the SVT-V type indicates that the first width of the conversion block of the residual block is narrower than the second width of the residual block, and the first height of the conversion block is the same as the second height of the residual block, and the SVT-H type indicates that the first width is the same as the second width, and the first height is lower than the second height. The transformation block position of the transformation block is determined according to the syntax elements, The conversion type of the conversion block is determined from the lookup table according to the SVT type and the conversion block position, Parsing the bitstream to obtain the conversion coefficients of the conversion block, Reconstructing the residual block based on the conversion type and the conversion coefficient, Based on the aforementioned prediction information, the prediction block corresponding to the residual block is obtained, The image block is acquired based on the residual block and the prediction block. A video decoding method having the following characteristics.
2. The transformation type indicates a horizontal transformation for the transformation block and a vertical transformation for the transformation block, the SVT type is the SVT-V type, the transformation block position covers the upper left corner of the residual block, the horizontal transformation is based on the discrete cosine transform (DCT)-8, and the vertical transformation is based on the discrete sine transform (DST)-7. The video decoding method according to claim 1.
3. The conversion type indicates a horizontal conversion for the conversion block and a vertical conversion for the conversion block, the SVT type is the SVT-V type, the conversion block position covers the lower right corner of the residual block, the horizontal conversion is based on the discrete sine transform (DST)-7, and the vertical conversion is based on the DST-7. The video decoding method according to claim 1.
4. The transformation type indicates a horizontal transformation for the transformation block and a vertical transformation for the transformation block, the SVT type is the SVT-H type, the transformation block position covers the upper left corner of the residual block, the horizontal transformation is based on the discrete sine transform (DST)-7, and the vertical transformation is based on the discrete cosine transform (DCT)-8. The video decoding method according to claim 1.
5. The conversion type indicates a horizontal conversion for the conversion block and a vertical conversion for the conversion block, the SVT type is the SVT-H type, the conversion block position covers the lower right corner of the residual block, the horizontal conversion is based on the discrete sine transform (DST)-7, and the vertical conversion is based on the DST-7. The video decoding method according to claim 1.
6. A video encoding method, Obtaining predicted blocks based on prediction information, The residual block is obtained based on the prediction block and the image block corresponding to the prediction block, Obtaining a conversion block corresponding to the aforementioned residual block, This involves setting a first value to a first flag, the first flag indicating that a spatially variable transformation (SVT) is used for the residual block, The second flag is set to a second value, the second flag indicating the SVT type for the residual block, the SVT type being either an SVT-vertical (SVT-V) type or an SVT-horizontal (SVT-H) type, the SVT-V type indicating that the first width of the conversion block of the residual block is narrower than the second width of the residual block, and the first height of the conversion block is the same as the second height of the residual block, the SVT-H type indicating that the first width is the same as the second width, and the first height is lower than the second height, The process involves obtaining a syntax element indicating the conversion block position of the conversion block, wherein the conversion type of the conversion block corresponds to the conversion block position and the second value. Obtaining the conversion coefficient of the aforementioned conversion block, The first value, the second value, the syntax element, the prediction information, and the conversion coefficient are included in the bitstream, The bitstream to be stored or transmitted A video encoding method having
7. The transformation type of the transformation block indicates a horizontal transformation and a vertical transformation for the transformation block, the SVT type is the SVT-V type, the transformation block position covers the upper left corner of the residual block, the horizontal transformation is based on the discrete cosine transform (DCT)-8, and the vertical transformation is based on the discrete sine transform (DST)-7. The video encoding method according to claim 6.
8. The transformation type of the transformation block indicates a horizontal transformation for the transformation block and a vertical transformation for the transformation block, the SVT type is the SVT-V type, the transformation block position covers the lower right corner of the residual block, the horizontal transformation is based on the discrete sine transform (DST)-7, and the vertical transformation is based on the DST-7. The video encoding method according to claim 6.
9. The transformation type of the transformation block indicates a horizontal transformation and a vertical transformation for the transformation block, the SVT type is the SVT-H type, the transformation block position covers the upper left corner of the residual block, the horizontal transformation is based on the discrete sine transform (DST)-7, and the vertical transformation is based on the discrete cosine transform (DCT)-8. The video encoding method according to claim 6.
10. The transformation type of the transformation block indicates a horizontal transformation for the transformation block and a vertical transformation for the transformation block, the SVT type is the SVT-H type, the transformation block position covers the lower right corner of the residual block, the horizontal transformation is based on discrete sine transform (DST)-7, and the vertical transformation is based on DST-7. The video encoding method according to claim 6.
11. A device for transmitting a bitstream, A receiver configured to receive an encoded bitstream of a video signal, wherein the encoded bitstream is The first value of the first flag indicates that a spatially varying transformation (SVT) is used for the residual block corresponding to the image data, The SVT type for the residual block is indicated, wherein the SVT type is either an SVT-vertical (SVT-V) type or an SVT-horizontal (SVT-H) type, the SVT-V type indicates that the first width of the conversion block of the residual block is narrower than the second width of the residual block, and the first height of the conversion block is the same as the second height of the residual block, the SVT-H type indicates that the first width is the same as the second width, and the first height is lower than the second height, and the second value of the second flag is indicated. The conversion block position of the conversion block is indicated, and the conversion type of the conversion block is a syntax element corresponding to the conversion block position and the second value, The conversion coefficient of the aforementioned conversion block and The receiver includes, A storage unit coupled to the receiver and configured to store the encoded bitstream, A transmitter coupled to the storage unit and configured to transmit the encoded bitstream, A device having.
12. The transformation type of the transformation block indicates a horizontal and vertical transformation for the transformation block, the SVT type is the SVT-V type, the transformation block position covers the upper left corner of the residual block, the horizontal transformation is based on the discrete cosine transform (DCT)-8, and the vertical transformation is based on the discrete sine transform (DST)-7. The apparatus according to claim 11.
13. The transformation type of the transformation block indicates horizontal and vertical transformations for the transformation block, the SVT type is the SVT-V type, the transformation block position covers the lower right corner of the residual block, the horizontal transformation is based on discrete sine transform (DST)-7, and the vertical transformation is based on DST-7. The apparatus according to claim 11.
14. The transformation type of the transformation block indicates a horizontal and vertical transformation for the transformation block, the SVT type is the SVT-H type, the transformation block position covers the upper left corner of the residual block, the horizontal transformation is based on the discrete sine transform (DST)-7, and the vertical transformation is based on the discrete cosine transform (DCT)-8. The apparatus according to claim 11.
15. The transformation type of the transformation block indicates horizontal and vertical transformations for the transformation block, the SVT type is the SVT-H type, the transformation block position covers the lower right corner of the residual block, the horizontal transformation is based on discrete sine transform (DST)-7, and the vertical transformation is based on DST-7. The apparatus according to claim 11.
16. A video decoding device, One or more processors, A non-temporary computer-readable storage medium coupled to the processor and storing the programming executed by the processor, It has, When the aforementioned programming is executed by the processor, it causes the video decoding device to execute the video decoding method described in any one of claims 1 to 5. Video decoding device.
17. A video encoding device, One or more processors, A non-temporary computer-readable storage medium coupled to the processor and storing the programming executed by the processor, It has, When the aforementioned programming is executed by the processor, it causes the video encoding device to execute the video encoding method described in any one of claims 6 to 10. Video encoding device.