Method and apparatus for dependent quantization for video coding

Abstract

A method and an apparatus for video coding and decoding using dependent quantization (DQ) with symbol hiding are provided. The method and the apparatus for video coding and decoding are provided. At the decoder end, at least one or both of a symbol hiding state for selected coefficients of the current segment and parity information for a plurality of quantized transform coefficients associated with the current segment is determined. The current segment includes a symbol hiding coefficient. And deciding symbols related to one or more symbol hiding quantization coefficients corresponding to one or more target coefficients in the current fragment according to at least one or both of the symbol hiding state and parity information of the current fragment. Using each of the plurality of quantizers, the plurality of quantized coefficients are de-quantized using the recovered symbols. A method and apparatus for a corresponding encoder are also disclosed.

Description

[0001] Cross-references

[0002] This application is a non-provisional application of and claims priority to U.S. Provisional Application No. 63 / 367,654, filed on July 5, 2022. The U.S. Provisional Application is incorporated herein by reference in its entirety. Technical Field

[0003] The present invention relates to a video coding system using a dependent quantization coding tool. In particular, the present invention relates to a scheme that allows dependent quantization with sign data hiding capability. Background Art

[0004] Versatile video coding (VVC) is the latest international video coding standard developed by the Joint Video Experts Team (JVET) of the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Moving Picture Experts Group (MPEG). The standard has been published as an ISO standard: ISO / IEC 23090-3:2021, Information technology - Codecs for the representation of immersive media - Part 3: Versatile video coding, which was published in February 2021. VVC builds on its predecessor High Efficiency Video Coding (HEVC) by adding more codec tools to improve codec efficiency and handle various types of video sources including three-dimensional (3D) video signals.

[0005] Figure 1AAn adaptive inter / intra video coding system including loop processing is shown. For intra prediction, prediction data is derived based on previously encoded and decoded video data in the current picture. For inter prediction 112, motion estimation (ME) is performed at the encoder end, and motion compensation (MC) is performed based on the results of ME to provide prediction data and motion data derived from other pictures. Switch 114 selects intra prediction 110 or inter prediction 112, and the selected prediction data is provided to adder 116 to form a prediction error, also known as a residual. It is then processed by transform (Transform, T) 118 and quantization (Quantization, Q) 120 in sequence. The transformed and quantized residuals are then encoded and decoded by an entropy encoder 122 to be included in a video bitstream corresponding to the compressed video data. The bitstream associated with the transform coefficients is then packaged together with auxiliary information such as motion mode and coding mode associated with intra prediction and inter prediction, and other information such as parameters associated with the loop filter applied to the underlying image area. As shown Figure 1A As shown, auxiliary information related to intra-picture prediction 110, inter-picture prediction 112 and in-loop filter (ILPF) 130 is provided to the entropy encoder 122. When the inter-picture prediction mode is used, one or more reference pictures must also be reconstructed at the encoder end. Therefore, the transformed and quantized residual is processed by inverse quantization (IQ) 124 and inverse transformation (IT) 126 to recover the residual. Then, at reconstruction (REC) 128, the residual is added back to the prediction data 136 to reconstruct the video data. The reconstructed video data can be stored in the reference picture buffer 134 and used for prediction of other frames.

[0006] like Figure 1AAs shown, the input video data undergoes a series of processes in the encoding system. The reconstructed video data from REC 128 may suffer various losses due to the series of processes. Therefore, in order to improve the video quality, a loop filter 130 is usually applied to the reconstructed video data before the reconstructed video data is stored in the reference picture buffer 134. For example, a deblocking filter (DF), a sample adaptive offset (SAO), and an adaptive loop filter (ALF) may be used. The loop filter information may need to be incorporated into the bitstream so that the decoder can properly recover the required information. Therefore, the loop filter information is also provided to the entropy encoder 122 for incorporation into the bitstream. Figure 1A In FIG. 1 , a loop filter 130 is applied to the reconstructed video before the reconstructed samples are stored in a reference picture buffer 134 . Figure 1A The system in is intended to illustrate an exemplary structure of a typical video encoder. It may correspond to a High Efficiency Video Coding (HEVC) system, VP8, VP9, ​​H.264, VVC, or any other video codec standard.

[0007] like Figure 1B As shown, since the decoder only needs inverse quantization 124 and inverse transform 126, the decoder can use similar or partially identical functional blocks as the encoder except for transform 118 and quantization 120. The decoder uses an entropy decoder 140 instead of an entropy encoder 122 to decode the video bitstream into quantized transform coefficients and required coding information (e.g., ILPF information, intra-picture prediction information, and inter-picture prediction information). The intra-picture prediction 150 on the decoder side does not need to perform a pattern search. Instead, the decoder only needs to generate an intra-picture prediction based on the intra-picture prediction information received from the entropy decoder 140. In addition, for inter-picture prediction, the decoder only needs to perform motion compensation 152 based on the inter-picture prediction information received from the entropy decoder 140, without the need for motion estimation.

[0008] According to VVC, similar to HEVC, the input picture is divided into a plurality of non-overlapping square block areas, called coding tree units (CTU). Each CTU can be divided into one or more coding units (CU) of smaller size. The resulting CU segmentation can be square or rectangular. At the same time, VVC divides the CTU into a plurality of prediction units (PU) as units to apply prediction processing, such as inter-picture prediction, intra-picture prediction, etc.

[0009] Compared with the HEVC standard, the VVC standard includes various new codec tools to further improve the codec efficiency. Among the various new codec tools, some codec tools related to the present invention are summarized as follows. In particular, VVC uses dependent quantization as a way to improve codec performance. Although dependent quantization can improve codec performance, due to the limitations of dependent quantization, the sign bit hiding tool is turned off. Therefore, the present invention discloses a scheme that allows dependent quantization to be combined with sign bit hiding to further improve the codec efficiency of dependent quantization. Summary of the invention

[0010] The present invention discloses a method and device for quantizing transform coefficients. According to the method, at the encoder end, a plurality of transform coefficients of a residual block are determined. The transform coefficients of the residual block are divided into one or more segments, each of the one or more segments having a preset number or range of transform coefficients. A plurality of states and a plurality of quantizers corresponding to the dependent quantization of the plurality of transform coefficients of the residual block are identified. For a current segment in the one or more segments, a plurality of quantization coefficients related to the plurality of transform coefficients in the current segment are determined, wherein at least one or both of the sign hiding state of the selected coefficient and the parity information of the current segment indicate one or more symbols related to one or more sign hiding quantization coefficients corresponding to one or more target coefficients in the current segment. The one or more sign hiding quantization coefficients are encoded by not signaling the one or more symbols related to the one or more sign hiding quantization coefficients. The proposed method and device can also be applied to encoding a transform skip block.

[0011] At a decoder, a plurality of quantized coefficients associated with transform coefficients of a residual block are received, wherein the plurality of quantized coefficients are divided into one or more segments, each segment of the one or more segments having a preset number or range of transform coefficients, and one or more symbols associated with one or more sign-hidden quantized coefficients corresponding to one or more target coefficients in a current segment are not signaled or parsed. A plurality of states and a plurality of quantizers corresponding to dependent quantization for quantizing transform coefficients of the residual block are identified. At least one or both of a sign-hiding state for selected coefficients of the current segment and parity information of the plurality of quantized coefficients of the current segment are determined. The one or more symbols associated with the one or more sign-hidden quantized coefficients corresponding to the one or more target coefficients in the current segment are determined based on at least one or both of the sign-hiding state and the parity information of the current segment. The plurality of quantized coefficients are dequantized using the one or more symbols recovered by each of the plurality of quantizers. The proposed method and apparatus can also be applied to encoding transform skip blocks.

[0012] In one embodiment, when the plurality of states corresponds to 4 states and the one or more target coefficients corresponds to a target coefficient, two of the 4 states represent positive signs, and the remaining two of the 4 states represent negative signs. In another embodiment, when the plurality of states corresponds to 8 states and the one or more target coefficients corresponds to a target coefficient, 4 of the 8 states represent positive signs, and the remaining 4 of the 8 states represent negative signs.

[0013] In one embodiment, the sign hiding state of the selected coefficient corresponds to the state of the first coefficient or the last coefficient of the current segment, or corresponds to the state of the first non-zero coefficient or the last non-zero coefficient of the current segment.

[0014] In one embodiment, the preset number or range of transform coefficients for each of the one or more fragments corresponds to N coefficients, one coefficient group, two coefficient groups, four coefficient groups, one transform unit or one transform block, where N corresponds to 16, 32, 48 or 64.

[0015] In one embodiment, the one or more target coefficients in the current segment correspond to the first non-zero coefficient, the Mth non-zero coefficient, or the last non-zero coefficient in the current segment.

[0016] In one embodiment, when the one or more segments correspond to at least two segments, after the plurality of quantization coefficients are determined for the first segment, the dependent quantization states are reset to initial states or remain unchanged through the rest of the target coefficients in the residual block.

[0017] In one embodiment, the parity information of the current segment corresponds to a sum of quantization coefficient levels or a sum of absolute quantization coefficient levels. In another embodiment, the parity information of the current segment corresponds to a sum of a plurality of states associated with the plurality of quantization coefficients.

[0018] In one embodiment, one or more target coefficients of the current segment correspond to two target coefficients. In one embodiment, two symbols of the two target coefficients are determined based on at least one or both of the symbol hiding state and the parity information of the current segment. In one embodiment, one of the two symbols is determined based on the symbol hiding state and the other of the two symbols is determined based on the parity information of the current segment.

[0019] In one embodiment, the one or more sign-hidden quantized coefficients are allowed when one or more conditions are met. In another embodiment, the one or more conditions include that the number of non-zero coefficients in the current segment or the residual block is greater than one or more thresholds. In another embodiment, the one or more conditions include that the distance between the first non-zero coefficient and the last non-zero coefficient in the current segment or the residual block is greater than one or more thresholds. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1A An adaptive inter / intra video coding system incorporating loop processing is shown.

[0021] Figure 1B Shows Figure 1A The corresponding decoder of the encoder.

[0022] Figure 2 Trellis-Coded-Quantization (TCQ) is shown, in which there are two scalar quantizers, denoted by Q0 and Q1, and the position of the available reconstruction levels is uniquely specified by the quantization step size Δ.

[0023] Figure 3 A finite state machine corresponding to a trellis structure with 4 states used in dependent scalar quantization is shown, wherein 4 states (ie, state 0, state 1, state 2, and state 3 in circles) and transitions between the states are shown.

[0024] Figure 4 Shows Figure 3 The corresponding Trellis-Coded-Quantization (TCQ) structure has 4 states.

[0025] Figure 5 Context modeling and binarization based on local neighborhood are shown, where the small black square represents the current scan position and the gray square represents the local neighborhood used.

[0026] Figure 6 A Trellis-Coded-Quantization (TCQ) structure with 8 states is shown.

[0027] Figure 7 A trellis traversing according to an embodiment of the present invention is shown, where sign bit hiding is implemented in last-state-based dependent quantization, and the target sign-hidden coefficient is the first coefficient in the forward scan order.

[0028] Figure 8A trellis traversal according to an embodiment of the present invention is shown, where sign bit hiding is implemented in last-state-based dependent quantization, and the target sign-hidden coefficient is the last non-zero coefficient in the forward scan order.

[0029] Fig. 9 A flow chart of an exemplary video decoding using combined dependent quantization and sign bit hiding according to an embodiment of the present invention is shown.

[0030] Fig.10 A flowchart of an exemplary video encoding using combined dependent quantization and sign bit hiding according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0031] It is readily understood that, as generally described and illustrated in the drawings herein, the components of the present invention may be arranged and designed in a variety of different configurations. As shown, the following detailed description of the embodiments of the present system and method is not intended to limit the scope of the present invention as required, but only represents the embodiments selected by the present invention. In the specification, reference to "one embodiment", "embodiment" or similar language means that a specific feature, structure or characteristic described in conjunction with the embodiment may be included in at least one embodiment of the present invention. Therefore, the phrases "in one embodiment" or "in an embodiment" appearing in various places in the specification do not necessarily all refer to the same embodiment.

[0032] In addition, the described features, structures or characteristics may be combined in any suitable manner in one or more embodiments. However, those skilled in the art will recognize that the present invention may be implemented without one or more specific details, or using other methods, components, etc. In other cases, in order to avoid confusing the various aspects of the present invention, known structures or operations are not shown or described in detail. The illustrated embodiments of the present invention will be best understood by reference to the drawings, in which the same parts are represented by the same numerals throughout. The following description is only by way of example, and simply shows certain selected embodiments of the apparatus and method consistent with the claimed invention.

[0033] Quantization in VVC

[0034] In a typical video codec system, such as Figure 1A and Figure 1B As shown, the prediction residuals obtained from inter-picture prediction and intra-picture prediction are transformed and decoded by transforming, quantizing and entropy coding the residuals. The input video data is usually represented by 8 / 10 / 12-bit data. However, the transform coefficients of the residual data usually use higher data precision, such as 32-bit or 64-bit data. Quantization is a lossy process (i.e., introduces distortion) that maps high-precision transform data to the input coefficients t i The smaller number of quantization levels (i.e., qi ). VVC supports basic quantization (i.e., uniform reconstruction quantization), in which the set of acceptable reconstruction values ​​is determined by a single parameter (i.e., Δ i ) specifies. The reconstructed level can be simply derived as t' i =Δ i q i . Similar to previous video codec standards (such as HEVC), VVC also uses a quantization weighting matrix through which the quantization step size can be varied in the transform coefficients of a block in order to take into account the human visual sensitivity response to spatial frequencies. Therefore, the quantization step size Δ i Depends on the coefficient position i within the block, i.e. Δ i =a i Δ., where a i is a weighting factor that depends on the coefficient t in the transform block i , Δ is the quantization step size. The quantization step size Δ is used to control the bit rate / image quality. In VVC, the quantization parameter QP is used to derive the quantization step size.

[0035] Sign Bit Hiding (SBH)

[0036] Like HEVC, VVC also supports sign bit hiding as a way to improve quantization performance. During the quantization process, the sign bit associated with the non-zero transform coefficient is encoded and decoded separately from the amplitude of the non-zero transform coefficient. The basic idea of ​​SDH is to skip the encoding and decoding of the sign of a non-zero coefficient in the non-zero transform coefficient. On the contrary, the sign of a non-zero coefficient in the non-zero transform coefficient is derived from the parity of the sum of the absolute values ​​of the non-zero coefficients. In order to save a sign bit, the encoder needs to adjust the value of the non-zero coefficient to meet the parity condition, which will bring a small additional distortion. Although the term "the value of the non-zero coefficient" is used here, it is understood that "the value of the non-zero coefficient" refers to the "value of the quantized non-zero coefficient" because the quantized non-zero coefficient can be determined at the encoder and decoder ends. Therefore, both the encoder and the decoder can obtain the same parity information (i.e., the sum of the absolute values ​​of the quantized non-zero coefficients) for sign bit hiding. In the following disclosure, the transform coefficient may refer to the quantized transform coefficient in any appropriate place. SDH is usually used for each coefficient group (CG) in HEVC and VVC.

[0037] Trellis-Coded-Quantization (TCQ) / Dependent Quantization (DQ)

[0038] Trellis-Coded-Quantization (TCQ), also known as Dependent Quantization (DQ), is a combination of the trellis structure and the idea of ​​set partitioning. By finding the path with the minimum cost along the trellis structure, the codec output of the sample group with the minimum cost measured by MSE and the number of bits used for signaling can be found.

[0039] A method for implementing dependent scalar quantization using TCQ has been disclosed, wherein a set of acceptable reconstruction values ​​for transform coefficients depends on the values ​​of transform coefficient levels preceding the current transform coefficient level in reconstruction order. Figure 2 A TCQ is shown in which there are two scalar quantizers, denoted Q0 and Q1, which are selected based on a quantization state defined by the value of the transform coefficient level preceding the current transform coefficient level and the previous quantization state. The positions of the available reconstruction levels are uniquely specified by the quantization step size.

[0040] The characteristics of the two scalar quantizers Q0 and Q1 are as follows:

[0041] Q0: reconstruction stage of the first quantizer Q0 ( Figure 2 denoted by black or gray dots in the figure) is given by an even integer multiple of the quantization step size Δ (i.e. Figure 2 When using this quantizer, the reconstructed transform coefficient t' is calculated by the following formula:

[0042] t'=2·k·Δ, k={…,-4,-3,-2,-1,0,1,2,3,4,…}

[0043] Wherein k represents the relevant transform coefficient level (ie, the transmitted quantization index).

[0044] Q1: The reconstruction stage of the second quantizer Q1 (in Figure 2 denoted by thin or thick circles in the figure) is given by an odd integer multiple of the quantization step size Δ. In addition, the quantizer Q1 also includes a reconstruction level equal to zero. The mapping relationship between the transform coefficient level k and the reconstructed transform coefficient t' is specified as follows:

[0045] t'=(2·k-sgn(k))·Δ,k={…,-5,-4,-3,-2,-1,0,1,2,3,4,5,…}

[0046] Where sgn(·) represents the sign function:

[0047] sgn(x)=(x==0?0:(x<0?-1:1)).

[0048] exist Figure 2In , the output quantization coefficient levels Q0 and Q1 can have even parity and odd parity. Figure 2 In , for Q0, the output coefficient level with even parity (i.e.) is labeled "A", and the output coefficient level with odd parity (i.e.) is labeled "B". Figure 2 In the example above, for Q1, the output coefficient level with even parity (i.e.) is labeled "C" and the output coefficient level with odd parity (i.e.) is labeled "D". The scalar quantizer used (Q0 or Q1) is not explicitly signaled in the bitstream. It is determined by the parity of the transform coefficient level that precedes the current transform coefficient in the encoding / reconstruction order. Figure 3 As shown, a finite state machine with four states determines the quantizer switching.

[0049] The finite state machine corresponding to the lattice structure used in dependent scalar quantization is Figure 3 Shown in. Figure 3 Four states (i.e., state 0, state 1, state 2, and state 3 in the circle) and transitions between the states are shown. The upper two states, i.e., state 0 and state 1, are associated with quantizer Q0, and the lower two states, i.e., state 2 and state 3, are associated with quantizer Q1. The operation initially starts from state 0, and Q0 is used to quantize the current coefficient value. If the obtained quantization index has an even parity (i.e.), the next state is still state 0, as shown in transition 310, and quantizer Q0 is used for the next coefficient. If the obtained quantization index has an odd parity (i.e.), the next state is state 2, as shown in transition 312, and quantizer Q1 is used for the next coefficient. For state 2, if the obtained quantization index (Q1 used) has an even parity (i.e.), the next state is state 1, as shown in transition 320, and quantizer Q0 is used for the next coefficient. For state 2, if the obtained quantization index (Q1 used) has an odd parity (i.e.), the next state is state 3, as shown in transition 322, and quantizer Q1 is used for the next coefficient. As shown in Figure 3 As shown, the state transitions of state 1 and state 3 can be determined similarly.

[0050] Table 1. Transition table for 4 states

[0051]

[0052] Table 1 is Figure 3 The transition table corresponding to the state transition diagram shows that the next state of the current state depends on the parity of the current quantization index (i.e. k&1).

[0053] The quantization process can be represented by a grid structure that defines the states and the quantization process of the transform coefficients. Figure 4As shown, the encoder will use the Viterbi algorithm (also known as dynamic programming) to traverse the trellis structure to determine the best path for the coefficient group. Figure 3 As shown, a grid structure can be generated according to the state transition diagram. For example, in the i-th stage, if the quantization index satisfies (k&1)==0, then state 0 will remain at state 0. The condition (k&1)==0 corresponds to the quantization index being an even number (i.e. Figure 2 symbol "A" in ). Therefore, state 0 at stage i will go to state 0 at stage i+1 via path 410 (i.e., parity 0 (A)). Similarly, state 0 at stage i will go to state 2 at stage i+1 via path 412 (i.e., parity 1 (B)). Using the same technique, we can determine Figure 4 The rest of the grid structure in . The grid structure has a great advantage in reducing the complexity of finding the minimum cost path to quantize the coefficient group. Since two quantizers can be used to quantize each coefficient, there will be 2 n possible combinations of quantizers are selected for n coefficients. When n is large, from 2 n Selecting the best quantizer combination from the possible combinations would be a difficult task. However, as described below, the trellis structure can reduce the complexity to be linear in n.

[0054] During the quantization process, at each stage, the path with the smaller cost for each state will be retained by the encoder. When a new coefficient is quantized, the smaller cumulative cost of each state is updated. Let ADx(i) be the minimum cost of state Sx at the i-th stage, where x = 0, 1, 2 or 3. For example, S0 at the i+1th stage can be reached from S0 at the i-th stage through the parity 0 (A) path, or from S1 at the i-th stage through the parity 1 (B) path. Comparing the cumulative costs of S0 and S1 at the i-th stage, the smaller one is retained for S0 at the i+1th stage. The same process applies to other states at the i+1th stage. Therefore, each state only retains one best cumulative cost and related path. This process continues until all coefficients in the transform block are processed. Therefore, when performing backward traversal, the path can be uniquely determined. Finding the level of the sample set with the minimum cost is equivalent to finding the end of the path with the minimum cost.

[0055] In VVC, TCQ, also known as dependent quantization (DQ), is one of the codec tools for quantization and residual. In VVC, when TCQ is selected, sign bit hiding is not used because the parity of the quantization index is used to generate the state transition of the grid structure.

[0056] Syntactic signaling and context modeling for DQ

[0057] Four passes of syntax signaling for the coefficients in each CG are disclosed.

[0058] Round 1: Transmit the following flags for each scan location (using normal mode):

[0059] sig_coeff_flag, when sig_coeff_flag is equal to 1, par_level_flag and rem_abs_gt1_flag;

[0060] Round 2: For all scan positions where rem_abs_gt1_flag is equal to 1, rem_abs_gt2_flag is encoded and decoded using the normal mode of the arithmetic codec engine;

[0061] Round 3: For all scan positions where rem_abs_gt2_flag is equal to 1, encode and decode the non-binary syntax element abs_remainder in the bypass mode of the arithmetic codec engine;

[0062] Round 4: For all scan positions where sig_coeff_flag is equal to 1, encode and decode sign_flag in the bypass mode of the arithmetic codec engine.

[0063] like Figure 5 As shown, context modeling and binarization depend on the following measures on the local neighborhood, where the small black square 510 represents the current scanning position and the gray square represents the local neighborhood used.

[0064] numSig: The number of non-zero orders in the local neighborhood.

[0065] sumAbs1: the sum of the partially reconstructed absolute levels (absLevel1) after the first round in the local neighborhood;

[0066] sumAbs: The sum of reconstructed absolute levels in the local neighborhood.

[0067] d=x+y, where x and y are the positions of the current transform block (TB) or transform unit (TU) on the x-axis and the y-axis respectively.

[0068] The context model of sig_flag depends on the current state and can be derived as follows:

[0069] Luminance component,

[0070] ctxIdSig=18*max(0,state-1)+min(sumAbs1,5)+(d<2?12:(d<5?6:0)).

[0071] Chroma component,

[0072] ctxIdSig=12*max(0,state-1)+min(sumAbs1,5)+(d<2?6:0)).

[0073] The context model of par_level_flag is described as follows:

[0074] If the current scan position is equal to the position of the last non-zero level (indicated by the transmitted x and y coordinates), ctxIdPar is set to 0;

[0075] Otherwise, if the current color component is a luma component, the context index is set to:

[0076] ctxIdPar=1+min(sumAbs1–numSig,4)+(d==0?15:(d<3?10:(d<10?5:0))).

[0077] Otherwise (the current color component is a chrominance component), the context index is set to:

[0078] ctxIdPar=1+min(sumAbs1–numSig,4)+(d==0?5:0).

[0079] The context of rem_abs_gtx_flag is described as follows:

[0080] ctxIdGt1=ctxIdPar

[0081] ctxIdGt2=ctxIdPar

[0082] The non-binary syntax element abs_remainder is binarized using the same Rice codec category as in HEVC. The Rice parameter ricePar is determined as follows:

[0083] If sumAbs-numSig is less than 12, ricePar is set to 0;

[0084] Otherwise, if sumAbs-numSig is less than 25, ricePar is set to 1;

[0085] Otherwise, RicePar is set to 2.

[0086] DQ with 8 states

[0087] In ECM-4.0 (Muhammed Coban, et al., “Algorithm description of Enhanced Compression Model 4 (ECM 4)”, Joint Video Experts Team (JVET) of ITU-T SG 16WP3 and ISO / IEC JTC 1 / SC 29, 23rd Meeting, by teleconference, 7–16 July 2021, Document: JVET-Y2025), 8-state DQ is supported. Figure 6 The state transition of 8-state DQ is shown. The state transition table is shown in Table 2. The number of states is increased to 8 states. Even states use Q0 and odd states use Q1.

[0088] Table 2.8 state transition table

[0089] Q Status Parity = 0 Parity = 1 0 0 2 1 5 7 2 1 3 3 6 4 4 2 0 5 4 6 6 3 1 7 7 5

[0090] TCQ has been adopted in VVC, and the next generation of international video coding standards is also considering adopting TCQ. Although the performance of TCQ is better than that of conventional uniform quantizers, TCQ cannot be used with sign bit hiding because the parity of the quantization index is used for grid path decision. The present invention proposes a TCQ that can use sign bit hiding to improve the performance of traditional TCQ.

[0091] DQ with sign bit hiding

[0092] In the present invention, it is proposed to apply sign hiding to DQ. According to the sign hiding state of the selected coefficient (e.g., DC coefficient) or the parity information of the quantized coefficient, the sign bit of the target sign hidden coefficient (e.g., the last non-zero coefficient or DC coefficient) can be restored or hidden. When sign hiding is used with DQ, some states (e.g., half of the states, or more than half of the states, or less than half of the states) are selected to represent the positive sign of the target coefficient, and some states (e.g., other states) are selected to represent the negative sign of the target coefficient. For example, in a 4-state DQ, state 0 / 1 (i.e., state 0 or state 1) can represent a positive sign, and state 2 / 3 (i.e., state 2 or state 3) can represent a negative sign. For another example, state 0 / 1 can represent a negative sign, and state 2 / 3 can represent a positive sign. For another example, state 0 / 2 can represent a positive sign, and state 1 / 3 can represent a negative sign. For another example, state 0 / 2 can represent a negative sign, and state 1 / 3 can represent a positive sign. For another example, state 0 / 3 can represent a positive sign, and state 1 / 2 can represent a negative sign. For another example, state 0 / 3 may represent a negative sign, and state 1 / 2 may represent a positive sign.

[0093] In another embodiment, in an 8-state DQ, states 0 / 7 / 1 / 4 may represent a positive (or negative) symbol. In another example, states 2 / 5 / 3 / 6 may represent a positive (or negative) symbol. In another example, states 0 / 2 / 4 / 6 may represent a positive (or negative) symbol. In another example, states 0 / 2 / 5 / 7 may represent a positive (or negative) symbol. In another example, states 0 / 1 / 2 / 3 may represent a positive (or negative) symbol.

[0094] In DQ, each state can be set to represent a positive coefficient or a negative coefficient. In the encoder, after traversing a preset number or range of coefficients, a path with the minimum cost and a symbol in the final state equal to the symbol value of the target coefficient is selected. However, although the symbol in the final state is mentioned as an example for symbol hiding, the present invention can use any state of the selected path (referred to as a symbol hiding state) for sign bit hiding. For example, the symbol hiding state can be the DQ state of the DC coefficient of the TB, the first or last coefficient in the fragment or TB, or the first or last non-zero coefficient in the fragment or TB. The preset number or range of coefficients can be N coefficients (e.g., N=16, 32, 48, or 64), 1 CG (or 1 aligned CG), 2 CGs (or 2 aligned CGs), 4 CGs (or 4 aligned CGs), or 1 TU / TB. The target symbol can be the first valid coefficient (non-zero coefficient) in the range, or the Mth valid coefficient, or the last valid coefficient. For example, it may be the non-zero coefficient with the smallest scan index of the TB (eg, the DC coefficient), or the non-zero coefficient with the smallest scan index of the fragment, or the last non-zero coefficient of the TB, or the non-zero coefficient with the largest scan index of the fragment.

[0095] After selecting the path with correct sign information and lower cost, DQ can be reset to the initial state, or continue to traverse the remaining coefficients, or continue to traverse the coefficients whose signs are hidden. Figure 7 As shown. In this example, the target coefficient (i.e., the sign of the target coefficient to be hidden according to an embodiment of the present invention) is the last quantized transform coefficient (in the backward scanning order, i.e., the DC coefficient), i.e., 3. In this example, the sign hiding state is the DQ state of the DC coefficient, state 1 / 3 represents a negative sign, and state 0 / 2 represents a positive sign. It should be noted that with sign hiding, only path ends in states with correct signs can be selected. Therefore, in this example, although the cost of the path end in state 3 is less than the cost of the path end in state 2, the path end in state 2 is selected instead of the path end in state 3 (because state 3 represents a negative sign). As mentioned above, although the last non-zero coefficient is taken as an example, the target coefficient can be any non-zero coefficient within a default number or range of coefficients. For example, in Figure 8 In this example, the target coefficient is the last non-zero quantized transform coefficient (in the forward scan order, that is, the coefficient at scan position 5), that is, 1. In this example, the sign hiding state is the DQ state of the DC coefficient, and state 1 / 3 indicates a negative sign, and state 0 / 2 indicates a positive sign. Therefore, in this example, although the cost of the path end in state 3 is less than the cost of the path end in state 2, the path end in state 2 is selected instead of the path end in state 3 (because state 3 indicates a negative sign).

[0096] For using DQ to enable symbol hiding, some conditions can be defined. When one or more conditions or all conditions are met, one or more sign bits are hidden in a default number or range of coefficients. In one example, for each preset number or range of coefficients, the number of non-zero coefficients needs to be greater than a threshold. The threshold is an integer and can be equal to 1, 2, 3, 4, 5, 6 or 7, or 8, 16, 24 or 32. In another example, the distance between the first non-zero coefficient and the last non-zero coefficient, or the distance between the last non-zero coefficient and the first coefficient in the preset number or range of coefficients needs to be greater than the threshold. The threshold is an integer and can be equal to 1, 2, 3, 4, 5, 6 or 7, or 8, 16, 24 or 32.

[0097] In another embodiment, the sum of coefficients or the sum of absolute coefficients can be used as a hint for the hidden sign. For example, if the sum of absolute coefficients is an even number, the hidden sign is a positive sign or a negative sign. Otherwise (i.e., the sum of absolute coefficients is an odd number), the hidden sign is a negative sign or a positive sign.

[0098] In another embodiment, the sum of the DQ states of the coefficients may be used as a hint for the hidden sign. For example, if the sum of the DQ states is an even number, the hidden sign is a positive sign or a negative sign. Otherwise (i.e., the sum of the DQ states is an odd number), the hidden sign is a negative sign or a positive sign. In another embodiment, the sum of the number of DQ quantizers of the coefficients (e.g., the number of quantizers Q0 is 0 and the number of quantizers Q1 is 1) may be used as a hint for the hidden sign. For example, if the sum of the number of DQ quantizers is an even number, the hidden sign is a positive sign or a negative sign. Otherwise (i.e., the sum of the number of DQ quantizers is an odd number), the hidden sign is a negative sign or a positive sign. As previously mentioned, the coefficients here refer to quantized transform coefficients or quantization levels.

[0099] In another embodiment, multiple symbols may be hidden in a selected path. For example, two symbols may be hidden in each TB by using the DQ state, the sum of coefficients or the absolute sum of coefficients, the sum of the DQ states of the coefficients, or a combination of the above information. It should be noted that the sum of coefficients, the absolute sum of coefficients, and the sum of the DQ states of the coefficients are all relevant parity information associated with the selected path. For example, state 0 / 7 represents ++, 1 / 4 represents +-, 2 / 5 represents -+, and 3 / 6 represents -. This means that if the selected symbol hiding state (e.g., the state of the DC coefficient in the TB or the state of the coefficient with the smallest scan index in the segment) is 0 or 7, the signs of the last significant coefficient and the penultimate significant coefficient are both positive signs. For another example, state 0 / 2 represents ++, 4 / 6 represents +-, 5 / 7 represents -+, and 1 / 3 represents -. For another example, state 0 / 4 represents ++, 2 / 6 represents +-, 5 / 3 represents -+, and 1 / 7 represents -. For another example, state 0 / 6 represents ++, 2 / 4 represents +-, 5 / 1 represents -+, and 7 / 3 represents --.

[0100] In another embodiment, the sum of the absolute coefficients modulo 4 (ie, modulo 4 operation) equal to 0 represents ++, equal to 1 represents +-, equal to 2 represents -+, and equal to 3 represents --.

[0101] In another embodiment, the state and parity information can be used together to hide the sign information. For example, some states (such as 0 / 7 / 1 / 4) indicate the positive or negative sign of the last significant coefficient symbol, and the parity of the absolute coefficient sum indicates the sign of the penultimate significant coefficient symbol.

[0102] In another example, some states (eg, 0 / 7 / 1 / 4) represent the positive or negative sign of the second-to-last significant coefficient symbol, and the parity of the absolute coefficient sum represents the sign of the last significant coefficient symbol.

[0103] Different thresholds may be used to enable hiding of the second symbol bit. For example, a preset number or range, the number of non-zero coefficients within the range, and / or the distance between non-zero coefficients within the range may be different from the condition for hiding the first symbol. The threshold may be greater than, or equal to, or greater than or equal to, or less than, or less than or equal to the threshold of the first symbol.

[0104] In another embodiment, the sign hiding conditions or thresholds may be different for the chrominance components. These values ​​may be different from the thresholds for the luma component.

[0105] Any of the methods proposed above can be applied to transformed TBs or to skip transformed TBs.

[0106] Any of the above-mentioned methods of TCQ with sign bit hiding can be implemented in an encoder and / or a decoder. For example, it can be implemented in a quantization module (e.g. Figure 1A Q 120 in), dequantization module (e.g. Figure 1A ) and residual coding modules, and / or in the dequantization module of the decoder (e.g. Figure 1B Any method of implementing the proposed TCQ with sign bit hiding in the IQ 124 in the encoder and the residual coding module. Alternatively, any proposed method can be implemented as a circuit coupled to the quantization module, dequantization module and residual coding module of the encoder and / or the dequantization module and residual coding module of the decoder to provide the information required by the predictor derivation module. The module for implementing the TCQ with sign bit hiding related to the present invention can also correspond to executable software or firmware code stored on a medium such as a hard disk or flash memory, for a central processing unit (CPU) or a programmable device (such as a digital signal processor (DSP) or a field programmable gate array (FPGA)).

[0107] Fig. 9A flowchart of an exemplary video decoding using combined dependent quantization and sign bit hiding according to an embodiment of the present invention is shown. The steps shown in the flowchart can be implemented as program codes that can be executed on one or more processors (e.g., one or more CPUs) at the encoder end. The steps shown in the flowchart can also be implemented based on hardware, such as one or more electronic devices or processors to perform the steps in the flowchart. According to the method, in step 910, a plurality of quantized coefficients related to a plurality of transform coefficients of a residual block are received, wherein the plurality of quantized coefficients are divided into one or more segments, each of the one or more segments has a preset number or range of transform coefficients, and one or more signs related to one or more sign-hidden quantized coefficients corresponding to one or more target coefficients in the current segment are not signaled or resolved. In step 920, a plurality of states and a plurality of quantizers corresponding to dependent quantization for the residual block are identified. In step 930, at least one or both of the sign-hidden state of the selected coefficients of the current segment and the parity information of the plurality of quantized coefficients of the current segment are determined. In step 940, one or more signs associated with one or more sign-hidden quantized coefficients corresponding to one or more target coefficients in the current segment are determined based on at least one or both of the sign-hiding state and the parity information of the current segment. In step 950, the plurality of quantized coefficients are dequantized using the one or more signs recovered by each of the plurality of quantizers.

[0108] Fig.10 A flowchart of an exemplary video encoding using combined dependent quantization and sign bit hiding according to an embodiment of the present invention is shown. According to the method, at the encoder end, a plurality of transform coefficients of a residual block are determined in step 1010. In step 1020, the plurality of transform coefficients of the residual block are divided into one or more segments, each of the one or more segments having a preset number or range of transform coefficients. In step 1030, a plurality of states and a plurality of quantizers corresponding to the dependent quantization of the plurality of transform coefficients of the residual block are identified. In step 1040, for a current segment in the one or more segments, a plurality of quantization coefficients associated with the plurality of transform coefficients in the current segment are determined, wherein at least one or both of the sign hiding state of the selected coefficient and the parity information of the current segment indicate one or more signs associated with one or more sign-hidden quantization coefficients corresponding to one or more target coefficients in the current segment. In step 1050, the one or more sign-hidden quantized coefficients are encoded by not signaling one or more signs associated with the one or more sign-hidden quantized coefficients.

[0109] The flowchart shown in the present invention is used to illustrate an example of video encoding and decoding according to the present invention. Without departing from the spirit of the present invention, a person of ordinary skill in the art can modify each step, reorganize the steps, separate a step, or combine the steps to implement the present invention. In the present invention, specific syntax and semantics have been used to illustrate different examples to implement embodiments of the present invention. Without departing from the spirit of the present invention, by replacing the syntax and semantics with equivalent syntax and semantics, a person of ordinary skill in the art can implement the present invention.

[0110] The above description enables the ordinary general knowledge of the art to implement the present invention in the content of a specific application and its requirements. For the ordinary knowledge of the art, the various deformations of the described embodiments will be apparent, and the general principles defined herein can be applied in other embodiments. Therefore, the present invention is not limited to the specific embodiments shown and described, but will be given the maximum scope consistent with the principles and novel features disclosed herein. In the above detailed description, various specific details are described to thoroughly understand the present invention. Nevertheless, it will be understood by the ordinary knowledge of the art that the present invention can be put into practice.

[0111] The embodiments of the present invention as described above can be implemented in various hardware, software codes or a combination of the two. For example, an embodiment of the present invention can be a circuit integrated in a video compression chip, or a program code integrated into a video compression software to perform the processing described herein. An embodiment of the present invention can also be a program code executed on a digital signal processor (DSP) to perform the processing described herein. The present invention can also include several functions executed by a computer processor, a digital signal processor, a microprocessor or a field programmable gate array (FPGA). According to the present invention, these processors can be configured to perform specific tasks by executing machine-readable software code or firmware code that defines the specific method implemented by the present invention. The software code or firmware code can be developed in different programming languages ​​and different formats or styles. The software code can also be compiled for different target platforms. However, different code formats, styles and languages ​​of software codes and other forms of configuration codes for performing the tasks of the present invention will not deviate from the spirit and scope of the present invention.

[0112] The invention may be implemented in other specific forms without departing from its spirit or essential characteristics. The examples described are illustrative in all respects and are not restrictive. Therefore, the scope of the invention is indicated by the appended claims rather than by the foregoing description. The meaning of the claims and all variations within the same scope are intended to be included within their scope.

Claims

1. A method for dequantizing a quantized transform coefficient, for processing video data, the method comprising: Receiving a plurality of quantized coefficients associated with a plurality of transform coefficients of a residual block, wherein the plurality of quantized coefficients are divided into one or more segments, each of the one or more segments having a predetermined number or range of transform coefficients, wherein one or more symbols associated with one or more sign-hidden quantized coefficients corresponding to one or more target coefficients in a current segment are not signaled or resolved; Identifying a plurality of states and a plurality of quantizers corresponding to dependent quantization for the residual block; determining at least one or both of a sign hiding state of the selected coefficients of the current segment and parity information of the plurality of quantized coefficients of the current segment; determining, based on at least one or both of the sign hiding state and the parity information of the current segment, the one or more signs associated with the one or more sign-hidden quantized coefficients corresponding to the one or more target coefficients in the current segment; and The plurality of quantized coefficients are dequantized using the one or more symbols recovered by each of the plurality of quantizers.

2. The method according to claim 1, characterized in that: in, When the plurality of states corresponds to four states and the one or more target coefficients corresponds to one target coefficient, two of the four states represent positive signs, and the remaining two of the four states represent negative signs.

3. The method according to claim 1, characterized in that: in, When the plurality of states corresponds to 8 states and the one or more target coefficients corresponds to a target coefficient, 4 of the 8 states represent positive signs, and the remaining 4 of the 8 states represent negative signs.

4. The method according to claim 1, characterized in that: in, The sign hiding state of the selected coefficient corresponds to the state of the first coefficient or the last coefficient of the current segment, or corresponds to the state of the first non-zero coefficient or the last non-zero coefficient of the current segment.

5. The method according to claim 1, characterized in that: in, The preset number or range of transform coefficients for each of the one or more fragments corresponds to N coefficients, one coefficient group, two coefficient groups, four coefficient groups, one transform unit or one transform block, where N corresponds to 16, 32, 48 or 64.

6. The method according to claim 1, characterized in that: in, The one or more target coefficients in the current segment correspond to the first non-zero coefficient, the Mth non-zero coefficient or the last non-zero coefficient in the current segment.

7. The method according to claim 1, characterized in that: in, When the one or more segments correspond to at least two segments, after the plurality of quantization coefficients are determined for the first segment, the dependent quantization states are reset to initial states or remain unchanged through the remaining target coefficients in the residual block.

8. The method according to claim 1, wherein: in, The parity information of the current segment corresponds to the sum of quantization coefficient levels or the sum of absolute quantization coefficient levels.

9. The method according to claim 1, wherein: in, The parity information of the current segment corresponds to a sum of a plurality of states associated with the plurality of quantized coefficients.

10. The method according to claim 1, wherein: in, The one or more target coefficients in the current segment correspond to two target coefficients.

11. The method according to claim 10, characterized in that: in, The two signs of the two target coefficients are determined according to at least one or both of the sign hiding state and the parity information of the current segment.

12. The method according to claim 11, characterized in that: in, One of the two symbols is determined according to the symbol hiding state, and the other of the two symbols is determined according to the parity information of the current segment.

13. The method of claim 1, wherein: in, When one or more conditions are met, the one or more sign-hidden quantization coefficients are enabled.

14. The method according to claim 13, characterized in that: in, The one or more conditions include that the number of non-zero coefficients in the current segment or the residual block is greater than one or more thresholds.

15. The method of claim 13, wherein: in, The one or more conditions include that a distance between a first non-zero coefficient and a last non-zero coefficient in the current segment or the residual block is greater than one or more thresholds.

16. A method for quantizing transform coefficients, for processing video data, the method comprising: Determine the complex transform coefficients of the residual block; Dividing the plurality of transform coefficients of the residual block into one or more segments, each segment of the one or more segments having a preset number or range of transform coefficients; Identifying a plurality of states and a plurality of quantizers corresponding to dependent quantization of the plurality of transform coefficients of the residual block; For a current segment of the one or more segments, determining a plurality of quantized coefficients associated with the plurality of transform coefficients in the current segment, wherein at least one or both of a sign hiding state of the selected coefficient and parity information of the current segment indicate one or more signs associated with one or more sign-hidden quantized coefficients corresponding to one or more target coefficients in the current segment; and The one or more sign-hidden quantized coefficients are encoded by signaling the one or more signs associated with the one or more sign-hidden quantized coefficients.

17. A dequantization device for quantized transform coefficients, for processing video data, the device comprising one or more electronic devices or processors for: A plurality of quantized coefficients associated with a plurality of transform coefficients of a residual block are received, wherein: The plurality of quantized coefficients are divided into one or more segments, each segment of the one or more segments having a predetermined number or range of transform coefficients, wherein one or more symbols associated with one or more sign-hidden quantized coefficients corresponding to one or more target coefficients in a current segment are not signaled or resolved; Identifying a plurality of states and a plurality of quantizers corresponding to dependent quantization for the residual block; determining at least one or both of a sign hiding state of the selected coefficients of the current segment and parity information of the plurality of quantized coefficients of the current segment; determining, based on at least one or both of the sign hiding state and the parity information of the current segment, the one or more signs associated with the one or more sign-hidden quantized coefficients corresponding to the one or more target coefficients in the current segment; and The plurality of quantized coefficients are dequantized using the one or more symbols restored by each of the plurality of quantizers.

18. A transform coefficient quantization device for processing video data, the device comprising one or more electronic devices or processors for: Determine the complex transform coefficients of the residual block; Dividing the plurality of transform coefficients of the residual block into one or more segments, each segment of the one or more segments having a preset number or range of transform coefficients; Identifying a plurality of states and a plurality of quantizers corresponding to dependent quantization of the plurality of transform coefficients of the residual block; For a current segment of the one or more segments, a plurality of quantization coefficients associated with the plurality of transform coefficients in the current segment are determined, wherein: at least one or both of a sign hiding state of the selected coefficient and parity information of the current segment, indicating one or more signs associated with one or more sign-hidden quantized coefficients corresponding to one or more target coefficients in the current segment; and The one or more sign-hidden quantized coefficients are encoded by signaling the one or more signs associated with the one or more sign-hidden quantized coefficients.

Images