Video encoding method, device and video decoding method, device

Abstract

The present invention provides a video encoding method, apparatus, and video decoding method and apparatus. The video decoding method includes: receiving decoded transform coefficients of a residual block corresponding to a current block, including decoded symbol residuals associated with a set of jointly predicted coefficient symbols; determining an allowed maximum number; decoding the decoded symbol residuals into decoded symbol residuals, wherein the total number of the set of jointly predicted coefficient symbols is equal to or less than the allowed maximum number; selecting a hypothesis to determine a joint symbol prediction for the set of jointly predicted coefficient symbols; and reconstructing the set of jointly predicted coefficient symbols based on the decoded symbol residuals and the joint symbol prediction. The video encoding method, apparatus, and video decoding method and apparatus of the present invention can improve the performance of transform coefficient symbol decoding.

Description

Technical field

[0001] The present invention relates to a video coding system, and in particular to coding of signs of transform coefficients of a residual block in the video coding system. [Background Technology]

[0002] Versatile Video Coding (VVC) is the latest international video coding standard developed jointly by the ITU-T Video Coding Experts Group (VCEG) and the Joint Video Experts Team (JVET) of the ISO / IEC Moving Picture Experts Group (MPEG). The standard was published as ISO / IEC 23090-3:2021, Information technology — Coded representation of immersive media — Part 3: Versatile video coding, in February 2021. VVC builds on its predecessor, High Efficiency Video Coding (HEVC), by adding additional coding tools to improve coding efficiency and handle a variety of video sources, including 3D video signals.

[0003] Figure 1A An exemplary adaptive inter / intra video decoding system incorporating loop processing is described. For intra prediction, prediction data is derived based on previously encoded video data in the current picture. For inter prediction 112, motion estimation (ME) is performed on the encoder side and motion compensation (MC) is performed based on the results of ME to provide prediction data derived from other pictures and motion data. Switch 114 selects intra prediction 110 or inter prediction 112, and the selected prediction data is provided to adder 116 to form a prediction error, also called a residual. The prediction error is then processed by transform (T) 118 and subsequently quantization (Q) 120. The transformed and quantized residual is then encoded by entropy encoder 122 to include it in a video bitstream corresponding to the compressed video data. The bitstream associated with the transform coefficients is then packaged with auxiliary information such as motion and decoding modes associated with intra prediction and inter prediction, and other information such as parameters associated with the loop filter applied to the underlying image area. Figure 1AAs shown, side information associated with intra prediction 110, inter prediction 112, and loop filter 130 is provided to entropy encoder 122. When using inter prediction mode, the reference picture must also be reconstructed at the encoder end. Therefore, the transformed and quantized residual is processed by inverse quantization (IQ) 124 and inverse transform (IT) 126 to recover the residual. The residual is then added back to the predicted data 136 in reconstruction (REC) 128 to reconstruct the video data. The reconstructed video data can be stored in a reference picture buffer 134 and used for prediction of other frames.

[0004] like Figure 1A As shown, the input video data undergoes a series of processes in the encoding system. Due to the series of processes, the reconstructed video data from REC 128 may be subject to various impairments. Therefore, before the reconstructed video data is stored in the reference picture buffer 134, a loop filter 130 is often applied to the reconstructed video data to improve the video quality. For example, a deblocking filter (DF), sample adaptive offset (SAO), and adaptive loop filter (ALF) may be used. It may be necessary to incorporate the loop filter information into the bitstream so that the decoder can correctly recover the required information. Therefore, the loop filter information is also provided to the entropy encoder 122 to be incorporated into the bitstream. Figure 1A In FIG, a loop filter 130 is applied to the reconstructed video before the reconstructed samples are stored in a reference picture buffer 134. Figure 1A The system in is intended to illustrate an exemplary structure of a typical video encoder. It may correspond to the High Efficiency Video Coding (HEVC) system, VP8, VP9, ​​H.264, or VVC.

[0005] Figure 1B Another example decoding system is shown. Figure 1B As shown, the decoder can use similar or partially identical functional blocks as the encoder, except for transform 118 and quantization 120, because the decoder only needs inverse quantization 124 and inverse transform 126. Instead of the entropy encoder 122, the decoder uses an entropy decoder 140 to decode the video bitstream into quantized transform coefficients and required decoding information (e.g., ILPF information, intra-frame prediction information, and inter-frame prediction information). The intra-frame prediction 150 on the decoder side does not need to perform a pattern search. Instead, the decoder only needs to generate an intra-frame prediction based on the intra-frame prediction information received from the entropy decoder 140. In addition, for inter-frame prediction, the decoder only needs to perform motion compensation (MC 152) based on the intra-frame prediction information received from the entropy decoder 140 without motion estimation.

[0006] In VVC, a coded picture is partitioned into non-overlapping square block regions represented by associated coding tree units (CTUs). A coded picture can be represented by a set of slices, each slice comprising an integer number of CTUs. Each CTU in a slice is processed in raster scan order. Bidirectionally predicted (B) slices can be decoded using intra prediction or inter prediction with at most two motion vectors and reference indices to predict the sample values ​​of each block. Predicted (P) slices are decoded using intra prediction or inter prediction with at most one motion vector and reference index to predict the sample values ​​of each block. Intra (I) slices are decoded only using intra prediction.

[0007] A quadtree (QT) with a nested multi-type-tree (MTT) structure can be used to divide a CTU into one or more non-overlapping coding units (CUs) to accommodate various local motion and texture characteristics. Figure 2 One of the five split types shown (quad-tree partitioning 210, vertical binary tree partitioning 220, horizontal binary tree partitioning 230, vertical center-side triple-tree partitioning 240, horizontal center-side triple-tree partitioning 250) further splits the CU into smaller CUs. Figure 3An example of a CTU recursively partitioned by a QT with a nested MTT is provided. Each CU contains one or more prediction units (PUs). A prediction unit, together with the associated CU syntax, serves as the basic unit for transmitting prediction sub-information. A specified prediction process is used to predict the values ​​of the relevant pixel samples within the PU. Each CU may contain one or more transform units (TUs) for representing prediction residual blocks. A transform unit (TU) includes a transform block (TB) of luma samples and two corresponding transform blocks of chroma samples, with each TB corresponding to a block of residual samples from one color component. An integer transform is applied to the transform blocks. The level values ​​of the quantized coefficients are entropy decoded in the bitstream along with other auxiliary information. The terms coding tree block (CTB), coding block (CB), prediction block (PB), and transform block (TB) are defined as a two-dimensional array of samples specifying one color component associated with a CTU, CU, PU, ​​and TU, respectively. Therefore, a CTU consists of one luma CTB, two chroma CTBs, and related syntax elements. Similar relationships apply to CUs, PUs, and TUs.

[0008] In order to achieve high compression efficiency, the context-based adaptive binary arithmetic coding (CABAC) mode, also known as the normal mode, is used to perform entropy decoding on the values ​​of syntax elements in HEVC and VVC. Figure 4 An exemplary block diagram of the CABAC process is shown. Since the arithmetic decoder in the CABAC engine can only decode binary symbol values, the CABAC process requires the use of a binarizer (410) to convert the value of the syntactic element into a binary string. The conversion process is usually called binarization. During the decoding process, a probability model is gradually built up from the decoded symbols of different contexts. The context modeler (420) is used for modeling purposes. In the decoding process based on normal context, a conventional decoding engine (430) corresponding to the binary arithmetic decoder is used. The selection of the modeling context for the next binary symbol can be determined by the decoded information. Symbols can also be encoded without a context modeling stage and assuming an equal probability distribution, usually called bypass mode, to reduce complexity. For bypassed symbols, a bypass decoding engine (440) can be used. As Figure 4 As shown, switches (S1, S2, and S3) are used to direct the data flow between the normal CABAC mode and the bypass mode. When the normal CABAC mode is selected, the switch is switched to the upper contact. When the bypass mode is selected, the switch is flipped to the lower contact, as shown in FIG. Figure 4 shown.

[0009] In VVC, correlated scalar quantization can be used to quantize transform coefficients. The selection of one of the two quantizers is determined by a state machine with four states. The state of the current transform coefficient is determined by the absolute level value state and parity of the previous transform coefficient in the scan order. The transform block is divided into non-overlapping sub-blocks. The transform coefficient level in each sub-block is entropy coded using multiple sub-block decoding channels. The syntax elements sig_coeff_flag, abs_level_gt1_flag, par_level_flag, and abs_level_gt3_flag are decoded in normal mode in the first sub-block decoding channel. The elements abs_level_gt1_flag and abs_level_gt3_flag indicate whether the absolute value of the current coefficient level is greater than 1 and greater than 3, respectively. The syntax element par_level_flag represents the parity bit of the absolute value of the current level. The partially reconstructed absolute value of the transform coefficient level of the first channel is given by:

[0010] AbsLevelPass1=sig_coeff_flag+par_level_flag+abs_level_gt1_flag+2*abs_level_gt3_flag.(1)

[0011] The context selection for entropy decoding sig_coeff_flag depends on the state of the current coefficient. Therefore, par_level_flag is signaled in the first decoding pass to derive the state of the next coefficient. The syntax elements abs_remainder and coeff_sign_flag are further decoded in bypass mode in subsequent sub-block decoding passes to indicate the coefficient level value and sign of the residual, respectively. The absolute value of the fully reconstructed transform coefficient level is given by:

[0012] AbsLevel = AbsLevelPass1 + 2 * abs_remainder. (2)

[0013] The transform coefficient levels are given by:

[0014] TransCoeffLevel=(2*AbsLevel-(QState>1?1:0))*(1-2*

[0015] coeff_sign_flag),(3)

[0016] Where QState represents the state of the current transform coefficient.

[0017] The present invention aims to further improve the performance of transform coefficient decoding of residual data in a video decoding system. [Summary of the invention]

[0018] In view of this, the present invention provides the following technical solutions:

[0019] The present invention provides a video decoding method, comprising: receiving decoded transform coefficients of a residual block corresponding to a current block, wherein the decoded transform coefficients include decoded symbol residuals associated with a set of jointly predicted coefficient symbols; determining an allowed maximum number based on a decoding context associated with the current block; decoding the decoded symbol residuals into decoded symbol residuals, wherein the total number of the set of jointly predicted coefficient symbols is equal to or less than the allowed maximum number; determining a joint symbol prediction of the set of jointly predicted coefficient symbols by selecting a hypothesis corresponding to a minimum cost from a set of hypotheses for the set of jointly predicted coefficient symbols, wherein a cost between a boundary pixel of the assumed current block and a corresponding adjacent pixel of the current block is calculated for each hypothesis in the set, and the boundary pixels of the current block associated with each hypothesis are reconstructed using information including said each hypothesis; and reconstructing the set of jointly predicted coefficient symbols based on the decoded symbol residuals and the joint symbol prediction.

[0020] The present invention provides a video decoding device, which includes one or more electronic circuits or processors for: receiving decoded transform coefficients of a residual block corresponding to a current block, wherein the decoded transform coefficients include decoded symbol residuals associated with a group of jointly predicted coefficient symbols; determining an allowed maximum number based on a decoding context associated with the current block; decoding the decoded symbol residuals into decoded symbol residuals, wherein the total number of the group of jointly predicted coefficient symbols is equal to or less than the allowed maximum number; determining a joint symbol prediction of the group of jointly predicted coefficient symbols by selecting a hypothesis corresponding to a minimum cost from a set of hypotheses for the group of jointly predicted coefficient symbols, wherein a cost between a boundary pixel of the hypothesized current block and a corresponding adjacent pixel of the current block is calculated for each hypothesis in the set of hypotheses, and the boundary pixels of the current block associated with each hypothesis are reconstructed using information including said each hypothesis; and reconstructing the group joint prediction based on the decoded symbol residuals and the joint symbol prediction.

[0021] The present invention provides a video encoding method, which includes: receiving transform coefficients corresponding to a residual block of a current block; determining a maximum allowed number based on a decoding context associated with the current block; determining a set of jointly predicted coefficient symbols associated with a set of selected transform coefficients, wherein the total number of the set of jointly predicted coefficient symbols is equal to or less than the maximum allowed number; determining a joint symbol prediction of the set of jointly predicted coefficient symbols by selecting a hypothesis corresponding to a minimum cost from a set of hypotheses of the set of jointly predicted coefficient symbols, wherein the cost between the assumed boundary pixels of the current block and the corresponding adjacent pixels of the current block is calculated for each hypothesis in the set of hypotheses, and the boundary pixels of the current block associated with each hypothesis are reconstructed using information including said each hypothesis; determining a symbol residual between the set of jointly predicted coefficient symbols and the joint symbol prediction; and applying context decoding to the symbol residual to generate a decoded symbol residual.

[0022] The present invention also provides a video encoding device, which includes one or more electronic circuits or processors for: receiving transform coefficients corresponding to a residual block of a current block; determining the maximum number allowed based on a decoding context associated with the current block; determining a group of jointly predicted coefficient symbols associated with a group of selected transform coefficients, wherein the total number of group jointly predicted coefficient symbols is equal to or less than the maximum number allowed; determining a joint symbol prediction of the group of jointly predicted coefficient symbols by selecting a hypothesis corresponding to a minimum cost from a group of hypotheses of the group of jointly predicted coefficient symbols, wherein the cost between the assumed boundary pixels of the current block and the corresponding adjacent pixels of the current block is calculated for each hypothesis in the group of hypotheses, and the boundary pixels of the current block associated with each hypothesis are reconstructed using information including each hypothesis; determining a symbol residual between the group jointly predicted coefficient symbols and the joint symbol prediction; and applying context decoding to the symbol residual to generate a decoded symbol residual.

[0023] The video encoding method and device and the video decoding method and device of the present invention can improve the performance of transform coefficient symbol decoding.

Brief Description of the Drawings

[0024] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the present invention and, together with the description, serve to explain the principles of the present invention:

[0025] Figure 1A An exemplary adaptive inter / intra video decoding system incorporating loop processing is shown.

[0026] Figure 1B Another example decoding system is shown.

[0027] Figure 2 A diagram showing a CU that can be split into smaller CUs using one of five split types.

[0028] Figure 3 An example of a CTU recursively partitioned by a QT with a nested MTT is provided.

[0029] Figure 4 An exemplary block diagram of a CABAC process is illustrated.

[0030] Figure 5 The computation of a cost function for deriving the best sign prediction hypothesis for a residual transform block according to Enhanced Compression Model 2 (ECM 2) is shown.

[0031] Figure 6 An example of collecting the signs of the first Nsp coefficients in a diagonal scanning order in a forward sub-block manner starting from the DC coefficient according to one embodiment of the present invention is illustrated.

[0032] Figure 7 A flow chart of an exemplary video decoding system using joint symbol prediction according to an embodiment of the present invention is shown.

[0033] Figure 8 A flowchart of an exemplary video encoding system using joint symbol prediction according to an embodiment of the present invention is shown. [Specific implementation method]

[0034] It will be readily understood that the components of the present invention, as generally described and illustrated in the figures herein, may be arranged and designed in a variety of different configurations. Accordingly, the following more detailed description of the embodiments of the systems and methods of the present invention, as represented in the figures, is not intended to limit the scope of the claimed invention, but rather represents selected embodiments of the invention. References throughout this specification to "one embodiment," "an embodiment," or similar language mean that a particular feature, structure, or characteristic described in connection with that embodiment may be included in at least one embodiment of the present invention. Therefore, appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment.

[0035] In addition, the described features, structures or characteristics may be combined in any suitable manner in one or more embodiments. However, those skilled in the relevant art will recognize that the present invention may be practiced without one or more specific details, or using other methods, components, etc. In other cases, well-known structures or operations are not shown or displayed. Detailed description is provided to avoid obscuring aspects of the present invention. The illustrated embodiments of the present invention will be best understood by reference to the accompanying drawings, in which like parts are represented by like numbers throughout. The following description is provided by way of example only and simply illustrates certain selected embodiments of the apparatus and method consistent with the present invention as claimed herein.

[0036] The Joint Video Experts Group (JVET) of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11 is currently exploring the next generation video coding standard.

[0037] Enhanced Compression Model 2 (ECM 2) uses some promising new decoding tools (M. Coban et al., “Algorithm description of Enhanced Compression Model 2 (ECM 2)”, Joint Video Experts Group of ITU-TSG16 WP3 and ISO / IEC JTC1 / SC29 / WG11, 23rd Meeting, Teleconference, July 7–16, 2021, document JVET-W2025) to further improve VVC. The new tools have been implemented in the reference software ECM-2.0 (ECM Reference Software ECM-2.0, available at https: / / vcgit.hhi.fraunhofer.de / ecm / ECM [online]). In particular, a new method for jointly predicting the set of transform coefficient level signs in a residual transform block has been developed (JVET-D0031, Felix Henry et al., “Residual Coefficient Sign Prediction”, Joint Video Experts Group ITU-TSG16WP3 and ISO / IEC JTC1 / SC29 / WG11, 4th Meeting: Chengdu, China, October 15-21, 2016, document JVET-D0031). In ECM 2, to derive the best sign prediction hypothesis for the residual transform block, the cost function is defined as spanning Figure 5The discontinuity measure for the block boundaries is shown, where block 510 corresponds to the transformed block, circle 520 corresponds to the neighboring samples, and circle 530 corresponds to the reconstructed samples associated with the symbol candidate of block 510. The cost function is defined as the sum of the absolute second derivatives in the residual domain of the above rows and left columns as follows:

[0038]

[0039] In the above equation, R is the reconstructed neighbor, P is the prediction of the current block, and r is the residual hypothesis. The maximum number of allowed prediction symbols N for each symbol prediction hypothesis in the transform block is signaled in the sequence parameter set (SPS). sp , and in ECM-2.0 it is restricted to be less than or equal to 8. The cost function of all hypotheses is measured and the one with the lowest cost is selected as the predictor of the coefficient symbol. Only the coefficient symbols from the upper left 4x4 transform sub-block area in the transform block (the coefficients with the lowest frequency) are allowed to be included in the hypothesis. The first N are collected and decoded according to the raster scan order on the upper left 4x4 sub-block. sp non-zero coefficients (first N sp The sign of the non-zero coefficients (if available) is signaled for those predicted coefficients. For those coefficients, a sign prediction bin is signaled that indicates whether the predicted sign is equal to the selected hypothesis, without signaling the sign of the coefficient. The sign prediction bin is context-coded, where the selected context is derived from whether the coefficient is DC. The context is separate for intra and inter blocks, and for luma and chroma components. For other coefficients without sign prediction, the corresponding sign is decoded by CABAC in bypass mode.

[0040] According to one aspect of the present invention, context modeling for entropy decoding of the sign prediction bits for the current coefficient can be further conditioned on information about the absolute value of the current transform coefficient level. This is because coefficients with larger absolute level values ​​have a greater impact on the output value of the cost function and therefore tend to have a higher correct prediction rate. In the proposed method, the video decoder process includes using multiple context variables to decode syntax information related to the sign of the transform coefficient level related to sign prediction, wherein the selection of the context variable to decode the sign of the current coefficient level can further depend on the absolute value of the current transform coefficient level. In some embodiments, the context selection for entropy decoding the sign prediction bits of certain coefficients further depends on whether the absolute value of the current transform coefficient level is greater than or less than one or more thresholds. In one example, the context selection for entropy decoding of the sign prediction bits of certain coefficients further depends on whether the absolute value of the current transform coefficient level is greater than a first threshold T1. In some preferred embodiments, T1 can be equal to 1, 2, 3, or 4. In another example, the context selection for entropy decoding of the sign prediction bits of certain coefficients further depends on whether the absolute value of the current transform coefficient level is greater than a second threshold T2, where T2 is greater than T1. In some preferred embodiments, (T1, T2) may be equal to (1, 2), (1, 3), or (2, 4).

[0041] According to another aspect of the present invention, the processing for the video decoder may further include adaptively setting the values ​​of the one or more thresholds taking into account the coding context of the current transform block. In some embodiments, the derivation of the one or more thresholds may further depend on the transform block dimension, transform type, color component index, number of predicted symbols, number of non-zero coefficients, or position of the last significant coefficient associated with the current transform block. The derivation of the one or more thresholds may further depend on the prediction mode of the current CU. The derivation of the one or more thresholds may further depend on the position or index associated with the current coefficient in the transform block. The derivation of the one or more thresholds may also depend on the sum of the absolute values ​​of the coefficients predicted by the symbols in the current transform block.

[0042] According to another aspect of the present invention, context modeling for entropy decoding of a sign prediction bit for a current coefficient can be further conditioned on information derived from the absolute values ​​of the current coefficient level and other coefficient levels in the current transform block. In some embodiments, context selection for entropy decoding of a sign for a coefficient in the current transform block can further depend on the sum of the absolute values ​​of the coefficients predicted by the symbol in the current transform block. In some embodiments, context selection for entropy decoding of a sign for a coefficient in the current transform block can further depend on the absolute value of the next coefficient predicted by the symbol or the sum of the absolute values ​​of the remaining coefficients predicted by the symbol in the current transform block.

[0043] In some proposed embodiments, context selection that depends on absolute coefficient level may be employed only for a specified plurality of transform coefficients. When a current coefficient does not belong to the specified plurality of transform coefficients, context selection for the current coefficient is independent of the absolute coefficient level. In some embodiments, the specified plurality of transform coefficients corresponds to the first N1 coefficients associated with symbol prediction according to a predefined scan order in the transform block. When the current coefficient does not belong to the first N1 coefficients, context selection is independent of the absolute coefficient level. In some embodiments, the predefined order is an entropy decoding order of the symbol prediction bits. In some embodiments, N1 is equal to 1, 2, 3, or 4. In some embodiments, the specified group of transform coefficients corresponds to coefficients from a transform coefficient region or scan index range.

[0044] In some embodiments, the designated transform coefficient group corresponds to the DC coefficient in the transform block. When the current transform coefficient is a DC coefficient, the context selection for sign coding can depend on the absolute value of the current transform coefficient level. Otherwise, the context selection for sign coding is independent of the absolute value of the current transform coefficient level. In some embodiments, the designated plurality of transform coefficients comes only from the luma block. The context selection for sign coding can depend on the absolute value of the current transform coefficient level in the luma TB and be independent of the absolute value of the current transform coefficient level in the chroma TB. In some embodiments, the designated plurality of transform coefficients is associated only with certain transform block dimensions, transform types, or CU coding modes.

[0045] According to another aspect of the present invention, context modeling for entropy decoding of the symbol prediction bits of the current coefficient may be further conditioned on information about the encoded symbol prediction bits in the current transform block. In some embodiments, context selection for entropy decoding of the symbol prediction bits of certain coefficients may further depend on whether the first decoded symbol prediction or DC symbol prediction in the current transform block is correct. In some embodiments, context selection for entropy decoding of the symbol of the current coefficient may further depend on the accumulated number of symbol prediction bits corresponding to incorrect symbol predictions. In some embodiments, context selection for entropy decoding of the symbol prediction bits of certain coefficients depends on whether the accumulated number of symbol prediction bits corresponding to incorrect symbol predictions is greater than one or more specified thresholds. In one embodiment, context selection for entropy decoding of the symbol prediction bits of certain coefficients depends on whether the accumulated number of symbol prediction bits corresponding to incorrect symbol predictions is greater than T ic , where T ic= 0, 1, 2 or 3. According to another aspect of the present invention, when the accumulated number of coded symbol prediction bits corresponding to erroneous symbol predictions is greater than a specified threshold, entropy decoding of the remaining symbol prediction bits may be switched to a bypass mode.

[0046] According to another aspect of the present invention, context modeling for entropy decoding of the symbol prediction bits of the current coefficient may be further conditioned on the total number of symbol prediction bits in the current transform block. In some embodiments, context selection for entropy decoding of the symbol prediction bits of certain coefficients in the current transform block may further depend on whether the total number of symbol prediction bits in the current transform block is greater than one or more non-zero thresholds. According to yet another aspect of the present invention, the process for a video decoder may further include adaptively setting the values ​​of the one or more thresholds based on the coding context of the current transform block. In some embodiments, the derivation of the one or more thresholds may further depend on the transform block dimension, transform type, color component index, number of predicted symbols, number of non-zero coefficients, or position of the last significant coefficient associated with the current transform block. The derivation of the one or more thresholds may further depend on the prediction mode of the current CU. The derivation of the one or more thresholds may further depend on the position or index associated with the current coefficient in the transform block. The derivation of the one or more thresholds may also depend on the sum of the absolute values ​​of the symbol-predicted coefficients in the current transform block.

[0047] According to another aspect of the present invention, the context modeling for entropy decoding of the symbol prediction bit of the current coefficient can be further conditioned on information about the index or position of the current transform coefficient in the transform block, wherein the index of the current transform coefficient can correspond to the scan order of the decoded predicted symbols, or can be in a raster scan order, a diagonal scan order (such as Figure 6 In some embodiments, the context selection for entropy coding of the sign prediction bits of certain coefficients depends on whether the index of the current transform coefficient level is greater than or less than one or more non-zero thresholds.

[0048] In some other embodiments, the context selection for entropy decoding the sign prediction bits of certain coefficients depends on whether the distance between the upper left block origin at position (0,0) and the current coefficient position (x,y) is greater than or less than another non-zero threshold or thresholds, where the distance is defined as (x+y). According to another aspect of the present invention, the process for the video decoder may further include adaptively setting the value of the one or more thresholds or another non-zero threshold or thresholds taking into account the coding context of the current transform block. In some embodiments, the derivation of the one or more thresholds or another non-zero threshold or thresholds may further depend on the transform block dimension, transform type, color component index, number of prediction symbols, number of non-zero coefficients, or position of the last significant coefficient associated with the current transform block. The derivation of the one or more thresholds or another non-zero threshold or thresholds may also depend on the prediction mode of the current CU. The derivation of the one or more thresholds may further depend on the absolute level associated with the current coefficient or further depend on the sum of the absolute values ​​of the sign-predicted coefficients in the current transform block.

[0049] According to another aspect of the present invention, context modeling for entropy decoding of the sign prediction bits of the current coefficient in the current transform block can be further conditioned on the width, height, or block size of the current transform block. In some embodiments, context selection for entropy decoding of the sign prediction bits of certain coefficients in the current transform block depends on whether the width, height, or block size of the current transform block is greater than or less than one or more thresholds.

[0050] According to another aspect of the present invention, context modeling for entropy decoding of sign prediction bits for a current coefficient in a current transform block may be further conditioned on a transform type associated with the current transform block. In some embodiments, context selection for entropy decoding of sign prediction bits for certain coefficients in the current transform block may further depend on the transform type associated with the current transform block. In some exemplary embodiments, when the current block transform type belongs to a low-frequency non-separable transform (LFNST) or multiple transform selection (MTS), the video decoder may allocate a separate set of contexts for entropy decoding of sign prediction bits for certain transform coefficients in the current transform block.

[0051] On the decoder side, the corresponding cost functions are evaluated for the possible hypotheses, as disclosed in JVET-D0031. The decoder parses the coefficients, signs, and sign residuals as part of its parsing process. The signs and sign residuals are parsed at the end of the TU, at which point the decoder knows the absolute values ​​of all coefficients. It can therefore determine which symbols were predicted, and for each predicted symbol, it can determine the context for parsing the symbol prediction residuals based on the dequantized coefficient values. The knowledge of a "correct" or "incorrect" prediction is simply stored as part of the CU data for the block being parsed. At this point, the true sign of the coefficient is unknown. During reconstruction, the decoder performs operations similar to those of the encoder to determine the hypothesis costs. The hypothesis that achieves the minimum cost is determined to be the symbol prediction.

[0052] In ECM-2.0, when applying symbol prediction to the current transform block, the first N sp The number of all possible hypotheses subject to the discontinuity measure is equal to 1< <N sp and increases significantly with the number of predicted symbols, where "<<" is a bitwise up-shift operator. In one proposed method, a video decoder may include a constraint on the minimum absolute value of the transform coefficient levels applicable to sign prediction, where the minimum absolute value may be greater than 1 for certain specified transform coefficients. The sign of the current coefficient under the constraint is only allowed to be included in the sign prediction hypothesis if the absolute value of the current transform coefficient level is greater than or equal to the specified minimum value. In this way, coefficients with smaller magnitudes can be excluded from sign prediction, thereby reducing the number of prediction symbols in the transform block.

[0053] In some embodiments, the current coefficient level is only valid if the absolute value of the current coefficient level is greater than a specified threshold value T sp The sign of the current coefficient is included in the hypothesis of sign prediction only when T sp is greater than 0. In one embodiment, the sign of the current coefficient is allowed to be included in the hypothesis for sign prediction only if the absolute value of the current coefficient level is greater than 1 for some specified coefficients. The proposed method may further include adaptively determining T sp In one example, T can be adaptively derived according to the block width, height or size of the current transform block. sp In another example, T can be adaptively derived according to the quantization parameter selected for the current transform block. sp For another example, T can be adaptively derived according to the position or index of the current coefficient in the current block.sp In one exemplary embodiment, T can be derived as follows sp :

[0054] When the index of the current coefficient is less than C1, T sp =0;

[0055] Otherwise, when the index of the current coefficient is less than C2, T sp is equal to 1;

[0056] Otherwise, T sp is equal to 2,

[0057] Where 0≤C1 <C2≤N sp .

[0058] In ECM-2.0, when symbol prediction is applied to the current transform block, the first N sp The symbols of the non-zero coefficients (when available). The transform coefficient area applicable to symbol prediction is fixed to the upper left corner 4x4 sub-block of each transform block. In the proposed method, the processing of the video decoder may further include adaptively setting the transform coefficient area or index range applicable to symbol prediction in the current transform block according to the decoding context associated with the current transform block. The video decoder may include a method for collecting the first N sp The video decoder may further include determining a maximum index for which symbol prediction is applicable in the current transform block according to the specified scan order, wherein a symbol of any transform coefficient having an index greater than the maximum index according to the specified scan order in the current transform block is not applicable for symbol prediction.

[0059] Alternatively, the video decoder may include determining whether a transform coefficient or subblock is eligible for symbol prediction in the current transform block based on a position (x, y) of the transform coefficient or subblock in the current transform block and one or more specific thresholds, where the top-left block origin of the current transform block corresponds to position (0, 0). In some embodiments, in certain transform blocks, the transform coefficient or subblock at position (x, y) is eligible for symbol prediction only when x is less than a first threshold and y is less than a second threshold. In some other embodiments, in certain transform blocks, the transform coefficient or subblock at position (x, y) is eligible for symbol prediction only when a distance between the transform block origin and the position of the current coefficient or subblock (equal to (x+y)) is less than another specified threshold.

[0060] In some embodiments, the transform coefficient region or index range applicable to the sign prediction in the current transform block can be adaptively set according to the block dimension, transform type, color component index, the position of the last significant coefficient, or the number of non-zero coefficients associated with the current transform block. In some embodiments, when the width or height of the current transform block is less than a specified threshold or the size of the current transform block is less than another specified threshold, the video decoder can reduce the transform coefficient region or index range applicable to the sign prediction in the current transform block. In some exemplary embodiments, for a block width or height less than a threshold M, the transform coefficient region or index range is less than a threshold M. sp The transform block or block size is smaller than another threshold MN sp The transform block can set the transform coefficient index range suitable for symbol prediction to be from 0 to R1 sp , where R1 sp is the maximum index in the specified scan order. In some embodiments, when M sp When MN is equal to 4, 8, 16 or 32, or when sp When R1 is equal to 16, 64, 256 or 1028, sp It can be equal to 2, 3, 5 or 7. sp 、M sp and MN sp The values ​​are only used as examples, and other appropriate values ​​can be selected or determined as needed.

[0061] In some embodiments, when the width or height of the current transform block is greater than a specified threshold or the size of the current transform block is greater than another specified threshold, the video decoder may increase the transform coefficient region or index range applicable to sign prediction in the current transform block.

[0062] In some exemplary embodiments, when the width or height of the current transform block is greater than a specified threshold or the size of the current transform block is greater than another specified threshold, the video decoder may include more than one sub-block from the low-frequency transform block region. In some embodiments, the video decoder may reduce the transform coefficient region or index range applicable to sign prediction of one or more specified transform types. In some exemplary embodiments, for one or more specified transform types, the transform coefficient index range applicable to sign prediction is set to be from 0 to R2 sp , where R2 sp is the maximum index according to the specified scan order.

[0063] In some other exemplary embodiments, the transform block is considered to be a positive coefficient only when the distance (x+y) between the transform block origin and the position equal to the current coefficient is less than another specified threshold D1. spIn some embodiments, the one or more specified transform types include certain low frequency non-separable transform (LFNST) types and / or certain transform types associated with multiple transform selection (MTS), R2 sp Can be equal to 2, 3, 5, and D1 sp Can be equal to 1, 2, 3, 4, 5, 6, or 7. In some embodiments, for certain MTS and / or LFNST types, the transform coefficient region or index range applicable to sign prediction is reduced compared to DCT type II. In some embodiments, the video decoder may expand the transform coefficient region or index range applicable to sign prediction for one or more specified transform types. In some exemplary embodiments, when the transform type associated with the current residual block belongs to one or more specified transform types, the video decoder may include more than one sub-block from the low-frequency transform block region. In some embodiments, the one or more specified transform types include DCT type II.

[0064] The proposed method may further include signaling information about the transform coefficient region or index range applicable to sign prediction in the bitstream. In some embodiments, the video coder may signal one or more syntax elements for deriving the transform coefficient region or index range applicable to sign prediction in one or more high-level parameter sets (e.g., a sequence parameter set (SPS), a picture parameter set (PPS), a picture header (PH), and / or a slice header (SH)).

[0065] In some other embodiments, the video coder may signal more than one syntax set to derive information about more than one transform coefficient region or index range for use in symbol prediction in one or more higher-level syntax sets, wherein each syntax set corresponds to information about deriving a specific transform coefficient region or index range applicable to symbol prediction for a certain specified transform block, and the selection of the specific transform coefficient region or index range for the current transform block depends on the original context associated with the current transform block. For example, the selection of the specific transform coefficient region or index range for the current transform block may depend on the transform block size, transform type, color component index, number of prediction symbols, position of the last significant coefficient, or number of non-zero coefficients associated with the current transform block. The selection of the specific transform coefficient region or index range for the current transform block may also depend on the prediction mode of the current CU.

[0066] In ECM-2.0, the maximum number of prediction symbols N in a transform block is signaled in the sequence parameter set (SPS). sp , and N spis limited to be less than or equal to 8. In the proposed method, the video decoder may adaptively set the maximum allowed number of prediction symbols in the current transform block according to the decoded original context associated with the current transform block. In some embodiments, the video decoder may adaptively derive the maximum allowed number of prediction symbols in the current transform block from one or more reference syntax values ​​based on the translated context associated with the current transform block. In some embodiments, the derivation of the maximum number of prediction symbols allowed in the current transform block depends on the transform block dimension, the transform type, the color component index, the position of the last significant coefficient, or the number of non-zero coefficients associated with the current transform block. For example, when the reference syntax value N specified in the high-order syntax set is sp When set to greater than 4, for smaller transform block sizes (such as 4*4), the maximum number allowed can be set to 4. In one example, the maximum number of prediction symbols allowed in the current transform block is set equal to the reference syntax value N sp Multiply by a scaling factor, where the scaling factor value is chosen depending on the current block dimension.

[0067] In some other embodiments, a video coder may signal more than one syntax set from one or more advanced parameter sets, such as a sequence parameter set (SPS), a picture parameter set (PPS), a picture header (PH), and / or a slice header (SH), where each syntax set corresponds to information regarding a specific parameter value for deriving the maximum number of prediction symbols allowed in a transform block for a particular specified transform block. Furthermore, the selection of the specific parameter value for the maximum number of prediction symbols allowed in the current transform block depends on the current coding context. For example, the selection of the specific parameter value for the maximum number of prediction symbols allowed in the current transform block may depend on the transform block dimension, transform type, color component index, the position of the last significant coefficient, or the number of non-zero coefficients associated with the current transform block. In one embodiment, the selection of the specific parameter value for the maximum number of prediction symbols allowed in the current transform block may be determined by comparing the current block size to one or more predefined thresholds. In another embodiment, the selection of the specific parameter value is determined by whether a secondary transform, such as LFNST, is applied to the current transform block. The selection of the specific parameter value for the current transform block may also depend on the prediction mode of the current CU. In the proposed method, the processing of the video decoder may also include more than one upper bound constraint, each constraint limiting the maximum parameter value for the maximum number of prediction symbols allowed in the transform block for some specified transform block. The determination of the upper bound constraint for the current transform block may depend on the transform block dimension, transform type, color component index, position of the last significant coefficient, or number of non-zero coefficients associated with the current transform block. In one embodiment, for transform blocks with a block size less than or equal to 16, the upper bound constraint is set to 4.

[0068] In ECM-2.0, when the transform block width or height of the current transform block is greater than 128 or less than 4, the coefficient sign prediction tool is disabled for the current transform block. According to one aspect of the present invention, a video decoder can use different constraints on block dimensions to implement sign prediction for different transform types. In the proposed method, the video decoder includes more than one set of dimensional constraints for implementing sign prediction for different transform types. The video decoder further includes determining a selected set of dimensional constraints for enabling sign prediction in the current transform block based on the current transform type. In some embodiments, for one or more transform types belonging to MTS and / or LFNST types, the video decoder can select a set including a lower maximum dimensional constraint value than DCT type II.

[0069] In ECM-2.0, the first N sp The signs of the first N non-zero coefficients are collected and decoded according to the forward raster scan order on the first sub-block for sign prediction. According to another aspect of the present invention, the video decoder can determine the first N selected to be subject to sign prediction according to the order of decreasing coefficient magnitudes based on the expected distribution of transform coefficient levels. sp In one embodiment, the first N coefficients are collected sequentially according to a forward sub-block diagonal scan starting from the DC coefficient. sp The signs of the coefficients, where the transform coefficients in each sub-block are accessed in diagonal scan order, such as Figure 6 As shown, the sub-blocks in each transform block are also accessed in diagonal scanning order. In another embodiment, the first N sub-blocks are collected according to the diagonal scanning order of the forward sub-blocks. sp In another embodiment, the first N coefficients are sp The signs of the coefficients are collected in a forward sub-block-wise diagonal scan order and decoded in a backward sub-block-wise diagonal scan order.

[0070] In ECM-2.0, the best symbol prediction hypothesis for the current transform block is determined by minimizing a cost function corresponding to a discontinuity measure by the sum of absolute differences (SAD) across block boundaries. In the proposed method, the best symbol prediction hypothesis for the current transform block is determined by minimizing a cost function corresponding to a discontinuity measure by the sum of squared errors across block boundaries, as shown below:

[0071]

[0072] In ECM-2.0, when symbol prediction is enabled for the current transform block, the upper left (lowest frequency) transform sub-block is subject to symbol prediction, and during the residual decoding process for entropy decoding the current transform block, the symbol entropy decoding of the upper left transform sub-block is skipped. After entropy decoding all transform blocks (through the transform block-by-transform block residual decoding process) and the syntax elements for indicating the LFNST and MTS indexes associated with the current CU, a separate symbol decoding process is used to entropy decode the symbol information on the transform sub-blocks of each residual block for which symbol prediction is enabled in the current CU. In this way, each transform block in the current CU that can be symbol predicted needs to be revisited for symbol decoding after entropy decoding each transform block through the residual decoding process.

[0073] In the proposed method, a video decoding system includes entropy decoding multiple residual blocks in a current CU, wherein each of the multiple residual blocks is decoded by a residual decoding process. Symbol prediction bits in a current transform block are also entropy decoded in the residual decoding process used to entropy decode the current transform block. Each decoded symbol prediction bit of a current coefficient indicates whether the predicted symbol of the current coefficient is correct. The separate symbol decoding process in ECM-2.0 is eliminated. In some embodiments, each residual block is divided into one or more non-overlapping sub-blocks, and the residual decoding process includes entropy decoding each sub-block in the current transform block according to a specified sub-block scanning order. The residual decoding process may also include one or more sub-block decoding passes for entropy decoding information regarding the absolute values ​​of the transform coefficient levels in the current sub-block. The residual decoding process may also include determining a set of coefficients for symbol prediction in the current transform block according to a specified rule. The residual decoding process may also include entropy decoding the symbol prediction bits in the current transform block according to a specified scanning order.

[0074] In some embodiments, sign prediction applies only to transform coefficients from the top-left transform sub-block within each transform block. The residual decoding process may also include a sub-block sign decoding pass followed by one or more sub-block decoding passes. The one or more sub-block decoding passes are used to entropy decode the absolute values ​​of the transform coefficient levels in the top-left sub-block, and the sub-block sign decoding pass entropy decodes coefficient sign information according to a specified scan order within the top-left sub-block. In the sub-block sign decoding pass, a syntax flag may be signaled for the current non-zero coefficient in the top-left sub-block. When the current non-zero coefficient is sign predicted, the syntax flag signals the value of the sign prediction bit to indicate whether the sign prediction for the current coefficient is correct. Otherwise, the syntax flag signals the sign of the current non-zero coefficient. In some embodiments, the sub-block sign decoding pass for entropy decoding the sign information follows a reverse diagonal scan order within the top-left sub-block. In some other embodiments, the sub-block sign decoding pass for entropy decoding the sign information follows a forward diagonal scan order within the top-left sub-block. Context modeling of the entropy decoding symbol prediction bits may be further conditioned on the decoded symbol prediction bits. In some embodiments, the sign prediction is applied to transform coefficients of more than one sub-block from the upper left transform block region in the transform block.The proposed sub-block sign decoding channel can be similarly applied to each sub-block from the upper left transform block region for sign prediction.

[0075] In ECM-2.0, if Figure 5 As shown, the best symbol prediction hypothesis is determined in the current transform block by minimizing the cost function corresponding to the discontinuity measure across the left block boundary and the upper block boundary. According to another aspect of the present invention, the video decoder can use only one side (left or upper) of the block boundary samples to derive the prediction symbols or completely turn off the symbol prediction by considering the decoding context associated with the current transform block. In the proposed method, the video decoder includes a symbol prediction mode that corresponds to deriving the prediction symbols in the transform block based on the discontinuity measure across both the upper block boundary and the left block boundary. The video decoder also includes two additional symbol prediction modes, wherein the first additional mode corresponds to deriving the prediction symbols in the transform block based only on the discontinuity measure across the left block boundary, and the second additional mode corresponds to deriving the prediction symbols in the transform block based only on the discontinuity measure across the upper block boundary. In some embodiments, the cost functions corresponding to the first additional mode and the second additional mode are given by equations (6) and (7) respectively as follows:

[0076]

[0077]

[0078] The video decoder further includes determining a selected symbol prediction mode for the current transform block. The video decoder may determine the selected symbol prediction mode based on the luma component only for all transform blocks in the CU. In some embodiments, the video decoder may derive the selected symbol prediction mode in the current transform block while taking into account a selected intra-prediction direction in the current CU. In one example, the video decoder may set the selected symbol prediction mode to a first additional prediction mode when the selected intra-prediction direction is close to a horizontal prediction direction, and set the selected symbol prediction mode to a second additional prediction mode when the selected intra-prediction direction is close to a vertical prediction direction.

[0079] In some other embodiments, the video decoder may derive a selected symbol prediction mode in the current transform block while taking into account block boundary conditions associated with the current block. For example, when a particular current block boundary overlaps a tile or slice boundary, the video decoder may determine not to use reconstructed samples from that particular boundary to derive prediction symbols. When reconstructed neighboring samples are not available at both the upper and left block boundaries, the video decoder may disable symbol prediction for that block. For another example, when it is determined from reconstructed boundary samples that an image edge may exist at a particular current block boundary, the video decoder may determine not to use reconstructed samples from that particular boundary to derive predicted symbols.

[0080] In some other embodiments, when a residual signal is present in a current transform block or coding unit, the video coder may signal one or more syntax elements for deriving a selected symbol prediction mode in the current transform block or coding unit. In some embodiments, the video coder may signal one or more syntax elements for deriving a selected symbol prediction mode in the current transform block only when the number of prediction symbols in the current transform block is greater than a threshold.

[0081] In some embodiments, the video coder may vertically downsample the left block boundary and horizontally downsample the upper block boundary in the cost function for discontinuity measurement with reduced computational complexity. In the proposed embodiment, when only one of the two additional prediction modes is selected for the current transform block, the downsampling rate of the selected block boundary may be reduced by a factor of 2.

[0082] In ECM-2.0, when symbol prediction is applied to the current transform block, the cost function Eqn. (1) is minimized to jointly predict up to N sp The number of possible hypotheses for which discontinuity measurements need to be performed is equal to 1< <N spand increases significantly with the number of symbols to be jointly predicted. According to another aspect of the present invention, the set of symbols subject to symbol prediction in the transform block can be predicted in subsets to reduce the number of all possible hypotheses subject to discontinuity measurement. The set of symbols subject to symbol prediction is called a jointly predicted coefficient symbol set (also called a set of jointly predicted coefficient symbols). In the proposed method, the set of symbols subject to symbol prediction in the transform block can be divided into one or more subsets of symbols. Predicting and decoding the set of symbols subject to symbol prediction includes predicting and decoding each subset of symbols, wherein the symbols in each subset are jointly predicted for all possible hypotheses of each subset by minimizing a cost function such as Eqn. (1). The symbols derived from the decoded subsets can be used to reconstruct boundary samples in the current transform block to predict the symbols of the current subset. By dividing the set of symbols subject to symbol prediction into subsets, the maximum number of predicted symbols allowed in the transform block can be further increased to 8.

[0083] In some embodiments, the video decoder may use the same subset size S sp , to evenly divide the symbol set into one or more subsets, where the subset sizes are all equal to S sp , except that a subset can have a set size of S sp The remaining size after division. In some embodiments, S sp The value of can be equal to 1, 2, 3, 4, 5, 6, 7 or 8, or can be signaled in the bitstream. In some other embodiments, the video decoder can adaptively set the subset size S sp For example, S may be adaptively determined according to the transform block dimension, transform type, color component index, number of non-zero coefficients, or number of prediction symbols associated with the current transform block. sp . S sp Adaptation can also be performed based on the prediction mode of the current CU. In some embodiments, symbol prediction can be skipped for a subset, and all symbols in the subset are decoded in bypass mode. For example, the video decoder can skip symbol prediction for a subset in the current transform block, where the subset is not the first subset in the current transform block and the number of predicted symbols or non-zero coefficients in the subset is less than S. sp or other specified threshold.

[0084] According to another aspect of the present invention, the video decoder may divide the set of symbols in the transform block that are subject to symbol prediction into subsets of different sizes. (i) sp Can be used to indicate the subset size of subset i. In one example, when up to 8 symbols are subject to symbol prediction in the transform block, for i equal to 0, 1 and 2, the subset size S (i) spIn another example, when up to 16 symbols are subject to symbol prediction in a transform block, the subset size S is equal to 2, 2, and 4 when i is equal to 0, 1, 2, and 3, respectively. (i) sp In another example, when up to 16 symbols are subject to symbol prediction in a transform block, the subset size S is equal to 2, 2, 4, and 8, respectively, when i is equal to 0, 1, 2, 3, and 4, respectively. (i) sp Which are equal to 1, 2, 3, 4 and 6 respectively.

[0085] In some embodiments, the video decoder may determine the associated subset index for sign prediction of the current transform coefficient based on the position or scan index of the current transform coefficient in the current transform block. For example, when the scan index of the current coefficient is greater than a specified threshold T sub , the associated subset index is set equal to 1; otherwise, the associated subset index is set equal to 0. In some embodiments, T sub Equal to 0, 1, 2, 3, 4, or 5.

[0086] In some embodiments, the video decoder may adaptively divide the set of symbols subject to symbol prediction into subsets based on the transform block dimension, transform type, color component index, number of non-zero coefficients, or number of prediction symbols associated with the current transform block. In some other embodiments, the video decoder may adaptively divide the set of symbols subject to symbol prediction into subsets based on the prediction mode of the current CU. In some other embodiments, the video decoder may adaptively consider the absolute values ​​of the coefficient levels in the current transform block to determine whether to divide the set of symbols subject to symbol prediction into subsets of variable sizes in the current transform block.

[0087] In some embodiments, information regarding the partitioning of the set of symbols subject to symbol prediction into subsets may be signaled in the bitstream. In some specific embodiments, information regarding the partitioning of the set of symbols subject to symbol prediction into subsets may be signaled in a sequence parameter set (SPS), a picture parameter set (PPS), a picture header (PH), and / or a slice header (SH). In some embodiments, context modeling for entropy coding the symbol of a current coefficient associated with symbol prediction may be further conditioned on information derived from the partitioning of the set of symbols subject to symbol prediction into subsets. In some embodiments, context selection for entropy coding the symbol of the current transform coefficient in a subset may further depend on a subset index associated with the subset.

[0088] In the present invention, entropy decoding of the sign of the current coefficient may refer to entropy decoding of the sign prediction bits of the current coefficient in any of the above-mentioned methods.

[0089] The proposed aspects, methods and related embodiments may be implemented individually and jointly in image and video coding systems.

[0090] Any of the aforementioned proposed methods can be implemented in an encoder and / or a decoder. For example, any of the proposed methods can be implemented in an encoder (e.g., Figure 1A The coefficient decoding module and / or decoder (e.g., Figure 1B Alternatively, any of the proposed methods may be implemented as a circuit integrated into the coefficient decoding module of the encoder and / or the coefficient decoding module of the decoder.

[0091] Figure 7 A flowchart of an exemplary video decoding system using joint symbol prediction according to an embodiment of the present invention is shown. The steps shown in the flowchart can be implemented as program code executable on one or more processors (e.g., one or more CPUs) on the encoder side. The steps shown in the flowchart can also be implemented based on hardware, such as one or more electronic devices or processors that are arranged to perform the steps in the flowchart. In step 710, a video decoder according to the method will receive encoded transform coefficients of a residual block corresponding to a current block, wherein the encoded transform coefficients include encoded symbol residuals. In step 720, the maximum allowed number is determined based on the decoding context associated with the current block. In step 730, the encoded symbol residuals are decoded into decoded symbol residuals, wherein the total number of coefficient symbols of the set of joint predictions is equal to or less than the allowed number. In step 740, a joint sign prediction for the set of jointly predicted coefficient symbols is determined by selecting a hypothesis corresponding to a minimum cost from a set of hypotheses for the set of jointly predicted coefficient symbols, wherein a cost between a boundary pixel of the current block of the hypothesis and a corresponding neighboring pixel of the current block is calculated for each hypothesis in the set of hypotheses, and the boundary pixels of the current block associated with each hypothesis are reconstructed using information including each hypothesis. In step 750, the set of jointly predicted coefficient symbols is reconstructed based on the decoded symbol residual and the joint sign prediction.

[0092] Figure 8A flowchart of an exemplary video encoding system utilizing joint symbol prediction according to an embodiment of the present invention is shown. In step 810, a video encoder according to the method receives transform coefficients corresponding to a residual block of a current block. In step 820, a maximum allowed number is determined based on a decoding context associated with the current block. In step 830, a set of jointly predicted coefficient symbols associated with a selected set of transform coefficients is determined, wherein the total number of the jointly predicted coefficient symbols in the set is equal to or less than the allowed maximum number. In step 840, a joint symbol prediction for the set of jointly predicted coefficient symbols is determined by selecting a hypothesis corresponding to a minimum cost from a set of hypotheses for the set of jointly predicted coefficient symbols, wherein a cost is calculated for each hypothesis in the set between a boundary pixel of the current block and a corresponding neighboring pixel of the current block, and wherein the boundary pixels of the current block associated with each hypothesis are reconstructed using information including each hypothesis. In step 850, a symbol residual is determined between the set of jointly predicted coefficient symbols and the joint symbol prediction. In step 860, context decoding is applied to the symbol residual to generate an encoded symbol residual.

[0093] The flowchart is intended to illustrate an example of video encoding according to the present invention. Those skilled in the art may modify each step, rearrange steps, split steps, or combine steps to implement the present invention without departing from the spirit of the present invention. In this disclosure, specific syntax and semantics have been used to illustrate examples to implement embodiments of the present invention. Those skilled in the art may practice the present invention by replacing syntax and semantics with equivalent syntax and semantics without departing from the spirit of the present invention.

[0094] The above description is presented to enable those skilled in the art to practice the present invention provided in the context of a specific application and its requirements. Various modifications to the described embodiments will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments. Therefore, the present invention is not intended to be limited to the specific embodiments shown and described, but rather to conform to the widest scope consistent with the principles and novel features disclosed herein. In the above detailed description, various specific details have been described in order to provide a thorough understanding of the present invention. However, it will be understood by those skilled in the art that the present invention can be put into practice.

[0095] The embodiments of the present invention as described above can be implemented in various hardware, software codes, or a combination of the two. For example, one embodiment of the present invention can be a program code integrated into one or more circuits in a video compression chip or integrated into video compression software to perform the processing described herein. An embodiment of the present invention can also be a program code to be executed on a digital signal processor (DSP) to perform the processing described herein. The present invention can also involve many functions performed by a computer processor, a digital signal processor, a microprocessor, or a field programmable gate array (FPGA). These processors can be configured to perform specific tasks according to the present invention by executing machine-readable software code or firmware code that defines the specific methods embodied in the present invention. The software code or firmware code can be developed in different programming languages ​​and different formats or styles. The software code can also be compiled for different target platforms. However, different code formats, styles, and languages ​​of the software code and other ways of configuring the code to perform tasks according to the present invention will not depart from the spirit and scope of the present invention.

[0096] The present invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described examples are to be considered in all respects only as illustrative and not restrictive. The scope of the present invention is, therefore, indicated by the appended claims rather than by the foregoing description. All changes that come within the meaning and range of equivalency of the claims are intended to be embraced within their scope.

Claims

1. A video decoding method, the method comprising: receiving decoded transform coefficients of a residual block corresponding to a current block, wherein the decoded transform coefficients include decoded symbol residuals associated with a set of jointly predicted coefficient symbols; determining a maximum allowed number based on a decoding context associated with the current block; decoding the decoded symbol residual into a decoded symbol residual, wherein a total number of the set of jointly predicted coefficient symbols is equal to or less than the allowed maximum number; determining a joint sign prediction for the set of jointly predicted coefficient signs by selecting a hypothesis corresponding to achieving a minimum cost from a set of hypotheses for the set of jointly predicted coefficient signs, wherein a cost between a boundary pixel of the current block of the hypothesis and a corresponding neighboring pixel of the current block is calculated for each hypothesis in the set of hypotheses, and the boundary pixel of the current block associated with each hypothesis is reconstructed using information included in said each hypothesis; and The set of jointly predicted coefficient symbols is reconstructed based on the decoded symbol residual and the joint symbol prediction.

2. The video decoding method according to claim 1, wherein: One or more syntax elements used to derive the allowed maximum number are parsed from one or more high-level parameter sets.

3. The video decoding method according to claim 2, wherein: The one or more advanced parameter sets include a sequence parameter set, a picture parameter set, a picture header, a slice header, or a combination thereof.

4. The video decoding method according to claim 1, wherein: The maximum allowed number is determined according to the block size of the current block.

5. The video decoding method according to claim 4, wherein: When the block size is equal to 4×4, the allowed maximum number is less than or equal to 4. The video decoding method according to claim 1 , wherein: The allowed maximum number is determined according to the transform type of the current block.

7. The video decoding method according to claim 1, wherein: The bits of the decoded symbol residual associated with the set of jointly predicted coefficient symbols are entropy decoded in a residual decoding process for entropy decoding of the current block.

8. The video decoding method according to claim 7, wherein: The residual decoding process also includes a sub-block symbol decoding channel for entropy decoding the set of jointly predicted coefficient symbols of one or more sub-blocks in the upper left area of ​​the current block after one or more sub-block decoding channels for entropy decoding the absolute values ​​of the transform coefficient levels of one or more sub-blocks.

9. The video decoding method according to claim 1, wherein: The cost is calculated between boundary pixels of the current block and corresponding neighboring pixels of the current block that are only on the top boundary of the current block or only on the left boundary of the current block. 10 . The video decoding method of claim 9 , wherein the selection between the top boundary of the current block and the left boundary of the current block depends on the selected intra prediction direction of the current block.

11. The video decoding method according to claim 1, wherein: The set of jointly predicted coefficient symbols is divided into a plurality of subsets, wherein each subset includes one or more of the set of jointly predicted coefficient symbols.

12. A video decoding apparatus comprising one or more electronic circuits or processors configured to: receiving decoded transform coefficients of a residual block corresponding to a current block, wherein the decoded transform coefficients include decoded symbol residuals associated with a set of jointly predicted coefficient symbols; determining a maximum allowed number based on a decoding context associated with the current block; decoding the decoded symbol residual into a decoded symbol residual, wherein a total number of the set of jointly predicted coefficient symbols is equal to or less than the allowed maximum number; determining a joint sign prediction for the set of jointly predicted coefficient signs by selecting a hypothesis corresponding to achieving a minimum cost from a set of hypotheses for the set of jointly predicted coefficient signs, wherein a cost between a boundary pixel of the current block of the hypothesis and a corresponding neighboring pixel of the current block is calculated for each hypothesis in the set of hypotheses, and the boundary pixel of the current block associated with each hypothesis is reconstructed using information included in said each hypothesis; and The set of jointly predicted coefficient symbols is reconstructed based on the decoded symbol residual and the joint symbol prediction.

13. A video encoding method, the method comprising: Receive the transform coefficients corresponding to the residual block of the current block; determining a maximum allowed number based on a decoding context associated with the current block; determining a set of jointly predicted coefficient symbols associated with a selected set of transform coefficients, wherein a total number of the set of jointly predicted coefficient symbols is equal to or less than the allowed maximum number; determining a joint sign prediction for the set of jointly predicted coefficient signs by selecting a hypothesis corresponding to achieving a minimum cost from a set of hypotheses for the set of jointly predicted coefficient signs, wherein a cost between a boundary pixel of the current block of the hypothesis and a corresponding neighboring pixel of the current block is calculated for each hypothesis in the set of hypotheses, and the boundary pixel of the current block associated with each hypothesis is reconstructed using information included in said each hypothesis; determining a sign residual between the signs of the coefficients of the set of joint predictions and the joint sign prediction; as well as Context coding is applied to the symbol residual to generate a decoded symbol residual. The video encoding method of claim 13 , further comprising signaling one or more syntax elements for deriving the allowed maximum number.

15. The video encoding method according to claim 14, wherein: The one or more syntax elements are signaled in a sequence parameter set, a picture parameter set, a picture header, a slice header, or a combination thereof.

16. The video encoding method according to claim 13, wherein: The decoding context corresponds to the block size of the current block.

17. The video encoding method according to claim 13, wherein: The bits of the decoded symbol residual associated with the set of jointly predicted coefficient symbols are entropy decoded in a residual decoding process for entropy decoding of the current block.

18. The video encoding method according to claim 17, wherein: The residual decoding process also includes a sub-block symbol decoding channel for entropy decoding the set of jointly predicted coefficient symbols of one or more sub-blocks in the upper left area of ​​the current block after one or more sub-block decoding channels for entropy decoding the absolute values ​​of the transform coefficient levels of one or more sub-blocks.

19. The video encoding method according to claim 13, wherein: The cost is calculated between boundary pixels of the current block and corresponding neighboring pixels of the current block that are only on the top boundary of the current block or only on the left boundary of the current block.

20. The video encoding method of claim 19, wherein the selection between the top boundary of the current block and the left boundary of the current block depends on the selected intra prediction direction of the current block.

21. The video encoding method according to claim 13, wherein: The set of jointly predicted coefficient symbols is divided into a plurality of subsets, wherein each subset includes one or more of the set of jointly predicted coefficient symbols.

22. A video encoding apparatus comprising one or more electronic circuits or processors configured to: Receive the transform coefficients corresponding to the residual block of the current block; determining a maximum allowed number based on a decoding context associated with the current block; determining a set of jointly predicted coefficient symbols associated with a selected set of transform coefficients, wherein a total number of the set of jointly predicted coefficient symbols is equal to or less than the allowed maximum number; determining a joint sign prediction for the set of jointly predicted coefficient signs by selecting a hypothesis corresponding to achieving a minimum cost from a set of hypotheses for the set of jointly predicted coefficient signs, wherein a cost between a boundary pixel of the current block of the hypothesis and a corresponding neighboring pixel of the current block is calculated for each hypothesis in the set of hypotheses, and the boundary pixel of the current block associated with each hypothesis is reconstructed using information included in said each hypothesis; determining a sign residual between the signs of the coefficients of the set of joint predictions and the joint sign prediction; as well as Context coding is applied to the symbol residual to generate a decoded symbol residual.

Images