QUANTIZATION MATRIX PREDICTION FOR VIDEO ENCODING AND DECODING
Patent Information
- Authority / Receiving Office
- MX · MX
- Patent Type
- Patents
- Current Assignee / Owner
- INTERDIGITAL CE PATENT HOLDINGS SAS
- Filing Date
- 2022-02-11
- Publication Date
- 2026-05-19
Smart Images

Figure MX433852B0
Abstract
Description
QUANTIZATION MATRIX PREDICTION FOR VIDEO ENCODING AND DECODING FIELD OF INVENTION This disclosure involves video compression and, more specifically, a video compression quantization step. BACKGROUND OF THE INVENTION To achieve high compression efficiency, video and image coding schemes generally employ prediction and transform to address spatial and temporal redundancy in the video content. In general, intra- or inter-prediction is used to exploit intra- or inter-frame correlation. The differences between the original image block and the predicted image block, often denoted as prediction errors or prediction remainder, are then transformed, quantized, and encoded by entropy. To reconstruct the video, the compressed data is decoded using the inverse processes corresponding to prediction, transform, quantization, and entropy encoding. HEVC (High Efficiency Video Coding) is an example of a compression standard. HEVC was developed by the Joint Collaborative Team on Video Coding (JCT-VC) (see, for example, “ITU-T H.265 TELECOMMUNICATION SECTOR STANDARDIZATION (10 / 2014), H SERIES: AUDIOVISUAL AND MULTIMEDIA SYSTEMS, Audiovisual Services Infrastructure - Motion Video Coding, High Efficiency Video Coding, ITU-T H.265 Recommendation”). Another example of a compression standard is one under development by the Joint Video Expert Team (JVET) and associated with the designated Versatile Video Coding (VVC) development effort. WC is intended to provide improvements to HEVC. BRIEF DESCRIPTION OF THE INVENTION In general, at least one example of a modality may involve a method comprising: obtaining, from a bitstream that includes encoded video information, information representing at least one coefficient of a quantization matrix and a syntax element; determining, based on the syntax element, that the information representing the at least one coefficient should be interpreted as a remainder; and decoding at least a portion of the encoded video information based on a combination of a prediction of the quantization matrix and the remainder. In general, at least one example of a modality may involve apparatus comprising: One or more processors configured to obtain, from a bitstream including encoded video information, information representing at least one coefficient of a quantization matrix and a syntax element; determine, based on the syntax element, that the information representing the at least one coefficient should be interpreted as a remainder; and decode at least a portion of the encoded video information based on a combination of a prediction of the quantization matrix and the remainder. In general, at least one example of a modality may involve a method comprising: aro Lnn / zznz / E / Yi obtain video information and information representing at least one coefficient of a predicted quantization matrix associated with at least one portion of the video information; determine that the at least one coefficient is to be interpreted as a remainder; and encode the at least one portion of the video information based on a combination of the predicted quantization matrix and the remainder, and encode a syntax element indicating the at least one coefficient to be interpreted as a remainder. In general, at least one example of a modality may involve apparatus comprising: one or more processors configured to obtain video information and information representing at least one coefficient of a predicted quantization matrix associated with at least one portion of the video information; determining that the at least one coefficient is to be interpreted as a remainder; and encoding the at least one portion of the video information based on a combination of the predicted quantization matrix and the remainder, and encoding a syntax element indicating the at least one coefficient that is to be interpreted as a remainder. In general, another example of a modality may involve a bitstream formatted to include image information, wherein the image information is encoded by processing the image information based on any one or more of the modality examples of methods in accordance with this disclosure. In general, one or more of the other examples of modalities may also provide a computer-readable storage medium, for example, a non-volatile computer-readable storage medium, having instructions stored therein for encoding or decoding image information such as video data according to the methods or apparatus described herein. One or more modalities may also provide a computer-readable storage medium having a bitstream stored therein generated according to the methods or apparatus described herein. One or more modalities may also provide methods and apparatus for transmitting or receiving the bitstream generated according to the methods or apparatus described herein. Several modifications and modalities are contemplated as explained below that can provide improvements to a video encoding and / or decoding system including but not limited to one or more of increased compression efficiency and / or encoding efficiency and / or processing efficiency and / or decreased complexity. The foregoing presents a simplified summary of the subject matter to provide a basic understanding of some aspects of this disclosure. This summary is not a comprehensive overview of the subject matter. It is not intended to identify critical / key elements of the modalities or to outline the scope of the subject matter. Its sole purpose is to present some concepts of the subject matter in a simplified form as a prelude to the more detailed description provided later. BRIEF DESCRIPTION OF THE FIGURES This disclosure can be better understood by considering the detailed description below in conjunction with the accompanying figures, in which: Figure 1 shows an example of inferred zero gold transform / quantization coefficients Lnn / zznz / E / Yi for block sizes larger than 32 in VVC; Figure 2 shows a comparison of predefined intra and inter quantization matrices in H.264 (top: inter (solid) and intra (grid); bottom: inter (solid), scaled intra (grid)); Figure 3 shows a comparison of intra and inter default quantization matrices in HEVC (top: inter (solid) and intra (grid); bottom: inter (solid), scaled intra (grid)); Figure 4 illustrates an example of a quantization matrix (QM) decoding workflow; Figure 5 illustrates an example of one modality of a modified QM decoding workflow; Figure 6 illustrates another example of a modality of a modified QM decoding workflow; Figure 7 illustrates another example of a modality of a QM decoding workflow that involves prediction (e.g., copy) and a variable-length remainder; Figure 8 illustrates another example of a modality of a QM decoding workflow that involves prediction (e.g., scaling) and a variable-length remainder; Figure 9 illustrates another example of a modality of a QM decoding workflow that always involves using prediction (e.g., scaling) and a variable-length remainder; Figure 10 illustrates, in the form of a block diagram, an example of one encoder modality, e.g., video encoder, suitable for implementing various aspects, features, and modalities described herein; Figure 11 illustrates, in the form of a block diagram, an example of a decoder modality, e.g., video decoder, suitable for implementing various aspects, features, and modalities described herein; Figure 12 illustrates, in the form of a block diagram, an example of a system suitable for implementing various aspects, features, and modalities described herein; Figure 13 illustrates, in the form of a block diagram, an example of a system suitable for implementing various aspects, features, and modalities described herein; Figure 14 illustrates another example of a modality of an encoder according to the present description. It should be understood that the figures are for the purpose of illustrating examples of various aspects and modalities and are not necessarily the only possible configurations. Through the various figures, as reference designators, they refer to the same or similar characteristics. DETAILED DESCRIPTION OF THE INVENTION The HEVC specification allows the use of quantization matrices in the dequantization process, where code-frequency transformed coefficients are scaled by the current quantization step and further scaled by means of a quantization matrix (QM) as follows: gold Lnn / zznz / E / Yi d[ x ][ y ]=C1ip3( coeffMm, coeffMax, ((TransCoeffLeveif xTbY ][ yTb¥ ][ cldx ][ x ][ y ] * m[ x H yi * leveIScale| qP%6 ] « (qP / 6 )) + (1 «(bdShift - 1 )))» bdShift) where: • TransCoeffLeve[...] are the absolute values of transformed coefficients for the current block identified by its spatial coordinates xTbY, yTbY and its component index cldx. • xyy are the horizontal / vertical frequency indices. • qP is the current quantization parameter. • Multiplication by levelScale[qP%6] and left shift by (qP / 6) is equivalent to multiplication by the quantization step qStep=(levelScale[qP%6]«(qP / 6)). • m[...][...] is the two-dimensional quantization matrix. • bfShift is an additional scaling factor to account for the bit depth of the image sample. The term (1 «(bdShift-1)) serves the purpose of rounding to the nearest integer. • d[...] are the absolute values of the resulting discounted transformed coefficients The syntax used by HEVC to transmit quantization matrices is as follows: oro Lnn / zznz / E / Yi scaling feet daía() { Descriptor sizeld « 0; sizeld < 4; $izeld++) ibr( maírixld ~ 0: matrixid < 6: matrixid + ( sizeld “ 3 ) ? 3 :1 ) { sealing list prcd mede flagf sizeld |[ mtrixid ] «(IJ di Iscabag list pred mode ítagl sizeld || maírixld [) scalhig líst pred «¡atm id ddM sizeld || matnxld ] »e(v) dse ¡ HcxíCocf™ 8 coefNüsn ~ Mh# 64. í 1 «( 4 + ( sizeld « 1 )))) if( size id > 1 ) | scalíng Hst de eeef sizekl seíy) nexlCoef ~ scalísg líst de co©f mimis8| sizeld-2 ][ matrixTd]+8 fon t - 0; i < oodNpnt; i++) { sealing list delta eoef se(v) __________nextCoef - ( nexiCoef + scalíng líst delta coef 4 256) % 2S6__________ ScalingList[ sizeld ]:[ matrixld i ] -- nextCoef 1 f It can be noted that: • A different matrix is specified for each transform size (sizeld). • For a given transform size, six matrices are specified, for intra / inter coding and Y / Cb / Cr components. • A matrix can be either copied from a previously transmitted matrix of the same size, if scaling_list_pred_mode_flag is zero (the reference matrixld is obtained as matr¡xld-scal¡ng_l¡st_pred_matr¡x_¡d_delta), or copied from defaults specified in the standard (if both scaling_list_pred_mode_flag and scaling_l¡st_pred_matr¡x_¡d_delta are zero), or fully specified in DPCM coding mode, using exp-Golomb entropy coding, in upper right diagonal scan order. • For block sizes larger than 8x8, only the coefficients are transmitted to signal the quantization matrix in order to save encoded bits. The coefficients are then interpolated using zero hold (i.e., repetition), except for the DC coefficient, which is transmitted explicitly. The use of HEVC-like quantization matrices has been adopted in the VVC 5 project based on contribution JVET-N0847 (see O. Chubach, T. Toma, SC Lim, et al., “CE7-related: Quantization matrix support for VVC”, JVET-N0847, CH, March 2019). The scaling_list_data syntax has been adapted to the WC codec as shown below. gold Lnn / zznz / B / Yi hst dataí 1 i Year descriptor siskl -1; score < 7; sh'eJíH-í·) tai mainrfd ~ Ó; matrixld < fc matrtxM -h- ) ·; ίΠ * ( (( SKStld ~~ 1 ) && ( Hiatnxid % 3 01) || í( stzeM -- 6 ) && ( Hctinxld % 3 !“ ......'............................' matrix id delt^'si^eld M wtíixid J ue(v! dse { nextCoef « 8 coefNnsi~ MiiX 64.( i « ( stald « 1))) siwtci > 3) scatag iiw d« «wf - 4 H taalrixid l ..»SdingÍ lis.............. iK smld -4 y atairixld |i < coelKwn; s+j- ) fx - DiagSss»Ordeif 3 ) | 3 jj ij μη v - IJjaftS&MiOitfeíf 3 y 3 H i |[ lj if («(sízeld^ && && y>4) ) f «ealí^ list detíá éwef ..^1........... nexiCoef - (list «íef+256 í% 256 ΙΠ1 ~ «exlCoef I 1 1. Compared to HEVC, WC requires more quantization arrays due to a higher number of block sizes. In the VVC 5 project (with adoption of JVET-N0847), as in HEVC, a QM is identified by means of two parameters, matrixld and sizeld. This is illustrated in the following two tables. Table 1: Block size identifier (JVET-N0847) gold ιηη / ζζηζ / Β / γι _ Lama | _ Sizel chroma 0 j 2x2 1 4x4 | 4x4 2 8x8 i 3x8 I 16x16 16x16 4 32x32 j 32x32 5 64x64 ] 6 Table 2: QM type identifier (JVET-N0847) CuPredMode cldx (Color Component) matrixld MODE INTRA o (Y) 0 MODE INTRA I (Cb) 1 MODE INTRA 2 (Cr) 2 MODE INTER 0 (Y) 3 MODE INTER i (Cb) 4 MODE INTER 2 (Cr) 5 NOTE: QMs MODEJNTER are also used for MODEJBC (Intra Block Copy). The combination of both identifiers is shown in the following table: Table 3: (matrixld,sizeld) combinations (JVET-N0847) INTRA ¥ 0.2 1 0.3 0.1 0.5 0.6 Cb i, i 1.2 | 1.3 1.4 L 5 Cr 2.1 2.2 1 2.3 2.4 2.5 INTER ¥ 3.2 | 3.3 3.4 3.5 3.6 Cb 4.1 4.2 | 4.3 4.4 4.5 Cr 5.1 5.2 | 5.3 5.4 and > Max block size (width, height) 2 4 i 8 16 32 64 Signaled QM coefficients 2x2 4x4 i 8x8 8x8 + DC As in HEVC, for block sizes larger than 8x8, only the 8x8 + DC coefficients are transmitted. The correct-sized QM is reconstructed using zero-retention interpolation. For example, for a 16x16 block, each coefficient is repeated twice in both directions, then the DC coefficient is replaced by the transmitted one. For rectangular blocks, the size retained for the QM selection (sizeld) is the largest dimension, i.e., maximum width and height. For example, for a 4x16 block, a QM is selected for a block size of 16x16. Then, the reconstructed 16x16 matrix is vertically decimated by a factor of 4 to obtain the final 4x16 quantization matrix (i.e., three out of four rows are skipped). For the following, we refer to the QMs for a given family of block sizes (square or rectangular) as size-N, in relation to sizeld and the square block size used for: for example, for block sizes of 16x16 or 16x4, the QMs are identified as size-16 (sizeld 4 in Draft 5 WC). The size-N notation is used to differentiate from the exact block shape, and from the number of signaled QM coefficients (limited to 8x8, as shown in Table 3). Furthermore, in VVC Project 5, for size 64, the coefficients for the lower right quadrant are not transmitted (they are inferred to be 0, which is referred to as “null” below). This is implemented using the condition “x>=4 && y>=4” in the scaling_list_data syntax. This avoids transmitting QM coefficients that are never used by the quantization / transformation process. In fact, in VVC, for transform block sizes larger than 32 in any dimension (64Xn, Nx64, with N <=64), any transformed coefficient with a frequency coordinate x / y greater than or equal to 32 is not transmitted and is inferred to be zero. Consequently, no quantization matrix coefficient is needed for quantization. This is illustrated in Figure 1, where the shaded area corresponds to transform coefficients inferred to be zero. In general, one aspect of at least one modality described herein may involve saving bits in the transmission of custom QMs, while keeping the specification as simple as possible. In general, another aspect of this disclosure may involve improving QM prediction so that simple adjustments are possible at low bit cost. For example, two common adjustments applied to QM are: -control of the dynamic range (ratio between lowest value - generally for low frequencies, and highest value - generally for high frequencies), which is easily achieved with global scaling of matrix coefficients around a neutral value of choice; and - a general trade-off either to fine-tune the bit rate, extend the QP range, or balance the quality between different transform sizes. An example is the default 8x8 “intra” and “inter” matrices for h264 illustrated in Figure 2. Figure 2 shows a comparison of the default h264 intra and inter matrices. The top illustration in Figure 2 shows inter (solid) and intra (grid). The bottom illustration in Figure 2 shows inter (solid) and scaled intra (grid). In Figure 2, the “inter” matrix is simply a scaled version of the “intra” matrix, where “scaled intra” is (intra_QM-16)*11 / 16+17. A similar trend is observed for 8x8 HEVC default arrays, as illustrated in Figure 3. Figure 3 shows a comparison of intra and inter default h264 arrays. The upper illustration in Figure 2 shows inter (solid) and intra scaled (grid) where “intra gold Lnn / zznz / E / Yi scaled” is (intra_QM-16)*12 / 16+16). These simple adjustments require full matrix registration in HEVC and VVC, which can be expensive (e.g., 240 bits to encode the scaled intra-HEVC matrix shown above). Also, in both HEVC and WC, the prediction (=copy) and explicit QM coefficients are mutually exclusive. That is, it is not possible to refine the prediction. In the WC context, modifications to QM syntax and prediction have been proposed. For example, in one proposal (JVET-O0223), QMs are transmitted from largest to smallest order, and prediction is allowed for all previously transmitted QMs, including those intended for larger block sizes. Prediction here means copying, or decimating if the reference is larger. This takes advantage of the similarity between QMs of the same type (Y / Cb / Cr, lntra / lnter) but intended for different block sizes. The QM index matrixld included in the described proposal is a composite of a size identifier sizeld (shown in Table 4) and a type identifier matrixTypeld (shown in Table 5), which are combined using the following equation, resulting in the single identifier matrixld shown in Table 6. matr¡xld=6*s¡zeld+matrixTypeld(3-1) Table 4: Size identifier (JVET-O0223) gold Lnn / zznz / E / Yi Croma plank 64x64 32x32 0 32x32 16x16 1 16x16 8x8 2 8x8 4x4 3 4x4 2x2 4 Table 5: Array type identifier (JVET-O0223) CuPredMode ddx (Color Component) mauixTypetd MODE INTRA 0 (Y) 0 MODEINTER 0 íY) 1 MODE INTRA 1 (Cb) 2 MODE ...INTER 1 (Cb) 3 MODE INTRA 2 (Cn 4 MODE INTER 2 (Cr) 5 Table 6: Unified matrixld (JVET.00223) j INTRÁ (1 ; 6 : i 12 ; 18 ¡ 24 : Ά INTER 1 : 7 : i 19 ! 25 : j INTRA ; *> > 8 ji 14 ; 20 : 26 vl> :........................................... INTER ; 3 9 j 15 21 ; 27 ÍNTRA : 4 ; 10 j 16 ; 22 : 28 ÍNTER : 5 : 11 ir í 17 ; 23 : 29 TU size: luma max (width,height) 64 32 j 16 8 4 Block size: max(width,height) 64 j 32 16 8 | 4 2 QM size marked 8x8+ »C 8x8 1 4x4 2x2 gold Lnn / zznz / E / Yi And the syntax is modified as indicated by highlighting it in the following: scantiií >st date! { Descriptor 1 fu; 6 '! ) í seaiíttg list pred mode fUtil i tf (Iscaling list fj<.d nwdc matrixld 11 xcalin« list pml ntaüix id deíta| imrtwUd | ite(v) j else { I nextCod - 8 coefNum u&HnxM < 20j'* ύ 1, umunxld'. 2*4 7 ih, 4 if < pKtfn.\{d The QM decoding workflow is illustrated in Figure 4. In Figure 4: -Input is the encoded bit stream. -Output is a set of ScalingMatrix. -“Decode QM prediction mode”, obtain a bitstream prediction indicator. -“Is it predicted?”: determines whether the QM is inferred (predicted) or signaled in the bit stream, depending on the aforementioned indicator. -“Decode QM prediction data”: obtain prediction data from the bitstream, necessary to infer QM when not signaled, for example a QM index difference scaling_list_pred_matrix_id_delta - “is predetermined”: determines whether the QM is predicted from predetermined values (e.g., if scaling_list_pred_matrix_id_delta is zero), or from the previously decoded QM - “Reference QM is a specific QM”: Selects a specific QM as the reference QM. There can be several specific QMs to choose from, for example, depending on the matrixld pandad. - “Obtains reference QM”: Selects a previously decoded QM as the reference QM. The reference QM index is derived from matrixld and the aforementioned index difference. -“Copy or scale down the reference QM”; predicts the QM of the reference QM. The prediction comprises a simple copy if the reference QM is the correct size, or a decimation if it is larger than expected. The result is stored in ScalingMatrix[matrixld]. -“Get number of coefficients.”: determines the number of QM coefficients to be decoded from the bit stream, depending on matrixld. For example: 64 if matrixld is less than 20, 16 if matrixld is between 20 and 25, 4 otherwise. -“Decode QM coefficients”: decodes the relevant number of QM coefficients from the bit stream. -“diagonal scan”: organizes the decoded QM coefficients into a 2D matrix. The result is stored in ScalingMatrix[matr!xld] -“last QM”: either loop or stop when all QMs are decoded from the bit stream. -Details about the DC value are omitted for clarity purposes. The proposal described above (JVET-O0223) attempts to reduce the bit cost of custom-encoded QMs in the context of WC by enabling the prediction of a QM with a different size. However, the prediction is limited to copying (or decimating). In the HEVC context, JCTVC-A114 (a response to the HEVC Call for Proposal) has proposed an inter-QM prediction technique with scalability. The following syntax and semantics are found in JCTVC-A114 Appendix A, with the most relevant part shown in bold: gold Lnn / zznz / E / Yi Description! Type Coniste «¿date (Setíd, Malrixld, MaxCedfj { qscsHngmsttRx opdirte lype .ffag wnHJ Vise 50 / 50 íf( úrdate tepe fias) { bstSeak ~ 8 «extScate = 8 ιίϊ nextíkak !KO ) f delía scate W50 nmSeate - (.bstScak +· delta seat + 2S6) % 256 useDefaiihScaln^MatrixFW ® f .i88 ® $ && íiexiSaile ~ S! 0)} scaids®inMnx| j ] - (tmlScafe ~ ü) ? tetScafe : fmScsüs iastSeafe ® «alisámatrix (i 1 1 ^xtScale - (testScak + ddta scafe + 256) % 256 fj ~ - 0 && He.xtS«ite - ~ 0 ) j Use { m&tm headquarters 50 / 50 matrix slope iKHin .rae 50 / 50 ttiiai dd8a sc«fe tKSHtekiC 50 / 50 fer( j = 8; j < mtn flete seáis; )++·) j dete «cate liten > u' 50 / 50} i í gold Lnn / zznz / E / YiAi where -qscalingmatrix_update_type_flag: indicates the methods for updating the scaling matrices. A value of 0 means a new matrix is adapted from the existing matrix and a value of 1 means a new matrix is recovered from an existing matrix. -delta_scale: Indicates a delta scale value. Delta-scale values are decoded in the zigzag scan order. If not present, the default value for delta_scale is zero. -matrix_scale: specifies a parameter to scale all matrix values. -matrix_slope: indicates a parameter to adjust the gradient of a non-planar matrix. -num_delta_scale: indicates the number of delta_scale values to be decoded when qscalingmatrix_update_type_flag is equal to 0. -Depending on qscalingmatrix_update_type_flag, the ScaleMatrix scaling matrix values are defined as follows: -if qscalingmatrix_update_type_flag is equal to 0, ScaleMatr¡x[¡]=(ScaleMatr¡x[¡]*(matr¡x_scale+16)+matr¡x_slope* (ScaleMatrix[i]-16)+8)»4+delta_scale, With !=0...MaxCoeff -otherwise (qscalingmatrix_update_type_flag equals 1) ScaleMatrix[i] is derived as shown in Section 5.10. Note-ScaleMatrix[¡] is equal to 16 per determination. Although the JCTVC-A114 approach allows QM prediction with a variable length scaling and remaining factor, there are two redundant scaling factors and there is no simple way to specify a global offset (one would have to encode all delta_scale values because they are not DPCM encoded). In general, at least one aspect of this disclosure may involve one or more of extending the prediction of QMs by applying a scaling factor in addition to copying and sampling down, providing a simple way to specify an overall offset, or allowing prediction refinement with a variable length remainder. In general, at least one aspect of this disclosure may involve combining enhanced prediction with a remaining, to further refine QMs, at a potentially lower cost than full coding. In general, at least one example of a modality in accordance with this disclosure may include QM prediction with offset in addition to copy / scaling down. In general, at least one example of a modality in accordance with this disclosure may comprise QM prediction plus remaining (variable-length). In general, at least one example of a modality in accordance with this disclosure may include QM prediction with a scaling factor in addition to copying / scaling down. In general, at least one example of a modality according to this disclosure may comprise a combination of scale factor with either compensation or remainder. In general, at least one example of a modality in accordance with this disclosure can provide to reduce the number of bits needed to transmit QMs, and allow user-defined trade-off between accuracy and bit cost, while keeping the specification simple. In general, at least one example of a modality according to this disclosure may comprise one or more of the following features to provide a reduction in the number of bits needed to transmit QMs, and allow user-defined trade-off between accuracy and bit cost, while keeping the specification simple. • Add a remainder to the QM prediction • Add a scaling factor to the QM prediction • Combine remainder and scaling • Always use prediction (remove prediction mode marker) Examples of various modalities incorporating one or more of the described features and aspects are provided later. The following examples are presented in the form of syntax and text descriptions based on Project 5 VVC, including proposal JVET-O023. However, this context is merely a selected example for ease and clarity of description and does not restrict either the scope of disclosure or the scope of application of the principles described in the present document. That is, the same principles can be applied in many different contexts, as will be apparent to someone skilled in the art, for example, to HEVC or Project 5 VVC, to QMs interpreted as QP offsets instead of scaling factors, to QM coefficients encoded with 7 bits instead of 8, and so on. In general, at least one example of a modality comprises a modification to the top of the copy for QM prediction, including adding a global offset. As an example, a global offset can be explicitly specified as indicated in the following syntax and specification text modification based on Project 5 WC, including JVET-O023, where additions are highlighted in shading and deletions are struck through. gold Lnn / zznz / E / Yi | scalíng list data)) { Descriptor ij scaling l&t beep mode flagí matiixMl ti(I) ί ! if('scaliiig list pred mode fiagf awrtxld Π I list pml matrix id dehal matrixld I iietYl ¡ i clse I $£ÍV) | | nextCoef-8 5 I COefNom ~ (matrixld < 20) ? 64 ; (wtnxfd < 26) ? 16 : 4 I if (tnatóxld < 14 ) { j «haze list of coef nnmis8| matiixld ] se(v) j 1 nextCoef ~ scaling list of coef matrixld | 8 i ) | fbr( i “ 0: i < cocfNam; i++1 { i scalíng list delta cocí se(v) | An example of a modified QM decoding workflow corresponding to the modality described above is shown in Figure 5 where changes from JVET-O023 are in bold text and thick outline. Alternatively, a global offset can be specified as the first coefficient of a DPCM-coded remainder (described below). In general, at least one example of a modality involves adding a remainder to the prediction. For example, it may be desirable to transmit a remainder on top of the QM prediction to refine the prediction or to provide an optimized coding scheme for QMs (prediction plus remainder instead of direct coding). The syntax used to transmit QM coefficients when they are not predicted can be reused to transmit a remainder when QM is predicted. Furthermore, the number of remainder coefficients can vary. When these coefficients are DPCM-coded, as in HEVC, WC, and JVET-O023, and the missing coefficients are inferred to be zero, this means that the remainder carryover coefficients are repetitions of the last transmitted remainder coefficient. Transmitting a single remainder coefficient is then equivalent to applying a constant offset to the entire QM. In general, at least one variant of the described modality, including adding a remainder, may further include the use of an indicator in prediction mode to show that the coefficients can still be transmitted as non-prediction mode but interpreted as remainders. When not in prediction mode, to unify the design, the coefficients can still be interpreted as remainders on top of a prediction of all 8 QMs (the value 8 is an imitation of HEVC and WC behavior but is not limited to that). An example of syntax and semantics based on JVET-O0223 is illustrated later. gold Lnn / zznz / E / Yi scaling list dataf) { Destripan· | forf matrixld ~ 0; matrixld < 30; iiiatrixld+ r ) { scaling lisi prcd mude Oagf matrixld | u{0 1 if (Scaling iist pred asede flagf matrixld | ) ^ vcalinj» iÍM pred matrix id delta| enter | ue(Vi seslñij» Irit pmm Üag utn 1 d híd ceef (U# tj .nextCW - g coeíNiita ~ (matrixld. < 20) ? 64 ' (matrixld < 26}) ? 16 : 4 ií'( matrixld 1 of scalin < 14) nexiCoef- scaling lísi de cosí-mi wtts&j matrixld 1 *4 fort i ~ 0; _____________________________________________________________________________________ ________________________________________________________________________________________________________ 1 ! and Ϊ Scaling_list_pred_mode_flag[matrixld] equal to 0 specifies that the scaling matrix is specified by scaling_list_pred_matrix_id_delta[matrixld]. Scaling_list_pred_mode_flag[matrixld] equal to 1 specifies that the scaling list values are explicitly flagged. When scaling„list„pred_modo„ftag[mafrixld] is equal to L all values of the set (matrixSfze)x(matríxSíze) predScalirtgMafríxf matrixld ] and the value of the variable predSoaiingDCf matrixld ] are inferred to be equal to 8. Scal¡ng_l¡st_pred_matr¡x_¡d_delta[ matrixld ] specifies the reference scaling matrix used to derive the predicted scaling matrix, as follows. The value of scal¡ngl¡st_pred_matr¡x_¡d_delta[ matrixld ] will be in the range of 0 to matrixld, inclusive. When scaling_list_pred_mode_flag[ matrixld ] is equal to zero: -The refMatrixSize variable and the refScalingMatrix set are first derived as follows: If scalingjist_pred_matrix_id_delta[ matrixld ] is equal to zero, the following applies to setting default values: refMatrixSize is set to 8, if matrixld is an even number, refScalingMatrix= (6-1) { {16, 16, 16, 16, 16, 16, 16, 16} / / bookmark for default values INTRA {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16} otherwise refScalingMatrix= (6-2) { {16, 16, 16, 16, 16, 16, 16, 16} / / marker for default values INTER {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16} 16, 16} {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16}}, or otherwise (if scaling_list_pred_matrix_id_delta[ matrixld ] is greater than zero), the following applies: refMatrixld=matr¡xld-scal¡ng_l¡st_pred_matrix_¡d_delta[ matrixld ](6-3) refMatrixSize=(refMatr¡xld<20)?8:(refMatr¡xld<26)?4:2(6-4) refScalingMatr¡x=ScalingMatr¡x[ refMatrixId ](6-5) -The set Ü®Écal¡ngMatr¡x[ matrixld ] is then derived as follows: Matnxld ss and «(iag2(mfMWixSize) - tog2( matrixSize)) hoop Lnn / zznz / B / Yi -When matrixld is less than 14, if scalingjist_pred_matrixjd_delta[ matrixld ] is equal to zero, the variable predScalingDC[ matrixld ] is set equal to 16, otherwise the ScalingDCj refMatrixId ] is set equal to 16. ScalingJísLooefjiresenLflag equal to 1 specifies that the coefficients in the scaling list are passed on, and interpreted as a remainder that will be added to the predicted quantization matrix. scalingjist_coef_presení_fíag equal to ao specifies that no remainder is added. When not present, scalingjist_coef_presént_flag is inferred to be equal to 1. scalinglistdccoef^nmusBI matrixld ] plus 8 is used to derive ScalingDCjmatrixld], the MI specifies the first value of the scaling matrix when relevant, as described in clause xxx. The value of scaling_list_dc_coef_minus8[ matrixld ] will be in the range of -128 to 127, inclusive. When not present, the value of sMngjístjlc_coef[ matrixld ] is inferred to be equal to 0 When matrixld is less than 14, the variable ScalingDC[ matrixld j is derived as follows: ScalingDC[ matrixld j“(predScalingDC| matrixld ]+scalingJisWc_coefi matrixld >256)%256 iliillie When scaling_list_pred_mode_flag[matrixld] is equal to zero, scaling_list_pred_matrix_id_delta[matrixld] is greater than zero, and refMatrixld < 14, then the following applies: If matrixld < 14, scaling_list_dc_coef_minus8[ matrixld ] is inferred to be equal to scaling_list_dc_coef_minus8[ refMatrixId ], Be—eirá—manera,—ScalingMatrix[—matrixld—H—θ—H—0—]—se—establece—igual—a scaling_list_dc_coef_minus8[ refMatrixId ]+8 When scaling_list_pred_mode_flag[matrixld] is equal to zero (indicating default values), and matrixld < 14, then scaling_list_dc_coef_minus8[matrixld] is inferred to be equal to 8. Scaling_list_delta_coef specifies the difference between the current array coefficient ScalingList[matrixld][i] and the previous array coefficient ScalingList[matrixld][i-1], when $Galing_|i$t_coef_present_flag scaling_list_pred_mode_flag[matrixld] is equal to 1. The value of scaling_list_delta_coef will be in the range of -128 to 127, inclusive. The value of ScalingList[matrixld][i] will be greater than 0. When scaling_list_coef_presení_fíag is equal to 0, all values of the set SeaímgLístj matrixld are inferred to be equal to zero. The carpanta (mafríSiz^^ matrixld ] is derived as follows: SGalingMairtx[i]|]]=(predScalingMatrix[í][j]+Scal¡ngL¡st[ matrixld ][k]+256)%256 (6-8) with k=0...coefNum-1 i=diagScanOrder[ log2(coefNum) / 2][log2(coefNum) / 2][k][0], andj=diagScanOrder[ log2(coefNum) / 2][log2(coefNum) / 2][k][1 ] Furthermore, in the section on “scaling matrix derivation process”, the text is changed as follows: If matrixld is less than 14, m[0][0] is further modified as follows: m [0] [0]=Scal tngDC [matnxld]scaling_list_dc_coef_minus8[matrixld]+8 (6-9) An example of a QM decoding workflow corresponding to the preceding modality is shown in Figure 6 where the changes in JVET-O0223 are in bold text and thick outline. In general, at least one example of a modality can involve the use of a variable-length remainder on the prediction. A variable-length remainder can be useful because, for example: • Low-frequency coefficients, which are transmitted first due to diagonal scanning, have the greatest impact. • Because the coefficients are DPCM-encoded, if the missing delta coefficients are inferred to be zero, this results in the last remaining coefficient being repeated. This means that the last transmitted remainder is reused for the portion of QM where no remainder is transmitted. In this way, transmitting a single remainder coefficient is equivalent to applying compensation to the entire QM, without using any additional specific syntax. The number of remaining coefficients can be indicated by, for example: • An explicit number of coefficients, either encoded in exp-golomb encoding or fixed-length encoding (with length depending on matrixld), or • An indicator showing the presence of a remainder, followed by the number of remaining coefficients less 1, which may be restricted to powers of 2, or equivalently, 1 + 1 / 2 of the number of remaining coefficients, with zero indicating no remainder. This may be encoded as fixed length (length potentially depending on matrixld) or exp-golomb encoding, or • An index in a list, where the list contains numbers of coefficients. It is recommended that this list be fixed by means of the standard (it need not be transmitted), for example {0, 1, 2, 4, 8, 16, 32, 64} The number of remaining coefficients does not need to take into account the potential extra DC remaining coefficient. This extra DC remaining coefficient can be transmitted as soon as at least one regular remaining coefficient is sent. An example of syntax and semantics, based on the immediately preceding exemplary modality, is provided further on. gold Lnn / zznz / E / Yi scalúiK bst daiat j J Descriptor fon nulnvíd b nrumíd - · í : ΐΒΜηχΛίΛ' Mitrad '' K, (iwntmhf c 3*»? 1 ntíKCWMiní ™ tnslmSíl 0 ………………………………………………………………….. uc(vi ewtfN ................' .......... ......................................................................................... aexOef “ i if (niatridd < 14 && > > (D ( scsling list of eWi Bwtnxíd s se(v€) ncíjmCo ] í' Ion ! - ú í ΒκηίκΓΜπ», i ® J { «xtong tist delta enri sri v) u£«Ceef= («exóef+scahng list delta coef-f-256)%256 Sctifoxild}Hsí matri. `Scaling_list_pred_mode_flag[ matrixld ]` equal to 0 specifies that the scaling matrix is derived from the values of a reference scaling matrix. The reference scaling matrix is specified by means of `scaling_list_pred_matr¡x_¡d_delta[ matrixld ].` `Scaling_list_pred_mode_flag[ matrixld ]` equal to 1 specifies that the values in the scaling list are explicitly flagged. When scaling_list_pred_mode_flag[ matrixld ] is equal to 1, all values in the matrixSize x matrixSize set predScalingMatrix[ matrixld ] and the value of the variable predScalingDC[ matrixld ] are inferred to be equal to 8. Scaling_list_pred_matr¡x_¡d_delta[ matrixld ] specifies the reference scaling matrix used to derive the predicted scaling matrix, as follows. The value of scaling_list_pred_matr¡x_¡d_delta[ matrixld ] will be in the range of 0 to matrixld, inclusive. When scaling_list_pred_mode_flag[ matrixld ] is equal to zero: -The refMatrixSize variable and the refScalingMatrix set are first derived as follows: If scal¡ng_l¡st_pred_matr¡x_¡d_delta[ matrixld ] is equal to zero, the following applies to setting default values: refMatrixSize is set to 8, if matrixld is an even number, refScalingMatrix= (6-10) { {16, 16, 16, 16, 16, 16, 16, 16} / / bookmark for default values INTRA {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16} otherwise refScalingMatrix= (6-11) { {16, 16, 16, 16, 16, 16, 16, 16} / / marker for default values INTER {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16}}, or otherwise (if scaling_list_pred_matr¡x_id_delta[ matrixld ] is greater than zero), the following applies: refMatrixld=matr¡xld-scal¡ng_list_pred_matr¡x_¡d_delta[ matrixld ] (6-12) refMatrixS¡ze=(refMatrixld<20)?8:(refMatrixld<26)?4:2) (6-13) refScalingMatrix=ScalingMatrix[ refMatrixId ] (6-14) -The set predScalingMatrix[ matrixld ] is then derived as follows: wnxM | * Hj í with r- x *' i * κ vi -¾ ' logJyvtAtwixSizel - 1i gold Lnn / zznz / E / Yi -when matrixld<14, if scaling_list_pred_matrix_d_delta[ matrix Id ] is equal to zero, the variable predScalingDC[ matrixld ] is set equal to 16, otherwise it is set equal to ScalingDC[ refMatrixId ]. coefNum specifies the number of remaining coefficients that are passed on. The value of coefNum will be in the range of 0 to maxCoefNum, inclusive. scaling_list_coef_present_flag equal to 1 specifies that the coefficients of the scaling list are passed on and interpreted as a remainder to be added to the predicted quantization matrix. scaling_list_coef_present_flag equal to 0 specifies that no remainder is added. When not present, scaling_list_coef_present_flag is inferred to be equal to 1. Scaling_list_dc_coef[ matrixld ] is used to derive ScalingDc[ matrixld ], which specifies the first value of the scaling matrix when relevant, as described in clause xxx. The value of scaling_list_dc_coef[ matrixld ] will be in the range of -128 to 127, inclusive. When not present, the value of scaling_list_dc_coef[ matrixld ] is inferred to be 0. When matrixld is less than 14, the variable ScalingDC[ matrixld ] is derived as follows: ScalingDC[ matrixld ]=(predScalingDC[ matrixld ]+ scaling_list_dc_coef[ matrixld ]+256)%256 (6-16) Scaling_list_delta_coef specifies the difference between the current matrix coefficient ScalingList[ matrixld ][!] and the previous matrix coefficient ScalingList[ matrixld ][!-1], for ( between 0 and CoefNum-1, inclusive. For ! between CoefNum and maxCoefNum -1 inclusive, scaling_list_delta_coef is inferred to be equal to 0 when scaling_list_coef_present_flag is equal to 1. The value of scaling_list_delta_coef will be in the range of -128 to 127, inclusive. When scaling_list_coef_present_flag is equal to 0, all values in the SoalingList[ matrixld ] set are inferred to be equal to zero. The MatrixScaling of the set (matrixSize)x(matrixSize) [matrixld] is derived as follows: ScalúlgMatrix | i lí i ] i pred^alíngMatrñí i ][ j ] ' S^íihiigListj matrixld j| k ] + 256 I % 256 (617) with k:::O iuaxGxTKwn - 1, i “ diagSc.inOnier| [| ki[0 ]. yj = diagYes.niurdvrj || kj| 1 ] An example of a QM decoding workflow is shown in Figure 7. The modality example shown in Figure 7 involves: prediction=copy; variable-length remainder. In Figure 7, the modality changes of the example illustrated in Figure 6 are in bold and thick outline. In general, at least one example of a modality involves adding a scale factor to the prediction. For example, the prediction of QMs can be improved by adding a scale factor to the copy / tenth. When combined with the previous modality example, this allows for two common adjustments made to QMs—scaling and offset, as explained above—and also permits user-defined trade-offs between accuracy and bit cost with a variable-length remainder. The scale factor can be an integer, normalized to a number N (e.g., 16). The scaling formula can scale the reference QM around a neutral value of choice (e.g., 16 for HEVC, VVC, or JVET-O0223; but which would be zero if QMs are interpreted as gold Lnn / zznz / E / Yi QP offsets). An example formula with N=(1" offset)=16 is shown below: ScaledQM=((refQM-16)*scale+rnd)+16 (6-18) Where: • shiñ=4 and rnd=8, that is N=(1«4)=16 • refQM is the input QM that will be scaled • scale is the scaling factor • scaledQM is the scaled output QM The scaledQM result must also be trimmed to the QM range (e.g., 0...255 or 1.255). The scaling factor can be transmitted as an offset to N (in the example above, the transmitted value would be: scale-("offset")), so that the value 0 represents a neutral value (no scaling) with a minimum bit cost: 1 bit when encoded as exp-golomb. Furthermore, the value can be limited to a practical range: for example, a number between -32 and 32, i.e., a scale between -1 and +3, or a number between -16 and 16, i.e., a scale between 0 and +2 (thus limiting the scale to positive values). An example syntax is shown below. Note that the scale factor is not necessarily combined with an offset or remainder; this can be applied alone or with JVET00223, VVC, or HEVC. gold Lnn / zznz / E / Yi scaimg fet data( j { Fon descriptor miatnxid = 0: mstñsdd < 30: mírtnxW+) { matrixSuíe «(minxld < 20) ? 8 : (mythM < 26) ? 4 : 2 mxCoefNum ® i«rixSize * íaatrixSize sralmg List pred asede fbuj sunuxEd j »U) if< iscaW list pted nwds ñajd niairixM 1) ¡ walin» Ksí pivd maim id ddtaj muinvld i im W) enefNtt mxCodNmn nexOef « 0 if (matmld < 14 && codNum > 0) { search iist of coefl müixid 1 cm) «extCoef = sciditix list of coefl insitródd 1} íbif i ~ 0; i < KiíixCoefNaim.: M { id i < coefÑnm ) ( smíing iot deha ewf sen) nextCoef = (imlC <hrf>scalw Jist delta eoef + 256 ) H 256} ScaliagListf mtródd Π i | - «extCoef i i__________________________________________________________________________________________________________________________________________________________________________________________________________ 1 `scaling_list_pred_mode_flag[ matrixld ] equal to 0` specifies that the scaling matrix is derived from the values of a reference scaling matrix. The reference scaling matrix is specified by means of `scaling_list_pred_matr¡x_¡d_delta[ matrixld ].` `scaling_list_pred_mode_flag[ matrixld ] equal to 1` specifies that the values in the scaling list are explicitly flagged. When scaling_list_pred_mode_flag[ matrixld ] is equal to 1, all values in the matrixSize x matrixSize set predScalingMatrix[ matrixld ] and the value of the variable predScalingDC[ matrixld ] are inferred to be equal to 8. Scaling_list_pred_matrix_id_delta[ matrixld ] specifies the reference scaling matrix used to derive the predicted scaling matrix, as follows. The value of scaling_list_pred_matrix_id_delta[ matrixld ] will be in the range of 0 to matrixld, inclusive. scaling_list_pred_minus16[ matrixld ] plus 16 specifies the scaling factor to be applied to the reference scaling matrix used to derive the predicted scaling matrix, as follows. The value of scaling_list_pred_scaling_minus16[ matrixld ] will be in the range of -16 to +16, inclusive. When scaling_list_pred_mode_flag[ matrixld ] is equal to zero: -The refMatrixSize variable and the refScalingMatrix set are first derived as follows: If scal¡ng_l¡st_pred_matr¡x_¡d_delta[ matrixld ] is equal to zero, the following applies to setting default values: refMatrixSize is set to 8, if matrixld is an even number, refScalingMatrix= (6-19) { {16, 16, 16, 16, 16, 16, 16, 16} / / marker for default values INTRA {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16} otherwise refScalingMatrix= (6-20) { {16, 16, 16, 16, 16, 16, 16, 16} / / marker for default values INTER {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16} gold Lnn / zznz / E / Yi {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16}}, or otherwise (if scaling_list_pred_matr¡x_¡d_delta[ matrixld ] is greater than zero), the following applies: refMatrixld=matr¡xld-scal¡ng_list_pred_matr¡x_¡d_delta[ matrixld ] (6-21) refMatrixS¡ze=(refMatr¡xld<20)?8:(refMatr¡xld<26)?4:2) (6-22) refScalingMatr¡x=Scal¡ngMatrix[refMatr¡xld ] (6-23) -The set predScalingMatrix[ matrixld ] is then derived as follows: predScalingMatrix[ matrixld ][x][y]= Chp3(1.255,(((refScal ing Matnx[i][jM 6*SCale+8)»4}+16) (6-24) with x=0...matrixSize-1, y=0...matrixSize-1, i=x«(log2(refMatrixSíze)-log2( matrixSize ))yy j=y«(log2(refMatrixSize)-log2( matrixSize ))ry matrixld}+i 6 -When matrixld<14, if scal¡ng_l¡st_pred_matr¡x_¡d_delta[ matrixld ] is equal to zero, the variable predScalingDC[ matrixld ] is set equal to 16, otherwise it is set equal to ScalingDCj refMatrixid ] derived as follows: predScalingDCE matrixld Clip3( 1,255,((( ScaüngDC] refMatnxld $cale+8)»4)+16) (6-25) with scafe^scalíng matrixld 1+16 coefNum specifies the number of remaining coefficients that are passed on. The value of coefNum will be in the range 0 to maxCoefNum, inclusive. `scaling_list_dc_coef[matrixld]` is used to derive `ScalingDC[matrixld]`, which specifies the first value of the scaling matrix when relevant, as described in clause xxx. The value of `scaling_list_dc_coef[matrixld]` will be in the range of -128 to 127, inclusive. When not present, the value of `scaling_list_dc_coef[matrixld]` is inferred to be 0. When matrixld is less than 14, the variable ScalingDC[ matrixld ] is derived as follows: ScalingDC[ matrixld ]=(predScalingDC[ matrixld ] + scaling_list_dc_coef[ matrixld ]+256)% (6-26) scalinq list delta coef specifies the difference between the current matrix coefficient ScalingList[ matrixld ][i] and the previous matrix coefficient ScalingList[ matrixld ][i-1], for i between 0 and CoefNum-1, inclusive. For i between CoefNum and maxCoefNum -1, inclusive, scaling_list_delta_coef is inferred to be equal to 0. The value of scaling_list_delta_coef will be in the range of -128 to 127, inclusive. The set (matr¡xS¡ze)x(matr¡xS¡ze) ScalingMatrixf matrixld ] is derived as follows: ScalingMatrix [i][j]=(predScal¡ngMatr¡x[i][j]+ScalingL¡st[matrixld][k]+256)%256 (6-27) gold Lnn / zznz / E / YiA ScalíngMatrix Γ i H j ] — (. pxedScalingMatrix{ i ][j ] + ScalñigListj .matrixld ][ k ] + 256) % 256 (6-27) with k = 0.. maxCoeíNum - 1, i = diagScaiiOrderf logíúnaírixSize) ][ Jog2(matnxÍze) ][ k ](0 ], yj - diagScanOrder[ logífmafrixSszí:) ][ log2(matnxÍze) ][ k ][ i ] gold Lnn / zznz / E / Yi An example of a modified QM decoding workflow is shown in Figure 8. The example modality shown in Figure 9 involves: prediction=scale; variable length remainder. In Figure 8, the modality changes from the example illustrated in Figure 7 are in bold text with a black outline. In general, at least one example of a mode involves removing a prediction mode flag. For example, whenever a complete remainder can be transmitted as in the mode examples described above, because the complete QM specification is possible without considering the scaling_list_pred_mode_flag flag, this flag can be removed, which simplifies the specification text with a marginal impact on bit cost. The following provides an example of syntax and semantics for the current modality based on changes to the previous modality. Descriptor scalipg fet datat, H________________ _____________________________________ for( matrixld® 0; matrixld < 30; matrixId-H-) { maxCoeíNimi ~ maUixSizo * suatrixSsze ( j uí44 sealhtg líst prcd matrix id delta | matrixld ] | nihiiLslO nuitrixld 1 eoefNsm jsg(y) | ue(v) '4'^^ $ γ. mxCoefNíim: tH-) f ift i < coeíNnm) í ( sealing íist deha eoef | se(v) aextCeef ® (ttextCoef + scaling list delta coef ·» 25fs) % 256 | 1 ScalinjdJsd matrixld ] [ i ] nextC&. .6 * 0 in scaling_list_pred_mode_flag[ matrixld ] equal to 0 specifies that the scaling matrix is derived from the values of a reference scaling matrix. The reference scaling matrix is specified by scaling_list_pred_matrix_ide_delta[—matrixld—F. Scaling_list_pred_mode_flag[ matrixld ] equal to 1 specifies that the scaling list values are explicitly flagged. When scaling_list_pred_mode_flag[ matrixld ] is equal to 1, all values in the matrixSize x matrixSize set predScalingMatrix[ matrixld ] and the value of the variable predScalingDC[ matrixld ] are inferred to be equal to 8. scaling_list_pred_matrix_id_delta[ matrixld ] specifies the reference scaling matrix used to derive the predicted scaling matrix. The value of scaling_list_pred_matrix_id_delta[ matrixld ] will be in the range of 0 to matrixld, inclusive. scaling_list_pred_scale_minus16[ matrixld ] plus 16 specifies the scale factor to be applied to the reference scaling matrix used to derive the predicted scaling matrix, as follows. The value of scaling_list_pred_scale_minus16[ matrixld ] will be in the range of -16 to +16, inclusive. When scaling_list_pred_mode_flag[ matrixld ] is equal to zero: -The refMatrixSize variable and the refScalingMatrix set are first derived as follows: If scal¡ng_l¡st_pred_matr¡x_¡d_delta[ matrixld ] is equal to zero, the following applies to setting default values: refMatrixSize is set to 8, if matrixld is an even number, refScalingMatrix= (6-28) { {16, 16, 16, 16, 16, 16, 16, 16} / / bookmark for default values INTRA {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16} otherwise refScalingMatrix= (6-29) { {16, 16, 16, 16, 16, 16, 16, 16} / / marker for default values INTER {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16} gold Lnn / zznz / E / Yi {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16} {16, 16, 16, 16, 16, 16, 16, 16}}, or otherwise (if scaling_list_pred_matr¡x_id_delta[ matrixld ] is greater than zero), the following applies: refMatrixld=matr¡xld-scal¡ng_l¡st_pred_matr¡x_¡d_delta[ matrixld ] (6-30) refMatrixS¡ze=(refMatrixld<20)?8: (refMatrixld<26)?4:2) (6-31) refScalingMatr¡x=ScalingMatr¡x[ refMatrixId ] (6-32) -The set predScalingMatrix[ matrixld ] is then derived as follows: predScalingMatrix[ matrixld ][x][y]= Clip3( 1, 255, (((refScalingMatrix) i ][ j ] - 16} * scale + 8)» 4 ) + 16 ) (6-33) with x - 0 . matrixSize - 1, and - 0 „ niaírixSize - 1. i = x « (]og2(relMatrixSize) - Jog2( matnxSize)) j - y «(kig2(refMatrixSize) - Jog2( matfixSize)), y scale = scaliiigJistjíredjscale_minusl6[ matrixld ] +· 16 -When matrild<14, if scaling_list_predjd_delta[ matrixld ] is equal to zero, the variable predScalingDC[ matrixld ] is set equal to 16, otherwise it is derived as follows: predScalingDCj matrixld j = Clip.3( 1, 255, ((( ScakagDCf refMatrixId ] - 16 ) * scale + 8) » 4 H 16) (6-34) with scale = scakaglistpredscale mimslóf matrixld] + 16 coefNum specifies the number of remaining coefficients that are passed on. The value of coefNum will be in the range 0 to maxCoefNum, inclusive. Scaling_list_dc_coef[ matrixld ] is used to derive ScalingDC[ matrixld ], which specifies the first value of the scaling matrix when relevant, as described in clause xxx. The value of scaling_list_dc_coef[ matrixld ] will be in the range of -128 to 127, inclusive. When not present, the value of scaling_list_dc_coef[ matrixld ] is inferred to be 0. When matrixld is less than 14, the variable ScalingDC[ matrixld ] is derived as follows: ScalingDC[ matrixld ]=(predScalingDC[ matrixld ]+scaling_list_dc_coef[ matrixld ]+256)%256 (6-35) scalinq list delta coef specifies the difference between the current matrix coefficient ScalingList[ matrixld ][i] and the previous matrix coefficient ScalingList[ matrixld ][i-1], for i between 0 and CoefNum-1, inclusive. For i between CoefNum and maxCoefNum -1, inclusive, scaling_list_delta_coef is inferred to be equal to 0. The value of scaling_list_delta_coef will be in the range of -128 to 127, inclusive. The set (matr¡xS¡ze)x(matr¡xS¡ze) ScalingMatrix[ matrixld ] is derived as follows: gold Lnn / zznz / E / YiA ScalmgMatrix | i ][ j ] = (predScaliBgMalnxf i ][ j ] + ScalingListj matrixld ][ k ] + 256)% 256 (6-36) with k - 0 .. maxCoefNnm - I, i ~ daygScanOrder[ log2(malrixSizc) ][ log2(matrixSize) ][ k ][ 0 ], yj = diagScanOrder) log2(ma(rixSize) ][ logZíroatrixSizc) H k |f 1 ] gold Lnn / zznz / B / Yi An example of a modified QM decoding workflow is shown in Figure 9. The example modality shown in Figure 9 involves: always use prediction; prediction=scale; variable length remainder. In Figure 9, changes to the example modality illustrated in Figure 8 include feature removals from Figure 8. At least several examples of various other approaches are described below, involving diverse contributions submitted for the 16th JVET meeting, which are considering changes to QMs, using the current WC technique described above as a reference. Most would consider changes equivalent to the technique proposed in this disclosure without altering its essence, but impacting the syntax and semantics of examples, affecting the number of Qs, QM identifier mapping, QM array size formulas, QM selection, and / or the resizing process. Examples of modalities providing adaptations to changes are detailed in the following subsections. At least one example of a mode may involve the removal of some or all of the 2x2 chroma QMs. VVC recently removed 2x2 chroma transform blocks in intra mode, therefore transmitting intra QMs from 2x2 QMs becomes useless, although the remainder may be limited to 2x3 = 6 bits (3 bits are required for the use of the default matrix signal with the syntax example described above). Removing QMs from luma 2x2 would affect the number of QMs transmitted, with a simple syntax change: scaling list dataf) { Descriptor for( matrixld = 0; matrixld < 1^30; matrixld++) { ! Yo And the identifier mapping would also be affected (ids 26 and 28 in Table 6 are canceled), which could be reflected by adjusting the matrixld selection formula in "quantization matrix derivation process" of JVET-O0223 (or the update in JVET-P0110), for example: matrixld = 6 * sizeld + malrixTypeld with log2TnWidth - Iog2( bikWidth) + (ddx > 0 ) ? Iog2( SubWidthC): 0, log2TnHeight = log2( blkFieight) + ( cldx > 0 ) ? Iog2( SubHeightC): (λ sizeld = 6 - max( iog2TuWidtlL log2TuHeight), and matrixTypeld = (2 * cldx + (predMode = - MODE INTRA ?0 : 1)) matrixld -- »matnxld > 25) ? (malnxld > 27 ? ; 2:1): 0 Removing all 2x2 QMs and disabling QMs for inter-block chroma 2x2 would be even less complex, reducing the number of QMs to 26, as shown in the table below: scalmg hsi dataí) ( Descriptor for( inairixld = 0; niatrixfd 4; niaínxíd > ) { inatrixSize - (matrixld x 20) ? 8: 4 4 I · · í gold Lnn / zznz / E / Yi And not to call the "scaling matrix derivation process" in the "scaling process for transform coefficients" of VVC, but to add that case to the exception list instead: - The set of intermediate scaling factors (nTbW)x(nTbH) m is derived as follows: - If one or more of the following conditions are true, m[x][y] are set equal to 16: - sps_scaling_list_enabled_flag is equal to 0. -transform_skip_flag[ xTbY ][ yTbY ] is equal to 1. - both nTbWy and nTbH are equal to 2 -Otherwise, m is the output of the derivation process for scaling matrix as specified in clause 8.7.1, invoked with prediction mode CuPredMode[ xTbY ][ yTbY ], color component variable cldx, block width nTbW and block height nTbH as inputs. Or alternatively, assign all 16 values to the set m[][] within the "derivation process for scaling matrix" when the block size is 2x2 or a similar condition. At least one other modality involves a larger size for luma QMs of size 64. The matrix size formula can be changed to allow, for example, 16x16 coefficients for luma QMs of size 64; an example syntax is shown below: scalínx list data() { Descriptor _______forjJnatrixM^______________________ nhitnxbne _ tmatnxld 2) 'ib tnKitiixld ' nnatnxld < 26)? 4:2 M______________________ It will be clear to an expert in the technique that refMatrixSize in the example semantics of a modality described above follows the same rules and will be changed accordingly. At least one other approach involves adding joint CbCr QMs. Block-specific QMs with remaining CbCr sets might be desirable. These could be added after regular chromaing, resulting in the new identification mapping table below: Table 7: Matrixld mapping with extra joint CbCR QMs AND INTRA 0 : 8 16 j 24 : 32 INTER 1 : S 17 25 ; 33 Cb INTRA: 2 10 ; 18 : 26 34 INTER ' 3 11 : 19 27 35 Cr INTRA : 4 12 : 20 i 28 36 INTER i 5 13 : 21 : 29 37 Mi INTRA j | 14 22 j 30 38 INTER ii 15 23 Yes, 39 TU Size: Sum maxwidth,height) 64 32 16 8 4 Block Size: n QM Size lax (width,height) 64 j 32 16 8 4 2 ized 8x8 + DC 8x8 4x4 2x2 gold Lnn / zznz / E / YiAi This affects the number of QMs, matrixSize formula, DC coefficient presence condition (moving from <14 to <18) and matrixld formula in the QM selection process, as shown below: scalinfi 1ist data() {__________________________________________________ ton. itidmUd ¡Lnulnxld nuimld1' ){ Descriptor matrixSize = (matrixld < 269) ? 8 : (matrixld - >426) ? 4; 2 1 > · ] if (matrixld < Í844 && coefNum > 0) { 1-1 matrixld = Í& * sizeld + matrixiypeld with log2YourWídth = Iog2( blkWidíh) + (cldx > 0 ) ? Iog2( SubWidthC ): 0, log2TuHeight = Iog2( blkHeight) + (cldx > 0) ? Iog2( SubHeightC ): Or, sizeld ~ 6 - max( log2YourWídth, log2T«Heighl), and matrixTypeld = (2 ♦ cldx + (predMode ~ ~ MODE INTRA ? 0 : 1)) Other semantically related places are affected (refMatrixSize, etc.) with corresponding changes that will be clear to an expert in the technique. At least one other modality involves adding QMs for LNFST. LNFST is a secondary transform used in WC that changes the spatial frequency meaning of the final transform coefficients, thus affecting the meaning of the QM coefficients as well. Specific QMs may be desirable. Because the LNFST output is limited to 4x4 and has two alternating transform cores, these QMs could be added as a dummy TU size inserted before or after the regular size 4 (keeping QMs in descending order of size), as shown in the following table. Because LNFST is for INTRA mode only, the INTRA / INTER lines would be used to differentiate the core transform. Table 6: Matrixld mapping with extra QMs for LNFST AND INTRA 0 6 12 18 24 j 30 INTER 1 7 13 19 75 O 31 Cb INTRA 2 8 14 20 26 j 32 INTER 3 9 15 21 27 | 33 Cr INTRA 4 10 1S 22 2S ! 34 _______.· INTER 5 11 17 23 29 O 35 TU size: max (width, height) 64 32 16 8 j 4 Block size: max (width, height) 64 32 16 8 4 MVJI i O * 2 Signaled QM size 8x8 + DC 8x8 4x4 2x2 gold Lnn / zznz / E / YiAi The syntax and semantics will need to be modified accordingly, as in the previous examples, to reflect the new matrixld mapping: number of QMs, matrixSize, DC coefficient condition, etc. The derivation of matrixld in "quantization matrix derivation process" to select the correct QM should also be updated, either by a tabular description as above, or by altering equations / pseudo-code as in the following example: if(tfnstJdx[ xTbY H YtBy ]E"0 matnxld"24+2^ xTbY ][ yTbY ] also${ matrixld=[...] s((matnxld>23) matrixld+=6 Ϊ At least one other method involves handling QMs with LNFST by forcing the use of 4x4 QMs. This can be implemented by forcing specific matrixlds in the "quantization matrix derivation process" when LNFST is used for the current block, for example: $¡(|fnst_ídx[ xTbY ][ yTbY ] !=0) matnxld";20+((qídx"^ xTbY ][ yTbY ] also matrixld=[...] This request describes a variety of aspects, including tools, features, modalities, models, approaches, etc. For example, the various aspects, etc., include, but are not limited to: - save bits in the transmission of custom QMs, while keeping the specification as simple as possible; - improve QM prediction, so that simple adjustments are possible with low bit cost; - provide a simple way to specify an overall compensation; - one or more ways to extend the prediction of QMs by applying a scaling factor in addition to copying and sampling down, provide a simple way to specify an overall offset, and allow refining the prediction with a variable-length remainder; - combine enhanced prediction with a remainder, to further refine QMs, at a potentially lower cost than full coding; - allow QM prediction with compensation in addition to copying / scaling down; - allow prediction of more remaining QM (variable length); - allow QM prediction with scaling factor in addition to copying / scaling down; - allow QM prediction including a combination of the scale factor either with compensation or remaining; - to allow reducing the number of bits needed to transmit QMs, and to allow user-defined trade-off between bit cost and reliability, while keeping the specification simple. - to allow reducing the number of bits needed to transmit QMs, and to allow user-defined trade-offs between reliability and bit cost, while keeping the specification simple based on the inclusion of one or more of the following features: or add a remainder to the QM prediction; or add a scaling factor to the QM prediction; or Combine remaining and scaling; or Always use prediction (remove prediction mode indicator). Many of these aspects are described in detail and, at least to highlight their individual characteristics, are often described in a way that might sound limiting. However, this is for the sake of clarity and does not restrict the application or scope of these aspects. In fact, all the different aspects can be combined and interchanged to provide additional perspectives. Furthermore, aspects can be combined and interchanged with aspects described in previous presentations as well. The aspects described and contemplated in this application can be implemented in many different ways. Figures 10, 11, and 12 below provide some examples, but other approaches are contemplated, and the discussion in Figures 10, 11, and 12 does not limit the scope of implementations. At least one aspect generally relates to video encoding and decoding, and at least one other aspect generally relates to transmitting a generated or encoded bitstream. These and other aspects can be implemented as a method, a device, a computer-readable storage medium that has instructions stored therein for encoding or decoding video data according to any of the methods described, and / or a computer-readable storage medium that has stored therein a bitstream generated according to any of the methods described. In this application, the terms "reconstructed" and "decoded" may be used interchangeably, the terms "pixel" and "sample" may be used interchangeably, and the terms "image", "photograph" and "frame" may be used interchangeably. gold Lnn / zznz / E / Yi Several methods are described herein, and each method comprises one or more steps or actions to achieve the described outcome. Unless a specific order of steps or actions is required for the proper operation of the method, the order and / or use of specific steps and / or actions may be modified or combined. Several methods and other aspects described in this application can be used to modify modules, for example, the quantization and inverse quantization modules (130, 140, 240), of a video encoder 100 and decoder 200 as shown in Figures 10 and 11. Furthermore, these aspects are not limited to VVC or HEVC and can be applied, for example, to other standards and recommendations, whether pre-existing or developed in the future, and extensions of any such standards and recommendations (including VVC and HEVC). Unless otherwise stated or technically excluded, the aspects described in this application may be used individually or in combination. Several numerical values are used in this application, for example, the maximum quantization array size and the number of block sizes considered. These specific values are for illustrative purposes only, and the aspects described are not limited to them. Figure 10 illustrates an encoder 100. Variations of this encoder 100 are contemplated, but the encoder 100 is described later for clarity purposes without describing all expected variations. Before being encoded, the video sequence may undergo pre-coding processing (101), for example, by applying a color transform to the input color image (e.g., converting RGB 4:4:4 to YCbCr 4:2:0), or by remapping the input image components to obtain a signal distribution more resistant to compression (e.g., by using histogram equalization of one of the color components). Metadata may be associated with the pre-processing and attached to the bitstream. In the encoder 100, an image is encoded by means of encoder elements as described below. The image to be encoded is divided (102) and processed into units of, for example, CUs. Each unit is encoded using, for example, either an intra or inter mode. When a unit is encoded in an intra mode, it performs intra prediction (160). In an inter mode, motion estimation (175) and compensation (170) are performed. The encoder decides (105) which mode, intra or inter, to use to encode the unit and indicates the intra / inter decision by means of, for example, a prediction mode indicator. The remaining predictions are calculated, for example, by subtracting (110) the predicted block from the original image block. The remaining prediction values are then transformed (125) and quantized (130). The quantized transform coefficients, as well as the movement vectors and other syntax elements, are encoded by entropy (145) to produce a bitstream. The encoder can skip the transform and apply quantization directly to the remaining untransformed signal. Alternatively, the encoder can derive both the transform and quantization; that is, the remainder is encoded directly without applying either the transform or quantization processes. gold Lnn / zznz / E / Yi The encoder decodes an encoded block to provide a reference for further predictions. The quantized transform coefficients are dequantized (140) and inversely transformed (150) to decode the remaining predictions. By combining (155) the decoded predictions and the predicted block, an image block is reconstructed. Loop filters (165) are applied to the reconstructed image to perform, for example, SAO (Sample Adaptive Compensation) unblocking / filtering to reduce encoding artifacts. The filtered image is stored in a reference image buffer memory (180). Figure 11 illustrates a block diagram of a 200 video decoder. In the 200 decoder, a bit stream is decoded by the decoding elements as described later. The 200 video decoder typically performs a decoding pass that is the reciprocal of the encoding pass, as described in Figure 10. The 100 encoder also typically performs video decoding as part of the video data encoding. In particular, the decoder input includes a video bitstream, which can be generated by the video encoder 100. The bitstream is first entropy-decoded (230) to obtain transform coefficients, motion vectors, and other encoded information. The image splitting information indicates how the image is split. The decoder can then split (235) the image according to the decoded image splitting information. The transform coefficients are dequantized (240) and inversely transformed (250) to decode the prediction remainder. By combining (255) the decoded prediction remainder and the predicted block, an image block is reconstructed. The predicted block can be obtained (270) from intra-prediction (260) or motion-compensated prediction (i.e., inter-prediction) (275). Loop filters (265) are applied to the reconstructed image.The filtered image is stored in a reference image buffer memory (280). The decoded image may undergo post-decoding processing (285), for example, an inverse color transform (e.g., YCbCr 4:2:0 to RGB 4:4:4 conversion) or inverse remapping by performing the reverse of the remapping process carried out in the pre-coding processing (101). The post-decoding processing may use metadata derived in the pre-coding processing and signaled in the bitstream. Figure 12 illustrates a block diagram of an example system in which various aspects and modes are implemented. System 1000 can be exemplified as a device including the various components described later and is configured to perform one or more of the aspects described in this document. Examples of such devices include, but are not limited to, various electronic devices such as laptops, smartphones, tablet computers, digital media set-top boxes, digital television receivers, personal video recording systems, connected appliances, and servers. Elements of System 1000, alone or in combination, can be exemplified by a single integrated circuit (IC), multiple ICs, and / or discrete components.For example, in at least one mode, the processing and encoder / decoder elements of System 1000 are distributed across multiple ICs and / or discrete components. In several modes, System 1000 is communicatively coupled to one or more different systems, or other electronic devices, via, for example, a communication bus or through dedicated input and / or output ports. In several modes, System 1000 is configured to implement one or more of the aspects described in this document. System 1000 includes at least one processor 1010 configured to execute instructions loaded thereon to implement, for example, the various aspects described in this document. The processor 1010 may include integrated memory, an input / output interface, and various other circuit assemblies as known in the art. System 1000 includes at least one memory 1020 (for example, a volatile memory device, and / or a non-volatile memory device). System 1000 includes a storage device 1040, which may include non-volatile and / or volatile memory, including, but not limited to, Electrically Erasable Programmable Read-Only Memory (EEPROM), Read-Only Memory (ROM), Programmable Read-Only Memory (PROM), Random Access Memory (RAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), flash memory, a magnetic disk drive, and / or an optical disk drive.The 1040 storage device may include an internal storage device, an attached storage device (including removable and non-removable storage devices), and / or a network-accessible storage device, as non-limiting examples. The 1000 system includes a 1030 encoder / decoder module configured, for example, to process data to provide encoded or decoded video, and the 1030 encoder / decoder module may include its own processor and memory. The 1030 encoder / decoder module represents one or more modules that can be included in a device to perform encoding and / or decoding functions. As is known, a device may include one or both of the encoding and decoding modules. Additionally, the 1030 encoder / decoder module can be implemented as a separate element of the 1000 system or can be incorporated within the 1010 processor as a combination of hardware and software as known to those experts in the art. The program code to be loaded into processor 1010 or encoder / decoder 1030 to perform the various functions described herein may be stored in storage device 1040 and subsequently loaded into memory 1020 for execution by processor 1010. Depending on the configuration, one or more of the processor 1010, memory 1020, storage device 1040, and encoder / decoder module 1030 may store one or more items during the execution of the processes described herein. Such stored items may include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams, arrays, variables, and intermediate or final results of processing equations, formulas, operations, and operational logic. In some configurations, the memory within the 1010 processor and / or the 1030 encoder / decoder module is used to store instructions and provide working memory for processing required during encoding or decoding. In other configurations, however, memory external to the processing device (for example, the processing device may be either the 1010 processor or the 1030 encoder / decoder module) is used for one or more of these functions. External memory can be either 1020 memory and / or 1040 storage device, for example, dynamic volatile memory and / or non-volatile flash memory. In some cases, external non-volatile flash memory is used to store the operating system of, for example, a television. In at least one modality, a fast external dynamic volatile memory such as RAM is used as working memory for video encoding and decoding operations, such as MPEG-2 (MPEG refers to the Moving Image Expert Group, MPEG-2 is also referred to as ISO / IEC 13818, and 13818-1 is also known as H.222, and 13818-2 is also known as H.262), HEVC (HEVC refers to High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2), or WC (Versatile Video Coding, a new standard being developed by JVET, the Joint Video Expert Group). Input to the 1000 system elements can be provided through various input devices as indicated in block 1130. Such input devices include, but are not limited to, (i) a radio frequency (RF) portion receiving a transmitted RF signal, for example, over the air by means of a broadcaster, (ii) a component input (COMP) terminal (or a series of COMP input terminals), (iii) a Universal Serial Bus (USB) input terminal, and / or (iv) a High Definition Multimedia Interface (HDMI) input terminal. Other examples, not shown in Figure 10, include composite video. In several configurations, the input devices of block 1130 have respective associated input processing elements as known in the art. For example, the RF portion may be associated with elements suitable for (i) selecting a desired frequency (also referred to as selecting a signal, or band-limiting a signal to a frequency band), (ii) down-converting the selected signal, (iii) band-limiting it again to a narrower frequency band to select (for example) a signal frequency band which may be referred to as a channel in certain configurations, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select the desired stream of data packets.The RF portion of various modes includes one or more elements to perform these functions, for example, frequency selectors, signal selectors, band limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF portion may include a tuner that performs several of these functions, including, for example, downconverting the received signal to a lower frequency (for example, an intermediate frequency or a near-baseband frequency) or to baseband. In a decoder mode, the RF portion and its associated input processing element receive an RF signal transmitted over a connected medium (for example, cable) and performs frequency selection by filtering, downconverting, and filtering again to a desired frequency band.Several configurations rearrange the order of the elements described above (and others), remove some of these elements, and / or add other elements performing similar or different functions. Adding elements may include inserting elements between existing elements, such as, for example, inserting amplifiers and an analog-to-digital converter. In several configurations, the RF portion includes an antenna. Additionally, the USB and / or HDMI terminals may include respective interface processors for connecting the 1000 system to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing, such as Reed-Solomon error correction, may be implemented, for example, within a separate input processing IC or within the 1010 processor as required. Similarly, aspects of USB or HDMI interface processing may be implemented within separate interface ICs or within the 1010 processor as required. The demodulated, error-corrected, and demultiplexed stream is provided to various processing elements, including, for example, the 1010 processor and the 1030 encoder / decoder, operating in conjunction with memory and storage elements to process the data stream as needed for presentation on an output device. Several elements of the 1000 system can be provided within an integrated housing. Within the integrated housing, the various elements can be interconnected and transmit data between them using a suitable connection arrangement, such as an internal bus as known in the art, including the Inter-IC (I2C) bus, wiring, and printed circuit board. The 1000 system includes a 1050 communication interface that enables communication with other devices via the 1060 communication channel. The 1050 communication interface may include, but is not limited to, a transceiver configured to transmit and receive data over the 1060 communication channel. The 1050 communication interface may also include, but is not limited to, a modem or network card, and the 1060 communication channel can be implemented, for example, within a wired and / or wireless medium. Data is transmitted, or otherwise provided, to System 1000 in several modes using a wireless network such as Wi-Fi, for example, IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers). The Wi-Fi signal in these modes is received over communication channel 1060 and communication interface 1050, which are adapted for Wi-Fi communication. Communication channel 1060 in these modes is typically connected to an access point or router that provides access to external networks, including the Internet, to enable the transmission of applications and other extensive communication. Other modes provide data transmitted to System 1000 using a decoder that outputs data over the HDMI connection of input block 1130. Still other modes provide data to System 1000 using the RF connection of input block 1130.As mentioned above, several modes provide data in a non-transmittable manner. Additionally, several modes utilize wireless networks other than Wi-Fi, such as a cellular network or a Bluetooth network. The 1000 system can provide an output signal to various output devices, including a display 1100, speakers 1110, and other peripheral devices 1120. The 1100 gold Lnn / zznz / E / Yi display in various configurations includes one or more of, for example, a touchscreen, an organic light-emitting diode (OLED) display, a curved screen, and / or a foldable screen. The 1100 display can be for a television, a tablet, a laptop, a cell phone (mobile phone), or another device. The 1100 display can also be integrated with other components (for example, as in a smartphone) or separate (for example, an external monitor for a laptop). The other peripheral devices 1120 include, in various configuration examples, one or more of a standalone digital video disc (or digital versatile disc) (DVR, for both terms), a disc player, a stereo system, and / or a lighting system.Several modes utilize one or more peripheral devices 1120 that provide a function based on the output of system 1000. For example, a disc player performs the function of playing the output of system 1000. In several configurations, control signals are communicated between System 1000 and the display 1100, speakers 1110, or other peripheral devices 1120 using signaling such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols that enable device-to-device control with or without user intervention. Output devices can be communicatively coupled to System 1000 via dedicated connections through respective interfaces 1070, 1080, and 1090. Alternatively, output devices can be connected to System 1000 using communication channel 1060 through communication interface 1050. The display 1100 and speakers 1110 can be integrated into a single unit with the other System 1000 components in an electronic device such as, for example, a television.In several modes, the 1070i deployment interface includes a deployment unit, such as, for example, a timing controller chip (T Con). The display 1100 and speakers 1110 can be alternatively separated from one or more of the other components, for example, if the RF input portion 1130 is part of a separate decoder. In various configurations where the display 1100 and speakers 1110 are external components, the output signal can be provided through dedicated output connections, including, for example, HDMI ports, USB ports, or COMP outputs. The modes can be implemented by computer software using the 1010 processor, by hardware, or by a combination of both. For example, the modes can be implemented using one or more integrated circuits. The 1020 memory can be of any type appropriate to the technical environment and can be implemented using any suitable data storage technology, such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory. The 1010 processor can be of any type appropriate to the technical environment and can include one or more microprocessors, general-purpose computers, special-purpose computers, and processors based on a multi-core architecture. Another example of a modality is illustrated in Figure 13. In Figure 13, the input information, such as a bitstream, includes encoded video information and other information, such as information associated with the encoded video information, for example, a quantization matrix or Lnn / zznz / E / Yi matrices, and control information, such as one or more syntax elements. In 1310, information representing at least one coefficient of a quantization matrix and one syntax element are obtained from the input, for example, the input bitstream. In 1320, it was determined, based on a syntax element, that the information representing at least one coefficient should be interpreted as a remainder. Then, in 1330, at least a portion of the encoded video information is decoded based on a combination of a quantization matrix prediction and the remainder.One or more examples of features shown and described in relation to Figure 13 were previously illustrated and described herein, for example in relation to Figure 6. Another example of a modality is illustrated in Figure 14. In Figure 14, an input including video information is processed in 1410 to obtain video information and information representing at least one coefficient of a predicted quantization matrix associated with at least a portion of the video information. In 1420, it was determined that the at least one coefficient is to be interpreted as a remainder. For example, video information processing may indicate that the use of a remainder provides advantageous processing of certain video information; for example, it provides improved compression efficiency when encoding the video information or improved video quality when decoding the encoded video information. Thus, in 1430, at least a portion of the video information is encoded based on a combination of the predicted quantization matrix and the remainder.The encoding in 1430 also encodes a syntax element that indicates at least one coefficient to be interpreted as a remainder. Several implementations and examples of the modes described herein involve decoding. "Decoding," as used in this application, may encompass all or part of the processes performed, for example, on a received encoded sequence in order to produce a final output suitable for deployment. In several modes, such processes include one or more of the processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In several modes, such processes also, or alternatively, include processes performed by a decoder of the various implementations described in this application. As further examples, in one modality "decoding" refers only to entropy decoding, in another modality "decoding" refers only to differential decoding, and in yet another modality "decoding" refers to a combination of entropy decoding and differential decoding. Whether the phrase "decoding process" is intended to refer specifically to a subset of operations or generally to a broader decoding process will be clear from the context of the specific descriptions and is believed to be well understood by those experts in the technique. Several implementations involve encoding. Analogous to the earlier discussion of "decoding," "encoding" as used here can encompass all or part of the processes performed, for example, on an input video sequence to produce an encoded bitstream. In various modalities, such processes include one or more of the processes typically performed by an encoder, such as division, differential encoding, transformation, Lnn / zznz / E / Yi quantization, and entropy encoding. As further examples, in one modality "coding" refers only to entropy coding, in another modality "coding" refers only to differential coding, and in yet another modality "coding" refers to a combination of differential coding and entropy coding. Whether the phrase "coding process" is intended to refer specifically to a subset of operations or generally to the broader coding process will be clear from the context of the specific descriptions and is believed to be well understood by those skilled in the technique. Note that syntax elements as used herein, for example, scaling_list_pred_mode_flag, scaling_list_pred_matr_d_delta, scaling_list_dc_coef_minus8, are descriptive terms. As such, they do not preclude the use of other syntax element names. When a figure is presented as a flowchart, it should be understood that it also provides a block diagram of a corresponding device. Similarly, when a figure is presented as a block diagram, it should be understood that it also provides a flowchart of a corresponding method / process. The implementations and aspects described here can be implemented in, for example, a method or process, a device, a software program, a data stream, or a signal. Even if discussed only in the context of a simple form of implementation (e.g., discussed only as a method), the implementation of the discussed features can also be implemented in other forms (e.g., a device or program). A device can be implemented in, for example, appropriate hardware, software, and firmware. Methods can be implemented in, for example, a processor, which refers to processing devices in general, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device.Processors also include communication devices, such as, for example, computers, cell phones, portable / personal digital assistants ("PDAs"), and other devices that facilitate the communication of information between end users. Reference to "a modality" or "an implementation" or "an implementation," as well as other variations thereof, means that a particular feature, structure, characteristic, etc., described in relation to the modality is included in at least one modality. Therefore, occurrences of the phrase "in a modality" or "in an implementation," as well as any other variations, appearing in various places throughout this application are not necessarily all referring to the same modality. Additionally, this request may refer to "determining" various pieces of information. Determining information may include one or more of the following: estimating information, calculating information, predicting information, or retrieving information from memory. Furthermore, this request may refer to "having access" to various pieces of information. Having access to information may include one or more of the following: receiving the information, retrieving the information (e.g., from memory), storing the information, moving the information, copying the information, calculating the information, determining the information, predicting the information, or estimating the information. Additionally, this request may refer to "receiving" various pieces of information. "Receiving," as with "having access," is intended to be a broad term. Receiving information may include one or more actions, such as accessing information or retrieving information (e.g., from memory). Furthermore, "receiving" is typically involved, in one way or another, during operations such as storing information, processing information, transmitting information, moving information, copying information, deleting information, calculating information, determining information, predicting information, or estimating information. It will be appreciated that the use of any of the following, such as “7”, “and / or”, and “at least one of”, for example, in the cases of “A / B”, “A and / or B”, and “at least one of A and B”, is intended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of both options (A and B). As a further example, in the cases of “A, B, and / or C” and “at least one of A, B, and C”, these phrases are intended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of the third listed option (C) only, or the selection of the first and second listed options (A and B) only, or the selection of the first and third listed options (A and C) only, or the selection of the second and third listed options (B and C) only, or the selection of all three options. (A and B and C).This can be extended, as is clear to an expert in this and related techniques, to as many elements as are listed. As will be evident to someone skilled in the art, implementations can produce a variety of formatted signals to carry information that can be, for example, stored or transmitted. The information might include, for example, instructions for carrying out a method or data produced by one of the described implementations. For instance, a signal might be formatted to carry the bitstream of a described mode. Such a signal might be formatted, for example, as an electromagnetic wave (e.g., using a portion of the radio frequency spectrum) or as a baseband signal. The formatting might include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information carried by the signal might be, for example, analog or digital. The signal might be transmitted over a variety of different wired or wireless links, as they are known.The signal can be stored on a processor-readable medium. Several generalized and specific modalities are supported and contemplated throughout this disclosure. Examples of modalities according to this disclosure include, but are not limited to, the following. In general, at least one example of a modality may involve a method comprising: obtaining, from a bitstream that includes encoded video information, information representing at least one coefficient of a quantization matrix and a syntax element; determining, based on the syntax element, that the information representing the at least one coefficient is to be interpreted as a remainder; and decoding at least a portion of the encoded video information based on a combination of a prediction of the quantization matrix and the remainder. In general, at least one example of a modality may involve apparatus comprising: one or more processors configured to obtain, from a bitstream including encoded video information, information representing at least one coefficient of a quantization matrix and a syntax element; determine, based on the syntax element, that the information representing the at least one coefficient is to be interpreted as a remainder; and decode at least a portion of the encoded video information based on a combination of a prediction of the quantization matrix and the remainder. In general, at least one example of a modality may involve a method comprising: obtaining video information and information representing at least one coefficient of a quantization matrix associated with at least one portion of the video information; determining that the at least one coefficient is to be interpreted as a remainder; and encoding the at least one portion of the video information based on a combination of the predicted quantization matrix and the remainder, and encoding a syntax element indicating the at least one coefficient that is to be interpreted as a remainder. In general, at least one example of a modality may involve apparatus comprising: one or more processors configured to obtain video information and information representing at least one coefficient of a predicted quantization matrix associated with at least one portion of the video information; determining that the at least one coefficient is to be interpreted as a remainder; and encoding the at least one portion of the video information based on a combination of the predicted quantization matrix and the remainder, and encoding a syntax element indicating the at least one coefficient that is to be interpreted as a remainder. In general, at least one example of a modality may involve a method or apparatus including a remainder as described herein, wherein the remainder comprises a remainder of variable length. In general, at least one example of a modality may involve a method or apparatus as described herein, wherein at least one coefficient comprises a plurality of predicted quantization matrix coefficients and a combination comprises adding the remainder to the plurality of predicted quantization matrix coefficients. In general, at least one example of a modality may involve a method or apparatus as described here, where a combination also includes applying a scaling factor. In general, at least one example of a modality may involve a method or apparatus as described herein, wherein a combination comprises applying a scaling factor to a plurality of predicted quantization matrix coefficients and then adding a remainder. In general, at least one example of a modality may involve a method or apparatus as described herein, where a combination also includes applying compensation. In general, at least one example of a modality may involve a method or apparatus as described herein, wherein a bitstream includes one or more syntax elements used to transmit at least one coefficient when the quantization matrix is not predicted that may be reused to transmit the remainder when the quantization matrix is predicted. In general, at least one example of a modality may involve a method or apparatus as described herein, producing a bitstream including one or more syntax elements used to transmit at least one coefficient when the quantization matrix is not predicted that can be reused to transmit the remainder when the quantization matrix is predicted. In general, one or more of the other examples of modalities may also provide a computer-readable storage medium, for example, a non-volatile computer-readable storage medium, that has instructions stored therein for encoding or decoding image information such as video data according to the methods or apparatus described herein. One or more modalities may also provide a computer-readable storage medium that has stored therein a bitstream generated according to the methods or apparatus described herein. One or more modalities may also provide methods and apparatus for transmitting or receiving the bitstream generated according to the methods or apparatus described herein. In general, another example of a modality may comprise a signal including data generated according to any of the methods described herein. In general, another example of a modality may involve a signal or bitstream as described here and comprise at least one syntax element used to represent at least one coefficient of the quantization matrix when the quantization matrix is not predicted, and wherein at least one syntax element is used to represent the remainder when the quantization matrix is predicted. In general, another example of a modality may comprise a device including an apparatus according to any example of a modality described herein, and at least one of (i) an antenna configured to receive a signal, the signal including data representative of the image information, (ii) a band limiter configured to limit the received signal to a frequency band that includes data representative of the image information, and (ii) a display configured to display an image of the image information. In general, another example of a modality may involve a device as described herein and comprising one of a television, a television signal receiver, a set-top box, a door device, a mobile device, a cell phone, a tablet, or other electronic device. Several embodiments are described herein. Features of these embodiments may be provided alone or in any combination, across various categories and types of claims. Furthermore, embodiments may include one or more of the following features, devices, or aspects, alone or in any combination, across various types and categories of claims: • Providing video encoding and / or decoding, including saving bits in the transmission of custom QMs, while keeping the specification as simple as possible; • Providing video encoding and / or decoding, including improved QM prediction, so that simple adjustments are possible with low bit cost; gold Lnn / zznz / B / Yi • Providing video encoding and / or decoding, understanding a simple way to specify an overall offset; • Providing video encoding and / or decoding comprising one more way to extend QM prediction by applying a scaling factor in addition to copying and downsampling, providing a simple way to specify an overall offset, and allowing the prediction to be refined with a remaining variable; gold Lnn / zznz / E / Yi • Providing video encoding and / or decoding comprising combining enhanced prediction with a remaining, to further refine QMs, at a potentially lower cost than full encoding; • Providing encoding and / or decoding QM with compensation in addition to copy / scale down; • Providing encoding and / or decoding QM plus remaining (variable length); • Providing encoding and / or decoding QM with scale factor in addition to copy / scale down; • Providing video encoding and / or decoding, including video prediction QM including a combination of scale factor with either compensation or remaining balance; • Providing video encoding and / or decoding, including reducing the number of bits required to transmit QMs, and allowing user-defined trade-off between accuracy and bit cost, while keeping the specification simple; • Providing video encoding and / or decoding by reducing the number of bits required to transmit QMs, and allowing user-defined trade-offs between accuracy and bit cost, while keeping the specification simple by including one or more of the following features: or Add a remainder to the QM prediction; or Add a scaling factor to the QM prediction; or Combine remaining and scaling; or Always use prediction (remove prediction mode indicator); • Providing video encoding and / or decoding comprising a modification to the top of the copy for QM prediction including adding an overall offset, wherein an overall offset may be explicitly specified. • Providing video encoding and / or decoding comprising an overall offset that can be specified as the first coefficient of a remainder encoded by DPCM; • Providing video encoding and / or decoding, including adding a remainder to the prediction; • Providing video encoding and / or decoding, including including a remainder on top of the QM prediction to refine the prediction or providing an optimized encoding scheme for QMs (prediction plus remainder instead of direct encoding); • Providing video encoding and / or decoding, understanding the syntax used to transmit QM coefficients when not predicted, which can be reused to transmit a remaining QM when predicted; • Providing video encoding and / or decoding comprising adding a remainder, where the number of coefficients of the remainder can be variable; • Providing video encoding and / or decoding comprising adding a remainder, wherein adding the remainder may involve the use of an indicator when in prediction mode it indicates that the coefficients may still be transmitted as non-prediction mode but interpreted as remainders; • Providing video encoding and / or decoding, understanding when not in prediction mode, coefficients that can still be interpreted as remaining on a predetermined QM prediction; • Providing video encoding and / or decoding comprising the use of a variable-length remainder over prediction; • Providing video encoding and / or decoding comprising the use of a variable-length remainder over prediction, wherein a number of remainder coefficients may be indicated by one or more of: o An explicit number of coefficients; o An indicator indicating the presence of a remainder followed by the number of remainder coefficients less 1, which may be restricted to powers of 2; o 1 + 1 / 2 of the number of remainder coefficients, indicating zero remainder, wherein this may be encoded as fixed length (length potentially depending on matrixld) or exp-golomb encoding; o An index in a list, wherein the list contains numbers of coefficients, e.g., this list may be fixed to a standard and need not be transmitted, such as {0, 1, 2, 4, 8, 16, 32, 64};oo The number of remaining coefficients does not need to take into account the extra potential DC remaining coefficient which can be transmitted as soon as at least one regular remaining coefficient is to be sent; • Providing video encoding and / or decoding, including adding a scaling factor for prediction, e.g., adding a scaling factor for copying / decimation; • Providing video encoding and / or decoding comprising QM prediction including a variable length remainder and a scaling factor, thereby allowing adjustment to QMs based on scaling and / or compensation, and allowing user-defined trade-off between accuracy and bit cost based on the variable length remainder; • Providing video encoding and / or decoding comprising QM prediction including a variable-length remainder and a scale factor, thereby enabling adjustments for QMs based on scaling and / or compensation, and allowing user-defined trade-off between accuracy and bit cost based on the variable-length remainder, wherein the scale factor can be an integer, normalized to a number N (e.g., 16), and wherein a scaling formula scales the reference QM around a neutral value; • Providing video encoding and / or decoding comprising adding a scaling factor for prediction, wherein the scaling factor can be an integer, normalized to a number N, and wherein the scaling factor can be transmitted as an offset to N; • Providing video encoding and / or decoding, including removing a prediction mode indicator; • A bitstream or signal that includes one or more of the described syntax elements, or variations thereof; • A bit stream or signal that includes syntax carrying information generated according to any of the described modalities; • By inserting into the signaling syntax elements that allow the decoder to operate in a manner corresponding to that used by an encoder; • Create and / or transmit and / or receive and / or decode a bit stream or signal that includes one or more of the described syntax elements, or variations thereof; • Create and / or transmit and / or receive and / or decode according to any of the modalities described; • A method, process, apparatus, means of storing instructions, means of storing data, or signal in accordance with any of the modalities described; • A TV, set-top box, cell phone, tablet, or other electronic device that provides implementation of the described modalities; • A TV, set-top box, cell phone, tablet, or other electronic device that provides the ability to implement any of the described modalities, and that displays (for example, using a monitor, screen, or other type of display) a resulting image; • A TV, set-top box, cell phone, tablet, or other electronic device that selects (for example, uses a tuner) a channel to receive a signal including an encoded image, and that provides the ability to implement any of the described modes; • A TV, set-top box, cell phone, tablet, or other electronic device that receives (for example, uses an antenna) an over-the-air signal that includes an encoded image, and provides implementation of any of the described modalities.< / hrf>
Claims
1. A method, characterized in that it comprises: obtaining, from a bitstream including encoded video information, information representing at least one coefficient of a quantization matrix and a syntax element; determining, based on the syntax element, that the information representing the at least one coefficient is to be interpreted as a remainder; and decoding at least a portion of the encoded video information based on a combination of a prediction of the quantization matrix and the remainder.
2. -An apparatus, characterized in that it comprises: one or more processors configured to obtain, from a bitstream including encoded video information, information representing at least one coefficient of a quantization matrix and a syntax element; determine, based on the syntax element, that the information representing the at least one coefficient is to be interpreted as a remainder; and decode at least a portion of the encoded video information based on a combination of a prediction of the quantization matrix and the remainder.
3. A method, characterized in that it comprises: obtaining video information and information representing at least one coefficient of a predicted quantization matrix associated with at least one portion of the video information; determining that the at least one coefficient is to be interpreted as a remainder; and encoding the at least one portion of the video information based on a combination of the predicted quantization matrix and the remainder, and encoding a syntax element indicating the at least one coefficient that is to be interpreted as a remainder.
4. An apparatus, characterized in that it comprises: one or more processors configured to obtain video information and information representing at least one coefficient of a predicted quantization matrix associated with at least one portion of the video information; determining that the at least one coefficient is to be interpreted as a remainder; and encoding the at least one portion of the video information based on a combination of the predicted quantization matrix and the remainder, and encoding a syntax element indicating that the at least one coefficient is to be interpreted as a remainder.
5. The method or apparatus according to any of the preceding claims, further characterized in that the remainder comprises a remainder of variable length.
6. The method or apparatus according to any of claims 1-5, further characterized in that the at least one coefficient comprises a plurality of predicted quantization matrix coefficients and the combination comprises adding the remainder to the plurality of predicted quantization matrix coefficients.
7. The method or apparatus according to any of the preceding claims, gold Lnn / zznz / E / YiAi further characterized in that the combination further comprises applying a scaling factor.
8. The method or apparatus according to claim 7 combined with claim 6, further characterized in that the combination comprises applying the scaling factor to the plurality of predicted quantization matrix coefficients and then adding the remainder.
9. The method or apparatus according to any of the preceding claims, further characterized in that the combination additionally comprises applying a compensation.
10. The method according to claim 1 or the apparatus according to claim 2, further characterized in that the bit stream includes one or more syntax elements used to transmit at least one coefficient when the quantization matrix does not predict that it can be reused to transmit the remainder when the quantization matrix is predicted.
11. The method according to claim 3 wherein the encoding, or the apparatus of claim 4, further characterized in that the at least one processor being configured to encode, produces a bit stream including one or more syntax elements used to transmit at least one coefficient when the quantization matrix does not predict that can be reused to transmit the remainder when the quantization matrix is predicted.
12. A non-transient, computer-readable medium, characterized by the fact that it stores a signal.
13. A non-transient, computer-readable medium, characterized by storing a stream of bits.
14. The non-transient computer-readable medium according to claim 12 or 13, further characterized in that it comprises at least one syntax element used to represent at least one coefficient of the quantization matrix when the quantization matrix is not predicted, and wherein the at least one syntax element is used to represent the remainder when the quantization matrix is predicted.