Video encoding and decoding method and device

Abstract

A method and apparatus for video encoding and decoding are disclosed. A piece-wise linear mapping is derived to map a target luminance sample from a first signal domain to a second signal domain, and vice versa. The piece-wise linear mapping process includes determining a first pivot point to perform video encoding or decoding of the luminance sample using the piece-wise linear mapping. The first pivot point is determined such that: if a target image pivot point corresponding to a first pivot point in the second signal domain has a multiple of a target interval value, the target image pivot point and the next mapping pivot point are allowed to be located in the same second segment; otherwise, the target image pivot point and the next mapping pivot point are not allowed to be located in the same second segment.

Description

Technical Field

[0001] The present invention relates to video coding for color video data, wherein a luma map is applied to the luma component. More particularly, the present invention discloses techniques for simplifying the derivation of an inverse luma map. Background Art

[0002] Versatile Video Coding (VVC) is an emerging video codec standard developed by the Joint Video Experts Team (JVET), which is composed of the Video Codec Experts Group of Study Group 16 of the ITU Telecommunication Standardization Sector (ITU-T) and the Moving Picture Experts Group (MPEG). VVC is based on the High Efficiency Video Coding (HEVC) video standard with improved and new codec tools. For example, the reshaping process is a new codec tool adopted in VTM-4.0 (VVC Test Model Version 4.0). The reshaping process is also called Luma Mapping and Chroma Scaling. Scaling (LMCS). When reshaping is applied, the video samples are encoded, decoded and reconstructed in the reshaped domain before loop filtering. The reshaped domain reconstructed samples are converted to the original domain using inverse reshaping. The loop filtered original domain reconstructed samples are stored in the decoded picture buffer. For inter mode, the motion compensated (MC) predictor is converted to the reshaped domain using forward reshaping. Figure 1 An example of the reshaping process on the decoder side is shown.

[0003] like Figure 1 As shown, the bitstream is decoded by a context-adaptive binary arithmetic coding (CABAC) decoder (i.e., CABAC -1 ), inverse quantization (i.e. Q -1 ) and the inverse transform (T -1 ) is processed to obtain the reconstructed brightness residual Y res. The reconstructed luma residual is provided to the luma reconstruction block 120 to generate a reconstructed luma signal. For intra mode, the predictor comes from the intra prediction block 130. For inter mode, the predictor comes from the motion compensation block 140. Since the reconstruction is applied to the luma signal on the encoder side, the forward reshaping block 150 is applied to the predictor. Before the predictor is provided to the reconstruction block 120, the forward reshaping block 150 is provided to the predictor from the motion compensation block 140. The inverse reshaping block 160 is applied to the reconstructed luma signal from the reconstruction block 120 to recover the unreshaped reconstructed luma signal. Then, a loop filter 170 is applied to the unreshaped reconstructed luma signal before the signal is stored in a decoded picture buffer (DPB) 180.

[0004] When reshaping is applied, chroma residual scaling is also applied. Chroma residual scaling compensates for the interaction of luma and chroma signals, such as Figure 2 As shown. Figure 2 In the figure, the upper part corresponds to luma decoding and the lower part corresponds to chroma decoding.

[0005] Chroma residual scaling is applied at TU level according to the following formulas at the encoder and decoder respectively:

[0006] Encoder side: C ResScale =C Res *C Scale =C Res / C ScaleInv (1)

[0007] Decoder side: C Res =C ResScale / C Scale =C ResScale *C ScaleInv (2)

[0008] In the above formula, C Res is the original chrominance residual signal, C ResScale is the scaled chroma residual signal. C Scale is the scaling factor for the inter mode predictor calculated using the FwdLUT (ie, forward lookup table), and is converted to its inverse C ScaleInv The multiplication is performed on the decoder side instead of division, thus reducing the implementation complexity. The scaling operation on both the encoder and decoder sides is implemented using fixed-point integer arithmetic via the following formula:

[0009] c'=sign(c)*((abs(c)*s+2 CSCALE_FP_PREC-1 )>>CSCALE_FP_PREC) (3)

[0010] In the above formula, c is the chroma residual, s is the chroma residual scaling factor in cScaleInv[pieceIdx], pieceIdx is determined by the corresponding average luminance value of the TU, and CSCALE_FP_PREC is a constant used to specify the precision. To derive the scaling factor, the predictor for the entire TU is used. ScaleInv The value of is calculated according to the following steps:

[0011] (1) If it is intra mode, the average value of intra predicted luminance values ​​is calculated; if it is inter mode, the average value of forward reconstructed inter predicted luminance values ​​is calculated. In other words, the average luminance value avgY'TU is calculated in the reconstructed domain.

[0012] (2) Find the index idx where avgY'TU belongs to the inverse mapping PWL.

[0013] (3)C ScaleInv =cScaleInv[idx]

[0014] Derive the chroma scaling factor C ScaleInv The steps are Figure 2 The derived chroma scaling factor C ScaleInv The chroma residual is used to convert the scaled chroma residual, which is decoded by CABAC (i.e., CABAC -1 ), inverse quantization (i.e. Q -1 ) and the inverse transform (T -1 ) to reconstruct. The reconstruction block 220 reconstructs the chroma signal by adding the predictor to the reconstructed chroma residual. For intra mode, the predictor comes from the intra prediction block 230. For inter mode, the predictor comes from the motion compensation block 240. Then, a loop filter 270 is applied to the reconstructed chroma signal before the signal is stored in the chroma DPB 280.

[0015] Figure 3 shows an example of a brightness map. Figure 3A In , a 1:1 mapping is shown, where the output (i.e., the reshaped brightness) is the same as the input. Since the histogram of brightness samples is usually not flat, using intensity reshaping can help improve performance in the sense of rate-distortion optimization (RDO). Statistics of brightness samples are calculated for an image region (e.g., image). A mapping curve is then determined based on the statistics. Typically, a piecewise linear (PWL) mapping curve is used. Figure 3BAn example of a PWL mapping with 3 segments is shown, where two adjacent segments have different slopes. The dashed line 340 corresponds to a 1:1 mapping. If the samples in the range from 0 to 340 have a large spatial variance and a small number of occurrences, the input range 0-340 is mapped to a smaller output range (i.e., 0-170), as shown in FIG. Figure 3B If the samples in the range from 340 to 680 have a smaller spatial variance and a larger number of occurrences, the input range 340-680 is mapped to a larger output range (i.e., 170-850), as shown in FIG. Figure 3B If the samples in the range from 680 to 1023 have a larger spatial variance and a smaller number of occurrences, the input range 680-1023 is mapped to a smaller output range (i.e., 850-1023), as shown in segment 330 of FIG. 3B . Figure 3B It is intended to show a simple PWL map. In practice, a PWL map may have more or fewer segments.

[0016] Intra sub-block partition (ISP) and sub-block transform (SBT)

[0017] In order to generate better intra mode predictors, ISP is applied. When ISP is applied, the luma component is divided into multiple sub-TBs. The sub-TBs are reconstructed one by one. For each sub-TU, the reconstructed samples of adjacent sub-TBs can be used as adjacent reconstructed samples for intra prediction. For the chroma component TB, it is not divided into multiple sub-TBs like luma.

[0018] Similar to ISP, SBT can be applied to inter-frame mode. When SBT is applied, only part of the CU data is converted. For example, the current block is divided into two partitions by horizontal or vertical partitioning. Only one of the partitions can be used for transform coding. The residual of the other partition is set to zero. For example, the CU is divided into two TUs or four TUs. Only one of the TUs has a non-zero coefficient.

[0019] Sending LMCS parameters

[0020] Table 1 shows the syntax table of LMCS parameters being considered by VVC.

[0021] Table 1

[0022]

[0023] In the above syntax table, the semantics of the syntax are defined as follows:

[0024] lmcs_min_bin_idx specifies the minimum bin index of the PWL (piecewise linear) model of the luminance image.

[0025] lmcs_delta_max_bin_idx specifies a delta value between 15 and the maximum bin index LmcsMaxBinIdx used in lmcs. The value should be between 1 and 15 (inclusive).

[0026] lmcs_delta_cw_prec_minus1 plus 1 is the number of bits used to represent the lmcs_delta_abs_cw[i] syntax.

[0027] lmcs_delta_abs_cw[i] is the absolute delta codeword value of the i-th bin.

[0028] lmcs_delta_sign_cw_flag[i] is the sign of the variable lmcsDeltaCW[i].

[0029] The variable lmcsDeltaCW[i] is derived as follows:

[0030] lmcsDeltaCW[i]=(1-2*lmcs_delta_sign_cw_flag[i])*lmcs_delta_abs_cw[i].

[0031] The variable lmcsCW[i] with i=0...15 specifies the number of codewords per interval in the image domain, which can be derived as follows:

[0032]

[0033] To represent the PWL model of the reshaping curve, three variables: LmcsPivot[i] with i=0…16, ScaleCoeff[i] with i=0…15, and InvScaleCoeff[i] with i=0…15 can be derived as follows:

[0034]

[0035] In the above derivation, SCALE_FP_PREC is a constant value used to specify the precision.

[0036] In the conventional LMCS process, inverse luminance mapping (eg, comparing the luminance samples to be mapped with multiple thresholds) is quite time consuming. Therefore, it is desirable to develop techniques that simplify the inverse mapping process. Summary of the invention

[0037] The present invention discloses a method and device for video encoding and decoding. According to a method of the present invention, a plurality of first segments are determined, wherein a range of luminance samples corresponding to luminance samples in a first signal domain is divided into a plurality of first segments, and each first segment corresponds to a linear segment for mapping luminance samples from the first signal domain to a second signal domain. In addition, a plurality of second segments are determined, wherein a range of luminance samples corresponding to luminance samples in a second signal domain is divided into a plurality of second segments, and all second segments have the same size equal to a target interval value. One or more first pivot points are determined or derived, wherein each first pivot point is associated with a first segment, and wherein the one or more first pivot points are determined or derived such that: if a target image pivot point corresponding to a first pivot point in the second signal domain has a multiple of the target interval value, the target image pivot point and the next mapping pivot point are allowed to be located in the same second segment; otherwise, the target image pivot point and the next mapping pivot point are not allowed to be located in the same second segment. Piecewise linear mapping is used to perform video encoding or decoding of luminance samples to map target luminance samples from the first signal domain to the second signal domain, or from the second signal domain to the first signal domain.

[0038] In one embodiment, the piecewise linear image is associated with the plurality of first segments, the plurality of second segments, and the one or more first pivot points.

[0039] In one embodiment, all first segments have the same first interval value. For example, the first interval value corresponds to 2 to a power of a first predefined number, and the first predefined number is an integer. In addition, the first predefined number may be equal to (bitDepth-4), and bitDepth represents the bit depth of the luma sample.

[0040] In one embodiment, the target interval value corresponds to 2 to a power of a second predefined number, and the second predefined number is an integer. For example, the second predefined number is equal to (bitDepth-5), and bitDepth represents the bit depth of the luma sample.

[0041] In one embodiment, each second segment has at most two mapped pivot points. In one embodiment, the brightness sample range is equal to 0 to 1023 (inclusive), and the target interval value is equal to 32. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 An exemplary block diagram of a video decoder incorporating a luminance reshaping process is shown.

[0043] Figure 2 An exemplary block diagram of a video decoder that combines a luma reshaping process with chroma scaling is shown.

[0044] Figure 3AAn example of a 1:1 brightness mapping is shown, where the output (i.e., the reshaped brightness) is the same as the input.

[0045] Figure 3B An example of a piecewise linear (PWL) brightness map with 3 segments is shown.

[0046] Figure 4 A flow chart illustrating an exemplary codec system for determining or deriving a pivot point for an inverse luma map according to an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The following description is the best mode of implementing the present invention. This description is made to illustrate the general principles of the present invention and should not be considered as limiting. The scope of the present invention is best determined by reference to the attached claims.

[0048] LMCS maps the brightness samples in the original domain to the reshaped domain for better data estimation. The mapping curve is estimated by a piecewise linear model. To convert the sample values ​​from the original domain to the reshaped domain, a look-up table (LUT) is used. The number of entries in the LUT is the same as the dynamic range of the input sample. For example, if a 10-bit input is used, a LUT of 1024 entries is used. If a 14-bit input is used, a LUT of 8192 entries is used. In hardware implementation, such a LUT is expensive. Therefore, a piecewise linear model can be used. The input can be compared with each fragment to find out which fragment the input belongs to. In each fragment, the corresponding output value can be calculated based on the characteristics of the fragment.

[0049] Method 1 - LMCS with default number of codewords

[0050] In one embodiment, the number of codewords per binary bit (bin) in the image domain (eg, lmcsCW[i]) is derived using a default number of codewords instead of using OrgCW, depending only on the bit depth of the input data.

[0051] In the proposed method, the variable lmcsCW[i] from i=lmcs_min_bin_idx to LmcsMaxBinIdx can be derived as follows:

[0052] lmcsCW[i]=default_CW+lmcsDeltaCW[i],

[0053] In the above formula, default_CW is derived at the decoder side or sent from the encoder side.

[0054] In one embodiment, if default_CW is derived at the decoder side, it can be derived according to lmcs_min_bin_idx and LmcsMaxBinIdx. If the sum of the number of bins less than lmcs_min_bin_idx and the number of bins greater than LmcsMaxBinIdx is greater, default_CW can be adjusted to a value greater than OrgCW.

[0055] For example, if the sum of the number of bins less than lmcs_min_bin_idx and the number of bins greater than LmcsMaxBinIdx is equal to 2, default_CW is derived as default_CW=OrgCW+A, where A is a positive integer (eg, 1, 2, 3, . . . ).

[0056] If the sum of the number of bins less than lmcs_min_bin_idx and the number of bins greater than LmcsMaxBinIdx is equal to 0, default_CW is equal to OrgCW.

[0057] In one embodiment, if default_CW is sent, two syntaxes, default_delta_abs_CW and default_delta_sign_CW_flag, are sent before lmcs_delta_cw_prec_minus1.

[0058] The variable default_delta_abs_CW indicates the absolute difference between default_CW and OrgCW, and the variable default_delta_sign_CW_flag indicates whether the delta value is positive or negative. default_delta_sign_CW_flag is issued only when default_delta_abs_CW is greater than 0.

[0059] In one embodiment, if default_CW is sent, then after lmcs_delta_cw_prec_minus1, the syntax default_delta_CW is sent.

[0060] In one embodiment, if default_CW is sent, before the syntax default_delta_CW, another syntax lmcs_delta_default_cw_prec_minus1 is sent to indicate how many bins are required for the syntax default_delta_CW.

[0061] The variable default_delta_CW represents the difference between default_CW and OrgCW.

[0062] Method 2 - Simplified inverse mapping

[0063] For the luminance mapping, the variable InputPivot[i] is defined, where InputPivot[i] represents the pivot point of the input luminance samples. The variable InputPivot[i] for i = 0...16 can be derived as follows:

[0064] InputPivot[i] = i * OrgCW.

[0065] In the original design, the inverse scaling values are stored in a table with 16 entries, and each entry corresponds to each piece of the PWL mapping. Therefore, when performing the inverse mapping, in order to know the piece to which the luminance sample to be mapped belongs, the luminance sample to be mapped needs to be compared with the pivots of all pieces in the reshaped domain to find the piece to which the luminance sample to be mapped belongs. For each luminance sample to be mapped, this process requires at most 15 comparisons, and is derived as follows.

[0066]

[0067] In the above derivation, lumaSample is the luminance sample to be mapped, LmcsPivot[i] for i = 0…16 is the pivot of each piece in the reshaped domain, lmcs_min_bin_idx is the minimum bin index used in the luminance mapping, LmcsMaxBinIdx is the maximum bin index used in the luminance mapping, and idxYInv is the piece to which the luminance sample lumaSample belongs.

[0068] Method 2.1: Reconstruct the inverse reshaping curve

[0069] To simplify the inverse mapping process, it is proposed to reconstruct the inverse mapping curve according to the transmitted forward mapping curve. The luminance samples in the reshaped domain are divided into pieces of size equal to (1<<LOG2_INVERSE_INTERVAL), and each piece has its own inverse scaling value and inverse input pivot. The inverse scaling value of each piece is implemented using a table of size equal to ((1<<bitDepth)>>LOG2_INVERSE_INTERVAL), where bitDepth is the internal bit depth. When looking for the piece to which the luminance sample to be mapped belongs, we only need to right-shift the luminance sample to be mapped by LOG2_INVERSE_INTERVAL (i.e., log2(INVERSE_INTERVAL)), and then the index of the piece and the inverse input pivot can be derived, so the inverse scaling value and the inverse input pivot can be found.

[0070] The inverse scaling value for each interval in the reshaping domain is derived by dividing the difference between the first inverse luminance sample and the last inverse luminance sample by the interval size in the reshaping domain.

[0071] In one example, the inverse scaling value table is constructed through the following steps:

[0072] Step 1. Derive the forward scaling value according to the sent LMCS data

[0073] To represent the PWL model of the reshaping curve, three variables: LmcsPivot[i] for i = 0...16, ScaleCoeff[i] for i = 0...15, and InvScaleCoeff_tmp[i] for i = 0...15 can be derived as follows:

[0074] Step 2. Find the inverse mapping pivot value

[0075] For luminance samples that are multiples within the range of 0 to (1<<BitDepth)-1 (including 0 and (1<<BitDepth)-1) equal to (1<<LOG2_INVERSE_INTERVAL), the variable InputPivot_inv[i>>LOG2_INVERSE_INTERVAL] can be derived as follows:

[0076]

[0077]

[0078] In the above formula, by setting lumaSample to i, idxYInv is derived, and then the following process is followed:

[0079]

[0080] Step 3. Derive the inverse scaling value for each segment

[0081] For segments with an index equal to i (i = 0...(LmcsPivot[LmcsMaxBinIdx+1]>>LOG2_INVERSE_INTERVAL)):

[0082] If((i == (LmcsPivot[LmcsMaxBinIdx+1]>>LOG2_INVERSE_INTERVAL)) && (LmcsPivot[LmcsMaxBinIdx+1]!= (i<<LOG2_INVERSE_INTERVAL))

[0083] InvScaleCoeff[i]=(InputPivot_inv[i+1]–InputPivot_inv[i])*(1< <SCALE_FP_PREC) / (LmcsPivot[LmcsMaxBinIdx+1]–(i<<LOG2_INVERSE_INTERVAL))

[0084] Otherwise,

[0085] InvScaleCoeff[i]=(InputPivot_inv[i+1]–InputPivot_inv[i])*(1< <SCALE_FP_PREC) / (1<<LOG2_INVERSE_INTERVAL)。

[0086] In another embodiment, for an index equal to i (i=0...(((1<<bitDepth)-1)> >LOG2_INVERSE_INTERVAL)-1) snippet:

[0087] InvScaleCoeff[i]=(InputPivot_inv[i+1]–InputPivot_inv[i])*(1< <SCALE_FP_PREC) / (1<<LOG2_INVERSE_INTERVAL)

[0088] Inverse Mapping

[0089] For inverse imaging, the variable idxYInv is derived from (lumaSample >> LOG2_INVERSE_INTERVAL), where lumaSample is the luma sample to be imaged. In other words, idxYInv is derived by right-shifting lumaSample by LOG2_INVERSE_INTERVAL bits. The variable invSample is derived as follows:

[0090] invSample=InputPivot_inv[idxYInv]+(InvScaleCoeff[idxYInv]*(lumaSample–(idxYInv< <LOG2_INVERSE_INTERVAL))+(1<<(SCALE_FP_PREC-1)))> >SCALE_FP_PREC

[0091] The chroma sample invLumaSample of the inverse image is derived as follows:

[0092] invLumaSample=Clip1(invSample).

[0093] In the above formula, Clip1(x) = Clip3(0, (1 << BitDepth) – 1, x), where BitDepth is the bit depth of the base data x. Clip3(x, y, z) is defined as:

[0094]

[0095] Case where chroma scaling is enabled

[0096] In this case, the variable ChromaScaleCoeff[i] (where i = 0…((((1 << bitDepth) - 1) >> LOG2_INVERSE_INTERVAL) - 1)) is derived as follows:

[0097] if(InvScaleCoeff[i] == 0)

[0098] ChromaScaleCoeff[i] = (1 << CSCALE_FP_PREC)

[0099] else

[0100] ChromaScaleCoeff[i] = InvScaleCoeff[i].

[0101] In the above formula, CSCALE_FP_PREC is a constant value used to specify the precision. The process of looking up the chroma scaling value becomes:

[0102] 1. Look up one or more reference luminance sample values

[0103] 2. Idx = (S >> LOG2_INVERSE_INTERVAL)

[0104] 3. Chroma scaling factor (varScale) = ChromaScaleCoeff[Idx]

[0105] Method 2.2: Construct a mapping table (MapIdxRtoIdxO[]) from piece_reshaped to piece_original

[0106] The method proposes to construct a mapping table from pieceIdx_reshaped to pieceIdx_original according to the sent forward mapping curve. The luminance samples in the reshaped domain are divided into intervals of size equal to (1<<LOG2_INVERSE_INTERVAL), and each interval has its corresponding segment index in the original domain. The size of the mapping table MapIdxRtoIdxO is equal to ((1<<bitDepth)-1>>LOG2_INVERSE_INTERVAL), where bitDepth is the internal bit depth. When looking up the segment to which the luminance sample to be mapped in the inverse mapping belongs, we only need to right-shift the luminance sample to be mapped by LOG2_INVERSE_INTERVAL and look up its corresponding segment index in the original domain, which is looked up through the table MapIdxRtoIdxO (e.g., MapIdxRtoIdxO[S>>LOG2_INVERSE_INTERVAL], where S is the luminance sample to be mapped).

[0107] In another embodiment, for each segment in the reshaped domain, the corresponding index of the segment in the original domain can be derived by using the indices of most of the luminance samples in the original domain.

[0108] In one embodiment, for each segment in the reshaped domain, the corresponding index of the segment in the original domain can be derived by using the index of the first luminance sample in the original domain. In one example, the table MapIdxRtoIdxO can be constructed as follows. For luminance samples equal to (i<<LOG2_INVERSE_INTERVAL), where i = 0…((((1<<bitDepth)-1)>>LOG2_INVERSE_INTERVAL):

[0109]

[0110]

[0111] In the above formula, idxYInv is derived by setting lumaSample equal to (i<<LOG2_INVERSE_INTERVAL), and the following process is followed:

[0112]

[0113] Inverse mapping

[0114] The variable idxYInv is derived from MapIdxRtoIdxO[(lumaSample>>LOG2_INVERSE_INTERVAL)], where lumaSample is the luminance sample to be mapped.

[0115] The variable invSample is derived as follows:

[0116] invSample = InputPivot[idxYInv] + (InvScaleCoeff[idxYInv] * (lumaSample – LmcsPivot[idxYInv]) + (1 << (SCALE_FP_PREC - 1))) >> SCALE_FP_PREC

[0117] The inverse mapped luma sample invLumaSample is derived as follows:

[0118] invLumaSample = Clip1(invSample)

[0119] Case where chroma scaling is enabled

[0120] In this case, the process of looking up the chroma scaling value becomes:

[0121] 1. Look up one or more reference luma sample values

[0122] 2. Idx = MapIdxRtoIdxO[(S >> LOG2_INVERSE_INTERVAL)]

[0123] 3. Chroma scaling factor (varScale) = ChromaScaleCoeff[Idx]

[0124] Method 2.3: Perform at most N comparisons when deriving the index of a segment in the original domain

[0125] The luma samples in the reshaped domain are divided into segments with an interval equal to (1 << LOG2_INVERSE_INTERVAL). For each interval, there are N thresholds. By comparing the luma sample to be mapped with the N thresholds, the inverse index can be found. To ensure that there are only N turning points in each interval, there is bitstream consistency, i.e., there can only be N turning points in each interval. If there are more than N turning points in the same interval, the number of codewords in that interval should be adjusted.

[0126] When looking up the inverse index of a luma sample, we first look up the interval index in the reshaped domain by (luma sample >> LOG2_INVERSE_INTERVAL), and then compare the luma sample with the N thresholds in that interval to find the correct index of the segment in the original domain. In one embodiment, N is equal to 1, 2, 3, 4,... 15.

[0127] Method 3 - Simplified Inverse Mapping - When deriving the index of a segment in the original domain, perform at most N comparisons

[0128] To simplify the process for deriving the interval index, it is recommended to perform at most N comparisons when looking up the interval index. To this end, the luminance samples in the reshaped domain are divided into multiple segments of size equal to (1<<LOG2_INVERSE_INTERVAL). For each segment, there are N thresholds (LmcsSegPivot[(((1<<bitDepth)>>LOG2_INVERSE_INTERVAL)]) and a base interval index (idxYBase[((1<<bitDepth)>>LOG2_INVERSE_INTERVAL)+1]). In one embodiment, LOG2_INVERSE_INTERVAL is equal to 5 and N is equal to 1. The following embodiments are for the case where INVERSE_INTERVAL is equal to 5 and N is equal to 1.

[0129] In another embodiment, the luminance samples in the reshaped domain are divided into (1<<K) equal-sized segments. For each segment, there are N thresholds (LmcsSegPivot) and a base interval index (idxYBase).

[0130] In one embodiment, bitDepth is equal to 10, K is equal to 5, and N is equal to 1. The following embodiments are for the case where bitDepth is equal to 10, K is equal to 5 and N is equal to 1.

[0131] In another embodiment, the number of segments is a multiple of the number of intervals of the LMCS segmentation mapping curve. In one embodiment, if the number of intervals of the LMCS segmentation mapping curve is equal to X, then the number of segments is equal to M*X, where M is an integer and can be 2, 3, 4, 5, 6, 7, etc.

[0132] In one embodiment, M is equal to 2. If the number of intervals of the LMCS segmentation mapping curve is equal to 16 (=2 4 ), then the number of segments is equal to 32 (=2 5 ).

[0133] Method 3.1: Inverse Luminance Mapping with Equal-Sized Segments

[0134] For each segment, the index (idxYSeg) (calculated by dividing the luma samples to be imaged by 32) specifies the index of the current segment. In another embodiment, idxYse is derived by taking the first 5 MSBs (most significant bits) of the luma samples. In another embodiment, idxYseg is derived by dividing the luma samples to be imaged by (1<<(bitDeptn-K)). idxYBase[idxYSeg] is the interval index of the lowest imaged luma sample in the segment whose index is equal to idxYSeg. idxYBase[idxYSeg] is derived by comparing the lowest luma sample imaged in the current segment with up to 15 LmcsPivot values. LmcsSegPivot[idxYSeg] is set equal to LmcsPivot[idxYBase[idxYSeg]+1]. When looking up the interval index (idxYinv) of any luma sample (lumaSample), the following steps are applied.

[0135]

[0136] In another embodiment, when finding the interval index (ie, idxYinv) of any luma sample (ie, lumaSample), the following steps are taken.

[0137] 1. Derive idxYSeg by dividing lumaSample by 32 or by taking the first 5MSBs of the luma sample.

[0138] 2. The inverse interval index (idxYInv) is found by comparing lumaSample with LmcsSegPivot[idxYSeg].

[0139] idxYInv=(lumaSample>=LmcsSegPivot[idxYSeg])? idxYBase[idxYSeg]+1:idxYBase[idxYSeg]

[0140] When chroma scaling is enabled, steps 1 and 2 are applied to derive the interval index of the reference luma samples.

[0141] 1. The inverse brightness sample is derived as follows:

[0142] invSample=InputPivot[idxYInv]+(InvScaleCoeff[idxYInv]*(lumaSample-LmcsPivot[idxYInv])+(1<<10))>>11

[0143] 2. Crop invSample to 10-bit value.

[0144] Method 3.2: Inverse Luma Map with Equal-Sized Segments and Reduced Bit Depth

[0145] In this method, the bit depth of the inverse imaging calculation is further reduced by limiting the terms used to multiply the scaling factors to the range [-32, 31] (eg, (lumaSample-LmcsPivot[idxYInv])).

[0146] For each segment, the index (idxYSeg) (calculated by dividing the luma samples to be mapped by 32) specifies the index of the current segment. In another embodiment, idxYseg is derived by taking the first 5 MSBs of the luma samples. In another embodiment, idxYseg is derived by dividing the luma samples to be mapped by (1<<(bitDeptn-K)). idxYBase[i], where i=0...((1<<bitDepth)> >5))-1, is the interval index of the lowest image brightness sample in the segment with index equal to i. Each of idxYBase[i] is derived by comparing the lowest image brightness sample in the (i+1)th segment with up to 15 LmcsPivot values. i=0…((1<<bitDepth)> >5)-1 is derived as follows. If there is only one turning point in the (i+1)th segment, LmcsSegPivot[i] is set equal to LmcsPivot[idxYBase[i]+1]; otherwise, it is set to (i+1)*32 (or equal to i*(1<<(bitDepth-K))).

[0147] Whether there is only one turning point in the segment with index "i" can be known by comparing idxYBase[i] and idxYBase[i+1]. If the two values ​​are the same, and LmcsPivot[LmcsMaxBinIdx+1] is not in the current segment, or LmcsPivot[LmcsMaxBinIdx+1] is in the current segment and LmcsPivot[LmcsMaxBinIdx+1] is equal to i*32 (or equal to i*(1<<(bitDepth-K)), it means there is no turning point; otherwise, there is a turning point in the (i+1)th segment. i=0…((1<<bitDepth)> >5)-1 is set to be equal to the inverse image luminance sample of LmcsSegPivot[i], and its derivation follows the original design. In another embodiment, if there is no turning point in the current segment, InputSegPivot[i] is set to be equal to the inverse image luminance sample of luminance sample i*32, and its derivation follows the original design.

[0148] When looking up the interval index (idxYinv) of any luma sample (lumaSample), the following steps are applied.

[0149]

[0150] In another embodiment, when looking up the interval index (idxYinv) of any luma sample (lumaSample), the following steps are applied.

[0151] 1. Derive idxYSeg by dividing lumaSample by 32 or by taking the 5MSB of the luma sample,

[0152] 2. By comparing lumaSample and LmcsSegPivot[idxYSeg], the inverse interval index (idxYInv) can be found.

[0153] idxYInv=(lumaSample>=LmcsSegPivot[idxYSeg])? idxYBase[idxYSeg]+1:idxYBase[idxYSeg]

[0154] When chroma scaling is enabled, steps 1 and 2 are applied to derive the interval index of the reference luma samples.

[0155] When deriving inverse luma samples, the following steps are applied:

[0156] 1. The inverse brightness sample is derived as follows:

[0157] dxYSeg=(lumaSample>>5)or idxYSeg=the first 5 MSBinvSample of lumaSample=InputSegPivot[idxYSeg]+(InvScaleCoeff[idxYInv]*(lumaSample-LmcsSegPivot[idxYSeg])+(1<<10))>>11

[0158] In another embodiment, if there is no turning point in the current segment, InputSegPivot[i] is set equal to the luma sample of the inverse image of luma sample idxYSeg*32, and its derivation follows the original design, and the inverse image can be derived as follows:

[0159] invSample=InputSegPivot[idxYSeg]+(InvScaleCoeff[idxYInv]*(lumaSample–(idxYSeg<<5))+(1<<10))>>11

[0160] 2. Crop invSample to 10-bit value.

[0161] Method 3.3: Constraints to ensure that there are only N turning points in a segment

[0162] To ensure that there are only N turning points in each interval (or each segment), there is a bitstream consistency or specification constraint, where there are only N turning points in each interval. If there are more than N turning points in the same interval, the number of codewords in the interval should be adjusted.

[0163] In one embodiment, for a variable LmcsPivot[i] with i=lmcs_min_bin_idx . . . LmcsMaxBinIdx, the requirement for bitstream consistency is that (LmcsPivot[i]>>5) and (LmcsPivot[i+1]>>5) must not be the same.

[0164] In another embodiment, for a variable LmcsPivot[i] with i=lmcs_min_bin_idx..LmcsMaxBinIdx, it is required that (LmcsPivot[i]>>LOG2_INVERSE_INTERVAL) and (LmcsPivot[i+1]>>LOG2_INVERSE) must not be the same. In one embodiment, LOG2_INVERSE_INTERVAL depends on the bit depth of the luma samples and is equal to bitDepth-5. In one embodiment, LOG2_INVERSE_INTERVAL is equal to 5 and N is equal to 1.

[0165] For the values ​​with idxYSeg=0…((1<<bitDepth)> >5)-1 The detection of "more than one turning point" for a segment can be performed by the following steps:

[0166] For interval index intervalIdx=0...15

[0167] 1.idxLow=LmcsPivot[intervalIdx]>>5

[0168] 2.idxHigh=LmcsPivot[intervalIdx+1]>>5

[0169] 3. If idexLow is equal to idxHigh, LmcsPivot[intervalIdx] is not equal to LmcsPivot[intervalIdx+1], and LmcsPivot[intervalIdx] is not a multiple of 32, then "multiple turning points" are detected.

[0170] When “more than one turning point” is detected in interval i, LmcsPivot[i+1] needs to be modified as follows.

[0171]

[0172] Although the above examples are for the case where LOG2_INVERSE_INTERVAL is equal to 5 and N is equal to 1, these methods can be extended to different settings of LOG2_INVERSE_INTERVAL and N.

[0173] In another embodiment, for a variable LmcsPivot[i] with i=lmcs_min_bin_idx..LmcsMaxBinIdx, the requirement for bitstream consistency is that the first 5 MSBs of LmcsPivot[i] and LmcsPivot[i+1] must not be the same.

[0174] Method 3.4: Constraint to ensure that all luma samples in a fragment belong to at most two different intervals

[0175] In order to use only one comparison for each luma sample to be mapped when deriving the interval index, there is a bitstream conformance, or encoder specification constraint or specification constraint, to ensure that all luma samples in a segment belong to at most two different intervals.

[0176] For the variable LmcsPivot[i] where i=lmcs_min_bin_idx…LmcsMaxBinIdx, the bitstream consistency requirement is: if LmcsPivot[i] is not a multiple of 32, then (LmcsPivot[i]>>5) and (LmcsPivot[i+1]>>5) must not be the same.

[0177] In another example, in order to use only one comparison for each luma sample to be mapped when deriving the interval index, the number of codewords in each interval is limited to a specific value. The following embodiment shows bitstream consistency to ensure that there is only one turning point in each segment of size equal to OrgCW / 2.

[0178] In one embodiment, for a variable lmcsCW[i] with i=lmcs_min_bin_idx . . . LmcsMaxBinIdx, the requirement for bitstream consistency is that lmcsCW[i] should be greater than OrgCW / 2−1.

[0179] In another embodiment, for a variable lmcsCW[i] with i=lmcs_min_bin_idx . . . LmcsMaxBinIdx, the requirement for bitstream consistency is: lmcsCW[i] should be greater than OrgCW / 2+1.

[0180] Method 4 - Chroma Scaling with CIIP

[0181] In one embodiment, different modes may use reference brightness samples at different locations.

[0182] In another embodiment, for a block whose prediction process can refer to adjacent reconstructed samples, the reference luminance samples used for scaling value derivation come from adjacent reconstructed samples of the current CU or TU. For example, if the current block is in intra-frame prediction mode, the reference luminance samples are the upper left, upper, and left reference boundary samples of the current CU. Therefore, for the intra-frame sub-block partitioning prediction (ISP) mode, the chroma residual scaling value is derived using the upper left, upper, and left L-shaped boundary reconstructed samples of the current CU / TU (not the sub-partition TU). In another example, if the current block is in intra-frame prediction mode, the reference luminance sample is the upper left reference boundary sample of the current TU. Therefore, for the intra-frame ISP mode, the chroma residual scaling value is derived using the upper left, upper, and left L-shaped boundary reconstructed samples of the current TU (sub-partition). The upper left L-shaped boundary reconstructed sample can be a sample.

[0183] In another embodiment, if the current block is in an inter prediction mode other than the CIIP mode, the reference luminance sample may be the upper left luminance prediction sample of the current CU or TU. For example, if the current CU is in an inter prediction mode, but is predicted by a combined Inter Intra mode (CIIP) or other prediction method requiring adjacent reconstructed samples, the reference luminance sample may be the aforementioned reference boundary reconstructed sample of the current CU or reference CU / TU. For example, the upper left adjacent reconstructed sample may be used.

[0184] In another embodiment, if the current block is in inter prediction mode, the reference luma sample may be the upper left luma prediction sample of the current CU or TU. For example, if the current block is in CIIP mode, the reference luma sample may be the upper left luma prediction sample of the current CU or TU.

[0185] In another embodiment, if the current block is in IBC mode, the determination of the reference luma sample is the same as that in the inter prediction mode.

[0186] In another embodiment, if the current block is in IBC mode, the determination of the reference luma sample is the same as that in intra prediction mode.

[0187] Any of the aforementioned proposed methods may be implemented in an encoder and / or a decoder. For example, any of the proposed methods may be implemented in a reshaping module or a residual scaling / reshaping module of an encoder and / or a reshaping module or a residual scaling / reshaping module of a decoder. Alternatively, any of the proposed methods may be implemented as a circuit coupled to a reshaping module or a residual scaling / reshaping module of an encoder and / or a reshaping module or a residual scaling / reshaping module of a decoder, thereby providing the information required by the reshaping module or the residual scaling / shaping module.

[0188] In some embodiments, the video encoder must follow the aforementioned syntax design in order to generate a legal bitstream, and the video decoder can correctly decode the bitstream only when the parsing process complies with the aforementioned syntax design. When the syntax is skipped in the bitstream, the encoder and decoder should set the syntax value to the inferred value to ensure that the encoding and decoding results match.

[0189] Figure 4A flowchart of an exemplary encoding system for determining or deriving a pivot point of an inverse brightness mapping according to an embodiment of the present invention is shown. The steps shown in the flowchart and other subsequent flowcharts in the present disclosure can be implemented as program code that can be executed on one or more processors (e.g., one or more CPUs) on the encoder side and / or the decoder side. The steps shown in the flowchart can also be implemented based on hardware, such as one or more electronic devices or processors arranged to perform the steps in the flowchart. According to the method, a plurality of first segments are determined in step 410, wherein a range of brightness samples corresponding to brightness samples in a first signal domain is divided into a plurality of first segments, and wherein each first segment corresponds to a line piece that maps brightness samples from the first signal domain to the second signal domain. A plurality of second segments are determined in step 420, wherein a range of brightness samples corresponding to brightness samples in a second signal domain is divided into a plurality of second segments, and wherein all second segments have the same size equal to the target interval value. In step 430, one or more first pivot points are determined or derived, wherein each first pivot point is associated with a first segment, and wherein the one or more first pivot points are determined or derived such that: if the value of the target image pivot point corresponding to a first pivot point in the second signal domain is a multiple of the target interval value, the target image pivot point and the next mapping pivot point are allowed to be located in the same second segment; otherwise, the target image pivot point and the next mapping pivot point are not allowed to be located in the same second segment. In step 440, video encoding or decoding of luma samples is performed using piecewise linear mapping to map the target luma samples from the first signal domain to the second signal domain, or from the second signal domain to the first signal domain.

[0190] The flow chart shown is intended to illustrate an example of video encoding according to the present invention. One of ordinary skill in the art may modify each step, rearrange the steps, split the steps or combine the steps to practice the present invention without departing from the spirit of the present invention. In the present disclosure, specific syntax and semantics are used to illustrate examples for implementing embodiments of the present invention. One of ordinary skill in the art may practice the present invention by replacing syntax and semantics with equivalent syntax and semantics without departing from the spirit of the present invention.

[0191] The above description is presented to enable one of ordinary skill in the art to practice the present invention provided in the context of a specific application and its requirements. Various modifications to the described embodiments will be apparent to one of ordinary skill in the art, and the general principles defined herein may be applied to other embodiments. Therefore, the present invention is not intended to be limited to the specific embodiments shown and described, but is consistent with the widest range of principles and novel features disclosed herein. In the above detailed description, various specific details are shown in order to provide a thorough understanding of the present invention. However, one of ordinary skill in the art will appreciate that the present invention may be implemented.

[0192] The aforementioned embodiments of the present invention can be implemented in various hardware, software codes or a combination of the two. For example, an embodiment of the present invention can be one or more circuits integrated into a video compression chip or integrated into a video compression software to perform the program code described herein. An embodiment of the present invention can also be a program code to be executed on a digital signal processor (Digital Signal Processor, referred to as DSP) to perform the processing described herein. The present invention may also be related to many functions performed by a computer processor, a digital signal processor, a microprocessor or a field programmable gate array (fieldprogrammable gate array, referred to as FPGA). These processors can be configured to perform specific tasks according to the present invention by executing machine-readable software code or firmware code that defines the specific method embodied in the present invention. The software code or firmware code can be developed in different programming languages ​​and different formats or styles. The software code can also be compiled by different target platforms. However, different code formats, styles and languages ​​of software codes and other means of configuring codes to perform tasks according to the present invention will not depart from the spirit and scope of the present invention.

[0193] The present invention may be implemented in other specific forms without departing from the spirit or essential features of the present invention. The described examples should be considered in all respects as illustrative only and not restrictive. Therefore, the scope of the present invention is indicated by the appended claims rather than the foregoing description. All changes within the meaning and scope of the claims that are equivalent should be included within their scope.

Claims

1. A video encoding and decoding method, comprising: determining a plurality of first segments, wherein a range of luma samples corresponding to a plurality of luma samples in a first signal domain is divided into the plurality of first segments, and each first segment corresponds to a line segment mapping the plurality of luma samples from the first signal domain to a second signal domain; determining a plurality of second segments, wherein the range of luma samples corresponding to the plurality of luma samples in the second signal domain is divided into the plurality of second segments, and all of the plurality of second segments have the same size equal to a target interval value; determining or deriving one or more first pivot points, wherein each first pivot point is associated with a first segment, and the one or more first pivot points are determined or derived such that: if the value of a target image pivot point corresponding to a first pivot point in the second signal domain is a multiple of the target interval value, the target image pivot point and the next image pivot point are allowed to be located in the same second segment; Otherwise, the target image pivot point and the next mapped pivot point are not allowed to be located in the same second segment; as well as Video encoding or decoding of the plurality of luma samples is performed using piecewise linear mapping to map target luma samples from the first signal domain to the second signal domain or from the second signal domain to the first signal domain.

2. The video encoding and decoding method according to claim 1, characterized in that: The piecewise linear image is associated with the plurality of first segments, the plurality of second segments, and the one or more first pivot points.

3. The video encoding and decoding method according to claim 1, characterized in that: All of the plurality of first segments have the same first interval value.

4. The video encoding and decoding method according to claim 3, characterized in that: The first interval value corresponds to 2 raised to a power of a first predefined number, and the first predefined number is an integer.

5. The video encoding and decoding method according to claim 4, characterized in that: The first predefined number is equal to (bitDepth-4), and bitDepth represents the bit depth of the plurality of luma samples.

6. The video encoding and decoding method according to claim 1, characterized in that: The target interval value corresponds to 2 raised to a power of a second predefined number, and the second predefined number is an integer.

7. The video encoding and decoding method according to claim 6, characterized in that: The second predefined number is equal to (bitDepth-5), and bitDepth represents the bit depth of the plurality of luma samples.

8. The video encoding and decoding method according to claim 1, wherein: Each second segment has at most two mapping pivot points.

9. The video encoding and decoding method according to claim 1, wherein: The brightness sample range is equal to 0 to 1023 (including 0 and 1023), and the target interval value is equal to 32.

10. A video encoding and decoding device, the device comprising one or more electronic circuits or processors configured to: determining a plurality of first segments, wherein a range of luma samples corresponding to a plurality of luma samples in a first signal domain is divided into the plurality of first segments, and each first segment corresponds to a line segment mapping the plurality of luma samples from the first signal domain to a second signal domain; determining a plurality of second segments, wherein the range of luma samples corresponding to the plurality of luma samples in the second signal domain is divided into the plurality of second segments, and all of the plurality of second segments have the same size equal to a target interval value; determining or deriving one or more first pivot points, wherein each first pivot point is associated with a first segment, and the one or more first pivot points are determined or derived such that: if the value of a target image pivot point corresponding to a first pivot point in the second signal domain is a multiple of the target interval value, the target image pivot point and the next image pivot point are allowed to be located in the same second segment; Otherwise, the target image pivot point and the next mapped pivot point are not allowed to be located in the same second segment; as well as Video encoding or decoding of the plurality of luma samples is performed using piecewise linear mapping to map target luma samples from the first signal domain to the second signal domain or from the second signal domain to the first signal domain.

11. The video encoding and decoding device according to claim 10, characterized in that: The piecewise linear image is associated with the plurality of first segments, the plurality of second segments, and the one or more first pivot points.

12. The video encoding and decoding device according to claim 10, characterized in that: All of the plurality of first segments have the same first interval value.

13. The video encoding and decoding device according to claim 12, characterized in that: The first interval value corresponds to 2 raised to a power of a first predefined number, and the first predefined number is an integer.

14. The video encoding and decoding device according to claim 13, characterized in that: The first predefined number is equal to (bitDepth-4), and bitDepth represents the bit depth of the plurality of luma samples.

15. The video encoding and decoding device according to claim 10, characterized in that: The target interval value corresponds to 2 raised to a power of a second predefined number, and the second predefined number is an integer.

16. The video encoding and decoding device according to claim 15, characterized in that: The second predefined number is equal to (bitDepth-5), and bitDepth represents the bit depth of the plurality of luma samples.

17. The video encoding and decoding device according to claim 10, characterized in that: Each second segment has at most two mapping pivot points.

18. The video encoding and decoding device according to claim 10, characterized in that: The brightness sample range is equal to 0 to 1023 (including 0 and 1023), and the target interval value is equal to 32.

Images