Luminance mapping (LMCS) LUT extension and clipping with chroma scaling

Abstract

Methods and apparatus provide mapping functions for video encoding or decoding by replacing a pivot point of an inverse mapping function of uniform sampling with another pivot point not on the original inverse mapping function of non-uniform sampling. In one embodiment, the last pivot point of the inverse mapping function is extended based on a preservation of its curve plotted between two previous pivot points of a previous slope or based on a slope dual to an existing slope of an equivalent section of a forward mapping function. Syntax is provided for auxiliary mapping and inverse mapping and clipping.

Description

Technical Field

[0001] The general aspects described in this article belong to the field of video compression, and they aim to improve compression efficiency compared to existing video compression systems. Background Technology

[0002] The aspects described can generally be applied to encoding and decoding video content (e.g., Standard Dynamic Range (SDR), High Dynamic Range (HDR)), but it is particularly applicable to using mapping and inverse mapping processes in encoders and decoders to achieve improved encoding and decoding performance. In fact, signal mapping can be used for better encoding and decoding efficiency. Summary of the Invention

[0003] These and other disadvantages and drawbacks of the prior art are addressed by the embodiments described herein, which are methods and apparatuses for managing the trade-off between the coding efficiency and complexity provided by FRUC tools.

[0004] According to one aspect of the described embodiments, a method is provided. The method includes the steps of: determining a non-uniformly sampled inverse mapping function from the forward mapping function by swapping the coordinates of two or more pivot points of a uniformly sampled forward mapping function and interpolating points between the swapped pivot points; selecting pivot points of the inverse mapping function to be aligned on a uniformly sampled grid to refine the inverse mapping function; replacing a portion of the uniformly sampled inverse mapping function with a pivot point located at the intersection of the x-coordinate of the relevant pivot point on the uniformly sampled grid and a line defined by two non-uniform pivot points preceding the relevant (incriminated) pivot point preserved at the last slope of the non-uniformity; if the replaced pivot point is the last pivot point before saturation, then replacing the y-coordinate values ​​of one or more pivot points after the replaced pivot point with the y-coordinate values ​​of the replaced pivot point; cropping the values ​​of the uniformly sampled inverse mapping based on values; and encoding video blocks using forward and inverse mapping functions including syntax elements indicating the values.

[0005] According to another aspect of the embodiments, a second method is provided. This method includes the following steps: parsing a bitstream to obtain syntactic information indicating a mapping function; determining a non-uniformly sampled inverse mapping function from the forward mapping function by swapping the coordinates of two or more pivot points of a uniformly sampled forward mapping function and interpolating points between the swapped pivot points; selecting pivot points of the inverse mapping function to be aligned on a uniformly sampled grid to refine the inverse mapping function; replacing a portion of the uniformly sampled inverse mapping function with a pivot point located at the intersection of the x-coordinate of the relevant pivot point on the uniformly sampled grid and a line defined by two non-uniform pivot points preceding the relevant pivot point retained at the last slope of the non-uniformity; if the replaced pivot point is the last pivot point before saturation, then replacing the y-coordinate values ​​of one or more pivot points after the replaced pivot point with the y-coordinate value of the replaced pivot point; cropping the value of the uniformly sampled inverse mapping based on the value; and decoding video blocks using the forward and inverse mapping functions based on the syntactic information.

[0006] According to another aspect of the described embodiments, an apparatus is provided. The apparatus includes a memory and a processor. The processor can be configured to encode or decode a portion of a video signal using any of the methods mentioned above.

[0007] According to another general aspect of at least one embodiment, an apparatus is provided, the apparatus comprising means according to any decoding embodiment; and at least one of the following: (i) an antenna configured to receive a signal comprising a video block; (ii) a bandwidth limiter configured to limit the received signal to a frequency band comprising the video block; or (iii) a display configured to display an output representing the video block.

[0008] According to another general aspect of at least one embodiment, a non-transitory computer-readable medium is provided that contains data content generated according to any described encoding embodiment or variant.

[0009] According to another general aspect of at least one embodiment, a signal is provided that includes video data generated according to any described encoding embodiment or variant.

[0010] According to another general aspect of at least one embodiment, the bitstream is formatted to include data content generated according to any described encoding embodiment or variant.

[0011] According to another general aspect of at least one embodiment, a computer program product is provided, the computer program product including instructions that, when executed by a computer, cause the computer to perform any of the described decoding embodiments or variations.

[0012] These and other aspects, features, and advantages of this principle will become apparent from the following detailed description of exemplary embodiments taken in conjunction with the accompanying drawings. Attached Figure Description

[0013] Figure 1 The diagram illustrates the in-loop shaping of the predicted signal - top: encoder, bottom: decoder.

[0014] Figure 2 This is a schematic diagram of the construction of the forward mapping function.

[0015] Figure 3 The construction of the inverse mapping function is illustrated.

[0016] Figure 4 The diagram illustrates the homogenization of the inverse mapping function.

[0017] Figure 5 The diagram illustrates the homogenization of the inverse mapping function.

[0018] Figure 6 The diagram illustrates a common video compression scheme.

[0019] Figure 7 The diagram illustrates a common video decompression solution.

[0020] Figure 8 The diagram illustrates a flowchart of existing technologies that oppose the proposal.

[0021] Figure 9 The diagram illustrates step 0 - FwdMap inversion (InvMap with non-uniform sampling).

[0022] Figure 10 The diagram illustrates step 1 - InvMap homogenization.

[0023] Figure 11 The illustration shows step 2 - pivot point replacement (alternative).

[0024] Figure 12 The diagram illustrates step 2 - InvMap slope expansion.

[0025] Figure 13 The diagram illustrates step 3 - saturation.

[0026] Figure 14 The illustration shows step 4 - clipping based on the original InvMap range.

[0027] Figure 15 The final InvMap is illustrated.

[0028] Figure 16 An embodiment of an apparatus for encoding or decoding video using mapping or inverse mapping is illustrated.

[0029] Figure 17 An embodiment of the encoding method under the general aspects described is illustrated.

[0030] Figure 18 An embodiment of the decoding method under the general aspects described is illustrated.

[0031] Figure 19 An embodiment of the apparatus under the general aspects described is illustrated. Detailed Implementation

[0032] The embodiments described herein are in the field of video compression and are designed to improve the video compression efficiency of existing video codec schemes. They aim to improve compression efficiency compared to existing video compression systems.

[0033] Mapping aims to better utilize the distribution of sample codeword values ​​in video images. Mapping and inverse mapping can occur outside the decoding loop. In this case, mapping is applied directly to the encoder's input samples before core encoding. The inverse mapping process is applied to the decoder's output samples. Mapping and inverse mapping can also occur within the decoding loop, as proposed in the first priori approach.

[0034] This method and another method describe in-loop shaping solutions to improve encoding / decoding efficiency. In these solutions, the mapping (also known as shaping) of the luminance or chrominance signal is implemented inside the encoding / decoding loop. Shaping is applied to the predicted signal, and additional shaping and inverse shaping blocks are added to the regular encoder and decoder block diagrams in the encoder or decoder. This is in Figure 1 The diagram illustrates the application of shaping to the predicted luminance or chrominance signal. Note that shaping is documented in detail in the JVET documentation, including Luminance Mapping with Chroma Scaling (LMCS).

[0035] At the encoder, the residual signal Res_source before quantization is generated according to the following formula:

[0036] Res_source(p)=map[Orig(p)]–map[Pred(p)] (Equation 1)

[0037] Where Orig(p) is the value of the source sample (to be encoded) at position p(x,y) in the image, Pred(p) is the value of the predicted sample, Res_source(p) is the predicted residual sample before quantization, and map[.] is the integer function.

[0038] Then, Res_source(p) is transformed and quantized. The residual signal from the inverse quantization and inverse transform is denoted as Res(p).

[0039] At the decoder, the signal is reconstructed according to the following formula (e.g.) Figure 1 (as shown in (b)):

[0040] Rec(p) = invmap[map[Pred(p)] + Res(p)] (Equation 2)

[0041] Preferably, it is implemented in 3 steps:

[0042] Step 1108:

[0043] Rec0(p) = map[Pred(p)] (Equation 3) Step 1103:

[0044] Rec1(p)=Rec0(p)+Res(p) (Equation 4) Step 1104:

[0045] Rec(p) = invmap[Rec1(p)] (Equation 5)

[0046] Where Rec(p) is the value of the reconstructed sample, and invmap[.] is the inverse integer function (the inverse of map[.], such that invmap[map[x]] = x). JVET-M0427 and JVET-L0247 The method described herein performs this process on each sample of the processed block. The mapping and inverse mapping functions are global to the entire image or regions of the image. The same functions are used between different blocks of the image or regions of blocks.

[0047] Implementation of mapping functions (in) JVET-M0427 and JVET-L0247 (In Chinese) Based on a scaling table, scale[k], k = 0 to N-1, where N is a power of 2 (usually 16 or 32). Each scale applies to a range of brightness values. The range R is fixed and is a power of 2 (R = 2^32). K R is equal to the full luminance range (rangeY) divided by N. For example, for a 10-bit luminance signal, and for N=16, R=1024 / N=64=2. 6 (K=6). The mapping function FwdMap is conceptually defined as follows:

[0048] -FwdMap[0] = 0

[0049] Then, for k = 0 to N, calculate the value at each k*R index as follows:

[0050] οFwdMap[(k+1)*R]=FwdMap[k*R]+R*scale[k]

[0051] - The intermediate values ​​at indices x = (k*R+1) to ((k+1)*R-1) are obtained by linear interpolation from their surrounding values ​​FwdMap[(k+1)*R] and FwdMap[k*R]:

[0052] οFwdMap[x]=FwdMap[k*R]+(x–k*R)*(FwdMap[(k+1)*R]-FwdMap[k*R

[0053] ]) / R

[0054] Note that this process requires the LUT FwdMap to have (rangeY+1) elements, from Y=0 to rangeY, even if the actual maximum sample value Y is equal to (rangeY-1).

[0055] This process is in Figure 2 As shown in the diagram. Each size is associated with a segment of a piecewise linear mapping function. Each segment is specified over a brightness range of the same length (R).

[0056] The advantage of using a uniform sampling interval of length R (the same interval length) is that the mapping of sample values ​​Y can be easily and dynamically performed by simply accessing a lookup table of finite size (size N) using an index calculated from the value Y shifted by K bits. This process is conceptually as follows:

[0057] - Index calculation: k = Y / R = Y >> K

[0058] -Ymap = scale[k]*Y + A[k]

[0059] Where A is a pre-constructed lookup table of size (N+1), derived as follows (for k = 0 to N):

[0060] -A[k]=FwdMap[k*R]–k*R*scale[k]

[0061] JVET-M0427 The current core syntax described in [the document] is as follows:

[0062]

[0063] Basically:

[0064] `reshaper_model_min_bin_idx` represents the amount of data carried in the bitstream. Figure 2 The x-coordinate (x) of scale[i] on the scale.

[0065] `reshaper_model_delta_max_bin_idx` represents the amount of data carried in the bitstream. Figure 2The x-coordinate (x) before scale[i] on the array. This is relative to the reshaper_model_min_bin_idx encoding / decoding.

[0066] reshaper_model_bin_delta_abs_cw_prec_minus1 represents the encoding and decoding process. Figure 2 The number of bits in scale[i] or FwdMap[i].

[0067] reshape_model_bin_delta_abs_CW[i] represents Figure 2 The absolute scale[i] or FwMap[i] on the map.

[0068] reshaper_model_bin_delta_sign_CW_flag[i] represents Figure 2 The symbol for scale[i] or FwdMap[i] on the map.

[0069] Due to the syntax elements provided in the previous section, a forward mapping function representing the samples of FwdMap can be transmitted in the bitstream. FwdMap is required during the forward mapping phase (1108). In the current design, FwdMap is a uniformly sampled mapping function, which can be represented as a lookup table (LUT) – see [link to relevant documentation] Figure 2 .

[0070] The inverse mapping phase (1104) requires the inverse mapping function InvMap, which is constructed as the inverse of the forward mapping function. Mathematically, it is constructed as a symmetric function of FwdMap related to the line defined by y = x, such as... Figure 3 As shown in the image.

[0071] Generally, when FwdMap is sampled uniformly, InvMap is sampled non-uniformly by design (see...). Figure 3 (The position of the index "idx" in the middle).

[0072] For several reasons, such as when you want to carry pivot data representing the inverse mapping function (or inverse lookup table or inverse LUT) instead of pivot data representing the forward mapping function, InvMap sampling can be a uniformly sampled process (see [link to InvMap sampling]). Figure 4 This homogenization process outputs an approximate inverse mapping function, such as... Figure 4 As can be seen in the image (dashed line to solid line). This approximation can lead to a decrease in shaping performance, mainly for two reasons: the forward mapping function and the approximate mapping function are not inverses of each other, and the forward mapping function can be highly sensitive to statistics, so any deviation function of the original FwdMap or the original InvMap may distort the results.

[0073] Figure 5 The illustration shows how the homogenization of InvMap produces a deviation between the non-uniform InvMap and the uniform InvMap on a portion of the mapping curve. Typically, because the pivot point between the original InvMap and the approximate InvMap differs, the deviation appears as a different slope than the original InvMap on a given segment. This problem can occur in any part of the mapping function, but it is especially true when considering the last part of the inverse mapping function, which is subject to pruning, such as limiting the LUT precision to a finite bit depth (e.g., 10 bits). Therefore, in this last part, the inverse and forward mapping functions may not have double slopes, thus jeopardizing the invertibility of this segment.

[0074] As an example, when the InvMap is carried with uniform sampling instead of the original uniform sampling FwdMap, the sequence BQSquare shows a penalty of 0.41% brightness.

[0075] The solution is to modify the forward mapping function so that the problem is minimized using the reverse mapping function. However, the forward mapping function is calculated based on content statistics, and modifying the forward function will degrade the performance of both the forward and reverse mapping functions.

[0076] The creation (and bearing) of the inverse shaping function for uniform sampling has been proposed in previous disclosures. The intention of these described aspects is to reduce the bias or error between the forward and inverse shaping functions for uniform sampling, as MSE optimization minimizes the error.

[0077] Another proposed approach is to use an inverse shaping function with uniform sampling instead of a forward shaping function. The aforementioned problems arise when implementing one of the earlier methods.

[0078] The general aspect described in this paper proposes replacing the pivot point of the (uniformly sampled) inverse mapping function with another pivot point that is not located on the original (non-uniformly sampled) inverse mapping function; usually by extending the last pivot point of the inverse mapping function, either because of the previous slope (whose curve can be plotted between the two previous pivot points) or because the slope in the forward mapping function is preserved in pairs with the existing slope of the equivalent segment.

[0079] The pivot point for this replacement could be:

[0080] - The last pivot point of the inverse mapping function

[0081] -Signal notification in bitstream

[0082] -Outside the domain where the forward or inverse mapping function is defined

[0083] - Combined with a syntax element to trim the slope (the associated curve connects the pivot point to the pivot point just before the pivot point) to a value indicated by this syntax element.

[0084] Example - Inverse Mapping Function Extension

[0085] Basically, one idea is to replace the pivot point of a directly created uniformly sampled inverse mapping function with the pivot point obtained by directly homogenizing the non-uniformly sampled inverse mapping function, instead of using the pivot point obtained by homogenizing the non-uniformly sampled inverse mapping function, by preserving the slope of the associated curve portion of the non-uniformly sampled inverse mapping function.

[0086] Note that steps 0 and 1 provided below correspond to the prior art and are described in detail in another disclosure.

[0087] Note that this document mentions binary bits (bin) and pivot points. This document uses the same definitions as VVC Draft 5, where, for simplicity, a binary bit can be the difference between the y-coordinate values ​​of two consecutive pivot points of an integer function, i.e.,

[0088] bin[i]=reshapePivot[i+1]-reshapePivot[i]

[0089] The VVC draft 5 specification specifies the carrying of binary bits between two limits (the minimum and maximum values ​​of minBinIdx and maxbinIdx, respectively). Outside these limits, reconstructed integer functions represented on the full precision range (e.g., [0-1023] for a 10-bit integer function) are specified to be truncated toward these limits.

[0090] Step 0: Inverse FwdMap - From Uniform FwdMap to Non-Uniform InvMap

[0091] By exchanging the coordinates (x) of the pivot point of the forward mapping function k y k And perform linear interpolation between these exchanged pivot points to create a (non-uniformly sampled) inverse mapping function (InvMap) from the (uniformly sampled) forward mapping function (FwdMap).

[0092] {(x i ,y i )}FwdMap-->{(y i ,x i InvMap

[0093] Where {(x i ,y i {} represents a list of coordinates of the pivot point (x,y).

[0094] Note that:

[0095] -FwdMap is a piecewise linear monotonic (ascending) function, therefore its invertibility is guaranteed.

[0096] Saturation / clipping in FwdMap implies clipping in InvMap (see diagram below).

[0097] Step 1: InvMap homogenization

[0098] Non-uniform sampling in the InvMap is homogenized by selecting pivot points that are aligned on the sampling grid (ordinary circles). Problems arise when the values ​​interpolated between the uniformly sampled pivot points cannot be superimposed on the values ​​of the inverse mapping function representing the non-uniform sampling. Typically, the incorrect pivot point is the point just before the inverse / forward mapping function saturates.

[0099] Step 2: Expansion / replacement of a portion of the curve from a uniformly sampled InvMap (Step 1 involves the method described in the first step) noodle)

[0100] If the pivot point is the last pivot point (x-coordinate order) of the uniform InvMap or the first pivot point (x-coordinate order) of the uniform InvMap generated in step 1 with the maximum value (or the minimum value of the monotonically decreasing function) of the y-coordinate, then the pivot point is considered to be replaced (related pivot point).

[0101] The relevant pivot point is replaced with the pivot point located at the intersection of the line defined by the relevant pivot point's x-coordinate (at the uniformly sampled x-coordinate) and two non-uniform InvMap pivot points preceding the relevant pivot point (with the last slope of the non-uniform InvMap preserved) (in x-coordinate order). More generally, the relevant pivot point is replaced with the pivot point located at the intersection of the line defined by two (sequential or non-sequential) non-uniform InvMap pivot points preceding the relevant pivot point (with the last slope of the non-uniform InvMap preserved) (in x-coordinate order). An example of the process is given below:

[0102] 1) Determine the index idx_end of the last FwdMap pivot point (or the second to last, in case the last part of the FwdMap (or InvMap) curve is constant or saturated).

[0103] idx_end = FwdMaxBinIdx + 1

[0104] Where FwdMaxBinIdx represents the last pivot point information of the forward shaping function transmitted in the bitstream, or the last binary bit carried in the bitstream, see the VVC draft 5 definition of lmcs_max_bin_idx.

[0105] 2) If this index is less than the maximum number of bits / pivot points that can be carried in the bitstream, the InvMap curve is expanded using the latest slope (or previous slope) determined between two previous x-coordinate values ​​of the pivot point in the non-uniformly sampled InvMap (or equivalently, two y-coordinate values ​​of the pivot point in the uniformly sampled FwdMap, since the coordinates of the pivot points in the uniform FwdMap and the non-uniform InvMap are interchangeable versions of each other). Linear interpolations larger than the target range (e.g., 1023 for a 10-bit integer curve) are truncated.

[0106]

[0107] Let FwdReshapePivot represent the pivot point of the uniformly sampled FwdMap.

[0108] The replacement pivot can be signaled in the same syntactic way as the replaced pivot (but their signal values ​​are different because their ordinates are different).

[0109] The inverse shaper function InvMaxBinIdx, representing a uniform sample of the last, second (or third) to last binary bit / pivot point carried in the bitstream, is calculated as follows:

[0110] InvMaxBinIdx=FwdMap[idx_end]>>log2PwlInvBinLen

[0111] Where log2PwlInvBinLen = log2(reshaperMapSize / reshaperMaxBins) uses reshaperMapSize to represent the size of the FwdMap and InvMap LUT functions (usually 1024), and reshaperMaxBIs to represent the maximum number of bits or pivot points authorized to represent the mapping function (inverse or forward) (usually 16).

[0112] Once InvMap has been expanded, determining the pivot point for uniform sampling in the InvMap function, which represents uniform sampling, is straightforward:

[0113] for(i=0; i <InvMaxBinIdx+1;i++)

[0114] InvReshapePivot[i]=InvMap(i*pwlInvBinLen)

[0115] Where pwlInvBinLen = reshaperMapSize / reshaperMaxBins

[0116] It is possible that the replacement pivot point ordinate value may exceed the mapping range or precision range associated with the content (e.g., when the precision / mapping range is 10 bits, the value = 1055, i.e., [0; 1023]).

[0117] Step 3: Clip to the y-coordinate of the pivot point to be replaced

[0118] If the pivot point being replaced is the last pivot point before saturation, then the ordinate value of the pivot points after the replaced pivot point is replaced by the ordinate value of the replaced pivot point.

[0119] In terms of process, this can be written as follows:

[0120] for(i=InvMaxbinIdx+1; i<=reshaperMaxBins; i++)

[0121] InvReshapePivot[i]=invReshapePivot[InvMaxBinIdx+1]

[0122] Values ​​located between these pivot points are also clipped.

[0123] Step 4: Clip to uniform FwdMap range - FwdMapmaxBinIdx (alternative or option)

[0124] Optionally, the clipped ordinate values ​​of the uniform InvMap are stored or carried. Due to the linear interpolation between these two points, the InvMap point located just before the replacement pivot point and between the replacement pivot point is calculated, and the result (ordinate value) is clipped by the clipping value so that the replaced portion of the curve can be superimposed onto the original portion of the inverse mapping function curve. This clipping value can be identified as representing the penultimate or last binary bit or the last pivot point information of the forward mapping function (maxBinIdx - FwdMaxBinIdx of the uniformly sampled function of FwdMap). maxBinIdx can be determined by reshaper_model_delta_max_bin_idx and / or reshaper_model_min_bin_idx, as indicated in Section 7.4.6.4, Luma mapping with chroma scaling data semantics, of VVC Draft 5.

[0125] This cropping operation can be described as a two-stage process:

[0126] i) Crop the generated non-uniformly sampled function FwdMap over the range of the initially uniformly sampled FwdMap

[0127] for(i = pwlInvBinLen * (FwdMaxBinIdx + 1); i < lutSize)

[0128] FwdMap[i] = FwdMap[pwlInvBinLen * (FwdMaxbinIdx + 1)]

[0129] ii) Crop the uniformly sampled function InvMap over the mapped range of the FwdMap function

[0130] for(i = FwdMap[pwlInvBinLen * (FwdMaxBinIdx + 1)]; i < lutSize; i++)

[0131] InvMap[i] = InvMap[FwdMap[pwlInvBinLen * (FwdMaxBinIdx + 1)]]

[0132] As a variant, the value used to crop InvMap can be equal to FwdMaxBinIdx * OrgCW, where OrgCW = (1 << BitdepthY) / 16. OrgCW represents the uniform sampling step of the piecewise linear mapping function.

[0133] As a variant, the value used to crop InvMap can be equal to InvMaxBinIdx * OrgCW, where OrgCW = (1 << BitdepthY) / 16. OrgCW represents the step of the uniform sampling of the piecewise linear mapping function.

[0134] Note that step 4 can directly replace step 3.

[0135] The finally obtained uniformly sampled inverse mapping function is more consistent with the non-uniformly sampled inverse mapping function.

[0136] Syntax / Semantic Adjustment:

[0137] Step 3:

[0138] If the cropping operation is performed using the value of the replaced pivot point ordinate (i.e., step 3, without step 4), then no syntax change is required (in VVC draft 5). However, in terms of semantics, reshaper_model_delta_max_bin_idx corresponds to InvMapmaxBinIdx (instead of FwdMapmaxBinIdx).

[0139] If the signaled replaced pivot point ordinate value for authorization is greater than the precision upper bound (e.g., 1024 for 10-bit signals), then the following constraints of VVC Draft 5 (Equations (7-90))

[0140] The requirements for bitstream consistency are that the following conditions are true:

[0141]

[0142] are relaxed as follows:

[0143] - Removed from the specification text

[0144] - Or (1<<BitDepthY-1) is replaced by another threshold greater than (1<<BitDepthY-1).

[0145] For example, the threshold is set to be equal to (1<<BitDepthY-1) plus one or one and a half uniform sampling steps. For example, the uniform sampling step is equal to (1<<BitDepthY)>>4 and BitDepthY = 10.

[0146] Step 4:

[0147] If the cropping operation to be applied is the operation described in Step 4 (i.e., using FwdMapmaxBinIdx), then an additional syntax element representing this cropping value will be added to the syntax of the shaping function. In fact, when the inverse mapping function is carried, its range is also carried due to the syntax elements reshaper_model_min_bin_idx and reshaper_model_delta_max_bin_idx, but the maximum range of the forward mapping function is not transmitted. The cropping described in Step 4 (also applied by the decoder) requires knowledge of the original range of the uniform sampling forward mapping function (i.e., the maxBinIdx of FwMap). Considering the syntax for carrying the inverse mapping function described in the previous disclosure, this syntax element can be transmitted as follows:

[0148]

[0149] Where `reshaper_model_fwd_delta_delta_max_bin_idx` specifies the increment between the maximum binary index of the (uniformly sampled) forward `MaxBinIdx` and the maximum binary index of the inverse `MaxBinIdx` (when `reshaper_model_inv_direction_flag` equals 1, i.e., when the inverse mapping function is carried instead of the forward mapping function, the latter is derived from `reshaper_model_delta_max_bin_idx`). In this case, the value of the clipped value `FwdMaxBinIdx` is set to equal to...

[0150] 15-reshaper_model_delta_max_bin_idx+

[0151] reshaper_model_fwd_delta_delta_max_bin_idx.

[0152] The clipping value could also be carried directly as a syntax element of `reshaper_model_fwd_delta_max_bin_idx`, where it specifies the difference between 15 and the maximum binary bit index of the (uniformly sampled) forward mapping function. In that case, the clipping value `FwdMaxBinIdx` would be set to equal 15 - `reshaper_model_fwd_delta_max_bin_idx`.

[0153] It's possible that InvMapmaxBinIdx is carried in the additional (compared to existing techniques) syntax element presented above, rather than FwdMapmaxBinIdx. In any case, step 4 requires the transport of both InvMapmaxBinIdx and FwdMapmaxBinIdx.

[0154] Note that the shaper is also called LMCS (Luminance Map with Chroma Scaling). The term "shaper" or "shaper" can be replaced with the term "lmcs". MaxBInIdx can also be called ReshaperMaxBinIdx or LmcsMaxBinIdx.

[0155] When the mapping function is homogenized, the performance improvement is typically described when a uniformly sampled (inverse) mapping function is derived from a non-uniformly sampled (inverse) mapping function and / or a uniformly sampled (forward) mapping function.

[0156] As an example, using a combination of inverse mapping function expansion and appended pruning syntax elements (such as those described in one of the embodiments in this document) significantly improves the performance of BQSquare sequences by a penalty of 0.41% to 0.07%.

[0157] An embodiment of method 1700 under the general aspects described herein is in Figure 17 As shown in the diagram. The method begins at start block 1701 and control advances to block 1710 to determine a non-uniformly sampled inverse mapping function from a uniformly sampled forward mapping function by exchanging the coordinates of two or more pivot points of the forward mapping function and interpolating points between the exchanged pivot points. Control advances from block 1710 to block 1720 to select the pivot point of the inverse mapping function to be aligned on the uniformly sampled grid to refine the inverse mapping function. Control advances from block 1720 to block 1730 to replace a portion of the uniformly sampled inverse mapping function with a pivot point located at the intersection of the x-coordinate of the relevant pivot point in the uniformly sampled grid and the line defined by the two non-uniform pivot points preceding the relevant pivot point retained at the last slope of the non-uniformity. If the replaced pivot point is the last pivot point before saturation, then control advances from block 1730 to block 1740 to replace the y-coordinate values ​​of one or more pivot points after the replaced pivot point with the y-coordinate values ​​of the replaced pivot point. Control proceeds from block 1740 to block 1750 for inverse mapping values ​​based on value-based cropping of uniformly sampled values. Control proceeds from block 1750 to block 1760 for encoding video blocks using forward and inverse mapping functions that include syntax elements indicating the values.

[0158] An embodiment of method 1800 under the general aspects described herein is in Figure 18As shown in the diagram. The method begins at start block 1801 and control advances to block 1810 for parsing the bitstream to obtain syntactic information indicating the mapping function. Control advances from block 1810 to block 1820 for determining the non-uniformly sampled inverse mapping function from the forward mapping function by swapping the coordinates of two or more pivot points of the uniformly sampled forward mapping function and interpolating points between the swapped pivot points. Control advances from block 1820 to block 1830 for selecting the pivot point of the inverse mapping function to be aligned on the uniformly sampled grid to refine the inverse mapping function. Control advances from block 1830 to block 1840 for replacing a portion of the uniformly sampled inverse mapping function with a pivot point located at the intersection of the x-coordinate of the relevant pivot point on the uniformly sampled grid and the line defined by the two non-uniform pivot points preceding the relevant pivot point retained in the non-uniform final slope. If the pivot point being replaced is the last pivot point before saturation, then control advances from block 1840 to block 1850 to replace the y-coordinate value of one or more pivot points following the replaced pivot point with the y-coordinate value of the replacement pivot point. Control advances from block 1850 to block 1860 to trim the inverse map value of the uniformly sampled data based on the value. Control advances from block 1860 to block 1870 to decode the video block using the forward and inverse map functions based on the grammatical information.

[0159] Figure 19 An embodiment of an apparatus 1900 for compressing, encoding, or decoding video using encoding or decoding tools is shown. The apparatus includes a processor 1910 and can be interconnected to a memory 1920 via at least one port. The processor 1910 and the memory 1920 may also have one or more additional interconnects for external connection.

[0160] The processor 1910 is also configured to either insert or receive information in the bitstream, and to perform compression, encoding, or decoding using various encoding and decoding tools.

[0161] This application describes multiple aspects, including tools, features, embodiments, models, methods, etc. Many of these aspects are specifically described and are often described in a way that may sound limiting, at least for the purpose of illustrating the various features. However, this is for clarity of description and does not limit the application or scope of those aspects. In fact, all the different aspects can be combined and interchanged to provide other aspects. Moreover, these aspects can also be combined and interchanged with aspects described in previous applications.

[0162] The aspects described and considered in this application can be implemented in many different forms. Figure 6 , 7 Some embodiments are provided in 16, but other embodiments are conceived, and Figure 6 , 7The discussion in section 16 does not limit the breadth of implementations. At least one of these aspects generally relates to video encoding and decoding, and at least one other aspect generally relates to the transmission of a generated or encoded bitstream. These and other aspects can be implemented as methods, apparatus, computer-readable storage media having instructions stored thereon for encoding or decoding video data according to any of the described methods, and / or computer-readable storage media having a bitstream generated according to any of the described methods stored thereon.

[0163] In this application, the terms "reconstruction" and "decoding" are used interchangeably, as are the terms "pixel" and "sample," and the terms "image," "picture," and "frame." Typically, but not necessarily, the term "reconstruction" is used on the encoder side, while "decoding" is used on the decoder side.

[0164] This document describes various methods, and each method includes one or more steps or actions for implementing the described method. Unless the correct operation of the method requires a specific order of steps or actions, the order and / or use of specific steps and / or actions may be modified or combined.

[0165] The various methods and other aspects described in this application can be used to modify modules of the video encoder 100 and decoder 200, such as intra-frame prediction, entropy coding, and / or decoding modules (160, 360, 145, 330), as... Figure 6 and Figure 7 As shown. Furthermore, the aspects given are not limited to VVC or HEVC, and can be applied to, for example, other standards and recommendations, whether pre-existing or developed in the future, and any extensions of such standards and recommendations (including VVC and HEVC). Unless otherwise instructed or technically prohibited, the various aspects described in this application may be used individually or in combination.

[0166] Various numerical values ​​are used in this application. Specific values ​​are for illustrative purposes, and the aspects described are not limited to these specific values.

[0167] Figure 6 The encoder 100 is illustrated. Variations of this encoder 100 are considered, but for clarity, the encoder 100 is described below without depicting all anticipated variations.

[0168] Before being encoded, the video sequence can undergo pre-coding processing (101), such as applying color transformations to the input color image (e.g., a conversion from RGB 4:4:4 to YCbCr 4:2:0), or performing remapping on the input image components to obtain a signal distribution that is more resilient to compression (e.g., histogram equalization using one of the color components). Metadata can be associated with the pre-processing and appended to the bitstream.

[0169] In encoder 100, the image is encoded by encoder elements as described below. The image to be encoded is segmented (102) and processed in units such as CUs. Each unit is encoded using, for example, either an intra-frame or inter-frame mode. When a unit is encoded in intra-frame mode, it performs intra-frame prediction (160). In inter-frame mode, motion estimation (175) and compensation (170) are performed. The encoder determines (105) which of the intra-frame or inter-frame modes to use to encode the unit, and indicates the intra-frame / inter-frame decision by, for example, a prediction mode flag. For example, the prediction residual is calculated by subtracting (110) the predicted block from the original image block.

[0170] The predicted residual is then transformed (125) and quantized (130). The quantized transform coefficients, along with the motion vector and other syntax elements, are entropy encoded (145) to output a bitstream. The encoder can skip the transform and apply the quantization directly to the untransformed residual signal. The encoder can bypass both the transform and quantization, i.e., the residual is directly decoded without applying the transform or quantization process.

[0171] The encoder decodes the encoded blocks to provide a reference for further prediction. The quantized transform coefficients are dequantized (140) and inversely transformed (150) to decode the prediction residuals. The decoded prediction residuals and the predicted blocks are combined (155) to reconstruct the image blocks. An in-loop filter (165) is applied to the reconstructed image to perform, for example, deblocking / SAO (Sample Adaptive Offset) filtering to reduce coding artifacts. The filtered image is stored in a reference image buffer (180).

[0172] Figure 7 A block diagram of a video decoder 200 is shown. In decoder 200, the bitstream is decoded by decoder elements as described below. Video decoder 200 generally performs a decoding traversal that is the reverse of the encoding traversal, such as... Figure 6 As described above. Encoder 100 generally also performs video decoding as part of the encoding of video data.

[0173] Specifically, the decoder's input includes a video bitstream, which can be generated by the video encoder 100. First, the bitstream is entropy-decoded (230) to obtain transform coefficients, motion vectors, and other encoding / decoding information. Image segmentation information indicates how the image is segmented. Therefore, the decoder can segment (235) the image based on the decoded image segmentation information. The transform coefficients are dequantized (240) and inverse transformed (250) to decode the prediction residuals. The decoded prediction residuals and predicted blocks are combined (255) to reconstruct image blocks. The predicted blocks (270) can be obtained from intra-frame prediction (260) or motion-compensated prediction (i.e., inter-frame prediction) (275). An in-loop filter (265) is applied to the reconstructed image. The filtered image is stored in a reference image buffer (280).

[0174] The decoded image can also undergo post-decoding processing (285), such as inverse color transformation (e.g., conversion from YCbCr 4:2:0 to RGB 4:4:4) or the inverse of the remapping process performed in pre-encoding processing (101). Post-decoding processing can utilize metadata derived in pre-encoding processing and signaled in the bitstream.

[0175] Figure 16 A block diagram illustrating an example system implementing various aspects and embodiments thereof is shown. System 1000 can be implemented as a device including the various components described below and configured to perform one or more of the aspects described in this document. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. Elements of system 1000 may be implemented individually or in combination in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one embodiment, the processing and encoder / decoder elements of system 1000 are distributed across multiple ICs and / or discrete components. In various embodiments, system 1000 is communicatively coupled to one or more other systems or other electronic devices via, for example, a communication bus or through dedicated input and / or output ports. In various embodiments, system 1000 is configured to implement one or more of the aspects described in this document.

[0176] System 1000 includes at least one processor 1010 configured to execute instructions loaded therein for implementing various aspects, such as those described in this document. Processor 1010 may include embedded memory, input / output interfaces, and various other circuitry known in the art. System 1000 includes at least one memory 1020 (e.g., a volatile memory device and / or a non-volatile memory device). System 1000 includes a storage device 1040, which may include non-volatile memory and / or volatile memory, including but not limited to electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, disk drives, and / or optical disk drives. As a non-limiting example, storage device 1040 may include internal storage devices, additional storage devices (including removable and non-removable storage devices), and / or network-accessible storage devices.

[0177] System 1000 includes an encoder / decoder module 1030 configured to, for example, process data to provide encoded or decoded video, and the encoder / decoder module 1030 may include its own processor and memory. The encoder / decoder module 1030 represents one or more modules that can be included in a device to perform encoding and / or decoding functions. As is known, a device may include one or both encoding and decoding modules. Furthermore, the encoder / decoder module 1030 may be implemented as a separate element of system 1000, or it may be incorporated into processor 1010 as a combination of hardware and software known to those skilled in the art.

[0178] Program code that can be loaded onto processor 1010 or encoder / decoder 1030 to execute the various aspects described in this document can be stored in storage device 1040 and subsequently loaded onto memory 1020 for execution by processor 1010. According to various embodiments, during execution of the processes described in this document, one or more of processor 1010, memory 1020, storage device 1040, and encoder / decoder module 1030 can store one or more of a variety of items. Such stored items may include, but are not limited to, input video, decoded video or a portion of decoded video, bitstreams, matrices, variables, and intermediate or final results of equations, formulas, operations, and operational logic.

[0179] In some embodiments, the memory within processor 1010 and / or encoder / decoder module 1030 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, external memory (e.g., the processing device may be processor 1010 or encoder / decoder module 1030) is used for one or more of these functions. External memory may be memory 1020 and / or storage device 1040, such as volatile memory and / or non-volatile flash memory. In several embodiments, external non-volatile flash memory is used to store, for example, the operating system of a television. In at least one embodiment, a fast external dynamic volatile memory (such as RAM) is used as working memory for video decoding and decoding operations, such as for MPEG-2 (MPEG stands for Moving Picture Experts Group, MPEG-2 also refers to ISO / IEC 13818, and 13818-1 is also called H.222, and 13818-2 is also called H.262), HEVC (HEVC stands for High Efficiency Video Coding, also called H.265 and MPEG-H Part 2) or VVC (Various Video Coding, a new standard developed by the Joint Video Experts Group JVET).

[0180] As indicated in box 1130, inputs to the components of system 1000 can be provided through various input devices. Such input devices include, but are not limited to, (i) RF sections that receive radio frequency (RF) signals transmitted over the air, for example by broadcasters, (ii) component input terminals (or sets of COMP input terminals), (iii) universal serial bus (USB) input terminals, and / or (iv) high-definition multimedia interface (HDMI) input terminals. Figure 16 Other examples not shown include composite video.

[0181] In various embodiments, the input device of block 1130 has associated corresponding input processing elements, as known in the art. For example, the RF section may be associated with elements suitable for: (i) selecting a desired frequency (also known as selecting a signal, or limiting a signal to a frequency band), (ii) down-converting the selected signal, (iii) further limiting the frequency band to a narrower band to select (e.g.,) a signal band that may be referred to as a channel in some embodiments), (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired data packet stream. The RF section of various embodiments includes one or more elements performing these functions, such as frequency selectors, signal selectors, band limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF section may include a tuner performing various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., intermediate frequency or near-baseband frequency) or baseband. In one set-top box embodiment, the RF section and its associated input processing elements receive RF signals transmitted over a wired (e.g., cable) medium and perform frequency selection by filtering, down-converting, and re-filtering to a desired frequency band. Various embodiments rearrange the order of the aforementioned (and other) components, remove some of these components, and / or add other components that perform similar or different functions. Adding components may include inserting components between existing components, such as, for example, inserting amplifiers and analog-to-digital converters. In various embodiments, the RF section includes an antenna.

[0182] Furthermore, the USB and / or HDMI terminals may include corresponding interface processors for connecting the system 1000 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Reed-Solomon error correction) may be implemented, for example, within a separate input processing IC or within the processor 1010. Similarly, various aspects of USB or HDMI interface processing may be implemented, as needed, within a separate interface IC or within the processor 1010. The demodulated, error-corrected, and demultiplexed stream is provided to various processing elements, including, for example, the processor 1010, and an encoder / decoder 1030 operating in conjunction with memory and storage elements, to process the data stream as needed for presentation on an output device.

[0183] Various components of the system 1000 can be provided within an integrated housing. Within the integrated housing, various components can be interconnected and data can be transferred between them using suitable connection arrangements (e.g., internal buses known in the art, including inter-IC (I2C) buses, wiring, and printed circuit boards).

[0184] System 1000 includes a communication interface 1050 that enables communication with other devices via a communication channel 1060. The communication interface 1050 may include, but is not limited to, a transceiver configured to transmit and receive data on the communication channel 1060. The communication interface 1050 may include, but is not limited to, a modem or network interface card (NIC), and the communication channel 1060 may be implemented, for example, within a wired and / or wireless medium.

[0185] In various embodiments, wireless networks such as Wi-Fi networks, such as IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers), are used to stream or otherwise provide data to system 1000. In these embodiments, Wi-Fi signals are received via a communication channel 1060 and a communication interface 1050 suitable for Wi-Fi communication. The communication channel 1060 in these embodiments is typically connected to an access point or router that provides access to external networks, including the Internet, to allow streaming applications and other over-the-air communications. Other embodiments use a set-top box to provide streaming data to system 1000, delivering data via an HDMI connection of input block 1130. Still other embodiments use an RF connection of input block 1130 to provide streaming data to system 1000. As indicated above, various embodiments provide data in a non-streaming manner. Furthermore, various embodiments use wireless networks other than Wi-Fi, such as cellular networks or Bluetooth networks.

[0186] System 1000 can provide output signals to various output devices, including display 1100, speaker 1110, and other peripheral devices 1120. Display 1100 in various embodiments includes one or more of, for example, a touchscreen display, an organic light-emitting diode (OLED) display, a flexible display, and / or a foldable display. Display 1100 can be used in televisions, tablets, laptops, cellular phones (mobile phones), or other devices. Display 1100 can also be integrated with other components (e.g., as in a smartphone) or separate (e.g., an external monitor for a laptop computer). In various examples of embodiments, other peripheral devices 1120 include one or more of a standalone digital video disc (or digital multifunction disc) (DVR, for both terms), a disc player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 1120 that provide functionality based on the output of system 1000. For example, a disc player performs the function of playing the output of system 1000.

[0187] In various embodiments, control signals communicate between system 1000 and display 1100, speaker 1110, or other peripheral devices 1120 using signaling such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols enabling device-to-device control, with or without user intervention. Output devices can be coupled to system 1000 via dedicated connection communication through their respective interfaces 1070, 1080, and 1090. Alternatively, output devices can be connected to system 1000 via communication interface 1050 using communication channel 1060. Display 1100 and speaker 1110 can be integrated into a single unit with other components of system 1000 in electronic devices such as, for example, televisions. In various embodiments, display interface 1070 includes a display driver, such as, for example, a timing controller (TCon) chip.

[0188] For example, if the RF portion of input 1130 is part of a separate set-top box, then display 1100 and speaker 1110 can alternatively be separate from one or more of the other components. In various embodiments where display 1100 and speaker 1110 are external components, output signals can be provided via dedicated output connections, including, for example, HDMI ports, USB ports, or COMP outputs.

[0189] The embodiments can be implemented by computer software implemented by processor 1010, or by hardware, or by a combination of hardware and software. As a non-limiting example, the embodiments can be implemented by one or more integrated circuits. Memory 1020 can be of any type suitable for the technical environment and can be implemented using any suitable data storage technology (as a non-limiting example, such as optical storage devices, magnetic storage devices, semiconductor-based storage devices, fixed memory, and removable memory). As a non-limiting example, processor 1010 can be of any type suitable for the technical environment and can include one or more of microprocessors, general-purpose computers, special-purpose computers, and processors based on multi-core architectures.

[0190] Various implementations involve decoding. As used herein, “decoding” can encompass all or part of a process, such as performing on a received encoded sequence to produce a final output suitable for display. In various embodiments, such processes include one or more processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various embodiments, such processes also, or alternatively, include processes performed by a decoder of the various embodiments described herein.

[0191] As a further example, in one embodiment, "decoding" refers only to entropy decoding; in another embodiment, "decoding" refers only to differential decoding; and in yet another embodiment, "decoding" refers to a combination of entropy decoding and differential decoding. It will be clear, and those skilled in the art, whether the phrase "decoding process" is intended to specifically refer to a subset of the operations or generally to the broader decoding process, based on the specific context of the description.

[0192] Various implementations involve encoding. In a manner similar to the discussion above regarding “decoding,” the term “encoding” as used herein can encompass all or part of a process performed on an input video sequence to produce an encoded bitstream. In various embodiments, such processes include one or more processes typically performed by an encoder, such as segmentation, differential coding, transform, quantization, and entropy coding. In various embodiments, such processes also, or alternatively, include processes performed by an encoder of the various embodiments described herein.

[0193] As another example, in one embodiment, “encoding” refers only to entropy encoding; in another embodiment, “encoding” refers only to differential encoding; and in yet another embodiment, “encoding” refers to a combination of differential and entropy encoding. It will be apparent, and believed that those skilled in the art, whether the phrase “encoding process” is intended to specifically refer to a subset of operations or to refer more broadly to a wider encoding process, based on the context of the particular description.

[0194] Note that the grammatical elements used in this article are descriptive terms. Therefore, the use of other grammatical element names is not excluded.

[0195] When a figure is presented as a flowchart, it should be understood that a block diagram of the corresponding apparatus is also provided. Similarly, when a figure is presented as a block diagram, it should be understood that a flowchart of the corresponding method / process is also provided.

[0196] Various implementations may refer to parametric models or rate-distortion optimization. In particular, during the encoding process, constraints on computational complexity are often given, typically considering a balance or trade-off between rate and distortion. This can be measured by rate-distortion optimization (RDO) or by least mean square (LMS), mean absolute error (MAE), or other such measures. Rate-distortion optimization is typically formulated as minimizing a rate-distortion function, which is a weighted sum of rate and distortion. There are various approaches to solving the rate-distortion optimization problem. For example, these approaches can be based on extensive testing of all encoding options, including all considered modes or decoding parameter values, and a complete evaluation of their decoding costs and the associated distortion of the reconstructed signal after decoding and decoding. Faster methods can also be used to save encoding complexity, particularly by calculating approximate distortion based on predicting or predicting the residual signal rather than the reconstructed signal. A hybrid of these two approaches can also be used, such as by using approximate distortion only for some possible encoding options and full distortion for others. Other methods evaluate only a subset of possible encoding options. More generally, many methods employ any of a variety of techniques to perform optimization, but optimization is not necessarily a complete assessment of both decoding cost and associated distortion.

[0197] The implementations and aspects described herein can be implemented, for example, as methods or processes, apparatus, software programs, data streams, or signals. Even if discussed only in a single form of implementation (e.g., discussed only as a method), implementations of the discussed features can also be implemented in other forms (e.g., apparatus or program). Apparatus can be implemented, for example, in suitable hardware, software, and firmware. Methods can be implemented, for example, in a processor, which generally refers to a processing device, including, for example, a computer, microprocessor, integrated circuit, or programmable logic device. Processors also include communication devices, such as, for example, computers, mobile phones, portable / personal digital assistants (“PDAs”), and other devices that facilitate information communication between end users.

[0198] References to "an embodiment" or "an embodiment" or "an implementation" or "implementation," and other variations thereof, refer to specific features, structures, characteristics, etc., described in connection with that embodiment, which are included in at least one embodiment. Therefore, the appearance of the phrases "in one embodiment" or "in an embodiment" or "in one implementation" or "in one implementation," and any other variations throughout this application, does not necessarily refer to all the same embodiment.

[0199] Furthermore, this application may refer to "determining" various types of information. Determining information may include, for example, one or more of the following: estimated information, calculated information, predicted information, or information retrieved from memory.

[0200] Additionally, this application may refer to "accessing" various types of information. Accessing information may include, for example, one or more of the following: receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0201] Furthermore, this application may refer to "receiving" various types of information. Like "accessing," receiving is a broad term. Receiving information may include one or more of the following: for example, accessing information or retrieving information (e.g., from memory). Additionally, "receiving" generally refers to actions such as storing information, processing information, sending information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information, in one or more ways.

[0202] It should be recognized that, for example, the use of any of the following " / ", "and / or", and "...at least one of A and B" in the context of "A / B", "A and / or B", and "at least one of A and B" is intended to cover selecting only the first listed option (A), or only the second listed option (B), or both options (A and B). As another example, in the context of "A, B, and / or C" and "at least one of A, B, and C", this wording is intended to cover selecting only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A, B, and C). This can be extended to multiple listed items, as will be apparent to those skilled in the art and related fields.

[0203] Moreover, as used herein, the word “signal” specifically refers to instructing the corresponding decoder to do something. For example, in some embodiments, the encoder signals a particular one of multiple transforms, encoding modes, or flags. In this way, in embodiments, the same transform, parameter, or mode is used on both the encoder and decoder sides. Thus, for example, the encoder can transmit (explicit signaling) a specific parameter to the decoder so that the decoder can use the same specific parameter. Conversely, if the decoder already has the specific parameter as well as other parameters, then signaling can be used without transmission (implicit signaling) to simply allow the decoder to know and select the specific parameter. Bit savings are achieved in various embodiments by avoiding the transmission of any actual functionality. It should be recognized that signaling can be accomplished in a variety of ways. For example, in various embodiments, one or more syntax elements, flags, etc., are used to signal information to the corresponding decoder. Although the verb form of the word “signal” has been referred to above, the word “signal” can also be used as a noun herein.

[0204] As will be apparent to those skilled in the art, implementations can generate various signals that are formatted to carry, for example, information that can be stored or transmitted. The information may include, for example, instructions for performing a method or data generated by one of the described implementations. For example, the signal may be formatted to carry a bitstream of the described embodiment. Such a signal may be formatted as, for example, electromagnetic waves (e.g., using the radio frequency portion of the spectrum) or baseband signals. Formatting may include, for example, encoding the data stream and modulating a carrier wave with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, signals can be transmitted via a variety of different wired or wireless links. The signal may be stored on a processor-readable medium.

[0205] We have described several embodiments spanning various claim classes and types. Features of these embodiments may be provided individually or in any combination. Furthermore, across various claim classes and types, embodiments may include one or more of the following features, means, or aspects, individually or in any combination:

[0206] • Includes bitstreams or signals of one or more of the described syntax elements or their variants.

[0207] • Includes bitstreams or signals that convey information according to any of the described embodiments.

[0208] • Create and / or transmit and / or receive and / or decode according to any of the embodiments described.

[0209] • Methods, processes, apparatus, media for storing instructions, media for storing data or signals according to any of the described embodiments.

[0210] • Insert elements into the signaling syntax elements that enable the decoder to determine the encoding mode in a manner corresponding to that used by the encoder.

[0211] • Creating and / or transmitting and / or receiving and / or decoding bitstreams or signals including one or more of the described syntax elements or variations thereof.

[0212] • A TV, set-top box, cellular phone, tablet computer or other electronic device that performs one or more transformation methods according to any of the described embodiments.

[0213] A TV, set-top box, cellular phone, tablet computer, or other electronic device that performs one or more transformation methods according to any of the described embodiments to determine and display (e.g., using a monitor, screen, or other type of display) the resulting image.

[0214] A TV, set-top box, cellular phone, tablet computer, or other electronic device that selects, bandwidth limits, or tunes (e.g., uses a tuner) a channel to receive signals including encoded images and performs one or more transformation methods according to any of the described embodiments.

[0215] • A TV, set-top box, cellular phone, tablet computer, or other electronic device that receives signals including encoded images over the air (e.g., using an antenna) and performs one or more transformation methods.

Claims

1. A method of encoding comprising: determining a non-uniformly sampled inverse mapping function from a uniformly sampled forward mapping function by swapping the coordinates of two or more pivot points of the forward mapping function and interpolating points between the swapped pivot points; selecting pivot points of the inverse mapping function to be aligned on a grid of uniform sampling to refine the inverse mapping function; replacing a relevant pivot point of the uniformly sampled inverse mapping function with a pivot point located at an intersection between a line defined by the relevant pivot point horizontal coordinate of the uniform sampling grid and a line defined by two non-uniform pivot points of the inverse mapping function before the relevant pivot point; if the replaced pivot point is the last pivot point before saturation, replacing the vertical coordinate values of one or more pivot points after the replaced pivot point with the vertical coordinate value of the replacing pivot point; clipping values of the uniformly sampled inverse mapping based on a clipping value; and, encoding a video block using the forward and inverse mapping functions including syntax elements indicating the values.

2. The method of claim 1, wherein the replaced pivot point is the second to last in case the last portion of the forward mapping function curve or the inverse mapping function curve is constant.

3. The method of claim 1, wherein clipping is performed on values greater than a target range.

4. The method of claim 3, wherein the target range is 1023 for a 10-bit reshaping curve.

5. The method of claim 1, wherein a previous slope is used to extend the mapping curve.

6. The method of claim 1, wherein a previous slope is used to extend the inverse mapping curve.

7. The method of claim 1, wherein the value used to clip the inverse mapping function is equal to FwdMaxBinIdx * OrgCW, where OrgCW = (1 « BitdepthY) / 16, OrgCW represents the uniformly sampled step size of the piecewise linear mapping function.

8. The method of claim 1, wherein a syntax element specifies an incremental value between the maximum bin index of the uniformly sampled forward MaxBinIdx and the maximum bin index of the inverse MaxBinIdx.

9. An apparatus for encoding comprising: a processor configured to perform: determining a non-uniformly sampled inverse mapping function from a uniformly sampled forward mapping function by swapping the coordinates of two or more pivot points of the forward mapping function and interpolating points between the swapped pivot points; selecting pivot points of the inverse mapping function to be aligned on a grid of uniform sampling to refine the inverse mapping function; replacing a relevant pivot point of the uniformly sampled inverse mapping function with a pivot point located at an intersection between a line defined by the relevant pivot point horizontal coordinate of the uniform sampling grid and a line defined by two non-uniform pivot points of the inverse mapping function before the relevant pivot point; if the replaced pivot point is the last pivot point before saturation, replacing the vertical coordinate values of one or more pivot points after the replaced pivot point with the vertical coordinate value of the replacing pivot point; clipping values of the uniformly sampled inverse mapping based on a clipping value; and, encoding a video block using the forward and inverse mapping functions including syntax elements indicating the values. encoding a video block using the forward mapping function and the inverse mapping function including a syntax element indicating the value.

10. The apparatus of claim 9, wherein in case the last portion of the forward mapping function curve or the inverse mapping function curve is constant, the replaced pivot point is the second last.

11. The apparatus of claim 9, wherein clipping is performed on values greater than a target range.

12. The apparatus of claim 11, wherein the target range is 1023 for a 10-bit reshaping curve.

13. The apparatus of claim 9, wherein a previous slope is used to extend the mapping curve.

14. The apparatus of claim 9, wherein a previous slope is used to extend the inverse mapping curve.

15. The apparatus of claim 9, wherein the value for clipping the inverse mapping function is equal to FwdMaxBinIdx * OrgCW, where OrgCW = (1 « BitdepthY) / 16, OrgCW represents the uniform sampling step size of the piecewise linear mapping function.

16. The apparatus of claim 9, wherein a syntax element specifies the delta value between the maximum bin index of the forward MaxBinIdx and the maximum bin index of the inverse MaxBinIdx of the uniform sampling.

17. A decoding method comprising: parsing a bitstream to obtain syntax information indicating mapping functions; determining an inverse mapping function of non-uniform sampling from a forward mapping function of uniform sampling by swapping coordinates of two or more pivot points of the forward mapping function and interpolating points between the swapped pivot points; selecting pivot points of the inverse mapping function to be aligned on a grid of uniform sampling to refine the inverse mapping function; replacing a relevant pivot point of the inverse mapping function of uniform sampling with a pivot point located at an intersection of a line defined by a relevant pivot point horizontal coordinate of the uniform sampling grid and a line defined by two non-uniform pivot points of the inverse mapping function before the relevant pivot point; if the replaced pivot point is the last pivot point before saturation, replacing vertical coordinate values of one or more pivot points after the replaced pivot point with a vertical coordinate value of the replaced pivot point; clipping values of the inverse mapping of uniform sampling based on a clipping value; and, decoding a video block using the forward mapping function and the inverse mapping function based on the syntax information.

18. The method of claim 17, wherein in case the last portion of the forward mapping function curve or the inverse mapping function curve is constant, the replaced pivot point is the second last.

19. The method of claim 17, wherein clipping is performed on values greater than a target range.

20. The method of claim 19, wherein the target range is 1023 for a 10-bit reshaping curve.

21. The method of claim 17, wherein a previous slope is used to extend the mapping curve.

22. The method of claim 17, wherein a previous slope is used to extend the inverse mapping curve.

23. The method of claim 17, wherein the value to clip the inverse mapping function is equal to FwdMaxBinIdx * OrgCW, where OrgCW = (1 « BitdepthY) / 16, OrgCW represents a uniform sampling step size of the piecewise linear mapping function.

24. The method of claim 17, wherein a syntax element specifies an increment value between a maximum bin index of the uniform sampled forward MaxBinIdx and a maximum bin index of the inverse MaxBinIdx.

25. A decoding apparatus comprising: a processor configured to perform: parsing a bitstream to obtain syntax information indicating mapping functions; determining, from a uniform sampled forward mapping function, a non-uniform sampled inverse mapping function by swapping coordinates of two or more pivot points of the forward mapping function and interpolating points between the swapped pivot points; selecting pivot points of the inverse mapping function to be aligned on a uniform sampling grid to refine the inverse mapping function; replacing a relevant pivot point of the uniform sampled inverse mapping function with a pivot point located at an intersection of a line defined by a relevant pivot point horizontal coordinate of the uniform sampling grid and a line defined by two non-uniform pivot points of the inverse mapping function before the relevant pivot point; if the replaced pivot point is a last pivot point before saturation, replacing vertical coordinate values of one or more pivot points after the replaced pivot point with a vertical coordinate value of the replaced pivot point; clipping values of the uniform sampled inverse mapping based on a clipping value; and, decoding a video block using the forward mapping function and the inverse mapping function based on the syntax information.

26. The apparatus of claim 25, wherein the replaced pivot point is a second to last in case a last portion of the forward mapping function curve or the inverse mapping function curve is constant.

27. The apparatus of claim 25, wherein clipping is performed on values greater than a target range.

28. The apparatus of claim 27, wherein the target range is 1023 for a 10-bit reshaping curve.

29. The apparatus of claim 25, wherein a previous slope is used to extend the mapping curve.

30. The apparatus of claim 25, wherein a previous slope is used to extend the inverse mapping curve.

31. The apparatus of claim 25, wherein the value to clip the inverse mapping function is equal to FwdMaxBinIdx * OrgCW, where OrgCW = (1 « BitdepthY) / 16, OrgCW represents a uniform sampling step size of the piecewise linear mapping function.

32. The apparatus of claim 25, wherein a syntax element specifies an increment value between a maximum bin index of the uniform sampled forward MaxBinIdx and a maximum bin index of the inverse MaxBinIdx.

33. An electronic device, comprising: the apparatus of any of claims 9-16 and 25-32; and a communication interface configured to receive the bitstream. At least one of: (i) an antenna configured to receive a signal, the signal comprising the video block, (ii) a band limiter configured to limit the received signal to a frequency band comprising the video block, and (iii) a display configured to display an output representative of the video block.

34. A computer program product comprising instructions which, when executed by a computer, cause the computer to carry out the method of any one of claims 1 to 8 and 17 to 24.

Images