Quantization of residuals in video coding

Abstract

A method for decoding an encoded signal is provided, wherein the encoded signal is received within a bitstream, and the method comprises the steps of: receiving a quantized data set associated with the encoded signal; determining at least one dequantization parameter; and dequantizing the quantized data at least partially based on at least one dequantization parameter.

Description

Technology Field

[0001] The present invention relates to data compression, particularly to methods for compressing and decoding image and video signals. Data compression may include, but is not limited to, acquiring, deriving, encoding, outputting, receiving, decoding, and reconstructing data encoded by a hierarchical (tier-based) coding format, wherein in a hierarchical coding format, a video signal is encoded into an echelon of data (e.g., a tier or tier) and subsequently decoded from the tier at a higher quality level. Different tiers of the signal may also be encoded according to different coding formats. Background Technology

[0002] Hybrid backward compatible coding techniques have been previously proposed, for example, in WO 2014 / 170819 and WO 2018 / 046940, the contents of which are incorporated by reference into this application.

[0003] These publications propose a method for parsing a data stream into a first part of encoded data and a second part of encoded data, implementing a first decoder to decode the first part of encoded data into a first rendition of a signal; implementing a second decoder to decode the second part of encoded data into reconstructed data—the reconstructed data specifies a method for modifying the first rendition of a signal—and generating a second rendition of a signal by applying the reconstructed data to the first rendition of a signal.

[0004] These publications further propose that a set of residual elements can be used to reconstruct the rendition of a first time sample of a signal. A set of spatial-temporal correlation elements associated with the first time sample is generated. The set of spatial-temporal correlation elements represents the degree of spatial correlation between multiple residual elements, and the degree of temporal correlation between the second reference data based on the rendition of a second time sample of the signal and the first reference data based on the rendition. The set of spatial-temporal correlation elements is used to generate output data. As mentioned, the set of residuals is encoded to reduce the overall data size.

[0005] Encoding applications typically use quantization operations. This compression process, in which each of one or more ranges of data values ​​is compressed into a single value, reduces the number of distinct values ​​in a set of video data, thereby making the data more compressible. In this way, quantization is useful for converting signals into quanta so that specific variables in some videos can take only specific discrete sizes. Typically, a video codec divides visual data in the form of video frames into discrete blocks of typically predetermined size or number of pixels. Then, a transformation is typically applied to the blocks to represent the visual data in terms of the sum of frequency components. This transformed data can then be pre-multiplied by a quantization scale code, then division by an element by a quantization matrix, each element is transformed by the matrix elements, and the output element of the division of the pre-multiplied elements is then rounded. Divisors, that is, the processing of different transformed elements using different elements of the quantization matrix, are typically used to enable more data or resolution to be more effectively allocated to these frequency elements—which have a greater impact on the visual appearance of the video to the viewer—than to the less perceptible components.

[0006] When the signal is reconstructed, optimization is pursued to further reduce the overall data size while balancing the goals of optimizing processing speed and complexity without compromising the overall impression on the user.

[0007] Most data compression methods include a quantization stage typically applied to a domain of invertible transformed coefficients with respect to display setting coordinates (e.g., luminance and chrominance values, RGB values, etc.). When quantized data is received by a decoding module, the decoding module must be able to implement a dequantization process to reconstruct the original coefficients.

[0008] In the following description, all embodiments will be described with reference to video signals for the sake of simplification. However, it should be understood that the same embodiments are applicable to other types of data with necessary modifications. Furthermore, for the sake of simplification, coding and decoding algorithms / processes are referred to as "codecs."

[0009] Typical codecs operate using a single-layer block-based approach, whereby an original signal is processed using multiple coding tools to generate an encoded signal, which can then be reconstructed by a corresponding decoding process. Such typical codecs include, but are not limited to, codecs currently under development by international standards organizations such as MPEG-2, AVC / H.264, HEVC / H.265, VP8, VP9, ​​AV1, and industry consortia such as MPEG / ISO / ITU and the Alliance for Open Media (AoM).

[0010] There are also codec classes that operate using a multi-layer block-based approach. These codecs, known in the industry as "scalable," typically encode (and decode) upper layers by downsampling the source signal and then replicating the operations performed by a single-layer block-based approach across multiple acquired layers by reusing as much information as possible from the lower layers. Importantly, these scalable codecs are intended to provide scalability features to operators, not only in that it is necessary to ensure that the quality of the down-decoded signal (e.g., a lower-resolution signal) satisfies quality requirements for existing services, but also in that it ensures the quality of the un-scaled decoded signal (e.g., a higher-resolution signal) is similar to that generated by the corresponding single-layer codec.

[0011] An example of such a codec is SVC, a scalable version of AVC (see, for example, the literature ["The Scalable Video Coding Extension of the H.264 / AVC Standard", H. Schwarz and M. Wien, IEEE Signal Processing Magazine, March 2008]), where each scalable layer is processed using the same AVC-based process, and the upper layer receives as much information as possible from the lower layer (e.g., inter-layer prediction including residual information and motion information), which is used for encoding the upper layer to reduce the information encoded in the upper layer. Conversely, for decoding, the SVC decoder needs to receive various overhead information as well as decode the lower layer so that it can decode the upper layer.

[0012] Another example is SHVC, a scalable version of HEVC (see, for example, the literature ["Overview of SHVC: Scalable Extensions of the High Efficiency Video Coding Standard", J. Boyce, Y. Ye, J. Chen and A. Ramasubramonian, IEEE Trans. On Circuits and Systems for Video Technology, Vol. 26, No. 1, Jan 2016]). Similar to SVC, SHVC also uses the same HEVC-based process for the upper layer, but this allows the lower layer to use either AVC or HEVC. Crucially, in SHVC, the upper layer also receives as much information as possible from the lower layer during encoding to reduce the encoded information in the upper layer (e.g., interlayer processing including motion information, the lower layer upsampled as an additional reference picture for upper layer coding). Also, similar to SVC, the SHVC decoder needs to receive various overhead information as well as decode the lower layer to enable decoding of the upper layer.

[0013] Recently, a new multi-layered non-block-based approach has been developed—for example, U.S. Patents No. US8,977,065, US8,948,248, US8,711,943, US9,129,411, US8,531,321, US9,510,018, US9,300,980, and US9,626,772, and PCT applications PCT / EP2013 / 059833, PCT / EP2013 / 059847, PCT / EP2013 / 059880, PCT / EP2013 / 059853, PCT / EP2013 / 059885, PCT / EP2013 / 059886, and Refer to PCT / IB2014 / 060716, all of which are incorporated by reference into this application. This new approach is currently being developed as an international standard by MPEG / ISO as MPEG-5 Part 2 LCEVC (Low Complexity Enhancement Video Coding).

[0014] Unlike scalable codecs, this new approach is completely agnostic to the codec used to encode the sublayer. This is because, as illustrated in FIG. 1, the upper layer can be decoded without any information about the sublayer. The decoder receives five streams generated by the encoder. The five streams include: a first encoded stream (encoded base) is generated by feeding a downsampled version of the input video to a base codec (e.g., AVC, HEVC, or any other codec); a second encoded stream (L1 coefficient layer) is generated by processing the residuals obtained by taking the difference between the reconstructed base codec video and the downsampled version of the input video (L1 residual); and a third encoded stream (L0 coefficient layer) is generated by processing the residuals obtained by taking the difference (L0 residual) between the upsampled version of the corrected version of the reconstructed base-coded video and the input video; A fourth encoded stream (e.g., a time layer) is generated from time processing to command a decoder; and a fifth stream (header) is generated to configure the decoder. The encoded base stream is decoded by a base decoder that implements a decoding algorithm corresponding to the encoding algorithm implemented by the base codec used in the encoder, and the output is the decoded base. Separately and independently, a group of L1 coefficients is decoded to obtain L1 residual data. Also, individually and independently, a group of L0 coefficients is decoded to obtain L0 residual data. Subsequently, the decoded base, L1 residual data, and L0 residual data are combined. In particular, the decoded base is combined with the L1 residual data to produce an intermediate picture. Subsequently, the intermediate picture is upsampled and can be further combined with the L0 residual data.

[0015] In addition, the new approach uses an encoding and decoding process to process the picture without using any inter-block prediction. Rather, it processes the picture by transforming NxN blocks of picture elements (e.g., 2x2 or 4x4) and processing the blocks independently of each other. This results in efficient processing as well as a state without dependencies on neighboring blocks, thus enabling the processing of the picture to be parallelized. means of solving the problem

[0016] According to one embodiment, a method for decoding an encoded signal may be provided, wherein the encoded signal is received in a bitstream, and the method comprises the steps of: receiving a quantized data set associated with the encoded signal; determining at least one dequantization parameter; and dequantizing the quantized data at least partially based on at least one dequantization parameter.

[0017] According to another aspect, a method for decoding an encoded stream into a reconstructed output signal is provided, the method comprising: receiving a first output signal decoded from a first base encoded stream according to a first codec; receiving a level encoded stream; decoding the level encoded stream to obtain a residual set; and combining the residual set with the first output signal to generate a reconstructed signal, wherein the step of decoding the level encoded stream comprises: decoding a set of quantized coefficients from the level encoded stream; and dequantizing the set of quantized coefficients. The combination may include combining with an upsampled version of the first output signal.

[0018] The level-encoded stream may be a first-level encoded stream; the set of quantized coefficients may be a first set of quantized coefficients; and the set of residuals may be a first set of residuals, wherein the method may further include the steps of receiving a second-level encoded stream; decoding the second-level encoded stream to obtain a second set of residuals; and combining an upsampled version of the reconstructed signal with the second set of residuals to produce a reconstruction of the original resolution input signal, and the step of decoding the second-level encoded stream includes the step of decoding a second set of quantized coefficients from the second-level encoded stream; and the step of dequantizing the second set of quantized coefficients.

[0019] The method can advantageously improve the efficiency of the encoding and decoding processes by changing the degree and / or method of compression applied to the coefficients of the quantization process, depending on any of a number of factors based on the video data to be encoded. Thus, the manner in which the typically lossy quantization procedure is performed during video stream encoding can be adapted so that an appropriate balance—a relationship between encoding or compression efficiency and the visually perceptible compression of the input video, which can vary significantly depending on different video frames and streams—can be applied according to the characteristics and content of the input video. In particular, the method enables the use of such an adaptable form of quantization in the dequantization process at the receiving decoder, for example, in cooperation with the preceding encoding process, through the manner in which quantization is performed, or, for example, the degree of change from the default mode signaled to the decoder. This can be achieved through the transmission of parameters having values ​​that express or represent the corresponding information. For example, it has been found that including such information or parameters in the data received from the decoder while increasing the size of the data payload can provide compression improvements, even though they are potentially called frequently in the compressed data stream from the encoder to the decoder.

[0020] In some embodiments, the determining step further comprises: deriving a mark of a dequantization operation to be applied to dequantize quantized data from a bitstream including an encoded stream; and determining at least one dequantization parameter based at least partially on said mark. The dequantization operation may include using at least one predetermined dequantization parameter to determine at least one dequantization parameter.

[0021] In such cases, typically at least one predetermined dequantization parameter corresponds to a default dequantization parameter. In such cases, preferably, the encoded stream contains a sequence of samples, and at least one predetermined dequantization parameter corresponds to a dequantization parameter used to decode the previous sample.

[0022] In some embodiments, the dequantization operation includes updating at least one predetermined dequantization parameter using at least one updated parameter received in a bitstream containing an encoded stream.

[0023] In some embodiments, at least one dequantization parameter includes a quantization step width parameter, and said quantization step width parameter is received in a bitstream. In such cases, preferably, at least one dequantization parameter further includes at least one scaling factor. In these embodiments, the step of determining at least one dequantization parameter may include the step of processing the quantization step with at least one scaling factor.

[0024] In some embodiments, the set of quantized data, particularly quantized coefficients, may include a plurality of subsets of quantized data, and at least one dequantization parameter includes a plurality of dequantization parameters, and each of the plurality of dequantization parameters corresponds to each subset of the plurality of subsets of quantized data. Such correspondence can be understood as each of the plurality of quantization parameters being related to or corresponding to each subset in such a manner that, for example, a given subset is dequantized according to each corresponding dequantization parameter.

[0025] In these embodiments, typically, the dequantization operation further comprises the step of dequantizing each of a plurality of subsets of quantized data based at least partially on each of a plurality of dequantization parameters. The plurality of dequantization parameters may include at least a quantization step width parameter—the quantization step width parameter is received in a bitstream—and at least one scaling factor. In this case, preferably, the at least one scaling factor includes a plurality of scaling factors, and each of the plurality of scaling factors corresponds to each subset of the plurality of subsets of quantized data. In some of these embodiments, the step of determining at least one dequantization parameter may include the step of deriving a plurality of dequantization parameters, and each of the plurality of dequantization parameters is obtained by processing the quantization step width parameter with each of the plurality of scaling factors.

[0026] The method may include a step of calculating actual quantization parameters from received quantization step width parameters based, for example, on a predetermined function. In some embodiments, the actual quantization parameters is a scaling factor Received quantization step width parameter is calculated by processing it with the function of these two arguments according to the following:

[0027] .

[0028] In these embodiments, typically the function f(ㆍ) is a multiplication according to the following.

[0029]

[0030] In such cases, preferably, the dequantization operation is It includes dequantized data calculated as, where is a quantized data subset, and is the actual quantization step for the i-th quantized data subset.

[0031] In some embodiments, actual quantization parameters is the step width modifier ( Calculated based additionally on ), the dequantization offset can be used in the dequantization operation according to the following:

[0032] .

[0033] In these embodiments, preferably f(ㆍ) is a function according to the following:

[0034] .

[0035] The dequantization operation may include dequantized data calculated as follows:

[0036]

[0037] Here is a quantized data subset and is the actual quantization step for the i-th quantized data subset and DO i is the dequantization offset for a specific i-th quantized data subset.

[0038] In some embodiments, the linear quantizer uses a non-central dequantization offset. The linear quantizer may use dead zones and non-central dequantization offsets of different sizes related to the quantization step. In some embodiments, the value of the dequantization offset may be received from the encoder. Preferably, the received offset value is added to the received quantized value prior to dequantization based on the step width.

[0039] In another embodiment, a method for encoding an input signal into a plurality of encoded streams may be provided, and the encoded streams may be combined to reconstruct the input signal, and the method comprises the steps of: receiving an input signal; downsampling the input signal to generate a downsampled signal; commanding the encoding of the downsampled signal using a base encoder to generate a base encoded stream; commanding the decoding of the base encoded stream using a base decoder to generate a reconstructed signal; generating a set of residuals by comparing the reconstructed signal with an input video; and encoding the set of residuals to generate a level encoded stream, wherein the step of encoding the set of residuals comprises the steps of: applying a transformation to the set of residuals to generate a set of transformed coefficients; applying a quantization operation to the set of transformed coefficients to generate a set of quantized coefficients; and applying an encoding operation to the quantized coefficients.

[0040] The residual set may be a first residual set, and the comparison step may include a step of generating a first residual set such that the encoded stream becomes a first level encoded stream by comparing the reconstructed signal with the downsampled signal, and the method further includes a step of decoding the first residual set to generate a decoded first residual set; a step of correcting the reconstructed signal using the decoded first residual set to generate a corrected reconstructed signal; a step of upsampling the corrected reconstructed signal to generate an upsampled reconstructed signal; a step of comparing the upsampled reconstructed signal with an input signal to generate a second residual set; and a step of encoding the second residual set to generate a second level encoded stream, and the step of encoding the second residual set includes a step of applying a transformation to the second residual set to generate a second coefficient set; a step of applying a quantization operation to the second coefficient set to generate a second set of quantized coefficients; and a step of applying an encoding operation to the second set of quantized coefficients.

[0041] A bin folding process may also be used in some embodiments to improve coding efficiency. In particular, in such cases, the quantization operation further includes the step of applying a bin folding operation to a set of coefficients, wherein the bin folding operation includes each coefficient having a value exceeding a predetermined maximum value, and the maximum value is defined by the upper limit of a first quantization bin of a plurality of quantization bins having a defined step width, wherein the maximum value is quantized to have a quantized value corresponding to the first quantization bin. This may be performed to place all residual or coefficient values ​​residing over the selected quantization bin into the selected bin. In relation to what can be understood as the endpoint of a range of related values, the first bin may be considered to correspond to the bin corresponding to the upper value, or the highest (absolute) quantized value. Bin folding may be implemented at either or both of the upper endpoint and the lower endpoint. A similar process may be performed for negative values ​​of the range. Bin folding may be configured to adjust or reduce the bit rate based on at least one of network conditions and base stream processing. Accordingly, the bin folding process may be configured itself, for example, with conditions and base stream processing or parameters derived therefrom used to configure the bin folding, for example, any one or both of the parameters defining the bin folding in various embodiments.

[0042] In some embodiments, the quantization operation further includes the step of using a quantization offset, which may be a non-central quantization offset.

[0043] Typically, the step of using a quantization offset includes applying the quantization offset to a plurality of quantization bins having a defined step width, thereby adjusting each of the corresponding values ​​of one or more of the plurality of quantization bins by or based on the value of the quantization offset. In this embodiment, each of the plurality of quantization bins may have its corresponding value adjusted. Alternatively, either or both of the value corresponding to the beginning of the first bin of the plurality of bins and the value corresponding to the end of the last bin of the plurality of bins are not adjusted by the value of the quantization offset, that is, are maintained without being adjusted by it. The first bin may be understood as corresponding to the numerically lowest value or the minimum value of the range. Likewise, the last bin may be understood as representing the maximum value of the range or the numerically largest value. Such adjustment and lack thereof may be applied in conjunction with quantization operations involving dead zones and bin folding, which are described later in this disclosure.

[0044] Typically, the value of the quantization offset is adjustable or configurable. In some embodiments, the value of the quantization offset may be changed based on data indicating the operating conditions under which encoding is being performed.

[0045] In some embodiments, the method further includes the step of signaling a quantization offset value to a decoder to receive the encoded stream. This signaling may be performed, for example, in an implementation where the quantization offset value changes dynamically during encoding.

[0046] Quantization operations typically involve subtracting a quantization offset value from the residual or coefficient value prior to quantization, based on the quantization step width.

[0047] In some embodiments, the value of the quantization offset is adjusted based on the sign of the residual or coefficient. This can be influenced (i.e., implemented) to allow for symmetry behavior with respect to a zero value.

[0048] The method can be performed such that when the value of the quantization offset is set to a first predetermined value, the application of the offset to the empty value is disabled. For example, this can be done by setting the quantization or dequantization offset value to 0.

[0049] In some embodiments, the value of the quantization offset is adjusted based on the defined width of the dead zone. In these embodiments, the quantization operation may be performed using the dead zone, which is described in detail later in this disclosure.

[0050] In some embodiments, the method may involve changing the step width used in the quantization operation according to a step width parameter. In particular, the step width may be changed for each of one or more coefficients within a set of coefficients, for example, for different coefficients within a 2x2 or 4x4 block of coefficients. For example, the step width may be changed so that a smaller step width value has a greater influence on the perception of the decoded signal for one or more of the predetermined coefficients. The degree of influence is typically determined experimentally, thereby obtaining information indicating which coefficient has a greater influence on the perception of the decoded signal by the viewer.

[0051] The step width is typically assigned a default value based on the base step width parameter. One or more modified step widths can be obtained based on the base step width and step width modifier parameters. For example, this can be accomplished by obtaining the modified step width according to the formula modified_stepwidth = base_stepwidth * modifier, where the modifier can be set based on a specific coefficient within a block or unit.

[0052] In these embodiments, each step width modifier parameter can be used to modify the step width used for each of one or more coefficients.

[0053] Each step width value may be used for or associated with two or more encoded streams, including a base encoded stream and one or more enhancement level encoded streams in some embodiments, or for each enhancement level.

[0054] In some embodiments, the step width modifier parameter depends on the enhancement level, that is, varies according to the level of enhancement used. The step width modifier may be changed so that a smaller step width is used for the first-level encoded stream and a larger step width is used for the base-encoded stream.

[0055] In some preferred embodiments, the quantization operation uses a quantization matrix defined as a set of step width modifier parameter values ​​for different coefficients and different enhancement levels. Thus, the method may involve a step width modified parameter value for each coefficient and each enhancement level. The quantization matrix may be obtained by various means in different embodiments by an encoder performing the method and by a decoder performing a corresponding decoding process. In particular, the quantization matrix may be pre-set in at least one of the encoder and the decoder, or the quantization matrix may be signaled between the encoder and the decoder, and additionally or alternatively may be dynamically configured in at least one of the encoder and the decoder.

[0056] The method may further include the step of constructing a quantization matrix as a function of at least one of one or more stored and one or more signaled parameters.

[0057] In some embodiments, for x = 0...nTbS⁻¹, y = 0...nTbS⁻¹ and a given quantization matrix qm[x][y], the scaled transform coefficient d[x][y] can be derived according to the following formula:

[0058] d[x][y] = (TransformCoeffQ[x][y] * ((qm[x + (levelIdxSwap * nTbS)][y] + stepWidthModifier[x][y]) + appliedOffset [x][y]),

[0059] Here, TransformCoeffQ is an array of size (nTbS)x(nTbS) containing entropy-decoded quantized transform coefficients, and

[0060] levelIdx is a variable specifying the index of the improvement sub-level, and appliedOffset[x][y] and stepWidthModifier[x][y] are variables. appliedOffset[x][y] may correspond to a dead zone described elsewhere in this application.

[0061] For example, levelIdx can be equal to 1 for improvement sub-level 1 and equal to 2 for improvement sub-level 2.

[0062] Typically, the variable stepWidthModifier[x][y] is derived according to the following:

[0063] When dequant_offset_signalled_flag = 0, stepWidthModifier [x][y] = ((((Floor(-Cconst * Ln (qm[x + (levelIdxSwap * nTbS)][y]))) + Dconst) * (qm[x + (levelIdxSwap * nTbS)][y]2))) / 32768), where Cconst and Dconst are constants and can have values ​​of 5242 and 99614, respectively, in one example.

[0064] In some embodiments, the quantization operation may include quantizing coefficients using a linear quantizer, and it is preferable for the linear quantizer to use a dead zone of variable size. In these embodiments, the size of the dead zone may be set as a predetermined multiple of the step width used in the quantization operation, for example, as a linear function of the step width value. Alternatively, a non-linear function of the step width value may be used.

[0065] The size of the step width used in the quantization operation is variable in some preferred embodiments, and the size of the dead zone is more preferably adapted according to the variable step width.

[0066] The size of the dead zone can be set by multiplying the step width used in the quantization operation by a multiplier parameter, and the value of the multiplier parameter is changed based on data representing the operating conditions under which encoding is being performed, e.g., the available bit rate. Thus, in some embodiments, the multiplier may also be adaptive.

[0067] Quantization operations are typically performed according to one or more quantization parameters. The one or more quantization parameters are typically set to at least one of controlling and providing a desired bitrate in one or more encoded streams. That is, they may be set to control and / or provide a desired bitrate in one or more encoded streams. The desired bitrate is a common bitrate for all streams to produce a common encoded stream, or different bitrates are provided for different encoded streams.

[0068] In some embodiments, one or more quantization parameters are set to provide a desired quality level or maximize the quality level within a predefined set of bit rate constraints.

[0069] The method may include the step of determining quantization parameters by receiving the state of a buffer receiving one or more encoded streams and a base encoded stream; and the step of using this state to determine quantization parameters.

[0070] The buffer is preferably used to store and / or combine an encoded base stream and an encoded enhancement stream, and is configured to receive input at a variable bitrate while the output is read at a constant rate. A rate controller can read the state from the buffer to ensure that the buffer is not overflowed or empty and that data is always available for reading from the output. The state of the buffer can also be used to generate one or more quantization parameters. One or more quantization parameters can be controlled based on the amount of data in the buffer.

[0071] Typically, the value of the quantization parameter is inversely proportional to the amount of data in the buffer.

[0072] In some embodiments, quantization parameters are determined for each frame, residual and / or group of residuals, i.e., for at least one of each frame, residual, and group of residuals. Typically, quantization parameters for a frame are determined based on the target data size for the frame and the current data size for the frame using a previous set of quantization parameters. In any of these embodiments, quantization parameters may be based on a previous set of quantization parameters.

[0073] The method may include the step of defining a set of curves to map a normalized size to one or more quantization parameters, wherein each curve includes one or more of a multiplier and an offset that depends on the properties of the current frame. A set of curves may be defined to map the normalized size to quantization parameters. Each curve may have one or more of a multiplier and an offset that may depend on the properties of the current frame (e.g., the complexity of the information to be encoded within the frame). The multiplier and the offset may define the shape of the curve. The multiplier may be applied to a size normalization function that is a function of the quantization parameter Q. In the case where the current size (i.e., Q) t-1 Size of frame t encoded as) and Q t-1 can be used to define a point within the space of a set of curves. This point can be used to select the set of nearest curves from the set of curves. These can be curves above and below the point, or the highest and lowest curves relative to the point. The set of nearest curves can be used in an interpolation function with the point to determine a new curve associated with the point. Once this new curve is determined, a multiplier and an offset for the new curve can be determined. These values ​​can then be used with the received target size to determine the value for Qt (for example, the curve can define a function of size and Q). Therefore, typically, the multiplier is applied to a size normalization function that is a function of the quantization parameter Q. t-1 Current size and Q of frame t encoded as t-1 It can be used to define a point in space of a set of curves, and this point is used to select the set of nearest curves from the set of curves. The nearest curves can be curves above the point and curves below the point, or the highest or lowest curves relative to the point.

[0074] In such cases, a set of nearest curves is typically used in an interpolation function with the point to determine a new curve associated with the point, and a multiplier and offset for the determined new curve can be determined, and the values ​​of the multiplier and the offset for the determined new curve are used with the received target size to Q t It further includes a step of determining the value for.

[0075] A set of curves is stored in accessible memory and can be updated based on a set of curves determined for the previous frame. In certain cases, adaptive quantization may be applied differently to different coefficient positions within a coding unit or block, for example, to different elements within an array of 4 or 16 coefficients (e.g., for a 2x2 or 4x4 transformation).

[0076] In some embodiments, the quantization operation is performed using a quantization matrix, and the quantization matrix is ​​derived based on the acquired value of a quantization matrix mode parameter. Preferably, in this case, the quantization matrix mode parameter specifies the quantization matrix to be used in the encoding process.

[0077] Typically, each, preferably different, quantization matrix is ​​used for each of two or more levels of an encoded stream. More preferably, different quantization matrices are used for each encoded stream, wherein a default quantization configuration is predetermined and a change from the default configuration is signaled between the encoder and the decoder.

[0078] In these embodiments, the method may include the step of using a different quantization matrix for each encoded stream through at least one of the default configurations, and the step of using a common quantization matrix for each encoded stream through signaling to invalidate the default configurations.

[0079] In some embodiments, the quantization matrix is ​​used for only one of a plurality of refinement levels. The quantization matrix is ​​typically indexed by the position of the coefficients within a block in which the coefficients are arranged.

[0080] In some embodiments, the base quantization matrix is ​​defined as a set of values, and the base quantization matrix is ​​modified according to a scaling factor which is a function of step width for one or more refinement levels. The scaling factor may be calculated, for example, as a clamped function of the step width parameter. In some embodiments, each entry of the quantization matrix is ​​scaled using an exponential function of the scaling factor.

[0081] According to a fourth embodiment, a decoder is provided for decoding an encoded stream into a reconstructed output video, and the decoder is configured to perform the method of a first or second embodiment.

[0082] According to an additional embodiment, a method for decoding an encoded stream into a reconstructed output signal (i.e., a method for decoding) may be provided, the method comprising the steps of: receiving a first output signal decoded from a first base encoded stream according to a first codec; receiving a level encoded stream; decoding the level encoded stream to obtain a set of residuals; and combining the set of residuals with the first output signal to generate a reconstructed signal, wherein the step of decoding the level encoded stream comprises the step of decoding a set of quantized coefficients from the level encoded stream; and the step of dequantizing the set of quantized coefficients. The combination may include combining with an upsampled version of the first output signal. The level encoded stream may be a first level encoded stream; the set of quantized coefficients is a first set of quantized coefficients; and the set of residuals is a first set of residuals, wherein the method comprises the steps of: receiving a second level encoded stream; and decoding the second level encoded stream to obtain a second set of residuals. The method may further include the step of combining an upsampled version of the reconstructed signal and a second residual set to generate a reconstruction of the original resolution input signal, and the step of decoding the second level encoded stream includes the step of decoding a second set of quantized coefficients from the second level encoded stream; and the step of dequantizing the second set of quantized coefficients.

[0083] According to an additional embodiment, an encoder for encoding an input signal is provided. According to an additional embodiment, a decoder for decoding an input signal is provided.

[0084] According to an additional embodiment, a non-transient computer-readable storage medium is provided that stores instructions that cause a processor to perform a method when executed by a processor.

[0085] According to an additional embodiment, a signal in the form of a bitstream is provided, said signal comprising at least a first part and a second part, wherein the first part generates a decoded signal when decoded by a decoding module and the second part provides information necessary for decoding the first part, said second part comprises a first sub-part representing a dequantization parameter to be used to dequantize one or more data of the first part; and a second sub-part representing a process to be used to dequantize one or more data of the first part. The second part may further comprise a third sub-part conditionally existing for the process indicated in the second sub-part, and the third sub-part represents a new set of dequantization parameters to be used to dequantize one or more data of the first part. For example, the third sub-part may exist in the bitstream only when the process indicated in the second sub-part requires that a new set of dequantization parameters be signaled and provided.

[0086] The embodiments described above provide technical advantages. In particular, the approach of the present disclosure enables an efficient and flexible method for decoding and receiving signaled dequantization parameters. Rather than always using the same quantization parameters for all quantized data, this approach provides for the flexible use of quantization parameters used to dequantize quantized data based on a combination of predetermined information and signaled information. This, in turn, enables more effective, granular, and accurate dequantization, thereby generating better decoded data. Secondly, the signaling of data can be performed in a manner that optimizes the bitstream and saves data when not needed. For example, in some implementations, modification parameters are sent only when the quantization parameters need to be modified. Brief explanation of the drawing

[0087] Figure 1 illustrates a high-level schematic diagram of the encoding process. Figure 2 illustrates a high-level schematic diagram of the decoding process. Figure 3 illustrates a high-level schematic diagram of the encoding process and a specific encoding step. Figure 4 illustrates a high-level schematic diagram of the decoding process and a specific decoding step. Figure 5 illustrates a high-level schematic diagram of the encoding process. Figure 6 illustrates a high-level schematic diagram of the additional decoding process. FIG. 7 illustrates a flowchart of the concept described in the present application. FIGS. 8a through 8d illustrate a method in which the quantization of an encoding process can be performed according to a specific example. FIG. 9 illustrates an exemplary decoding module. Figure 10 illustrates an exemplary decoding process. Figure 11 illustrates an exemplary encoding module. Figure 12 is a schematic diagram of an encoder. Figure 13 is a schematic diagram of a decoder. Specific details for implementing the invention

[0088] The present invention relates to a method. In particular, the present invention relates to a method for encoding and decoding a signal (i.e., a method for decoding). Data processing may include, but is not limited to, the acquisition, derivation, output, reception, and reconstruction of data.

[0089] The coding technology described in this application is a flexible, adaptable, highly efficient, and computationally inexpensive coding format that combines a video coding format, a base codec (e.g., AVC, HEVC, or any other current or future codec), with an enhancement level of coded data encoded using other technologies. This technology uses a downsampled source signal encoded using a base codec to form a base stream. An enhancement stream is formed using an encoded set of residuals that corrects or enhances the base stream, for example, by increasing the resolution or the frame rate. There may be multiple levels of enhancement data in a hierarchical structure. In a particular device, the base stream may be decoded by a hardware decoder, while the enhancement stream may be suitable for software implementation.

[0090] It is important that any optimization used in the new coding technique is tailored to the specific requirements or constraints of the enhancement stream and has low complexity. These requirements or constraints include: a potential reduction in computational power due to the need for software decoding of the enhancement stream; the necessity of combining the decoded residual set with the decoded frame; the possible structure of the residual data, namely, a relatively high proportion of zero values ​​along with highly variable data values ​​over a wide range; the nuances of the quantized blocks of coefficients; and the structure of the enhancement stream, which is a set of discrete residual frames separated into various components. Note that the constraints imposed on the enhancement stream imply that simple and fast entropy coding operations are essential to enable the enhancement stream to effectively correct or enhance individual frames of the base-decoded video. Note that in some scenarios, the base stream is also decoded substantially simultaneously before combination, placing a burden on resources.

[0091] In one case, the method described in this application may be applied to so-called planes of data that reflect different color components of a video signal. For example, the method described in this application may be applied to different planes of YUV or RGB data that reflect different color channels. Different color channels may be processed in parallel. Accordingly, a reference to a set of residuals as described in this application may include a plurality of sets of residuals, wherein each color component has a different set of residuals that forms part of a combined enhancement stream. The components of each stream may be contrasted in any logical order, for example, each plane of the same level may be grouped and sent together, or alternatively, sets of residuals for different levels of each plane may be sent together.

[0092] This document preferably satisfies the requirements of the following ISO / IEC documents: documents ["Call for Proposals for Low Complexity Video Coding Enhancements" ISO / IEC JTC1 / SC29 / WG11 N17944, Macao, CN, Oct. 2018] and ["Requirements for Low Complexity Video Coding Enhancements" ISO / IEC JTC1 / SC29 / WG11 N18098, Macao, CN, Oct. 2018] (incorporated by reference in this application).

[0093] The general structure of the proposed encoding method to which the currently described technology can be applied utilizes a downsampled source signal encoded by a base codec, adds a first level of correction data to the decoded output of the base codec to generate a corrected picture, and then adds an additional level of enhancement data to an upsampled version of the corrected picture. Thus, the stream is considered to consist of a base stream and an enhancement stream. This structure generates multiple degrees of freedom that allow for excellent flexibility and adaptability to many situations, and thus forms a coding format suitable for many use cases, including Over-The-Top (OTT) transmission, live streaming, and live Ultra High Definition (UHD) broadcasting. Although the decoded output of the base codec is not intended for viewing, it is video fully decoded at low resolution, and can also be used as a low-resolution output where the output is compatible with existing decoders and deemed appropriate. In specific cases, a base stream can be generated using the base codec. The base codec may include independent codecs controlled in a modular or "black box" manner. The method described in this application may be implemented through computer program code that is executed by a processor and performs function calls to a hardware and / or software implementation base codec.

[0094] Generally, the term “residual” as used in this application refers to the difference between the values ​​of a reference array or reference frame and the actual array or frame of data. An array may be a one-dimensional or two-dimensional array representing a coding unit. For example, a coding unit may be a 2x2 or 4x4 set of residual values ​​corresponding to an area of ​​similar size of an input video frame. It should be noted that these generalized examples are agnostic to the encoding operation performed and the characteristics of the input signal. Reference to “residual data” as used in this application refers to a set of residuals, e.g., the residual set itself or data derived from the output of a set of data processing operations performed on the residual set. Throughout this description, a residual set generally comprises a plurality of residuals or residual elements, each residual or residual element corresponds to a signal element, that is, an element of the signal or source data. The signal may be an image or video. In such examples, the residual set corresponds to an image or frame of the video, each residual is associated with a pixel of the signal, and the pixel is a signal element. The examples disclosed in this application describe how these residuals can be modified (i.e., processed) to affect the encoding pipeline or the finally decoded image while reducing the overall data size. Residuals or sets may be processed on a per-residual element (or residual) basis, or on a group basis, such as per tile or per coding unit, where a tile or coding unit is a neighboring subset of the residual set. In the case where a tile may include a group of smaller coding units. Note that processing may be performed for each frame of the video or only for a set number of frames of the sequence.

[0095] Generally, each or both enhancement streams can be encapsulated within one or more enhancement bitstreams using a set of Network Abstraction Layer Units (NALUs). The NALUs are intended to encapsulate the enhancement bitstreams to apply the enhancement to the correct base-reconstructed frame. The NALUs may include, for example, reference indices to NALUs containing the base-decoder-reconstructed frame bitstream to which the enhancement must be applied. In this way, the enhancement is synchronized with the base stream, and the frames of each bitstream are combined to generate a decoded output video (i.e., the residuals of each frame of the enhancement level are combined with the frames of the base-decoded stream). A picture group may represent multiple NALUs.

[0096] Referring to the initial process described above, the base stream is provided with two levels (or lower levels) of improvement within the improvement stream, and an example of a generalized encoding process is illustrated in the block diagram of FIG. 1. The input full-resolution video (100) is processed to generate various encoded streams (101, 102, 103). The first encoded stream (encoded base stream) is generated by feeding a downsampled version of the input video to a base codec (e.g., AVC, HEVC, or any other codec). The encoded base stream may be referred to as the base layer or base level. The second encoded stream (encoded level 1 stream) is generated by processing the residual obtained by taking the difference between the reconstructed base codec video and the downsampled version of the input video. The third encoded stream (encoded level 2 stream) is generated by processing the residual obtained by taking the difference between the upsampled version of the corrected version of the reconstructed base-coded video and the input video. In certain cases, the component of FIG. 1 may generally provide a low-complexity encoder. In certain cases, an enhancement stream may be generated by an encoding process that forms part of the low-complexity encoder, and the low-complexity encoder may be configured to control independent base encoders and decoders (e.g., packaged as a base codec). In other cases, the base encoder and decoder may be provided as part of the low-complexity encoder. In one case, the low-complexity encoder of FIG. 1 may be viewed as a wrapper for a base codec, where the function of the base codec may be hidden in an entity implementing the low-complexity encoder.

[0097] The downsampling operation exemplified by the downsampling component (105) can be applied to the input video to produce a downsampled video to be encoded by the base encoder (113) of the base codec. Downsampling can be performed in both the vertical and horizontal directions, or alternatively, only in the horizontal direction. The base encoder (113) and the base decoder (114) can be implemented by the base codec (e.g., as different functions of a common codec). One or more of the base codec, and / or the base encoder (113) and the base decoder (114) may include computer program code executed by a suitably configured electronic circuit (e.g., a hardware encoder / decoder) and / or a processor.

[0098] Each enhancement stream encoding process may not necessarily include an upsampling step. For example, in FIG. 1, the first enhancement stream is conceptually a correction stream, whereas the second enhancement stream is upsampled to provide an enhancement level.

[0099] Looking more specifically at the process of generating an improved stream, the encoded base stream is decoded by a base decoder (114) to generate an encoded level 1 stream (i.e., a decoding operation is applied to the encoded base stream to generate a decoded base stream. Decoding can be performed by the decoding function or mode of the base codec). Then, the difference between the decoded base stream and the downsampled input video is generated in a level 1 comparator (110) (i.e., a subtraction operation is applied to the downsampled input video and the decoded base stream to generate a first set of residuals). The output of the comparator (110) may be referred to as the first set of residuals, for example, as a surface or frame of residual data, where the residual value is determined for each picture element at the output of the downsampling block (105), the resolution of the base decoder (114) and the base encoder (113).

[0100] Next, the difference is encoded by a first encoder (115) (i.e., a level 1 encoder) to generate an encoded level 1 stream (102) (i.e., an encoding operation is applied to a first residual set to generate a first improvement stream).

[0101] As previously described, the enhancement stream may include a first enhancement level (102) and a second enhancement level (103). The first enhancement level (102) may be considered as a stream that provides a correction level to a corrected stream, for example, a video signal base-encoded / decoded at a lower resolution than the input video (100). The second enhancement level (103) may be considered as an additional enhancement level that applies an enhancement or correction level to a signal reconstructed from the corrected stream, for example, converting the corrected stream to the original input video (100).

[0102] In the example of FIG. 1, the second enhancement level (103) is generated by encoding an additional residual set. The additional residual set is generated by a level 2 comparator (119). The level 2 comparator (119) determines the difference between the input video (100) and the output of an upsampled version of the decoded level 1 stream, for example, an upsampling component (117). The input to the upsampling component (117) is generated by applying a first decoder (i.e., a level 1 decoder) to the output of a first encoder (115). This generates a decoded level 1 residual set. This is then combined with the output of a base decoder (114) in a summing component (120). This effectively applies the level 1 residual to the output of the base decoder (114). This allows the loss of the level 1 encoding and decoding process to be corrected by the level 2 residual. The output of the sum component (120) can be seen as a simulated signal representing the output of applying level 1 processing to the base stream (101) encoded in the decoder and the encoded level 1 stream (102).

[0103] As mentioned, the upsampled stream is compared with the input video to generate an additional residual set (i.e., a difference operation is applied to the upsampled regenerated stream to generate an additional residual set). Then, the additional residual set is encoded by a second encoder (121) (i.e., a level 2 encoder) as an encoded level 2 enhancement stream (i.e., then, an encoding operation is applied to the additional residual set to generate an encoded additional enhancement stream).

[0104] Accordingly, as exemplified in FIG. 1 and described above, the output of the encoding process is a base stream (101) and, preferably, one or more enhancement streams (102, 103) including a first enhancement level and an additional enhancement level. The three streams (101, 102, and 103) may be combined with or without additional information, such as a control header, to create a combined stream for a video encoding framework representing the input video (100). It should be noted that the component illustrated in FIG. 1 may operate on blocks of data or coding units corresponding to a 2x2 or 4x4 portion of a frame at a specific resolution level, for example. Since the component operates without any inter-block dependencies, it can be applied in parallel to multiple blocks or coding units within a frame. This differs from the video encoding method of comparison, which has dependencies between blocks (e.g., spatial or temporal dependencies). The dependencies of the video encoding method of comparison limit the level of parallelization and require much higher complexity.

[0105] A corresponding generalized decoding process is illustrated in the block diagram of FIG. 2. FIG. 2 can be said to illustrate a low-complexity decoder corresponding to the low-complexity encoder of FIG. 1. The low-complexity decoder receives three streams (101, 102, 103) generated by the low-complexity encoder along with a header (204) containing additional decoding information. The encoded base stream (101) is decoded by a base decoder (210) corresponding to the base codec used in the low-complexity encoder. The encoded level 1 stream (102) is received by a first decoder (211) (i.e., a level 1 decoder) that decodes a first residual set encoded by the first encoder (115) of FIG. 1. In the first summing component (212), the output of the base decoder (210) is combined with the decoded residuals obtained from the first decoder (211). The combined video, which can be described as a level 1 reconstructed video signal, is upsampled by the upsampling component (213). The encoded level 2 stream (103) is received by the second decoder (214) (i.e., the level 2 decoder). The second decoder (214) decodes a second set of residuals encoded by the second encoder (121) of FIG. 1. Although the header (204) is shown in FIG. 2 as being used by the second decoder (214), it can also be used by the first decoder (211) as well as the base decoder (210). The output of the second decoder (214) is the decoded residuals of the second set. These may have a higher resolution for the first residual set and the input to the upsampling component (213). In the second summing component (215), the second residual set from the second decoder (214) is combined with the output of the upsampling component (213), i.e., the upsampled reconstructed level 1 signal, to reconstruct the decoded video (250).

[0106] Depending on the low-complexity encoder, the low-complexity decoder of FIG. 2 may operate in parallel for a coding unit or different blocks of a given frame of a video signal. Additionally, two or more of the base decoder (210), the first decoder (211), and the second decoder (214) may perform decoding in parallel. This is possible because there is no dependency between blocks.

[0107] In the decoding process, the decoder can parse a header (204) (which may include global configuration information, picture or frame configuration information, and data block configuration information) and configure a low-complexity decoder based on this header. To regenerate the input video, the low-complexity decoder can decode a base stream, a first enhancement stream, and an additional or second enhancement stream, respectively. Frames of the streams can be synchronized and then combined to yield a decoded video (250). The decoded video (250) may be a lossy or lossless reconstruction of the original input video (100) depending on the configuration of the low-complexity encoder and decoder. In many cases, the decoded video (250) may be a lossy reconstruction of the original input video (100) in which the loss is reduced or has minimal effect on the recognition of the decoded video (250).

[0108] In FIG. 1 and FIG. 2, respectively, the Level 2 and Level 1 encoding operations may include the steps of transform, quantization, and entropy encoding (e.g., in this order). Similarly, in the decoding stage, the residual may be passed through an entropy decoder, a dequantizer, and an inverse transform module (e.g., in this order). Any suitable encoding and corresponding decoding operations may be used. However, preferably, the Level 2 and Level 1 encoding steps may be performed in software (e.g., executed by one or more central or graphics processing units in the encoding device).

[0109] The transformations described in this application may use directional decomposition transformations, such as Hadamard-based transformations. Both may include small kernels or matrices applied to flattened coding units of residuals (i.e., 2x2 or 4x4 blocks of residuals). Further details regarding the transformations can be found, for example, in patent applications PCT / EP2013 / 059847 or PCT / GB2017 / 052632 incorporated by reference in this application. The encoder may select between different transformations to be used, for example, between the sizes of the kernels to be applied.

[0110] The transformation can transform residual information into four surfaces. For example, the transformation can generate the following components: mean, vertical, horizontal, and diagonal. As previously described in the present disclosure, these components output by the transformation can be taken in these embodiments as coefficients to be quantized according to the described method.

[0111] In summary, the method and apparatus of the present application are based on a holistic approach built upon existing encoding and / or decoding algorithms (e.g., MPEG standards such as AVC / H.264, HEVC / H.265, etc., and other non-standard algorithms such as VP9, ​​AV1, etc.), which thus serve as a baseline for enhancement layers operating on different encoding and / or decoding approaches. The idea behind the holistic approach of the example is to hierarchically encode / decode video frames, as opposed to the use of block-based approaches used in MPEG-family algorithms. Hierarchically encoding frames involves generating residuals for the entire frame and then generating decimated frames, etc.

[0112] Video compression residual data for full-size video frames may be referred to as LoQ-2 (e.g., 1920 x 1080 for HD video frames or higher for UHD frames), while that of decimated frames may be referred to as LoQ-x, where x represents a number corresponding to the hierarchical decimation. In the examples described in FIGS. 1 and 2, the variable x may have values ​​of 1 and 2, representing the first and second refinement streams. Thus, there are two hierarchical levels at which compression residuals are generated. Other naming conventions for the levels may also be applied without any functional change (e.g., the Level 1 and Level 2 refinement streams described in this application may alternatively be referred to as Level 1 and Level 2 streams—indicating a countdown from the highest resolution).

[0113] A more detailed encoding process is illustrated in the block diagram of FIG. 3. The encoding process is divided into two halves as indicated by the dotted line. Below the dotted line is the base level of the encoder (300), which can be usefully implemented in hardware or software. Above the dotted line is the enhancement level, which can be usefully implemented in software. The encoder (300) may include only the enhancement level process, or, if necessary, a combination of the base level process and the enhancement level process. The encoder (300) can be usefully implemented in software, particularly at the enhancement level. This arrangement allows for the upgrade of a legacy hardware encoder providing the base level, for example, using a firmware (e.g., software) update, where the firmware is configured to provide the enhancement level. In newer devices, both the base level and the enhancement level may be provided in hardware and / or a combination of hardware and software.

[0114] A general level encoder topology is as follows: The encoder (300) includes an input I for receiving an input signal (30). The input signal (30) may include an input video signal, wherein the encoder is applied on a frame-by-frame basis. The input I is connected to a downsampler (305D) and a processing block (300-2). The downsampler (305D) may correspond to the downsampling component (105) of FIG. 1, and the processing block (300-2) may correspond to the second encoder (121) of FIG. 1. The downsampler (305D) outputs to a base codec (320) at the base level of the encoder (300). The base codec (320) may implement the base encoder (113) and base decoder (114) of FIG. 1. The downsampler (305D) also outputs to a processing block (300-1). The processing block (300-1) may correspond to the first encoder (115) of FIG. 1. The processing block (300-1) transmits the output to the upsampler (305U), and the upsampler (305U) outputs to the processing block (300-2) in turn. The upsampler (305U) may correspond to the upsampling component (117) of FIG. 1. Each processing block (300-2 and 300-1) includes one or more of the following modules: a conversion block (310), a quantization block (320), an entropy encoding block (330), and a residual processing block (350). The residual block (350) may occur before the conversion block (310) and / or may control residual processing in the processing block (300). The processing order may be as described in the drawings.

[0115] In this example, an input signal (30), such as full (or highest) resolution video, is processed by an encoder (300) to generate various encoded streams. A base encoded stream is generated by using a downsampler (305D) to feed a downsampled version of the input video (30) at the base level to a base codec (320) (e.g., AVC, HEVC, or any other codec). The base encoded stream may include the output of the base encoder of the base codec (320). A first encoded stream (encoded level 1 stream) is generated by reconstructing the encoded base stream to generate a base reconstruction, and then taking the difference between the base reconstruction and the downsampled version of the input video (30). Reconstructing the encoded base stream may include receiving a decoded base stream from the base codec (i.e., the input to the processing block (300-1) includes a base decoded stream as illustrated in FIG. 1). Next, the difference signal is processed in block (300-1) to generate an encoded level 1 stream. Block (300-1) includes a transform block (310-1), a quantization block (320-1), and an entropy encoding block (330-1). A second encoded stream (encoded level 2 stream) is generated by upsampling a corrected version of the base reconstruction using an upsampler (305U) and taking the difference between the corrected version of the base reconstruction and the input signal (30). Next, this difference signal is processed in block (300-2) to generate an encoded level 2 stream. Block (300-2) includes a transform block (310-2), a quantization block (320-2), an entropy encoding block (330-2), and a residual processing block (350-2). According to the processing block (300-1), the blocks can be performed in the order shown in the drawing (e.g., residual processing, then, transformation, then, quantization, then, entropy encoding).

[0116] A quantization method may be useful for generating residual signals into quanta so that specific variables can take only specific discrete sizes. In this case, quantization involves performing division by a predetermined step width. This can be applied to both levels (1 and 2). For example, quantization in block (320) may involve dividing the transformed residual value by a step width (e.g., where an integer quotient is used to generate the quantized value and the remainder is ignored). The step width may be predetermined and, for example, selected based on the desired quantization level. In this case, division by a step width may be converted into multiplication by an inverse step width and can be implemented more efficiently in hardware. In this case, dequantization such as in block (320) may include multiplication by a step width. Entropy encoding as described in this application may include run length encoding (RLE), wherein processing of the encoded output is performed using a Haffman encoder. In specific cases where entropy encoding is required, only one of these methods can be used.

[0117] An encoded base stream can be referred to as a base-level stream.

[0118] As previously explained, the residual is calculated by comparing the original form of the image signal with the reconstructed form of the image signal. For example, in one case, the residual for the L-2 enhancement stream is determined by subtracting the output of the upsampling from the original form of the image signal (e.g., the input video as shown in the figure). The input for the upsampling can be described as the reconstruction of the signal after simulated decoding. In another case, the residual for the L-1 enhancement stream is determined by subtracting the image stream output by the base decoder from the downsampled form of the original image signal (e.g., the output of the downsampling).

[0119] A decoder (400) that performs a decoding process corresponding to the encoder of FIG. 3 is illustrated in the block diagram of FIG. 4. The decoding process is divided into two halves as indicated by the dotted line. Below the dotted line is the base level of the decoder (400), which can be usefully implemented in hardware. Above the dotted line is an enhancement level, which can be usefully implemented in software. The decoder (400) may include only the enhancement level process, or, if necessary, a combination of the base level process and the enhancement level process. The decoder (400) can be usefully implemented in software, particularly at the enhancement level, and can be appropriately placed over legacy decoding technology, particularly legacy hardware technology. Legacy technology refers to older technology that was previously developed and sold, which is already on the market, inconvenient or costly to replace, and can still be used for signal decoding purposes. In other cases, the base level may include any existing and / or future video encoding tools or technologies.

[0120] A typical level decoder topology is as follows. The decoder (400) includes an input (not shown) for receiving one or more input signals, including an encoded base stream, an encoded level 1 stream, and an encoded level 2 stream, along with an optional header containing additional decoding information. The decoder (400) includes a base decoder (420) at the base level and processing blocks (400-1, 400-2) at the enhancement level. An upsampler (405U) is also provided between processing block (400-1) and processing block (400-2) to provide an upsampled version of the signal output by processing block (400-1) to processing block (400-2). The base decoder (420) may correspond to the base decoder (210) of FIG. 2, the processing block (400-1) may correspond to the first decoder (211) of FIG. 2, the processing block (400-2) may correspond to the second decoder (214) of FIG. 2, and the upsampler (405U) may correspond to the upsampler (213) of FIG. 2.

[0121] The decoder (400) receives one or more input signals and derives three streams generated by the encoder (300). The encoded base stream is transmitted to the base decoder (420) and decoded by it, and the base decoder corresponds to the base codec (420) used in the encoder (300) and serves to reverse the encoding process at the base level. The encoded level 1 stream is processed by a block (400-1) of the decoder (400) to regenerate a first residual set generated by the encoder (300). The block (400-1) corresponds to the processing block (300-1) of the encoder (300) and acts to reverse or substantially reverse the processing of the block (300-1) at the base level. The output of the base decoder (420) is combined with the first residual set obtained from the encoded level 1 stream. The combined signal is upsampled by an upsampler (405U). The encoded level 2 stream is processed by a block (400-2) to regenerate additional residuals generated by the encoder (300). The block (400-2) corresponds to the processing block (300-2) of the encoder (300) and acts to reverse or substantially reverse the processing of the block (300-2) at the base level. The upsampled signal from the upsampler (405U) is combined with additional residuals obtained from the encoded level 2 stream to generate a level 2 reconstruction of the input signal (30). The output of the processing block (400-2) can be seen as a decoded video similar to the decoded video (250) of FIG. 2.

[0122] As previously explained, the enhancement stream may include two streams, namely an encoded Level 1 stream (first enhancement level) and an encoded Level 2 stream (second enhancement level). The encoded Level 1 stream provides a set of enhancement data that can be combined with a decoded version of the base stream to produce a corrected picture.

[0123] FIG. 5 illustrates the encoder (300) of FIG. 1 in more detail. The encoded base stream is generated directly by the base encoder (320E) and may be quantized and entropy encoded as needed. In certain cases, these latter processes may be performed as part of the encoding by the base encoder (320E). To generate an encoded level 1 stream, the encoded base stream is decoded in the encoder (300) (i.e., a decoding operation is applied to the encoded base stream in the base decoding block (320D)). The base decoding block (320D) is illustrated as part of the base level of the encoder (300) and is illustrated separately from the corresponding base encoding block (320E). For example, the base decoder (320D) may be a decoding component that complements an encoding component in the form of a base encoder (320E) having a base codec. In another example, the base decoding block (320D) may instead be part of the improvement level and, in particular, part of the processing block (300-1).

[0124] Referring to FIG. 5, a difference is generated between the decoded base stream output from the base decoding block (320D) and the downsampled input video (i.e., a subtraction operation (310-S) is applied to the downsampled input video and the decoded base stream to generate a first set of residuals). Here, the term residual is used in the same manner as known in the art; that is, residual represents an error or difference between a reference signal or frame and a desired signal or frame. Here, the reference signal or frame is the decoded base stream and the desired signal or frame is the downsampled input video. Thus, the residual used in the first enhancement level can be considered as a correction signal because it can 'correct' the future decoded base stream to the downsampled input video used in the base encoding operation or a closer approximation. This is useful because it can correct singularities or other characteristics of the base codec. These include, above all, motion compensation algorithms applied by the base codec, quantization and entropy encoding applied by the base codec, and block control applied by the base codec.

[0125] The components of block (300-1) of FIG. 3 are illustrated more specifically in FIG. 5. In particular, the first residual set is transformed, quantized, and entropy-encoded to produce an encoded Level 1 stream. In FIG. 5, a transformation operation (310-1) is applied to the first residual set; a quantization operation (320-1) is applied to the transformed residual set to produce a quantized residual set; and an entropy encoding operation (330-1) is applied to the quantized residual set to produce a Level 1 stream encoded at the first enhancement level. However, it should be noted that in other examples, only the quantization step (320-1) may be performed, or only the transformation step (310-1) may be performed. Entropy encoding may not be used, or optionally may be additionally used in either or both of the transformation step (110-1) and the quantization step (320-1). The entropy encoding operation can be any appropriate type of entropy encoding, such as a Haffman encoding operation, a run-length encoding (RLE) operation, or a combination of both Haffman encoding and RLE operations.

[0126] The selection of such an entropy coding method, along with the described quantization, can have a favorable effect on coding performance. This can be understood in terms of the tendency to apply a high degree of quantization to residual data, as described in this disclosure, which results in a high proportion of zero values. Run-length encoding as mentioned above is particularly suitable for encoding data having such a distribution, and thus, this approach can synergistically improve the efficiency of the entire process. Likewise, for embodiments where quantization is applied with larger step width values ​​and the distribution of the quantized data consists of a relatively larger number of integer values, the efficiency of the encoding process will typically benefit from the use of prefix / Haffman encoding, which is particularly suitable for such distributions. This is especially true when a residual distribution is given in which higher integer values ​​have a lower frequency. This type of distribution can be efficiently encoded by Haffman encoding, which operates by allocating fewer bits to high-frequency symbols. In this way, the quantization and entropy encoding operations are complementary.

[0127] As previously described, the enhancement stream may include an encoded Level 1 stream (first enhancement level) and an encoded Level 2 stream (second enhancement level). The first enhancement level may be considered to enable video corrected at the base level, that is, for example, to correct encoder and / or decoder artifacts. The second enhancement level may be considered as an additional enhancement level that can be used to convert the corrected video to the original input video or to convert it to an approximation close to it (for example, to add detail or sharpness). For example, the second enhancement level may help to add fine details lost during downsampling and / or correct errors introduced by one or more of the conversion operation (310-1) and the quantization operation (320-1).

[0128] Referring to FIGS. 3 and 5, additional level enhancement information is generated by generating and encoding an additional residual set in block (300-2) to create an encoded level 2 stream. The additional residual set is the difference between the upsampled version (via the upsampler (305U)) of the corrected version of the decoded base stream (reference signal or frame) and the input signal (30) (desired signal or frame).

[0129] To achieve the reconstruction of a corrected version of the decoded base stream generated in the decoder (400), at least some of the processing steps of block (300-1) are inverted to mimic the process of the decoder (200) and to account for at least some losses and singularities of the transformation and quantization processes. To this end, block (300-1) includes an inverse quantization block (320-1i) and an inverse transformation block (310-1i). A quantized first residual set is inversely quantized in the inverse quantization block (320-1i) and inversely transformed in the inverse transformation block (310-1i) of the encoder (100) to regenerate a decoder-side version of the first residual set.

[0130] The decoded base stream from the decoder (320D) is combined with this improved decoder-side version of the first residual set (i.e., a summation operation (310-C) is performed on the decoded base stream and the decoder-side version of the first residual set). The summation operation (310-C) generates a reconstruction of a downsampled version of the input video that will be generated in all possible ways in the decoder—i.e., a reconstructed base codec video. As illustrated in FIGS. 3 and 5, the reconstructed base codec video is then upsampled by an upsampler (305U).

[0131] Then, the upsampled signal (i.e., reference signal or frame) is compared with the input signal (30) (i.e., desired signal or frame) to generate a second residual set (i.e., a difference operation (300-S) is applied to the upsampled regenerated stream to generate an additional residual set). Then, the second residual set is processed in block (300-2) to become an encoded level 2 stream (i.e., an encoding operation is then applied to the additional or second residual set to generate an encoded additional or second improved stream).

[0132] In particular, the second residual set is transformed (i.e., the transformation operation (310-2) is performed on the additional residual set to generate an additional transformed residual set). Then, the transformed residuals are quantized and entropy encoded in the manner previously described with respect to the first residual set (i.e., the quantization operation (320-2) is applied to the transformed residual set to generate a quantized additional residual set; and the entropy encoding operation (320-2) is applied to the quantized additional residual set to generate an encoded level 2 stream containing additional levels of improvement information). However, only the quantization step (20-1) may be performed, or only the transformation and quantization steps may be performed. Entropy encoding may additionally be optionally used. Preferably, the entropy encoding operation may be a Haffman encoding operation or a run length encoding (RLE) operation, or both.

[0133] Accordingly, as illustrated in FIGS. 3 and 5 and as previously described, the output of the encoding process is a base stream of the base level and preferably includes one or more enhancement streams of enhancement levels, including a first enhancement level and an additional enhancement level. As described with reference to the previous example, the operation of FIG. 5 can be applied in parallel to blocks or coding units of the color components of a frame because there is no inter-block dependency. The encoding of each color component within a set of color components can also be performed in parallel (e.g., so that the operation of FIG. 5 is replicated according to (number of frames) * (number of color components) * (number of coding units per frame). Additionally, it should be noted that different color components may have different numbers of coding units per frame, and, for example, since human vision can detect brightness changes more than color changes, the luma (e.g., Y) component may be processed at a higher resolution than the set of chroma (e.g., U or V) components.

[0134] An encoded base stream and one or more enhancement streams are received at the decoder (400). FIG. 6 illustrates the decoder of FIG. 4 in more detail.

[0135] The encoded base stream is decoded in a base decoder (420) to generate a base reconstruction of the input signal (30) received from the encoder (300). This base reconstruction can actually be used to provide a visible rendition of the signal (30) at a lower quality level. However, the primary purpose of this base reconstruction signal is to provide a base for a high-quality rendition of the input signal (30). To this end, the decoded base stream is provided to a processing block (400-1). The processing block (400-1) also receives the encoded level 1 stream and reverses any encoding, quantization, and transformation applied by the encoder (300). The block (400-1) includes an entropy decoding process (430-1), an inverse quantization process (420-1), and an inverse transformation process (410-1). Optionally, only one or more of these steps may be performed depending on the operation performed in the corresponding block (300-1) in the encoder. By performing these corresponding steps, a decoded level 1 stream containing a first residual set becomes available in the decoder (400). The first residual set is combined with the decoded base stream from the base decoder (420) (i.e., a summation operation (410-C) is performed on the decoded base stream and the decoded first residual set to produce a reconstructed downsampled version of the input video—i.e., a reconstructed base codec video). As illustrated in FIGS. 4 and 6, the reconstructed base codec video is then upsampled by the upsampler (405U).

[0136] Additionally, and optionally in parallel, the encoded Level 2 stream is processed in block (400-2) of FIG. 2 to generate a decoded additional residual set. Similar to processing block (300-2), processing block (400-2) includes an entropy decoding process (430-2), an inverse quantization process (420-2), and an inverse transform process (410-2). Of course, these operations will correspond to the operations performed in block (300-2) of the encoder (300), and one or more of these steps may be omitted as necessary. Block (400-2) generates a decoded Level 2 stream containing the additional residual set, which is summed with the output of the upsampler (405U) in operation (400-C) to generate a Level 2 reconstruction of the input signal (30). The Level 2 reconstruction can be viewed as an output decoded video, such as 250 of FIG. 2. In a specific example, it may also be possible to acquire and view the reconstructed video that is passed to the upsampler (405U)—which has a first enhancement level but may be at a lower resolution than the level 2 reconstruction.

[0137] Accordingly, as exemplified and described above, the output of the decoding process is (optional) base reconstruction and original signal reconstruction at a higher level. This example is particularly suitable for generating encoded and decoded video at various frame resolutions. For example, the input signal (30) may be an HD video signal containing frames of 1920 x 1080 resolution. In certain cases, both base reconstruction and Level 2 reconstruction may be used by the display device. For example, in the case of network traffic, the Level 2 stream may be interrupted more than the Level 1 and base streams (since downsampling reduces the dimension of each direction by 2, this can contain up to 4x the amount of data). In this case, when traffic occurs, the display device may revert to displaying the base reconstruction while the Level 2 stream is interrupted (e.g., while Level 2 reconstruction is unavailable) and then revert to displaying the Level 2 reconstruction when network conditions improve. A similar approach may be applied when the decoding device is experiencing resource constraints, for example, a set-top box performing a system update may have an operation base decoder (220) for outputting a base reconstruction but may not have processing capability for calculating a level 2 reconstruction.

[0138] The encoding device also enables the video distributor to distribute the video to a heterogeneous set of devices; while a device with only a base decoder (220) can view the base reconstruction, a device with an enhancement level can view the higher quality level 2 reconstruction. In the case of comparison, two full video streams at distinct resolutions were required to service both sets of devices. As the level 2 and level 1 enhancement streams encode residual data, the level 2 and level 1 enhancement streams can be encoded more efficiently, for example, the distribution of residual data typically has most of its mass around zero (i.e., when there is no difference) and typically takes values ​​in a small range of about zero. This may be especially the case after quantization. In contrast, the full video streams at different resolutions will have a different distribution with a non-zero mean or median that requires a higher bit rate for transmission to the decoder.

[0139] In certain examples, residuals may be considered as errors or differences at a specific quality level or resolution. In the described example, there are two quality levels or resolutions, and thus two sets of residuals (L-1 and L-2). Each set of residuals described in this application models different forms of errors or differences. L-1 residuals correct, for example, typically the characteristics of the base encoder and correct artifacts introduced by the base encoder as part of the encoding process. In contrast, L-2 residuals correct, for example, complex effects introduced by shifting the quality level and differences introduced by L-1 correction (e.g., artifacts generated over a wider spatial scale, such as an area of ​​4 or 16 pixels by the L-1 encoding pipeline). This implies that it is not obvious that an action performed on one set of residuals will necessarily provide the same effect on another set of residuals (e.g., each set of residuals may have a different set of statistical patterns and correlations).

[0140] In the examples described in this application, residuals are encoded by an encoding pipeline. This may include transform, quantization, and entropy encoding operations. It may also include residual ranking, weighting, and filtering. Such pipelines are illustrated in FIG. 1 and FIG. 3a and 3b. Then, the residuals are transmitted to a decoder, for example, as L-1 and L-2 enhancement streams, which may be combined with the base stream as a hybrid stream (or transmitted separately). In this case, a bit rate is set for the hybrid data stream containing both the base stream and the enhancement streams of the quanta, and then different adaptive bit rates are applied to the individual streams based on the data being processed to satisfy the set bit rate (for example, high-quality video perceived as having low-level artifacts may be configured by adaptively assigning bit rates to different individual streams, even at the per-frame level, so that the constrained data can be used by the most perceptually influential individual streams where the image data may change).

[0141] The residual set described in this application can be viewed as sparse data, for example, in many cases there is no difference for a given pixel or area and the resulting residual value is 0. When examining the distribution of residuals, most of the probability mass is assigned to small residual values ​​located near 0—for example, for a specific video, values ​​such as -2, -1, 0, 1, 2, etc., occur most frequently. In certain cases, the distribution of residual values ​​is symmetric or nearly symmetric with respect to 0. In a specific test video case, the distribution of residual values ​​was found to take a shape similar to a log- or exponential distribution with respect to 0 (e.g., symmetric or nearly symmetric). The exact distribution of residual values ​​may vary depending on the content of the input video stream.

[0142] Residuals can be treated as a two-dimensional image in themselves, for example, as a delta image of the difference. Viewed in this way, the sparsity of the data may appear to relate features such as "points," small "lines," "edges," and "corners" visible in the residual image. These features have typically been found to be completely uncorrelated (e.g., in space and / or time). This is because they have characteristics different from the properties of the image data from which they are derived (e.g., pixel characteristics of the original video signal).

[0143] Since the characteristics of residuals differ from those of the image data from which they are derived, it is generally impossible to apply standard encoding approaches, such as those found in traditional Moving Picture Experts Group (MPEG) encoding and decoding standards. For instance, many comparable methods utilize large transformations (e.g., transformations of large pixel areas in a typical video frame). For instance, due to the characteristics of residuals as previously described, using such relatively large transformations on residual images would be highly inefficient. For instance, it would be very difficult to encode small dots in a residual image using large blocks designed for areas of a typical image.

[0144] The specific examples described in this application address this problem by using small and simple transformation kernels instead (e.g., 2x2 or 4x4 kernels as presented in this application—direction decomposition and direction decomposition squared—). The transformations described in this application can be applied using Hadamard matrices (e.g., a 4x4 matrix for a flattened 2x2 coding block or a 16x16 matrix for a flattened 4x4 coding block). This moves in a different direction from the video encoding approaches being compared. Applying these new approaches to residual blocks generates compression efficiency. For example, certain transformations generate uncorrelated coefficients (e.g., in space) that can be efficiently compressed. While correlations between coefficients could be utilized, for example, for lines within the residual image, this can lead to encoding complexity, which is difficult to implement on legacy and low-resource devices and often generates other complex artifacts that need to be corrected. Preprocessing residuals by setting specific residual values ​​to 0 (i.e., not forwarding these values ​​for processing) can provide a controllable and flexible way to manage resource usage as well as bitrate and stream bandwidth.

[0145] For completeness, FIG. 7 illustrates the broad principles of the concept described in this application in the form of a flowchart. The method (1000) comprises the steps of receiving an input video (step 1001); downsampling the input video to generate a downsampled video (step 1002); commanding the encoding of the downsampled video using a base encoder to generate a base-encoded stream (step 1003); commanding the decoding of the base-encoded stream using a base decoder to generate a reconstructed video (step 1004); comparing the reconstructed video with the downsampled video to generate a first set of residuals (step 1005); and encoding the first set of residuals to generate a first level-encoded stream, wherein the step of encoding the first set of residuals comprises the step of applying a transformation to the first set of residuals to generate a first set of coefficients (step 1006). The method comprises: a step of generating a first set of quantized coefficients by applying a quantization operation to a first set of coefficients (step 1007); and a step of applying an encoding operation to the first set of quantized coefficients (step 1008), wherein the step of applying the quantization operation includes a step of adapting the quantization based on the first set of coefficients to be quantized, including changing the step width used for different coefficients of the first set of coefficients, and a first set of parameters derived from the adaptation is signaled to a decoder to enable dequantization of the first set of quantized coefficients.

[0146] FIG. 8a provides an example of how the quantization of residuals and / or coefficients (transformed residuals) can be performed based on bins having a defined step width. The x-axis of FIG. 8a represents the residual or coefficient values. In this example, multiple bins are defined with a step width of 5. The step width can be understood as the quantization step size illustrated in the figure. The size of the step width may be selectable, for example, based on parameter values. In certain cases, the size of the step width may be dynamically set, for example, based on the rate control example described earlier. In FIG. 8a, the step width results in bins corresponding to residual values ​​in the range of 0-4, 5-9, 10-14, and 15-19 (i.e., 0 to 4, including both 0 and 4 quanta). The bin width can be configured to include or exclude endpoints as needed. In this example, quantization is performed by replacing all values ​​belonging to a bin with integer values ​​(e.g., residual values ​​between 0 and 4 (including boundary values) have a quantized value of 1). In FIG. 8a, quantization can be performed by dividing by a step width (e.g., 5), taking the floor of the result (i.e., the integer closest to the decimal point for positive values), and then adding 1 (e.g., 3 / 5 = 0.6, floor(0.6) = 0, 0+1 = 1; or 16 / 5 = 3.2, floor(3.2) = 3, 3+1 = 4). Negative values ​​can be handled in a similar manner, for example, by working on the absolute value and then converting it to a negative value after calculation (e.g., abs(-9) = 9, 9 / 5 = 1.8, floor(1.8) = 1, 1+1 = 2, 2*-1 = -2). FIG. 8a illustrates the case of linear quantization where all bins have a common step width. It should be noted that various different implementations based on this approach may be defined, for example, that the first bin may have a quantized value of 0 instead of 1, or may include values ​​from 1 to 5 (including boundary values).Figure 8a is just a simple example of quantization according to bins of a given step width.

[0147] FIG. 8b illustrates a method by which a so-called "dead zone" (DZ) can be implemented. This can be understood as a region where the output value of a quantizer is around zero, which is a band containing a zero signal and has a size that may be equal to or different from the step width. Thus, for this band of input close to zero, the signal can be effectively attenuated, and therefore, data is not unnecessarily allocated to low-level signals that correspond to noise in visual data. In FIG. 8b, residuals or coefficients having values ​​within a predefined range are set to zero. In FIG. 8b, the predefined range is a range around the zero value. In FIG. 8b, values ​​less than 6 and greater than -6 are set to zero. The dead zone can be set to a fixed range (e.g., -6 to 6) or based on the step width. In the case where, the dead zone can be set as a predefined multiple of the step width, for example, as a linear function of the step width value. In the example of Fig. 8b, the dead zone is set to 2.4*stepwidth. Thus, if the step width is 5, the dead zone extends from -6 to +6. In other cases, the dead zone can be set as a non-linear function of the step width value.

[0148] In this case, the dead zone is set based on the dynamic step width and may be adaptive, for example. In this case, the dead zone may change as the step width changes. For example, if the step width is updated to 3 instead of 5, the dead zone of 2.4 * step width may change from the range of -6 to +6 to the range of -3.6 to 3.6; or, if the step width is updated to 10, the dead zone may change to expand from -12 to 12. In this case, the multiplier for the step width may be in the range of 2 to 4. In this case, the multiplier may also be adaptive based on operating conditions, such as the available bit rate, for example.

[0149] Having a dead zone can help reduce the amount of data transmitted over a network, for example, by helping to reduce the bit rate. When using a dead zone, residual or coefficient values ​​within the dead zone are effectively ignored. This approach can also help remove low-level residual noise. Having an adaptive dead zone rather than a constant one means that when the step width decreases (e.g., when more bandwidth is available), smaller residual or coefficient values ​​are not over-filtered, and when the step width increases, the bit rate is reduced appropriately. The dead zone only needs to be defined in the encoder, while the decoder simply receives a quantized value of 0 for any residual or coefficient within the dead zone.

[0150] FIG. 8c illustrates how an approach referred to as bin folding can be applied. In the example of FIG. 8c, bin folding is used with a dead zone; however, in other cases, it can be used without a dead zone and / or with other quantization approaches. In FIG. 8c, bin folding acts to place all residual or coefficient values ​​residing on a selected quantization bin into the selected bin. For example, this can be viewed as a form of clipping.

[0151] In Fig. 8c, a step width of 5 is applied again. A dead zone of the range of 2.4*stepwidth is also applied, so that values ​​between -6 and 6 are set to 0. This can also be seen as leading to a first quantization bin larger (having a value of 0). Subsequently, two quantization bins with a width of 5 are defined for positive and negative values. For example, a bin with a quantization value of 1 is defined between 6 and 11 (e.g., having a step width of 5), and a bin with a quantization value of 2 is defined between 11 and 16. In this example, to define bin folding, all residuals or coefficients having values ​​that generally belong to a bin above the second bin (e.g., having a value greater than 16) are "folded" into the second bin and clipped to have a quantization value of 2. This can be done by setting all values ​​greater than a threshold to the maximum bin value (e.g., 2). A similar process occurs for negative values ​​as well. This is exemplified in Fig. 8c by the large arrow.

[0152] Bin folding may be a selectable processing option in the encoder. It does not need to be specified during dequantization in the decoder (e.g., a "folded" or "clipped" value of 2 is simply dequantized as if it were in the second bin). Bin folding may be specified to reduce the number of bits sent to the decoder over the network. Bin folding may be configurable to reduce the bit rate based on network conditions and / or base stream processing.

[0153] Figure 8d illustrates how a quantization offset can be used in specific cases. The quantization offset can be used to shift the position of the quantization bin. Figure 8d illustrates a line representing possible actual counts along a range of x-axis residual or coefficient values. In this example, many values ​​are near 0, and as one moves away from 0, the counts of higher values ​​decrease. When the count values ​​are normalized, the line can also represent a probability distribution for the residual or coefficient values.

[0154] The dashed line on the right and the bar on the left of Fig. 8d illustrate a histogram modeling quantization. For convenience of explanation, count values ​​for the first through third bins following the dead zone are shown (for both positive and negative values, negative values ​​are indicated by stripes to illustrate the bar). For example, the bar illustrates the counts for the quantized values ​​1, 2, 3 and -1, -2, -3. Due to quantization, the distribution modeled by the histogram differs from the actual distribution illustrated by the line. For example, an error - e - is shown to indicate how the bar differs from the line.

[0155] To change the properties of the error (e), a quantization offset—qO—can be applied. The quantization offset can be understood as a parameter that defines whether and to what extent a quantization interval or bin should be shifted from a predetermined or default position or set of values. For a positive value, a positive quantization offset acts to shift each bin to the right, and a negative quantization offset acts to shift each bin to the left. The quantization offset can be applied together with a dead zone. In this case, the dead zone can be applied based on a first set of thresholds, for example, all values ​​smaller than (n*stepwidth) / 2 and larger than (n*stepwidth*-1) / 2 are set to 0.

[0156] In some examples, the quantization offset can be signaled to the decoder for use in dequantization.

[0157] In one case, in the encoder, the quantization offset may be subtracted from the residual or coefficient value before quantization based on the step width. Thus, the signaled offset may be added to the quantized value received in the decoder to dequantize based on the step width. In certain cases, the offset may be adjusted based on the sign of the residual or coefficient to allow for symmetric operation with respect to a value of 0. In one case, the use of the offset may be disabled by setting the quantization or dequantization offset value to 0. In another case, the applied quantization offset may be adjusted based on a defined dead zone width. In yet another case, the dead zone width may be calculated in the decoder as a function of the quantization parameter and the step width received from the encoder, for example.

[0158] In one case, the step width for quantization may vary for different coefficients within a 2x2 or 4x4 block of coefficients. For example, a smaller step width may be assigned to coefficients determined experimentally to have a greater impact on the recognition of the decoded signal; for instance, in the 2x2 or 4x4 directional decomposition (DD-squared or "DDS") described earlier, the A, H, V, and D coefficients may be assigned a smaller step width, while the later coefficients may be assigned a larger step width. In this case, a base_stepwidth parameter may be defined to set the default step width, which can then be applied to calculate the modified_stepwidth to be used for quantization (and dequantization); for instance, modified_stepwidth = base_stepwidth * modifier (where the "modifier" can be set based on specific coefficients within a block or unit and can be derived from signaling, such as the variable "qm" described later).

[0159] In certain cases, the modifier may also, or alternatively, vary depending on the enhancement level. For example, the step width may be smaller for a level 1 enhancement stream because it can affect a large number of reconstructed pixels at a higher quality level.

[0160] In certain cases, modifiers may be defined based on both coefficients and refinement levels within a block. In such cases, the quantization matrix may be defined as a set of modifiers for different coefficients and different refinement levels. This quantization matrix may be preset (e.g., in the encoder and / or decoder), signaled between the encoder and decoder, and / or dynamically configured in the encoder and / or decoder. For example, in the latter case, the quantization matrix may be configured in the encoder and / or decoder as a function of other stored and / or signaled parameters, e.g., those received through a configuration interface.

[0161] In this case, different quantization modes, or different ways of defining the quantization matrix to be applied to a given set of coefficients, may be defined. In one mode, a common quantization matrix may be used for all quantization levels; in another mode, distinct matrices may be used for different levels; in yet another mode, the quantization matrix may be used for only one improvement level, e.g., only for level 2. The quantization matrix may be indexed by the position of the coefficients within a block (e.g., 0 or 1 in the x direction and 0 or 1 in the y direction for a 2x2 block, or 0 to 3 for a 4x4 block).

[0162] In one case, the base quantization matrix can be defined as a set of values. This base quantization matrix can be modified by a scaling factor that is a function of the step width for one or more enhancement levels. In another case, the scaling factor can be a clamped function of the step width variable. In the decoder, the step width variable may be received from the encoder for one or more of the level-1 stream and the level-2 stream.

[0163] Through some additional examples of processes involving the principles described above, advantageous modes for configuring and adapting quantization to further improve encoding and decoding procedures can be understood from the following additional perspective. Data block unit improvement payload semantics can be applied, involving multiple parameters in which the properties of the quantization and dequantization steps can be signaled and configured.

[0164] In one example, the dequant_offset_signaled parameter specifies whether the value of the offset parameter to be applied during dequantization is signaled. In this way, it is possible to signal whether the offset is sent. In this example, if the offset is sent, it is then used. If it is not sent, the default offset is used, or the offset may not be used.

[0165] In an example of a method for encoding an input video into a plurality of encoded streams, wherein the encoded streams can be combined to reconstruct the input video, the method may include the step of receiving an input video and the step of downsampling the input video to generate a downsampled video.

[0166] The method typically comprises the steps of: commanding the encoding of a downsampled video using a base encoder to generate a base-encoded stream (the base-encoded stream); commanding the decoding of the base-encoded stream using a base decoder to generate a reconstructed video; comparing the reconstructed video with the downsampled video to generate a first residual set; and further comprising the step of encoding the first residual set to generate a first level-encoded stream. This preferably comprises the steps of: applying a transformation to the residual set to generate a coefficient set; applying a quantization operation to the coefficient set to generate a quantized coefficient set; and applying an encoding operation to the quantized coefficients, wherein the quantization operation is performed using a quantization matrix, and the quantization matrix is ​​derived according to the acquired values ​​of the quantization matrix mode parameters.

[0167] As described above in the present disclosure, the quantization matrix mode parameter may be advantageously used to specify the quantization matrix to be used in the encoding process. In some examples, when the quantization matrix mode parameter value is equal to a predetermined value, for example, when it is equal to zero (0), the method may involve using a default quantization matrix for each of the two quality levels. These levels typically correspond to Level 1 and Level 2 enhancement streams. When the quantization matrix mode parameter value is equal to 1, a first quantization matrix may be used for each of the two quality levels, and the first quantization matrix may be signaled, for example, from an encoder to a decoder or to a device to which the encoded stream will be transmitted. When the quantization matrix mode parameter value is equal to 2, a second quantization matrix may be used for Quality Level 2, and the second quantization matrix may be signaled. In this case, a quantization matrix may not be used for Quality Level 1, or a default value may be used for this level. When the quantization matrix mode parameter value is equal to 3, the third quantization matrix is ​​preferably used for quality level 1, or for a first-level encoded stream, and the third quantization matrix is ​​signaled. In this case, the quantization matrix may not be used for quality level 2, or a default value may be used for this level. When the quantization matrix mode parameter value is equal to 4, the fourth quantization matrix may be used for a first-level encoded stream and the fifth quantization matrix may be used for a second-level encoded stream (e.g., two matrices may be used), each of which may or may not be identical to each other and may be any one of the first to third matrices described above. In this fifth mode, the fourth quantization matrix and the fifth quantization matrix may be signaled to a decoder or other device to which the encoded stream is to be transmitted.

[0168] In the procedure described in this disclosure, as previously described, all groups of transform factors passed to this process typically belong to specific planes and layers. Typically, they have been scaled using a linear quantizer that uses a non-centered dequantization offset in some examples. The scaling process for the transform factors may be applied as follows. The process may take location information for the block along with a set of parameters that can indicate properties of the block, such as its size, properties of the quantization operation, such as step width and offset values, and the enhancement level applied. For example, a location may be used that can be indicated as a pair of coordinate values ​​or, for example, by a parameter (xTbP, yTbP) specifying the top-left sample of the current lumina or chroma transform block for the top-left lumina or chroma sample of the current picture. This may be associated with a specific part of the data representing the image, and may be associated with either the lumina plane or the chroma plane depending on the plane to which the transform factors belong.

[0169] The previously described parameter specifying the size of the current transform block, which may be referred to as nTbS in some examples, may have a value that depends on the parameter value defining the transform type, specifically the transform used for decoding. This type parameter may be referred to as transform_type in some examples and may have a value of 0, 1, or 2-3, corresponding to a value or parameter that specifies a 2x2 directional decomposition transform, a 4x4 directional decomposition transform, or zero because these elements are not used in the bitstream in some applications. In some examples, this transform_type parameter with a value of 0 may correspond to a number of layers equal to 4, and if transform_type is equal to 1, the corresponding number of layers may be 16. The size (nTbS) parameter may have a value of 2 if transform_type is equal to 0 and a value of 4 if transform_type is equal to 1.

[0170] Additional input to the process is typically in the form of an array of entropy-decoded quantized coefficients. This may be referred to as TransCoeffQ and may be of a size related to the previously described size parameter, specifically having dimensions (nTbS)x(nTbS), that is, the array may include coding units or blocks as described in this application. This array may be referred to as TransCoeffQ. The step width value may be specified by a parameter referred to as stepWidth. The index of the refinement level may also be specified by a parameter, and in some examples, may be referred to as idxLevel. If a dequantization offset is used, it may be specified by a parameter referred to, for example, as dQuantOffset. The parameter typically specifies the value of the dequantization offset parameter to be applied. This process typically generates an array of dequantized transform coefficients. This may be in the form of an array of dimensions (nTbS)x(nTbS) containing, for example, d[ x ][ y ] elements.

[0171] The value of the output array, which may be referred to as d[ x ][ y ], is typically derived based on an arbitrary index of the enhancement level applied to the quantization matrix, which may be referred to as the step width and / or qm described earlier, and the corresponding element, and the entropy-decoded quantized coefficient, which may be named TransCoeffQ[ xTbP ][ yTbP ]. Additionally, the value of the output array element may be derived by additionally applying an offset to the result of the operation described earlier. For example, the output element value may be generated by adding a parameter, which may be named appliedOffset, to the value.

[0172] This, therefore, may entail the following relationship to calculate each element d of the array.

[0173] d[ x ][ y ]=(TransCoeffQ[ xTbP ][ yTbP ]*(stepWidth * qm[yTbP + (idxLevel * 4)][xTbP]))+ appliedOffset

[0174] In the above calculation, which is one example of how dequantization can be performed in a decoder, idxLevel can be 0 or 1, representing levels 2 and 1, respectively. In the above calculation, and as described in other examples of this application, values ​​from the quantization matrix are selected based on specific coefficients and specific enhancement levels.

[0175] With respect to the offset that may be applied in some examples, this parameter, which may be referred to as appliedOffset as illustrated in the relationship above, is typically derived as follows below.

[0176] In certain cases, appliedOffset may be applied conditionally based on whether residual filtering is applied. This filtering may be applied after an inverse transform (e.g., 410-1 in FIG. 6) and may be a deblocking filter. In such cases, the offset may be applied only when residual filtering is applied. For example, if the parameter specifying whether to apply a deblocking filter, which may be named deblocking_signaled, is a specific value, e.g., 1, the offset is calculated based on the previously described parameter specifying the dequantization offset, which may be named dQuantOffset.

[0177] appliedOffset can also be derived from a single value of dQuantOffset and can be adapted for application to positive and negative coefficient values. For example, appliedOffset can typically consist of a negative or positive value having the same absolute value as the dequantization offset parameter and a positive or negative sign, respectively. This can be applied when the TransCoeffQ value has a value less than 0; otherwise, if the TransCoeffQ value is greater than 0, the applied offset can be assigned the same value as the dequantization offset parameter. If the TransCoeffQ value is equal to 0, then the applied offset can also be assigned a value of 0.

[0178] Therefore, in a specific example, the derivation of an offset that may be added to obtain an output array element can be derived by the following (where TransCoeffQ may be an array of typically size (nTbS)x(nTbS) containing entropy-decoded quantized coefficients):

[0179] If deblocking_signaled is equal to 1

[0180] If TransCoeffQ[xTbP][yTbP] < 0

[0181] appliedOffset = (dQuantOffset * -1)

[0182] else If TransCoeffQ [ xTbP ][ yTbP ] > 0

[0183] appliedOffset = dQuantOffset

[0184] else

[0185] appliedOffset = 0

[0186] else

[0187] appliedOffset = 0

[0188] For example, a parameter or a set of parameters can be signaled as previously described and can be used to specify how the quantization matrix is ​​applied to a set of coefficients. This parameter may be the quant_matrix_mode parameter, which specifies the quantization matrix to be used in the decoding process according to Table 1 below. The parameter may be 1 byte. The quant_matrix_mode parameter may be used to construct how the quantization matrix is ​​derived (for example, as described later below).

[0189]

[0190] Quantization matrix data semantics can be applied according to the following specific example. A quantization matrix that can be referred to as qm[y][x] is derived.

[0191] A matrix qm can have k*M x N dimensions. That is to say, the matrix can be defined as corresponding to the quantization coefficients contained in an MxN matrix, and qm can contain this data in the form of an MxN matrix of quantization coefficients for each of the k refinement levels, for example.

[0192] In this example, two improvement levels, namely levels 1 and 2 described in this disclosure, are involved, and k is equal to 2. Since the largest transformation described in relation to this procedure is 4x4, M and N can each be equal to 4.

[0193] In some examples, as previously explained, by defining qm with M and N values ​​corresponding to the maximum available transformation, which is a 4x4 transformation and therefore contains 16 coefficients in these examples, the application of the quantization matrix in which the smaller transformation is used can be performed by reading from qm and applying a subset of coefficients in each M x N matrix.

[0194] In a specific example, when the value of the quantization matrix mode parameter is equal to 0, that is, when quant_matrix_mode is equal to 0, the following default quantization matrix is ​​used for the quantization operation:

[0195] qm[ y ][ x ] =

[0196] { 0.500 0.500 0.500 0.617}

[0197] { 0.862 0.610 1.064 0.781}

[0198] { 0.500 0.500 0.500 0.617}

[0199] { 3.125 1.851 1.851 1.316}

[0200] { 0.500 0.500 0.500 0.617}

[0201] { 0.862 0.610 1.064 0.781}

[0202] { 0.862 1.064 0.610 0.781}

[0203] { 3.125 1.851 1.851 1.316},

[0204] Here, y = 0..3 is the coefficient used for improvement level 2 and y = 4..7 is the coefficient used for improvement level 1;

[0205] If the value of the quantization matrix mode parameter is equal to 1, a single modifier matrix is ​​signaled and used at both enhancement levels. The matrix can be constructed through an interactive process in some applications. Thus, the qm[x][y] values ​​can be obtained row by row, and accordingly, when a row is processed, the value of that row is filled for each row, and each column of the matrix is ​​filled. As described below, the step width modifier parameter can be used to obtain these matrix values. In this example, the number of rows to be filled can be 8, and the number of columns is 4, as previously described in relation to the matrix dimensions of this particular example. Specifically, the iterative process of this example can be written as follows:

[0206] for (y = 0; y < 8; y++)

[0207] for (x = 0; x < 4; x++)

[0208] qm[y][ x ] = step_width_modifier_2[x + y*4],

[0209] Here, the step width modifier parameter, which may be referred to as step_width_modifier_2 in this example, is a parameter that specifies the value of the Level 2 enhancement 16 factor to be applied to different layers of the transformation factor;

[0210] If the value of the quantization matrix mode parameter is equal to 2, one modifier matrix is ​​signaled and used at refinement level 2. Likewise, an iterative procedure corresponding to the previously described one can be used specifically as follows:

[0211] for (y = 0; y < 4; y++)

[0212] for (x = 0; x < 4; x < 4)

[0213] qm[ y][ x ] = step_width_modifier_2[x + y*4]; and

[0214] When the value of the quantization matrix mode parameter is 3, one modifier matrix is ​​signaled and used for enhancement level 1:

[0215] for (y = 0; y < 4; y++)

[0216] for (x = 0; x < 4; x < 4)

[0217] qm[y + 4][ x ] = step_width_modifier_2[x + y*4]; and

[0218] When the value of the quantization matrix mode parameter is 4, two modifier matrices are signaled, the first matrix is ​​for refinement level 2 and the second matrix is ​​for refinement level 1:

[0219] for (y = 0; y < 4; y++)

[0220] for (x = 0; x < 4; x < 4)

[0221] qm[ y][ x ] = step_width_modifier_2[x + y*4]

[0222] for (y = 0; y < 4; y++)

[0223] for (x = 0; x < 4; x < 4)

[0224] qm[ y][ x ] = step_width_modifier_1[x + y*4],

[0225] Here, step_width_modifier_1 is a parameter that specifies the value of the level 1 improvement 16 factor to be applied at different levels of the transformation factor.

[0226] As previously described, the d[ x ][ y ] value of the matrix can be calculated by multiplying, for example, the product of the respective transformation coefficients in the relevant elements of the matrix, the corresponding quantization matrix element values ​​in the column identified by the product of the levelIdxSwap parameter and the parameter specifying the size of the current transformation block, and the sum of the step width modifier parameter values ​​corresponding to the elements in the corresponding rows, with the previously described offset typically applied to this product as well.

[0227] The dequantization process described above can be performed according to the following ordered steps. The dequantization process according to the example described above can be called using as input the luminance location (xTbY, yTbY), a set of transformation sizes identical to nTbS, i.e., the size of the current transformation block, an array TransCoeffQ of size (nTbS)x(nTbS) as described above, and a step width parameter. Then, the output can be an array of dequantized coefficients (e.g., dequantized transformed residuals) of dimension (nTbS)x(nTbS), which can be referred to as dequantCoeff in this example.

[0228] In the following additional examples, for ease of understanding, the signal is referred to as a video signal. In these examples, the signal consists of a sequence of samples, and in the case of a video signal, these may be frames. If the video is a picture element or a pixel, each sample (e.g., a frame) consists of multiple data elements.

[0229] Additionally, in this example, quantized data is referenced. This quantized data may consist of multiple subsets of encoded data. As previously described, the quantized data may correspond to transformed coefficients, that is, data generated by applying a transformation process to the original data set of the signal. This data may be data elements (e.g., pixels, etc.). This data may be residual data, which may be obtained as the difference between a prediction of a sample or data element and a reference of the same sample or data element. The transformation may be a DCT-like transformation, a wavelet transform, an Hadamard-like transformation, a directional decomposition transformation such as that described in patent applications PCT / EP2013 / 059847 and PCT / GB2017 / 052632 (both incorporated by reference in this application), or any other type of transformation appropriate for a particular signal. Each of the multiple subsets of quantized data may correspond to one of the coefficients of a specific transformed quantized data. As a non-limiting example, in the case of a 4x4 transformation, 16 coefficients can be generated, and thus, there can be 16 subsets of quantized data.

[0230] Quantized data may include two or more layers of quantized data. For example, in a multi-layer encoding scheme such as MPEG-5 Part 2 (LCEVC), there may be at least two refinement sublayers, as previously described. Each of these refinement sublayers has its own quantized data, and each is quantized in a different way from the others.

[0231] Referring to FIG. 9, an example of a decoding module (1100) is illustrated. The decoding module (1100) receives a plurality of input bitstreams including an encoded base (1101), an L1 count group (1121), an L0 count group (1131), a time count group (1136), and a header (1140).

[0232] Generally, the decoding module (1100) processes two data layers. The first layer, namely the base layer (1110), includes a received data stream (1111) containing an encoded base. The encoded base (1111) is then sent to a base decoding module (1112) that decodes the encoded base (1111) to generate a decoded base picture (1113). The base decoding may be a decoder that implements any existing base codec algorithm, such as AVC, HEVC, AV1, VVC, EVC, VC-6, VP9, ​​etc., depending on the encoded format of the encoded base (1111).

[0233] The second layer, i.e., the enhancement layer, is further composed of two enhancement sub-layers. The decoding module receives a first group of coefficients, i.e., the L1 coefficient group (1121), which is then passed to the entropy decoding module (1122) to generate a decoded coefficient group. This is then passed to the inverse quantization module (1123), which uses one or more dequantization parameters to generate a dequantized coefficient group. This is then passed to the inverse transform module (1124), which performs an inverse transform on the dequantized coefficient group to generate a residual (L1 residual) in the enhancement sub-layer 1. The residual can then be filtered by a smoothing filter (1125). The L1 residual (i.e., the decoded first enhancement sub-layer) is applied to the processed output of the base picture (1113).

[0234] The decoding module receives a second group of coefficients, namely the L0 coefficient group (1131), which is then passed to the entropy decoding module (1132) to generate a decoded coefficient group. This is then passed to the inverse quantization module (1134), which uses one or more dequantization parameters to generate a dequantized coefficient group. The dequantization parameters used in the improvement sublayer 0 may differ from the dequantization parameters used in the improvement sublayer 1. The dequantized coefficient group is then passed to the inverse transform module (1135), which performs an inverse transform on the dequantized coefficient group to generate a residual in the improvement sublayer 0 (L0 residual).

[0235] Referring to FIG. 10, an example of a decoding process is illustrated. A decoded base picture is acquired from a base decoder. This picture may be optionally upsampled to generate a preliminary intermediate picture. A first layer of residuals is added to the preliminary intermediate picture to generate a combined intermediate picture. The first layer of residuals (L1 residuals) serves to correct the decoded base picture, for example, by correcting blocking artifacts generated by the decoded base picture. The combined intermediate picture may then be upsampled to generate a preliminary output picture. A second layer of residuals is added to the preliminary output picture to generate a combined output picture. The second layer of residuals (L0 residuals) serves to add detail and sharpness to the preliminary output picture. Data may also be present in a time buffer used with the L0 residuals.

[0236] Referring to FIG. 11, a non-limiting exemplary embodiment according to the present invention is illustrated. A bitstream (200) is received by a decoding module. The bitstream includes a series of bits corresponding to metadata and / or encoded data and any other information that may be useful for decoding the encoded data. In an example, the bitstream may comply with a bitstream of standard video coding. In a non-limiting example, the standard may be an MPEG family standard such as MPEG 5 Part 2 (LCEVC), MPEG 5 Part 1 (EVC), VVC, AVC, HEVC, etc., or a standard created by another standard organization such as AoM (e.g., AV1) or SMPTE (e.g., VC-6 / ST-2117).

[0237] In the bitstream (1200), there may be some information related to determining the dequantization parameters required to decode one or more subsets of the quantized data.

[0238] Information of the bitstream (1200) can be obtained by a decoding module. The first information may include a quantization step (1210). The quantization step relates to the step width used to quantize the data. As a non-limiting example, if the original data can take values ​​between +32,768 and -32,768, a quantization step of 256 would mean that 256 symbols will be generated from these original values, and each symbol will contain a value within the range of 256. Thus, for example, values ​​between 0 and 255 are all quantized into the same symbol, and values ​​between 256 and 512 are quantized into the same symbol, and so on. The quantization step may be signaled per sample of the signal (e.g., per frame) and may differ per sublayer (e.g., in the case of a multilayer encoding scheme). For example, there may be a first step width for a first refinement sublayer and a second step width for a second refinement sublayer.

[0239] The information in the bitstream (1200) may also include information (220) that signals the type of dequantization process to be used. This information may take the form of a look-up-table (LUT) (1230) indicating which process to use based on values ​​provided in the information, for example. For example, a value 0 may be associated with process #0 and a value 1 may be associated with process #1, and so on. As a non-limiting example, process #0 may indicate that the same set of dequantization parameters used in the previous sample should be used for the current sample. As a non-limiting example, process #1 may indicate that the default set of dequantization parameters should be used for the current sample. As a non-limiting example, process #2 may indicate that the signaled set of quantization parameters should be used for the current sample instead of the default set of quantization parameters. In the example, the set of quantization parameters may include two or more subsets, each subset corresponding to a layer of quantized data. For example, a first quantization parameter subset may correspond to a first refinement sublayer of the quantized data, and a second quantization parameter subset may correspond to a second refinement sublayer of the quantized data. As a non-limiting example, process #3 may indicate that a signaled quantization parameter subset is used to modify a portion of a default quantization parameter set (e.g., one of the subsets) to generate a modified quantization parameter set, and the modified quantization parameter set is used for the current sample instead of the default quantization parameter set. In the example, the quantization parameter set may consist of multiple scaling factors. Then, the scaling factors may be used to generate scaled quantization parameters to be used to dequantize the quantized data.

[0240] The information in the bitstream (1200) may also include information (1240) that signals a new set of quantization parameters to be used. In the example, the set of quantization parameters may consist of a number of scaling factors. This information may be included in the bitstream (1200) only when a process indicating the use of the signaled quantization parameters is signaled via the information (1220). In this way, this information is sent only when necessary, thus preventing the size of the bitstream (1200) from being increased when not needed.

[0241] Next, the decoding module may use the dequantization parameter module (1270) to generate actual dequantization parameters (1280) to be used when dequantizing the quantized data. The dequantization parameter module (1270) may receive various information as input. For example, in one embodiment, the module (1270) may receive a quantization step (1210), a dequantization selection indicator (220), scaling factor(s) (1240), a predetermined scaling factor(s) (1250), and other parameters (1260). The predetermined scaling factor(s) (1250) may correspond to the scaling factor used in the previous sample or to a default set of scaling factors. Other parameters may include other elements that modify the quantization parameters, such as one or more parameters that modify the step width based on whether a block of data elements is considered static, semi-static, or non-static (as also described in patent application GB 1911545.0, which is incorporated by reference into this application), or one or more dequantization offsets to be applied (as also described in patent application GB 1914413.8, which is incorporated by reference into this application).

[0242] As explained earlier, the actual quantization parameters ( ) is a scaling factor determined based on the signaled process as explained earlier ( ) received as a quantization step (210)( It can be calculated by processing ). The calculation can be performed by a function of at least these two arguments as follows:

[0243]

[0244] Here, f(ㆍ) can be the following multiplication:

[0245]

[0246] After that, dequantized data can be calculated as follows:

[0247]

[0248] Here is a quantized data subset and is the actual quantization step for the i-th quantized data subset.

[0249] Next, actual quantization parameters can be used to dequantize the quantized data. In the case of quantized data containing multiple quantized data subsets, there may be multiple actual quantization parameters, each of which is associated with another of the multiple quantized subsets.

[0250] In addition, as explained earlier, in other examples, the actual quantization parameters ( ) is a step width modifier( It can be further calculated based on other information such as ), and a dequantization offset (DO(s)) can be used for dequantization.

[0251]

[0252] Here, f(ㆍ) can be as follows:

[0253]

[0254] After that, dequantized data can be calculated as follows:

[0255]

[0256] Here is a quantized data subset and is the actual quantization step for the i-th quantized data subset and DO i is the dequantization offset for a specific i-th quantized data subset.

[0257] Exemplary implementation

[0258] According to a specific example, the scaled transformed coefficient can be given as d[x][y] and can be determined according to the following:

[0259] d[x][y] = (TransformCoeffQ[x][y] * ((qm[x + (levelIdxSwap * nTbS)][y] + stepWidthModifier[x][y]) + appliedOffset [x][y])

[0260] Here:

[0261] TransformCoeffQ[x][y] represents an array of coefficients;

[0262] qm[x][y] represents the quantization parameter or the quantization matrix of the step width;

[0263] stepWidthModifier[x][y] represents an array of modification parameters;

[0264] appliedOffset[x][y] is an array that functions as a dead zone as described elsewhere in this application, that is, in a specific signaled scenario, a dequantization offset can be used to change the dead zone or can be zero.

[0265] If the flag signals that the dequantization offset is not used, then the step width modifier may be a function of the step width, for example, as follows:

[0266] stepWidthModifier [x][y] = ((((Floor(-Cconst * Ln (qm[x + (levelIdxSwap * nTbS)][y]))) + Dconst) * (qm[x + (levelIdxSwap * nTbS)][y] 2 ))) / 32768) >> 16

[0267] If the flag signals that a dequantization offset is used, then the step width modifier may be based on a set of offset values, for example, as follows:

[0268] stepWidthModifier [x][y] = (Floor((dQuantOffsetActual [x][y]) * (qm[x + (levelIdxSwap * nTbS)][y])) / 32768)

[0269] Here, the offset value is a function of the step width (i.e., qm[x][y]), and, for example, as follows:

[0270] dQuantOffsetActual [x][y] = ((Floor(-Cconst * Ln(qm[x + (levelIdxSwap * nTbs)][y]) + (dQuantOffset << 9) + Floor(Cconst * Ln(StepWidth)))) * (qm[x + (levelIdxSwap * nTbs)][y])) >> 16

[0271] In a specific example, the quantization matrix qm[x][y] contains the actual quantization step width used to decode each group of coefficients. In a specific mode, the matrix can be based on (QuantScalerDDBuffer[x][y] * stepWidth), where QuantScalerDDBuffer[x][y] is an array of scaling values.

[0272] Here, the buffer is essentially a scaling buffer that modifies the step width. In this particular implementation, the value scales the width rather than acting as a mask as specified in alternative implementations. The buffer is affected by mode signaling. That is, the matrix may be a default 2x6 matrix or may differ for different layers and upscaling. For example, for 1D or 2D upscaling, if parameters must be signaled for each frame, signaling the matrix alone requires a high signaling cost. The default scaling factor may be more efficient. However, improvements may be made to signal a different matrix for a specific frame or to change the matrix from the default. Here, the buffer may be persistent for a picture group or a specific subset. A mechanism may also be provided to change the matrix back to the default using mode signaling parameters. Changing the mode signaling, rather than re-signaling the matrix, allows the scaling factor to be used once again in the buffer. The signaling described in this application is provided to maintain the previously signaled matrix. The signaling described in this application provides a reset (or refresh) of parameters at the end of a picture group unless otherwise signaled.

[0273] It should be noted that the matrix signaling described in this application provides significant advantages. For example, if there are 16 values, 1 byte is required to signal the parameters. Given that there may be 60 frames per second for video, 7.6 Kbps of signaling alone may be required to send the parameters for each frame.

[0274] The following table describes examples of signaling modes. It should be noted that these may be alternative to or additional to Table 1 above.

[0275]

[0276] Note that in this table, IDR refers to a frame in which the global composition is sent along with the frame. An IDR picture is a frame that does not reference any other picture. One specific advantage of an IDR picture is that it allows for a reset (or refresh) and can be used, for example, when there is a change in the scene. It should be noted that while an IDR can be considered an i-frame, an i-frame is not an IDR because they possess different properties.

[0277] Here, it can be seen that mode 0 is used to signal that the parameters to be used have been previously signaled. In this way, the cheapest signaling is used to signal that the buffer should be maintained and used. That is, the signaling is the most efficient. In the example, when 0 is used, it can be efficiently entropy-coded.

[0278] Therefore, efficient scaling can be performed depending on different mode combinations. Accordingly, the default can be signaled to skip scaling stored in memory. Memory can be updated by signaling from the encoder, or it can be predetermined or calculated. Modes and scaling factors can be signaled per frame, per picture group, or per tile. Mode signaling can cause the system to maximize results for small bitrates available in the residual set.

[0279] Additional implementation

[0280] It should be noted that in certain embodiments as described in this application, in exemplary implementations, the following may be implemented for lossless compression:

[0281] Here, for stepWidth > 16, deadZoneWidthOffset is derived as follows:

[0282] deadZoneWidthOffset [x][y] = (((1 << 16) - ((Aconst * (qm[x + (levelIdxSwap * nTbs)][y] + stepWidthModifier [x][y])) + Bconst) >> 1) * (qm[x + (levelIdxSwap * nTbs)][y] + stepWidthModifier [x][y])) >> 16

[0283] Here, for stepWidth <= 16, deadZoneWidthOffset is derived as follows:

[0284] deadZoneWidthOffset [x][y] = stepWidth >> 1

[0285] That is, when the step width is less than 16, the deadzonewidthoffset does not depend on the parameter matrix.

[0286] Additional exemplary implementations

[0287] The appendix attached below provides examples of non-limiting implementations of the principles described elsewhere in this application. These examples may be used to provide context for the described features.

[0288] Additional exemplary encoders and decoders

[0289] FIGS. 25 and 26 illustrate variations of the encoder architecture of FIGS. 1, 3a and 3b and the decoder architecture of FIGS. 2, 5a and 5b, respectively.

[0290] An encoding process (2500) for generating a bitstream is illustrated in FIG. 12. First, an input sequence (2502) is fed to a first downsampler (2504), and then a second downsampler (2506) (i.e., a successive downsampler referred to as a down-scaler in the drawing) is processed according to a selected scaling mode. The variation of FIG. 12 differs from the variation of the previous example in that additional downsampling and upsampling stages exist prior to the base layer, for example, the additional downsampling stage illustrated as the second down-scaler (2506) is possible before the data is passed to the base encoder (2512), and the additional upsampling stage (illustrated as the first up-scaler (2508) in FIG. 12) is possible after receiving the data decoded from the base layer. In a particular example, a given scaling mode may be used to turn on and turn off a pair of down-scalers and up-scalers at each stage. In this case, the scaling mode may indicate the direction of scaling, for example, according to the horizontal single downsampling / upsampling described in the present application. When the second down-scaler (2506) and the first up-scaler (2508) are turned off, the spatial scaling is similar to that of FIG. 1, 3a and 3b.

[0291] In FIG. 12, a base codec is used to generate a base bitstream (2516) according to its own specifications, as in the previous example. This encoded base may be included as part of a combined bitstream for the current video coding framework.

[0292] With or without additional upscaling, a reconstructed base picture, e.g., a decoded version of a base-encoded frame, is subtracted from the first downscaled input sequence in the first subtraction component (2520) to generate a sub-layer 1 residual (level 1 residual data as described in the present application). This residual forms a starting point for the encoding process of the first enhancement layer. The transform component (2521), quantization component (2523), and entropy encoding component (2524) as described in the present application (among other things) process a first set of (level 1) residuals to generate (level 1) entropy-encoded quantized transform coefficients (2526).

[0293] In FIG. 12, according to the previous example, the entropy-encoded quantized transform coefficients from sub-layer 1 are processed by an in-loop decoder that performs an inverse operation or a decoding operation. This operation simulates the decoding process for a first set of residuals to be performed in the decoder. In one example of FIG. 12, this includes an entropy decoding component (2525), an inverse quantization component (2527), an inverse transform component (2528), and a level 1 filter (2530). These may be similar to the components described earlier. The processed or "decoded" first set of residuals is added to the data derived from the output of the base encoder (e.g., decoded and optionally upscaled) in the summing component (2532) to generate a reconstructed frame. In FIG. 12, the reconstructed frame is processed by a second upscaler (2534). The use of the upscaler may again depend on the selected scaling mode. Finally, the residuals for the second sublayer 2 (which may also be referred to as the L2 layer) are calculated in the second subtraction component (2536) by subtracting the input sequence and the upscaled reconstruction. These form a second set of (level 2) residuals, and these residuals are processed by a coding component or tool set, which includes a transform component (2541), a time prediction component (2542), a quantization component (2543), and an entropy encoding component (2544). The output is a set of level 2 coefficient layers (2546). As described in other examples, when the time mode is activated, additional time prediction may be applied to the transform coefficients by the time prediction component (2542) to remove specific temporal redundancy information and reduce the energy of the level 2 residual stream (e.g., the number of values ​​and the number of non-zero residual values). The time layer (2556) that specifies the use of block-based time prediction as well as the entropy-encoded quantized transform coefficients of sub-layer 2 is included in the improved bitstream.The time layer (2556) may include the time signaling described with reference to the previous example. It may be entropy encoded by the entropy encoding component (2557). The entropy encoding component (2557) may apply at least run-length encoding as described with reference to the example.

[0294] The encoder (2500) may be composed of a set of encoder configuration information (2565), for example, as described with reference to the examples in FIGS. 14a through 14c. This information may be transmitted to the decoder as a set of headers (2566) for the output bitstream. In FIG. 12, the combined bitstream for the encoder may include a header (2566), a time layer (2556), a level 2 (L2) encoded coefficient (2546), a level 1 (L1) encoded coefficient (2526), ​​and an encoded base stream (2516).

[0295] FIG. 13 illustrates a variation of a decoder (2600) according to one example. The decoder may include a variation of the decoder illustrated in any of the other drawings of the present application. The decoder of FIG. 13 may be used together with the encoder of FIG. 12.

[0296] First, to generate an output sequence of frames, the decoder (2600) analyzes the bitstream. As can be seen in FIG. 13, the process can be divided into three parts.

[0297] To generate a decoded base picture (e.g., at layer 0), the extracted base bitstream (2616) is supplied to the base decoder (2618). According to the selected scaling mode, this reconstructed picture may be upscaled by an additional first upscaler (2608) prior to a summing component (2630) that adds a first set of (level 1) residuals. The input from the first upscaler (2608) to the summing component (2630) may be referred to as a preliminary intermediate picture.

[0298] Following (or in parallel with) base layer decoding, an enhancement layer bitstream (containing two sub-layers of residuals) needs to be decoded. First, the coefficients (2626) belonging to sub-layer 1 (L1) are decoded using the inverse version of the coding component or tool used during the encoding process. Thus, the level 1 coefficient layer (2626) is processed, in turn, by the entropy decoding component (2671), the inverse quantization component (2672), and the inverse transform component (2673). Additionally, a sub-layer 1 (L1) filter (2632) may be applied to smooth the boundaries of the transform block (i.e., the coding unit). The output of the sub-layer 1 (L1) decoding process may be referred to as the enhancement sub-layer 1 output. This enhancement sub-layer 1 output is added to a preliminary intermediate picture in the first (sub) summing component (2630) to produce a combined intermediate picture. Also, depending on the scaling mode, a second up-scaler (2687) may be applied, and a resulting preliminary output picture may be generated. The preliminary output picture is provided to a second upper summing component (2658). It has the same dimensions as the full output picture.

[0299] As a final step, the encoded coefficients (2646) for the second enhancement sublayer (2) are decoded. Again, this uses a set of inverse coding components or tools as described in other examples in this application. In FIG. 13, this component includes an entropy decoding component (2681), an inverse quantization component (2682), and an inverse transform component (2683). When the time mode is activated, the time prediction component (2685) can apply a time prediction. The time prediction can be applied at any point within the second enhancement sublayer (2). In this case, it is applied to the quantized transform coefficients. The time prediction can be applied based on the received signaling as the time layer (2656). In FIG. 13, the time layer (2656) is decoded by the entropy decoding component (2690) (e.g., run-length decoding). The output of the time prediction is provided to the second upper summing component (2658) as an improved sub-layer 2 output. Then, this is added to the preliminary output picture by the summing component (2658) to form a combined output picture (2660) as the final output of the decoding process.

[0300] In other words, the decoding process can be controlled according to the decoder configuration (2692) transmitted within the header (2666) of the bit stream.

[0301] As described with reference to the example above, unlike the scalable codec of comparison, the novel approach described in this application can be completely agnostic to the codec used to encode the sublayer. This is because the upper layer can be decoded without any information about the sublayer. As illustrated in FIG. 13, the decoder receives a plurality of streams generated by the encoder. These may be about five streams including: a first encoded stream (encoded base) generated by feeding a downsampled version of the input video to a base codec (e.g., AVC, HEVC, or any other codec); a second encoded stream (level 1 coefficient layers) generated by processing the residuals obtained by taking the difference between the reconstructed base codec video and the downsampled version of the input video (level 1 residuals); A third encoded stream (level 2 coefficient layers) generated by processing residuals obtained by taking the difference (level 2 residuals) between the upsampled version of the corrected version of the reconstructed base-coded video and the input video; a fourth encoded stream (e.g., in the form of a time layer) generated from time processing to command the decoder; and a fifth stream (header) generated to configure the decoder. The encoded base stream is decoded by a base decoder that implements a decoding algorithm corresponding to the encoding algorithm implemented by the base codec used in the encoder, and the output thereof is the decoded base. Separately and independently, a level 1 coefficient group is decoded to obtain level 1 residual data. Also, individually and independently, a level 2 coefficient group is decoded to obtain level 2 residual data. Subsequently, the decoded base, level 1 residual data, and level 2 residual data are combined. In particular, the decoded base is combined with the level 1 residual data to produce an intermediate picture. Next, the intermediate picture is upsampled and can be further combined with level 2 residual data.

[0302] In addition, the new approach uses an encoding and decoding process to process the picture without using any inter-block prediction. Rather, it processes the picture by transforming NxN blocks of picture elements (e.g., 2x2 or 4x4) and processing the blocks independently of each other. This results in efficient processing as well as a state without dependencies on neighboring blocks, thus enabling the processing of the picture to be parallelized.

[0303] In summary, with reference to FIG. 13, a non-limiting exemplary embodiment is illustrated. FIG. 13 illustrates an exemplary decoding module (2600). The decoding module (2600) receives a plurality of input bitstreams including an encoded base (2616), a level 1 count group (2626), a level 2 count group (2646), a time count group (2656), and a header (2666).

[0304] Generally, the decoding module (2600) processes two data layers. The first layer, i.e., the base layer, includes a received data stream (2616) containing an encoded base. Then, the encoded base (2616) is sent to a base decoding module (2618), and the base decoding module (2618) decodes the encoded base (2616) to produce a decoded base picture. The base decoding may be a decoder that implements any existing base codec algorithm, such as AVC, HEVC, AV1, VVC, EVC, VC-6, VP9, ​​etc., depending on the encoded format of the encoded base.

[0305] The second layer, i.e., the enhancement layer, is further composed of two enhancement sub-layers. The decoding module receives a first group of coefficients, i.e., a level 1 coefficient group (2626), which is then passed to the entropy decoding module (2671) to generate a decoded coefficient group. This is then passed to the inverse quantization module (2672), which uses one or more dequantization parameters to generate a dequantized coefficient group. This is then passed to the inverse transform module (2673), which performs an inverse transform on the dequantized coefficient group to generate residuals (level 1 residuals) in the enhancement sub-layer 1. The residuals can then be filtered by a smoothing filter (2632). The level 1 residuals (i.e., the decoded first enhancement sub-layer) are applied to the processed output of the base picture.

[0306] The decoding module receives a second group of coefficients, i.e., a Level 2 coefficient group (2646), which is then passed to an entropy decoding module (2681) to generate a decoded coefficient group. This is then passed to an inverse quantization module (2682), which uses one or more dequantization parameters to generate a dequantized coefficient group. The dequantization parameters used in the improvement sublayer 2 may differ from the dequantization parameters used in the improvement sublayer 1. The dequantized coefficient group is then passed to an inverse transform module (2683), which performs an inverse transform on the dequantized coefficient group to generate a residual in the improvement sublayer 2 (Level 2 residual).

[0307] avatar

[0308] In both the encoder and the decoder, for example, in a streaming server or client device, or in a client device decoding from a data storage, the method and process described in this application may be implemented as code (e.g., software code) and / or data. The encoder and the decoder may be implemented in hardware or software, as is well known in the field of data compression. For example, hardware acceleration using a specially programmed graphics processing unit (GPU) or a specially designed Field Programmable Gate Array (FPGA) may provide specific efficiencies. For completeness, such code and data may be stored on one or more computer-readable media, which may include any device or medium capable of storing code and / or data for use in a computer system. When a computer system reads and executes code and / or data stored on a computer-readable medium, the computer system performs the method and process implemented by the data structure and code stored on the computer-readable storage medium. In certain embodiments, one or more steps of the method and process described in this application may be performed by a processor (e.g., a processor of a computer system or a data storage system).

[0309] In general, any of the functions described in the text or illustrated in the drawings may be implemented using software, firmware (e.g., fixed logic circuits), programmable or non-programmable hardware, or a combination of such implementations. The terms “component” or “function” as used in this application generally refer to software, firmware, hardware, or a combination thereof. For example, in the case of a software implementation, the terms “component” or “function” may refer to a processing device or program code that performs a specified task when executed on the device. The illustrated separation of components and functions into distinct units may reflect any actual or conceptual physical grouping and assignment of such software and / or hardware and tasks.

[0310] supplement

[0311] The following describes examples of non-limiting implementations that embody the principles described elsewhere in this application. These examples may be used to provide context for the described features.

[0312] Syntax and Semantics

[0313] The syntax table specifies the superset of all allowed bitstream syntax. Additional constraints on syntax can be specified directly or indirectly as needed.

[0314] Note: The actual decoder must implement some means for identifying entry points into the bitstream and some means for identifying and processing non-compliant bitstreams. Methods for identifying and processing errors and other

[0315] Process Payload - Picture Configuration

[0316]

[0317]

[0318]

[0319] Data Block Unit Picture Composition Semantics

[0320] no_enhancement_bit_flag specifies that there is no enhancement data for all layerIdx < nLayers of the picture.

[0321] quant_matrix_mode Specifies the quantization matrix to be used in the decoding process according to the table below. When quant_matrix_mode does not exist, it is inferred to be 0.

[0322]

[0323] dequant_offset_signalled_flag Specifies whether the offset method and the offset parameter value to be applied during dequantization are signaled. If equal to 1, the values ​​of the dequantization offset method and the dequantization offset parameter are signaled. If dequant_offset_signalled_flag does not exist, it is inferred to be equal to 0.

[0324] picture_type_bit_flag Specifies whether the encoded data is sent frame-based (e.g., progressive mode or interlaced mode) or field-based (e.g., interlaced mode) according to the table below. Error! Reference source not found.

[0325] Picture type

[0326]

[0327] field_type_bit_flag If picture_type is equal to 1, it specifies whether the sent data is in the top or bottom field according to the table below. Error! Reference source not found.

[0328] Field Type

[0329]

[0330] temporal_refresh_bit_flagSpecifies whether the time buffer should be refreshed for the picture. If it is 1, the time buffer must be refreshed. For IDR pictures, temporal_refresh_bit_flag must be set to 1.

[0331] temporal_signalling_present_flag Specifies whether a time signaling factor group exists in the bitstream. If temporal_signaling_present_flag does not exist, it is inferred to be 1 if temporal_enabled_flag is 1 and temporal_refresh_bit_flag is 0, and 0 otherwise.

[0332] step_width_level2 Specifies the step width value to be used when decoding the residuals encoded in the enhancement sub-level 2 for the lumina plane. The step width value used when decoding the residuals encoded in the enhancement sub-level 2 for the chroma plane is calculated as Clip3(1, 32,767, ((step_width_level2 * chroma_step_width_multiplier)>>6)).

[0333] step_width_level1_enabled_flag Specifies whether the step width value used when decoding the residuals encoded in the enhancement sub-level 1 is the default value or signaled. This must be either 0 (default value) or 1 (value signaled by step_width_level1). The default value is 32,767. When step_width_level1_enabled_flag does not exist, it is inferred to be 0.

[0334] dithering_control_flag Specifies whether dithering should be applied. This must be either 0 (dithering disabled) or 1 (dithering enabled). If dithering_control_flag does not exist, it is inferred to be 0.

[0335] step_width_level1 specifies the step width value to be used when decoding the residual encoded in the improvement sublayer 1.

[0336] level1_filtering_enabled_flag Specifies whether to use the level1 deblocking filter. This must be either 0 (filtering disabled) or 1 (filtering enabled). If level1_filtering_enabled_flag does not exist, it is inferred to be 0.

[0337] qm_coefficient_0[layerIdx] Specifies the value of the quantization matrix scaling parameter when quant_matrix_mode is equal to 2, 3, or 5.

[0338] qm_coefficient_1[layerIdx] Specifies the value of the quantization matrix scaling parameter when quant_matrix_mode is equal to 4 or 5.

[0339] dequant_offset_mode_flag Specifies how to apply the dequantization offset. If equal to 0, the default method is applied using the signaled dequant_offset as a parameter. If equal to 1, the constant offset method is applied using the signaled dequant_offset parameter.

[0340] dequant_offset Specifies the value of the dequantization offset parameter to be applied. The value of the dequantization offset parameter must be between 0 and 127 (including boundary values).

[0341] dithering_type Specifies the dithering type applied to the final reconstructed picture according to the table below. Error! Reference source not found.

[0342] Dithering

[0343]

[0344] dithering_strength specifies a value between 0 and 31.

[0345] Decoding process

[0346] The decoding process is specified to produce numerically identical truncated decoded output pictures when any decoder compliant with a specified profile and level calls the decoding process associated with that profile for a bitstream compliant with that profile and level. Any decoding process that produces the same truncated decoded output picture as produced by the process described in this application (with the exact output order or output timing as specified) complies with the decoding process requirements.

[0347] General decoding process for L-2 encoded data blocks

[0348] The input for this process is as follows:

[0349] Sample position (xTb0, yTb0) specifying the top-left sample of the current transform block relative to the top-left sample of the current picture, variable nTbS specifying the size of the current transform block derived from the value of variable transform_type in sub-clause error! Reference source not found (nTbS = 2 if transform_type is 0, nTbS = 4 if transform_type is 1),

[0350] Variables derived elsewhere temporal_enabled_flag and temporal_refresh_bit_flag, variables derived elsewhere temporal_signaling_present_flag and temporal_step_width_modifier specified elsewhere

[0351] recL2ModifiedUpsampledSamples, an array of size (nTbS)x(nTbS) specifying upsampled reconstructed samples resulting from a process specified elsewhere in the current block

[0352] TransformCoeffQ, an array of size (nTbS)x(nTbS) specifying L-2 entropy-decoded quantized transform coefficients

[0353] If the variable temporal_signalling_present_flag is equal to 1 and temporal_tile_intra_signalling_enabled_flag is equal to 1, the variable TransformTempSig corresponding to the value of TempSigSurface at position (xTb0 >> nTbs, yTb0 >> nTbs); and additionally, if temporal_tile_intra_signalling_enabled_flag is set to 1, the variable TileTempSig corresponding to the value of TempSigSurface at position ((xTb0%32) * 32, (yTb0%32) * 32),

[0354] The stepWidth value derived elsewhere from the value of the variable step_width_level2,

[0355] Variable IdxPlanes that specifies the plane to which the transformation coefficients belong.

[0356] The output of this process is an array of (nTbS)x(nTbS) L-2 residuals resL2Residuals with elements resL2Residuals[x][y].

[0357] The sample position (xTbP, yTbP) specifying the top-left sample of the current transform block for the top-left sample of the current picture is derived as follows.

[0358] (xTbP, yTbP) = (IdxPlanes == 0) ? (xTb0, yTb0) : (xTb0 >> ShiftWidthC, yTb0 >> ShiftHeightC)

[0359] P may be associated with either the lumens or chroma plane depending on the plane to which the transformation coefficient belongs. ShiftWidthC and ShiftHeightC are specified elsewhere.

[0360] If no_enhancement_bit_flag is set to 0, then the following ordered steps are applied:

[0361] If the variable temporal_enabled_flag is equal to 1 and the temporal_refresh_bit_flag is equal to 0, a time prediction process specified elsewhere is called with inputs including the luminance position (xTbY, yTbY), the transform size set to nTbS, the variable TransformTempSig, and the variable TileTempSig, and the output is an array tempPredL2Residuals of size (nTbS)x(nTbS).

[0362] If the variable temporal_enabled_flag is equal to 1 and temporal_refresh_bit_flag is equal to 1, the array tempPredL2Residuals of size (nTbS)x(nTbS) is set to contain only 0s.

[0363] If variable temporal_enabled_flag is equal to 1, temporal_refresh_bit_flag is equal to 0, temporal_tile_intra_signaling_enabled_flag is equal to 1 (sub-clause error! Reference source not found), and TransformTempSig is equal to 0, variable stepWidth is modified to Floor(stepWidth * (1 - (Clip3(0, 0.5, (temporal_step_width_modifier / 255))))).

[0364] The dequantization process specified elsewhere is called with a transform size set equal to nTbS, an array TransformCoeffQ of size (nTbS)x(nTbS), and a variable stepWidth as input, and the output is an array dequantCoeff of size (nTbS)x(nTbS).

[0365] The transformation process specified elsewhere is called with input dequantCoeff, an array of size (nTbS)x(nTbS), a transformation size set to be equal to the luminance location (xTbY, yTbY) and nTbS, and output is an array resL2Residuals of size (nTbS)x(nTbS).

[0366] If the variable temporal_enabled_flag is equal to 1, the tempPredL2Residuals array of size (nTbS)x(nTbS) is added to the (nTbS)x(nTbS) array resL2Residuals, and the resL2Residuals array is stored in temporalBuffer at the luminance position (xTbY, yTbY).

[0367] If no_enhancement_bit_flag is set to 1, the following ordered steps are applied:

[0368] When the variable temporal_enabled_flag is equal to 1, temporal_refresh_bit_flag is equal to 0, and the variable temporal_signaling_present_flag is equal to 1, a time prediction process specified elsewhere is called with inputs including the luminance position (xTbY, yTbY), the transform size set to nTbS, the variable TransformTempSig, and the variable TileTempSig, and the output is an array tempPredL2Residuals of size (nTbS)x(nTbS).

[0369] When the variable temporal_enabled_flag is equal to 1, temporal_refresh_bit_flag is equal to 0, and the variable temporal_signaling_present_flag is equal to 0, a time prediction process specified elsewhere is called with inputs including the luminance position (xTbY, yTbY), the transform size set to nTbS, the variable TransformTempSig set to 0, and the variable TileTempSig set to 0, and the output is an array tempPredL2Residuals of size (nTbS)x(nTbS).

[0370] If the variable temporal_enabled_flag is equal to 1 and temporal_refresh_bit_flag is equal to 1, the array tempPredL2Residuals of size (nTbS)x(nTbS) is set to contain only 0s.

[0371] If the variable temporal_enabled_flag is equal to 1, the tempPredL2Residuals array of size (nTbS)x(nTbS) is stored in the (nTbS)x(nTbS) array resL2Residuals, and the resL2Residuals array is stored in temporalBuffer at the luma position (xTbY, yTbY).

[0372] Otherwise, the array resL2Residuals of size (nTbS)x(nTbS) is set to contain only 0.

[0373] Each plane picture reconstruction process specified in Subclause Error! Reference source not found is called with the transform block location (xTb0, yTb0), transform block size nTbS, variable IdxPlanes, (nTbS)x(nTbS) array resL2Residuals and (xTbY)x(yTbY) recL2ModifiedUpsampledSamples as input.

[0374] Decoding process for dequantization

[0375] All groups of transformation coefficients passed to this process belong to specific planes and enhancement sublayers. They were scaled using a uniform quantizer with a dead zone. The quantizer may use a non-centralized dequantization offset.

[0376] Scaling process for transformation coefficients

[0377] The input for this process is as follows:

[0378] A variable nTbS specifying the size of the current transformation block (nTbS = 2 if transform_type is 0 and nTbS = 4 if transform_type is 1),

[0379] An array of size (nTbS)x(nTbS) TransformCoeffQ containing entropy-decoded quantized transform coefficients

[0380] variable stepWidth, which specifies the step width value parameter

[0381] Variable levelIdx specifying the index of the improvement sub-level (levelIdx = 1 for improvement sub-level 1 and levelIdx = 2 for improvement sub-level 2),

[0382] Variables dQuantOffset and dequant_offset specifying the dequantization offset,

[0383] If quant_matrix_mode is different from 0, an array QmCoeff0 of size 1 x nTbS2 (identical to array variable qm_coefficient_0), and additionally, if quant_matrix_mode is equal to 4, an array QmCoeff1 of size 1 x nTbS2 (identical to array variable qm_coefficient_1),

[0384] If nTbS == 2, an array of size (3 * nTbS)x(nTbS) QuantScalerDDBuffer containing the array of scaling parameters used in the previous picture;

[0385] If nTbS == 4, QuantScalerDDSBuffer is an array of size (3 * nTbS)x(nTbS) containing the array of scaling parameters used in the previous picture.

[0386] The output of this process is an array d of (nTbS)x(nTbS) dequantized transformation coefficients with elements d[x][y] and an updated array QuantMatrixBuffer.

[0387] To derive the scaled transformation coefficient d[x][y] for a given matrix qm[x][y] as specified in subsection 8.6.2, x = 0...nTbS - 1, y = 0...nTbS - 1, and the following formula is used:

[0388] d[x][y] = (TransformCoeffQ[x][y] * ((qm[x + (levelIdxSwap * nTbS)][y] + stepWidthModifier[x][y]) + appliedOffset [x][y]) (1)

[0389] Deduction of Deprotonation Offset and Step Width Modifier

[0390] The variables appliedOffset[x][y] and stepWidthModifier[x][y] are derived as follows:

[0391] if (dequant_offset_signalled_flag == 0) {

[0392] stepWidthModifier [x][y] = ((((Floor(-Cconst * Ln (qm[x + (levelIdxSwap * nTbS)][y]))) + Dconst) *

[0393] (qm[x + (levelIdxSwap * nTbS)][y]2))) / 32768) >> 16

[0394] if (TransformCoeffQ[x][y] < 0)

[0395] appliedOffset [x][y] = (-1 * (-deadZoneWidthOffset [x][y]))

[0396] else if (TransformCoeffQ [x][y] > 0)

[0397] appliedOffset [x][y] = -deadZoneWidthOffset [x][y]

[0398] else

[0399] appliedOffset [x][y] = 0

[0400] } else if (dequant_offset_signalled_flag == 1) && (dequant_offset_mode_flag ==1) {

[0401] stepWidthModifier [x][y] = 0

[0402] if (TransformCoeffQ[x][y] < 0)

[0403] appliedOffset = (-1 * (dQuantOffsetActual [x][y] - deadZoneWidthOffset [x][y]))

[0404] else if (TransformCoeffQ [x][y] > 0)

[0405] appliedOffset [x][y] = dQuantOffsetActual [x][y] - deadZoneWidthOffset [x][y]

[0406] else

[0407] appliedOffset [x][y] = 0}

[0408] } else if (dequant_offset_signalled_flag == 1) && (dequant_offset_mode_flag == 0) {

[0409] stepWidthModifier [x][y] = (Floor((dQuantOffsetActual [x][y]) * (qm[x + (levelIdxSwap * nTbS)][y]))

[0410] / 32768)

[0411] if (TransformCoeffQ[x][y] < 0)

[0412] appliedOffset = (-1 * (-deadZoneWidthOffset[x][y]))

[0413] else if (TransformCoeffQ[x][y] > 0)

[0414] appliedOffset [x][y] = -deadZoneWidthOffset [x][y]

[0415] else

[0416] appliedOffset[x][y] = 0

[0417] }

[0418] Here, if stepWidth > 16 the deadZoneWidthOffset is derived as follows:

[0419] deadZoneWidthOffset [x][y] = (((1 << 16) - ((Aconst * (qm[x + (levelIdxSwap * nTbs)][y] + stepWidthModifier [x][y])) + Bconst) >> 1) * (qm[x + (levelIdxSwap * nTbs)][y] + stepWidthModifier [x][y])) >> 16

[0420] Here, for stepWidth <= 16, deadZoneWidthOffset is derived as follows:

[0421] deadZoneWidthOffset [x][y] = stepWidth >> 1

[0422] Here:

[0423] Aconst = 39

[0424] Bconst = 126484

[0425] Cconst = 5242

[0426] Dconst = 99614

[0427] Here, dQuantOffsetActual[x][y] is calculated as follows:

[0428] if (dequant_offset == 0)

[0429] dQuantOffsetActual [x][y] = dQuantOffset

[0430] else {

[0431] if (dequant_offset_mode_flag == 1)

[0432] dQuantOffsetActual [x][y] = ((Floor(-Cconst * Ln(qm[x + (levelIdxSwap * nTbs)][y]) +

[0433] (dQuantOffset << 9) + Floor(Cconst * Ln(StepWidth)))) * (qm[x + (levelIdxSwap * nTbs)][y])) >> 16

[0434] else if (dequant_offset_mode_flag == 0)

[0435] dQuantOffsetActual [x][y] = ((Floor(-Cconst * Ln(qm[x + (levelIdxSwap * nTbs)][y]) +

[0436] (dQuantOffset << 11) + Floor(Cconst * Ln(StepWidth)))) * (qm[x + (levelIdxSwap * nTbs)][y]))

[0437] >>16

[0438] }

[0439] Here, levelIdxSwap is derived as follows:

[0440] if (levelIdx == 2)

[0441] levelIdxSwap = 0

[0442] else

[0443] levelIdxSwap = 1

[0444] Derivation of the quantization matrix

[0445] The quantization matrix qm[x][y] contains the actual quantization step width used to decode each group of coefficients.

[0446] if (levelIdx == 2) {

[0447] if (scaling_mode_level2 == 1) {

[0448] for (x = 0; x < nTbS; x++) {

[0449] for (y = 0; y < nTbs; y++)

[0450] qm [x][y] = qm_p [x][y]

[0451] }

[0452] } else {

[0453] for (x = 0; x < nTbS; x++) {

[0454] for (y = 0; y < nTbS; y++)

[0455] qm [x][y] = qm_p [x + nTbS][y]

[0456] }

[0457] }

[0458] } else {

[0459] for (x = 0; x < nTbS; x++) {

[0460] for (y = 0; y < nTbs; y++)

[0461] qm [x][y] = qm_p [x + (2 * nTbS)][y]

[0462] }

[0463] }

[0464] Here, qm_p[x][y] is calculated as follows:

[0465] if (nTbs == 2) {

[0466] for (x = 0; x < 6; x++) {

[0467] for (y = 0; y < nTbs; y++)

[0468] qm_p[x][y] = (Clip3 (0, (3 << 16),[(QuantScalerDDBuffer [x][y] * stepWidth) + (1 << 16)]) *

[0469] stepWidth) >> 16

[0470] }

[0471] } else {

[0472] for (y = 0; y < 12; y++) {

[0473] for (x = 0; x < nTbs; x++)

[0474] qm_p[x][y] = (Clip3 (0, (3 << 16),[(QuantScalerDDSBuffer [x][y] * stepWidth) + (1 << 16)]) *

[0475] stepWidth) >> 16

[0476] }

[0477] }

[0478] And, here, QuantScalerDDBuffer [x][y] is derived elsewhere and QuantScalerDDSBuffer [x][y] is derived elsewhere.

[0479] Derivation of Scaling Parameters for 2x2 Transformation

[0480] When the variable nTbS is equal to 2, the default scaling parameter is as follows:

[0481] default_scaling_dd[x][y] =

[0482] {

[0483] { 0, 2}

[0484] { 0, 0}

[0485] { 32, 3)

[0486] { 0, 32}

[0487] { 0, 3}

[0488] { 0, 32}

[0489] }

[0490] In the first step, the array QuantScalerDDBuffer[x][y] is initialized as follows:

[0491] If the current picture is an IDR picture, QuantScalerDDBuffer[x][y] is initialized to be equal to default_scaling_dd[x][y]. If the current picture is not an IDR picture, the QuantScalerDDBuffer[x][y] matrix remains unchanged.

[0492] After initialization, the array QuantScalerDDBuffer[x][y] is processed as follows based on the quant_matrix_mode value:

[0493] If quant_matrix_mode is equal to 0 and the current picture is not an IDR picture, QuantScalerDDBuffer[x][y] remains unchanged.

[0494] If quant_matrix_mode is equal to 1, QuantScalerDDBuffer[x][y] is equal to default_scaling_dd[x][y].

[0495] If quant_matrix_mode is equal to 2, QuantScalerDDBuffer[x][y] is modified as follows:

[0496] for (MIdx = 0; MIdx < 3; MIdx++)

[0497] for (x = 0; x < 2; x++)

[0498] for (y = 0; y < 2; y++)

[0499] QuantScalerDDBuffer [x + (MIdx * 2)][y] = QmCoeff0[(x * 2) + y]

[0500] If quant_matrix_mode is equal to 3, QuantScalerDDBuffer[x][y] is modified as follows:

[0501] for (MIdx = 0; MIdx < 2; MIdx++)

[0502] for (x = 0; x < 2; x++)

[0503] for (y = 0; y < 2; y++)

[0504] QuantScalerDDBuffer [x + (MIdx * 2)][y] = QmCoeff0 [(x * 2) + y]

[0505] If quant_matrix_mode is equal to 4, QuantScalerDDBuffer[x][y] is modified as follows:

[0506] for (x = 0; x < 2; x++)

[0507] for (y = 0; y < 2; y++)

[0508] QuantScalerDDBuffer [x + 4][y] = QmCoeff1 [(x * 2) + y]

[0509] If quant_matrix_mode is equal to 5, QuantScalerDDBuffer is modified as follows:

[0510] for (MIdx = 0; MIdx < 2; MIdx++)

[0511] for (x = 0; x < 2; x++)

[0512] for (y = 0; y < 2; y++)

[0513] QuantScalerDDBuffer [x + (MIdx * 2)][y] = QmCoeff0[(x * 2) + y]

[0514] for (x = 4, x< 6; x++)

[0515] for (y = 0; y < 2; y++)

[0516] QuantScalerDDBuffer [x][y] = QmCoeff1[(x * 2) + y]

[0517] Derivation of Scaling Parameters for 4x4 Transformation

[0518] When the variable nTbS is equal to 4, the default scaling parameter is as follows:

[0519] default_scaling_dds[x][y] =

[0520] {

[0521] { 13, 26, 19, 32}

[0522] { 52, 1, 78, 9}

[0523] { 13, 26, 19, 32}

[0524] { 150, 91, 91, 19}

[0525] { 13, 26, 19, 32}

[0526] { 52, 1, 78, 9}

[0527] { 26, 72, 0, 3}

[0528] { 150, 91, 91, 19}

[0529] { 0, 0, 0, 2}

[0530] { 52, 1, 78, 9}

[0531] { 26, 72, 0, 3}

[0532] { 150, 91, 91, 19}

[0533] }

[0534] In the first step, the array QuantScalerDDSBuffer[][] is initialized as follows:

[0535] If the current picture is an IDR picture, QuantScalerDDSBuffer[x][y] is initialized to be equal to default_scaling_dds[x][y]. If the current picture is not an IDR picture, the QuantScalerDDSBuffer[x][y] matrix remains unchanged.

[0536] After initialization, the array QuantScalerDDSBuffer[x][y] is processed as follows based on the quant_matrix_mode value:

[0537] If quant_matrix_mode is equal to 0 and the current picture is not an IDR picture, QuantScalerDDSBuffer remains unchanged.

[0538] If quant_matrix_mode is equal to 1, QuantScalerDDSBuffer is equal to default_scaling_dds[x][y].

[0539] If quant_matrix_mode is equal to 2, QuantScalerDDSBuffer is modified as follows:

[0540] for (MIdx = 0; MIdx < 3; MIdx++)

[0541] for (x = 0; x < 4; x++)

[0542] for (y = 0; y < 4; y++)

[0543] QuantScalerDDSBuffer [x + (MIdx * 4)][y] = QmCoeff0[(x * 4) + y]

[0544] If quant_matrix_mode is equal to 3, QuantScalerDDSBuffer is modified as follows:

[0545] for (MIdx = 0; MIdx < 2; MIdx++)

[0546] for (x = 0; x < 4; x++)

[0547] for (y = 0; y < 4; y++)

[0548] QuantScalerDDSBuffer [x + (MIdx * 4)][y] = QmCoeff0[(x * 4) + y]

[0549] If quant_matrix_mode is equal to 4, QuantScalerDDSBuffer is modified as follows:

[0550] for (x = 0; x < 4; x++)

[0551] for (y = 0; y < 4; y++)

[0552] QuantScalerDDSBuffer [x + 8][y] = QmCoeff1[(x * 4) + y]

[0553] If quant_matrix_mode is equal to 5, QuantScalerDDSBuffer is modified as follows:

[0554] for (MIdx = 0; MIdx < 2; MIdx++)

[0555] for (x = 0; x < 4; x++)

[0556] for (y = 0; y < 4; y++)

[0557] QuantScalerDDSBuffer [x + (MIdx * 4)][y] = QmCoeff0[(x * 4) + y]

[0558] for (x = 8, x < 12; x++)

[0559] for (y = 0; y < 4; y++)

[0560] QuantScalerDDSBuffer [x][y] = qm_coefficient_1[(x * 4) + y]

Claims

Claim 1 A method for decoding an encoded signal comprising a sample sequence, wherein the encoded signal is received in a bitstream, and the method comprises: receiving a quantized data set associated with the encoded signal; determining at least one dequantization parameter; and dequantizing a sample of the quantized data at least partially based on the at least one dequantization parameter, wherein the dequantization operation comprises using at least one predetermined dequantization parameter to determine the at least one dequantization parameter, and wherein the at least one predetermined dequantization parameter corresponds to a dequantization parameter used to decode a previous sample. Claim 2 The method of claim 1, wherein the determining step further comprises: deriving a representation of the dequantization operation to be applied to dequantize the quantized data from a bitstream containing the encoded signal; and determining the at least one dequantization parameter based at least partially on the representation. Claim 3 A method according to claim 1, wherein the at least one predetermined deprotonation parameter corresponds to a default deprotonation parameter. Claim 4 A method according to claim 1, wherein the dequantization operation comprises the step of updating at least one predetermined dequantization parameter using at least one updated parameter received in the bitstream. Claim 5 A method according to claim 1, wherein the at least one dequantization parameter includes a quantization step width parameter, and the quantization step width parameter is received in a bitstream. Claim 6 The method of claim 5, wherein the at least one deprotonation parameter further comprises at least one scaling factor. Claim 7 A method according to claim 6, wherein the step of determining the at least one dequantization parameter comprises the step of processing the quantization step with the at least one scaling factor. Claim 8 A method according to claim 1, wherein the set of quantized coefficients comprises a plurality of subsets of quantized data, the at least one dequantization parameter comprises a plurality of dequantization parameters, and each of the plurality of dequantization parameters corresponds to each subset of the plurality of subsets of quantized data. Claim 9 A method according to claim 8, wherein the dequantization operation further comprises the step of dequantizing each of a plurality of subsets of quantized data based at least partially on each of the plurality of dequantization parameters. Claim 10 A method according to claim 8, wherein the plurality of dequantization parameters includes at least one quantization step width parameter, and the quantization step width parameter is received in a bitstream and includes at least one scaling factor. Claim 11 A method according to claim 10, wherein the at least one scaling factor comprises a plurality of scaling factors, and each of the plurality of scaling factors corresponds to each subset of the plurality of subsets of the quantized data. Claim 12 A method according to claim 11, wherein the step of determining at least one dequantization parameter includes the step of deducing a plurality of dequantization parameters, and each of the plurality of dequantization parameters is obtained by processing the quantization step width parameter as each of the plurality of scaling factors. Claim 13 In claim 1, the actual quantization parameter is a scaling factor The received quantization step width parameter above by the function f(ㆍ) of these two arguments according to the following A method calculated by processing. Claim 14 In claim 13, the function f(ㆍ) is A method that is the multiplication according to Claim 15 In claim 14, the deprotonation operation is It includes dequantized data calculated as, where is a quantized data subset, and is a method that is the actual quantization step for the i-th quantized data subset. Claim 16 In claim 1, the actual quantization parameter is the step width modifier ( It is calculated based additionally on ), and the dequantization offset is A method used for dequantization operations according to Claim 17 In claim 16, f(ㆍ) is It is a function according to, where the above dequantization operation includes dequantized data calculated as follows, and Here, is a quantized data subset, and is the actual quantization step for the i-th quantized data subset, and DO i A method in which is the dequantization offset for a specific i-th quantized data subset. Claim 18 A method according to claim 1, wherein the set of quantized data comprises a group of transformed coefficients scaled using a linear quantizer, the linear quantizer uses a non-centered dequantization offset, and the value of the dequantization offset is received from an encoder. Claim 19 A method according to claim 18, wherein the received offset value is added to the received quantized value prior to dequantization based on the step width. Claim 20 A method according to claim 1, further comprising the step of receiving a base-decoded signal coded by a first codec, wherein the dequantized data represents a residual set, and the method further comprises the step of combining the dequantized data with the base-decoded signal to generate a reconstructed signal. Claim 21 The method of claim 20 further comprises: receiving an additional set of quantized data associated with an additional encoded signal—the additional encoded signal is received within the bitstream—; receiving an additional set of quantized data associated with the additional encoded signal; determining at least one additional dequantization parameter; dequantizing the additional quantized data at least partially based on the at least one dequantization parameter; upsampling the reconstructed signal to generate an upsampled reconstructed signal; and combining the additional dequantized data with the upsampled reconstructed signal to generate an additional reconstructed signal. Claim 22 A decoder for decoding an encoded stream into a reconstructed output video, wherein the decoder is configured to perform the method of claim 1. Claim 23 A non-transient computer-readable storage medium storing instructions that cause a processor to perform the method of claim 1 when executed by a processor. Claim 24 Computer program code for recording a signal in the form of a bitstream, wherein the computer program code is executed by a processor and stored in a non-transient computer-readable storage medium, the signal comprises a sample sequence and at least a first part and a second part, wherein the first part generates a decoded signal when decoded by a decoding module; the second part provides information necessary for decoding the first part; and the second part comprises a first sub-part and a second sub-part, wherein the first sub-part indicates a dequantization parameter to be used to dequantize one or more data within the first part, and the second sub-part indicates a process to be used to dequantize one or more data within the first part, and the dequantization operation of the dequantization process for a sample comprises using at least one predetermined dequantization parameter to determine at least one dequantization parameter for the sample, and the at least one predetermined dequantization parameter corresponds to a dequantization parameter used to decode a previous sample. Claim 25 delete Claim 26 delete Claim 27 delete Claim 28 delete Claim 29 delete Claim 30 delete Claim 31 delete Claim 32 delete Claim 33 delete Claim 34 delete Claim 35 delete Claim 36 delete Claim 37 delete Claim 38 delete Claim 39 delete Claim 40 delete Claim 41 delete Claim 42 delete Claim 43 delete Claim 44 delete Claim 45 delete Claim 46 delete Claim 47 delete Claim 48 delete Claim 49 delete Claim 50 delete Claim 51 delete Claim 52 delete Claim 53 delete Claim 54 delete Claim 55 delete Claim 56 delete Claim 57 delete Claim 58 delete Claim 59 delete Claim 60 delete Claim 61 delete Claim 62 delete Claim 63 delete

Images