Video compression and storage system and frame compression method for the video compression and storage system - Patents.com

Abstract

To provides video compression, storage, and decompression system and method that can ensure a non-negative bounded reconstruction error.SOLUTION: A frame compression method of video compression and a storage system according to an embodiment of the present invention includes applying modulo addition to a residual of an original sample of a frame to generate a deflection residual on the basis of the bit depth and maximum margin of error of the original sample, quantizing the deflection residual on the basis of the maximum tolerance to generate a quantized deflection residual, and encoding a value corresponding to the quantized deflection residual to generate a coded value with a non-negative reconstruction error.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a video compression and storage system and a method for video compression and decompression.

[0002] This application claims priority to U.S. Patent Application Nos. 63 / 063,004 and 63 / 062,963, filed in the United States Patent and Trademark Office on August 7, 2020, the entire contents of which are incorporated herein by reference.

[0003] This application is also related to a U.S. patent application entitled "DPCM CODEC WITH HIGHER RECONSTRUCTION QUALITY ON IMPORTANT GRAY LEVELS," filed on the same date in the United States. [Background technology]

[0004] 2. Description of the Related Art Flat panel displays, such as liquid crystal displays (LCDs) and light emitting diode (LCD) displays, have become ubiquitous due to their small size, excellent image quality, and low power consumption.

[0005] The pixels of a display panel have a certain response time. The data supplied to the pixel to be displayed can change faster than the pixel can react. This can result in undesirable effects such as motion blur and ghosting. To make the displayed image more beautiful, image compensation techniques are required. Overdrive algorithms have been developed to compensate the image data, thereby reducing the motion blur that occurs in display pixels.

[0006] The information described in the background art is intended to deepen understanding of the present invention and may include information that does not constitute prior art already known to those skilled in the art. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] U.S. Patent No. 6,324,216 [Patent Document 2] U.S. Patent No. 8,447,121 [Patent Document 3] U.S. Patent No. 9,319,684 Summary of the Invention [Problem to be solved by the invention]

[0008] The problem that the present invention seeks to solve is to provide a video compression, storage and decompression system and method that can ensure a non-negative bounded reconstruction error. [Means for solving the problem]

[0009] A method for compressing a frame in a video compression and storage system according to an embodiment of the present invention includes applying modulo addition to a residue of original samples of a frame to generate a biased residue based on a bit depth of the original samples and a maximum allowable error, quantizing the biased residue based on the maximum allowable error to generate a quantized biased residue, and encoding a value corresponding to the quantized biased residual to generate an encoded value having a non-negative reconstruction error.

[0010] According to one embodiment of the invention, generating the biased residual may include calculating the biased residual based on the following formula, where the original samples correspond to color values ​​of pixels of the frame:

number

[0011] According to one embodiment of the present invention, the deflection residual is -δ≦e′≦2 bitdepth It may be within the range represented by -1.

[0012] According to one embodiment of the invention, the method may further include receiving the original sample and subtracting a prediction of a previous reconstructed sample from the original sample to generate the residual of the original sample.

[0013] According to an embodiment of the present invention, the value corresponding to the quantized biased residual may be a quantization index corresponding to the quantized biased residual.

[0014] According to one embodiment of the invention, the quantization of the biased residual may be based on a quantization step size equal to the maximum allowed error plus one.

[0015] According to one embodiment of the invention, generating the quantized biased residual may include calculating a quantization index for the quantized biased residual based on the following equation:

number

[0016] According to one embodiment of the invention, generating the quantized biased residual may include using non-uniform quantization bands.

[0017] According to an embodiment of the present invention, the method may further include performing inverse quantization on the quantized biased residual to generate a reconstructed biased residue, adding a prediction of a previous reconstructed sample to the reconstructed biased residual to generate a reconstructed biased sample, applying a modulo subtraction and clipping operation to the reconstructed biased sample based on a bit depth of the original sample and the maximum allowed error to generate a reconstructed sample, and generating a prediction of a next reconstructed sample based on the reconstructed sample.

[0018] According to an embodiment of the present invention, the step of generating the reconstructed sample may include calculating the reconstructed sample based on the following formula:

number

[0019] According to one embodiment of the present invention, performing the inverse quantization may be based on a quantization band size and a quantization index that are the same as the maximum allowed error.

[0020] According to an embodiment of the present invention, the generating of the reconstructed biased residual may include calculating the reconstructed biased residual based on the following equation:

number

[0021] According to one embodiment of the present invention, performing inverse quantization on the quantized biased residual may be based on a non-uniform quantization table.

[0022] According to one embodiment of the invention, generating the coded value may include entropy coding a value corresponding to the quantized biased residual to generate the coded value.

[0023] According to one embodiment of the invention, the method further includes truncating the frame to generate the original representation by reducing a bit depth of color values ​​of pixels of the frame, the frame may be in Red-Green-Blue (RGB) format or Red-Green-Blue-Green (RGBG) format.

[0024] In accordance with one embodiment of the present invention, a decompression method is provided for decompressing stored video data corresponding to a frame in a video compression and storage system, the method including: decoding encoded values ​​corresponding to original samples of the frame to generate decoded values ​​corresponding to quantized biased residues; performing inverse quantization on the decoded values ​​to generate reconstructed biased residues; adding a prediction of a previous reconstructed sample to the reconstructed biased residual to generate reconstructed biased samples; and applying modulo subtraction to the reconstructed biased samples based on a bit depth of the original samples and a maximum allowed error to generate reconstructed samples.

[0025] According to one embodiment of the invention, the method may further comprise generating the prediction of a next reconstructed sample based on the reconstructed sample.

[0026] According to one embodiment of the invention, generating the reconstructed representation may include calculating the reconstructed representation based on the following formula:

number

[0027] According to one embodiment of the invention, the decoded values ​​are quantization indices, and generating the reconstructed biased residual may include calculating the reconstructed biased residual based on the following formula:

number

[0028] According to an embodiment of the present invention, a video compression and storage system includes a processor and a processor memory local to the processor, the processor memory storing instructions, the processor executing the instructions to apply modulo addition to a residue of original samples of a frame to generate a biased residue based on a bit depth of the original samples and a maximum allowed error, quantize the biased residue based on the maximum allowed error to generate a quantized biased residue, encode a value corresponding to the quantized biased residue to generate an encoded value for storing in a storage medium, retrieve the encoded value from the storage medium, decode the encoded value to generate a decoded value corresponding to the quantized biased residue, perform inverse quantization on the decoded value to generate a reconstructed biased residue, and apply the inverse quantization to the reconstructed biased residue. generating a reconstructed biased sample by adding a prediction of the original sample to the reconstructed biased sample based on a bit depth of the original sample and the maximum allowed error, and generating a reconstructed sample by applying a modulo subtraction to the reconstructed biased sample based on a bit depth of the original sample and the maximum allowed error, wherein a difference between the original sample and the reconstructed sample is non-negative. Effect of the Invention

[0029] This ensures a non-negative bounded reconstruction error in the video compression, storage and decompression system and method. [Brief description of the drawings]

[0030] [Figure 1] 1 is a schematic diagram of a frame compensation system using a video compression and storage system according to an embodiment of the present invention; [Figure 2A]2 is a schematic diagram of an encoder of a video compression and storage system according to one embodiment of the present invention; [Figure 2B] 2 is a schematic diagram of a decoder of a video compression and storage system according to one embodiment of the present invention; [Figure 3A] FIG. 2 is a schematic diagram of part of an encoder together with part of a decoder according to an embodiment of the invention; [Figure 3B] FIG. 3B is a simplified diagram of FIG. 3A illustrating the identity operation of the entropy encoder and decoder according to one embodiment of the present invention. [Figure 4A] 4 illustrates a quantization table used by a quantizer according to an embodiment of the present invention. [Figure 4B] 4 illustrates the operation of a quantizer according to one embodiment of the present invention. [Figure 4C] 4 illustrates uniform and non-uniform quantization tables used by a quantizer according to one embodiment of the present invention. [Figure 4D] 4 illustrates uniform and non-uniform quantization tables used by a quantizer according to one embodiment of the present invention. [Figure 5A] FIG. 2 is a schematic block diagram of an encoder with in-loop mapping according to an embodiment of the present invention; [Figure 5B] FIG. 2 is a schematic block diagram of an encoder with offline / outer-loop mapping according to an embodiment of the present invention; [Figure 6A] 3A-3C are schematic block diagrams of a decoder with inner ring mapping and a decoder with offline / outer ring mapping, respectively, according to an embodiment of the present invention; [Figure 6B] 3A-3C are schematic block diagrams of a decoder with inner ring mapping and a decoder with offline / outer ring mapping, respectively, according to an embodiment of the present invention; [Figure 7A] 11 is a table showing numerical examples of potential reconstruction errors for different sample values ​​given different values ​​of maximum error δ and a constant c according to an embodiment of the present invention. [Figure 7B]11 is a table showing numerical examples of potential reconstruction errors for different sample values ​​given different values ​​of maximum error δ and a constant c according to an embodiment of the present invention. [Figure 8] 11 is a table showing example values ​​of potential reconstruction error for different sample values ​​considering the importance of non-zero sample values ​​according to an embodiment of the present invention; [Figure 9A] 11 is a table illustrating example values ​​of potential reconstruction errors for different sample values ​​based on mapping and inverse mapping functions of a video compression and storage system that does not guarantee a non-negative reconstruction error, according to an embodiment of the present invention. [Figure 9B] 11 is a table illustrating example values ​​of potential reconstruction errors for different sample values ​​based on mapping and inverse mapping functions of a video compression and storage system that does not guarantee a non-negative reconstruction error, according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0031] The detailed description below with reference to the accompanying drawings relates to embodiments of a data compression system and method, and does not represent all aspects realized or utilized by the present invention. Hereinafter, the embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the same or equivalent functions and structures realized in different embodiments are also included within the scope of the present invention. The same reference numerals are used throughout the specification to refer to the same or similar components.

[0032] FIG. 1 is a schematic diagram of a frame compensation system 1 that uses a video compression and storage system 10 according to one embodiment of the present invention.

[0033] According to an embodiment of the present invention, the video compression and storage system 10 includes an encoder 100, a storage medium 200, and a decoder 300. The encoder 100 encodes (e.g., compresses) data to reduce its size, the storage medium 200 stores the encoded data, and the decoder 300 decodes (e.g., decompresses) the stored data to reconstruct the data. Storing some data, such as a 4K video frame, consumes a lot of memory space. Here, the encoder 100 and the decoder 300 allow the use of a smaller storage medium 200, which can save space and cost. As shown in FIG. 1, according to an embodiment of the present invention, the video compression and storage system 10 can be part of a frame compensation system 1, in which an overdrive system 20 uses a previous frame to compensate a current frame (e.g., to reduce ghosting effects or blurring). According to one embodiment of the present invention, the truncation and delay circuit 30 receives a frame (e.g., 8 bits (bpc) per hue), truncates the frame (i.e., reduces the number of bpcs of the pixel values ​​of the frame to, for example, 3 bpc), and delays the frame to generate a truncated previous frame. The video compression and storage system 10 then compresses the previous frame and stores it for later use by the overdrive system 20. Here, the truncation and compression can result in a stored frame that is much smaller in size than the original (previous) frame. For example, for an RGB input frame in which each color value is represented with 8 bits (or 24 bits (bpp) per pixel), the truncation can result in a 3 bpc / 9 bpp frame representation, which can be reduced again to 2 bpc / 6 bpp when the encoder 100 performs a 1.5:1 compression. However, embodiments of the present invention are not limited to RGB input frames, and any suitable input frame format can be used, such as RGBG (for a pentile sub-pixel arrangement).For example, for an RGBG input frame, truncation can reduce the frame size from 16 bpp to 6 bpp, and the encoder 100 can perform 1.5:1 compression to reduce the size again to 4 bpp. Also, embodiments of the present invention are not limited to using the truncation operation, and in accordance with one embodiment of the present invention, the truncation operation can be omitted by replacing the truncation and delay circuit 30 in the process described above with a delay block.

[0034] The performance of the overdrive system 20 can be improved if the reconstruction error, i.e., the error introduced by the video compression and storage system 10 (both of which are quantization errors), is simply non-negative. Conventional encryption methods typically optimize the mean square error (MSE) or peak signal to noise ratio (PSNR), which means that the polarity of the error in the reconstructed image can be either positive or negative. According to an embodiment of the present invention, the video compression and storage system 10 improves the performance of the overdrive system 20 by ensuring that the reconstruction error (i.e., the quantization error), defined as the difference between the reconstructed frame and the original frame, is always non-negative (i.e., greater than or equal to zero) and limited to a set maximum value.

[0035] Throughout this specification, a frame refers to a video frame of data that includes a number of pixels, each pixel of which is represented by a number of color values ​​(eg, red, green and blue).

[0036] Figure 2A is a schematic diagram of an encoder 100 of the video compression and storage system 10 according to one embodiment of the present invention. Figure 2B is a schematic diagram of a decoder 300 of the video compression and storage system 10 according to one embodiment of the present invention.

[0037] Referring to FIG. 2A, according to an embodiment of the present invention, the encoder 100 uses a predictive encryption scheme that encodes the residual of an original sample (e.g., a truncated frame) rather than the original sample itself. The residual (also called error e) is defined as the difference between the original sample and a prediction based on one or more adjacent samples (e.g., a prediction of a previous sample). According to an embodiment of the present invention, the sample predictor 110 generates a prediction of a previous sample, and the subtractor 120 calculates a residual value based on the original sample and the predicted sample. The calculated residual may be a positive or negative value. The modulo adder 130 adds a bias value to the residual to reduce the cost of signaling 1 bit of code information. The quantizer 140 quantizes the biased residual value to generate a quantized biased value and transmits the corresponding quantization index to the entropy encoder 150 for further compression (e.g., lossless compression such as entropy coding). This compressed data (or encoded value) is stored on storage medium 200. As will be explained further below, according to one embodiment of the present invention, the bias value is a function of the quantization step size.

[0038] According to one embodiment of the present invention, to generate the predicted samples, inverse quantizer 160 performs an inverse operation of the operation performed by quantizer 140 on the quantized biased residual to generate a reconstructed biased residue, adder 170 adds a prediction of a previous reconstructed sample to the reconstructed biased residual to generate a reconstructed biased sample, and modulo subtractor 180 subtracts the value added by modulo adder 130 to generate a reconstructed sample. According to one embodiment of the present invention, a clipping function is applied to the reconstructed samples to obtain a clipping function within the range [0,2 bitDepth The reconstructed sample may be within a desired dynamic range, expressed as [1 / 2, 0 / 1 -1]. The sample predictor 110 uses this reconstructed sample (eg, the clipped reconstructed sample) to predict the next sample.

[0039] 2B, according to one embodiment of the present invention, the decoder 300 includes an entropy decoder 310 for decompressing stored samples received from the storage medium 200, the same inverse quantizer 160, the adder 170, the modulo subtractor 180, and the sample predictor 110 as previously described. Apart from the entropy decoder 310, the remaining parts of the decoder 300 are identical to the encoder 100 and are constructed in the same manner, and therefore will not be described in detail for the sake of brevity.

[0040] Because the decoder 300 does not have access to the original samples, but only to the reconstructed samples, the sample predictor 110 of the decoder 300 uses the reconstructed samples to generate predictions of previous samples. Also, because the operation of the encoder 100 and the decoder 300 is mirrored, the sample predictor 110 of the encoder 100 also uses the same reconstructed samples to generate predictions of previous samples, thereby allowing the encoder 100 to be in-sync with the decoder 300 even though it does not have access to the original samples.

[0041] According to one embodiment of the present invention, the encoder 100 and the decoder 300 can operate on a single color value of a pixel. Thus, multiple encoders 100 and decoders 300 can operate in parallel (e.g., simultaneously) to process different hues of a pixel.

[0042] Figure 3A is a schematic diagram of part of an encoder together with part of a decoder according to an embodiment of the present invention, and Figure 3B is a simplified diagram of Figure 3A illustrating the identity operation of the entropy encoder and decoder according to an embodiment of the present invention.

[0043] According to one embodiment of the present invention, all operations performed by the encoder 100 and decoder 300 architecture are lossless, except for the quantizer 140. For example, the sequential operations of the entropy encoder 150 and entropy decoder 310 shown in Fig. 3A (block A) are identity operations and may be omitted as in Fig. 3B. Thus, the error in signal space, defined as the difference between the reconstructed samples at the output of the decoder 300 and the original samples at the input of the encoder 100, is equivalent to the error in residual space. Error (residual) = R-e' = Q[e'] - e' (Equation 1)

[0044] where e′ is the biased residual input to quantizer 140, R is the output of inverse quantizer 160, and Q[.] is a function of the combined quantization of quantizer 140 and the inverse quantization of inverse quantizer 160.

[0045] Therefore, if the error in the residual space is kept non-negative, the error in the signal space should also be non-negative. Also, if the error in the residual space is bounded, i.e., R-e'≦δ, where δ is a positive integer representing the maximum allowable error, then the error in the signal space is also bounded by δ. This relationship is maintained even in the presence of modulo adder 130 and modulo subtractor 180. In contrast, conventional transform coding (e.g., high efficiency video coding (HEVC), JPEG-2000, or VESA display compression-M (VDC-M)) is not suitable to guarantee a non-negative reconstruction error, because a positive error in the transformed domain does not guarantee a non-negative reconstruction error in the signal space. This is because transforms such as the discrete cosine transform (DCT) store the L2-norm (or energy) between the original and the transform coefficients, but not the L1-norm.

[0046] Thus, in accordance with one embodiment of the present invention, modulo adder 130 and quantizer 140 are configured identically in signal space, ensuring that the error in residual space remains non-negative and bounded (i.e., limited to δ).

[0047] The quantizer 140 and the inverse quantizer 160 ensure a non-negative bounded error by appropriately adjusting the quantization band size and the reconstruction point in each band. According to an embodiment of the present invention, for a maximum allowable error (δ), the quantizer 140 uses uniform or non-uniform quantization bands with a maximum quantization step size of δ+1 and uses a reconstruction point at the right edge (i.e., the maximum value) of the band. Also, according to an embodiment of the present invention, the value zero is at the right edge of the band (e.g., the first quantization band with index 1), so zero appears in the reconstruction value. This is believed to be due to the residual peaking at zero.

[0048] Figure 4A illustrates quantization tables used by quantizer 140 and inverse quantizer 160 according to one embodiment of the invention. Figure 4B illustrates the operation of quantizer 140 and inverse quantizer 160 according to one embodiment of the invention, where the input is e' and the output is R. Figures 4C and 4D illustrate uniform and non-uniform quantization tables, respectively, used by quantizer 140 according to one embodiment of the invention.

[0049] 4A, the quantizer 140 uses uniform bands of quantization step size δ+1 with the reconstruction point at the right end of each band. According to an embodiment of the present invention, the first quantization band has a right end at 0 (i.e., the reconstruction point is 0), where the biased residual e′ is an integer. According to an embodiment of the present invention, when using modulo addition and subtraction parameters defined in Equation 6 and Equation 8 (also referred to as optimized modulo addition and subtraction parameters) as described below, the number of bands NB can be expressed as follows:

number

[0050] where δ represents the maximum allowable error and Ceiling(.) represents the ceiling function, where δ+1 is the maximum allowable step size, i.e., a smaller step size can be selected. In one embodiment of the present invention, when using un-optimized modulo addition and subtraction parameters (i.e., parameters not defined in Equation 6 and Equation 8), the number of bands NB can be expressed as:

number

[0051] According to one embodiment of the present invention, when quantizing the biased residual, quantizer 140 calculates quantization indices for the biased residual as follows:

number

[0052] Here, when using the optimized modulo addition and subtraction parameters of Equation 6 and Equation 8, −δ≦e′≦2 bitdepth -1. In contrast, when using unjustified modulo addition and subtraction parameters, -δ≦e′≦2 bitdepth -1+δ.

[0053] I[e'] represents the quantization index, Ceiling(.) represents the ceiling function, e' represents the bias residual, and δ represents the maximum allowable error.

[0054] Similarly, applying inverse quantization to the quantized biased residual may include calculating a reconstructed biased residual as follows:

number

[0055] where Q(e') represents the reconstructed biased residual, I[e'] represents the quantization index, and e' represents the biased residual. Equations 2 and 3 apply to an embodiment using uniform quantization. When non-uniform quantization is used, the maximum step size may be δ+1.

[0056] FIG. 4B shows an example where δ=2 and the bit depth (i.e., the number of bits per hue) is 3. Here, bit depth refers to the color information stored in the image and affects the dynamic range of the image. The higher the bit depth of the image, the more colors it can store. The simplest image, a 1-bit image, can only show two colors, black and white. This is because 1 bit can only store one of two values, 0 (white) and 1 (black). The higher the bit depth, the larger the dynamic range, and therefore the number of quantization bands.

[0057] As shown in the example of FIG. 4B, a deflection residual e′ of 1 results in a reconstruction value of 3.

[0058] According to one embodiment of the present invention, modulo adder 130 calculates the biased residual e′ as follows:

number

[0059] This means that the deflection residual is in the range expressed as:

number

[0060] An upper bound on the bias residual e' reduces the error for certain bias values, thereby allowing for non-uniform quantization bands which improves performance.

[0061] For example, for δ=2 and bit depth=3, −2≦e′≦7, and quantizer 140 can use the uniform quantization table shown in FIG. 4C and the non-uniform quantization table shown in FIG. 4D. In FIG. 4C, where the uniform band size is 2, the quantization error is bounded by δ (i.e., 2) for all four bands. However, embodiments of the present invention also allow for non-uniform bands in the quantization table shown in FIG. 4D, where the third and fourth bands have a smaller quantization error bounded by 1 (instead of 2). In one embodiment of the present invention, quantizer 140 reserves narrower bands for statistically more important bands (e.g., bands where more bias residuals are located, or bands of values ​​that have a greater impact on the performance of overdrive system 20). This can improve the performance of overdrive system 20 because the quantization error is reduced by the modulo adder operation.

[0062] Modulo subtractor 180 may perform the inverse operation of modulo adder 130 by removing the bias value that modulo adder 130 added to the residual. According to one embodiment of the present invention, for an input x, the output y of modulo subtractor 180 may be expressed as:

number

[0063] As mentioned above, according to one embodiment of the present invention, a clipping function is applied to the reconstructed samples to produce a clipping function within the range [0,2 bitDepth The subtractor output can be made to lie within a given dynamic range, represented as [0, -1]. In the 8 bpc example, a clipping function can limit the subtractor output to [0, 7].

[0064] Thus, as described above, by using appropriate quantization band sizes and applied bias, the video compression and storage system 10 ensures (i.e., guarantees) a non-negative bounded error for the reconstructed image, which can improve the performance of the overdrive system 20.

[0065] While ensuring a non-negative bounded reconstruction error, according to one embodiment of the present invention, it is preferred that a particular gray level (e.g., a gray value of 0) has an error of 0 or at least less than that of other gray values. For example, for a color depth of 3, the gray values ​​may range from 0 to 7. According to one embodiment of the present invention, the overdrive correction gives increasing importance (or priority) to gray values ​​going down from 7 to 0, with 0 being the most important (or most preferred). In such an example, a larger reconstruction error can be tolerated for gray values ​​with lower priority, such as 5, 6, and 7. This is contrary to conventional codecs, which treat all gray levels equally and where the reconstruction error is independent of the amplitude (or gray level) of the pixel samples.

[0066] According to one embodiment of the present invention, the video compression and storage system 10 ensures smaller reconstruction errors for important gray levels and non-negative reconstruction errors for other gray levels. According to one embodiment of the present invention, the encoder 100 modifies the original gray values ​​to achieve a lower error for a particular gray level by allowing a larger error for other non-important gray levels.

[0067] Figure 5A is a schematic block diagram of an encoder 400 with in-loop mapping according to an embodiment of the present invention. Figure 5B is a schematic block diagram of an encoder 400-1 with offline / outer-loop mapping according to an embodiment of the present invention. Figures 6A and 6B are schematic block diagrams of a decoder 500 with in-loop mapping and a decoder 500-1 with offline / outer-loop mapping, respectively, according to an embodiment of the present invention.

[0068] According to one embodiment of the present invention, the encoder 400 / 400-1 includes a mapper (e.g., gray value mapper circuit) 202 that maps the gray values ​​x of the original samples with a mapped value (also called mapped sample) f(.) prior to encoding, and further includes an inverse mapper (e.g., gray value inverse mapper circuit) 204 that performs the inverse operation g(.) of the mapper 202. Thus, the encoder 400 / 400-1 encodes the original samples in a mapped space, rather than in a signal space, where the dynamic range of the mapped space is the same as the original space. In other words, the bit depths of the mapped space and the original space are the same.

[0069] According to one embodiment of the present invention, for the inner ring scheme shown in Fig. 5A, the encoder 400 includes a first mapper 202 operating on the original samples before the adder 170, a second mapper 203 operating on the sample predictor 110, and an inverse mapper 204 operating on the output of the modulo subtractor 180. In this embodiment, the inverse mapper 204 performs an inverse mapping operation on the mapped reconstructed samples generated by the modulo subtractor 180 to generate unmapped reconstructed samples, which the sample predictor 110 uses to generate a prediction of the previous unmapped reconstructed sample. The second mapper 203 maps this value with the previous mapped reconstructed sample and sends it to the subtractor 120, which subtracts the previous reconstructed sample from the mapped sample.

[0070] According to one embodiment of the present invention, in the case of the offline / outer ring approach shown in FIG. 5B, the inverse mapper 204 does not operate on the input of the sample predictor 110, thereby eliminating the need for a second mapper 203 operating on the output of the sample predictor 110. In such an embodiment, the mapping operation is performed completely outside the encoder loop. Such an outer ring approach is easy to use with existing codecs, since the operations of the mapper 202 and the inverse mapper 204 are performed outside the codec. The outer ring approach is also less complex than the inner ring approach, since it requires fewer operations / calculations to be performed. However, higher performance can be obtained using the inner ring approach.

[0071] As shown in FIGS. 6A and 6B, inner ring mapping decoder 500 uses both mapper 203 and inverse mapper 204, while outer ring mapping decoder 500-1 uses a single inverse mapper 204 (no mapper required).

[0072] In both approaches, except for the mappers 202, 203 and the inverse mapper 204, the encoder 400 / 400-1 includes the same components as the encoder 100. Similarly, in both approaches, the inline and the offline, the decoder 500 / 500-1 includes the same components as the decoder 300, except for the mapper 203 and the inverse mapper 204. Therefore, for the sake of brevity, the components of the encoder 400 / 400-1 and the decoder 500 / 500-1 that are common to the encoder 100 and the decoder 300 will not be described again. The modulo adder 130, the quantizer 140, the inverse quantizer 160 and the modulo subtracter 180 ensure a non-negative bounded reconstruction error, but the encoders 400, 400-1 and the decoders 500, 500-1 are not limited thereto. For example, if negative reconstruction errors are tolerated (eg, by overdrive system 20), then any suitable modulo adder, quantizer, inverse quantizer and modulo subtractor may be used.

[0073] According to one embodiment of the present invention, if a gray level of 0 is important (eg, a reconstruction error of 0 is desired), the mapping function f(.) of mapper 202 is expressed as follows:

number

[0074] where δ denotes the maximum positive error between the reconstructed sample and the original sample, c is a constant non-negative integer less than δ (i.e., c<δ), and maxVal is the maximum value of the mapper's input sample (e.g., the original sample) x for a given bit depth (or bits per hue), which can be expressed as:

number

[0075] Moreover, the inverse mapping function g(.) of the inverse mapper 204 can be expressed as follows:

number

[0076] where x' represents the input to inverse mapper 204 (e.g., the mapped reconstruction sample output by modulo subtractor 180) and Clip(0,maxVal,x'-δ+c) is a clipping function that limits the output value x'-δ+c to a minimum value of 0 and a maximum value of maxVal. In other words, the clipping function computes 0 for all x'-δ+c values ​​less than 0, maxVal for all x'-δ+c values ​​greater than maxVal, and x'-δ+c otherwise.

[0077] As will be explained later, when the constant c=0, there is no reconstruction error for original sample value 0, but other original sample values ​​that exceed the maximum error δ may result in large errors.

[0078] 7A and 7B are tables showing numerical examples of potential reconstruction errors for different sample values ​​given different values ​​of maximum error δ and constant c according to one embodiment of the present invention. In the tables shown in Figures 7A and 7B, the values ​​x' represent the possible outputs of modulo subtractor 180 in the in-line implementation of encoder 400 (of Figure 5A), and y indicates the reconstructed sample at the output of inverse mapper 204.

[0079] In FIG. 7A, assume that the bit depth is 3 (i.e., the original sample values ​​range from 0 to 7), the maximum error δ=1, and the constant c=0. In this table, the value x' for each value x can have one or more possibilities depending on the predicted reconstructed sample. For example, if x=3 and f(x)=4, the number of possible scenarios is as follows: 1) If the sample predictor 110 predicts a value of 3, the residual e' is 1 (=4-3), the quantization index=1, the output of the inverse quantizer 160 (R)=2, the output of the modulo subtractor 180 (x')=5 (=3+2), which results in a reconstructed value y=4. 2) If the sample predictor 110 predicts a value of 2, the residual e'=2 (=4-2), the quantization index=1, R=2, x'=4 (=2+2), which results in a reconstructed value y=3. 3) If the sample predictor 110 predicts a value of 5, then the residual e'=-1 (=4-5), the quantization index=0, R=0, and x'=5 (=0+5), which again results in a reconstructed value y=4.

[0080] As is evident from Fig. 7A, for constant c = 0, the reconstruction error for original sample 0 (i.e., the statistically most important sample value) is 0. However, the error for original sample 5 (i.e., the less preferred sample) can reach 2, which is larger than the maximum desired error δ of 1.

[0081] In Figure 7B, assume that the bit depth is 3, the maximum error δ = 2, and the constant c = 1, where the constant c is greater than 0 and cannot guarantee that the possible reconstruction error for original sample 0 is 0, but the maximum error for original sample 0 is less than or equal to the maximum error for any other original sample value.

[0082] According to one embodiment of the present invention, when gray levels i>0 are of interest, the mapping function f(.) of the mapper 202 is expressed as follows:

number

[0083] The inverse mapping function g(.) of the inverse mapper 204 is expressed as follows:

number

[0084] FIG. 8 is a table illustrating example values ​​of potential reconstruction error for different sample values ​​taking into account the importance of non-zero sample values ​​according to an embodiment of the present invention.

[0085] In the example of Figure 8, assume that the bit depth is 3, the maximum error δ = 1, and the constant c = 0. In this embodiment, original sample 4 is statistically more important to the overdrive system 20, and the reconstruction error for this sample is 0. However, this could be achieved by making the maximum error for original samples 3 and 5 larger than the desired maximum error.

[0086] In the tables of Figures 7A, 7B and 8, the encoder 400 / 400-1 and the decoder 500 / 500-1 guarantee a non-negative reconstruction error. However, in an example where a negative reconstruction error is allowed (e.g., an embodiment using appropriate modulo adders / subtractors and quantizers / dequantizers, but not necessarily the encoder 400 / 400-1 and the decoder 500 / 500-1), and the most significant original value is 0, the mapping function F(.) and the inverse mapping function G(.) replacing the functions f(.) and g(.) can be expressed as follows:

number

[0087] In such an example, the mapping function F(.) and the inverse mapping function G(.) may alternatively be expressed as:

number

[0088] FIG. 9A is a table showing example values ​​of potential reconstruction errors for different sample values ​​based on the mapping and inverse mapping functions of Equations 9-10 according to an embodiment of the present invention. FIG. 9B is a table showing example values ​​of potential reconstruction errors for different sample values ​​based on the mapping and inverse mapping functions of Equations 11-12 according to an embodiment of the present invention. In the tables of FIG. 9A and FIG. 9B, it is assumed that the bit depth is 3 and the maximum error δ=1. Here, original sample 0 is the most important from the perspective of the overdrive system 20, and the reconstruction error for this sample is 0 in both tables.

[0089] In general, conventional image / video encryptors such as DSC and DPCM do not attach special importance to specific gray values. Therefore, the magnitude of the reconstruction error is usually independent of the gray value. Also, transform encryption used in codecs such as HEVC, JPEG-2000, VDC-M, etc. is not suitable to satisfy non-negative errors because in the conventional techniques, quantization is applied to the transform coefficients, and a positive error in the transformed domain does not mean that it also becomes a positive error in the signal space.

[0090] In contrast, according to an embodiment of the present invention, the image storage and compression system gives high priority to certain gray values ​​and exhibits low reconstruction error for such gray values. This is achieved by pixel-wise mapping the original image to another space and compressing the image in the mapped space. Along with maintaining low error for important gray levels, according to an embodiment of the present invention, the encoder and decoder of the image storage and compression system guarantee non-negative error for the reconstructed image, which can improve the performance of the corresponding overdrive system.

[0091] The operations or calculations performed by the components of the encoder 100 / 400 / 400-1 and the decoder 300 / 500 / 500-1 can be realized by a "processing circuit" or a "processor" or a "processor". The "processing circuit" can be realized using hardware, firmware, software, or a combination of these. The processing circuit can include, for example, an application specific integrated circuit (ASIC), a general-purpose or dedicated central processing unit (CPU), a digital signal processor (DSP), a graphics processing unit (GPU), an FPGA, or other programmable logic device. Each function in the processing circuit is performed by a wired hardware that performs the function, or a general-purpose hardware such as a CPU that executes instructions stored in a non-transitory storage medium. The processing circuit is formed on one printed wiring board (PWB) or distributed on PWBs that are connected to each other. The processing circuit can include other processing circuits, but can also include, for example, an FPGA and a CPU that are connected to each other on a PWB.

[0092] Terms such as "first," "second," and "third" are used to refer to various elements, components, regions, layers, portions, and the like, but are not limited by such modifiers. Such terms are used to distinguish one element, component, region, layer, portion from another element, component, region, layer, portion, and the like, without departing from the spirit and scope of the present invention.

[0093] The terms used herein are used only for the purpose of describing particular embodiments and are not intended to limit the present invention. In this specification, unless a number is specifically mentioned, all cases of singular or plural are included. The term "comprising" a feature, step, operation, part, component, etc. means that other features, steps, operations, parts, components, etc. may be included in addition to the part in question. The term "and / or" includes all combinations of one or more of the listed items. In addition, the term "can" used in describing the embodiments of the present invention means that it is applicable to "one or more embodiments of the present invention." The term "exemplary" refers to an example or drawing.

[0094] The words "use", "utilization", and the like, along with other similar terms, may be used interchangeably.

[0095] The phrases "at least one of X, Y, and Z" and "at least one selected from the group including X, Y, and Z" refer to X only, Y only, Z only, or a combination of two or more of X, Y, and Z, for example, XYZ, XYY, YZ, and ZZ.

[0096] Although the present invention has been described in detail with reference to specific embodiments, the embodiments described herein are not intended to limit the scope of the present invention to those described. Those skilled in the art will appreciate that the above-described combinations and structures and methods of operation may be modified or altered without significantly departing from the principles and scope of the invention as expressed in the following claims and their equivalents. [Explanation of symbols]

[0097] 1: Frame Compensation System 10: Video compression and storage systems 20: Overdrive System 30: Disconnect and delay circuit 100, 400, 400-1: encoder 110: Sample Predictor 120: Subtractor 130: Modulo adder 140: Quantizer 150: Entropy Encoder 160: Inverse quantizer 170: Adder 180: Modulo subtractor 200: Storage media 203, 203: mapper 204: Inverse mapper 300, 500, 500-1: Decoder 310: Entropy decoder

Claims

1. receiving an original sample of a frame; selecting, by a quantizer, an index corresponding to a quantization band based on the original samples; outputting, by an inverse quantizer, the maximum value of the quantization band based on the index as a reconstructed biased residual; and generating a reconstructed version of the original representation based on the reconstruction bias residual; Including, The quantizer adjusts the quantization band size and the reconstruction point in each band so that the quantization error is not negative.

2. subtracting a prediction of a previous reconstructed sample from the original sample to generate a residual for the original sample; and applying modulo summation to the residuals of the original samples to generate a biased residual based on a bit depth of the original samples and a maximum allowable error; The method of claim 1 further comprising:

3. Applying the modulo sum to the residual includes calculating the biased residual as: the original samples correspond to color values ​​of pixels of the frame; The deflection residual is −δ≦e′≦2 ｂｉｔｄｅｐｔｈ－１ The method of claim 2, wherein the range is within the range represented by [0010] (where e′ represents the bias residual, e represents the residual of the original samples, bitdepth represents the bit depth of the original samples, and δ represents the maximum allowable error.)

4. Selecting the index corresponding to the quantization band comprises: quantizing the biased residual based on the maximum allowable error to generate a quantized biased residual; and identifying the index corresponding to the quantized biased residual; The method of claim 2 , comprising:

5. The method of claim 4 , wherein quantization of the biased residual is based on a quantization step size equal to the maximum allowable error plus one.

6. The method of claim 4 , wherein quantizing the biased residual comprises calculating the index of the quantized biased residual as follows: [0025] (I[e'] represents the index, Ceiling(.) represents the ceiling function, e' represents the bias residual, and δ represents the maximum allowable error.)

7. The method of claim 4 , wherein quantizing the biased residual comprises using non-uniform quantization bands.

8. performing inverse quantization on the quantized biased residual to generate the reconstructed biased residual; adding a prediction of a previous reconstructed sample to the reconstructed deflection residual to generate a reconstructed deflection sample; applying a modulo subtraction and clipping operation to the reconstructed deflection samples based on a bit depth of the original samples and the maximum allowable error to generate reconstructed samples; and The method of claim 4 , further comprising generating a prediction of a next reconstruction sample based on the reconstruction sample.

9. Applying the modulo subtraction to the reconstructed deflection samples converts the reconstructed samples into The method of claim 8 , comprising calculating: [0030] (where y represents the reconstructed sample, x represents the reconstructed biased sample, bitdepth represents the bit depth of the original sample, and δ represents the maximum allowable error.)

10. The method of claim 8 , wherein the inverse quantization is performed based on a quantization band size equal to the maximum allowable error and the index.

11. The method of claim 8 , wherein performing inverse quantization on the quantized biased residual comprises calculating the reconstructed biased residual as follows: [0045] (Q(e') represents the reconstruction deflection residual associated with the deflection residual e', I[e'] represents the index, and δ represents the maximum allowable error.)

12. The method of claim 8 , wherein performing inverse quantization on the quantized biased residual is based on a non-uniform quantization table.

13. encoding the index via entropy coding to generate encoded values ​​having a non-negative reconstruction error; and storing the pre-encoded values ​​in a memory; The method of claim 1 further comprising:

14. Receiving the original sample of the frame includes: generating the original samples by reducing a bit depth of color values ​​of pixels of the frame; The method of claim 1 , wherein the frames are in Red-Green-Blue (RGB) format or Red-Green-Blue-Green (RGBG) format.

15. Outputting the maximum value of the quantization band as the reconstructed bias residual decoding encoded values ​​corresponding to the original samples of the frame to generate the index; and The method of claim 1 , comprising: performing inverse quantization based on the index to generate the reconstructed biased residual.

16. 16. The method of claim 15, wherein performing inverse quantization based on the index comprises calculating the reconstructed biased residual as follows: [0050] (Q(e') represents the reconstruction bias residual, I[e'] represents the index, e' represents the bias residual, and δ represents the maximum allowable error.)

17. generating the reconstructed version of the original exemplar based on the reconstruction bias residuals, adding a prediction of a previous reconstructed sample to the reconstructed bias residual to generate a reconstructed bias sample; and The method of claim 1 , comprising applying a modulo subtraction to the reconstructed biased samples based on a bit depth of the original samples and a maximum allowable error to generate the reconstructed version of the original samples.

18. The method of claim 17 , further comprising generating a prediction of a reconstructed version of a next exemplar based on the reconstructed version of the original exemplar.

19. 20. The method of claim 17, wherein applying modulo subtraction to the reconstructed biased samples comprises calculating the reconstructed version of the original samples as follows: [006] (where y represents the reconstructed version of the original sample, x represents the reconstructed biased sample, bitdepth represents the bit depth, and δ represents the maximum allowable error.)

20. A processor; a processor memory local to said processor; Including, The processor memory, when executed by the processor, causes the processor to: receiving an original sample of a frame; selecting, by a quantizer, an index corresponding to a quantization band based on the original samples; outputting, by an inverse quantizer, the maximum value of the quantization band based on the index as a reconstructed biased residual; and generating a reconstructed version of the original representation based on the reconstruction bias residual; and storing an instruction to execute the The quantizer and the inverse quantizer adjust the quantization band size and the reconstruction point in each band so that the quantization error is not negative; the difference between the original sample and the reconstructed version of the original sample is non-negative; Video compression and storage system.

Images