Method and system for compressing and decompressing images

Abstract

The present disclosure relates to methods and systems for compressing and decompressing images. A method for compressing a frame in an image compression and storage system, the method comprising: applying modulo addition to residuals of original samples of the frame to generate biased residuals based on a bit depth of the original samples and a maximum allowed error; quantizing the biased residuals based on the maximum allowed error to generate quantized biased residuals; and encoding values corresponding to the quantized biased residuals to generate encoded values having non-negative reconstruction errors.

Description

[0001] Cross-references to related applications

[0002] This application claims priority and benefit to U.S. Provisional Application No. 63 / 063,004 (“Compression with Positive Reconstruction Error”), filed August 7, 2020, and U.S. Provisional Application No. 63 / 062,963 (“DPCM Codec with Higher Reconstruction Quality on Important Gray Levels”), both filed August 7, 2020, the entire contents of which are incorporated herein by reference.

[0003] This application also claims priority and benefit to U.S. Patent Application No. 17 / 070,898, filed on October 14, 2020, entitled "DPCM CODEC WITH HIGHER RECONSTRUCTION QUALITY ON IMPORTANT GRAY LEVELS". Technical Field

[0004] The embodiments of this disclosure generally relate to image processing systems and methods. Background Technology

[0005] Flat panel displays, such as liquid crystal displays (LCDs) and light-emitting diode (LED) displays, have become very common due to their small size, excellent image quality, and lower power consumption.

[0006] Pixels in a display panel have specific response times. The data provided to the pixels for display may change faster than the pixels can respond. Therefore, undesirable effects such as motion blur or ghosting may occur. To make the displayed image more satisfactory, image compensation techniques are necessary. Overdrive algorithms have been developed to compensate for image data, thereby reducing motion blur produced by the display pixels.

[0007] The information disclosed in this background section is intended only to enhance the understanding of this disclosure, and therefore may contain information that does not form prior art known to those skilled in the art. Summary of the Invention

[0008] Embodiments of this disclosure relate to systems and methods for image compression, storage, and decompression, which ensure bounded and non-negative reconstruction errors. In some embodiments, the system achieves this by encoding the image or frame, in part, using an appropriate quantization band size and an added bias. The system and method can improve the image compensation performance of overdrive systems.

[0009] Embodiments of this disclosure relate to systems and methods for image compression, storage, and decompression, which ensure zero reconstruction error for a particular high-priority grayscale value or at least a lower reconstruction error for a particular high-priority grayscale value relative to other grayscale values.

[0010] According to some embodiments of the present invention, a method for compressing frames in an image compression and storage system is provided, the method comprising: applying a modulo-addition method to the residuals of the original samples of the frame to generate a bias residual based on the bit depth and maximum permissible error of the original samples; quantizing the bias residual based on the maximum permissible error to generate a quantized bias residual; and encoding a value corresponding to the quantized bias residual to generate an encoded value with a non-negative reconstruction error.

[0011] In some embodiments, applying the modulo addition method to the residuals includes calculating the bias residuals as:

[0012]

[0013] Where e′ represents the bias residual, e represents the residual of the original sample, bitdepth represents the bit depth of the original sample, and δ represents the maximum permissible error, where the original sample corresponds to the color value of the pixel in the frame.

[0014] In some embodiments, the bias residual is expressed as -δ≤e′≤2 bitdepth Within the range of -1.

[0015] In some embodiments, the method further includes: receiving the original sample; and subtracting the prediction of the previously reconstructed sample from the original sample to generate the residual of the original sample.

[0016] In some embodiments, the value corresponding to the quantized bias residual is the quantization index corresponding to the quantized bias residual.

[0017] In some embodiments, the bias residual is quantized based on a quantization step size equal to the maximum permissible error plus 1.

[0018] In some embodiments, quantizing the bias residual includes calculating the quantization index of the quantized bias residual as follows:

[0019]

[0020] Where I[e′] represents the quantization index, Ceiling(.) represents the upper bound function, e' represents the bias residual, and δ represents the maximum permissible error.

[0021] In some embodiments, quantizing the bias residual includes utilizing a non-uniform quantization band.

[0022] In some embodiments, the method further includes: performing inverse quantization on the quantized bias residual to generate a reconstructed bias residual; adding the prediction of the previously reconstructed sample to the reconstructed bias residual to generate a reconstructed bias sample; applying a modulo subtraction and clipping operation to the reconstructed bias sample based on the bit depth and maximum permissible error of the original sample to generate a reconstructed sample; and generating a prediction of a subsequently reconstructed sample based on the reconstructed sample.

[0023] In some embodiments, applying the modulus subtraction method to the reconstructed bias samples includes calculating the reconstructed samples as:

[0024]

[0025] Where y represents the reconstructed sample, x represents the reconstructed bias sample, bitdepth represents the bit depth of the original sample, and δ represents the maximum permissible error.

[0026] In some embodiments, inverse quantization is performed based on a quantization band size equal to the maximum permissible error and further based on the quantization index.

[0027] In some embodiments, performing inverse quantization on the quantized bias residual includes calculating the reconstructed bias residual as:

[0028] Q[e′]=(δ+1)I[e′]

[0029] Where e' represents the bias residual, Q[e'] represents the reconstructed bias residual associated with the bias residual, I[e′] represents the quantization index, and δ represents the maximum permissible error.

[0030] In some embodiments, inverse quantization is performed on the quantized bias residuals based on a non-uniform quantization table.

[0031] In some embodiments, encoding the value corresponding to the quantized bias residual includes performing entropy encoding on the value corresponding to the quantized bias residual to generate an encoded value.

[0032] In some embodiments, the method further includes: truncating the frame by reducing the bit depth of the color values ​​of the pixels of the frame to generate an original sample, wherein the frame has a red, green and blue (RGB) format or a red, green, blue and green (RGBG) format.

[0033] According to some embodiments of the present invention, a method is provided for decompressing stored image data corresponding to a frame in an image compression and storage system. The method includes: decoding encoded values ​​corresponding to original samples of the frame to generate decoded values ​​corresponding to quantized bias residuals; performing inverse quantization on the decoded values ​​to generate reconstructed bias residuals; adding predictions of previously reconstructed samples to the reconstructed bias residuals to generate reconstructed bias samples; and applying modulo subtraction to the reconstructed bias samples based on the bit depth and maximum permissible error of the original samples to generate reconstructed samples.

[0034] In some embodiments, the method further includes: generating predictions of subsequently reconstructed samples based on the reconstructed samples.

[0035] In some embodiments, applying the modulus subtraction method to the reconstructed bias samples includes calculating the reconstructed samples as:

[0036]

[0037] Where y represents the reconstructed sample, x represents the reconstructed bias sample, bitdepth represents the bit depth, and δ represents the maximum permissible error.

[0038] In some embodiments, the decoded value is a quantization index, and performing inverse quantization on the decoded value includes calculating the reconstructed bias residual as:

[0039] Q[e′]=(δ+1)I[e′]

[0040] Where Q[e'] represents the reconstructed bias residual, I[e′] represents the quantization index, e' represents the bias residual, and δ represents the maximum permissible error.

[0041] According to some embodiments of the present invention, an image compression and storage system is provided, comprising: a processor; and a processor memory local to the processor, wherein the processor memory has instructions stored thereon, the instructions causing the processor to perform the following when executed by the processor: applying modulo addition to the residuals of original samples of a frame to generate a bias residual based on the bit depth and maximum permissible error of the original samples; quantizing the bias residual based on the maximum permissible error to generate a quantized bias residual; encoding a value corresponding to the quantized bias residual to generate an encoded value for storage in a storage medium; retrieving the encoded value from the storage medium; decoding the encoded value to generate a decoded value corresponding to the quantized bias residual; performing inverse quantization on the decoded value to generate a reconstructed bias residual; adding a prediction of a previously reconstructed sample to the reconstructed bias residual to generate a reconstructed bias sample; and applying modulo subtraction to the reconstructed bias sample based on the bit depth and maximum permissible error of the original sample to generate a reconstructed sample, wherein the difference between the original sample and the reconstructed sample is only non-negative. Attached Figure Description

[0042] The accompanying drawings, together with the specification, illustrate exemplary embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0043] Figure 1 This is a schematic diagram of a frame compensation system utilizing an image compression and storage system according to some embodiments of the present disclosure.

[0044] Figure 2A This is a schematic diagram of an encoder of an image compression and storage system according to some embodiments of the present disclosure.

[0045] Figure 2B This is a schematic diagram of a decoder for an image compression and storage system according to some embodiments of the present disclosure.

[0046] Figure 3A This is a schematic diagram illustrating a portion of an encoder and a portion of a decoder according to some embodiments of the present disclosure.

[0047] Figure 3B This is an illustration of the identity operation of the entropy encoder and entropy decoder according to some embodiments of this disclosure. Figure 3A A simplified diagram.

[0048] Figure 4A The illustration shows a quantization table utilized by a quantizer according to some embodiments of the present disclosure.

[0049] Figure 4B The illustration shows the operation of a quantizer according to some embodiments of the present disclosure.

[0050] Figure 4C and Figure 4D Uniform quantization tables and non-uniform quantization tables utilized by a quantizer according to some embodiments of the present disclosure are illustrated respectively.

[0051] Figure 5A This is a schematic diagram of an encoder with in-line / in-loop mapping according to some embodiments of this disclosure.

[0052] Figure 5B This is a schematic diagram of an encoder with offline / loop external mapping according to some embodiments of the present disclosure.

[0053] Figures 6A to 6B These are schematic diagrams of decoders with online / in-loop mapping and decoders with offline / out-loop mapping, respectively, according to some embodiments of this disclosure.

[0054] Figures 7A to 7B This is a table illustrating numerical examples of potential reconstruction errors for different sample values ​​given different values ​​of the maximum permissible error δ and the constant c, according to some embodiments of the present disclosure.

[0055] Figure 8 This is a table illustrating numerical examples of potential reconstruction errors for different sample values, given the importance of non-zero sample values, according to some embodiments of the present disclosure.

[0056] Figures 9A to 9B This is a table illustrating numerical examples of potential reconstruction errors for different sample values ​​based on mapping and inverse mapping functions of image compression and storage systems that cannot guarantee non-negative reconstruction errors, according to some embodiments of the present disclosure. Detailed Implementation

[0057] The detailed description set forth below is intended as a description of exemplary embodiments of systems and methods for image compression, storage, and decompression provided in this disclosure, and is not intended to represent the only form in which this disclosure may be constructed or utilized. The description illustrates features of this disclosure in conjunction with the illustrated embodiments. However, it will be understood that the same or equivalent functions and structures may be implemented by different embodiments that are also intended to be covered within the scope of this disclosure. As noted elsewhere herein, the same element reference numerals are intended to indicate the same elements or features.

[0058] Figure 1 This is a schematic diagram of a frame compensation system 1 utilizing an image compression and storage system 10 according to some embodiments of the present disclosure.

[0059] In some embodiments, the image compression and storage system 10 includes an encoder 100 for encoding (e.g., compressing) data to reduce its size, a storage medium 200 for storing the encoded data, and a decoder 300 for decoding (e.g., decompressing) the stored encoded data to reconstruct the data. Storing specific data, such as 4K image frames, consumes a large amount of memory space. Here, the encoder 100 and decoder 300 allow for the use of a smaller storage medium 200, which saves space and cost. Figure 1 As illustrated, in some examples, the image compression and storage system 10 may form part of the frame compensation system 1, in which the overdrive system 20 uses a previous frame to compensate the current frame (e.g., to reduce ghosting or blurring) to produce a compensated current frame. In some examples, the truncation and delay circuit 30 receives a frame (e.g., 8 bits per color (bpc)), i.e., the current frame, truncates the frame (i.e., reduces the number of bpc of the frame's pixel values ​​to, for example, 3 bpc), and delays the frame to produce a truncated previous frame (3 bpc). The image compression and storage system 10 then compresses the truncated previous frame and stores it for later use by the overdrive system 20. Here, truncation and compression can make the stored frame significantly smaller in size than the original (previous) frame. For example, in the case of RGB (red, green, and blue) input frames, where each color value is represented by 8 bits (or 24 bits per pixel (bpp)), truncation can result in a 3bpc / 9bpp frame representation, and the 1.5:1 compression performed by encoder 100 can further reduce the frame representation to 2bpc / 6bpp. However, embodiments of this disclosure are not limited to RGB input frames, and any suitable input frame format such as RGBG (red, green, blue, and green) (for pentile subpixel arrangements) can be used. For example, in the case of RGBG input frames, truncation can reduce the frame size from 16bpp to 6bpp, and the 1.5:1 compression performed by encoder 100 can further reduce the frame size to 4bpp. Furthermore, embodiments of this disclosure are not limited to using the above truncation operations, and in some examples, the truncation and delay circuitry 30 can be replaced by delay blocks, and the truncation operation can be omitted from the process described herein.

[0060] If the reconstruction error, that is, the error introduced by the image compression and storage system 10 (all of which is quantization error), is only non-negative, the performance of the overdrive system 20 can be improved. Coding schemes of related techniques typically optimize the mean square error (MSE) or peak signal-to-noise ratio (PSNR), meaning the polarity of the error in the reconstructed image can be either positive or negative. According to some embodiments, the image compression and storage system 10 improves the performance of the overdrive system 20 relative to related techniques by ensuring that the reconstruction error (i.e., quantization error), defined as the difference between the reconstructed frame and the original frame, is always non-negative (i.e., greater than or equal to zero) and is also limited to a set maximum value.

[0061] Throughout this disclosure, a frame refers to data comprising an image frame containing multiple pixels. Each pixel of a frame can be represented by multiple color values ​​(e.g., red, green, and blue). The original sample of a frame corresponds to the color values ​​of the pixels in the frame.

[0062] Figure 2A This is a schematic diagram of the encoder 100 of an image compression and storage system 10 according to some embodiments of the present disclosure. Figure 2B This is a schematic diagram of the decoder 300 of an image compression and storage system 10 according to some embodiments of the present disclosure.

[0063] refer to Figure 2A According to some embodiments, encoder 100 utilizes a predictive coding scheme to encode the residual of the original sample (e.g., a truncated frame) in the predictive coding scheme, rather than encoding the original sample itself. The residual (also called error e) is defined as the difference between the original sample and a prediction based on one or more neighboring samples (e.g., a prediction of a previous (reconstructed) sample). In some embodiments, sample predictor 110 generates a prediction of a previous sample, and subtractor 120 calculates the residual value based on the original sample and the predicted sample. In some embodiments, subtractor 120 subtracts the prediction of a previously reconstructed sample from the original sample to generate the residual of the original sample. The calculated residual can be positive or negative. Modulo adder 130 adds a bias value to the residual to produce a bias residual value in order to reduce the signaling cost of 1 bit of symbol information. Quantizer 140 quantizes the bias residual value to generate a quantized bias value (i.e., a quantized bias residual), and passes the quantization index corresponding to the quantized bias residual to entropy encoder 150 for further compression (e.g., lossless compression such as entropy coding). The compressed data (or encoded value) is stored in storage medium 200. As will be explained further below, in some embodiments, the bias value is a function of the quantization step size.

[0064] According to some embodiments, in order to generate predicted samples, decoder 300 decodes the encoded values ​​corresponding to the original samples of the frame to generate decoded values ​​(i.e., quantization indexes) corresponding to the quantized bias residuals. In some embodiments, inverse quantizer 160 performs inverse quantization on the decoded values ​​to generate reconstructed bias residuals.

[0065] In some embodiments, inverse quantizer 160 performs the inverse operation of the quantization operation performed by quantizer 140 on the quantized bias residual to generate a reconstructed bias residual. Adder 170 adds the prediction of the previously reconstructed sample to the reconstructed bias residual to generate a reconstructed bias sample. Modulus subtractor 180 subtracts the bias value added by modulus adder 130 from the reconstructed bias sample to generate a reconstructed sample. In some examples, a limiting function can be applied to the reconstructed sample to ensure it is within the desired dynamic range, which for a given bit depth can be represented as [0, 2]. bitdepth -1]. The reconstructed sample (e.g., the reconstructed sample after clipping) is used by the sample predictor 110 to predict the next sample.

[0066] refer to Figure 2B In some embodiments, decoder 300 includes entropy decoder 310 for decompressing stored samples (or, stored encoded values) received from storage medium 200, and the same inverse quantizer 160, adder 170, modulo subtractor 180, and sample predictor 110 as described above. Except for entropy decoder 310, the remaining components of decoder 300 are identical to those in encoder 100 and configured in the same manner. Therefore, for the sake of brevity, their detailed description will not be repeated here.

[0067] Because decoder 300 cannot access the original samples and can only access the reconstructed samples, sample predictor 110 of decoder 300 uses the reconstructed samples to generate predictions for previous samples. Furthermore, because the operations of encoder 100 and decoder 300 are mirror images of each other, sample predictor 110 of encoder 100 also uses the same reconstructed samples to generate predictions for previous samples, enabling encoder 100 to synchronize with decoder 300 even if encoder 100 can access the original samples.

[0068] In some embodiments, encoder 100 and decoder 300 operate on a single color value of a pixel. Therefore, multiple encoders 100 and decoders 300 can operate in parallel (e.g., concurrently / simultaneously) to process various colors of a pixel.

[0069] Figure 3AThis is a schematic diagram illustrating a portion of an encoder 100 and a portion of a decoder 300 according to some embodiments of the present disclosure. Figure 3B The illustration of some embodiments of this disclosure describes the identity operations of the entropy encoder 150 and the entropy decoder 310. Figure 3A A simplified diagram.

[0070] According to some embodiments, the operations performed by components of the encoder 100 and decoder 300, excluding the quantizer 140, are lossless operations. For example, Figure 3A The sequential operations of the entropy encoder 150 and entropy decoder 310 (as block A) shown are identity operations and can therefore be effectively omitted, resulting in Figure 3B Therefore, the error in the signal space, defined as the difference between the reconstructed sample at the output of decoder 300 and the original sample at the input of encoder 100, is equivalent to the error in the residual space:

[0071] Error (residual) = R – e' = Q[e'] – e' Equation (1)

[0072] Where e' is the bias residual input to quantizer 140, R is the output of inverse quantizer 160, and Q[.] is the combined quantization and inverse quantization function of quantizer 140 and inverse quantizer 160.

[0073] Therefore, if the error in the residual space is kept non-negative, the error in the signal space will also be non-negative. Furthermore, if the error in the residual space is bounded, that is, if R–e'≤δ, where δ is a positive integer representing the maximum permissible error, then the error in the signal space will also be bounded to δ. This relationship holds even with the presence of modulo adders 130 and modulo subtractors 180. Conversely, transform encoding / decoding used in related techniques (e.g., High Efficiency Video Codec (HEVC), JPEG-2000, or VESA Display Compression-M (VDC-M)) may not be suitable for ensuring non-negative reconstruction errors because positive errors in the transform domain cannot guarantee non-negative reconstruction errors in the signal space. This may be because transforms such as the Discrete Cosine Transform (DCT) preserve the L2 norm (or energy) between the original coefficients and the transform coefficients, rather than the L1 norm.

[0074] Therefore, in some embodiments, the modulus adder 130 and the quantizer 140 are configured in a way that ensures the error in the residual space remains non-negative and bounded (i.e., restricted to δ), which ensures that the error in the signal space is the same.

[0075] Quantizer 140 and inverse quantizer 160 ensure non-negative, bounded errors by appropriately adjusting the quantization band size and the reconstruction point within each band. In some embodiments, given a maximum permissible error δ, quantizer 140 utilizes a uniform or non-uniform quantization band with a maximum quantization step size of δ+1, and uses the reconstruction point at the right edge of that band (i.e., the maximum value). Furthermore, in some embodiments, the value zero is located at the right edge of the band (e.g., the first quantization band at index 1), such that zero appears in the reconstructed value. This may be due to the residual peaking at zero.

[0076] Figure 4A The illustration shows a quantization table utilized by quantizer 140 according to some embodiments of the present disclosure. Figure 4B The illustration shows the operation of a quantizer 140 according to some embodiments of the present disclosure, with input e' and output R. Figure 4C and Figure 4D Uniform quantization tables and non-uniform quantization tables utilized by quantizer 140 are illustrated in some embodiments of the present disclosure.

[0077] refer to Figure 4A Quantizer 140 uses a uniform quantization band with a quantization step size of δ+1, where the reconstruction point is at the right edge of each band. In some embodiments, the first quantization band has a zero right edge (i.e., a zero reconstruction point). Here, the bias residual e' is an integer. In some embodiments, when using the modulus addition and subtraction parameters (also known as optimized modulus addition and subtraction parameters) defined in equations (6) and (8), the number of bands NB can be expressed as:

[0078]

[0079] Where δ represents the maximum permissible error, and Ceiling(.) represents the upper bound function. Here, δ+1 can be the maximum permissible step size, i.e., a smaller step size can be selected. In some other embodiments, when using unoptimized modulo addition and subtraction parameters (i.e., parameters not defined by equations (6) and (8)), the number of bands NB can be expressed as:

[0080]

[0081] In some embodiments, when quantizing the bias residual, the quantizer 140 calculates the quantization index of the bias residual as follows:

[0082]

[0083] Among them, when using the optimized modulo addition and subtraction parameters defined by equations (6) and (8), -δ≤e′≤2 bitspth-1 Otherwise, when using unoptimized modulo addition and subtraction parameters, -δ≤e′≤2 bitdepth -1+δ.

[0084] I[e′] represents the quantization index, Ceiling(.) represents the upper bound function, e' represents the bias residual, and δ represents the maximum permissible error.

[0085] Similarly, performing inverse quantization on the quantized bias residuals can include calculating the reconstructed bias residuals as:

[0086]

[0087] Where Q[e'] represents the reconstructed bias residual, I[e′] represents the quantization index, and e' represents the bias residual. Equations (2) and (3) are applied to embodiments utilizing uniform quantization. When using non-uniform quantization, the maximum step size can be δ+1.

[0088] Figure 4B The illustration shows an example where δ = 2 and the bit depth (i.e., the number of bits per color) is 3. Here, bit depth refers to the color information stored in an image and affects the image's dynamic range. The higher the bit depth of an image, the more colors it can store. The simplest image, a 1-bit image, can only display two colors, black and white. This is because 1 bit can only store one of two values, 0 (white) and 1 (black). As the bit depth increases, the dynamic range also increases, which increases the number of quantization bands.

[0089] like Figure 4B As illustrated in the example, a bias residual e' of 1 results in a reconstructed value R of 3.

[0090] According to some embodiments, the modulus adder 130 calculates the bias residual e' as:

[0091]

[0092] This means that the bias residual e' is within the range that can be expressed as follows:

[0093] -δ≤e′≤2 bitdepth -1 Equation (7)

[0094] The upper limit of the bias residual e' allows for non-uniform quantization bands, which can reduce errors for a specific bias value and thus improve performance.

[0095] For example, when δ = 2 and bit depth = 3, –2 ≤ e' ≤ 7, and Figure 4C The uniform quantization table shown in the figure and Figure 4DThe non-uniform quantization table shown can be utilized by quantizer 140. In this table, a uniform quantization band size of 2 is used. Figure 4C In this context, the quantization error across all four bands is bounded by δ (i.e., 2). However, embodiments of this disclosure also allow... Figure 4D The non-uniform quantization bands of the quantization table shown have smaller quantization errors bounded by 1 (and not 2) for the third and fourth bands. In some examples, quantizer 140 reserves narrower bands for those bands that are statistically more significant (e.g., bands containing multiple values ​​in the bias residuals, or bands whose values ​​have a greater impact on the performance of overdrive system 20). Therefore, the smaller quantization error achieved through the operation of modulo adder 130 can improve the performance of overdrive system 20.

[0096] Modular subtractor 180 can perform the opposite operation of modular adder 130 by removing the bias value added to the residual by modular adder 130. According to some embodiments, for a given input x (reconstructed bias sample), the output y (reconstructed sample) of modular subtractor 180 is represented as:

[0097]

[0098] As mentioned above, in some examples, a clipping function can be applied to the reconstructed samples to ensure they are within the desired dynamic range, which for a given bit depth can be expressed as [0, 2]. bitdepth -1]. In the 8bpc example, the limiting function can limit the output of the modulo subtractor 180 to [0,7].

[0099] Therefore, as described above, by using an appropriate quantization band size and added bias, the image compression and storage system 10 ensures (e.g., guarantees) non-negative and bounded errors on the reconstructed image, which can improve the performance of the overdrive system 20.

[0100] In addition to ensuring bounded, non-negative reconstruction error, according to some embodiments, it is also desirable for certain gray levels (e.g., zero gray value) to have zero or at least less error than other gray values. For example, for a color depth of 3, the range of gray values ​​can be from 0 to 7. In some examples, as gray values ​​decrease from 7 to 0, overdrive compensation can assign greater importance (higher priority) to gray values, with 0 having the greatest importance (highest priority). In such examples, high reconstruction error is acceptable for low-priority gray values ​​such as 5, 6, and 7. This contrasts with related art codecs, where all gray levels are treated equally and the reconstruction error is independent of the magnitude (or gray level) of the pixel sample.

[0101] According to some embodiments, the image compression and storage system 10 is configured to ensure smaller reconstruction errors for important grayscale values, as well as non-negative reconstruction errors on other grayscale levels. In some embodiments, the encoder 100 modifies the original grayscale values ​​to achieve lower errors for specific grayscale levels. This may come at the cost of higher errors on other less important grayscale levels.

[0102] Figure 5A This is a schematic diagram of an encoder 400 with online / in-loop mapping according to some embodiments of the present disclosure. Figure 5B This is a schematic diagram of an encoder 400-1 with offline / loopout mapping according to some embodiments of the present disclosure. Figures 6A to 6B These are schematic diagrams of a decoder 500 with online / in-loop mapping and a decoder 500-1 with offline / out-loop mapping, respectively, according to some embodiments of the present disclosure.

[0103] According to some embodiments, encoder 400 / 400-1 includes a first mapper (e.g., grayscale mapping circuit) 202 that maps the grayscale value x of the original sample to the mapped value (also called the mapped sample) f(.) before encoding, and further includes an inverse mapper (e.g., inverse grayscale mapping circuit) 204 that performs the inverse operation g(.) of the first mapper 202. Therefore, encoder 400 / 400-1 encodes the original sample in the mapped space rather than the signal space. Here, the dynamic range of the mapped space can be the same as the dynamic range of the original space. In other words, the bit depth of the mapped space and the original space can be the same.

[0104] According to some embodiments, in Figure 5A In the online / loop implementation illustrated, encoder 400 includes a first mapper 202 that operates on the original samples before adder 170, a second mapper 203 that operates on sample predictor 110, and further includes an inverse mapper 204 that operates on the output of modulo subtractor 180. In these embodiments, inverse mapper 204 performs an inverse mapping operation on the reconstructed samples mapped by modulo subtractor 180 and generates unmapped reconstructed samples, which are used by sample predictor 110 to produce a prediction of the previously reconstructed unmapped samples. Second mapper 203 maps this value (i.e., the prediction of the previously reconstructed unmapped samples) to the mapped previously reconstructed samples so that the value is subtracted from the mapped samples at subtractor 120.

[0105] In some embodiments, Figure 5BIn the offline / out-of-loop implementation illustrated, the inverse mapper 204 does not operate on the input of the sample predictor 110, and therefore, the second mapper 203 does not need to operate on the output of the sample predictor 110. In such an embodiment, the mapping operation is performed entirely outside the encoder loop. Because the operations of the first mapper 202 and the inverse mapper 204 are performed outside the codec, this offline / out-of-loop implementation can be more easily used with existing codecs. Furthermore, the offline / out-of-loop implementation has lower complexity compared to the online / in-loop implementation because fewer operations / computations are performed. However, the online / in-loop implementation can lead to higher performance.

[0106] like Figures 6A to 6B As illustrated, the decoder 500 for the online / in-loop mapping implementation utilizes both the second mapper 203 and the inverse mapper 204, while the decoder 500-1 for the offline / out-of-loop mapping implementation utilizes a single inverse mapper 204 (no mapper required).

[0107] In either embodiment, encoder 400 / 400-1 includes the same components as encoder 100 and has the same configuration as encoder 100, except for first mapper 202, second mapper 203, and inverse mapper 204. Similarly, in both the in-loop and out-of-loop embodiments, decoder 500 / 500-1 includes the same components as decoder 300 and has the same configuration as decoder 300, except for second mapper 203 and inverse mapper 204. Therefore, for the sake of brevity, the descriptions of the components in encoder 400 / 400-1 and decoder 500 / 500-1 that are the same as encoder 100 and decoder 300, respectively, will not be repeated. Although modulus adder 130, quantizer 140, inverse quantizer 160, and modulus subtractor 180 ensure bounded, non-negative reconstruction errors, encoders 400 and 400-1 and decoders 500 and 500-1 are not limited thereto. For example, any suitable modulo adder, quantizer, inverse quantizer, and modulo subtractor can be used when negative reconstruction error can be tolerated (e.g., by overdrive system 20).

[0108] According to some embodiments, when 0 gray level is important (e.g., when 0 reconstruction error is expected), the mapping function f(.) of the first mapper 202 is expressed as:

[0109]

[0110] Where δ represents the maximum positive error between the reconstructed sample and the original sample, c is a constant non-negative integer less than δ (i.e., c < δ), and maxVal is the maximum value of the mapper input sample (e.g., the original sample) x for a given bit depth (or bits per color), maxVal can be expressed as:

[0111] maxVal = 2 bitdept h-1 Equation (10)

[0112] Furthermore, the inverse mapping function g(.) of inverse mapper 204 can be expressed as:

[0113]

[0114] Here, x′ represents the input to the inverse mapper 204 (e.g., a reconstructed sample of the map output by the modulo subtractor 180), and Clip(0, maxVal, x′-δ+c) is a clipping function that restricts the output value x′-δ+c to a minimum of 0 and a maximum of maxVal. In other words, the clipping function produces zero for all values ​​of x′-δ+c less than zero, maxVal for all values ​​of x′-δ+c greater than maxVal, and x′-δ+c in all other cases.

[0115] As will be explained below, when the constant c = 0, there is no reconstruction error at the original sample values ​​that are 0, although this may come at the cost of increased error for other original sample values ​​that exceed the maximum permissible error δ.

[0116] Figures 7A to 7B This is a table illustrating numerical examples of potential reconstruction errors for different sample values ​​given different values ​​of the maximum permissible error δ and the constant c, according to some embodiments of the present disclosure. Figures 7A to 7B In the table shown in the figure, the value x' represents the modulo subtractor 180 at ( Figure 5A The possible output of the encoder 400 in the online / loop implementation, and y represents the reconstructed sample (i.e. the unmapped reconstructed sample) at the output of the inverse mapper 204.

[0117] exist Figure 7AIn this table, we assume a bit depth of 3 (i.e., the original sample values ​​(also called raw samples or raw values) range from 0 to 7), a maximum permissible error δ = 1, and a constant c = 0. In this table, the value x' for each value of x can have one or more possibilities based on the predicted reconstructed sample. For example, with x = 3 and f(x) = 4, there are several possible scenarios: 1) When the sample predictor 110 predicts a value of 3, the residual e' is 1 (= 4 - 3), the quantization index = 1, the output R of the inverse quantizer 160 = 2, and the output x' of the modulo subtractor 180 = 5 (= 3 + 2), which produces the reconstructed value y = 4; 2) When the sample predictor 110 predicts a value of 2, the residual e' = 2 (= 4 - 2), the quantization index = 1, R = 2, and x' = 4 (= 2 + 2), which produces the reconstructed value y = 3; and 3) When the sample predictor 110 predicts a value of 5, the residual e' = -1 (= 4 - 5), the quantization index = 0, R = 0, and x' = 5 (= 0 + 5), which again produces the reconstructed value y = 4.

[0118] As from Figure 7A As is clearly visible, with the constant c = 0, the reconstruction error for the original sample 0 (e.g., the statistically most important sample value) is (i.e., Figure 7A The possible errors shown in the table are zero. However, the error for the original sample 5 (e.g., a low-priority sample) can reach 2, which is greater than the maximum expected error δ of 1.

[0119] exist Figure 7B In this context, we assume a bit depth of 3, a maximum permissible error δ = 2, and a constant c = 1. Here, when the constant c is greater than zero, the possible reconstruction error for the original sample 0 is no longer guaranteed (i.e., ...). Figure 7B The possible errors in the table shown are zero; however, the maximum error at original sample 0 is still lower than or equal to the maximum error of other original sample values.

[0120] According to some embodiments, when gray levels i>0 are important, the mapping function f(.) of the first mapper 202 is expressed as:

[0121]

[0122] Furthermore, the inverse mapping function g(.) of inverse mapper 204 can be expressed as:

[0123]

[0124] Figure 8 This is a table illustrating numerical examples of potential reconstruction errors for different sample values, given the importance of non-zero sample values, according to some embodiments of the present disclosure.

[0125] exist Figure 8In the example, assume a bit depth of 3, a maximum permissible error δ = 1, and a constant c = 0. In this example, the original sample 4 may be statistically more important to the drive system 20, and the reconstruction error for that sample (i.e., Figure 8 The possible errors shown in the table are zero. However, this comes at the cost of errors greater than the expected maximum error for the original samples 3 and 5.

[0126] exist Figures 7A to 7B as well as Figure 8 In the table, encoder 400 / 400-1 and decoder 500 / 500-1 ensure non-negative reconstruction errors. However, in examples where negative reconstruction errors can be tolerated (e.g., using any suitable modulus adder / subtractor and quantizer / inverse quantizer, and not necessarily those components of encoder 400 / 400-1 and decoder 500 / 500-1) and most importantly, where the original value is zero, the mapping function F(.) and inverse mapping function G(.) that can replace functions f(.) and g(.) can be expressed as:

[0127] F(x) = maxVal - x Equation (14)

[0128]

[0129] In such an example, the mapping function F(.) and the inverse mapping function G(.) can be alternatively represented as:

[0130]

[0131]

[0132] Figure 9A This is a table illustrating numerical examples of potential reconstruction errors for different sample values ​​based on the mapping function and inverse mapping function of equations (9) and (11) according to some embodiments of the present disclosure. Figure 9B This is a table illustrating numerical examples of potential reconstruction errors for different sample values ​​based on the mapping function and inverse mapping function of equations (12) and (13) according to some embodiments of this disclosure. Figures 9A to 9B In the table, it is assumed that the bit depth = 3 and the maximum permissible error δ = 1. Here, from the perspective of the overdrive system 20, the original sample 0 is the most important, and in both tables, the reconstruction error for that sample (i.e., Figures 9A to 9B The possible errors in the table shown are zero.

[0133] Typically, in image / video encoders using related techniques such as DSC and DPCM, no particular importance is given to specific grayscale values. Therefore, the magnitude of the reconstruction error is usually independent of the grayscale value. Furthermore, transform encoding and decoding used in codecs such as HEVC, JPEG-2000, and VDC-M are not well-suited to satisfying non-negative errors when quantization is applied to the transform coefficients in related techniques, and a positive error in the transform domain does not necessarily imply a positive error in the signal space.

[0134] Conversely, according to some embodiments of this disclosure, the image storage and compression system prioritizes certain grayscale values ​​and exhibits lower reconstruction errors for such grayscale values. This can be achieved by performing a pixel-by-pixel mapping of the original image to another space and compressing the image in the mapped space. In addition to maintaining lower errors for important grayscale levels, in some embodiments, the encoder and decoder of the image storage and compression system also ensure non-negative errors for the reconstructed image, which can improve the performance of the corresponding overdriven system.

[0135] The operations performed by the components of encoder 100 / 400 / 400-1 and decoder 300 / 500 / 500-1 can be performed by a "processing circuit" or "processor," which can include any combination of hardware, firmware, and software for processing data or digital signals. The processing circuit hardware can include, for example, application-specific integrated circuits (ASICs), general-purpose or special-purpose central processing units (CPUs), digital signal processors (DSPs), graphics processing units (GPUs), and programmable logic devices such as field-programmable gate arrays (FPGAs). In the processing circuit, as used herein, each function is either performed by hardware configured to perform that function (i.e., hardwired) or by more general-purpose hardware such as a CPU configured to execute instructions stored in a non-transitory storage medium. The processing circuit can be fabricated on a single printed circuit board (PWB) or distributed across several interconnected PWBs. The processing circuit can include other processing circuits; for example, the processing circuit can include two processing circuits, an FPGA and a CPU, interconnected on a PWB.

[0136] It will be understood that while the terms “first,” “second,” “third,” etc., may be used herein to describe various elements, components, regions, layers, and / or portions, these elements, components, regions, layers, and / or portions should not be limited by these terms. These terms are used to distinguish one element, component, region, layer, and / or portion from another element, component, region, layer, and / or portion. Therefore, the first element, first component, first region, first layer, and / or first portion discussed above may be referred to as a second element, second component, second region, second layer, and / or second portion without departing from the scope of the inventive concept.

[0137] The terminology used herein is for the purpose of describing particular embodiments and is not intended to limit the inventive concept. As used herein, the singular form “a” is intended to include the plural form as well, unless the context clearly indicates otherwise. It will be further understood that, when used in this specification, the terms “comprising” and / or “including” indicate the presence of stated features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof. As used herein, the terms “and / or” include any and all combinations of one or more of the associated listed items. Furthermore, in describing embodiments of the inventive concept, the use of “may” means “one or more embodiments of the inventive concept.” Moreover, the term “exemplary” is intended to indicate an example or illustration.

[0138] As used herein, the terms “use” and “being used” may be considered synonymous with the terms “exploitation” and “being exploited,” respectively.

[0139] For the purposes of this disclosure, "at least one of X, Y and Z" and "at least one selected from the group consisting of X, Y and Z" can be interpreted as only X, only Y, only Z, or any combination of two or more of X, Y and Z (such as, for example, XYZ, XYY, YZ and ZZ).

[0140] While this disclosure has been described in detail with specific reference to illustrative embodiments, the embodiments described herein are not intended to be exhaustive or to limit the scope of this disclosure to the exact forms disclosed. Those skilled in the art to which this disclosure pertains will recognize that changes and alterations to the structure and methods of the described components and operations can be practiced without intentionally departing from the principles and scope of this disclosure as set forth in the appended claims and their equivalents.

Claims

1. A method for compressing frames in an image compression and storage system, the method comprising: Modulo addition is applied to the residuals of the original samples of the frame to generate bias residuals based on the bit depth of the original samples and the maximum permissible error; The bias residual is quantized based on the maximum permissible error to generate a quantized bias residual; and Encoding the value corresponding to the quantized bias residual to generate an encoded value with a non-negative reconstruction error, wherein applying the modulo addition method to the residual includes calculating the bias residual as: in, This represents the bias residual. The residual represents the original sample, and bitdepth represents the bit depth of the original sample. The maximum permissible error is represented as, and the bias residual is expressed as Within the range, The original sample corresponds to the color value of the pixel in the frame.

2. The method according to claim 1, further comprising: Receive the original sample; and The predictions of the previously reconstructed samples are subtracted from the original samples to generate the residuals of the original samples.

3. The method according to claim 1, wherein, The value corresponding to the quantized bias residual is the quantization index corresponding to the quantized bias residual.

4. The method according to claim 1, wherein, The bias residual is quantized based on a quantization step size equal to the maximum permissible error plus 1.

5. The method according to claim 1, wherein, Quantizing the bias residual includes calculating the quantization index of the quantized bias residual as follows: in, Indicates the quantization index, Represents the upper limit function. This represents the bias residual, and This indicates the maximum permissible error.

6. The method according to claim 1, wherein, Quantizing the bias residuals includes using a non-uniform quantization band.

7. The method of claim 1, further comprising: The quantized bias residuals are inversely quantized to generate reconstructed bias residuals; The predictions of the previously reconstructed samples are added to the bias residuals of the reconstruction to generate the reconstructed biased samples. Based on the bit depth and the maximum permissible error of the original sample, modulo subtraction and amplitude limiting operations are applied to the reconstructed bias sample to generate a reconstructed sample; and Based on the reconstructed samples, predictions for subsequent reconstructed samples are generated, wherein applying the modulus subtraction method to the reconstructed bias samples includes calculating the reconstructed samples as follows: Where y represents the reconstructed sample, x represents the reconstructed bias sample, and bitdepth represents the bit depth of the original sample. This indicates the maximum permissible error.

8. The method according to claim 7, wherein, The inverse quantization is performed based on a quantization band size equal to the maximum permissible error and further based on the quantization index.

9. The method according to claim 7, wherein, Performing the inverse quantization on the quantized bias residual includes calculating the reconstructed bias residual as follows: in, Let Q[e'] represent the bias residual, Q[e'] represent the reconstructed bias residual associated with the bias residual, and I[e'] represent the quantization index. This indicates the maximum permissible error.

10. The method according to claim 7, wherein, The inverse quantization is performed on the quantized bias residual based on a non-uniform quantization table.

11. The method according to claim 1, wherein, Encoding the value corresponding to the quantized bias residual includes: Entropy encoding is performed on the value corresponding to the quantized bias residual to generate the encoded value.

12. The method of claim 1, further comprising: The original sample is generated by truncating the frame by reducing the bit depth of the color values ​​of the pixels in the frame. The frames may be in red, green and blue format or red, green, blue and green format.

13. A method for decompressing stored image data corresponding to frames in an image compression and storage system, the method comprising: The encoded values ​​corresponding to the original samples of the frame are decoded to generate decoded values ​​corresponding to the quantized bias residuals; The decoded value is inversely quantized to generate the reconstructed bias residual; The predictions of the previously reconstructed samples are added to the reconstructed bias residuals to generate reconstructed biased samples; and Based on the bit depth and maximum permissible error of the original sample, a modulo subtraction method is applied to the reconstructed bias sample to generate a reconstructed sample. Specifically, applying the modulus subtraction method to the reconstructed bias samples includes calculating the reconstructed samples as follows: Where y represents the reconstructed sample, x represents the reconstructed bias sample, and bitdepth represents the bit depth of the original sample. This indicates the maximum permissible error.

14. The method of claim 13, further comprising: Based on the reconstructed samples, predictions for subsequent reconstructed samples are generated.

15. The method according to claim 13, wherein, The decoded value is the quantization index, and Specifically, performing the inverse quantization on the decoded value includes calculating the reconstructed bias residual as follows: Where Q[e'] represents the reconstructed bias residual, and I[e'] represents the quantization index. This represents the bias residual, and This indicates the maximum permissible error.

16. An image compression and storage system, comprising: processor; and The processor's local processor memory, wherein the processor memory has instructions stored in the processor memory, the instructions causing the processor to execute when executed by the processor: Modulo addition is applied to the residuals of the original samples of the frame to generate bias residuals based on the bit depth and maximum permissible error of the original samples; The bias residual is quantized based on the maximum permissible error to generate a quantized bias residual; The value corresponding to the quantized bias residual is encoded to generate an encoded value for storage in a storage medium; Retrieve the encoded value from the storage medium; The encoded value is decoded to generate a decoded value corresponding to the quantized bias residual; The decoded value is inversely quantized to generate the reconstructed bias residual; The predictions of the previously reconstructed samples are added to the reconstructed bias residuals to generate reconstructed biased samples; and Based on the bit depth and the maximum permissible error of the original sample, a modulo subtraction method is applied to the reconstructed bias sample to generate a reconstructed sample. Wherein, the difference between the original sample and the reconstructed sample is only non-negative. Applying the modulus addition method to the residual includes calculating the bias residual as follows: in, This represents the bias residual. The residual represents the original sample, and bitdepth represents the bit depth of the original sample. The maximum permissible error is represented as, and the bias residual is expressed as Within the range, Wherein, the original sample corresponds to the color value of the pixel in the frame, and Specifically, applying the modulus subtraction method to the reconstructed bias samples includes calculating the reconstructed samples as follows: Where y represents the reconstructed sample, x represents the reconstructed bias sample, and bitdepth represents the bit depth of the original sample. This indicates the maximum permissible error.

Images