Method of compressing frame in image compression and storage system

Abstract

A frame compression method of an image compression and storage system according to one embodiment of the present invention comprises the steps of: mapping an original sample to a mapped sample based on the bit depth of the original sample and a maximum allowable error so as to assign the lowest reconstruction error to the highest priority sample value; determining a residual of the mapped sample based on a mapped previous reconstructed sample; generating a biased residual by applying modulo addition to the residual; generating a quantized biased residual by quantizing the biased residual based on the maximum allowable error; and generating an encoded value by encoding a value corresponding to the quantized biased residual.

Description

Technology Field

[0001] The present invention relates to an image compression and storage system and a frame compression method.

[0002] The present application claims priority to U.S. Patent Application Nos. 63 / 063,004 and 63 / 062,963, filed with the U.S. Patent and Trademark Office on August 7, 2020, and incorporates the entire contents of the application by reference herein.

[0003] This application is also related to the U.S. patent application titled "COMPRESSION WITH POSITIVE RECONSTRUCTION ERROR" filed in the United States on the same date. Background Technology

[0005] Flat panel displays, such as liquid crystal displays (LCDs) and light emitting diode displays, are becoming common devices because they are small in size, have excellent image quality, and consume little power.

[0006] Pixels in a display panel have a specific response time. The data supplied to and displayed by the pixels can change faster than the pixels react. Because of this, undesirable effects such as motion blur or illusions may occur. Image correction technology is necessary to make the displayed image more satisfactory. Overdrive algorithms have been developed to correct image data, thereby reducing the motion blur exhibited by the display pixels.

[0007] The information described in the background description is intended to enhance understanding of the present invention and may include information that does not correspond to prior art already known to those skilled in the art. The problem to be solved

[0009] The problem that the present invention aims to solve is to provide an image compression, storage, and decompression system and method capable of securing a non-negative bounded reconstruction error. means of solving the problem

[0011] A frame compression method of an image compression and storage system according to one embodiment of the present invention comprises the steps of: mapping an original sample to a mapped sample based on the bit depth of the original sample and a maximum allowable error so as to assign the lowest reconstruction error to the highest priority sample value; determining a residual of the mapped sample based on a mapped previous reconstructed sample; generating a biased residual by applying modulo addition to the residual; generating a quantized biased residual by quantizing the biased residual based on the maximum allowable error; and generating an encoded value by encoding a value corresponding to the quantized biased residual.

[0012] The above priority sample value is 0, and the above mapping step is,

[0013]

[0014] [ f(x) represents the above-mentioned thought sample, and x represents the above original sample, and d represents the above maximum allowable error, and c Is d It is a constant non-negative integer smaller than, and maxVal [represents the maximum value of the original sample] may include a step of calculating the event sample.

[0015] The maximum value of the original sample above is,

[0016]

[0017] ( bit depth is represented as (indicating the bit depth of the original sample), the original sample corresponds to the color value of the pixel of the frame, and the frame may be in an RGB format or an RGBG format.

[0018] The bit depth of the original sample above is 3, and c is 0, and the above maximum allowable error may be 1.

[0019] The above highest priority sample value is greater than 0, and the above mapping step is,

[0020]

[0021] [ f(x) represents the above-mentioned thought sample, and i is the above-mentioned highest priority sample value, and x represents the above original sample, and d represents the above maximum allowable error, and c Is d It is a constant non-negative integer smaller than, and maxVal [represents the maximum value of the original sample] may include a step of calculating the event sample.

[0022] The above residual determination step may include the step of generating the residual of the original sample by subtracting the pre-mapping reconstruction sample from the mapping sample.

[0023] The value corresponding to the above quantization bias residual may be a quantization index corresponding to the above quantization bias residual.

[0024] A frame compression method of an image compression and storage system according to one embodiment of the present invention may further include the steps of: generating a reconstructed biased residue by performing inverse quantization on the quantization biased residue; generating a reconstructed biased sample by adding the pre-mapping reconstructed sample to the reconstructed biased residue; generating a mapped reconstructed sample by applying modulo subtraction to the reconstructed biased sample; and reconstructing the mapped reconstructed sample into a non-mapped reconstructed sample based on the highest priority sample value, the bit depth of the original sample, and the maximum allowable error.

[0025] The above highest priority sample value is 0, and the above historical step is,

[0026]

[0027] [ g(x') represents the above-mentioned non-ideal reconstruction sample, and x' represents the above-mentioned reconstruction sample, and d represents the above maximum allowable error, and c Is d It is a constant non-negative integer smaller than, and maxVal represents the maximum value of the original sample above, and Is The value of is minimum 0, maximum maxVal It may include a step of calculating the above-mentioned non-mapping reconstruction sample as [representing a clipping function that limits to].

[0028] The above highest priority sample value is greater than 0, and the above historical step is,

[0029]

[0030] [ g(x') represents the above-mentioned non-ideal reconstruction sample, and i is the above-mentioned highest priority sample value, and x'represents the above-mentioned reconstruction sample, and d represents the above maximum allowable error, and c Is d It is a constant non-negative integer smaller than, and maxVal represents the maximum value of the original sample above, and Is The value of is minimum 0, maximum maxVal Represents a clipping function that limits to, and Is The value of is minimum 0, maximum maxVal It may include a step of calculating the above-mentioned non-mapping reconstruction sample as [representing a clipping function that limits to].

[0031] A frame compression method of an image compression and storage system according to one embodiment of the present invention may further include the step of generating a prediction of a non-mapped next reconstruction sample based on the non-mapped reconstruction sample, and the step of mapping the prediction of the non-mapped next reconstruction sample to the pre-mapped reconstruction sample.

[0032] A frame compression method of an image compression and storage system according to one embodiment of the present invention may further include the step of generating a prediction of the next reconstruction sample based on the reconstruction sample.

[0033] The reconstruction error for the above-mentioned highest priority sample value may be less than or equal to other values ​​of the above-mentioned original sample.

[0034] The above bias residual generation step may be based on the bit depth of the original sample and the maximum tolerance.

[0035] A decompression method according to one embodiment of the present invention is a method for decompressing stored image data corresponding to a frame in an image compression and storage system, comprising: a step of decoding an encoded value corresponding to an original sample of the frame to generate a decoded value corresponding to a quantized biased residue; a step of generating a reconstructed biased residue by performing inverse quantization on the decoded value; a step of generating a reconstructed biased sample by adding a prediction of a mapped previous reconstructed sample to the reconstructed biased residue; a step of generating a mapped reconstructed sample by applying modulo subtraction to the reconstructed biased sample; and a step of reconstructing the mapped reconstructed sample into an unmapped reconstructed sample based on a highest priority sample value, the bit depth of the original sample, and a maximum allowable error.

[0036] The above highest priority sample value is 0, and the above historical step is,

[0037]

[0038] [ g(x') represents the above-mentioned non-ideal reconstruction sample, and x' represents the above-mentioned reconstruction sample, and d represents the above maximum allowable error, and c Is d It is a constant non-negative integer smaller than, and maxVal represents the maximum value of the original sample above, and Is The value of is minimum 0, maximum maxValIt may include a step of calculating the above-mentioned non-mapping reconstruction sample using a clipping function that limits to (representing).

[0039] The above highest priority sample value is greater than 0, and the above historical step is,

[0040]

[0042] [ g(x') represents the above-mentioned non-ideal reconstruction sample, and i is the above-mentioned highest priority sample value, and x' represents the above-mentioned reconstruction sample, and d represents the above maximum allowable error, and Is d It is a constant non-negative integer smaller than, and maxVal represents the maximum value of the original sample above, and Is The value of is minimum 0, maximum maxVal Represents a clipping function that limits to, and Is The value of is minimum 0, maximum maxVal It may include a step of calculating the above-mentioned non-mapping reconstruction sample using a clipping function that limits to (representing).

[0043] A decompression method according to one embodiment of the present invention may further include the step of generating a prediction of a next-generation reconstruction sample based on the non-generation reconstruction sample.

[0044] An image compression and storage system according to one embodiment of the present invention comprises a processor and a processor memory local to the processor, wherein the processor memory stores an instruction, and the processor executes the instruction to map an original sample to a mapped sample based on the bit depth of the original sample and a maximum allowable error, determines a residue of the mapped sample based on a mapped previous reconstructed sample, applies modulo addition to the residue to generate a biased residue, quantizes the biased residue based on the maximum allowable error to generate a quantized biased residue, encodes a value corresponding to the quantized biased residue to generate an encoded value for storing in a storage medium, decodes the encoded value to generate a decoded value corresponding to the quantized biased residue, and performs inverse quantization on the decoded value A reconstructed biased residue is generated by performing the operation, a reconstructed biased sample is generated by adding the prediction of a previous reconstructed sample to the reconstructed biased residue, a mapped reconstructed sample is generated by applying modulo subtraction to the reconstructed biased sample, and the mapped reconstructed sample is inversely mapped to an unmapped reconstructed sample based on the highest priority sample value, the bit depth of the original sample, and the maximum tolerance.

[0045] With respect to the above-mentioned highest priority sample value, the difference between the above-mentioned original sample and the above-mentioned non-event reconstruction sample may be less than or equal to another value of the above-mentioned original sample. Effects of the invention

[0047] By doing so, a non-negative bounded reconstruction error can be secured in the image compression, storage, and decompression system and method. Brief explanation of the drawing

[0049] FIG. 1 is a schematic diagram of a frame correction system using an image compression and storage system according to one embodiment of the present invention. FIG. 2a is a schematic diagram of an encoder of an image compression and storage system according to one embodiment of the present invention. FIG. 2b is a schematic diagram of a decoder of an image compression and storage system according to one embodiment of the present invention. FIG. 3a is a schematic diagram showing a part of an encoder together with a part of a decoder according to one embodiment of the present invention. FIG. 3b is a simplified drawing of FIG. 3a and shows the identity operation of an entropy encoder and a decoder according to one embodiment of the present invention. FIG. 4a shows a quantization table used by a quantizer according to one embodiment of the present invention. FIG. 4b shows the operation of a quantizer according to one embodiment of the present invention. FIGS. 4c and FIGS. 4d each show uniform and non-uniform quantization tables used by a quantizer according to one embodiment of the present invention. FIG. 5a is a schematic block diagram of an encoder with in-loop mapping according to one embodiment of the present invention. FIG. 5b is a schematic block diagram of an encoder with an offline / outer-loop concept according to one embodiment of the present invention. FIGS. 6a and FIGS. 6b are schematic block diagrams of a decoder with an internal exchange event and a decoder with an offline / external exchange event, respectively, according to one embodiment of the present invention. FIGS. 7A and FIGS. 7B show maximum errors of different values ​​according to one embodiment of the present invention. d and constant c This is a table showing numerical examples of potential reconstruction errors for given different sample values. FIG. 8 is a table showing numerical examples of potential reconstruction errors for different sample values ​​given the importance of non-zero sample values ​​according to one embodiment of the present invention. FIGS. 9a and 9B are tables showing numerical examples of potential reconstruction errors for different sample values ​​based on mapping and history functions of an image compression and storage system in which non-negative reconstruction errors are not guaranteed according to one embodiment of the present invention. Specific details for implementing the invention

[0050] The detailed description to be given below with reference to the attached drawings relates to embodiments of a data compression system and method, and does not represent all forms to be implemented or used by the present invention. Embodiments of the present invention will now be described in detail with reference to the attached drawings. However, functions and structures identical or equivalent to those implemented in different embodiments are also included within the scope of the present invention. Throughout the specification, identical or similar components are denoted by the same reference numerals.

[0051] FIG. 1 is a schematic diagram of a frame correction system (1) using an image compression and storage system (10) according to one embodiment of the present invention.

[0052] According to one embodiment of the present invention, a video compression and storage system (10) includes an encoder (100), a storage medium (200), and a decoder (300). The encoder (100) encodes (see: compresses) data to reduce its size, the storage medium (200) stores the encoded data, and the decoder (300) decodes (see: decompresses) the stored data to reconstruct the data. Storing data, such as 4K video frames, consumes a large amount of memory space. Here, the encoder (100) and the decoder (300) enable the use of a smaller storage medium (200) that can save space and cost. As illustrated in FIG. 1, according to one embodiment of the present invention, an image compression and storage system (10) may form part of a frame correction system (1), and an overdrive system (20) within the frame correction system (1) corrects the current frame using the previous frame [e.g., reduces ghosting effect or blurring]. According to one embodiment of the present invention, a truncation and delay circuit (30) receives a frame [e.g., 8 bits per color (bpc)], truncates the frame (i.e., reduces the bpc number of pixel values ​​of the frame to, for example, 3 bpc), and delays the frame to produce a truncation previous frame. The image compression and storage system (10) then compresses the previous frame and stores it so that the overdrive system (20) can use it later. Here, the truncation and compression may make the size of the stored frame much smaller than the original (previous) frame. For example, in the case of an RGB input frame where each color value is represented by 8 bits [or 24 bits per pixel (bpp)], the frame representation becomes 3 bpc / 9 bpp due to truncation, and if the encoder (100) performs 1.5:1 compression, it can be reduced again to 2 bpc / 6 bpp.However, embodiments of the present invention are not limited to RGB input frames, and any suitable input frame format, such as RGBG [in the case of a pentile sub-pixel array], may be used. For example, in the case of an RGBG input frame, the frame size can be reduced from 16 bpp to 6 bpp through truncation, and the encoder (100) can perform 1.5:1 compression to further reduce the size to 4 bpp. Additionally, embodiments of the present invention are not limited to using the aforementioned truncation operation, and according to one embodiment of the present invention, the truncation and delay circuit (30) in the previously described process may be replaced with a delay block and the truncation operation may be omitted.

[0053] The performance of the overdrive system (20) can be improved only when the reconstruction error, that is, the error introduced by the image compression and storage system (10) (all of which are quantization errors), is not negative. Conventional encryption methods typically optimize the mean square error (MSE) or the peak signal-to-noise ratio (PSNR), which means that the polarity of the error in the reconstructed image can be positive or negative. According to one embodiment of the present invention, the image compression and storage system (10) improves the performance of the overdrive system (20), which is made possible by ensuring that the reconstruction error (i.e., quantization error), defined as the difference between the reconstructed frame and the original frame, is always not negative (i.e., is greater than or equal to 0) and is limited to a set maximum value.

[0054] Throughout the entire specification, "frame" refers to data of an image frame comprising a plurality of pixels. Each pixel of the frame may be represented by a plurality of color values ​​(e.g., red, green, and blue).

[0055] FIG. 2a is a schematic diagram of an encoder (100) of an image compression and storage system (10) according to one embodiment of the present invention. FIG. 2b is a schematic diagram of a decoder (300) of an image compression and storage system (10) according to one embodiment of the present invention.

[0056] Referring to FIG. 2a, according to one embodiment of the present invention, an encoder (100) uses a predictive encryption method that encodes the residual of the original sample rather than the original sample (e.g., truncated frame) itself. Residual (error e A (also called) is defined as the difference between the original sample and a prediction based on one or more adjacent samples (e.g., a prediction of a previous sample). According to one embodiment of the present invention, a sample predictor (110) generates a prediction of a previous sample, and a subtractor (120) calculates a residual value based on the original sample and the predicted sample. The calculated residual may be a positive or negative value. A modulo adder (130) adds a bias value to the residual to reduce the cost of conveying 1-bit code information. A quantizer (140) quantizes the biased residual value to generate a quantized biased value and passes the corresponding quantization index to an entropy encoder (150) to further perform compression (e.g., lossless compression such as entropy encoding). This compressed data [or encoded value] is stored in a storage medium (200). As will be explained further later, according to one embodiment of the present invention, the bias value is a function of the quantization step size.

[0057] According to one embodiment of the present invention, to generate a prediction sample, an inverse quantizer (160) generates a reconstructed biased residue by performing the inverse operation of the operation performed by the quantizer (140) on the quantization biased residue, an adder (170) generates a reconstructed biased sample by adding the prediction of the previous reconstructed sample to the reconstructed biased residue, and a modulo subtractor (180) generates a reconstructed sample by subtracting the value added by the modulo adder (130). According to one embodiment of the present invention, a clipping function is applied to the reconstructed sample so that for a given bit depth, [0, 2 bitDepth It can be made to exist within a desired dynamic range that can be expressed as [-1]. The sample predictor (110) uses this reconstructed sample (e.g., clipped reconstructed sample) to predict the next sample.

[0058] Referring to FIG. 2b, according to one embodiment of the present invention, the decoder (300) includes an entropy decoder (310) that decompresses a stored sample received from a storage medium (200), and an inverse quantizer (160), an adder (170), a modulo subtractor (180), and a sample predictor (110) identical to those described above. Except for the entropy decoder (310), the rest of the decoder (300) is identical to the encoder (100) and is configured in the same manner. Therefore, for brevity, a detailed description of these is omitted.

[0059] Since the decoder (300) does not access the original sample but only the reconstructed sample, the sample predictor (110) of the decoder (300) uses the reconstructed sample to generate a prediction of the previous sample. Additionally, since the operation of the encoder (100) and the decoder (300) is mirrored, the sample predictor (110) of the encoder (100) also uses the same reconstructed sample to generate a prediction of the previous sample, and accordingly, the encoder (100) can be in-sync with the decoder (300) even without accessing the original sample.

[0060] According to one embodiment of the present invention, the encoder (100) and the decoder (300) can operate on a single color value of a pixel. Accordingly, a plurality of encoders (100) and decoders (300) can operate in parallel (e.g., simultaneously) to process multiple colors of a pixel.

[0061] FIG. 3a is a schematic diagram illustrating a part of an encoder together with a part of a decoder according to one embodiment of the present invention. FIG. 3b is a simplified diagram of FIG. 3a, showing the identity operation of an entropy encoder and a decoder according to one embodiment of the present invention.

[0062] According to one embodiment of the present invention, the operations performed by the components of the encoder (100) and the decoder (300) are all lossless operations, except for the quantizer (140). For example, the sequential operation of the entropy encoder (150) and the entropy decoder (310) illustrated in FIG. 3a (as block A) is an identity operation, so it may be omitted as shown in FIG. 3b. Accordingly, the error in the signal space, defined as the difference between the reconstructed sample of the decoder (300) output and the original sample of the encoder (100) input, is equal to the error in the residual space.

[0063] Error (residual) = R - e' = Q [ e' ] - e' (Formula 1)

[0064] Here, e' is the bias residual input to the quantizer (140), and R is the output of the inverse quantizer (160). Q [.] is a function that combines the quantization of the quantizer (140) and the inverse quantization of the inverse quantizer (160).

[0065] Therefore, if the error in the residual space remains non-negative, the error in the signal space will also be non-negative. Furthermore, if the error in the residual space is bounded, i.e., R - e' ≤ d (Here d If is a positive integer representing the maximum allowable error, then the error in the signal space is also d It will be limited to. This relationship is maintained even if a modulo adder (130) and a modulo subtractor (180) exist. In contrast, conventional transform coding [e.g., HEVC (high efficiency video coding), JPEG-2000, or VDC-M (VESA display compression-M)] is not suitable for guaranteeing non-negative reconstruction error, because a positive error in the transformed domain does not guarantee a non-negative reconstruction error in the signal space. This is because transforms such as the discrete cosine transform (DCT) preserve the L2-norm (or energy) between the original and the transform coefficients, but do not preserve the L1-norm.

[0066] Accordingly, according to one embodiment of the present invention, the modulo adder (130) and the quantizer (140) maintain a non-negative error in the residual space and are bounded (i.e., d It is configured to be the same in the signal space by ensuring that it is limited to ),

[0067] The quantizer (140) and the inverse quantizer (160) ensure a non-negative bounded error by appropriately adjusting the quantization band size and the reconstruction point in each band. According to one embodiment of the present invention, the maximum allowable error ( d Regarding ), the quantizer (140) d A uniform or non-uniform quantization band having a maximum quantization step size of +1 is used, and a reconstruction point is used at the right edge (i.e., maximum value) of the band. Additionally, according to one embodiment of the present invention, a value of zero is at the right edge of the band (e.g., the first quantization band of index 1), and a value of zero appears in the reconstruction value. This is because the residual peaks at zero.

[0068] FIG. 4a shows a quantization table used by a quantizer (140) and an inverse quantizer (160) according to one embodiment of the present invention. FIG. 4b shows the operation of a quantizer (140) and an inverse quantizer (160) according to one embodiment of the present invention, and the input is e' and the output is R. FIG. 4c and FIG. 4d each show uniform and non-uniform quantization tables used by a quantizer (140) according to one embodiment of the present invention.

[0069] Referring to FIG. 4a, the quantizer (140) has a quantization step size where the reconstruction point is at the right end of each band. d A uniform band of +1 is used. According to one embodiment of the present invention, the first quantization band has a right end of 0 (i.e., the reconstruction point is 0). Here, the bias residual e' is an integer. According to one embodiment of the present invention, when using the modulo addition and subtraction parameters defined in Equations 6 and 8 [also called optimized modulo addition and subtraction parameters], the number of bands NB can be expressed as follows.

[0070] ) (Equation 2)

[0071] Here d indicates the maximum tolerance, and Ceiling ( . ) represents the ceiling function. Here, d Since +1 is the maximum allowable step size, that is, a smaller step size can be selected. In one embodiment of the present invention, when using addition and subtraction parameters (i.e., parameters not defined by Equations 6 and 8) as an un-optimized module, the number of bands NB can be expressed as follows.

[0072] (Equation 3)

[0073] According to one embodiment of the present invention, when quantizing a bias residual, the quantizer (140) calculates the quantization index of the bias residual as follows.

[0074] (Equation 4)

[0075] Here, when using addition and subtraction parameters as the optimization modules for Equations 6 and 8, In contrast, when using addition and subtraction parameters with a non-optimization module .

[0076] I [ e' ] represents the quantization index, Ceiling ( . ) represents the ceiling function, e' represents the biased residual, and d represents the maximum allowable tolerance.

[0077] Likewise, applying inverse quantization to the quantization bias residuals may involve calculating the reconstruction bias residuals as follows.

[0078] (Equation 5)

[0079] Here Q ( e' ) represents the reconstruction bias residual, I [ e'] represents the quantization index, e' represents the biased residual. Equations 2 and 3 are applied to the embodiment using uniform quantization. When using non-uniform quantization, the maximum step size is d It can be +1.

[0080] Fig. 4b is d = 2, representing an example where the bit depth (i.e., the number of bits per color) is 3. Here, bit depth refers to the color information stored in the image and affects the image's dynamic range. The higher the image's bit depth, the more colors can be stored. The simplest image, a 1-bit image, can display only two colors: black and white. This is because 1 bit can only store one of two values: 0 (white) or 1 (black). As bit depth increases, the dynamic range expands, and consequently, the number of quantization bands also increases.

[0081] As illustrated in the example of Fig. 4b, the biased residual e' of 1 yields a reconstruction value of 3.

[0082] According to one embodiment of the present invention, the modulo adder (130) calculates the biased residual e' as follows.

[0083] (Equation 6)

[0084] This means that the biased residual is within the range expressed as follows.

[0085] (Equation 7)

[0086] Biased residuals e' The upper limit of allows for a non-uniform quantization band that reduces the error for some bias value and thereby improves performance.

[0087] For example, d In the case where = 2 and bit depth = 3, And, the quantizer (140) may use the uniform quantization table shown in FIG. 4c and the non-uniform quantization table shown in FIG. 4d. In FIG. 4c, where the uniform band size is 2, the quantization error for all four bands is d It is limited by (i.e., 2). However, embodiments of the present invention also allow for non-uniform bands of the quantization table illustrated in FIG. 4d, where the third and fourth bands have smaller quantization errors limited by 1 (rather than 2). In one embodiment of the present invention, the quantizer (140) leaves narrower bands for statistically more important bands [e.g., bands where more bias residuals are located, or bands of values ​​that have a greater impact on the performance of the overdrive system (20)]. Because of this, the quantization error is reduced by the operation of the modulo adder, so the performance of the overdrive system (20) can be improved.

[0088] The modulo subtractor (180) can perform the opposite operation of the modulo adder (130) by removing the bias value added to the residual by the modulo adder (130). According to one embodiment of the present invention, input x Regarding, the output of the modulo subtractor (180) y It is expressed as follows.

[0089] (Equation 8)

[0090] As previously mentioned, according to one embodiment of the present invention, a clipping function is applied to a reconstructed sample such that for a given bit depth, [0, 2 bitDepth It can be made to exist within the desired dynamic range represented by [-1]. In the 8 bpc example, the clipping function can limit the output of the subtractor to [0, 7].

[0091] Accordingly, as previously explained, by using an appropriate quantization band size and added bias, the image compression and storage system (10) secures (e.g., guarantees) a non-negative bounded error for the reconstructed image, which can improve the performance of the overdrive system (20).

[0092] While ensuring a non-bounded reconstruction error, according to one embodiment of the present invention, it is desirable that certain gray levels (e.g., a gray value of 0) are 0 or at least smaller than the error of other gray values. For example, for a color depth of 3, the gray values ​​may be in the range of 0 to 7. According to one embodiment of the present invention, overdrive correction may be such that the importance (or priority) increases as the gray value drops from 7 to 0, and 0 may be the most important (or highest priority). In this example, large reconstruction errors may be allowed for gray values ​​with low priority, such as 5, 6, and 7. This contrasts with conventional codecs that treat all gray levels equally and where the reconstruction error is independent of the amplitude (or gray level) of the pixel sample.

[0093] According to one embodiment of the present invention, an image compression and storage system (10) obtains a smaller reconstruction error for important gray levels and a non-negative reconstruction error for other gray levels. According to one embodiment of the present invention, an encoder modifies the original gray value to achieve a lower error for a specific gray level. This is possible by giving a larger error for other unimportant gray levels.

[0094] FIG. 5a is a schematic block diagram of an encoder (400) with an in-loop mapping according to one embodiment of the present invention. FIG. 5b is a schematic block diagram of an encoder (400-1) with an offline / outer-loop mapping according to one embodiment of the present invention. FIG. 6a and FIG. 6b are schematic block diagrams of a decoder (500) with an in-loop mapping and a decoder (500-1) with an offline / outer-loop mapping, respectively, according to one embodiment of the present invention.

[0095] According to one embodiment of the present invention, the encoder (400 / 400-1) has a gray value of the original sample prior to encoding. x mapped value (also called mapped sample) f It includes a mapper (e.g., gray value mapping circuit) (202) that maps to (.), and the inverse operation of the mapper (202). g It further includes an inverse mapper (e.g., gray value inverse circuit) (204) that performs (.). Thus, the encoder (400 / 400-1) encodes the original sample in the mapped space rather than in the signal space. Here, the dynamic range of the mapped space is the same as that of the original space. In other words, the bit depth of the mapped space and the original space are the same.

[0096] According to one embodiment of the present invention, in the case of the inner ring method illustrated in FIG. 5a, the encoder (400) includes a first mapper (202) that operates on the original sample in front of the adder (170), and a second mapper (203) that operates on the sample predictor (110), and further includes a reverser (204) that operates on the output of the modulo subtractor (180). In this embodiment, the reverser (204) performs a reverse operation on the mapped reconstructed sample generated by the modulo subtractor (180) to generate an unmapped reconstructed sample, and the sample predictor (110) uses this to generate a prediction of the unmapped reconstructed sample. The second mapper (203) maps this value to a mapped previous reconstructed sample and sends it to a subtractor (120), and the subtractor (120) subtracts the mapped previous reconstructed sample from the mapped sample.

[0097] According to one embodiment of the present invention, in the case of the offline / foreign exchange method illustrated in FIG. 5b, the history machine (204) does not operate on the input of the sample predictor (110), and accordingly, a second mapping machine (203) that operates on the output of the sample predictor (110) is not required. In this embodiment, the mapping operation is performed entirely outside the encoder loop. This foreign exchange method is easy to use with existing codecs because the operation of the mapping machine (202) and the history machine (204) takes place outside the codec. Additionally, the foreign exchange method is less complex than the internal method because the number of operations / calculations performed is small. However, using the internal method can improve performance.

[0098] As illustrated in FIGS. 6a and 6b, the internal event method decoder (500) uses both an event method (203) and a reverse method (204), while the foreign event method decoder (500-1) uses a single reverse method (204) (even if an event method is not required).

[0099] In both modes, except for the mapping unit (202, 203) and the reverse unit (204), the encoder (400 / 400-1) contains the same components having the same configuration as the encoder (100). Likewise, in both inline and offline modes, except for the mapping unit (203) and the reverse unit (204), the decoder (500 / 500-1) contains the same components having the same configuration as the decoder (300). Therefore, for the sake of brevity, the components of the encoder (400 / 400-1) and the decoder (500 / 500-1) that are common to the encoder (100) and the decoder (300) are not described repeatedly. The modulo adder (130), quantizer (140), inverse quantizer (160), and modulo subtractor (180) ensure a non-negative bounded reconstruction error, but the encoder (400, 400-1) and decoder (500, 500-1) are not limited thereto. For example, any suitable modulo adder, quantizer, inverse quantizer, and modulo subtractor may be used as long as [e.g., the overdrive system (20)] can withstand a negative reconstruction error.

[0100] According to one embodiment of the present invention, when the gray level of 0 is important (e.g., when a reconstruction error of 0 is desired), the mapping function of the mapping device (202) f (.) is expressed as follows.

[0101] (수식 9)

[0102] Here d represents the maximum positive error between the reconstructed sample and the original sample, and c Is d Smaller than (i.e., ) is a constant non-negative integer, and maxVal For a given bit depth (or bits per color), the mapper input sample (e.g., original sample) It is the maximum value of and can be expressed as follows.

[0103] (Equation 10)

[0104] Also, the historical function of the historical period (204) g (.) can be expressed as follows.

[0105] (Equation 11)

[0106] Here x represents an input to the historical period (204) [e.g., a historical reconstruction sample output by the modulo subtractor (180)], and Is Output value minimum value 0 and maximum value maxVal It is a clipping function that restricts to. In other words, the clipping function is for all less than 0 Calculate 0 for the value, and maxVal All greater Regarding the value maxVal Calculate, and in other cases Calculates.

[0107] As I will explain later, constant c When = 0, there is no reconstruction error from the original sample value of 0, which is the maximum error d This is possible by making the error larger for other original sample values ​​that exceed it.

[0108] FIGS. 7A and FIGS. 7B show maximum errors of different values ​​according to one embodiment of the present invention. d and constant c This is a table showing numerical examples of potential reconstruction errors for given different sample values. In the tables illustrated in Figs. 7a and 7b, the values x'...represents the possible output of the modulo subtractor (180) in an inline manner of the encoder (400) (of FIG. 5a), and y represents a reconstructed sample from the output of the historical period (204).

[0109] In Fig. 7a, the bit depth is 3 (i.e., the original sample value is in the range of 0 to 7), and the maximum error d = 1 and, constant c Assume = 0. In this table, each value x Value for x' It can have one or more possibilities depending on the predicted reconstructed sample. For example, x =3 and f ( x When )=4, the number of possible scenarios is as follows: 1) If the sample predictor (110) predicts a value of 3, the residual e' is 1 (= 4-3), quantization index = 1, output (R) of the inverse quantizer (160) = 2, output of the modulo subtractor (180) x' ) = 5 (=3+2), this is the reconstruction value y = 4 is calculated. 2) If the sample predictor (110) predicts a value of 2, the residual e' = 2 (= 4-2), quantization index = 1, R = 2, x' = 4 (= 2+2), this is the reconstruction value y = 3 is calculated. 3) If the sample predictor (110) predicts a value of 5, the residual e' = -1 (= 4-5), quantization index = 0, R = 0, x' = 5 (= 0+5), this is the reconstructed value y Produces = 4.

[0110] As is evident from Fig. 7a, the constant c= 0, the reconstruction error for the original sample 0 (Example: statistically, the most important sample value) is 0. However, the error for the original sample 5 (Example: a sample with low priority) is the maximum desired error d It can reach 2, which is greater than 1.

[0111] In Fig. 7b, the bit depth is 3, and the maximum error d = 2 and, constant c Assume = 1. Here, the constant c is greater than 0, and while it cannot be guaranteed that the possible reconstruction error for the original sample 0 is 0, the maximum error for the original sample 0 is less than or equal to the maximum error for other original sample values.

[0112] According to one embodiment of the present invention, gray level i If > 0 is important, the mapping function of the mapping device (202) f (.) is expressed as follows.

[0113] (Equation 12)

[0114] Historical function of the historical period (204) g (.) can be expressed as follows.

[0115] (Equation 13)

[0116] FIG. 8 is a table showing numerical examples of potential reconstruction errors for different sample values ​​given the importance of non-zero sample values ​​according to one embodiment of the present invention.

[0117] In the example of Fig. 8, the bit depth is 3, and the maximum error d = 1 and, constant c It is assumed that = 0. In this embodiment, the original sample 4 was statistically more significant for the overdrive system (20), and the reconstruction error for this sample is 0. However, this was made possible by making the maximum error for the original samples 3 and 5 larger than the desired maximum error.

[0118] In the table of FIGS. 7a, 7b, and FIG. 8, the encoder (400 / 400-1) and decoder (500 / 500-1) guarantee a non-negative reconstruction error. However, in an example where a negative reconstruction error can be tolerated [e.g., an example using appropriate modulo adder / subtractor and quantizer / inverse quantizer, but not necessarily those of the encoder (400 / 400-1) and decoder (500 / 500-1)] and the most important original value is 0, the function f (.) and g mapping function replacing (.) F (.) and historical functions G (.) can be expressed as follows.

[0119] (Equation 14)

[0120] (Equation 15)

[0121] In this example, the mapping function F (.) and historical functions G (.) can also be expressed as follows instead.

[0122] (Equation 16)

[0123] (Equation 17)

[0124] FIG. 9a is a table showing numerical examples of potential reconstruction errors for different sample values ​​based on the mapping and historical functions of Equations 9-10 according to one embodiment of the present invention. FIG. 9b is a table showing numerical examples of potential reconstruction errors for different sample values ​​based on the mapping and historical functions of Equations 11-12 according to one embodiment of the present invention. In the tables of FIG. 9a and FIG. 9b, the bit depth is 3, and the maximum error d It is assumed that = 1. Here, the original sample 0 is the most important from the perspective of the overdrive system (20), and the reconstruction error for this sample is 0 in both tables.

[0125] In general, conventional image / video ciphers such as DSC and DPCM do not attach special significance to specific gray values. Therefore, the magnitude of the reconstruction error is typically independent of the gray value. Furthermore, transform encryption used in codecs such as HEVC, JPEG-2000, and VDC-M is not suitable for satisfying non-negative errors, because in conventional technology, quantization is applied to transform coefficients, and a positive error in the transformed domain does not mean that a positive error in the signal space is also a positive error.

[0126] In contrast, according to an embodiment of the present invention, an image storage and compression system gives high priority to specific gray values ​​and exhibits low reconstruction error for such gray values. This can be achieved by mapping the original image to another space pixel-wise and compressing the image in the mapped space. In addition to maintaining low error for important gray levels, according to one embodiment of the present invention, the encoder and decoder of the image storage and compression system ensure non-negative error for the reconstructed image, and accordingly, the performance of the corresponding overdrive system can be improved.

[0127] The operations or operations performed by the components of the encoder (100 / 400 / 400-1) and decoder (300 / 500 / 500-1) may be implemented as a "processing circuit," "processor," or "processor." The "processing circuit" may be implemented using hardware, firmware, software, or a combination thereof. The processing circuit may include, for example, an application-specific integrated circuit (ASIC), a general-purpose or dedicated central processing unit (CPU), a digital signal processor (DSP), a graphics processing unit (GPU), an FPGA, or other programmable logic device. Each function in the processing circuit may be performed by wired hardware that performs the function or by general-purpose hardware, such as a CPU, that executes instructions stored on a non-transitory storage medium. The processing circuit may be fabricated on a single printed wiring board (PWB) or distributed across interconnected PWBs. The processing circuit may include other processing circuits, for example, an FPGA and a CPU interconnected on the PWB.

[0128] Terms such as "first," "second," and "third" are used for various elements, components, regions, layers, parts, etc., but they are not limited by such modifiers. These terms are used to distinguish one element, component, region, layer, or part from another and do not deviate from the spirit and scope of the present invention.

[0129] The terms used herein are for the purpose of describing specific embodiments only and are not intended to limit the invention. Unless otherwise specified, the singular or plural case is included. The expression “comprising” any feature, step, action, part, component, etc. means that other features, steps, actions, parts, components, etc., may be included in addition to said part. The expression “and / or” includes any combination of one or more of the listed items. Furthermore, the expression “may” used when describing embodiments of the invention means that it is applicable to “one or more embodiments of the invention.” The term “exemplary” indicates an example or drawing.

[0130] Words such as "use" and "utilization" can be used with similar meanings along with other similar expressions.

[0131] The expressions “at least one of X, Y, and Z” and “at least one selected from the group including X, Y, and Z” represent X only, Y only, Z only, or two or more combinations of X, Y, and Z, e.g., XYZ, XYY, YZ, ZZ, etc.

[0132] Although specific embodiments have been described in detail above, the embodiments described herein are not intended to limit the scope of the invention as described. Those skilled in the art will understand that the structure and method of the combination and operation described above can be changed or modified without significantly departing from the principles and scope of the invention expressed in the following claims and their equivalents. Explanation of the symbols

[0134] 1: Frame Correction System 10: Video Compression and Storage Systems 20: Overdrive System 30: Cut-off and delay circuits 100, 400, 400-1: Encoder 110: Sample Predictor 120: Subtractor 130: Modulo adder 140: Quantizer 150: Entropy encoder 160: Inverse quantum magnetizer 170: Adder 180: Modulo subtractor 200: Storage media 203, 203: Period of Thought 204: Historical Records 300, 500, 500-1: Decoder 310: Entropy decoder

Claims

Claim 1 A frame compression method for an image compression and storage system, comprising the steps of: mapping an original sample to a mapped sample based on the bit depth of the original sample and a maximum allowable error so as to assign the lowest reconstruction error to the highest priority sample value; determining a residue of the mapped sample based on a mapped previous reconstructed sample; generating a biased residue by applying modulo addition to the residue; generating a quantized biased residue by quantizing the biased residue based on the maximum allowable error; and generating an encoded value by encoding a value corresponding to the quantized biased residue, wherein the reconstruction error of the highest priority sample is less than or equal to the value of the original sample. Claim 2 In paragraph 1, the highest priority sample value is 0, and the mapping step is, [ f(x) represents the above-mentioned thought sample, and x represents the above original sample, and δ represents the above maximum allowable error, and c Is δ It is a constant non-negative integer smaller than, and maxVal A frame compression method for an image compression and storage system comprising the step of calculating the mapping sample as [representing the maximum value of the original sample]. Claim 3 In paragraph 2, the maximum value of the original sample is, ( bitdepth A frame compression method of an image compression and storage system, wherein the original sample is represented as (indicating the bit depth of the original sample), the original sample corresponds to the color value of a pixel of the frame, and the frame is in an RGB format or an RGBG format. Claim 4 In paragraph 2, the bit depth of the original sample is 3, and c A frame compression method for an image compression and storage system in which is 0 and the maximum allowable error is 1. Claim 5 In paragraph 1, the highest priority sample value is greater than 0, and the mapping step is [ f(x) represents the above-mentioned thought sample, and i is the above-mentioned highest priority sample value, and x represents the above original sample, and δ represents the above maximum allowable error, and c Is δ It is a constant non-negative integer smaller than, and maxVal A frame compression method for an image compression and storage system comprising the step of calculating the mapping sample as [representing the maximum value of the original sample]. Claim 6 A frame compression method of an image compression and storage system according to claim 1, wherein the residual determination step comprises the step of generating the residual of the original sample by subtracting the pre-mapping reconstruction sample from the mapping sample. Claim 7 A frame compression method of an image compression and storage system, wherein, in claim 1, the value corresponding to the quantization bias residual is a quantization index corresponding to the quantization bias residual. Claim 8 A frame compression method for an image compression and storage system according to claim 1, further comprising the steps of: generating a reconstructed biased residue by performing inverse quantization on the quantized biased residue; generating a reconstructed biased sample by adding the pre-mapping reconstructed sample to the reconstructed biased residue; generating a mapped reconstructed sample by applying modulo subtraction to the reconstructed biased sample; and reconstructing the mapped reconstructed sample into a non-mapped reconstructed sample based on the highest priority sample value, the bit depth of the original sample, and the maximum tolerance. Claim 9 In paragraph 8, the above highest priority sample value is 0, and the above historical step is, [ g(x') represents the above-mentioned non-ideal reconstruction sample, and x' represents the above-mentioned reconstruction sample, and δ represents the above maximum allowable error, and c Is δ It is a constant non-negative integer smaller than, and maxVal represents the maximum value of the original sample above, and Is The value of is minimum 0, maximum maxVal A frame compression method of an image compression and storage system comprising the step of calculating the above-mentioned non-mapping reconstruction sample as [representing a clipping function limited to]. Claim 10 In paragraph 8, the above highest priority sample value is greater than 0, and the above historical step is, [ g(x') represents the above-mentioned non-ideal reconstruction sample, and i is the above-mentioned highest priority sample value, and x' represents the above-mentioned reconstruction sample, and δ represents the above maximum allowable error, and c Is δ It is a constant non-negative integer smaller than, and maxVal represents the maximum value of the original sample above, and Is The value of is minimum 0, maximum maxVal Represents a clipping function that limits to, and Is The value of is minimum 0, maximum maxVal A frame compression method of an image compression and storage system comprising the step of calculating the above-mentioned non-mapping reconstruction sample as [representing a clipping function limited to].

Images