System and method for performing rate-distortion optimization

Abstract

A system and method for performing rate-distortion optimization includes receiving raw data corresponding to a block of an image frame; and identifying a reconstructed version of the raw data that minimizes a cost function. In response to a highest norm of an error between the raw data and a specific reconstructed version of the raw data being less than or equal to a threshold, the cost associated with the specific reconstructed version of the raw data may be based on a coding rate associated with the specific reconstructed version of the raw data and a distortion between the raw data and the specific reconstructed version of the raw data.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 135,462, filed on January 8, 2021, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present application relates generally to rate-distortion optimization and, more particularly, to systems and methods for compression with a constraint on maximum absolute error. Background Art

[0004] Compression techniques are often used to reduce storage requirements. However, with lossy compression, the quality of the reconstructed data may be prone to errors and inconsistencies, depending on the encoding and decoding techniques and other factors. Therefore, techniques are needed that provide maximum compression while maintaining the best reconstruction quality. Summary of the invention

[0005] According to an embodiment, a method for performing rate-distortion optimization is described. The method may include: receiving raw data corresponding to a block of an image frame; and identifying a reconstructed version of the raw data that minimizes a cost function. In response to a first highest norm of an error between the raw data and a first specific reconstructed version of the raw data being less than or equal to a first threshold, a first cost associated with the first specific reconstructed version of the raw data may be based on a coding rate associated with the first specific reconstructed version of the raw data and a distortion between the raw data and the first specific reconstructed version of the raw data.

[0006] A first cost associated with a first particular reconstructed version of the original data may be the product of the Lagrangian parameter and the coding rate plus the distortion.

[0007] In response to a second highest norm of errors between the original data and the second particular reconstructed version of the original data being greater than the first threshold, a second cost associated with the second particular reconstructed version of the original data may be infinite.

[0008] The method may further include setting the first threshold to a second threshold in response to a second highest norm of errors between the original data and a second particular reconstructed version of the original data being greater than the first threshold.

[0009] The method may further include setting a first threshold by: searching in the search window for a first value that is greater than or equal to a first highest norm of the error; and setting the first threshold to a first value.

[0010] The search window may include multiple values ​​arranged in ascending order.

[0011] The first value may be greater than a lower bound of the search window.

[0012] The lower bound of the search window may correspond to a threshold of a previous block of the image frame.

[0013] A first highest norm of the error between the original data and the first particular reconstructed version of the original data may be less than or equal to a first threshold and reduce a maximum absolute error of the first particular reconstructed version of the original data.

[0014] According to another embodiment, a method for performing rate-distortion optimization is described. The method may include receiving raw data corresponding to a block of an image frame; and identifying a reconstructed version of the raw data that minimizes a cost function. The cost associated with a particular reconstructed version of the raw data may be based on: a distortion between the raw data and the particular reconstructed version of the raw data, a coding rate associated with the particular reconstructed version of the raw data, and a highest norm of an error between the raw data and the particular reconstructed version of the raw data.

[0015] The cost associated with a particular reconstructed version of the original data may be the sum of the distortion, the product of the first Lagrangian parameter and the coding rate, and the product of the highest norm of the error between the original data and the particular reconstructed version of the original data and the second Lagrangian parameter.

[0016] According to another embodiment, an encoder system is described. The system may include: a memory storing computer executable instructions; and a processor configured to execute the instructions and cause the system to perform operations including: receiving raw data corresponding to a block of an image frame; and identifying a reconstructed version of the raw data that minimizes a cost function. In response to a highest norm of an error between the raw data and a specific reconstructed version of the raw data being less than or equal to a threshold, the cost associated with the specific reconstructed version of the raw data may be based on a coding rate associated with the specific reconstructed version of the raw data and a distortion between the raw data and the specific reconstructed version of the raw data.

[0017] The cost associated with a particular reconstructed version of the original data may be the product of the Lagrangian parameter and the coding rate plus the distortion.

[0018] In response to a second highest norm of errors between the original data and the second particular reconstructed version of the original data being greater than a threshold, a second cost associated with the second particular reconstructed version of the original data may be infinite.

[0019] The processor may be further configured to execute the instructions and cause the encoder system to perform operations comprising: in response to a second highest norm of errors between the original data and a second particular reconstructed version of the original data being greater than the threshold, setting the threshold to a second threshold.

[0020] The system may further include a threshold calculator, wherein the processor may be further configured to execute the instructions and cause the threshold calculator to determine the threshold.

[0021] The threshold calculator may be further configured to determine the threshold by: searching in the search window for a first value that is greater than or equal to a highest norm of the error; and setting the threshold to the first value.

[0022] The search window may include multiple values ​​arranged in ascending order.

[0023] The first value is greater than the lower bound of the search window.

[0024] The lower bound of the search window may correspond to a threshold of a previous block of the image frame.

[0025] The scope of the present invention is defined by the claims, which are incorporated by reference into this section. By considering the following detailed description of one or more embodiments, those skilled in the art will more completely understand the embodiments of the present invention and realize their additional advantages. Reference will be made to the accompanying drawings, which will first be briefly described. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a block diagram of an encoder and a decoder according to an embodiment.

[0027] Figure 2 is a block diagram of a mura compensation process according to an embodiment.

[0028] Figure 3 is a block diagram of a mura compensation process with compression according to an embodiment.

[0029] Figure 4 is a block diagram of an encoder according to an embodiment.

[0030] Figure 5 A cost calculation technique performed by an encoder according to a first method according to an embodiment is illustrated.

[0031] Figure 6 A process of determining a threshold according to an embodiment is illustrated.

[0032] Figure 7 is a block diagram of an encoder according to another embodiment.

[0033] Figure 8 A cost calculation technique performed by an encoder according to a second method according to another embodiment is illustrated.

[0034] Fig. 9 is a flowchart of a method for encoding compensation parameters according to an embodiment.

[0035] Fig.10is a flowchart of a method for encoding compensation parameters according to another embodiment.

[0036] The embodiments of the present disclosure and their advantages can be best understood by referring to the detailed description below. Unless otherwise specified, the same reference numerals represent the same elements throughout the drawings and written description, and therefore their description will not be repeated. In the drawings, the relative sizes of elements, layers and regions may be exaggerated for clarity. DETAILED DESCRIPTION

[0037] Hereinafter, example embodiments will be described in more detail with reference to the accompanying drawings. However, the present application may be embodied in various different forms and should not be construed as being limited to the embodiments illustrated herein. On the contrary, these embodiments are provided as examples so that this disclosure will be comprehensive and complete, and will fully convey aspects and features of the present invention to those skilled in the art. Accordingly, processes, elements, and techniques that are not necessary for a person of ordinary skill in the art to fully understand aspects and features of the present invention may not be described.

[0038] Systems and methods are disclosed for reducing the visible effects of compression on reconstructed images. Figure 1 An example block diagram of an encoder 102 and a decoder 104 is illustrated. The encoder 102 receives an image I, encodes the image I, and outputs the encoded image to the decoder 104. The decoder 104 then generates a reconstructed version of the original image based on the encoded image.

[0039] Some optimization techniques may attempt to generate a reconstructed version of the original image that is equal to the original image I using the following formula: The reconstructed version that minimizes the error E between

[0040] Satisfy R≤R b

[0041] in

[0042] Where: E is the original image I and the reconstructed version of the original image The error between b is the bit budget (which may be based on the compression ratio, available bandwidth between the encoder 102 and the decoder 104, storage requirements, or a combination thereof); R is the encoding rate of the encoder 102; D is the original image I and the reconstructed version of the original image and λ(≥0) is the Lagrangian parameter that balances distortion and bit rate.

[0043] Figures 2 to 3A system is illustrated in which compensation parameters are applied to an original image to generate a compensated image. In the illustrated example, the purpose of generating the compensated image is to reduce mura defects that the display device may be prone to (unless correction is performed). However, compensation techniques can be used in the display device to reduce other visual defects (e.g., ghosting, etc.).

[0044] Display devices are often prone to mura defects because, although the transistors and various electronic components that make up the pixels of the display device are manufactured within set manufacturing tolerances according to strictly checked specifications, no two components have exactly the same relationship. For example, the voltage-current relationship of two transistors in a display device may be slightly different, so that in response to a specific input code (e.g., input voltage), one pixel may be turned on with a brightness corresponding to a gray value of 128, while another pixel may be turned on with a brightness corresponding to a gray value of 127, and another pixel may be turned on with a brightness corresponding to a gray value of 130, and so on. In addition, these differences may vary from one display device to another. In other words, no two display devices are exactly the same. Therefore, mura compensation techniques can be implemented to compensate for this visual shortcoming in the display device so that the pixels in each display device will appear more consistent (e.g., always turned on with a brightness corresponding to a gray value of 128).

[0045] In some embodiments, mura defects can be compensated by applying compensation parameters to an image to be displayed. The display device can adjust input values ​​(e.g., input voltage, input code, etc.) supplied to pixels of the display device based on the compensation parameters to achieve reduced mura defects. Because the compensation parameters are static parameters, these parameters can be determined and implemented during the manufacture of the display device (e.g., in a factory).

[0046] Accordingly, Figure 2 A block diagram of an example mura compensation technique is illustrated. According to an embodiment, an image 202 to be displayed on a display device 210 is compensated by applying compensation parameters 204 to the original image at a mura compensation block 206. The compensation parameters 204 are display device dependent, so these parameters are calculated specifically for each display device 210 and stored in a memory 214. When the image 202 is to be displayed on the display device 210, the compensation parameters 204 can be obtained from the memory 214 in real time and applied to the image 202 to compensate for mura defects. Finally, a mura-compensated image 208 can be displayed on the display device 210.

[0047] However, the compensation parameters 204 may be memory intensive and occupy a large amount of memory space. Therefore, in some embodiments, the compensation parameters 204 may be compressed and then stored in memory to reduce the amount of memory required to store the compensation parameters.

[0048] Figure 3 FIG. 4 is a block diagram illustrating an example mura compensation technique in which compensation parameters are compressed according to an embodiment. Figure 2 , by applying compensation parameters 304 at mura compensation block 306 to compensate for mura defects, an image 302 to be displayed on a display device 310 is compensated. Figure 2 Differently, the compensation parameters 304 are compressed by the encoder 312, the compressed compensation parameters are stored in the memory 314, and the compressed compensation parameters stored in the memory 314 are decompressed by the decoder 316 before mura compensation occurs. Finally, the mura-compensated image 308 can be displayed on the display device 310. In some embodiments, the encoder 312 and the decoder 316 can implement a transform-based low complexity codec (TLC), but are not limited thereto, and other encoding / decoding techniques can be implemented.

[0049] Embodiments of the present disclosure will relate to rate-distortion optimization techniques for compensation parameters 304. According to an embodiment, compensation parameters 304 may be represented as data I, and data I may be encoded (e.g., compressed) by encoder 312. Encoding may be performed offline, such as during the manufacture of display device 310 in a factory. Therefore, in some embodiments, encoding is performed only once when display device 310 is manufactured. In other embodiments, encoding may be performed on an existing device, such as during a configuration process initiated by an end user. Once compensation parameters 304 represented by data I are compressed, compressed data I may be stored in memory 314 (e.g., on-board memory in display device 310). During operation, encoder 312 is not utilized because encoded compensation parameters 304 have been stored in memory 314.

[0050] In some embodiments, by encoding and thereby compressing the data I, the size of the encoded compensation parameters can be reduced, thereby reducing the storage requirements or the amount of memory 314 required to store such encoded compensation parameters. In some embodiments, the storage requirements can be reduced by a factor of one quarter to one eighth.

[0051] In some embodiments, the encoded compensation parameters may be retrieved from the memory 314 and decoded (eg, decompressed) by the decoder 316 so that a reconstructed version of the data may be obtained. Provided to the mura compensation block 306. In some embodiments, the decoder 316 can be implemented in hardware such as a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC). However, in other embodiments, the decoder 316 can be implemented by programming software.

[0052] Rate-distortion optimization of compensation parameters corresponding to an image is different from rate-distortion optimization of an image. For example, when an encoded image (e.g., a video image) is decoded, the decoded image (which is a reconstructed version of the original image) is shown to an end user (e.g., on a display device). Therefore, when performing rate-distortion optimization in the case of a reconstructed image, it may be desirable to increase (e.g., maximize) the peak signal-to-noise ratio (PSNR) to reduce or minimize the mean square error, for example, as is typically done in video encoding formats such as MPEG.

[0053] On the other hand, when the encoded compensation parameters are decoded (e.g., to compensate for mura defects), the decoded data (which is a reconstructed version of the original compensation parameter data) is not shown to the end user, because the reconstructed data is only the parameters (e.g., offset) for mura defect compensation. Instead, what the end user sees is the image compensated by the reconstructed compensation parameters. Therefore, when rate-distortion optimization is performed in the case of reconstructed compensation parameters, increasing (e.g., maximizing) PSNR alone may not provide the best results (e.g., the best reconstructed compensation parameters). Therefore, according to an embodiment, in addition to increasing (e.g., maximizing) PSNR, a technique for reducing (e.g., minimizing) the error between the original compensation parameters and the reconstructed version of the original compensation parameters may be implemented to reduce the maximum absolute error. In the following, the compensation parameters may also be referred to as "data".

[0054] In some embodiments, methods are described for optimizing a framework for reducing maximum absolute error without sacrificing (eg, while increasing) PSNR. In some embodiments, methods are determined to reduce the maximum absolute error without increasing the amount of time spent encoding data.

[0055] In some embodiments, systems and methods are described for including the highest norm in rate-distortion optimization to reduce the maximum absolute error (||E|| ∞ ). The disclosed systems and methods identify the reconstructed version of the original data according to the following equation

[0056] Satisfy R≤R b ,||E|| ∞ ≤τ

[0057] where τ is the threshold and ||E||∞ is the original data I and the reconstructed version of the original data The highest norm of the error between .

[0058] In some embodiments, a modified constraint problem with a Lagrangian parameter λ may be used:

[0059] Satisfy||E|| ∞ ≤τwhere Among them, when τ>||E|| ∞ When , the cost D+λR is set to D+λR=∞. Here, τ>||E|| ∞ is the highest norm constraint. In yet another embodiment, the distortion may be calculated based on the sum of absolute differences such that:

[0060] Satisfy||E|| ∞ ≤τwhere D=||E|| 1

[0061] Embodiments of the present disclosure provide two methods for enforcing the highest norm constraint by determining τ.

[0062] Figure 4 yes Figure 3 4. According to an embodiment, the encoder 312 is configured to obtain compensation parameters corresponding to each block of the image 402 and encode the compensation parameters based on a first cost calculation method (e.g., rate-distortion optimization) according to one of several possible modes (e.g., transform mode), which will refer to Figure 5 Describe in more detail.

[0063] In some embodiments, the encoder 312 includes a τ calculator 408 that provides a threshold τ to two or more transform modes 404A-404C and an alternate mode 406. The outputs from the transform modes 404A-404C and the alternate mode 406 are provided to a mode selection block 410 where one of the cost outputs from the transform modes 404A-404C and the alternate mode 406 is selected. In some embodiments, the τ calculator 408 sets a threshold τ applied to the transform modes 404A-404C that will be used to determine whether the cost passes or fails (e.g., whether the constraint enforced by the highest norm of the error is less than or equal to the threshold τ). Thus, when the threshold τ is found to satisfy (i.e., pass) the constraint for one of the modes, the mode selection block 410 selects the mode with the lowest cost (e.g., one of the transform modes 404A-404C). On the other hand, if the threshold τ does not satisfy (i.e., fails) the constraint for the mode, feedback can be provided to the τ calculator 408 to select another τ value (e.g., the next higher value), and the updated τ value is then provided to the transformation modes 404A-404C to determine whether the updated τ value satisfies the constraint for at least one of the modes. The mode that satisfies the constraint can be selected, and in the event that multiple modes satisfy the constraint, the mode with the lowest cost can be selected. In some embodiments, if no τ value (including the updated τ value) satisfies the constraint, the mode selection block 410 can select the backup mode 406.

[0064] In some embodiments, the encoder 312 includes a plurality of transform modes 404A-404C that can perform spatial intra prediction of compensation parameters corresponding to blocks of the image 402. According to one example, a vertical intra prediction mode can be performed. In another example, a horizontal intra prediction mode can be performed. Thus, different predictions of compensation parameters can be generated according to each of the different transform modes 404A-404C. The original compensation parameters can then be subtracted from the predictions of each of the transform modes 404A-404C to generate a residual. After generating the residual, a discrete cosine transform (DCT) such as that used in JPEG or H.264 can be performed. In another embodiment, a discrete sine transform (DST) such as that used in HEVC can be performed. After applying the transform, the current block is quantized using the quantization parameter, and a coding rate representing the quantized coefficients is calculated.

[0065] In some embodiments, Figure 4The two or more transform modes 404A-404C shown in the figure include eight transform modes. Therefore, the current block can be approximated or reconstructed in eight different ways by eight different transform modes 404A-404C, and the transform mode that generates the best output can be selected by the mode selection block 410 based on the distortion between the original block and the approximated / reconstructed block, the rate (e.g., the amount of bits used to represent the quantized transform coefficients), and the threshold τ. In addition, the selected transform mode can vary between blocks of the image 402. Therefore, in one example, transform mode 1 404A can best approximate the first block, while transform mode 2 404B can best approximate the second block, and so on. In some embodiments, the desired residual can be the residual that gives the minimum distortion, the minimum rate, and the minimum τ, or a combination thereof.

[0066] In some embodiments, the alternate mode 406 may have a fixed rate compression and may be used when the transform modes 404A-404C are expensive or the encoder 312 is running out of bits. In the alternate mode 406, the prepaid memory size may be fixed, and thus the compensation parameters may be compressed to fit the fixed memory size. Thus, exceeding the memory capacity by even one bit may result in an inability to decode (e.g., reconstruct) the compensation parameters, thereby completely losing the compensation parameters. Thus, use of the alternate mode 406 is typically reserved for stressful or extreme situations.

[0067] In some embodiments, for the compensation parameters corresponding to each block of the image 402, the mode selection block 410 selects the best transform mode that increases (e.g., maximizes) the PSNR or reduces (e.g., minimizes) the highest norm error. Figure 4 In the example encoder 312 illustrated in FIG. 4 , the mode selection block 410 selects one of the transform modes 404A-404C or the backup mode 406. As described earlier, because the probability of using the backup mode 406 is very small (e.g., approximately 0.001% of the time), and because the backup mode 406 is primarily reserved for stressful or extreme situations, the mode selection block 410 will select the transform mode 404A-404C most of the time (e.g., approximately 99.999% of the time). In addition, a bit budget may be calculated in the mode selection block 410 to determine a fixed rate.

[0068] In some embodiments, the encoder 312 may include a rate control block 412. The target bits and the bit budget may be provided as inputs to the rate control block 412, and depending on how far the encoding rate is from the target bits, the rate control block 412 may calculate a quantization parameter QP. The QP may then be provided to a mode (e.g., transform modes 404A-404C or alternate mode 406). Finally, the compensation parameters may be encoded at the encoding block 414 and then stored in the memory 314.

[0069] In some encoding schemes (such as VESA Display Compensation Motion (VDC-M) as an example), transform modes are used to encode specific types of visual information, for example, natural content is encoded using transform modes, or screen content and synthetic patterns are encoded using block prediction modes. In some embodiments, a backup mode is used when transform modes are expensive or the encoder is running out of bits.

[0070] In some embodiments, the highest norm of the error constraint is evaluated only in conventional modes such as transform mode and block prediction mode. This is because alternate modes are rarely used, and when they are used, they usually have high distortion, and there is at least one alternate mode available for each block.

[0071] Figure 5 The diagram shows Figure 4 312 according to the first method. More specifically, it illustrates the cost calculation performed by the transform modes 404A-404C of the encoder 312. Here, at block 502, the distortion D is calculated using both the original data and the reconstructed data, and compared with a particular reconstructed version of the original data. The associated cost is based on the specific reconstructed version of the original data. The associated coding rate and the original data I and a specific reconstructed version of the original data That is, the cost may be calculated as λ multiplied by the sum of the rate R and the distortion D, as calculated at block 506. λ may be calculated, for example, according to one or more techniques known in the art, such as described in "Ian E. Richardson, 2010, H.264 Advanced Video Compression Standard (Second Edition), Wiley Publishing"

[0072] λ=α*2 ((QP-β) / γ) , for example, α=0.852, β=12, γ=3,

[0073] Or as described in U.S. Patent No. 9,866,853 "System and Method for Lagrangian Parameter Computation for Display Stream Compression (DSC)"

[0074] in

[0075] Here, a, b, c, and d are constants; R is the coding rate of the current block; and maxBits is the maximum number of bits representing the current block. Figure 4The original data is encoded and then decoded using the encoding techniques selected in the transform modes 404A-404C and the alternate mode 406 in the encoder 312 to generate reconstructed data.

[0076] However, according to an embodiment of the present disclosure, an additional constraint is incorporated into the cost that enforces that the highest norm of the error between the original data and a particular reconstructed version of the original data is less than or equal to a threshold value τ. In some embodiments, the highest norm may be determined at the highest norm constraint block 504 by taking the absolute difference of the error and then taking the maximum of the absolute differences. If the highest norm is less than or equal to τ (which may be provided from the τ calculator 408), then the current cost is accepted as D+λR. If the highest norm of the error is greater than τ, then the cost of the selected transform mode is set to infinity, and thus the transform mode is not used to encode the data for that particular value of τ. However, as described above with reference to Figure 4 As explained, if the current value of τ does not satisfy the constraint, feedback can be provided to the τ calculator 408 to obtain a different τ value (e.g., the next higher τ value) and determine whether the τ value will satisfy the constraint. If not, the process can be repeated until a τ value that satisfies the constraint is found. If multiple τ values ​​are determined to satisfy the constraint, the τ value that generates the lowest cost can be selected. On the other hand, if no τ value satisfies the constraint and no more τ values ​​are left for the τ calculator 408 to select, the cost is set to infinity. Therefore, for all transform modes, the current reconstructed version of the data is rejected. Therefore, the first method provides a method of hard forcing the maximum absolute error (e.g., a binary decision of acceptance or rejection) by setting the cost of the encoding method to infinity in response to the maximum absolute error associated with using an encoding method that exceeds the threshold τ. When this occurs, the backup mode 406 can be selected.

[0077] A technique for determining the value of τ will be described. In some embodiments, an exhaustive method may be used to determine τ. For example, the minimum threshold τ in the search window S = {1, 2, 3, ... T} may be determined to satisfy the highest norm of the error constraint of at least one of the transformation modes 404A-404C, where:

[0078] Satisfy R≤R b ,||E|| ∞ ≤τ.

[0079] In other words, the search window S can be set, for example, from 1 to T, where 1 is a predetermined lower bound of the search window S, T is a predetermined upper bound of the search window S, and the values ​​of the search window S are provided in ascending order. The goal is to find the threshold τ of the highest norm that satisfies the error constraint from the search window S. Therefore, the threshold τ can be the minimum value in the search window S (for example, 1 in this case), and can be set for each transformation mode (for example, Figure 4 The transform modes 404A-404C in the example are used to calculate the cost (e.g., rate-distortion optimization). If the constraint ||E|| is not satisfied, ∞ ≤τ, then the next minimum value in S (e.g., 2 in this case) can be selected and the rate-distortion optimization can be calculated. This process can be repeated for each block until a threshold τ that satisfies all constraints in the cost equation is determined for each block. In some embodiments, if each τ value does not satisfy the constraints, an alternate mode 406 that does not depend on the τ value can be selected.

[0080] Figure 6 An example of an image frame divided into multiple blocks (e.g., N×M blocks) is illustrated, and corresponding compensation parameters can be applied to each block of the image to correct mura defects. The encoder 312 may lack a τ calculator that provides a threshold τ to two or more transform modes 604A-604C and a backup mode 606. In some embodiments, each block may include a number of pixels, such as 16 pixels arranged in 2 rows and 8 columns or 16 pixels arranged in 8 rows and 2 columns. In other embodiments, each block may have more or fewer pixels. Therefore, according to an example display device using an RGB arrangement, each of the red block, the green block, and the blue block includes 16 pixels of red sub-pixels, green sub-pixels, and blue sub-pixels. In an example display device using an RGBG arrangement (e.g., In another embodiment of the arrangement), each of the red block and the blue block includes 8 pixels of red sub-pixels and blue sub-pixels, and the green block includes 16 pixels of green sub-pixels.

[0081] In some embodiments, an exhaustive search may be performed for the compensation parameters of each block to determine a threshold τ for rate distortion optimization. According to an embodiment, the threshold τ is set to the minimum value (e.g., 1) in the search window S, and according to the above reference Figure 4The described coding technique in which the best coding solution is selected solves the rate-distortion optimization equation for the first block. Here, the first block is the top left block. If the highest norm of the error is less than a threshold τ (1 in this example) based on the selected coding technique (e.g., one of the transform modes 404A-404C), the first block passes, and the cost of the block is D+λR for the selected mode, and we proceed to the next block. Although the cost may pass for this threshold τ in the selected mode, it may fail at the same threshold in a different mode, in which case the cost may be infinity instead of D+λR. Therefore, the mode that best satisfies the constraints (e.g., also based on a smaller rate, minimum distortion, and minimum τ) can be selected.

[0082] The next block is the block to the right of the first block. The calculation is repeated at this block, and if the highest norm of the error is less than or equal to 1, then this block also passes, and the cost of this block is also D+λR. Accordingly, this process is repeated for all of the blocks in order from left to right and then from left to right in the next row below, and so on, until a block is reached where this calculation fails to satisfy the constraint. That is, the highest norm of the error is greater than a threshold value τ (e.g., 1). Here, in image frame 802, block 808 (see Figure 8 ) has failed. Therefore, the threshold τ is set to the next minimum value in the search window S (e.g., 2 in this case), and the rate-distortion optimization equation is solved again based on an exhaustive method starting from the first block at the upper left. Therefore, the lower bound of the search window value can correspond to the threshold τ of the previous block of the image frame. In some embodiments, the calculations performed for each block include cost calculations for each transform mode in the encoder. Therefore, when the calculation for a block is called failed, all transform modes have failed.

[0083] However, the exhaustive approach may be inefficient by recalculating the rate-distortion optimization equation from scratch. According to a more efficient approach, the rate-distortion optimization equation does not need to be solved for earlier blocks because these blocks have already passed at a threshold value of τ=1. Therefore, we know that these blocks will pass at a larger threshold value (e.g., when τ=2). Therefore, starting at block 808 (which is where the failure occurs when using a smaller threshold value of τ=1), the rate-distortion optimization equation can be solved using a larger threshold value of τ=2.

[0084] In this way, the threshold τ can be determined more efficiently according to an efficient method. Accordingly, at the threshold τ=2, the rate-distortion optimization equation can be solved starting at block 808, and if this block passes, the next block is solved, and so on, until we reach a block where the highest norm of the error is greater than 2 and therefore fails. Here, the threshold of τ=2 is determined at block 810 (see Figure 8) fails, such as shown in image frame 804 (see Figure 8 ). Thus, the threshold τ is set to the next minimum value in the search window S (e.g., 3 in this case), and the rate - distortion optimization equation is solved starting from block 810, and if this block passes, the equation is solved for the next block, and so on until all in the block pass, as shown in image frame 806 (see Figure 8 ). Thus, in this example, the threshold τ can be set to 3.

[0085] In some embodiments, if it is known that smaller search window values will not make all in the block pass, it may be inefficient to start the search by setting τ equal to the minimum search window value (e.g., the lower bound of S, which is τ = 1). Accordingly, in the case where it is known that a smaller threshold τ will fail at some point, a binary search method can be performed, where the initial value of τ can be set to a higher value, e.g., a value somewhere in the middle between the lower bound of the search window value and the upper bound of the search window value (e.g., the mid - point of the search window S). For example, if the search window S = {1, 2, 3, 4, 5,....., 100}, then the first threshold τ can be set to start at 50. If τ = 50 passes, the lower half of the search window S can be divided, for example, by choosing τ = 25 (which is a value between 1 and 50 (i.e., the lower half)). On the other hand, if τ = 50 fails, the upper half of the search window S can be divided, for example, by choosing τ = 75 (which is a value between 50 and 100 (i.e., the upper half)). This process can be repeated until the passing τ value is determined. Accordingly, time can be saved by not having to calculate using each threshold τ in the search window S, thereby improving the overall efficiency.

[0086] Thus, while the exhaustive method is the simplest, it is not the most efficient, and the efficient method and the binary search method are more complex but more efficient. For example, a display device with 1920×1080 pixels has approximately 129,600 blocks (or 2,073,600 pixels, where each block has 2 rows × 8 columns of pixels). The complexity can be analyzed as the cardinality of the search window S multiplied by the number of blocks (e.g., |S|*B = |S|*(N×M)), where S is the cardinality of the search window, B is the number of blocks, N is the number of rows of the block, and M is the number of columns of the block. Thus, the complexity of the efficient method can be expressed as O(B + |S|*B), but since the cardinality of the search window S is much smaller than B (e.g., |S| << B), then O(B + |S|*B)=O(B)=O(N*M). Thus, the complexity of the search can be roughly reduced to only O(N*M). On the other hand, the complexity of the exhaustive method is O(|S|*N*M), and the complexity of the binary search method is O(log 2 (|S|*N*M)).

[0087] Figure 7 is similar to Figure 4 A block diagram of an encoder 312 in FIG. 1 but not including a τ calculator and which can be used to encode compensation parameters according to a second method in which the cost calculation includes rate, distortion and highest norm constraints, which will be referred to as Figure 8 In some embodiments, similar to Figure 4 , the encoder includes two or more transform modes 604A-604C, each of which can be configured to perform spatial intra-frame prediction of compensation parameters corresponding to blocks of the image frame 802. According to one example, a vertical intra-frame prediction mode can be performed. In another example, a horizontal intra-frame prediction mode can be performed. Accordingly, different predictions of compensation parameters can be generated according to each of the different transform modes 604A-604C. The original compensation parameters can then be subtracted from the predictions of each of the transform modes 604A-604C to generate a residual. After generating the residual, DCT or DST can be performed. After applying the transform, the current block can be quantized using the quantization parameters, and a coding rate representing the quantization coefficients can be calculated. Accordingly, different predictions of compensation parameters can be generated according to each of the different transform modes 604A-604C, and a cost can be calculated for each transform mode. The residual block is compared with the reference Figure 4 The blocks described are essentially the same and will not be repeated here.

[0088] Figure 8 The diagram shows Figure 7 Another cost calculation technique performed by the encoder according to the second method is illustrated in FIG. More specifically, it illustrates the cost calculation performed by the encoder's transform modes 604A-604C. Figure 7 The example encoder illustrated in FIG. 6 may include two or more units (e.g., eight units) of the cost calculation transformation mode 604A-604C to perform two or more cost calculations to select the best result. In some embodiments, by using two constraints (λ 1 and λ 2 ) and then provide a second method for determining the cost by converting the constraints to an unconstrained problem using the following equation:

[0089] in

[0090] Here, λ 1 λ may depend on the quantization parameter or it may be calculated using the same technique for calculating λ as described above with reference to the first approach. 2 It can be calculated by trial and error based on experience or analytical error, and thus it can be an estimate. 2may depend on the quantization parameter and may be determined according to an exponential equation, where λ 2 Value and target λ 1 The calculated values ​​are different. In some embodiments, λ 2 can depend on the infinity norm (||E|| ∞ ). Therefore, according to the second method, we can use λ 1 and λ 2 To balance the rate R and the highest norm of the error, to determine the cost. Figure 7 As illustrated in FIG. , a reconstructed version of the original data may be provided to block 702 to calculate the distortion D. Here, the ... Figure 6 The original data is encoded using the encoding technique selected in the transform mode 604A-604C in the encoder and then decoded to generate a reconstructed version of the original data. 1 and λ 2 may be calculated as described above at block 704, and the cost may be estimated in this manner. Thus, since the absolute error is incorporated into the cost, the encoding with the lowest cost may have a reduced absolute error.

[0091] Fig. 9 is a flow chart of a method 900 for encoding compensation parameters according to an embodiment. According to an example embodiment, a method for performing rate distortion optimization is described. The method may include receiving raw data (I) corresponding to a block of an image frame (step 902). In some embodiments, the method may further include identifying a reconstructed version of the raw data that minimizes a cost function. (Step 904). A cost may be associated with a particular reconstructed version of the original data, and in response to a highest norm of errors between the original data and the particular reconstructed version of the original data being less than or equal to a threshold value (τ), the cost may be based on a coding rate (R) associated with the particular reconstructed version of the original data and a distortion (D) between the original data and the particular reconstructed version of the original data. In some embodiments, the method may further set the threshold to a second threshold in response to a second highest norm of errors between the original data and a second particular reconstructed version of the original data being greater than the threshold value (Step 906). In some embodiments, the method may further search for a first value greater than or equal to the highest norm of errors in the search window (S) (Step 908), and then set the threshold to the first value (Step 910).

[0092] Fig.10is a flow chart of a method 1000 for encoding compensation parameters according to another embodiment. According to an example embodiment, another method for performing rate distortion optimization is described. The method may include receiving raw data (I) corresponding to a block of an image frame (step 1002). In some embodiments, the method may further include identifying a reconstructed version of the raw data that minimizes a cost function. (Step 1004). A cost may be associated with a particular reconstructed version of the original data, and the cost may be based on: a distortion (D) between the original data and the particular reconstructed version of the original data, a coding rate (R) associated with the particular reconstructed version of the original data, and a highest norm of the error between the original data and the particular reconstructed version.

[0093] It should be understood that, although the terms "first", "second", "third", etc. may be used herein to describe various elements, components, regions, layers and / or parts, these elements, components, regions, layers and / or parts should not be limited by these terms. These terms are used to distinguish one element, component, region, layer or part from another element, component, region, layer or part. Therefore, the first element, component, region, layer or part described below may be referred to as a second element, component, region, layer or part without departing from the spirit and scope of the present invention.

[0094] For ease of illustration, spatially relative terms such as "below," "below," "down," "under," "above," and "on" may be used herein to describe the relationship of one element or feature relative to another element or feature as shown in the accompanying drawings. It should be understood that the spatially relative terms are intended to include different orientations of the device in use or operation in addition to the orientation depicted in the accompanying drawings. For example, if the device in the accompanying drawings is turned over, the elements described as being "below" or "below" or "below" other elements or features will then be oriented "above" the other elements or features. Therefore, the example terms "below" and "under" can include both above and below. The device can be oriented in other ways (e.g., rotated 90 degrees or in other orientations), and the spatially relative descriptors used herein should be interpreted accordingly.

[0095] It should be understood that when an element or layer is referred to as being "on," "connected to," or "coupled to" another element or layer, it can be directly on, directly connected to, or coupled to the other element or layer, or there can be one or more intervening elements or layers. In addition, it should be understood that when an element or layer is referred to as being "between" two elements or layers, it can be the only element or layer between the two elements or layers, or there can be one or more intervening elements or layers.

[0096] The terms used herein are used to describe the purpose of a particular embodiment, rather than being intended to limit the present invention. As used herein, the singular form "one" is intended to also include the plural form, unless the context clearly indicates otherwise. It will be further understood that when used in this specification, the terms "include" and "comprise" specify the existence of stated features, integral bodies, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, integral bodies, steps, operations, elements, parts and / or combinations thereof. As used herein, the term "and / or" includes any and all combinations of one or more of the listed associated items. Statements such as "at least one of" when located after a column of elements modify the entire column of elements rather than modifying the single element in the column.

[0097] As used herein, the terms "substantially," "approximately," and similar terms are used as terms of approximation rather than terms of degree, and are intended to account for the inherent deviations in measurements or calculations that one of ordinary skill in the art would recognize. In addition, when describing embodiments of the present invention, the use of "may" refers to "one or more embodiments of the present invention." As used herein, the term "use" and variations thereof may be considered synonymous with the term "utilizing" and variations thereof, respectively.

[0098] The electronic or electrical devices and / or any other related devices or components according to the embodiments of the present invention described herein can be implemented using any suitable hardware, firmware (e.g., a dedicated integrated circuit or a field programmable gate array), software, or a combination of software, firmware and / or hardware. For example, the various components of these devices can be formed on an integrated circuit (IC) chip or on a separate IC chip. In addition, the various components of these devices can be processes or threads running on one or more processors in one or more computing devices, executing computer program instructions and interacting with other system components to perform the functions described herein. Computer program instructions are stored in a memory that can be implemented in a computing device using a standard memory device (e.g., taking a random access memory (RAM) as an example). Computer program instructions can also be stored in other non-temporary computer-readable media such as, for example, a CD-ROM, a flash drive, etc. In addition, those skilled in the art should recognize that the functions of various computing devices can be combined or integrated into a single computing device, or the functions of a specific computing device can be distributed on one or more other computing devices without departing from the spirit and scope of the exemplary embodiments of the present invention.

[0099] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art to which the present invention belongs. It will also be understood that, unless expressly defined as such herein, terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and / or this specification, and should not be interpreted in an idealized or overly formal sense.

[0100] The embodiments described herein are merely examples. Those skilled in the art may recognize various alternative embodiments based on those embodiments specifically disclosed. Those alternative embodiments are also intended to fall within the scope of the present disclosure. Therefore, the embodiments are limited only by the appended claims and their equivalents.

Claims

1. A method for performing rate-distortion optimization, the method comprising: receiving raw data corresponding to a block of an image frame; identifying a reconstructed version of the original data that minimizes a cost function; as well as To set the threshold, do the following: searching in a search window for a first value that is greater than or equal to a highest norm of errors between the original data and a specific reconstructed version of the original data; and setting the threshold to the first value, Wherein, in response to the highest norm of the error being less than or equal to a threshold, the cost associated with the specific reconstructed version of the original data is based on: a coding rate associated with the specific reconstructed version of the original data and a distortion between the original data and the specific reconstructed version of the original data.

2. The method according to claim 1, wherein: The cost associated with the particular reconstructed version of the original data is the sum of the product of a Lagrangian parameter and the coding rate and the distortion.

3. The method according to claim 1, wherein: The search window includes a plurality of values ​​arranged in ascending order.

4. The method according to claim 3, wherein: The first value is greater than a lower bound of the search window.

5. The method according to claim 4, wherein: The lower limit of the search window corresponds to a threshold of a previous block of the image frame.

6. The method according to claim 1, wherein: A highest norm of the errors between the original data and the particular reconstructed version of the original data being less than or equal to a threshold reduces the maximum absolute error of the particular reconstructed version of the original data.

7. The method according to claim 1, wherein: The raw data includes compensation parameters for compensating for mura defects in a display device, and the reconstructed version of the raw data includes a reconstructed version of the compensation parameters.

8. A method for performing rate-distortion optimization, the method comprising: receiving raw data corresponding to a block of an image frame; identifying a reconstructed version of the original data that minimizes a cost function, wherein, in response to a highest norm of errors between the original data and the specific reconstructed version of the original data being less than or equal to a threshold, the cost associated with the specific reconstructed version of the original data is based on: a coding rate associated with the specific reconstructed version of the original data and a distortion between the original data and the specific reconstructed version of the original data, and Wherein, in response to a second highest norm of errors between the original data and a second particular reconstructed version of the original data being greater than the threshold, a second cost associated with the second particular reconstructed version of the original data is infinite. 9 . The method of claim 8 , further comprising setting the threshold to a second threshold in response to a second highest norm of the error between the original data and the second particular reconstructed version of the original data being greater than the threshold.

10. The method according to claim 9, further comprising setting the threshold by: searching in a search window for a first value of the highest norm that is greater than or equal to the error; and The threshold is set to the first value.

11. A method for performing rate-distortion optimization, the method comprising: receiving raw data corresponding to a block of an image frame; identifying a reconstructed version of the original data that minimizes a cost function, and Wherein the cost associated with a particular reconstructed version of the original data is the sum of: the distortion between the original data and the particular reconstructed version of the original data, the product of a coding rate associated with the particular reconstructed version of the original data and a first Lagrangian parameter, and the product of a highest norm of the error between the original data and the reconstructed version and a second Lagrangian parameter.

12. An encoder system comprising: a memory storing computer executable instructions; as well as A processor configured to execute the instructions and cause the system to perform operations including: receiving raw data corresponding to a block of an image frame; identifying a reconstructed version of the original data that minimizes a cost function; and To set the threshold, do the following: searching in a search window for a first value that is greater than or equal to a highest norm of errors between the original data and a specific reconstructed version of the original data; and setting the threshold to the first value, Wherein, in response to the highest norm of the error being less than or equal to a threshold, the cost associated with the specific reconstructed version of the original data is based on: a coding rate associated with the specific reconstructed version of the original data and a distortion between the original data and the specific reconstructed version of the original data.

13. The system according to claim 12, wherein: The cost associated with the particular reconstructed version of the original data is the sum of the product of a Lagrangian parameter and the coding rate and the distortion.

14. The system according to claim 12, wherein: The processor is further configured to execute the instructions to initiate determination of the threshold.

15. The system of claim 12, wherein: The search window includes a plurality of values ​​arranged in ascending order.

16. The system of claim 15, wherein: The first value is greater than a lower bound of the search window.

17. The system of claim 15, wherein: The lower limit of the search window corresponds to a threshold of a previous block of the image frame.

18. An encoder system comprising: a memory storing computer executable instructions; as well as A processor configured to execute the instructions and cause the system to perform operations including: receiving raw data corresponding to a block of an image frame; identifying a reconstructed version of the original data that minimizes a cost function, and wherein, in response to a highest norm of errors between the original data and the specific reconstructed version of the original data being less than or equal to a threshold, the cost associated with the specific reconstructed version of the original data is based on: a coding rate associated with the specific reconstructed version of the original data and a distortion between the original data and the specific reconstructed version of the original data, and Wherein, in response to a second highest norm of errors between the original data and a second particular reconstructed version of the original data being greater than the threshold, a second cost associated with the second particular reconstructed version of the original data is infinite.

19. The system of claim 18, wherein: The processor is further configured to execute the instructions and cause the system to perform operations including: in response to a second highest norm of the error between the original data and the second particular reconstructed version of the original data being greater than the threshold, setting the threshold to a second threshold.

20. The system of claim 18, wherein: The processor is further configured to execute the instructions to initiate determination of the threshold by: searching in a search window for a first value of the highest norm that is greater than or equal to the error; and The threshold is set to the first value.

Images