Method and apparatus for rendering color images

Abstract

To provide a method and apparatus for rendering color images.SOLUTION: The invention provides methods for driving an electro-optic display. A method for driving an electro-optic display having a plurality of display pixels comprises receiving an input image, processing the input image to create color separation cumulate, and using a threshold array to process the color separation cumulate to generate colors for the electro-optic display. The method also comprises dithering the input image by intersecting the color separation cumulate with a dither function.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] (Reference to Related Application) This application is related to and claims priority to U.S. Provisional Application No. 63 / 108,855, filed November 2, 2020.

[0002] The entire disclosure of the aforementioned application is incorporated herein by reference.

[0003] (Object of the invention) The present invention relates to a method for driving an electro-optic display, and more particularly to a driving method for dithering and rendering images on an electrophoretic display.

[0004] (background) The present invention relates to a method and apparatus for rendering color images. More particularly, the present invention relates to a method for multi-color dithering in which combinations of color intensities are converted into multi-color surface coverages. [Background technology]

[0005] The term "pixel" is used herein in its conventional sense in the display art to mean the smallest unit of a display that is capable of producing all the colors that the display itself can show.

[0006] Halftoning has been used for decades in the printing industry to represent shades of gray by covering varying percentages of each pixel on white paper with black ink. Similar halftoning schemes can be used with CMY or CMYK color printing systems, with the color channels being varied independently of each other.

[0007] However, there are many color systems in which the color channels cannot be varied independently of each other because each pixel can display any one of a limited set of primary colors (such systems may hereafter be referred to as "limited palette displays" or "LPDs"). ECD patent color displays are of this type. To create other colors, the primary colors must be spatially dithered to produce the correct color sensation.

[0008] An electronic display typically includes an active matrix backplane, a master controller, local memory, and a set of communication and interface ports. The master controller receives data via the communication / interface ports or reads it from the device memory. Once the data enters the master controller, it is converted into a set of instructions for the active matrix backplane. The active matrix backplane receives these instructions from the master controller and produces an image. For color devices, on-device color gamut calculation may require a master controller with increased computing power. As noted above, rendering methods for color electrophoretic displays are often computationally intensive, and although the present invention itself provides methods for reducing the computational load imposed by rendering, as discussed in detail below, the rendering (dithering) step and other steps of the overall rendering process may still impose a significant load on the device's computing processing system.

[0009] The increased computing power required for image rendering diminishes the advantages of electrophoretic displays in some applications. In particular, the cost of manufacturing the device increases when a master controller is configured to implement complex rendering algorithms, as does device power consumption. Furthermore, excessive heat generated by the controller requires thermal management. Therefore, it may be desirable to have an efficient method for dithering multicolor images in at least some cases, such as when ultra-high resolution images or multiple images need to be rendered in a short period of time. Summary of the Invention [Means for solving the problem]

[0010] (Summary of the Invention) Thus, in one aspect, the subject matter presented herein provides a method for driving an electro-optic display, the method may include receiving an input image, processing the input image to create a color separation accumulation, and dithering the input image by intersecting the color separation accumulation with a dither function.

[0011] In some embodiments, the dither function is a threshold array.

[0012] In another embodiment, the threshold array is a blue noise mask (BNM).

[0013] In yet another embodiment, the processing step is implemented by a look-up table. The present invention provides, for example, the following. (Item 1) 1. A method for driving an electro-optic display having a plurality of display pixels, the method comprising: receiving an input image; processing the input image to produce a color separation accumulation; dithering the input image by intersecting the color separation accumulation with a dither function; A method comprising: (Item 2) Item 10. The method of item 1, wherein the dither function is a threshold array. (Item 3) 3. The method of claim 2, wherein the threshold array is a blue noise mask (BNM). (Item 4) Item 1. The method according to item 1, wherein the step of processing the input image is implemented by a lookup table. (Item 5) 4. The method of claim 3, wherein the lookup table includes a mapping between color values ​​of the input image and the color separation accumulation. (Item 6) Item 10. The method of item 1, further comprising passing the input image through a sharpening filter before processing the input image. (Item 7) Item 6. The method of item 5, wherein the sharpening filter is a finite impulse response (FIR) filter. (Item 8) Item 10. The method of item 1, wherein the step of processing the input image and creating a color separation accumulation includes using a barycentric coordinate method. (Item 9) Item 1. An electro-optic display configured to perform the method of item 1, including an electrophoretic display. [Brief explanation of the drawings]

[0014] The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.

[0015] [Figure 1] FIG. 1 of the accompanying drawings is an image rendering model in accordance with the subject matter presented herein.

[0016] [Figure 2]FIG. 2 is an exemplary black and white dithering method using a mask in accordance with the subject matter presented herein.

[0017] [Figure 3] FIG. 3 illustrates various mask designs in accordance with the subject matter presented herein.

[0018] [Figure 4] FIG. 4 illustrates gamut color mapping in accordance with the subject matter disclosed herein.

[0019] [Figure 5] FIG. 5 illustrates a multi-color dithering method using a mask in accordance with the subject matter disclosed herein.

[0020] [Figure 6] FIG. 6 illustrates a multi-color dithering algorithm using a mask in accordance with the subject matter disclosed herein.

[0021] [Figure 7] 7-10 are various mask designs for multi-color dithering in accordance with the subject matter presented herein. [Figure 8] 7-10 are various mask designs for multi-color dithering in accordance with the subject matter presented herein. [Figure 9] 7-10 are various mask designs for multi-color dithering in accordance with the subject matter presented herein. [Figure 10] 7-10 are various mask designs for multi-color dithering in accordance with the subject matter presented herein. DETAILED DESCRIPTION OF THE INVENTION

[0022] (Detailed explanation) Standard dithering algorithms, such as error diffusion algorithms (where the "error" introduced by printing one pixel with a particular color different from the color theoretically desired at that pixel is distributed among neighboring pixels so as to produce an overall correct color sensation), can be employed with limited palette displays. An extensive literature exists on error diffusion. For a review, see Pappas, Thrasyvoulos N. "Model-based halftoning of color images" IEEE Transactions on Image Processing See 6.7 (1997): 1014-1024.

[0023] This application is a continuation of U.S. Patent Nos. 5,930,026, 6,445,489, 6,504,524, 6,512,354, 6,531,997, 6,753,999, 6,825,970, 6,900,851, 6,995,550, 7,012,600, 7,023,420, 7,034,783, 7,061,166, 7,061,662, 7,116,466, 7,119,772, 7,177,066, 7,193,625, 7,202,847, 7,242,514, No. 7,259,744, No. 7,304,787, No. 7,312,794, No. 7,327,511, No. 7,408,699, No. 7, No. 453,445, No. 7,492,339, No. 7,528,822, No. 7,545,358, No. 7,583,251, No. 7,602 ,374, No. 7,612,760, No. 7,679,599, No. 7,679,813, No. 7,683,606, No. 7,688,2 No. 97, No. 7,729,039, No. 7,733,311, No. 7,733,335, No. 7,787,169, No. 7,859,742 , No. 7,952,557, No. 7,956,841, No. 7,982,479, No. 7,999,787, No. 8,077,141, No. No. 8,125,501, No. 8,139,050, No. 8,174,490, No. 8,243,013, No. 8,274,472, No. 8,2 No. 89,250, No. 8,300,006, No. 8,305,341, No. 8,314,784, No. 8,373,649, No. 8,384 ,658, No. 8,456,414, No. 8,462,102, No. 8,514,168, No. 8,537,105, No. 8,558,78 No. 3, No. 8,558,785, No. 8,558,786, No. 8,558,855, No. 8,576,164, No. 8,576,259 , No. 8,593,396, No. 8,605,032, No. 8,643,595, No. 8,665,206, No. 8,681,191, No. 8 ,730,153, No.8,810,525, No.8,928,562, No.8,928,641, No.8,976,444, No.9,0 No. 13,394, No. 9,019,197, No. 9,019,198, No. 9,019,318, No. 9,082,352, No. 9,171,Nos. 508, 9,218,773, 9,224,338, 9,224,342, 9,224,344, 9,230,492, 9,251,736, 9,262,973, 9,269,311, 9,299,294, 9,373,289, 9,390,066, 9,390,661, and 9,412,314, and U.S. Patent Application Publication Nos. 2003 / 0102858, 2004 / 0246562, 2005 / 0253777, 2007 / 0091418, 2007 / 0103427, 2007 / 0176912, 2008 / 0024429, 2008 / 0024482, 2008 / 0136774, 2008 / 0291129, 2008 / 0303780, 2009 / 0174651, 2009 / 0195568, No. 2009 / 0322721, No. 2010 / 0194733, No. 2010 / 0194789, No. 2010 / 0220121, No. 2010 / 0265561, No. 2010 / 0283804, No. 2011 / 0063314, No. 2011 / 0175875, 2011 / 0193840, 2011 / 0193841, 2011 / 0199671, 2011 / 0221740, 2012 / 0001957, 2012 / 0098740, 20 No. 13 / 0063333, No. 2013 / 0194250, No. 2013 / 0249782, No. 2013 / 0321278, No. 2014 / 0009817, No. 2014 / 0085355, No. 2014 / 0204012, No. 201 No. 4 / 0218277, No. 2014 / 0240210, No. 2014 / 0240373, No. 2014 / 0253425, No. 2014 / 0292830, No. 2014 / 0293398, No. 2014 / 0333685, No. 2014 / Also related to Nos. 0340734, 2015 / 0070744, 2015 / 0097877, 2015 / 0109283, 2015 / 0213749, 2015 / 0213765, 2015 / 0221257, 2015 / 0262255, 2015 / 0262551, 2016 / 0071465, 2016 / 0078820, 2016 / 0093253, 2016 / 0140910, and 2016 / 0180777. These patents and applications may hereinafter, for convenience, be collectively referred to as the "MEDEOD" (Method for Driving an Electro-Optic Display) Application, and are incorporated herein by reference in their entirety.

[0024] ECD systems exhibit certain peculiarities that must be taken into account when designing dithering algorithms for use in such systems. Inter-pixel artifacts are a common feature in such systems. One type of artifact is caused by so-called "blooming." In both monochromatic and colorimetric systems, the electric field generated by a pixel electrode tends to affect an area of ​​the electro-optic medium larger than the area of ​​the pixel electrode itself, effectively spreading the optical state of one pixel into portions of the area of ​​adjacent pixels. Another type of crosstalk occurs when driving adjacent pixels results in a final optical state in the area between the pixels that is different from the area reached by either of the pixels themselves; this final optical state is caused by the average electric field experienced in the inter-pixel region. A similar effect occurs in monochromatic systems, but because such systems are one-dimensional in color space, the inter-pixel region typically displays a gray color state intermediate between the states of the two adjacent pixels; such an intermediate gray color state does not significantly affect the average reflectance of the region, or can be easily modeled as blooming in effect. However, in a color display, the inter-pixel regions can display colors that are not present in any of the adjacent pixels.

[0025] The aforementioned problems in color displays have serious consequences for the gamut and linearity of colors predicted by spatially dithering the primary colors. Consider attempting to create a desired orange color using a spatially dithered pattern of saturated red and yellow colors from the primary color palette of an ECD display. In the absence of crosstalk, the combination required to create orange can be perfectly predicted in the far field using the law of linear additive color mixing. Because red and yellow lie on the gamut boundary, this predicted orange color should also lie on the gamut boundary. However, if the aforementioned effects produce (for example) a bluish band in the inter-pixel region between adjacent red and yellow pixels, the resulting color will be much more neutral than the predicted orange. This results in a "dimple" within the gamut boundary, or more precisely, a scalloping, since the boundary is actually three-dimensional. Thus, not only can a simple dithering approach fail to accurately predict the required dithering, but it may also attempt to produce a color that is unavailable because it lies outside the achievable color gamut.

[0026] It may be desirable to be able to predict the achievable color gamut through extensive measurement or advanced modeling of patterns. This may not be feasible when the number of device primaries is large or when crosstalk error is large compared to the error introduced by quantizing pixels to the primaries. The present invention provides a dithering method that incorporates a model of blooming / crosstalk error so that the realized colors on the display are closer to the predicted colors. Furthermore, the method stabilizes error diffusion in cases where the desired color falls outside the achievable color gamut, since error diffusion would normally produce unbounded error when dithering to colors outside the convex hull of the primaries.

[0027] In some embodiments, image duplication may be performed using an error diffusion model, as illustrated in Figure 1 of the accompanying drawings. The method illustrated in Figure 1 begins with an input 102, where a color value x i,jare fed to the processor 104, which produce the corrected input u, which may hereafter be referred to as "error corrected input colors" or "EMIC." i,j The modified input u is added to the output of the error filter 106 to produce i、j is fed to the quantizer 108.

[0028] In some embodiments, processes that utilize model-based error diffusion may be unstable because the input image is assumed to lie within the (theoretical) convex hull (i.e., color gamut) of the primary colors, but the actual achievable color gamut is likely smaller due to loss of color gamut due to dot overlap. Thus, the error diffusion algorithm may attempt to achieve colors that cannot actually be achieved in practice, and the error continues to grow with each successive "correction." This problem can be prevented by clipping or otherwise limiting the error, but it has been suggested that this leads to other errors.

[0029] In practice, one solution would be to have a better non-convex estimate of the achievable color gamut when performing gamut mapping of the source image so that the error diffusion algorithm can always achieve its target color. One could consider the possibility of approximating this from the model itself or determining it empirically. In some embodiments, the quantizer 108 examines the primaries in terms of the effect that selecting each would have on the error, and the quantizer selects the primary with the least error (by some metric), if selected. However, the primaries fed to the quantizer 108 may be smaller than the system's natural primaries {P k}, but allows for the color of at least some neighboring pixels. ~ k} and their influence on the pixels is quantized by blooming or other inter-pixel interactions.

[0030] One embodiment of the above method uses a standard Floyd-Steinberg error filter, processing pixels in raster order. Assuming a display is processed top-to-bottom and left-to-right, as is conventional, it is logical to use a pixel's upper and left radix neighbors, which are considered to calculate blooming or other inter-pixel effects, since these two neighbors have already been determined. In this way, right and lower neighbor crosstalk is considered when those neighbors are visited, so all modeled errors caused by neighboring pixels are taken into account. If the model only considers upper and left neighbors, the adjusted set of primaries must be a function of the state of those neighbors and the primary color under consideration. The simplest approach is to assume that the blooming model is additive, i.e., the color shift due to the left neighbor and the color shift due to the upper neighbor are independent and additive. In this case, there are only "N_select2" (equal to N*(N-1) / 2) model parameters (color shifts) that need to be determined. For N=64 or less, these can be estimated from colorimetric measurements of a checkerboard pattern of all these possible primary color pairs by subtracting the ideal mixture values ​​from the measurements.

[0031] To take a specific example, consider the case of a display with 32 color primaries. If only the upper and left neighbors are considered, then with 32 primaries, there are 496 possible neighboring sets of primaries for a given pixel. Because the model is linear, only these 496 color shifts need to be stored, since the additive effect of both neighbors can be produced during run-time without much overhead. Thus, for example, if the unadjusted primary set comprises (P1...P32) and the current upper and left neighbors are P4 and P7, then the modified primary (P ~ 1...P ~ 32 ), i.e. the adjusted primaries fed to the quantizer, are given by: P ~ 1=P1+dP(1,4) +dP (1,7) ; ... P ~ 32 =P 32 +dP (32,4) +dP (32,7) In the formula, dP (i,j) is an empirically determined value in a color shift table.

[0032] More complex pixel-to-pixel interaction models are of course possible, such as nonlinear models, models that consider corner (diagonal) neighbors, or models that use non-causal neighborhoods in which the color shift at each pixel is updated as more of its neighbors are learned.

[0033] The quantizer 108 receives the adjusted input u' i,j The adjusted primary colors {P ~ k} and find the most suitable primary color y i,k to the output. Any suitable method of selecting appropriate primaries may be used, for example a minimum Euclidean distance quantizer in linear RGB space, which has the advantage of requiring less computational power than some alternative methods.

[0034] y from the quantizer 108 i,k In addition to being output, the output values ​​may also be fed into a neighborhood buffer 110 where they are stored for use in generating adjusted primary colors for subsequently processed pixels. i,j Value and output y i,j Both values ​​are fed to processor 112, which calculates: e i,j =u i,j -y i,j This error signal is passed onto error filter 106 in the same manner as described above with reference to FIG.

[0035] However, in practice, error diffusion-based methods can be slow for some applications because they are not easily parallelizable: in this case, the next pixel output cannot be completed until the previous pixel's output is available. Alternatively, mask-based methods can be employed for their simplicity: the output at each pixel depends only on that pixel's input and values ​​from a look-up table (LUT), and each output can be calculated completely independently of the other outputs.

[0036] Referring now to Figure 2, an exemplary black and white dithering method is illustrated. As shown, an input grayscale image, with normalized darkness values ​​between 0 (white) and 1 (black), is dithered at each output location by comparing the corresponding input darkness with a dither threshold. For example, if the input image darkness u(x) is higher than the dither threshold T(x), the output location is marked as black (i.e., 1); otherwise, it is marked as white (i.e., 0). Figure 3 illustrates several mask designs in accordance with the subject matter disclosed herein.

[0037] In practice, when implementing multicolor dithering, it is assumed that the input colors to the dithering algorithm can be represented as a linear combination of multiple primary colors. This can be achieved by dithering in source space using gamut angles, or by gamut mapping the input to the gamut of device space. Figure 4 illustrates one way to create color separations using a set of weights Px. In this case, each color C is defined as follows:

number

[0038] where the partial sum of these weights is the separated cumulative

number

number

[0039] In practice, dithering multiple colors consists in intersecting the relative cumulative amounts of the colors with a dither function (e.g., threshold array T(x) 502 of FIG. 5). Referring now to FIG. 5, illustrated here by way of example is a method for printing with four different colored inks: C1 512, C2 514, C3 516, and C4 518. At each pixel of the output pixmap, the color separation provides the relative proportions of each of the base colors, e.g., d1 for color C1 512, d2 for color C2 514, d3 for color C3 516, and d4 for color C4 518. In this case, one of the colors, e.g., C4 518, may be white.

[0040] Extending dithering to multiple colors consists in intersecting the relative cumulative amounts of colors Λ1(x) 504=d1, Λ2(x) 506=d1+d2, Λ3(x) 508=d1+d2+d3, and Λ4(x) 510=d1+d2+d3+d4 with a threshold array T(x), as shown in Figure 5. Illustrated in Figure 5 is a dithering example for purposes of illustrating the subject matter presented herein. In the interval where Λ1(x) 504 > T(x) 502, the output location or pixel region will be printed using the base color C1 512 (e.g., black); in the interval where Λ2(x) 506 > T(x) 502, the output location or pixel region will display color C2 514 (e.g., yellow); in the interval where Λ3(x) 508 > T(x) 502, the output location or pixel region will display color C3 516 (e.g., red); and in the remaining intervals where Λ4(x) 510 > T(x) 502 and Λ3(x) 508 ≦ T(x) 502, the output location or pixel region will display color C4 518 (e.g., white). Thus, the multi-color dithering presented herein converts the relative amounts of d1, d2, d3, d4 of colors C1512, C2514, C3516, and C4518 into relative coverage, ensuring that, by definition, the contributing colors are printed side-by-side.

[0041] In some embodiments, a multi-color rendering algorithm such as that illustrated in Figure 6 may be utilized in accordance with the subject matter disclosed herein. As shown, image data im i,j may first be fed through a sharpening filter 602, which may be optional in some embodiments. This sharpening filter 602 may be useful in some cases where the threshold array T(x) or filter is less sharp than an error diffusion system. This sharpening filter 602 may be a simple finite impulse response (FIR) filter, e.g., 3×3, which may be easily calculated. The color data may then be mapped in a color mapping step 604, and color separations may be generated in a separation generation step 606 by methods commonly available in the art, such as using barycentric coordinate methods. This color data may be used to index a CSC_LUT lookup table, which may have N entries per index, providing the desired separation information in a form directly required by the mask-based dithering step (e.g., step 612). In some embodiments, this CSC_LUT lookup table may be constructed by combining both the desired color enhancement and / or color gamut mapping and the selected separation algorithm, and is configured to include a mapping between color values ​​of the input image and color separation accumulations. In this scheme, a lookup table (e.g., CSC_LUT) can be designed to provide the desired separated accumulation information quickly and in the form directly required by the mask-based dithering step (e.g., step 612 using a quantizer). Finally, the separated accumulation data 608 is used in conjunction with a threshold array 610, and a quantizer 612 is used to generate the output y i,j, generating multiple colors. In some embodiments, the color mapping 604, separation generation 606, and accumulation 608 steps may be implemented as a single interpolated CSC_LUT lookup table. In this configuration, the separation stage may be implemented by a lookup table rather than by finding the barycentric coordinates in the tetrahedralization of the multi-primary colors, allowing for more flexibility. In addition, the outputs calculated by the methods illustrated herein are calculated completely independently of other outputs. Furthermore, the threshold array T(x) used herein may be a blue noise mask (BNM), and various BNM designs are presented in Figures 7-10.

[0042] It will be apparent to those skilled in the art that numerous changes and modifications can be made in the specific embodiments of the invention described above without departing from the scope of the invention. Accordingly, the whole of the foregoing description is to be interpreted in an illustrative sense, and not in a restrictive sense.

Claims

1. 1. A method for driving an electro-optic display having a plurality of display pixels, the method comprising: receiving an input image, the input image including color data for each display pixel; obtaining a plurality of color separation accumulation values ​​by accessing a lookup table using the color data for each display pixel as an index, the lookup table storing the plurality of color separation accumulation values ​​for a plurality of indexes, the color data for each display pixel being defined by a linear combination of a plurality of primary colors using a set of weights, and a color separation accumulation value of the plurality of color separation accumulation values ​​for a kth primary color at each display pixel being a partial sum of the set of weights from a first weight to a kth weight; Displaying each basic color in a plurality of sections based on a comparison between the acquired plurality of color separation accumulated values ​​and a value in a threshold array, For each pixel location, a first primary color is displayed if a first value of the obtained plurality of color separation cumulative values ​​is greater than a value of the threshold array; a second primary color is displayed when a second value of the acquired color separation accumulation values ​​is greater than the value of the threshold array and the first value of the acquired color separation accumulation values ​​is less than the value of the threshold array; A method comprising:

2. The method of claim 1 , wherein the threshold array is a blue noise mask (BNM).

3. The method of claim 1 , further comprising passing the input image through a sharpening filter before using the color data for each display pixel as the index into the lookup table.

4. The method of claim 3 , wherein the sharpening filter is a finite impulse response (FIR) filter.

5. 10. An electro-optic display configured to perform the method of claim 1, the electro-optic display comprising an electrophoretic display.

Images