Method and apparatus for rendering color images

Abstract

To provide a method and apparatus for rendering color images. [Solution] A method for driving an electro-optical display is provided. A method for driving an electro-optical display having a plurality of display pixels, the method comprising: receiving an input image; processing the input image and creating a color separation accumulation; and using a threshold array, processing the color separation accumulation and generating colors for the electro-optical display. The method also comprises dithering the input image by crossing the color separation accumulation with a dither function.

Description

[Technical Field]

[0001] (Reference to related applications) This application is related to and claims priority from U.S. Provisional Application No. 63 / 108,855, filed on November 2, 2020.

[0002] The entire disclosure of the aforementioned application is incorporated herein by reference.

[0003] (Purpose of the invention) The present invention relates to a method for driving an electro-optical display. More specifically, the present invention relates to a driving method for dithering and rendering images on an electrophoretic display.

[0004] (background) The present invention relates to a method and apparatus for rendering color images. More specifically, the present invention relates to a method for multicolor dithering in which a combination of color intensities is converted into a multicolor surface coverage ratio. [Background technology]

[0005] The term "pixel" is used herein in its conventional sense in the field of display technology to mean the smallest unit of a display capable of producing all the colors that the display itself can display.

[0006] Halftoning has been used in the printing industry for decades to represent gray tones by coating each pixel of white paper with a varying percentage of black ink. Similar halftoning schemes can be used in conjunction with CMY or CMYK color printing systems, which use color channels that vary independently of each other.

[0007] However, there are many color systems in which the color channels cannot be varied independently of each other, because each pixel can display any one of a limited set of primary colors (such systems may hereafter be referred to as “Limited Palette Displays” or “LPDs”). The ECD patent color display is of this type. In order to create other colors, the primary colors must be spatially dithered to produce the correct color perception.

[0008] An electronic display typically includes an active matrix backplane, a master controller, local memory, and a set of communication and interface ports. The master controller receives data via the communication / interface ports or reads it from device memory. Once data enters the master controller, it is converted into a set of instructions for the active matrix backplane. The active matrix backplane receives these instructions from the master controller and produces an image. In the case of color devices, on-device color gamut calculation may require a master controller with increased computational power. As shown above, rendering methods for color electrophoretic displays are often computationally intensive, and while the invention itself provides methods for reducing the computational load imposed by rendering, as will be discussed in detail below, the rendering (dithering) step and other steps of the overall rendering process can still impose a significant load on the device computation processing system.

[0009] The increasing computational power required for image rendering diminishes the advantages of electrophoretic displays in some applications. In particular, the cost of manufacturing the device increases, as does the device's power consumption, when the master controller is configured to implement complex rendering algorithms. Furthermore, the excess heat generated by the controller necessitates thermal management. Therefore, having an efficient method for dithering multicolor images may be desirable, at least in some cases, such as when ultra-high-resolution images or a large number of images need to be rendered in a short time. [Overview of the project] [Means for solving the problem]

[0010] (Summary of the invention) Therefore, in one aspect, the subject matter presented herein provides a method for driving an electro-optical display, the method including receiving an input image, processing the input image to create a color separation accumulation, and dithering the input image by crossing the color separation accumulation with a dither function.

[0011] In some embodiments, the dither function is a threshold array.

[0012] In another embodiment, the threshold array is a blue noise mask (BNM).

[0013] In yet another embodiment, the processing step is implemented by a lookup table. The present invention provides, for example, the following: (Item 1) A method for driving an electro-optical display having multiple display pixels, wherein the method is Receiving an input image, The input image is processed to create a color-separated cumulative image, By intersecting the aforementioned color separation accumulation with a dithering function, the input image is dithered. A method including (Item 2) The dither function is a threshold array, the method according to item 1. (Item 3) The threshold array is a blue noise mask (BNM), the method according to item 2. (Item 4) The step of processing the input image is implemented by a look-up table, the method according to item 1. (Item 5) The look-up table includes a mapping between the color values of the input image and the color separation accumulation, the method according to item 3. (Item 6) Before processing the input image, further including passing the input image through a sharpening filter, the method according to item 1. (Item 7) The sharpening filter is a finite impulse response (FIR) filter, the method according to item 5. (Item 8) The step of processing the input image and creating a color separation accumulation includes using a barycentric coordinate method, the method according to item 1. (Item 9) An electro-optical display configured to perform the method according to item 1, including an electrophoretic display. (Item 10) The display according to item 9, comprising a rotating dichroic member and an electrochromic or electro-wetting material. (Item 11) An electro-optical display according to item 9, comprising an electrophoretic material comprising a plurality of charged particles disposed in a fluid and capable of moving through the fluid under the influence of an electric field. (Item 12) The electro-optical display according to item 11, wherein the charged particles and the fluid are confined within a plurality of capsules or microcells. (Item 13) The electro-optical display according to item 11, wherein the charged particles and the fluid exist as a plurality of discrete droplets surrounded by a continuous phase comprising a polymer material. [Brief explanation of the drawing]

[0014] A patent or application file shall contain at least one drawing, which shall be in color. A copy of this patent or patent application publication, accompanied by the color drawing, shall be provided by the Patent Office upon request and payment of the required fees.

[0015] [Figure 1] Figure 1 of the accompanying drawings is an image rendering model of the subject presented herein.

[0016] [Figure 2] Figure 2 shows exemplary black and white dithering methods using masks, according to the subject matter presented herein.

[0017] [Figure 3] Figure 3 illustrates various mask designs based on the subject presented herein.

[0018] [Figure 4] Figure 4 illustrates the color gamut color mapping of the subject disclosed herein.

[0019] [Figure 5] Figure 5 illustrates a multicolor dithering method using a mask, according to the subject matter disclosed herein.

[0020] [Figure 6] Figure 6 illustrates a multicolor dithering algorithm using a mask, according to the subject matter disclosed herein.

[0021] [Figure 7] Figure 7-10 shows various mask designs for multicolor dithering according to the subject presented herein. [Figure 8] Figure 7-10 shows various mask designs for multicolor dithering according to the subject presented herein. [Figure 9] Figure 7-10 shows various mask designs for multicolor dithering according to the subject presented herein. [Figure 10] Figure 7-10 shows various mask designs for multicolor dithering according to the subject presented herein. [Modes for carrying out the invention]

[0022] (Detailed explanation) Standard dithering algorithms, such as error diffusion algorithms (where the "error" introduced by printing one pixel with a specific color different from the theoretically required color for that pixel is distributed among adjacent pixels so that an overall correct color perception is produced), can be employed with limited-palette displays. A vast amount of literature exists on error diffusion. For further examination, see Pappas, Thrasyvoulos N. "Model-based halftoning of color images" IEEE Transactions on Image Processing. See 6.7 (1997): 1014–1024.

[0023] This application is based on U.S. Patents No. 5,930,026, No. 6,445,489, No. 6,504,524, No. 6,512,354, No. 6,531,997, No. 6,753,999, No. 6,825,970, No. 6,900,851, No. 6,995,550, No. 7,012,600, No. 7,023,420, No. 7,034,783, No. 7,061,166, No. 7,061,662, No. 7,116,466, No. 7,119,772, No. 7,177,066, No. 7,193,625, No. 7,202,847, No. 7,242,514, No. 7,259,744, No. 7,304,787, No. 7,312,794, No. 7,327,511, No. 7,408,699, No. 7, No. 453,445, No. 7,492,339, No. 7,528,822, No. 7,545,358, No. 7,583,251, No. 7,602 ,374, No. 7,612,760, No. 7,679,599, No. 7,679,813, No. 7,683,606, No. 7,688,2 No. 97, No. 7,729,039, No. 7,733,311, No. 7,733,335, No. 7,787,169, No. 7,859,742 , No. 7,952,557, No. 7,956,841, No. 7,982,479, No. 7,999,787, No. 8,077,141, No. No. 8,125,501, No. 8,139,050, No. 8,174,490, No. 8,243,013, No. 8,274,472, No. 8,2 No. 89,250, No. 8,300,006, No. 8,305,341, No. 8,314,784, No. 8,373,649, No. 8,384 ,658, No. 8,456,414, No. 8,462,102, No. 8,514,168, No. 8,537,105, No. 8,558,78 No. 3, No. 8,558,785, No. 8,558,786, No. 8,558,855, No. 8,576,164, No. 8,576,259 , No. 8,593,396, No. 8,605,032, No. 8,643,595, No. 8,665,206, No. 8,681,191, No. 8 ,730,153, No.8,810,525, No.8,928,562, No.8,928,641, No.8,976,444, No.9,0 No. 13,394, No. 9,019,197, No. 9,019,198, No. 9,019,318, No. 9,082,352, No. 9,171,Nos. 508, 9,218,773, 9,224,338, 9,224,342, 9,224,344, 9,230,492, 9,251,736, 9,262,973, 9,269,311, 9,299,294, 9,373,289, 9,390,066, 9,390,661, and 9,412,No. 314, and U.S. Patent Application Publications No. 2003 / 0102858, 2004 / 0246562, 2005 / 0253777, 2007 / 0091418, 2007 / 0103427, 2007 / 0176912, 2008 / 0024429, 2008 / 0024482, 2008 / 0136774, 2008 / 0291129, 2008 / 0303780, 2009 / 0174651, 2009 / 0195568, No. 2009 / 0322721, No. 2010 / 0194733, No. 2010 / 0194789, No. 2010 / 0220121, No. 2010 / 0265561, No. 2010 / 0283804, No. 2011 / 0063314, No. 2011 / 0175875, 2011 / 0193840, 2011 / 0193841, 2011 / 0199671, 2011 / 0221740, 2012 / 0001957, 2012 / 0098740, 20 No. 13 / 0063333, No. 2013 / 0194250, No. 2013 / 0249782, No. 2013 / 0321278, No. 2014 / 0009817, No. 2014 / 0085355, No. 2014 / 0204012, No. 201 No. 4 / 0218277, No. 2014 / 0240210, No. 2014 / 0240373, No. 2014 / 0253425, No. 2014 / 0292830, No. 2014 / 0293398, No. 2014 / 0333685, No. 2014 / This also relates to issues 0340734, 2015 / 0070744, 2015 / 0097877, 2015 / 0109283, 2015 / 0213749, 2015 / 0213765, 2015 / 0221257, 2015 / 0262255, 2015 / 0262551, 2016 / 0071465, 2016 / 0078820, 2016 / 0093253, 2016 / 0140910, and 2016 / 0180777. These patents and applications may, for convenience, be collectively referred to hereafter as the "MEDEOD" (Method for Driving Electro-Optical Displays) applications and are incorporated herein by reference as a whole.

[0024] ECD systems exhibit certain peculiarities that must be considered when designing dithering algorithms for use within such systems. Inter-pixel artifacts are a common feature in such systems. One type of artifact is caused by so-called "blooming." In both monochromatic and colorimetric systems, the electric field generated by the pixel electrode tends to affect an area of ​​the electro-optic medium that is larger than the area of ​​the pixel electrode itself, so that the optical state of one pixel effectively spreads into a portion of the area of ​​an adjacent pixel. Another type of crosstalk is superimposed when driving an adjacent pixel results in a final optical state in the area between pixels, which is different from the area reached by either of the pixels themselves, and this final optical state is caused by the average electric field superimposed in the inter-pixel region. A similar effect is superimposed in monochromatic systems, but since such systems are one-dimensional in color space, the inter-pixel region usually displays an intermediate gray state between the states of two adjacent pixels, and such an intermediate gray state does not significantly affect the average reflectance of the region, or can be easily modeled as de facto blooming. However, in a color display, the inter-pixel region can display a color that does not exist in any of the adjacent pixels.

[0025] The aforementioned problem in color displays has serious consequences for the gamut and linearity of colors predicted by spatially dithering primary colors. Consider attempting to create a desired orange using a spatially dithered pattern of saturated red and yellow from the primary color palette of an ECD display. In the absence of crosstalk, the combination required to create orange can be perfectly predicted in the far field by using the laws of linear additive color mixing. Since red and yellow lie on the gamut boundary, this predicted orange should also lie on the gamut boundary. However, if the aforementioned effect produces a bluish band (for example) in the interpixel region between adjacent red and yellow pixels, the resulting color will be far more neutral than the predicted orange. This results in a "depression" within the gamut boundary, or more precisely, a scallop, since the boundary is actually three-dimensional. Thus, a simple dithering approach not only fails to accurately predict the required dithering, but may also attempt to produce a color that is unavailable because it lies outside the achievable gamut.

[0026] It may be desirable to be able to predict the achievable color gamut through extensive measurement or advanced modeling of the pattern. This may not be feasible when the number of device primary colors is large, or when crosstalk errors are large compared to the errors introduced by quantizing pixels to primary colors. The present invention provides a dithering method that incorporates a model of blooming / crosstalk errors so that the colors realized on the display are closer to the predicted colors. Furthermore, the method stabilizes error diffusion when the desired color falls outside the achievable color gamut, as error diffusion would normally produce unlimited errors when dithering to colors outside the convex hull of the primary colors.

[0027] In some embodiments, image copying may be performed using an error diffusion model illustrated in Figure 1 of the accompanying drawings. The method illustrated in Figure 1 starts with input 102, where the color value x i,jThese are fed to processor 104, and they are corrected input u, which may hereafter be referred to as “error-corrected input color” or “EMIC”. i,j The corrected input u is added to the output of the error filter 106 to produce the corrected input u. i、j This is fed into quantizer 108.

[0028] In some embodiments, processes utilizing model-based error diffusion assume that the input image lies within the (theoretical) convex hull (i.e., color gamut) of the primary colors, but this can be unstable because the actual achievable color gamut is smaller, likely due to color gamut loss caused by dot overlap. Thus, error diffusion algorithms may attempt to achieve colors that are not actually achievable in practice, and the error continues to increase with each successive "correction." This problem can be thwarted by trimming or otherwise limiting the error, but this has been suggested to lead to other errors.

[0029] In practice, one solution would be to have a better non-convex estimate of the achievable color gamut when performing color gamut mapping of the source image so that the error diffusion algorithm can always achieve its target color. This can be estimated from the model itself or determined empirically. In some embodiments, the quantizer 108 investigates the primary colors in terms of the impact that selecting each would have on the error, and the quantizer selects the primary color with the smallest error (by some metric) if selected. However, the primary colors fed to the quantizer 108 are the system's natural primary colors {P k} is not, but a modified set of primary colors that allows for the color of at least some adjacent pixels {P ~ k} and their effects on pixels are quantized by blooming or other inter-pixel interactions.

[0030] One embodiment of the above method uses a standard Floyd-Steinberg error filter and processes pixels in raster order. Assuming that, as is conventional, the display is processed from top to bottom and left to right, it is logical to use the cardinal neighbors above and to the left of the pixel, which are considered to calculate blooming or other inter-pixel effects since these two adjacent pixels have already been determined. In this way, all modeled errors caused by adjacent pixels are taken into account because right and bottom adjacent crosstalk is considered when their neighbors are visited. If the model considers only the upper and left neighbors, the adjusted set of primary colors must be a function of the state of those neighbors and the primary color under consideration. The simplest approach is to assume that the blooming model is additive, i.e., the color shift due to the left neighbor and the color shift due to the upper neighbor are independent and additive. In this case, only the "N choose 2" (equal to N*(N - 1) / 2) model parameters (color shifts) that need to be determined exist. For N = 64 or less, these can be estimated from the colorimetric measurements of the grid pattern of all these possible primary color pairs by subtracting the ideal mixing method value from the measured value.

[0031] To give a specific example, consider the case of a display with 32 primary colors. If only the upper and left neighbors are considered, there are 496 possible adjacent sets of primary colors for a given pixel with respect to the 32 primary colors. Since the model is linear, only these 496 color shifts need to be stored because the additive effects of both neighbors can be produced during runtime without much overhead. Thus, for example, if the unadjusted set of primary colors is (P1...P32) and the current upper and left neighbors are P4 and P7, the adjusted primary colors (P ~ 1...P ~ 32 ) fed to the quantizer, i.e., the adjusted primary colors, are determined as follows. P ~ 1=P1+dP(1,4) +dP (1,7) ; ... ... P ~ 32 =P 32 +dP (32,4) +dP (32,7) In the formula, dP (i,j) This is a value determined empirically in the color shift table.

[0032] More complex pixel interaction models, such as nonlinear models, models that consider angular (diagonal) neighbors, or models that use acausal neighborhoods where the color shift at each pixel is updated as more of those neighbors are known, are certainly possibilities.

[0033] The quantizer 108 takes the tuned input u' i,j Adjusted primary colors {P ~ k Compared to}, the most appropriate primary color y i,k The output is output to the output. Any suitable method for selecting appropriate primary colors, such as a minimum Euclidean distance quantizer in linear RGB space, may be used, which has the advantage of requiring less computational power than some alternative methods.

[0034] y from quantizer 10⁸ i,k The output value may be fed not only to the output but also to the neighbor buffer 110, where it is stored for use in generating the adjusted primary color for the pixels to be processed later. i,j Value and output y i,j Both values ​​are supplied to processor 112, which calculates the following: e i,j =u i,j -y i,j Refer to Figure 1 and pass the error signal onto the error filter 106 in the same manner as described above.

[0035] However, in practice, error diffusion-based methods are not easily parallelizable and can be slow for some applications. In this case, the output of the next pixel cannot be completed until the output of the previous pixel is available. As an alternative, a mask-based method may be adopted for its simplicity, where the output at each pixel depends only on the input and values ​​from the lookup table (LUT) for that pixel, and each output can be calculated completely independently of the other outputs.

[0036] Referring here to Figure 2, exemplary black and white dithering methods are illustrated. As shown, an input grayscale image with normalized darkness values ​​between 0 (white) and 1 (black) is dithered at each output location by comparing the corresponding input darkness with a dither threshold. For example, if the darkness u(x) of the input image is higher than the dither threshold T(x), the output location is marked as black (i.e., 1); otherwise, it is marked as white (i.e., 0). Figure 3 illustrates several mask designs according to the subject disclosed herein.

[0037] In practice, when performing multicolor dithering, it is assumed that the input colors to the dithering algorithm can be represented as a linear combination of multiple primary colors. This can be achieved by using the color gamut angle to dither in source space, or by color gamut mapping the input to the color gamut in device space. Figure 4 illustrates one method of creating color separation using a set of weighted Px. In this case, each color C is defined as follows:

number

[0038] In the formula, the partial sum of these weights is separated cumulative

number

number

[0039] In practice, dithering with multiple colors involves crossing the relative cumulative amount of the colors with a dither function (e.g., the threshold array T(x)502 in Figure 5). Referring to Figure 5, illustrated here as an example, is a method for printing using four different colored inks, C1512, C2514, C3516, and C4518. At each pixel of the output pixmap, the color separation gives the relative ratios of each of the base colors, e.g., d1 for color C1512, d2 for color C2514, d3 for color C3516, and d4 for color C4518. In this case, one of the colors, for example C4518, may be white.

[0040] Extending dithering to multiple colors involves crossing the relative cumulative amounts of the colors Λ1(x)504=d1, Λ2(x)506=d1+d2, Λ3(x)508=d1+d2+d3, and Λ4(x)510=d1+d2+d3+d4 with a threshold array T(x), as shown in Figure 5. Illustrated in Figure 5 is an example of dithering for the purpose of illustrating the subject matter presented herein. In the interval Λ1(x)504 > T(x)502, the output location or pixel area will be printed using the base color C1512 (e.g., black); in the interval Λ2(x)506 > T(x)502, the output location or pixel area will display the color C2514 (e.g., yellow); in the interval Λ3(x)508 > T(x)502, the output location or pixel area will display the color C3516 (e.g., red); and in the remaining intervals Λ4(x)510 > T(x)502 and Λ3(x)508 ≤ T(x)502, the output location or pixel area will display the color C4518 (e.g., white). Therefore, the multicolor dithering presented herein converts the relative amounts of d1, d2, d3, and d4 of colors C1512, C2514, C3516, and C4518 into relative coverage, ensuring that, by definition, the contributing colors are printed in parallel.

[0041] In some embodiments, a multicolor rendering algorithm, such as the one illustrated in Figure 6, may be used in accordance with the subject matter disclosed herein. As shown, image data im i,j The color data may first be fed through a sharpening filter 602, which may be optional in some embodiments. This sharpening filter 602 may be useful in some cases where the threshold array T(x) or filter is not sharper than the error diffusion system. This sharpening filter 602 may be a simple finite impulse response (FIR) filter, e.g., 3x3, which can be readily computed. Subsequently, the color data may be mapped in a color mapping step 604, and color separation may be generated in a separation generation step 606 by a method commonly available in the art, such as using a centroid coordinate method, and the color data may be used to index a CSC_LUT lookup table, which may have N entries per index, giving the desired separation information in a form directly required by a mask-based dithering step (e.g., step 612). In some embodiments, the CSC_LUT lookup table may be constructed by combining both the desired color enhancement and / or gamut mapping, as well as a selected separation algorithm, and is configured to include a mapping between the color values ​​of the input image and the cumulative color separation. In this scheme, the lookup table (e.g., CSC_LUT) can be designed to provide the desired separation and accumulation information quickly and in the form directly required by the mask-based dithering step (e.g., step 612 using a quantizer). Finally, the separation and accumulation data 608 is used in conjunction with the threshold array 610 and the quantizer 612 is used to produce the output y i,jThis generates multiple colors. In some embodiments, the color mapping 604, separation generation 606, and accumulation 608 steps may be implemented as a single interpolation CSC_LUT lookup table. In this configuration, the separation step may be implemented by a lookup table rather than by finding the centroid coordinates in the tetrahedronization of the multiple primary colors, which allows for greater flexibility. In addition, the output calculated by the method illustrated herein is calculated completely independently of other outputs. Furthermore, the threshold array T(x) used herein may be a blue noise mask (BNM), and various BNM designs are presented in Figure 7-10.

[0042] It will be apparent to those skilled in the art that numerous changes and modifications can be made without departing from the scope of the invention, in the specific embodiments of the invention described above. Therefore, the entire preceding description should be interpreted as illustrative, not restrictive.

Claims

[Claim 1] The invention described herein.

Images