Color uniformity correction of display devices

Abstract

Techniques for improving color uniformity of a display of a display device are disclosed. A plurality of images of the display is captured using an image capture device. The plurality of images is captured in a color space, where each image corresponds to one of a plurality of color channels. Global white balancing is performed on the plurality of images to obtain a plurality of normalized images. Local white balancing is performed on the plurality of normalized images to obtain a plurality of correction matrices. Performing local white balancing includes defining a set of weighting factors based on a figure of merit, and calculating a plurality of weighted images based on the plurality of normalized images and the set of weighting factors. The plurality of correction matrices is calculated based on the plurality of weighted images.

Description

[0001] Cross-references to related applications

[0002] This application claims the benefit of priority to U.S. Provisional Patent Application No. 63 / 044,995, filed June 26, 2020, entitled “COLOR UNIFORMITY CORRECTION OF DISPLAY DEVICE,” the entire contents of which are incorporated herein by reference for all purposes. Background Technology

[0003] A display or display device is an output device that presents information in a visual form by emitting light (usually through projection or emission) toward a light-receiving object (such as a user's eye). Many displays utilize an additive color model to achieve a wide range of colors by simultaneously or sequentially displaying several additive colors (such as red, green, and blue) at different intensities. For example, in some additive color models, white (or target white point) is achieved by displaying each of the additive colors simultaneously or sequentially at non-zero and relatively similar intensities, while black is achieved by displaying each of the additive colors at zero intensity.

[0004] The accuracy of a display's colors can be correlated with the actual intensity of each added color at each pixel of the display. For many display technologies, determining and controlling the actual intensity of added colors can be difficult, especially at the pixel level. Therefore, new systems, methods, and other technologies are needed to improve color uniformity across such displays. Summary of the Invention

[0005] This disclosure generally relates to techniques for improving color uniformity in displays and display devices. More specifically, embodiments of this disclosure provide techniques for calibrating multi-channel displays by capturing and processing images of displays for multiple color channels. Although portions of this disclosure are described with reference to augmented reality (AR) devices, this disclosure is applicable to a wide range of applications in computer vision and display technologies.

[0006] The following provides a summary of various embodiments of the invention as a list of examples. As used below, any reference to a series of examples will be understood as a separate reference to each of those examples (e.g., "Examples 1-4" will be understood as "Examples 1, 2, 3, or 4").

[0007] Example 1 is a method for displaying a video sequence comprising a series of images on a display, the method comprising: receiving the video sequence at a display device, the video sequence having a plurality of color channels; applying per-pixel correction to each of the plurality of color channels of the video sequence using a correction matrix of a plurality of correction matrices, wherein each of the plurality of correction matrices corresponds to one of the plurality of color channels; and wherein applying the per-pixel correction generates a corrected video sequence having the plurality of color channels; and displaying the corrected video sequence on the display of the display device.

[0008] Example 2 is the method according to Example 1, wherein the plurality of correction matrices are previously calculated by: capturing a plurality of images of the display using an image capture device, wherein the plurality of images are captured in a color space, and wherein each of the plurality of images corresponds to one of the plurality of color channels; performing global white balance on the plurality of images to obtain a plurality of normalized images, each normalized image corresponding to one of the plurality of color channels; and performing local white balance on the plurality of normalized images to obtain the plurality of correction matrices, wherein performing the local white balance includes: defining a set of weighting factors based on a figure of merit; calculating a plurality of weighted images based on the plurality of normalized images and the set of weighting factors; and calculating the plurality of correction matrices based on the plurality of weighted images.

[0009] Example 3 is the method according to Example 1, further comprising: using the plurality of correction matrices to determine a plurality of target source currents; and setting the plurality of source currents of the display device as the plurality of target source currents.

[0010] Example 4 is a non-transitory computer-readable medium including instructions that, when executed by one or more processors, cause the one or more processors to perform operations including: receiving at a display device a video sequence comprising a series of images, the video sequence having a plurality of color channels; applying per-pixel correction to each of the plurality of color channels of the video sequence using a correction matrix of a plurality of correction matrices, wherein each of the plurality of correction matrices corresponds to one of the plurality of color channels, and wherein the application of the per-pixel correction generates a corrected video sequence having the plurality of color channels; and displaying the corrected video sequence on a display of the display device.

[0011] Example 5 is a non-transitory computer-readable medium according to Example 4, wherein the plurality of correction matrices are previously calculated by: capturing a plurality of images of the display using an image capture device, wherein the plurality of images are captured in a color space, and wherein each of the plurality of images corresponds to one of the plurality of color channels; performing global white balance on the plurality of images to obtain a plurality of normalized images, each normalized image corresponding to one of the plurality of color channels; and performing local white balance on the plurality of normalized images to obtain the plurality of correction matrices, wherein performing the local white balance includes: defining a set of weighting factors based on a quality factor; calculating a plurality of weighted images based on the plurality of normalized images and the set of weighting factors; and calculating the plurality of correction matrices based on the plurality of weighted images.

[0012] Example 6 is a non-transitory computer-readable medium according to Example 4, wherein the operation further includes: determining a plurality of target source currents using the plurality of correction matrices; and setting the plurality of source currents of the display device as the plurality of target source currents.

[0013] Example 7 is a system comprising: one or more processors; and a non-transitory computer-readable medium including instructions that, when executed by the one or more processors, cause the one or more processors to perform operations including: receiving at a display device a video sequence comprising a series of images, the video sequence having a plurality of color channels; applying per-pixel correction to each of the plurality of color channels of the video sequence using a correction matrix from a plurality of correction matrices, wherein each of the plurality of correction matrices corresponds to one of the plurality of color channels, and wherein the application of the per-pixel correction generates a corrected video sequence having the plurality of color channels; and displaying the corrected video sequence on a display of the display device.

[0014] Example 8 is a system according to Example 7, wherein the plurality of correction matrices are previously calculated by: capturing a plurality of images of the display using an image capture device, wherein the plurality of images are captured in a color space, and wherein each of the plurality of images corresponds to one of the plurality of color channels; performing global white balance on the plurality of images to obtain a plurality of normalized images, each normalized image corresponding to one of the plurality of color channels; and performing local white balance on the plurality of normalized images to obtain the plurality of correction matrices, wherein performing the local white balance includes: defining a set of weighting factors based on a quality factor; calculating a plurality of weighted images based on the plurality of normalized images and the set of weighting factors; and calculating the plurality of correction matrices based on the plurality of weighted images.

[0015] Example 9 is a system according to Example 7, wherein the operation further includes: using the plurality of correction matrices to determine a plurality of target source currents; and setting the plurality of source currents of the display device to the plurality of target source currents.

[0016] Example 10 is a method for improving the color uniformity of a display, the method comprising: capturing a plurality of images of the display of a display device using an image capture device, wherein the plurality of images are captured in a color space, and wherein each of the plurality of images corresponds to one of a plurality of color channels; performing global white balance on the plurality of images to obtain a plurality of normalized images, each normalized image corresponding to one of the plurality of color channels; and performing local white balance on the plurality of normalized images to obtain a plurality of correction matrices, each correction matrix corresponding to one of the plurality of color channels, wherein performing the local white balance comprises: defining a set of weighting factors based on a quality factor; calculating a plurality of weighted images based on the plurality of normalized images and the set of weighting factors; and calculating the plurality of correction matrices based on the plurality of weighted images.

[0017] Example 11 is the method according to Example 10, further comprising: applying the plurality of correction matrices to the display device.

[0018] Example 12 is the method according to Examples 10 to 11, wherein the quality factor is at least one of the following: power consumption; color error; or minimum bit depth.

[0019] Example 13 is the method according to Examples 10 to 12, wherein defining the set of weighting factors based on the quality factor includes: minimizing the quality factor by changing the set of weighting factors; and determining the set of weighting factors that minimizes the quality factor.

[0020] Example 14 is the method according to Examples 10 to 13, wherein the color space is one of the following: CIELUV color space; CIEXYZ color space; or sRGB color space.

[0021] Example 15 is the method according to Examples 10 to 14, wherein performing the global white balance on the plurality of images includes: determining a target illuminance value in the color space based on a target white point, wherein the plurality of normalized images are calculated based on the target illuminance value.

[0022] Example 16 is the method according to Example 15, wherein the plurality of correction matrices are further calculated based on the target illuminance value.

[0023] Example 17 is the method according to Examples 10 to 16, wherein the display is a diffractive waveguide display.

[0024] Example 18 is a non-transitory computer-readable medium including instructions that, when executed by one or more processors, cause the one or more processors to perform operations including: capturing a plurality of images of a display of a display device using an image capture device, wherein the plurality of images are captured in a color space, and wherein each of the plurality of images corresponds to one of a plurality of color channels; performing global white balance on the plurality of images to obtain a plurality of normalized images, each normalized image corresponding to one of the plurality of color channels; and performing local white balance on the plurality of normalized images to obtain a plurality of correction matrices, each correction matrix corresponding to one of the plurality of color channels, wherein performing the local white balance includes: defining a set of weighting factors based on a quality factor; calculating a plurality of weighted images based on the plurality of normalized images and the set of weighting factors; and calculating the plurality of correction matrices based on the plurality of weighted images.

[0025] Example 19 is a non-transitory computer-readable medium according to Example 18, wherein the operation further includes applying the plurality of correction matrices to the display device.

[0026] Example 20 is a non-transitory computer-readable medium according to Examples 18-19, wherein the quality factor is at least one of the following: power consumption; color error; or minimum bit depth.

[0027] Example 21 is a non-transitory computer-readable medium according to Examples 18 to 20, wherein defining the set of weighting factors based on the quality factor includes: minimizing the quality factor by changing the set of weighting factors; and determining the set of weighting factors that minimizes the quality factor.

[0028] Example 22 is a non-transitory computer-readable medium according to Examples 18 to 21, wherein the color space is one of the following: CIELUV color space; CIEXYZ color space; or sRGB color space.

[0029] Example 23 is a non-transitory computer-readable medium according to Examples 18 to 22, wherein performing the global white balance on the plurality of images includes: determining a target illuminance value in the color space based on a target white point, wherein the plurality of normalized images are calculated based on the target illuminance value.

[0030] Example 24 is a non-transitory computer-readable medium according to Example 23, wherein the plurality of correction matrices are further calculated based on the target illuminance value.

[0031] Example 25 is a non-transitory computer-readable medium according to Examples 18 to 24, wherein the display is a diffractive waveguide display.

[0032] Example 26 is a system comprising: one or more processors; and a non-transitory computer-readable medium including instructions that, when executed by the one or more processors, cause the one or more processors to perform operations including: capturing a plurality of images of a display of a display device using an image capture device, wherein the plurality of images are captured in a color space, and wherein each of the plurality of images corresponds to one of a plurality of color channels; performing global white balance on the plurality of images to obtain a plurality of normalized images, each normalized image corresponding to one of the plurality of color channels; and performing local white balance on the plurality of normalized images to obtain a plurality of correction matrices, each correction matrix corresponding to one of the plurality of color channels, wherein performing the local white balance includes: defining a set of weighting factors based on a quality factor; calculating a plurality of weighted images based on the plurality of normalized images and the set of weighting factors; and calculating the plurality of correction matrices based on the plurality of weighted images.

[0033] Example 27 is a system according to Example 26, wherein the operation further includes: applying the plurality of correction matrices to the display device.

[0034] Example 28 is a system according to Examples 26 to 27, wherein the quality factor is at least one of the following: power consumption; color error; or minimum bit depth.

[0035] Example 29 is a system according to Examples 26 to 28, wherein defining the set of weighting factors based on the quality factor includes: minimizing the quality factor by changing the set of weighting factors; and determining the set of weighting factors that minimizes the quality factor.

[0036] Example 30 is a system according to Examples 26 to 29, wherein the color space is one of the following: CIELUV color space; CIEXYZ color space; or sRGB color space.

[0037] Example 31 is a system according to Examples 26 to 30, wherein performing the global white balance on the plurality of images includes: determining a target illuminance value in the color space based on a target white point, wherein the plurality of normalized images are calculated based on the target illuminance value.

[0038] Example 32 is a system according to Example 31, wherein the plurality of correction matrices are further calculated based on the target illuminance value.

[0039] Example 33 is a system according to Examples 26 to 32, wherein the display is a diffractive waveguide display.

[0040] The present disclosure offers numerous benefits compared to conventional techniques. For example, the embodiments described herein are capable of correcting high levels of color inhomogeneity. The embodiments also take into account robustness for various applications, including eye position, electrical power, and bit depth. The embodiments can further relax the manufacturing requirements and tolerances (such as TTV (related to wafer thickness variation), diffraction structure fidelity, layer-to-layer alignment, projector-to-layer alignment, etc.) required to produce a certain level of color uniformity in displays. The techniques described herein are applicable not only to displays employing diffractive waveguide eyepieces but also to a wide variety of displays, such as reflective holographic optics (HOE) displays, reflective combiner displays, birdbath combiner displays, embedded reflector waveguide displays, and so on. Attached Figure Description

[0041] The accompanying drawings, which are incorporated in and form a part of this specification to provide a further understanding of this disclosure, illustrate embodiments of the disclosure and, together with the detailed description, serve to explain the principles of the disclosure. No attempt is made to show the structural details of the disclosure in more detail than necessary for a basic understanding of the disclosure and for practicing the various modes of the disclosure.

[0042] Figure 1 An example of a calibration scheme is shown.

[0043] Figure 2 An example of the brightness uniformity pattern that may appear for different color channels in a diffractive waveguide eyepiece is shown.

[0044] Figure 3 A method for displaying a video sequence comprising a series of images on a monitor is shown.

[0045] Figure 4 A method for improving the color uniformity of a display is shown.

[0046] Figure 5 An example of improved color uniformity is shown.

[0047] Figure 6 It shows the use of Figure 5 The example shown is a set of error histograms.

[0048] Figure 7 An example correction matrix is ​​shown.

[0049] Figure 8 An example of a brightness uniformity pattern for a display color channel is shown.

[0050] Figure 9A method for improving color uniformity of a display for multiple eye positions is shown.

[0051] Figure 10 A method for improving color uniformity of a display for multiple eye positions is shown.

[0052] Figure 11 An example of improved color uniformity for multiple eye locations is shown.

[0053] Figure 12 A method for determining and setting the source current of a display device is shown.

[0054] Figure 13 A schematic diagram of an example wearable system is shown.

[0055] Figure 14 A simplified computer system is shown.

[0056] Several of the accompanying drawings include color features that have been converted to grayscale for reproduction purposes. The applicant reserves the right to reintroduce the color features at a later date. Detailed Implementation

[0057] Many types of displays, including augmented reality (AR) displays, suffer from color inhomogeneity across the user's field of view (FoV). The sources of this inhomogeneity vary across display technologies, but it is particularly problematic for diffractive waveguide eyepieces. A significant factor contributing to color inhomogeneity in these displays is the partial-to-partial variation in the local thickness profile of the eyepiece substrate, which can lead to large variations in the uniformity pattern of the output image. In eyepieces containing multiple layers, the uniformity patterns of the display channels (e.g., red, green, and blue display channels) can be significantly different, resulting in color inhomogeneity. Other factors that can contribute to color inhomogeneity include variations in the grating structure across the eyepiece, variations in the alignment of optical elements within the system, system differences in the optical paths between display channels, and so on.

[0058] Embodiments of this disclosure provide techniques for improving color uniformity in displays and display devices. Such techniques can correct color inhomogeneities caused by multiple displays, including AR displays, so that after correction, a user can see more uniform colors across the entire FoV of the display. In some embodiments, the techniques may include a calibration process and algorithm that generates a correction matrix corresponding to values ​​between 0 and 1 for each pixel and color channel used by the spatial light modulator (SLM). The generated correction matrix may be multiplied with each image frame sent to the SLM to improve color uniformity.

[0059] In the following description, various examples will be described. Specific configurations and details are illustrated for illustrative purposes to provide a thorough understanding of the examples. However, it will be apparent to those skilled in the art that the examples can be practiced without the specific details. Furthermore, well-known features may be omitted or simplified so as not to obscure the described embodiments.

[0060] Figure 1 Example display calibration schemes according to some embodiments of the present disclosure are illustrated. In the illustrated example, camera 108 is positioned at the user's eye position relative to the display 112 of wearable device 102. In some instances, camera 108 may be mounted near wearable device 102 at a station. Camera 108 can be used to simultaneously or sequentially measure the display output of the wearable device for the left and right eyes. Although each of the cameras 108 is shown positioned at a single eye position for the sake of simplicity, it should be understood that each of the cameras 108 may be shifted to several positions to account for possible color shifts due to changes in eye position, interpupillary distance, and user movement. By way of example only, each of the cameras 108 (or similarly, wearable device 102) may be shifted at three lateral positions: -3mm, 0mm, and +3mm. Additionally, the relative angle of wearable device 102 with respect to each of the cameras 108 may also be varied to provide additional calibration conditions.

[0061] Each of the displays 112 may include one or more light sources, such as light-emitting diodes (LEDs). In some embodiments, liquid crystal on silicon (LCOS) may be used to provide the displayed image. The LCOS may be built into the wearable device 102. During calibration, the wearable device 102 may project image light in field-order colors (e.g., in the order of red, green, and blue). In a field-order color system, primary color information is sent in successive images, relying on the human visual system to fuse successive images into a color picture. Each of the cameras 108 may capture an image in the camera's color space and provide the data to the calibration workstation. The color space may be converted from a first color space (e.g., the camera's color space) to a second color space before further processing of the captured image. For example, the captured image may be converted from the camera's RGB space to the XYZ color space.

[0062] In some embodiments, each of the displays 112 displays a separate image for each light source to generate the target white point. While each of the displays 112 is displaying each image, the corresponding camera can capture the displayed image. For example, when displaying a red image using a red light source, a first image of the display can be captured; when displaying a green image using a green light source, a second image of the same display can be captured; and when displaying a blue image using a blue light source, a third image of the same display can be captured. The three captured images, along with the three captured images for another display, can then be processed according to the described embodiments.

[0063] Figure 2 Examples of brightness uniformity patterns that may appear for different color channels in a diffractive waveguide eyepiece according to some embodiments of the present disclosure are shown. From left to right, brightness uniformity patterns for the red, green, and blue display channels in a diffractive waveguide eyepiece are shown. The combination of individual display channels results in the rightmost color uniformity image, which consistently exhibits non-uniform color. In the example shown, images were captured by a diffractive waveguide eyepiece consisting of 3 layers (one layer for each display channel) (gamma = 2.2). Each image corresponds to a 45° × 55° FoV. Figure 2 This includes color features that have been converted to grayscale for reproduction purposes.

[0064] Figure 3 A method 300 for displaying a video sequence comprising a series of images on a display is illustrated according to some embodiments of the present disclosure. One or more steps of method 300 may be omitted during execution of method 300, and the steps of method 300 need not be performed in the order shown. One or more steps of method 300 may be executed by one or more processors. Method 300 may be implemented as a computer-readable medium or computer program product including instructions that, when executed by one or more computers, cause one or more computers to perform the steps of method 300.

[0065] At step 302, a video sequence is received at a display device. The video sequence may include a series of images. The video sequence may include multiple color channels, each of which corresponds to one of multiple illumination sources of the display device. For example, the video sequence may include red, green, and blue channels, and the display device may include red, green, and blue illumination sources. The illumination sources may be LEDs.

[0066] At step 304, a plurality of correction matrices are determined. Each of the plurality of correction matrices may correspond to one of the plurality of color channels. For example, the plurality of correction matrices may include red, green, and blue correction matrices.

[0067] At step 306, per-pixel correction is applied to each of the multiple color channels of the video sequence using one of the multiple correction matrices. For example, a red correction matrix can be applied to the red channel of the video sequence, a green correction matrix can be applied to the green channel of the video sequence, and a blue correction matrix can be applied to the blue channel of the video sequence. In some embodiments, applying per-pixel correction results in the generation of a corrected video sequence with multiple color channels.

[0068] At step 308, the calibrated video sequence is displayed on a monitor of the display device. For example, the calibrated video sequence can be sent to a projector (e.g., LCOS) of the display device. The projector can project the calibrated video sequence onto the monitor. The monitor can be a diffractive waveguide display.

[0069] At step 310, a plurality of target source currents are determined. Each of the target source currents may correspond to one of a plurality of lighting sources and one of a plurality of color channels. For example, the plurality of target source currents may include red, green, and blue target source currents. In some embodiments, the plurality of target source currents are determined based on a plurality of correction matrices.

[0070] At step 312, multiple source currents of the display device are set to multiple target source currents. For example, the red source current (corresponding to the amount of current flowing through the red lighting source) can be set to the red target current by adjusting the red source current toward or equal to the value of the red target current; the green source current (corresponding to the amount of current flowing through the green lighting source) can be set to the green target current by adjusting the green source current toward or equal to the value of the green target current; and the blue source current (corresponding to the amount of current flowing through the blue lighting source) can be set to the blue target current by adjusting the blue source current toward or equal to the value of the blue target current.

[0071] Figure 4 A method 400 for improving the color uniformity of a display according to some embodiments of the present disclosure is illustrated. One or more steps of method 400 may be omitted during execution of method 400, and the steps of method 400 need not be performed in the order shown. One or more steps of method 400 may be executed by one or more processors. Method 400 may be implemented as a computer-readable medium or computer program product including instructions that, when executed by one or more computers, cause one or more computers to perform the steps of method 400. The steps of method 400 may include and / or may be used in combination with one or more steps of various other methods described herein.

[0072] When a white image is displayed on a monitor, the amount of color non-uniformity in the monitor can be characterized by the shift in color coordinates from the desired white point. To capture the amount of color variation across the FoV, the root mean square (RMS) of the deviation from the target white point (e.g., D65) in color coordinates at each pixel in the FoV can be calculated. When using the CIELUV color space, the RMS color error can be calculated as:

[0073]

[0074] Among them, u' px It is the u' value at pixel px, v' px It is the v' value at pixel px, D65 u' This is the u' value used for the D65 white point, D65 v' These are the v' value used for the D65 white point, and n px' It refers to the number of pixels.

[0075] One objective of color uniformity correction is to minimize the RMS color error as much as possible within the eye position range of the eye box, while minimizing the negative impact on display power consumption, display brightness, and color bit depth. The output of method 400 can be a set of correction matrices C. R,G,B It contains the values ​​between 0 and 1 at each pixel of the display for each color channel, as well as multiple target source currents I. R I G and I B .

[0076] A set of input data can be used to describe the display's output in sufficient detail to correct color inhomogeneity, white balance, and minimize power consumption. In some embodiments, this set of input data may include a mapping of CIE XYZ tristimulus values ​​across FoV, and data relating the brightness of each display channel to the electrical drive characteristics of the illumination source. This information can be collected and processed as described below.

[0077] At step 402, an image capturing device is used to capture multiple images of the display (e.g., image 450). Each of the multiple images may correspond to one of multiple color channels. For example, when displaying using a first illumination source corresponding to a first color channel, a first image of the display may be captured; when displaying using a second illumination source corresponding to a second color channel, a second image of the display may be captured; and when displaying using a third illumination source corresponding to a third color channel, a third image of the display may be captured.

[0078] Multiple images can be captured in a specific color space. For example, each pixel of each image can include a value for a specific color space. The color space can be CIELUV, CIEXYZ, sRGB, or CIELAB, etc. For example, each pixel of each image can include a CIE XYZ tristimulus value. This value can be captured across FoV using a colorimeter, spectrophotometer, or a calibrated RGB camera, etc. In some examples, if each color channel does not exhibit strong chromaticity variations across FoV, a simpler option can be used that combines a uniformity pattern captured by a monochrome camera with measurements of chromaticity at a single field point. The required resolution may depend on the angular frequency of color non-uniformity in the display. To correlate the display's output with the electrical drive characteristics of the illumination source, the output power or brightness of each display channel can be characterized as the current and temperature of the illumination source change.

[0079] The XYZ tristimulus image can be represented as:

[0080] X R,G,B (px,py,I R,G,B ,T)

[0081] Y R,G,B (px,py,I R,G,B ,T)

[0082] Z R,G,B (px,py,I R,G,B ,T)

[0083] Where X, Y, and Z are tristimulus values, R refers to the red / display channel, G refers to the green / display channel, B refers to the blue / display channel, px and py are pixels in FoV, I is the illumination source drive current, and T is the characteristic temperature of the display or display device.

[0084] The electrical power used to drive the lighting source can be a function of current and voltage. The current-voltage relationship can be known, and P(I R ,I G ,I B T can be used to represent electrical power. L can be used... Out R,G,B (I R,G,B The relationship between the lighting source current, characteristic temperature, and average display brightness is used as a reference.

[0085] At step 404, global white balance is performed on the multiple images to obtain multiple normalized images (e.g., normalized image 452). Each of the multiple normalized images may correspond to one of the multiple color channels. In order to perform global white balance (or to perform global white balance on the display or display channel), in some embodiments, the average value of the tristimulus images of FoV may be oriented towards X. Ill Y Ill Z Ill A set of target illuminance values ​​454 are increased or decreased. For a D65 target white point (at 100 nits luminance), the target illuminance value 454 has tristimulus values:

[0086] X Ill =95.047

[0087] Y Ill =100

[0088] Z Ill =108.883

[0089] The average measured tristimulus values ​​for each color / display channel (under certain test conditions for current and temperature) can be calculated using the following:

[0090]

[0091]

[0092]

[0093] Next, the target brightness for each color / display channel can be solved using matrix equations:

[0094]

[0095] Using the global balanced brightness for each color / display channel, normalized image 452 can be calculated using normalized image 450 as follows:

[0096]

[0097]

[0098]

[0099] At step 406, local white balance is performed on the multiple normalized images to obtain multiple correction matrices (e.g., correction matrix 456). Each of the multiple correction matrices may correspond to one of the multiple color channels. To perform local white balance, the correction matrices can be optimized in a manner that minimizes the total power consumption used to achieve the global white balance brightness target.

[0100] At step 408, a set of weighting factors is defined (e.g., weighting factor 458), denoted as W. R,G,B Each of these weighting factors can correspond to one of multiple color channels. This set of weighting factors can be defined based on a quality factor (e.g., quality factor 464). During each iteration through loop 460, this set of weighting factors is used to bias the correction matrix to favor the color / display channel with the lowest efficiency. For example, if the red channel is substantially less efficient than green and blue, it is desirable for the red correction matrix to have a value of 1 across the entire FoV, while lower values ​​will be used in the correction matrices for the green and blue channels to achieve better local white balance.

[0101] At step 410, multiple weighted images (e.g., weighted image 466) are calculated based on multiple normalized images and the set of weighting factors. Each of the multiple weighted images may correspond to one of multiple color channels. The multiple weighted images may be represented as X. Opt R,G,B Y Opt R,G,B Z Opt R,G,B As shown in the example, except for the first iteration, weighting factor 458 can be used as the set of weighting factors during each iteration through loop 460, with the initial weighting factor 462 used during the first iteration. The resolution used for local white balance is a selectable parameter and does not need to match the resolution of the display device (e.g., SLM). In some embodiments, after calculating the correction matrix 456, an interpolation step may be added to match the size of the calculated correction matrix to the resolution of the SLM.

[0102] The weighted image 466 can be calculated as:

[0103] X OptR，G，B (cx, cy) = W R，G，B ·imresize(X NormR，G，B (cx, cy), [n] cx n cy ])

[0104] Y OptR，G，B (cx, cy) = W R，G，B ·imresize(Y NormR，G，B (cx, cy), [n] cx n cy ])

[0105] Z OptR，G，B (cx, cy) = W R，G，B ·imresize(Z NormR，G，B (cx, cy), [n] cx n cy ])

[0106] Where cx and cy are n cx and n cy The coordinates in the correction matrix of the element.

[0107] At step 412, multiple relative ratio maps (e.g., relative ratio 468) are calculated based on multiple weighted images and multiple target illumination values. Each of the multiple relative ratio maps may correspond to one of multiple color channels. The multiple relative ratio maps can be represented as l R (cx, cy), l G (cx, cy), l B (cx, cy). For each pixel (cx, cy) in the correction, the relative ratio of the color channels required to reach the target white point can be determined. Similar to the process used for global correction, the relative ratio 468 can be calculated as follows:

[0108]

[0109] Quantity l R,G,B This can be interpreted as the relative weights of the pixels required to achieve the target white balance (e.g., D65). Since global white balance correction has been performed, resulting in a normalized image 452, if the image is perfectly uniform across cx and cy, the relative ratio 468 will be calculated as l. R =l G =l B Due to the non-uniformity on cx and cy, in l R l G and l B There may be changes between them.

[0110] At step 414, multiple correction matrices are calculated based on multiple relative ratio maps. In some embodiments, the correction matrix for each color channel can be calculated at each pixel as follows:

[0111]

[0112] Using this definition of the correction matrix, at each point in cx,cy, the relative ratios of the red, green, and blue channels will correctly generate the target white point (e.g., D65). Furthermore, at least one color channel has a value of 1 at each cx,cy, which minimizes optical loss due to the correction of color inhomogeneity, resulting in a decrease in perceived brightness for the user.

[0113] At step 416, a quality factor (e.g., quality factor 464) is calculated based on multiple correction matrices and one or more quality factor inputs (e.g., quality factor input 470). The calculated quality factor is used in conjunction with step 408 to calculate this set of weighting factors through the next iteration of loop 460. As an example, one quality factor to be minimized is electrical power consumption. The optimization can be described as follows:

[0114] (W R W G W B ) = fmin(FOM(X) R，G，B Y R，G，B Z R，G，B L OutR，G，B (I R，G，B W R0 W G0 W B0 )

[0115] Where fmin is the multivariate optimization function, FOM is the quality factor function, and W... R0 W G0 W B0 These are weighting factors derived from previous iterations or initial estimates. During each iteration through loop 460, it can be determined whether the calculated quality factors have converged, in which case method 400 can exit loop 460 and output the correction matrix 456.

[0116] Examples of usable quality factors include: 1) Power consumption P(I) R ,I G ,I B ), 2) a combination of RMS color error and power consumption at the eye position (in which case the angular frequency of the low-pass filter in the correction matrix can be included in the optimization), and 3) a combination of power consumption, RMS color error and minimum bit depth, etc.

[0117] In many system configurations, a correction matrix can reduce the maximum bit depth of pixels in a display device. Lower values ​​of the correction matrix may result in lower bit depths, while a value of 1 will keep the bit depth unchanged. Additional constraints may be desired for operation in the linear state of an SLM. Noise may occur when devices such as LCoS exhibit unpredictable responses at lower or higher gray levels due to liquid crystal (LC) switching (which is attributed to the dynamic optical response of the LC to the electronic video signal), temperature effects, or electronic noise. Constraints can be placed on the correction matrix to avoid reducing the bit depth below a desired threshold or operating in an undesirable state of the SLM, and the impact of RMS color error can be included in the optimization.

[0118] In some embodiments, global white balance can be re-performed, and the newly generated correction matrix can be applied to calculate the required source current. The target luminance L for each channel was previously calculated. R,G,B However, the efficiency η attributable to the correction matrix can be applied. CorrectionR，G，B The effective efficiency can be calculated as follows:

[0119]

[0120] The · operator represents element-wise multiplication.

[0121] The relationship between the luminance curve and current (and, if necessary, temperature), also known as the luminance response 472, can be updated using the following:

[0122] L CorrectedR，G，B =η CorrectionR，G，B L OutR，G，B (I R，G，B )

[0123] The target D65 luminance value L for each color channel, which was previously defined, can now be found from the luminance response 472. R,G,B Required current I R,G,B The luminance response 472 includes L CorrectedR，G，B vs I R，G，B Curve. Given the current, the efficiency of each color channel and the total power consumption P(I) can also be found. R ,I G ,I B ).

[0124] In some embodiments, once the optimal weighting factors are found, the same method described above can be followed one last time to generate the optimal correction matrix. Using L... CorrectedR，G，B (I R，G，B The system can perform global white balance to obtain the illumination source current required for all operating temperatures and target display illuminance.

[0125] In some embodiments, a matrix equation, such as that used for performing global white balance, can be used to determine the desired luminance L for each color channel. CorrectedR，G，B However, it is now possible to display the target brightness L. Target To scale the target white point tristimulus value X Ill Y Ill Z Ill For D65 white spots, this results in:

[0126] X Ill (L Target ) = 0.95047L Target

[0127] Y Ill (L Target )= LTar get

[0128] Z Ill (L Target = 1.08883L Target

[0129] Other target white dots can change X Ill Y Ill Z Ill The value of L. Now, L CorrectedR，G，B The solution can be found as follows:

[0130]

[0131] in It is the previously defined average tristimulus value for each display color channel.

[0132] The display shows data related to brightness, current, and temperature, generated by the function L. CorrectedR，G，B (I R，G，B (T) is known, and it can be included in the luminance response 472. This information can also be represented as I. R，G，B (L CorrectedR，G，B (T), which can be included in the luminance response 472. Using this and the result from the matrix equation above, we produce as L Target Source current I as a function of temperature R，G，B (L Target T).

[0133] At step 418, it is determined that the representation is L. Target The target brightness of the display (e.g., target brightness 472). In some embodiments, target brightness 472 can be determined by benchmarking the brightness of the wearable device against typical monitor brightness (e.g., against a desktop monitor or television).

[0134] At step 420, a luminance response (e.g., luminance response 472) between the target luminance and the display's luminance and current (and optionally temperature) is determined based on the luminance response (e.g., luminance response 472). R,G,B Multiple target source currents (e.g., target source current 474). In some embodiments, target source current 474 and correction matrix 456 are the outputs of method 400.

[0135] Various techniques can be employed to address the eye position dependence of the correction matrix 456. In a first approach, a low-pass filter can be applied to the correction matrix to reduce sensitivity to eye position. The angular frequency cutoff of the filter can be optimized for a given display. A Gaussian filter with σ = 2-10° might be a suitable range for such a filter. In a second approach, images can be acquired at multiple eye positions using a camera with a coarse 4mm incident pupil diameter, and the average value can be used to generate an effective eyebox image. The eyebox image can be used to generate a correction matrix that will be less sensitive to eye position than an image taken at a specific eye position.

[0136] In the third method, images can be acquired using a camera with an entrance pupil diameter as large as the designed eyebox (~10-20mm). Furthermore, eyebox images can produce a correction matrix that is less sensitive to eye position compared to images taken at a specific eye location with an entrance pupil of 4mm. In the fourth method, images can be acquired using a camera with an entrance pupil diameter of approximately 4mm located at the nominal center of the user's eye rotation to reduce the sensitivity of color uniformity correction in the FoV portion of the user's gaze to eye rotation. In the fifth method, images can be acquired at multiple eye locations using a camera with an entrance pupil diameter of approximately 4mm. A separate correction matrix can be generated for each camera location. These corrections can be used to apply eye position-related color correction using eye-tracking information from the wearable system.

[0137] Figure 5 Examples of improved color uniformity using methods 300 and 400 according to some embodiments of this disclosure are shown. In the illustrated example, a color uniformity correction algorithm is applied to an LED-illuminated LCOS SLM diffractive waveguide display system. The FoV of the image corresponds to 45° × 55°. A Gaussian filter with σ = 5° is applied to the correction matrix to reduce eye position sensitivity. The quality factor used in minimizing the optimization function is electrical power consumption. Both images were taken using a camera with a 4mm entrance pupil. The RMS color errors before and after performing the color uniformity correction algorithm are 0.0396 and 0.0191, respectively. Figure 5 The left and right sides respectively show the uncorrected and corrected images demonstrating improved color uniformity. Figure 5 This includes color features that have been converted to grayscale for reproduction purposes.

[0138] Figure 6 Some embodiments of the present disclosure are shown for use with Figure 5The example shown is a set of error histograms. Each error histogram shows the number of pixels in each of a set of error ranges in both the uncorrected and corrected images. The error is the u'v' error of D65 on the pixels within the FoV. The example shown demonstrates that applying correction significantly reduces color error.

[0139] Figure 7 An example correction matrix 700, considered as an RGB image according to some embodiments of this disclosure, is shown. The correction matrix 700 may consist of three separate correction matrices C. R,G,B The superposition of these factors. In the example shown, correction matrix 700 demonstrates that different color channels can exhibit varying levels of inhomogeneity along different areas of the display. Figure 7 This includes color features that have been converted to grayscale for reproduction purposes.

[0140] Figure 8 Examples of luminance uniformity patterns for a display color channel are shown according to some embodiments of the present disclosure. Each image corresponds to a 45° × 55° FoV image taken at different eye positions within the eye box of a single display color channel. (See also...) Figure 8 As can be observed, the brightness uniformity pattern can depend on the eye position in multiple directions.

[0141] Figure 9 A method 900 for improving color uniformity of a display for multiple eye positions within an eye box (or eye box location) according to some embodiments of the present disclosure is illustrated. One or more steps of method 900 may be omitted during execution of method 900, and the steps of method 900 need not be performed in the order shown. One or more steps of method 900 may be executed by one or more processors. Method 900 may be implemented as a computer-readable medium or computer program product including instructions that, when executed by one or more computers, cause one or more computers to perform the steps of method 900. The steps of method 900 may include and / or may be used in combination with one or more steps of various other methods described herein.

[0142] At step 902, an image capture device is used to capture a first plurality of images of the display. The first plurality of images can be captured at a first eye position within the eye box.

[0143] At step 904, global white balance is performed on the first plurality of images to obtain the first plurality of normalized images.

[0144] At step 906, local white balance is performed on the first plurality of normalized images to obtain a first plurality of correction matrices and optionally a first plurality of target source currents, which may be stored in a memory device.

[0145] At step 908, the position of the image capture device relative to the display is changed. During subsequent iterations through steps 902 to 906, a second plurality of images of the display are captured at a second eye position within the eye box, and local white balance is performed on the second plurality of normalized images to obtain a second plurality of correction matrices and optionally a second plurality of target source currents, which may be stored in a memory device. Similarly, during subsequent iterations through steps 902 to 906, a third plurality of images of the display are captured at a third eye position within the eye box, and local white balance is performed on the third plurality of normalized images to obtain a third plurality of correction matrices and optionally a third plurality of target source currents, which may be stored in a memory device.

[0146] Figure 10 A method 1000 for improving color uniformity of a display for multiple eye positions within an eye box (or eye box location) according to some embodiments of the present disclosure is illustrated. One or more steps of method 1000 may be omitted during execution of method 1000, and the steps of method 1000 need not be performed in the order shown. One or more steps of method 1000 may be performed by one or more processors. Method 1000 may be implemented as a computer-readable medium or computer program product including instructions that, when executed by one or more computers, cause one or more computers to perform the steps of method 1000. The steps of method 100 may include and / or may be used in conjunction with one or more steps of various other methods described herein.

[0147] At step 1002, an image of the user's eyes is captured using an image capture device. The image capture device may be an eye-facing camera of a wearable device.

[0148] In step 1004, the position of the eye within the eye box is determined based on the image of the eye.

[0149] At step 1006, multiple correction matrices are retrieved based on the position of the eye within the eye box. For example, the multiple correction matrices corresponding to multiple eye positions can be stored in a memory device, such as a reference. Figure 9 As described, multiple correction matrices corresponding to the eye positions closest to the determined eye positions can be retrieved. Optionally, at step 1006, multiple target source currents are also retrieved based on the position of the eye within the eye box. For example, multiple sets of target source currents corresponding to multiple eye positions can be stored in a memory device, such as a reference. Figure 9 As described, multiple target source currents corresponding to the eye positions closest to the determined eye position can be retrieved.

[0150] At step 1008, corrections are applied to the video sequence and / or image to be displayed using the multiple correction matrices retrieved at step 1006. In some embodiments, the corrections can be applied to the video sequence before it is sent to the SLM. In some embodiments, the corrections can be applied to the settings of the SLM. Other possibilities are anticipated.

[0151] At step 1010, the multiple source currents associated with the display are set to the multiple target source currents retrieved at step 1006.

[0152] Figure 11 Examples of improved color uniformity for multiple eye positions using various methods described herein are shown. In the example shown, a color uniformity correction algorithm is applied to an LED-illuminated LCOS SLM diffractive waveguide display system. Figure 11 The left and right sides respectively show the uncorrected and corrected images demonstrating improved color uniformity. Figure 11 This includes color features that have been converted to grayscale for reproduction purposes.

[0153] Figure 12 A method 1200 for determining and setting the source current of a display device according to some embodiments of the present disclosure is illustrated. One or more steps of method 1200 may be omitted during execution of method 1200, and the steps of method 1200 do not need to be performed in the order shown. One or more steps of method 1200 may be executed by one or more processors. Method 1200 may be implemented as a computer-readable medium or computer program product including instructions that, when executed by one or more computers, cause one or more computers to perform the steps of method 1200. The steps of method 1200 may include and / or may be used in combination with one or more steps of various other methods described herein.

[0154] At step 1202, multiple images of the display are captured using an image capture device. Each of the multiple images may correspond to one of the multiple color channels.

[0155] At step 1204, the multiple images are averaged on FoV.

[0156] At step 1206, the brightness response of the display is measured.

[0157] At step 1208, multiple correction matrices are output. In some embodiments, multiple correction matrices are output using a color correction algorithm.

[0158] At step 1210, multiple correction matrices are used to adjust the brightness response.

[0159] At step 1212, the target white point is determined.

[0160] In step 1214, the target display brightness is determined.

[0161] In step 1216, the required display channel brightness is determined based on the target white point and the target display brightness.

[0162] In step 1218, the temperature of the display is determined.

[0163] At step 1220, multiple target source currents are determined based on the brightness response, the required display channel brightness, and / or temperature.

[0164] At step 1222, the multiple source currents are set as multiple target source currents.

[0165] Figure 13 A schematic diagram of an example wearable system 1300, which may be used in one or more of the embodiments described above, according to some embodiments of the present disclosure, is shown. The wearable system 1300 may include a wearable device 1301 and at least one remote device 1303, which is remote from the wearable device 1301 (e.g., separate hardware, but communicatively coupled). When the wearable device 1301 is worn by a user (typically as a headband device), the remote device 1303 may be held by the user (e.g., as a handheld controller) or mounted in various configurations, such as being fixedly attached to a frame, fixedly attached to a helmet or hat worn by the user, embedded in headphones, or otherwise removably attached to the user (e.g., in a backpack configuration, in a belt-coupled configuration, etc.).

[0166] Wearable device 1301 may include a left eyepiece 1302A and a left lens assembly 1305A arranged side-by-side and forming a left optical stack. The left lens assembly 1305A may include an accommodating lens on the user side of the left optical stack and a compensation lens on the world side of the left optical stack. Similarly, wearable device 1301 may include a right eyepiece 1302B and a right lens assembly 1305B arranged side-by-side and forming a right optical stack. The right lens assembly 1305B may include an accommodating lens on the user side of the right optical stack and a compensation lens on the world side of the right optical stack.

[0167] In some embodiments, the wearable device 1301 includes one or more sensors, including but not limited to: a world camera 1306A facing left front, which is directly attached to or near the left eyepiece 1302A; a world camera 1306B facing right front, which is directly attached to or near the right eyepiece 1302B; a world camera 1306C facing left side, which is directly attached to or near the left eyepiece 1302A; a world camera 1306D facing right side, which is directly attached to or near the right eyepiece 1302B; a left eye tracking camera 1326A pointing towards the left eye; a right eye tracking camera 1326B pointing towards the right eye; and a depth sensor 1328 attached between the eyepieces 1302. Wearable device 1301 may include one or more image projection devices, such as a left projector 1314A optically linked to a left eyepiece 1302A, and a right projector 1314B optically linked to a right eyepiece 1302B.

[0168] Wearable system 1300 may include a processing module 1350 for collecting, processing, and / or controlling data within the system. Components of processing module 1350 may be distributed between wearable device 1301 and remote device 1303. For example, processing module 1350 may include a local processing module 1352 on the wearable portion of wearable system 1300 and a remote processing module 1356 physically separate from and communicatively linked to local processing module 1352. Each of local processing module 1352 and remote processing module 1356 may include one or more processing units (e.g., central processing unit (CPU), graphics processing unit (GPU), etc.) and one or more storage devices, such as non-volatile memory (e.g., flash memory).

[0169] The processing module 1350 can collect data captured by various sensors of the wearable system 1300, such as camera 1306, eye-tracking camera 1326, depth sensor 1328, remote sensor 1330, ambient light sensor, microphone, inertial measurement unit (IMU), accelerometer, compass, global navigation satellite system (GNSS) unit, radio equipment, and / or gyroscope. For example, the processing module 1350 can receive one or more images 1320 from camera 1306. Specifically, the processing module 1350 can receive one or more left-front images 1320A from the left-front-facing world camera 1306A, one or more right-front images 1320B from the right-front-facing world camera 1306B, one or more left-side images 1320C from the left-facing world camera 1306C, and one or more right-side images 1320D from the right-facing world camera 1306D. In some embodiments, one or more images 1320 may include a single image, a pair of images, a video including an image stream, a video including a pair of image streams, etc. One or more images 1320 may be generated periodically and sent to the processing module 1350 when the wearable system 1300 is powered on, or may be generated in response to an instruction sent by the processing module 1350 to one or more cameras.

[0170] Camera 1306 can be configured in various positions and orientations along the outer surface of wearable device 1301 to capture images of the user's surroundings. In some instances, cameras 1306A and 1306B can be positioned, respectively, to capture images substantially overlapping with the user's left and right eye FOVs. Thus, the arrangement of cameras 1306 can be near the user's eyes, but not so close as to blur the user's FOV. Alternatively or additionally, cameras 1306A and 1306B can be positioned, respectively, to align with the coupling positions of virtual image lights 1322A and 1322B. Cameras 1306C and 1306D can be positioned to capture images on one side of the user, for example, within or outside the user's peripheral vision. Images 1320C and 1320D captured using cameras 1306C and 1306D do not necessarily need to overlap with images 1320A and 1320B captured using cameras 1306A and 1306B.

[0171] In some embodiments, processing module 1350 may receive ambient light information from an ambient light sensor. The ambient light information may indicate a luminance value or a range of spatially resolved luminance values. Depth sensor 1328 may capture a depth image 1332 in the forward-facing direction of wearable device 1301. Each value of depth image 1332 may correspond to the distance between depth sensor 1328 and the most recently detected object in a particular direction. As another example, processing module 1350 may receive eye-tracking data 1334 from eye-tracking camera 1326, which may include images of the left and right eyes. As another example, processing module 1350 may receive projected image luminance values ​​from one or both of projectors 1314. Remote sensor 1330 located within remote device 1303 may include any sensor described above with similar functionality.

[0172] The projector 1314 and eyepiece 1302, along with other components in the optical stack, deliver virtual content to the user of the wearable system 1300. For example, eyepieces 1302A and 1302B may each include a transparent or semi-transparent waveguide configured to guide and couple light generated by projectors 1314A and 1314B. Specifically, processing module 1350 may cause the left projector 1314A to output left virtual image light 1322A to the left eyepiece 1302A, and may cause the right projector 1314B to output right virtual image light 1322B to the right eyepiece 1302B. In some embodiments, projector 1314 may include a microelectromechanical system (MEMS) SLM scanning device. In some embodiments, each of eyepieces 1302A and 1302B may include multiple waveguides corresponding to different colors. In some embodiments, lens assemblies 1305A and 1305B may be coupled to and / or integrated with eyepieces 1302A and 1302B. For example, lens assemblies 1305A and 1305B may be included in a multilayer eyepiece and may form one or more layers constituting one of eyepieces 1302A and 1302B.

[0173] Figure 14 A simplified computer system according to some embodiments of the present disclosure is shown. For example... Figure 14 The computer system 1400 shown may be included in the device described herein. Figure 14 A schematic diagram of one embodiment of a computer system 1400 capable of performing some or all of the steps of the methods provided by various embodiments is provided. It should be noted that... Figure 14 This is intended only to provide a generalized illustration of the various components that may be used at discretion in any or all of them. Therefore, Figure 14 It broadly demonstrates how individual system components can be implemented in a relatively discrete or relatively more integrated manner.

[0174] A computer system 1400 is shown, comprising hardware elements that can be electrically coupled via bus 1405 or otherwise communicated as appropriate. The hardware elements may include one or more processors 1410, including but not limited to one or more general-purpose processors and / or one or more special-purpose processors, such as digital signal processing chips, graphics accelerator processors, and / or the like; one or more input devices 1415, which may include, but are not limited to, a mouse, keyboard, camera, and / or the like; and one or more output devices 1420, which may include, but are not limited to, display devices, printers, and / or the like.

[0175] Computer system 1400 may also include one or more non-transitory storage devices 1425 and / or communicate with one or more non-transitory storage devices 1425, which may include, but are not limited to, local and / or network access storage, and / or may include, but are not limited to, disk drives, drive arrays, optical storage devices, solid-state storage devices, such as random access memory (“RAM”) and / or read-only memory (“ROM”), which may be programmable, flash-updateable, and / or the like. Such storage devices can be configured to implement any suitable data storage, including but not limited to various file systems, database structures, and / or the like.

[0176] Computer system 1400 may also include a communication subsystem 1419, which may include, but is not limited to, a modem, a network card (wireless or wired), an infrared communication device, a wireless communication device, and / or a chipset such as Bluetooth. TM Devices, 802.11 devices, WiFi devices, WiMax devices, cellular communication facilities, and / or the like. The communication subsystem 1419 may include one or more input and / or output communication interfaces that allow data exchange with networks (such as those described below, to name only), other computer systems, televisions, and / or any other devices described herein. Depending on desired functionality and / or other implementation issues, portable electronic devices or similar devices may transmit images and / or other information via the communication subsystem 1419. In other embodiments, a portable electronic device (e.g., a first electronic device) may be included in the computer system 1400, for example, as an electronic device serving as an input device 1415. In some embodiments, the computer system 1400 will also include working memory 1435, which may include RAM or ROM devices as described above.

[0177] Computer system 1400 may also include software elements, shown as currently residing within working memory 1435, including operating system 1440, device drivers, executable libraries, and / or other code, such as one or more application programs 1445, which may include computer programs provided by various embodiments and / or may be designed to implement methods and / or configure systems provided by other embodiments, as described above. By way of example only, one or more programs described with respect to the methods discussed above may be implemented as code and / or instructions executable by a computer and / or a processor within a computer; in this respect, such code and / or instructions may then be used to configure and / or adapt a general-purpose computer or other device to perform one or more operations according to the described methods.

[0178] These sets of instructions and / or code may be stored on a non-transitory computer-readable storage medium, such as storage device(s) 1425 described above. In some cases, the storage medium may be contained within a computer system, such as computer system 1400. In other embodiments, the storage medium may be separate from the computer system, for example, a removable medium, such as an optical disc, and / or provided in an installation package, such that the storage medium can be used to program, configure, and / or adapt to a general-purpose computer using the instructions / code stored thereon. These instructions may take the form of executable code that can be executed by computer system 1400 and / or may take the form of source and / or installable code, which is based on compilation and / or installation on computer system 1400, for example, using any generally available compiler, installer, compression / decompression utility, etc., and then takes the form of executable code.

[0179] It will be apparent to those skilled in the art that substantial modifications can be made to suit specific requirements. For example, custom hardware may also be used, and / or specific components may be implemented in hardware, software (including portable software such as applets), or both. Furthermore, connectivity to other computing devices, such as network input / output devices, may be employed.

[0180] As mentioned above, in one aspect, some embodiments may use a computer system (such as computer system 1400) to perform methods according to various embodiments of the art. According to one set of embodiments, some or all of the procedures of such a method are executed by computer system 1400 in response to processor 1410 executing one or more sequences of one or more instructions, which may be contained in operating system 1440 and / or other code, such as application program 1445 contained in working memory 1435. Such instructions may be read into working memory 1435 from another computer-readable medium (such as one or more storage devices 1425). By way of example only, execution of a sequence of instructions contained in working memory 1435 may cause processor(s) 1410 to perform one or more procedures of the methods described herein. Additionally or alternatively, portions of the methods described herein may be executed by dedicated hardware.

[0181] As used herein, the terms “machine-readable medium” and “computer-readable medium” refer to any medium that participates in providing data that causes a machine to operate in a particular manner. In embodiments implemented using computer system 1400, various computer-readable media may relate to providing instructions / code to processor(s) ...(s))(s)(s)(s)

[0182] Common forms of physical and / or tangible computer-readable media include, for example, floppy disks, hard disks, magnetic tapes, or any other magnetic media, CD-ROMs, any other optical media, punch cards, paper tapes, any other physical media with a pattern of holes, RAM, PROMs, EPROMs, FLASH-EPROMs, any other memory chips or tape cassettes, or any other media from which a computer can read instructions and / or code.

[0183] Various forms of computer-readable media may involve carrying one or more sequences of one or more instructions to processor(s)1410 for execution. For example only, the instructions may initially be carried on a disk and / or optical disk of a remote computer. The remote computer may load the instructions into its dynamic memory and transmit them as signals via a transmission medium for reception and / or execution by computer system 1400.

[0184] The communication subsystem 1419 and / or its components will typically receive signals, and the bus 1405 may then carry the signals and / or data, instructions, etc., carried by the signals to the working memory 1435, from which the processor(s) 1410 retrieves and executes the instructions. The instructions received by the working memory 1435 may optionally be stored on the non-transitory storage device 1425 before or after execution by the processor(s) 1410.

[0185] The methods, systems, and apparatus discussed above are examples. Various configurations may omit, substitute, or add various processes or components as appropriate. For example, in alternative configurations, methods may be performed in a different order than those described, and / or stages may be added, omitted, and / or combined. Moreover, features described with respect to certain configurations may be combined in various other configurations. Different aspects and elements of configurations may be combined in a similar manner. Furthermore, technology evolves, and therefore many elements are examples and do not limit the scope of this disclosure or the claims.

[0186] Specific details are given in the description, which provides a thorough understanding of exemplary configurations (including implementations). However, the configurations may be practiced without these specific details. For example, well-known circuits, processes, algorithms, structures, and techniques are shown without unnecessary detail to avoid obscuring the configurations. This specification provides only exemplary configurations and does not limit the scope, applicability, or configuration of the claims. Rather, the foregoing description of the configurations will provide those skilled in the art with an enabling description for implementing the described techniques. Various changes may be made to the function and arrangement of the elements without departing from the spirit or scope of this disclosure.

[0187] Furthermore, a configuration can be described as a process depicted as a schematic flowchart or block diagram. While each configuration can be described as a sequential process of operations, many operations can be executed in parallel or concurrently. Additionally, the order of operations can be rearranged. The process may have additional steps not included in the diagram. Moreover, examples of the method can be implemented using hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware, or microcode, the program code or code segments that perform the necessary tasks can be stored in a non-transitory computer-readable medium, such as a storage medium. The processor can execute the described tasks.

[0188] Several example configurations have been described, and various modifications, alternative constructions, and equivalents may be used without departing from the spirit of this disclosure. For example, the above elements may be components of a larger system, in which other rules may prevail over or otherwise modify the application of the technology. Moreover, multiple steps may be taken before, during, or after considering the above elements. Therefore, the above description is not confined to the scope of the claims.

[0189] As used herein and in the appended claims, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” include plural references. Thus, for example, a reference to “user” includes a plurality of such users, and a reference to “processor” includes one or more processors and their equivalents known to those skilled in the art, and so on.

[0190] Furthermore, when used in this specification and in the following claims, the words “comprising,” “including,” “accommodating,” “containing,” “comprise,” “containing,” “comprising,” “including,” and “containing” are intended to specify the presence of a descriptive feature, integral, component, or step, but do not exclude the presence or addition of one or more other features, integrals, components, steps, actions, or combinations.

[0191] It should also be understood that the examples and embodiments described herein are for illustrative purposes only and will be suggested to those skilled in the art for various modifications or alterations to be made thereto, and will be included within the spirit and scope of this application and the scope of the claims.

Claims

1. A method for improving the color uniformity of a display, the method comprising: An image capture device is used to capture multiple images of the display of the display device, wherein the multiple images are captured in a color space, and wherein each of the multiple images corresponds to one color channel of a plurality of color channels; Global white balance is performed on the multiple images to obtain multiple normalized images, each normalized image corresponding to one of the multiple color channels; Performing local white balance on the plurality of normalized images to obtain a plurality of correction matrices, each correction matrix corresponding to one of the plurality of color channels, wherein performing the local white balance includes: Based on the quality factor, a set of weighting factors is defined, wherein the quality factor is at least one of the following: power consumption; color error; or minimum bit depth, and wherein defining the set of weighting factors includes: minimizing the quality factor by changing the set of weighting factors; and determining the set of weighting factors that minimizes the quality factor. Based on the multiple normalized images and the set of weighting factors, calculate multiple weighted images; and Based on the multiple weighted images, calculate the multiple correction matrices; and The plurality of correction matrices are used to correct the video sequence to be displayed on the display device.

2. The method according to claim 1, wherein, The color space is one of the following: CIELUV color space; CIEXYZ color space; or sRGB color space.

3. The method according to claim 1, wherein, Performing the global white balance on the plurality of images includes: The target illuminance value in the color space is determined based on the target white point, wherein the plurality of normalized images are calculated based on the target illuminance value.

4. The method according to claim 3, wherein, The plurality of correction matrices are then calculated based on the target illuminance value.

5. The method according to claim 1, wherein, The display is a diffractive waveguide display.

6. A non-transitory computer-readable medium comprising instructions, said instructions, when executed by one or more processors, causing said one or more processors to perform operations including: Use an image capture device to capture multiple images of the display device's monitor, where... The plurality of images are captured in a color space, and each of the plurality of images corresponds to one color channel in a plurality of color channels; Global white balance is performed on the multiple images to obtain multiple normalized images, each normalized image corresponding to one of the multiple color channels; Performing local white balance on the plurality of normalized images yields a plurality of correction matrices, each correction matrix corresponding to one of the plurality of color channels. The local white balance process includes: Based on the quality factor, a set of weighting factors is defined, wherein the quality factor is at least one of the following: power consumption; color error; or minimum bit depth, and wherein defining the set of weighting factors includes: minimizing the quality factor by changing the set of weighting factors; and determining the set of weighting factors that minimizes the quality factor; Based on the multiple normalized images and the set of weighting factors, calculate multiple weighted images; and Based on the multiple weighted images, calculate the multiple correction matrices; and The plurality of correction matrices are used to correct the video sequence to be displayed on the display device.

7. The non-transitory computer-readable medium according to claim 6, wherein, The color space is one of the following: CIELUV color space; CIEXYZ color space; or sRGB color space.

8. The non-transitory computer-readable medium according to claim 6, wherein, Performing the global white balance on the plurality of images includes: The target illuminance value in the color space is determined based on the target white point, wherein the plurality of normalized images are calculated based on the target illuminance value.

9. The non-transitory computer-readable medium according to claim 8, wherein, The plurality of correction matrices are then calculated based on the target illuminance value.

10. The non-transitory computer-readable medium according to claim 6, wherein, The display is a diffractive waveguide display.

11. A system comprising: One or more processors; as well as A non-transitory computer-readable medium including instructions that, when executed by the one or more processors, cause the one or more processors to perform operations including: An image capture device is used to capture multiple images of the display of a display device, wherein the multiple images are captured in a color space, and wherein each of the multiple images corresponds to one color channel of a plurality of color channels; Global white balance is performed on the multiple images to obtain multiple normalized images, each normalized image corresponding to one of the multiple color channels; Performing local white balance on the plurality of normalized images yields a plurality of correction matrices, each correction matrix corresponding to one of the plurality of color channels. The local white balance process includes: Based on the quality factor, a set of weighting factors is defined, wherein the quality factor is at least one of the following: power consumption; color error; or minimum bit depth, and wherein defining the set of weighting factors includes: minimizing the quality factor by changing the set of weighting factors; and determining the set of weighting factors that minimizes the quality factor; Based on the multiple normalized images and the set of weighting factors, calculate multiple weighted images; and Based on the multiple weighted images, calculate the multiple correction matrices; and The plurality of correction matrices are used to correct the video sequence to be displayed on the display device.

Images