Method for virtual image compensation

Abstract

A method is provided for compensating a virtual image displayed by a near-eye display based on a source image, the method including: obtaining virtual images to be displayed by the near-eye display for three primary color channels, each of the virtual images being based on a primary color test pattern; obtaining a correction coefficient matrix including luma and chroma components of the three primary color channels; and performing compensation on the source image using the correction coefficient matrix.

Description

[Technical Field]

[0001] FIELD OF THE DISCLOSURE The present disclosure relates generally to microdisplay technology, and more particularly to a method for virtual image compensation. [Background technology]

[0002] Near-eye displays can be provided as augmented reality (AR) displays, virtual reality (VR) displays, head-up / head-mounted displays, or other displays. In general, near-eye displays typically include an image generator and an optical combiner that transfers a projected image from the image generator to the human eye. The optical combiner is a group of reflective and / or diffractive optical elements, such as a freeform mirror / prism, a birdbath, or a cascade mirror, and / or a grating coupler (waveguide). Furthermore, the projected image is a virtual image in front of the human eye. The image generator can be a microLED-based display, an LCOS (liquid crystal on silicon) display, or a DLP (digital light processing) display. The virtual image is rendered from the image generator and the optical combiner to the human eye.

[0003] Uniformity is one important performance metric for displays used to evaluate image quality. Uniformity usually refers to defects in the display matrix, also called non-uniformity. Non-uniformity includes variations in global distribution and local zones, also called mura. In the case of near-eye displays, such as AR / VR, visual artifacts, such as the appearance of mottled, bright, or black spots or darkness, may also be observed on virtual images rendered in display systems. In virtual images rendered in AR / VR displays, non-uniformity may be exhibited in luminance and / or chromaticity. Compared with traditional displays, non-uniformity artifacts are much more obvious due to their proximity to the human eye. Therefore, a method for improving virtual image quality is desirable. Summary of the Invention

[0004] An embodiment of the present disclosure provides a method for compensating a virtual image displayed by a near-eye display based on a source image, the method including: acquiring virtual images to be displayed by the near-eye display for three primary color channels, each of the virtual images being based on a primary color test pattern; obtaining a correction coefficient matrix including luma and chroma components of the three primary color channels; and performing compensation on the source image using the correction coefficient matrix.

[0005] An embodiment of the present disclosure provides a method for compensating a virtual image displayed by a near-eye display based on a source image, the method including: acquiring virtual images to be displayed by the near-eye display for three primary color channels, each of the virtual images based on a primary color test pattern; obtaining first chromaticity components for each of the primary color channels; determining target chromaticity components for each of the primary color channels based on the first chromaticity components; obtaining a correction coefficient matrix based on the target chromaticity components; and performing compensation on the source image based on the correction coefficient matrix.

[0006] An embodiment of the present disclosure provides a method for compensating a virtual image displayed by a near-eye display based on a source image, the method including: acquiring virtual images to be displayed by the near-eye display for three primary color channels, each of the virtual images being based on a primary color test pattern; obtaining an image data matrix for the three primary color channels based on the virtual image; inverting the image data matrix to obtain an inverted image data matrix; determining a target image data matrix for the three primary color channels; obtaining a correction coefficient matrix by multiplying the inverted image data matrix and the target image data matrix; and performing compensation on the source image based on the correction coefficient matrix.

[0007] Embodiments and various aspects of the present disclosure are illustrated in the following detailed description and accompanying drawings, in which various features are not drawn to scale. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 illustrates a framework for an equalization method for improving image quality, according to some embodiments of the present disclosure. [Figure 2] FIG. 1 illustrates another framework of an equalization method for improving image quality using three primary color channels, according to some embodiments of the present disclosure. [Figure 3] FIG. 1 illustrates a flowchart of an exemplary method for compensating for a virtual image, according to some embodiments of the present disclosure. [Figure 4A-C] FIG. 10 illustrates an example distribution of luminance and chromaticity for the green primary color channel, according to some embodiments of the present disclosure. [Figure 5] FIG. 2 illustrates an exemplary pixel distribution in color space and a color triangle, according to some embodiments of the present disclosure. [Figure 6A-B] 10A-10C are diagrams illustrating a nine-point color coordinator before and after Demura in color space, according to some embodiments of the present disclosure. [Figure 7A-B] 10A-10C are diagrams illustrating a nine-point color coordinator before and after Demura in another color space, according to some embodiments of the present disclosure. [Figure 8] FIG. 1 illustrates a flowchart of an exemplary pixel alignment method, in accordance with some embodiments of the present disclosure. [Figure 9] 1A-1C illustrate exemplary determined regions of interest from image data, according to some embodiments of the present disclosure. [Figure 10] 1A-1C illustrate exemplary images after distortion correction, according to some embodiments of the present disclosure. [Figure 11] FIG. 10 illustrates an example of pixel alignment from a virtual image to an image source using a mapping ratio of 5, according to some embodiments of the present disclosure. [Figure 12] FIG. 10 illustrates an example of a preprocessed image (640×480) in pixel alignment, according to some embodiments of the present disclosure. [Figure 13] FIG. 10 illustrates an exemplary 3×3 partial on-off positioning pattern, according to some embodiments of the present disclosure. [Figure 14] FIG. 1 shows a flowchart illustrating a target image data determination method according to some embodiments of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0009] Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. The following description refers to the accompanying drawings in which like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following description of exemplary embodiments do not represent all implementations consistent with the present invention. Instead, the implementations are merely examples of apparatus and methods consistent with aspects related to the present invention as recited in the appended claims. Particular aspects of the present disclosure are described in more detail below. In the event of a conflict with terms and / or definitions incorporated by reference, the terms and definitions provided herein shall control.

[0010] The International Commission on Illumination (known in English as the "Commission Internationale de I'eclaireage" in French) defined the CIE-1931 standard for quantifying the physical properties of color as perceived by the human eye. The CIE-1931 color spaces quantitatively define the connection between the distribution of wavelengths in the electromagnetic visible spectrum and the physiologically perceived colors of human color vision. The CIE-1931 mathematical relationships that define these color spaces are tools for color management, used, for example, when dealing with color inks, lighting displays, and recording devices such as digital cameras.

[0011] The CIE-1931 color space includes the CIE-RGB color space and the CIE-XYZ color space. The CIE-RGB color space is one of many RGB color spaces distinguished by a specific set of nonchromatic (single-wavelength) primaries. The CIE-XYZ color space encompasses all color sensations visible to humans with average vision. Therefore, CIE-XYZ (tristimulus values) is a device-invariant representation of color. CIE-XYZ serves as a standard reference against which many other color spaces are defined. The CIE-1931 color space defines the resulting tristimulus values, denoted by X, Y, and Z. Y is luminance, Z is quasi-equivalent to blue (in CIE-RGB), and X is a blend of the three CIE-RGB curves chosen to be nonnegative. Setting Y as luminance has the useful consequence that for any given Y value, the XZ plane contains all possible chromaticities at that luminance.

[0012] Because the human eye has three types of color sensors that respond to different ranges of wavelengths, a full plot of all visible colors would be a three-dimensional diagram. However, the concept of color can be divided into two parts: luminance and chromaticity. The CIE-XYZ color space was carefully designed so that the Y parameter is also a measure of the color's luminance. Chromaticity, in this case, is specified by two derived parameters, x and y, which are two of three normalized values ​​that are functions of all three tristimulus values ​​x, y, and z according to the following: TIFF2026504225000002.tif31170

[0013] The derived color space defined by x, y, and Y is known as the CIE-xyY color space. The X and Z tristimulus values ​​can be back-calculated from the chromaticity values ​​x and y and the Y tristimulus value according to the following formula: TIFF2026504225000003.tif32170

[0014] In colorimetry, the CIE-1976 L*,u*,v* color space, commonly known by its abbreviation CIE-LUV, is a color space adopted by the CIE in 1976 as an easily computed transformation of the 1931 CIE-XYZ color space that attempted to achieve perceptual uniformity. There are three components represented in the CIE-1976 color space: a luminance component (L) and two chrominance components, u and v.

[0015] The gray level or gray value of an image indicates the brightness of a pixel. The minimum gray value is 0. The maximum gray value depends on the digitization depth of the image. For example, for an 8-bit deep image, the maximum gray value is 255. In a color image, the gray value of each pixel can be calculated using the three primary color components as follows: Gray value = 0.299 × R + 0.587 × G + 0.114 × B (Equation 3)

[0016] where R refers to the red component, G refers to the green component, and B refers to the blue component. The formula takes into account the color sensitivity of the human eye, making the presentation of gray levels color-independent and limited only to the luminance of the individual pixels.

[0017] Non-uniformity can be compensated for to improve image quality by developing and integrating uniformity algorithms (also called demura) into the display driving system. Demura refers to a process for removing / suppressing visual artifacts and achieving relative uniformity in the brightness and / or color of a display.

[0018] According to some embodiments of the present disclosure, compensation methods and systems are provided for improving uniformity in near-eye displays.

[0019] FIG. 1 illustrates a framework of a uniformization method for improving image quality according to some embodiments of the present disclosure. Referring to FIG. 1 , a rendered virtual image 110 displayed by a near-eye display (NED) is acquired by an imaging light measurement device (LMD). After the virtual image is preprocessed 120 (including alignment processing), the uniformity of the virtual image is characterized 130 for compensation calculation by comparing it with a baseline (e.g., target image data, target matrix, etc.) 131 to obtain non-uniformity 132. Compensation coefficients for a pixel matrix are generated 140 taking into account the non-uniformity matrix and the target matrix. Gray values ​​of the pixel matrix are adjusted 150 according to the compensation coefficients for each pixel of the image generator to obtain a compensated rendered virtual image 160. The compensated rendered virtual image 160 may be re-evaluated. Finally, the compensated rendered virtual image 160 is compared 170 with the uncompensated rendered virtual image 110 to determine the uniformity improvement quality (e.g., NU(non-uniformity)≦10%).

[0020] FIG. 2 illustrates another framework of a uniformization method for improving image quality using three primary color channels according to some embodiments of the present disclosure. Referring to FIG. 2 , a rendered virtual image 210 may be generated, specifically, a multi-color virtual image 220 may be generated for three primary color channels, e.g., a red channel, a green channel, and a blue channel. Then, a non-uniformity characterization 230 may be extracted from the multi-color virtual image 220 for each primary color channel. In some embodiments, the characteristics of each virtual image 220 include image data in the CIE-XYZ color space (in other words, X, Y, Z). A correction coefficient matrix 240 may be obtained based on the non-uniformity characterization 230 extracted from each primary color channel. Compensation may be performed using the correction coefficient matrix 240, and a Demura virtual image 250 may be generated after the compensation.

[0021] Using correction coefficient matrices for the three primary color channels, the compensation can be more accurate.

[0022] In some embodiments, a method for compensating a virtual image is provided. Figure 3 illustrates a flowchart of an exemplary method 300 for compensating a virtual image, according to some embodiments of the present disclosure. Referring to Figure 3, method 300 includes steps 302-310.

[0023] In step 302, three virtual images are acquired to be displayed by a near-eye display (NED) based on the three primary color patterns. Each virtual image is rendered by the NED and displayed to the human eye by the NED's microdisplay projector. The virtual image is formed by a source image emitted from the microdisplay projector and transmitted through an optical combiner (e.g., a waveguide) toward the front of the human eye. To characterize the non-uniformity of the virtual image for further compensation calculations, the virtual image is captured by an imaging LMD (light measurement device). In some embodiments, the LMD may be a colorimeter or an imaging camera such as a CCD (charge-coupled device) or a CMOS (complementary metal-oxide semiconductor). The gray value and / or luminance and chromaticity distribution of the virtual image are obtained in the entire field of view of the virtual image. Therefore, the gray value and / or luminance and chromaticity values ​​of each pixel of the virtual image, also referred to as image data, are obtained. A test pattern may be applied as a source image. In some embodiments, the source image is an all-white image (e.g., an all-white test pattern), and the virtual image is an all-white image. In this example, three primary color test patterns (e.g., a red test pattern, a green test pattern, and a blue test pattern) are applied. Therefore, three primary color virtual images can be obtained using the three test patterns, respectively. In some embodiments, the source image includes multiple partial-on patterns instead of a full pattern. The multiple partial-on patterns are superimposed on each other to form a full pattern. For example, three partial-on patterns are rendered sequentially on the NED. Finally, a full-screen virtual image is obtained. In some embodiments, one or more images with various grays / intensities can be rendered on the NED.

[0024] In step 304, image data including luminance and chromaticity (CIE-XYZ) for the three primary color channels is obtained, respectively. In some embodiments, the image data in the CIE-xyY color space for the three primary color channels (i.e., x, y, Y) is obtained through a conversion from CIE-XYZ. FIGS. 4A-4C illustrate exemplary distributions of luminance and chromaticity for the green primary color channel according to some embodiments of the present disclosure. FIG. 4A shows an example of luminance (L) in the CIE-xyY color space corresponding to the Y value. FIGS. 4B and 4C show examples of chromaticity distributions in the CIE-xyY color space, i.e., CIE-x and CIE-y. FIG. 5 shows an exemplary pixel distribution and color triangle in the CIE-xyY color space according to some embodiments of the present disclosure. The x and y axes represent the relative values ​​of the two components of chromaticity. As shown in FIG. 5, the CIE-xy image data for the green primary color channel shown in FIGS. 4B-4C can be projected onto an XY plane, e.g., the upper-left green region 501, where Z=0. The x and y axes shown in FIG. 5 correspond to CIE-xy color coordination, with each x and y value between 0 and 1. Referring to FIG. 5, the image data for the three primary color channels can be projected onto the same XY plane, e.g., the right red region 502 and the lower-left blue region 503. A color triangle 510 illustrating the chromaticity distribution for the virtual image is then obtained by connecting regions 501, 502, and 503 with vertices A, B, and C. In some embodiments, the color triangle 510 is the triangle with the largest area for illustrating the chromaticity distribution of the virtual image.

[0025] In some embodiments, image data in the CIE-XYZ color space for the three primary color channels (in other words, X, Y, Z) is first extracted for each primary color channel, and then the image data (in other words, X, Y, Z) is converted to image data in the CIE-xyY color space (in other words, x, y, Y) according to Equation 1.

[0026] In some embodiments, the image data of a pixel for the three primary color channels in the CIE-XYZ color space is a matrix expressed, for example, as follows: TIFF2026504225000004.tif14170

[0027] In step 306, target image data is determined based on the image data obtained for the three primary color channels. Referring again to FIG. 5, in the CIE-xyY color space, the target image data may be determined according to a target color triangle 520 obtained based on pixel chromaticity distributions 501-503. Color triangle 520, having vertices A', B', and C' formed by the target image data for each primary color channel, is within color triangle 510. In some embodiments, target color triangle 520 is determined as the color triangle with the smallest area. In some embodiments, target color triangle 520 is determined based on a preset threshold. The target image data includes image data for each primary color channel (in other words, x, y, Y).

[0028] In some embodiments, the target image data in the CIE-xyY color space is converted to target image data in the CIE-XYZ color space (in other words, X, Y, Z). The conversion may be performed based on Equation 2 described above. It will be appreciated that the target image data of a pixel for the three primary color channels in the CIE-XYZ color space is a matrix.

[0029] In step 308, correction coefficients for each pixel are obtained based on the target image data. In some embodiments, the correction coefficients are obtained as a matrix having multiple components corresponding to components in the image data (e.g., X, Y, Z in CIE-XYZ color space, R, G, B in CIE-RGB color space, or x, y, Y in CIE-xyY color space). In some embodiments, the correction coefficient matrix is ​​obtained by calculating the target image data matrix obtained in step 302 and the image data matrix. In some embodiments, the correction coefficients for each pixel are calculated for luminance and chromaticity equalization. In some embodiments, a correction matrix including the correction coefficients may be obtained using the following Equation 4: TIFF2026504225000005.tif36170

[0030] where [M 3×3 ] target For example, [M 3×3 ] px represents the image data for each pixel for all three primary color channels obtained from the virtual image in step 302, for example. 3×3 ] corr is the correction coefficient matrix used for further compensation. inv[M 3×3 ] px is [M 3×3 ] px is the inverse matrix of

[0031] In some embodiments, the correction coefficient matrix is ​​a non-diagonal matrix, i.e., the correction coefficient matrix includes chromaticity correction components that improve the accuracy of compensation for each pixel.

[0032] In step 310, compensation is performed on the source image based on the correction coefficient. After the correction coefficient is obtained, the gray value for each pixel can be adjusted to remove non-uniformity across the display matrix. The compensated image data can be obtained using the following Equation 5: TIFF2026504225000006.tif39170

[0033] where: TIFF2026504225000007.tif14170 represents the gray values ​​for each primary color channel (e.g., red, green, and blue) after compensation for each pixel, TIFF2026504225000008.tif14170 represents the gray value for each primary color channel of the source image for each pixel (in other words, before compensation). α, β, γ represent the components in the correction coefficient matrix.

[0034] Therefore, the input gray values ​​for the three sub-pixels are TIFF2026504225000009.tif13170 is adjusted, It will be output as TIFF2026504225000010.tif13170.

[0035] In some embodiments, a re-evaluation of the virtual image after the correction of the source image may be performed. To evaluate the quality of the improvement using the compensation method described above, the uniformity on the rendered virtual image is evaluated before and after the correction for comparison. Several standards have been published by the International Electrotechnical Commission (IEC) for device measurement, for example, the standard IEC 63145 is used for eyewear displays. A nine-point measurement (according to IEC 63145) may be performed to evaluate the uniformity of luminance and chromaticity.

[0036] 6A and 6B show nine-point color coordinates of a virtual image in the CIE-1931 color space before and after demurration, according to some embodiments of the present disclosure. 7A and 7B show nine-point color coordinates in the CIE-1976 color space before and after demurration, according to some embodiments of the present disclosure. The u and v axes shown in FIGS. 7A and 7B correspond to color coordinates with relative values ​​between 0 and 1. As shown in FIGS. 6A and 7A, before demurration, the differences among the nine points are significant in terms of the dispersion of the points in color space. After demurration, as shown in FIGS. 6B and 7B, the nine points tend to coincide and converge in color space, indicating a dramatic improvement in uniformity.

[0037] In some embodiments, after step 302 in which a virtual image is acquired, a pixel alignment process is performed to map the captured virtual image to the matrix array of the image generator and pre-process the virtual image. In an example, the image data of the virtual image is 9000 x 6000 pixels, and the matrix array of the image generator is 640 x 480 pixels. Therefore, the 9000 x 6000 pixels need to be aligned to the 640 x 480 pixel array for further compensation. FIG. 8 shows a flowchart of an exemplary pixel alignment method 800 according to some embodiments of the present disclosure. Referring to FIG. 8, the pixel alignment method 800 includes steps 802-806.

[0038] In step 802, a region of interest (ROI) in the virtual image is determined. Image data in the ROI of the virtual image is subjected to compensation. In some embodiments, the ROI may be determined by a preset threshold. FIG. 9 shows an exemplary determined ROI 910 from image data, according to some embodiments of the present disclosure. Referring to FIG. 9, the ROI 910 of the full field of view of the virtual image is determined based on a predefined threshold. In some embodiments, the ROI is determined by comparing the value of the image data for each pixel with a threshold. For example, the ROI may be determined according to Equation 6. G pixel ≧G threshold (Formula 6)

[0039] Here, the threshold G threshold can be set according to the image histogram, and G pixel where σ represents the value of the image data of the pixel, and G represents the gray value. For example, the threshold may be set to a gray value that is less than 10 percent of the full grayscale (255), e.g., the threshold is set to 225.

[0040] In some embodiments, the virtual image is divided into a ROI and a dark region around the ROI, eg, dark region 920 with reference to FIG.

[0041] In some embodiments, noise spots are excluded from the ROI. In some embodiments, noise spots are excluded from the ROI by evaluating luminous areas and background regions of the virtual image.

[0042] In some embodiments, distortion correction is further performed on the ROI. The captured virtual image is distorted by the LMD lens and the DUT (Device Under Test) module. To obtain accurate distribution data, the captured image needs to be distortion-free, that is, the distortion needs to be corrected by remapping the geometric pixel matrix. Usually, the distortion is observed (e.g., barrel distortion), and an inverse transformation is applied to correct the distortion accordingly. In some embodiments, the distortion can be corrected by Equation 7-1 and Equation 7-2. x corr =x orig (1+k1r 2 +k2r 4 +k3r 6 ) (Formula 7-1) y corr =y orig (1+k1r 2 +k2r 4 +k3r 6 ) (Formula 7-2)

[0043] where (x corr , y corr ) is the original coordinate (x orig , y orig ) are the coordinates of the pixel after distortion correction, corresponding to the center of the virtual image. The term r represents the distance of the pixel to the center of the virtual image. The terms k1, k2, and k3 are coefficients of the distortion parameters. In some embodiments, tangential distortion may also be corrected. Figure 10 shows an example image after distortion correction, according to some embodiments of the present disclosure.

[0044] In step 804, pixels in the ROI of the virtual image are identified. That is, a mapping ratio between the source image and the virtual image is calculated. In some embodiments, each pixel of the image generator / source can be extracted by a method that evaluates the mapping ratio and the full field of view size of the virtual image.

[0045] Because the virtual image is captured by a higher-resolution imaging LMD, the virtual image is much larger than the source image. For example, the mapping ratio is 3 or 5 between the virtual image and the source image. FIG. 11 shows an example of pixel alignment from the virtual image to the image source using a mapping ratio of 5, according to some embodiments of the present disclosure. Each unit zone 1110 (shown as a cross) represents an extracted pixel of the image source. In some embodiments, the mapping ratio is determined by the full-field size of the virtual image, the full-field size of the source image, the dimensions of the virtual image, and the dimensions of the source image. For example, the mapping ratio is calculated according to Equations 8-1 through 8-3. R=R1 / R2 (formula 8-1) R1=D1 / FOV1 (formula 8-2) R2=D2 / FOV2 (formula 8-3)

[0046] where R is the mapping ratio, D1 is the dimension of the imaging LMD used to acquire the virtual image, FOV1 is the active field of view of the imaging LMD, D2 is the active light-emitting area of ​​the micro-light-emitting array in the microdisplay projector, and FOV2 is the active field of view of the microdisplay projector. In some embodiments, the microdisplay projector includes a microdisplay panel and a lens. The microdisplay panel includes a micro-light-emitting array that can form the active light-emitting area. For example, the microdisplay panel is a micro-inorganic LED (light-emitting diode) display panel, a micro-OLED (organic light-emitting diode) display panel, or a micro-LCD (liquid crystal display) display panel.

[0047] 12 illustrates an example preprocessed image (640x480) in pixel alignment according to some embodiments of the present disclosure. As shown in FIG. 12, 9000x6000 pixels were aligned into a 640x480 pixel array.

[0048] In step 806, image data for the virtual image is extracted based on the identified pixels. In some embodiments, the image data includes a gray value for each pixel. In some embodiments, the image data includes luminance (Lum) and chromaticity (x, y) for each pixel, which may be derived from the gray value.

[0049] Using a pixel alignment process, pixels in the virtual image can be correlated to pixels in the source image, e.g., the matrix array of the image generator. Compensation performed on the pixels of the matrix array can improve the display performance of the virtual image.

[0050] In some embodiments, pixels in a source image are identified by image processing, such as morphology and feature extraction. The positions of pixels in the source image can be determined through morphological image processing (e.g., dilation / erosion, etc.). To avoid crosstalk between pixels and accurately identify pixels in the source image, a partial on / off method can be used for position determination, in which pixels in the source image are turned on at intervals and no two adjacent pixels are turned on at the same time. FIG. 13 shows an exemplary 3×3 partial on / off position determination pattern according to some embodiments of the present disclosure. In this example, only one pixel is turned on in a 3×3 matrix space. The pixel positions for all pixels can be derived through one identified position determination pattern and integrated pixel distance. Furthermore, CIE-XYZ data for the matrix pixels is extracted according to the identified pixel positions / regions.

[0051] In some embodiments, a method for determining target image data is implemented to obtain target image data in the CIE-XYZ color space. Figure 14 shows a flowchart illustrating a target image data determination method 1400 according to some embodiments of the present disclosure. Referring to Figure 14, method 1400 includes steps 1402-1408.

[0052] In step 1402, image data for the three primary color channels is obtained, including luminance (Lum) and chromaticity (x, y) distribution / uniformity for each primary color channel. In some embodiments, image data in the CIE-xyY color space (in other words, x, y, Y) for the three primary color channels is obtained. Previously described FIGS. 4A-4C illustrate exemplary distributions of luminance and chromaticity for the green primary color channel, according to some embodiments of the present disclosure. FIG. 4A shows an example of luminance (L) in the CIE-xyY color space corresponding to a Y value. FIGS. 4B and 4C show examples of chromaticity distributions in the CIE-xyY color space, in other words, CIE-x and CIE-y.

[0053] In some embodiments, image data in the CIE-XYZ color space for the three primary color channels (in other words, X, Y, Z) is first extracted for each primary color channel, and then the image data (in other words, X, Y, Z) is converted to image data in the CIE-xyY color space (in other words, x, y, Y) according to Equation 1.

[0054] In some embodiments, the image data of a pixel for the three primary color channels in the CIE-XYZ color space is a matrix expressed, for example, as follows: TIFF2026504225000011.tif14170

[0055] In step 1404, a pixel chromaticity distribution and a first color triangle are determined according to the image data of the three primary color channels. Referring again to FIG. 5, the CIE-xyY image data for the green primary color channel shown in FIGS. 4A-4C can be projected onto an XY plane, for example, the upper-left green region 501, where Z=0. The image data of the three primary color channels can be projected onto the same XY plane, for example, the right red region 502 and the lower-left blue region 503, referring again to FIG. 5. Then, a first color triangle 510 illustrating the chromaticity distribution of the virtual image is obtained. In some embodiments, the color triangle 510 is the triangle with the largest area for illustrating the chromaticity distribution of the virtual image.

[0056] In step 1406, a target color triangle is determined based on the pixel chromaticity distribution and the first color triangle.

[0057] A target color triangle for determining the target image data may be determined or selected based on the first color triangle. Referring again to Figure 5, in the CIE-xyY color space, the target color triangle 520 may be determined according to the pixel chromaticity distribution and the color triangle 510.

[0058] In some embodiments, the target color triangle 520 is determined based on preset thresholds. For example, a first target vertex A' for green is determined by selecting a point whose x-value is greater than that of vertex A and / or whose y-value is less than that of vertex A. A second target vertex B' for red is determined by selecting a point whose x-value is less than that of vertex B. A third target vertex C' for blue is determined by selecting a point whose x-value is greater than that of vertex C and / or whose y-value is greater than that of vertex C.

[0059] In some embodiments, target color triangle 520 is determined by selecting vertices within color triangle 510. In some embodiments, target color triangle 520 is determined as the color triangle with the smallest area, where the three vertices of the target color triangle are on the edges of each color area.

[0060] In step 1408, a target luminance value (Y) for each channel is determined taking into account the luminance distribution across the matrix for each primary channel. In some embodiments, the Y component of the target image data, in other words, the target luminance component, is the same as the Y component of the acquired image data for each primary color channel. In some embodiments, the target luminance value for each channel is calculated by the mean value of the distribution, as shown in Equation 9. Y target|ch =Average(Ymatrix|ch ) (Formula 9)

[0061] Y target|ch represents the target luminance level, and Y matrix|ch is the luminance distribution across the pixel matrix in each primary channel (e.g., R, G, and B). The function mean(M) represents calculating the average value of matrix M. In some embodiments, the target luminance level may be determined by self-definition or a histogram for extraction of the majority luminance level of the image matrix.

[0062] In step 1410, target image data is obtained based on the second color triangle and the target luminance level. The target image data includes a target luminance component and a target chrominance component. In some embodiments, the target chrominance component includes an x ​​component and a y component. For example, the x and y values ​​of three vertices (e.g., A', B', and C') of the target color triangle 520 may be determined as the target chrominance components x and y, respectively, for the three primary color channels in the CIE-xyY color space. For example, the x and y values ​​of the upper left vertex A' of the target color triangle 520 are determined as the target chrominance components x and y for the green channel. The x and y values ​​of the lower left vertex C' of the target color triangle 520 are determined as the target chrominance components x and y for the blue channel. The x and y values ​​of the right vertex B' of the target color triangle 520 are determined as the target chrominance components x and y for the red channel. The target image data includes image data (in other words, x, y, Y) for each primary color channel.

[0063] According to the compensation method provided by the present disclosure, not only can the luminance non-uniformity be compensated, but also the chrominance non-uniformity, thus improving the compensation performance.

[0064] In some embodiments, a non-transitory computer-readable storage medium containing instructions is also provided, which can be executed by a device to implement the methods described above. Common forms of non-transitory media include, for example, floppy disks, flexible disks, hard disks, solid-state drives, magnetic tape, or any other magnetic data storage medium, CD-ROMs, any other optical data storage medium, any physical medium with a pattern of holes, RAM, PROMs, and EPROMs, flash EPROMs or any other flash memory, NVRAM, cache, registers, any other memory chip or cartridge, and networked versions thereof. A device may include one or more processors (CPUs), input / output interfaces, network interfaces, and / or memory.

[0065] It should be noted that relational terms herein, such as "first" and "second," are used only to distinguish one entity or operation from another and do not require or imply any actual relationship or order between those entities or operations. Moreover, the words "comprising," "having," "containing," and "including," as well as other similar forms, are intended to be equivalent in meaning and to be open-ended in that the one or more items following any one of these words are not intended to be an exhaustive listing of such one or more items or to be limited to only the listed one or more items.

[0066] Unless otherwise specified, as used herein, the term "or" encompasses all possible combinations unless infeasible. For example, if it is stated that a database may include A or B, then the database may include A, or B, or A and B, unless otherwise specified or infeasible. As a second example, if it is stated that a database may include A, B, or C, then the database may include A, or B, or C, or A and B, or A and C, or B and C, or A and B and C, unless otherwise specified or infeasible.

[0067] It should be appreciated that the above-described embodiments may be implemented by hardware, or software (program code), or a combination of hardware and software. If implemented by software, the software may be stored in the computer-readable medium described above. The software, when executed by a processor, may perform the disclosed methods. The computing units and other functional units described in this disclosure may be implemented by hardware, or software, or a combination of hardware and software. Those skilled in the art will also understand that multiple of the above-described modules / units may be combined into one module / unit, and that each of the above-described modules / units may be further divided into multiple sub-modules / sub-units.

[0068] In the foregoing specification, embodiments have been described with reference to numerous specific details that may vary from implementation to implementation. Certain adaptations and modifications of the described embodiments may be made. Other embodiments may be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the invention being indicated by the following claims. It is also intended that the order of steps depicted in the figures is for illustrative purposes only and is not intended to be limited to any particular order of steps. Thus, one skilled in the art will appreciate that these steps may be performed in different orders while implementing the same method.

[0069] Illustrative embodiments have been disclosed in the drawings and herein. However, many variations and modifications may be made to these embodiments. Therefore, although specific terms have been employed, they are used in a generic and descriptive sense only and not for purposes of limitation.

Claims

1. 1. A method for compensating a virtual image displayed by a near-eye display based on a source image, comprising: acquiring virtual images for display by the near-eye display for three primary color channels, each of the virtual images being based on a primary color test pattern; obtaining a correction coefficient matrix including luma and chroma components of the three primary color channels; performing compensation on the source image using the correction coefficient matrix; A method comprising:

2. Obtaining the correction coefficient matrix including the luma component and the chroma component of the three primary color channels comprises: obtaining first image data for each of the primary color channels; determining target image data for each of the primary color channels based on the first image data; Obtaining the correction coefficient matrix based on the first image data and the target image data, wherein the first image data includes a luminance component and a chrominance component, and the target image data includes a target luminance component and a target chrominance component; and The method of claim 1 further comprising:

3. obtaining the first image data for each primary color channel; obtaining a first luminance component and a first chrominance component for each of the primary color channels in a first color space; converting the first luminance component and the first chrominance component from the first color space to a second color space to obtain a second luminance component and a second chrominance component; Including, determining the target image data for each of the primary color channels based on the first image data; determining a first target luminance component and a first target chrominance component for each of the primary color channels in the second color space based on the second luminance component and the second chrominance component; converting the first target luminance component and the first target chrominance component from the second color space to the first color space to obtain the target luminance component and the target chrominance component; further comprising obtaining the correction coefficient matrix based on the first image data and the target image data; Obtaining the correction coefficient matrix based on the first luminance component, the first chrominance component, the target luminance component, and the target chrominance component. The method of claim 2 further comprising:

4. 4. The method of claim 3, wherein the first color space is a CIE-XYZ color space and the second color space is a CIE-xyY color space.

5. obtaining the first image data for each of the primary color channels; extracting grey values ​​of said acquired virtual image for each primary colour channel respectively; determining the luminance and chrominance components from the gray values ​​for each of the primary color channels; The method of any one of claims 2 to 4, further comprising:

6. The method of claim 1 , wherein the three primary color channels include a green primary color channel, a blue primary color channel, and a red primary color channel.

7. After acquiring the virtual image, the method further comprises: determining a region of interest (ROI) in each of the virtual images; identifying pixels in the ROI of each of the virtual images; extracting image data for each of the virtual images based on the identified pixels; 7. The method of claim 1, further comprising:

8. The method of claim 1 , wherein the correction coefficient matrix is ​​a non-diagonal matrix.

9. obtaining the correction coefficient matrix including the luma component and the chroma component of the three primary color channels, obtaining an image data matrix for the three primary color channels based on the virtual image, and inverting the image data matrix to obtain an inverted image data matrix; determining a target image data matrix for the three primary color channels; multiplying the inverted image data matrix and the target image data matrix to obtain a correction coefficient matrix; The method of claim 1 further comprising:

10. Determining the target image data for each of the primary color channels based on the first image data includes: determining a first color triangle formed by first chromaticity components of the first image data for each of the primary color channels; determining a second color triangle formed by target chromaticity components of the target image data for each of the primary color channels, the second color triangle having an area smaller than an area of ​​the first color triangle; determining the target chromaticity component values ​​for each of the primary color channels based on the second color triangle; The method of claim 2 further comprising:

11. The method of claim 10 , wherein the second color triangle is within the first color triangle.

12. determining the values ​​of the target chromaticity components for each of the primary color channels based on the second color triangle; determining the values ​​of the target chromaticity components for each primary color channel based on coordinate values ​​of vertices of the second color triangle; 12. The method of claim 10 or 11, comprising:

13. identifying the pixels in the ROI of each of the virtual images; locating the pixels in the ROI of each of the virtual images under a partial on-off pattern; identifying said pixels; The method of claim 7 further comprising:

14. 1. A method for compensating a virtual image displayed by a near-eye display based on a source image, comprising: acquiring virtual images for display by the near-eye display for three primary color channels, each of the virtual images being based on a primary color test pattern; obtaining a first chromaticity component for each of the primary color channels; determining a target chromaticity component for each of the primary color channels based on the first chromaticity component; obtaining a correction coefficient matrix based on the target chromaticity components; performing compensation on the source image based on the correction coefficient matrix; A method comprising:

15. determining the target chromaticity components for each of the primary color channels based on the first chromaticity components, determining a first color triangle formed by the chromaticity components of each primary color channel; determining a second color triangle formed by the target chromaticity components of each of the primary color channels, the second color triangle having an area smaller than an area of ​​the first color triangle; determining a value of the target chromaticity component for each of the primary color channels based on the second color triangle; 15. The method of claim 14, further comprising:

16. The method of claim 15 , wherein the second color triangle is within the first color triangle.

17. determining the values ​​of the target chromaticity components for each of the primary color channels based on the second color triangle; determining the values ​​of the target chromaticity components for each of the primary color channels based on coordinate values ​​of vertices of the second color triangle; 17. The method of claim 15 or 16, comprising:

18. obtaining the first chromaticity component for each of the primary color channels; obtaining a second chromaticity component for each of the primary color channels in a first color space; converting the second chromaticity component from the first color space to a second color space to obtain the first chromaticity component; further comprising determining the target chromaticity components for each of the primary color channels based on the first chromaticity components; determining the target chromaticity components for each of the primary color channels based on the first chromaticity components in the second color space; further comprising obtaining the correction coefficient matrix based on the target chromaticity components, converting the target chromaticity components from the second color space to the first color space to obtain second target chromaticity components; obtaining the correction coefficient matrix based on the second target chromaticity component; 15. The method of claim 14, further comprising:

19. 19. The method of claim 18, wherein the first color space is a CIE-XYZ color space and the second color space is a CIE-xyY color space.

20. Obtaining the correction coefficient matrix based on the target chromaticity components includes: obtaining an image data matrix for the three primary color channels based on the virtual image, and inverting the image data matrix to obtain an inverted image data matrix, the data matrix including a luminance component and a chrominance component for each of the primary color channels; determining a target image data matrix for the three primary color channels; obtaining a correction coefficient matrix by multiplying the inverted image data matrix and the target image data matrix, the target image data matrix including a target luma component and a target chroma component for each of the primary color channels; 15. The method of claim 14, further comprising:

21. 21. The method of any one of claims 14 to 20, wherein the three primary color channels include a green primary color channel, a blue primary color channel, and a red primary color channel.

22. After acquiring the virtual image, the method further comprises: determining a region of interest (ROI) in each of the virtual images; identifying pixels in the ROI of each of the virtual images; extracting image data for each of the virtual images based on the identified pixels; 22. The method of any one of claims 14 to 21, further comprising:

23. identifying the pixels in the ROI of each of the virtual images; locating the pixels in the ROI of each of the virtual images under a partial on-off pattern; identifying said pixels; 23. The method of claim 22, further comprising:

24. 24. The method of any one of claims 14 to 23, wherein the correction coefficient matrix is ​​a non-diagonal matrix.

25. 1. A method for compensating a virtual image displayed by a near-eye display based on a source image, comprising: acquiring virtual images for display by the near-eye display for three primary color channels, each of the virtual images being based on a primary color test pattern; obtaining an image data matrix for the three primary color channels based on the virtual image, and inverting the image data matrix to obtain an inverted image data matrix; determining a target image data matrix for the three primary color channels; multiplying the inverted image data matrix and the target image data matrix to obtain a correction coefficient matrix; performing compensation on the source image based on the correction coefficient matrix; A method comprising:

26. 26. The method of claim 25, wherein the correction coefficient matrix includes luma and chroma components of the three primary color channels.

27. obtaining the image data matrix for the three primary color channels based on the virtual image includes obtaining the image data matrix in a first color space; determining the target image data matrix for the three primary color channels; converting the image data matrix from the first color space to a second color space to obtain a second image data matrix; determining a first target image data matrix in the second color space based on the second image data matrix; converting the first target image data matrix from the second color space to the first color space to obtain the target image data matrix; 27. The method of claim 26, further comprising:

28. 28. The method of claim 27, wherein the first color space is a CIE-XYZ color space and the second color space is a CIE-xyY color space.

29. Obtaining the image data matrix for the three primary color channels comprises: extracting grey values ​​of the acquired virtual image for each of the primary colour channels respectively; determining a luminance component and a chrominance component from said gray value for each of said primary color channels; 29. The method of any one of claims 25 to 28, further comprising:

30. 30. The method of any one of claims 25 to 29, wherein the three primary color channels include a green primary color channel, a blue primary color channel, and a red primary color channel.

31. After acquiring the virtual image, the method further comprises: determining a region of interest (ROI) in each of the virtual images; identifying pixels in the ROI of each of the virtual images; extracting image data for each of the virtual images based on the identified pixels; 31. The method of any one of claims 25 to 30, further comprising:

32. identifying the pixels in the ROI of each of the virtual images; locating the pixels in the ROI of each of the virtual images under a partial on-off pattern; identifying said pixels; 32. The method of claim 31 , further comprising:

33. 33. The method of any one of claims 25 to 32, wherein the correction coefficient matrix is ​​a non-diagonal matrix.

Images