Method and system for virtual image compensation and evaluation

Abstract

A method for image compensation for a virtual image displayed by a near-eye display based on a microdisplay projector includes obtaining a virtual image formed by a source image emitted from the microdisplay projector and displayed by the near-eye display, preprocessing image data of the virtual image to obtain preprocessed image data, obtaining a relationship between the source image and the virtual image, determining an image baseline value of the virtual image, and obtaining a compensation coefficient matrix including a compensation coefficient for each pixel in the source image based on the relationship and the image baseline value.

Description

Technical Field

[0001] The present disclosure generally relates to microdisplay technology, and more particularly to methods and systems for virtual image compensation and evaluation.

Background Art

[0002] Near-eye displays may be provided as augmented reality (AR) displays, virtual reality (VR) displays, head-up / head-mounted, or other displays. Generally, a near-eye display typically includes an image generation device and an optical combiner that transfers a projected image from the image generation device to the human eye. The optical combiner is a group of reflective and / or diffractive optical systems such as a free-form mirror / prism, a bird bath, or a cascaded mirror, and / or a grating coupler (waveguide). Also, the projected image is a virtual image in front of the human eye. The image generation device can be a micro-LED-based display, a liquid crystal on silicon (LCOS) display, or a digital light processing (DLP) display. The virtual image is rendered from the image generation device and the optical combiner to the human eye.

[0003] Uniformity is one of the important performance indicators of a display used to evaluate image quality. This usually refers to the imperfection of the display matrix and is also called non-uniformity. Non-uniformity includes fluctuations in the overall distribution and local zones also called mura. In the case of near-eye displays such as AR / VR, visual artifacts such as speckled, bright, or dark spots, or a cloudy appearance can also be observed on the virtual image rendered by the display system. In the virtual image rendered by an AR / VR display, non-uniformity may be shown in luminance and / or chromaticity. Compared with conventional displays, non-uniformity artifacts are much more obvious because they are closer to the human eye. Therefore, a method for improving the quality of virtual images is desired.

Summary of the Invention

[0004] Embodiments of the present disclosure provide a method for compensating a virtual image displayed by a near-eye display based on a microdisplay projector. The method includes obtaining a virtual image formed by a source image emitted from a microdisplay projector and displayed by the near-eye display, preprocessing the image data of the virtual image to obtain preprocessed image data, obtaining the relationship between the source image and the virtual image, determining an image baseline value of the virtual image, and obtaining a compensation coefficient matrix including compensation coefficients for each pixel in the source image based on the relationship and the image baseline value.

[0005] Embodiments of the present disclosure also provide a method for evaluating the compensation quality of a virtual image displayed by a near-eye display based on a microdisplay projector. The method includes obtaining a first virtual image formed by a source image emitted from a microdisplay projector and displayed by the near-eye display, preprocessing the image data of the first virtual image to obtain preprocessed image data, obtaining the relationship between the source image and the virtual image, determining an image baseline value of the first virtual image, evaluating the non-uniformity of the first virtual image, obtaining a compensation coefficient matrix including compensation coefficients for each pixel in the source image based on the relationship and the image baseline value, adjusting the image data of each pixel for the first virtual image based on the compensation coefficient matrix and displaying a second virtual image, re-evaluating the non-uniformity of the second virtual image, and comparing the non-uniformity of the second virtual image with the non-uniformity of the first virtual image.

[0006] Embodiments of the present disclosure further provide an apparatus. The apparatus includes a memory configured to store instructions, and one or more processors configured to execute the instructions to cause the apparatus to implement the above-described method for compensating a virtual image displayed by a near-eye display based on a microdisplay projector.

[0007] Embodiments of the present disclosure further provide an apparatus. The apparatus includes a memory configured to store instructions, and one or more processors configured to execute the instructions to cause the apparatus to implement the above-described method for evaluating the compensation quality of a virtual image displayed by a near-eye display based on a microdisplay projector.

[0008] Embodiments and various aspects of the present disclosure are shown in the following detailed description and the accompanying drawings. The various features shown in the figures are not drawn to scale.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figures 3(a)-3(c)

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figures 13(a)-13(b)

Figures 14(a)-14(b)

Figure 15

Figure 16

DETAILED DESCRIPTION OF THE INVENTION

[0010] Here, exemplary embodiments are referred to in detail, and examples thereof are shown in the accompanying drawings. The following description refers to the accompanying drawings in which the same numbers represent the same or similar elements in different drawings, unless otherwise indicated. The embodiments described in the following description of the exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of apparatuses and methods consistent with aspects related to the present invention described in the appended claims. Specific aspects of the present disclosure are described in more detail below. The terms and definitions provided herein govern in case of conflict with the terms and / or definitions incorporated by reference.

[0011] By developing and integrating a homogenization (also called demura) algorithm into the display driving system, non-uniformity can be compensated and image quality can be improved. Demura refers to a process for eliminating / suppressing visual artifacts and achieving relative uniformity of luminance and / or color of the display.

[0012] According to the present disclosure, a compensation method and system for improving the uniformity of a near-eye display are provided.

[0013] FIG. 1 shows a framework of a homogenization method for improving image quality according to some embodiments of the present disclosure. Referring to FIG. 1, a rendered virtual image displayed by a near-eye display (NED) 110 is acquired by an imaging light measurement device (LMD). After the virtual image is pre-processed (including alignment processing) 120, the uniformity of the virtual image is characterized 130 for compensation calculation by comparing it with a baseline 131 to obtain a non-uniformity 132. Considering the non-uniformity matrix and the target matrix, a compensation coefficient for the pixel matrix is generated 140. The gray value of the pixel matrix is finally adjusted 150 according to the compensation coefficient of each pixel of the image generation device to obtain a rendered virtual image with compensation 160. The rendered virtual image with compensation 160 can be re-evaluated. Finally, the rendered virtual image with compensation 160 is compared with the rendered virtual image 110 without compensation to determine the uniformity improvement quality (e.g., NU (non-uniformity) <= 10%) 170.

[0014] FIG. 2 shows a flowchart illustrating an exemplary compensation method 200 according to some embodiments of the present disclosure. Referring to FIG. 2, method 200 includes steps 202 to 216.

[0015] In step 202, a virtual image displayed by a near-eye display (NED) is acquired. The virtual image is rendered by the NED and displayed to the human eye by the NED's microdisplay projector. The virtual image is formed by a source image that is emitted from the microdisplay projector and transmitted towards the front of the human eye. In order to characterize the non-uniformity of the virtual image for further compensation calculations, the virtual image is captured by an imaging LMD (light measurement device). In some embodiments, the LMD can be a colorimeter or an imaging camera such as a CCD (charge-coupled device) or a CMOS (complementary metal-oxide semiconductor). The grayscale and / or luminance distribution of the virtual image is acquired over the entire field of view of the virtual image. Thus, the gray value and / or luminance value of each pixel of the virtual image is acquired, which is also referred to as image data. For the measurement, a full-white test pattern can be applied. In some embodiments, the source image is a full-white image (e.g., a full-white test pattern), and the virtual image is also a full-white image. Thus, based on the full-white test pattern, the calculation of the compensation coefficients for the compensation model can be more accurate. In some embodiments, the source image includes a plurality of partial-on patterns instead of a complete pattern. The plurality of partial-on patterns are stacked on top of each other to form a full-white pattern. For example, three partial-on patterns are sequentially rendered to the NED. Finally, a full-screen virtual image is obtained. In some embodiments, one or more images having various grays / luminances can be rendered to the NED. FIG. 3(a) shows an example of a full-white test pattern according to some embodiments of the present disclosure, FIG. 3(b) shows a captured virtual image in color according to some embodiments of the present disclosure, and FIG. 3(c) shows a pseudo-color luminance image according to some embodiments of the present disclosure. Referring to FIGS. 3(a), 3(b), and 3(c), the virtual image is captured by a 2D colorimeter having an NED lens.

[0016] In step 204, the image data of the virtual image is preprocessed, and the preprocessed image data is obtained. The image data includes the gray value and / or luminance value of the image. FIG. 4 shows a flowchart of an exemplary preprocessing method 400 according to some embodiments of the present disclosure. Referring to FIG. 4, the preprocessing method 400 includes steps 402 to 406.

[0017] In step 402, a region of interest (ROI) in the virtual image is extracted. The image data within the ROI of the virtual image is preprocessed. In some embodiments, the ROI can be determined by a preset threshold. FIG. 5 shows an exemplary ROI 510 determined from the image data according to some embodiments of the present disclosure. Referring to FIG. 5, the ROI 510 for the entire field of view of the virtual image is determined based on a predetermined threshold. In some embodiments, the ROI is determined by comparing the average value of the image data with the threshold, and the average image data value within the region of interest is equal to or greater than the threshold. In some embodiments, the ROI is determined by comparing the value of the image data of each pixel with the threshold. For example, the ROI can be expressed by Equation 1, L pixel ≧L threshold (Equation 1) and can be determined according to the following formula, where the threshold L threshold can be set according to the image histogram, and L pixel represents the value of the image data of the pixel. For example, it is set as a gray value less than 10% of the full grayscale (255), for example, 225.

[0018] In some embodiments, the virtual image is divided into an ROI and a dark region around the ROI, for example, referring to FIG. 5, region 520.

[0019] In step 404, noise spots are excluded from the ROI. In some embodiments, the noise spots are excluded from the ROI by evaluating the light-emitting region and the background region of the virtual image.

[0020] In step 406, distortion correction is performed on the ROI. In some embodiments, the preprocessing further includes image distortion correction. The captured virtual image is distorted by the LMD lens and the DUT (device under test) module. To obtain accurate distribution data, it is necessary to correct the distortion of the captured image, that is, by remapping the geometric pixel matrix. Usually, distortion is observed (e.g., barrel distortion), and a reverse transformation is applied accordingly to correct the distortion. In some embodiments, the distortion can be corrected by equations 2-1 and 2-2. x corr =x orig (1 + k1r 2 + k2r 4 + k3r 6 )(Equation 2-1) y corr =y orig (1 + k1r 2 + k2r 4 + k3r 6 )(Equation 2-2)

[0021] Where (x corr , y corr ) are the coordinates of the pixel after distortion correction corresponding to the original coordinates (x orig , y orig ). The term r is the distance of the pixel from the center of the virtual image. The terms k1, k2, and k3 are the coefficients of the distortion parameters. In some embodiments, tangential distortion can also be corrected. FIG. 6 shows an exemplary image after distortion correction according to some embodiments of the present disclosure.

[0022] After step 204, the relationship between the source image and the virtual image is obtained. Referring again to FIG. 2, the relationship between the source image and the virtual image can be obtained by the following steps 206 and 208.

[0023] In step 206, the mapping ratio between the source image and the virtual image is calculated. To achieve compensation via the image generation device, pixel alignment is performed to map the captured virtual image to the matrix array of the image generation device. In some embodiments, each pixel of the image generation device / source can be extracted by a method that evaluates the mapping ratio and full field size of the virtual image. In some embodiments, pixels are identified by image processing such as morphology and feature extraction. The position of the pixels can be determined by morphological image processing (e.g., dilation / erosion, etc.).

[0024] Since the virtual image is captured by a higher resolution imaging LMD, the virtual image is much larger than the source image. For example, the mapping ratio between the virtual image and the source image is 3 or 5. FIG. 7 shows an example of pixel alignment from a virtual image having a mapping ratio of 5 to an image source according to some embodiments of the present disclosure. Each unit zone 710 (shown as a cross) represents an extracted pixel of the image source. In some embodiments, the mapping ratio is determined by the full field size of the virtual image, the full field size of the source image, the dimensions of the virtual image, and the dimensions of the source image. For example, the mapping ratio is calculated by Equations 3-1 to 3-3. R = R1 / R2 (Equation 3-1) R1 = D1 / FOV1 (Equation 3-2) R2 = D2 / FOV2 (Equation 3-3)

[0025] Wherein, R is the mapping ratio, D1 is the dimension of the imaging LMD used to obtain the virtual image, FOV1 is the active field of view of the imaging LMD, D2 is the active emission area of the micro-light-emitting array in the microdisplay projector, and FOV2 is the active field of view of the microdisplay projector. In some embodiments, the microdisplay projector includes a microdisplay panel and a lens. The microdisplay panel includes a micro-light-emitting array capable of forming an active emission area. For example, the microdisplay panel is a micro inorganic LED (light-emitting diode) display panel, a micro OLED (organic light-emitting diode) display panel, or a micro LCD (liquid crystal display) display panel.

[0026] In step 208, a source image data matrix is calculated based on the preprocessed image data and the mapping ratio. The source image data matrix includes the same pixel dimensions of the source image. In some embodiments, the source image data matrix is obtained by Equation 4. [M] orig =[M1] / R (Equation 4) Wherein, [M] orig is the source image data, R is the mapping ratio, and M1 is a preprocessed image data matrix composed of the preprocessed image data.

[0027] In step 210, an image baseline value is determined. In order to characterize image uniformity and perform compensation, the image baseline value needs to be set for a general overall representation and a compensation object. Therefore, the image baseline value is determined for the entire virtual image. In some embodiments, the image baseline value can be determined by analyzing an image histogram (e.g., the ratio of pixels corresponding to each gray value). In the histogram method, the gray distribution across the entire image is considered. FIG. 8 shows an example of an image histogram according to some embodiments of the present disclosure. Referring to FIG. 8, the proportional pixel numbers (i.e., the vertical axis) corresponding to each gray value (i.e., the horizontal axis) are shown. In some embodiments, the image baseline value is determined by the maximum ratio (i.e., the peak) of pixels. In some embodiments, the baseline value is determined by calculating the average gray value of all pixels. The baseline value can be obtained, for example, by Equation 5. TIFF2025523108000002.tif8170Where V baseline is the baseline value, n is the number of pixels, and GV i is the gray value of each pixel.

[0028] In step 212, a compensation coefficient matrix is calculated based on the source image data matrix and the image baseline value. The compensation coefficient matrix includes compensation coefficients for each pixel in the source image. In some embodiments, the compensation coefficient matrix for each pixel can be obtained by Equation 6. [M] comp =[M] baseline / [M] orig -1 (Equation 6) Where [M] comp represents the compensation coefficient matrix (e.g., 640×480) of the image generation device. [M] baseline is the baseline matrix that matches the image baseline value.

[0029] The compensation factor can be positive or negative. A positive correction factor means that as the original pixel value increases, the baseline value is raised. A negative compensation factor means that as the original pixel value decreases, the baseline value is lowered.

[0030] Accordingly, a compensation model (e.g., a compensation factor matrix) for improving image quality is established. Further compensation for non-uniformity can be performed using the compensation model. In some embodiments, the compensation model (e.g., the compensation factor matrix) can be stored in the hard disk of the display system or the memory of the microdisplay panel so as to be able to perform compensation for the entire display system.

[0031] In some embodiments, the compensation method 200 can further include step 214.

[0032] In step 214, the image data of each pixel of the virtual image is adjusted based on the compensation factor matrix. In some embodiments, the image data of each pixel in the source image is adjusted based on the compensation factor matrix. The source image is an image transmitted by the micro LEDs of the display device to form the virtual image.

[0033] In some embodiments, the adjusted image data of each pixel in the source image is obtained by Equation 7, GV comp =round(GV orig ×([M] comp +1))(Equation 7) where GV comp is an adjusted image data matrix including the adjusted image data of each pixel, GV orig is a source image data matrix including the image data of each pixel in the source image, and [M] comp is the compensation factor matrix.

[0034] In some embodiments, the image data includes the gray value of each pixel. For example, the original gray value of the pixel is 128. The corresponding compensation coefficient is 0.2. Therefore, the gray value after compensation is 153.6. In the integer function (e.g., round, or ceil / floor), the gray value after compensation is 154. In some embodiments, the compensation ability depends on the display driving system. In some embodiments, the gray value after compensation usually overflows the original gray value range (e.g., 0-255). Therefore, the gray value range after compensation includes the original gray value range and the extended gray value range. For example, the gray value range after compensation is in the range of 0-511 (e.g., 9 bits). In some embodiments, the gray values of some individual pixels still exceed the gray value range after compensation. The gray values exceeding the gray value range after compensation can be cut off at the boundary (e.g., at 0 or 511). FIG. 9 shows an example of a generated image with pseudo-color compensation according to some embodiments of the present disclosure. Referring to FIG. 9, compared with FIG. 3(c), the compensated image is adjusted in gray value and the uniformity is improved.

[0035] In some embodiments, the image data includes the luminance value of each pixel. The adjusted image data of each pixel in the source image is obtained by adjusting the luminance value of each pixel.

[0036] In some embodiments, the image data to the micro light-emitting array is adjusted based on the compensation coefficient matrix. In this example, the gray value of each pixel is adjusted to obtain an updated virtual image.

[0037] Therefore, in some embodiments, method 200 further includes step 216 of displaying the updated virtual image.

[0038] FIG. 10 shows another flowchart of an exemplary image compensation method 200 according to some embodiments of the present disclosure. As shown in FIG. 10, to consider the quality improvement of the compensation method, after step 210, method 200 further includes step 211.

[0039] In step 211, the non-uniformity of the virtual image is evaluated. Based on the image baseline value, the non-uniformity of the virtual image can be evaluated. The non-uniformity can be calculated according to Equation 8. [M] non =([M] orig -[M] baseline ) / [M] baseline (Equation 8) In the formula, [M] non represents the non-uniformity of the image, [M] orig is the source image data matrix, and [M] baseline is the baseline matrix that coincides with the image baseline value. FIG. 11 shows an example of the pseudo-color image non-uniformity according to some embodiments of the present disclosure. In some embodiments, the non-uniformity can be directly evaluated before mapping the virtual image to the source image matrix.

[0040] In some embodiments, method 200 may further include step 218.

[0041] In step 218, the non-uniformity of the updated virtual image is re-evaluated and compared with the non-uniformity of the original virtual image (i.e., the non-uniformity evaluated in step 211). The other steps in FIG. 10 are the same as those described above with reference to FIG. 2 and will not be repeated here.

[0042] FIG. 12 shows a flowchart of an exemplary method 1200 for re-evaluating the non-uniformity of the updated virtual image according to some embodiments of the present disclosure. As shown in FIG. 12, re-evaluating the non-uniformity of the updated virtual image includes steps 1202 to 1210.

[0043] In step 1202, a plurality of uniformity regions distributed within the updated virtual image are determined. For example, the plurality of regions can be determined as nine uniform uniformity regions distributed around the virtual image.

[0044] In step 1204, the luminance values of a plurality of regions are summed. Note that the luminance value can be represented by a gray value.

[0045] In step 1206, the average luminance value L av of the updated virtual image is calculated. In some embodiments, the average luminance value of the updated virtual image is obtained by Equation 9 L av = S / N (Equation 9) where S is the sum of the luminance values of the plurality of regions, N is the number of regions, for example, N is equal to 9.

[0046] In step 1208, the uniformity value of each of the plurality of regions is obtained. In some embodiments, the uniformity value U n of each of the plurality of regions is obtained by Equation 10 U n = L n / L av (Equation 10) where U n is the uniformity value in region n, L n is the luminance value of region n, and L av is the average luminance value of the updated virtual image. In this example, n ranges from 1 to 9. Therefore, the overall uniformity can be quantitatively evaluated using a method for re-evaluating the non-uniformity of the updated virtual image.

[0047] In some embodiments, method 1200 further includes step 1210 of calculating the non-uniformity (NU) of the updated virtual image with NU = 1 - U max , where U max is the maximum value in the previous uniformity calculation among the n regions.

[0048] In some embodiments, a plurality of regions within the original virtual image are determined, and the uniformity value of the regions within the original virtual image is calculated. Next, the non-uniformity of each region of the updated virtual image is compared with the non-uniformity of the same region of the original virtual image.

[0049] In some embodiments, after comparing the non-uniformity of the original virtual image and the non-uniformity of the updated virtual image, the virtual image with higher uniformity is finally displayed.

[0050] FIGS. 13(a), 13(b), and 13(c) show an exemplary comparison of uniformity before and after compensation according to some embodiments of the present disclosure. FIG. 13(a) shows the captured virtual image before compensation, and FIG. 13(b) shows the captured virtual image after compensation. FIGS. 14(a) and 14(b) show exemplary corresponding luminance distributions before and after compensation according to some embodiments of the present disclosure. FIG. 14(a) shows the luminance distribution before compensation, and FIG. 14(b) shows the luminance distribution after compensation. As shown in FIGS. 13(a), 13(b), 13(c), 14(a), and 14(b), the uniformity and the luminance distribution are significantly improved. FIG. 15 shows the uniformity at nine points before and after compensation according to some embodiments of the present disclosure. Referring to FIG. 15, the uniformity values in nine regions are plotted. FIG. 15 shows that the variation in the uniformity of the image distribution is significantly reduced and is close to the ideal smoothness (i.e., the uniformity value is equal to 1). The image quality after compensation has been improved dramatically.

[0051] The image quality of the virtual image rendered by NED is improved dramatically by the equalization / demura algorithm, and visual artifacts can be effectively removed after compensation.

[0052] FIG. 16 is a schematic diagram of an exemplary system 1600 according to some embodiments of the present disclosure. As shown in FIG. 16, the system 1600 is provided to improve the uniformity of virtual images rendered within a near-eye display and can implement the compensation method 200 described above. The system 1600 includes a near-eye display (NED) 1610 for displaying an image in front of a human eye, an imager provided as an imaging module 1620, a positioner provided as a positioning device 1630, and a processor provided as a processing module 1640. Further, ambient light can be provided by an ambient light module 1650. The near-eye display 1610 can be provided as an AR (augmented reality) display, a VR (virtual reality) display, a head-up display / head-mounted display, or other display. The positioning device 1630 is provided to set an appropriate spatial relationship between the near-eye display (NED) 1610 and the imaging module 1620. For example, the positioning device 1630 is configured to set the distance between the near-eye display 1610 and the imaging module 1620 in the range of 10 mm to 25 mm. Further, the positioning device 1630 can adjust the relative position (e.g., distance and spatial position) between the near-eye display 1610 and the imaging module 1620. The imaging module 1620 is configured to measure display optical characteristics by mimicking a human eye and observe display performance. In some embodiments, the imaging module 1620 can include an array light measurement device (LMD) 1622 and a near-eye display (NED) lens 1621. For example, the LMD 1622 can be a colorimeter or an imaging camera such as a CCD (charge-coupled device) or a CMOS (complementary metal-oxide semiconductor). The near-eye display (NED) lens 1621 of the imaging module 1620 is provided with a front opening having a small diameter of 1 mm to 6 mm. Therefore, the near-eye display (NED) lens 1621 can provide a wide field of view (e.g., 60 to 180 degrees) forward, and the near-eye display lens 1621 is configured to mimic a human eye for observing the near-eye display 1610.The optical characteristics of the virtual image are measured by the imaging module 1620 based on the positioning device 1630.

[0053] In some embodiments, the near-eye display 1610 can include an image generation device 1611, also referred to herein as an image source, and an optical combiner, also referred to herein as an image optical system (not shown in FIG. 16). The image generation device 1611 can be a microdisplay such as a micro LED, micro OLED, LCOS, or DLP display, and can be configured to form an optical engine with an additional projection lens. In some embodiments, the microdisplay projector includes a microdisplay panel and a plurality of lenses. The microdisplay panel includes a micro light-emitting array that can form an active emission region. For example, the microdisplay panel can be a micro inorganic LED display panel, a micro OLED display panel, or a micro LCD display panel. The image projected through the optical system designed from the optical engine is transmitted to the human eye through the optical combiner. The optical system of the optical combiner can be a reflective and / or diffractive optical system such as a free-form mirror / prism, a birdbath or cascaded mirror, or a grating coupler (waveguide).

[0054] The processing module 1640 is configured to calculate compensation coefficients and evaluate uniformity / non-uniformity, etc. In some embodiments, the processing module 1640 can be included in a computer or a server. In some embodiments, the processing module 1640 can be deployed in the cloud, which is not limited herein.

[0055] In some embodiments, a driver provided as a drive module (not shown in FIG. 16) can be further provided to compensate for the image generation device 1611. The compensation coefficient is calculated in the processing module 1640 and then transferred to the drive module. Thus, in the system 1600, a compensation method can be implemented. The drive system can be coupled to communicate with the near-eye display 1610, specifically, to communicate with the image generation device 1611 of the near-eye display 1610. For example, the drive module can be configured to adjust the gray values of the image generation device 1611. When integrating a drive system including the functions of display driving and compensation (gray value adjustment in image processing) into the near-eye display, the data of the compensation coefficient from the processing module 1640 can be transferred to the near-eye display system 1610.

[0056] In some embodiments, for example, for AR applications, the ambient light is supplied from the ambient light module 1650. The ambient light module 1650 is configured to generate a uniform light source having a corresponding color (such as D65) that can support measurements performed under an ambient light background and simulations of various scenarios such as daylight, outdoor, or indoor.

[0057] In some embodiments, a non-transitory computer-readable storage medium including instructions is also provided, and the instructions may be implemented by a device to implement the methods described above. Common forms of non-transitory media include, for example, floppy disks, flexible disks, hard disks, solid state drives, magnetic tapes, or any other magnetic data storage medium, CD-ROM, any other optical data storage medium, any physical medium having a pattern of holes, RAM, PROM, and EPROM, FLASH-EPROM, or any other flash memory, NVRAM, cache, registers, any other memory chip or cartridge, and network versions thereof. The device may include one or more processors (CPUs), an input / output interface, a network interface, and / or memory.

[0058] Note that relational terms such as "first" and "second" in this specification are used only to distinguish one entity or operation from another entity or operation, and do not require or imply any actual relationship or order between these entities or operations. Further, the words "comprising", "having", "containing", and "including", and other similar forms, are equivalent in meaning, and do not mean that the one or more items following any one of these words is an exhaustive listing of such one or more items, nor are they intended to be limited to only the one or more items listed, but are open-ended in the sense that they do not exclude additional, unrecited items.

[0059] As used herein, unless otherwise specified, the term "or" includes all possible combinations except when it is infeasible. For example, if it is stated that a database can include A or B, then unless otherwise specified or infeasible, the database can include A, or B, or both A and B. As a second example, if it is stated that a database can include A, B, or C, then unless otherwise specified or infeasible, the database can include A, or B, or C, or both A and B, or both A and C, or all of A, B, and C.

[0060] It should be understood that the above-described embodiments can be implemented by hardware, or software (program code), or a combination of hardware and software. When implemented by software, it can be stored in the above-described computer-readable medium. When the software is executed by a processor, the disclosed method can be implemented. The computing units and other functional units described in the present disclosure can be implemented by hardware, or software, or a combination of hardware and software. Those skilled in the art will also understand that a plurality of the above-described modules / units may be combined as one module / unit, and each of the above-described modules / units may be further divided into a plurality of sub-modules / sub-units.

[0061] In the foregoing specification, embodiments have been described with reference to numerous specific details which may vary from embodiment to embodiment. Specific adaptations and modifications of the described embodiments can be made. Other embodiments may be apparent to those of ordinary skill in the art from consideration of the specification and practice of the invention disclosed herein. The specification and examples are intended to be considered as exemplary only, and the true scope and spirit of the invention are indicated by the following claims. Also, the series of steps shown in the figures are intended to be illustrative only and not limited to a particular series of steps. Thus, those of ordinary skill in the art can understand that these steps can be performed in a different order while implementing the same method.

[0062] In the drawings and specification, exemplary embodiments are disclosed. However, many variations and modifications can be made to these embodiments. Therefore, although specific terms are used, they are used only in a general and descriptive sense and not for purposes of limitation.

Claims

1. A method for compensating a virtual image displayed by a near-eye display based on a microdisplay projector, comprising: obtaining a virtual image displayed by the near-eye display, wherein the virtual image is formed by a source image emitted from the microdisplay projector; preprocessing the image data of the virtual image to obtain preprocessed image data; obtaining the relationship between the source image and the virtual image; determining an image baseline value of the virtual image; obtaining a compensation coefficient matrix including compensation coefficients of each pixel in the source image based on the relationship and the image baseline value; A method comprising the above steps.

2. Obtaining the relationship between the source image and the virtual image further includes: calculating a mapping ratio between the source image and the virtual image; calculating a source image data matrix based on the preprocessed image data and the mapping ratio, wherein the source image data matrix includes the same pixel dimensions as the source image; determining the source image data matrix to represent the relationship between the source image and the virtual image; The method according to claim 1, further comprising the above steps.

3. The method according to claim 1, wherein the virtual image is obtained by an image light measurement device.

4. The method according to claim 1, wherein the source image is a full white image.

5. The method according to claim 4, wherein the source image includes a plurality of partial on patterns, and the partial on patterns are stacked to form the full white image.

6. Preprocessing the image data of the virtual image to obtain preprocessed image data further includes: determining a region of interest in the virtual image; processing the image data within the region of interest; The method according to claim 1, further comprising the above steps.

7. The method according to claim 6, wherein the region of interest is determined by a preset threshold, and an average image data value within the region of interest is greater than or equal to the preset threshold.

8. The method according to claim 7, wherein an image data value of each pixel within the region of interest is greater than or equal to the preset threshold.

9. The method according to claim 7, wherein the virtual image is divided into the region of interest and a dark region around the region of interest.

10. Processing the image data within the region of interest includes excluding noise spots from the region of interest and correcting distortion of the virtual image. The method according to claim 6, further comprising the above steps.

11. The distortion of the virtual image x ｃｏｒｒ = x ｏｒｉｇ (1 + k １ r ２ + k ２ r ４ + k ３ r ６ ) y ｃｏｒｒ = y ｏｒｉｇ (1 + k １ r ２ + k ２ r ４ + k ３ r ６ ) is corrected by (( ｘｃｏｒｒ , y ｃｏｒｒ ) is the coordinate of the pixel in the virtual image after distortion correction corresponding to the original coordinate (x ｏｒｉｇ , y ｏｒｉｇ ), r represents the distance of the pixel to the center of the virtual image, and k １ , k ２ , k ３ are the coefficients of the distortion parameters.) the method according to claim 10.

12. The method according to claim 2, wherein the mapping ratio is determined by the full field size of the virtual image, the full field size of the source image, the dimensions of the virtual image, and the dimensions of the source image.

13. The mapping ratio R = R １ / R ２ is calculated by, where R １ = D １ / FOV １ 、R ２ = D ２ / FOV ２ where R is the mapping ratio, D １ is the dimension of the imaging optical measurement device (LMD) used to obtain the virtual image, FOV １ is the active field of view of the imaging LMD, D ２ is the active emission area of the micro-light emitting array in the microdisplay projector, FOV ２ is the active field of view of the microdisplay projector. is as described in the method according to claim 12.

14. Determining the image baseline value of the virtual image includes obtaining an image histogram of the virtual image data and determining the maximum proportional value within the image histogram as the image baseline value. The method according to claim 1, further comprising the above steps.

15. Determining the image baseline value of the virtual image includes calculating the average gray value of all pixels within the virtual image and determining the average gray value as the image baseline value. The method according to claim 1, further comprising the above steps.

16. The source image data matrix [M] ｏｒｉｇ = [M １ / R is obtained by In the formula, [M] ｏｒｉｇ is the source image data, R is the mapping ratio, and M １ is the preprocessed image data matrix of the preprocessed image data the method according to claim 2.

17. The compensation coefficient matrix [M] ｃｏｍｐ = ([M] ｂａｓｅｌｉｎｅ / [M] ｏｒｉｇ ) - 1 is obtained by where [M] ｃｏｍｐ is the compensation coefficient matrix, and [M] ｂａｓｅｌｉｎｅ is the baseline matrix of the image baseline value. the method according to claim 16.

18. Adjusting the image data of each pixel of the virtual image based on the compensation coefficient matrix is further included in the method according to claim 2.

19. Adjusting the image data of each pixel of the virtual image based on the compensation coefficient matrix includes adjusting the image data of each pixel within the source image based on the compensation coefficient matrix. The method according to claim 18, including the above step.

20. The image data of each pixel within the source image GV ｃｏｍｐ = round(GV ｏｒｉｇ ×([M] ｃｏｍｐ + 1)) is adjusted by wherein, GV ｃｏｍｐ is an adjusted image data matrix including the adjusted image data of each pixel, GV ｏｒｉｇ is the source image data matrix, and [M] ｃｏｍｐ is the compensation coefficient matrix. the method according to claim 19.

21. The microdisplay projector includes a microdisplay panel and a lens, the microdisplay panel includes a micro light emitting array, and adjusting the image data of each pixel of the virtual image based on the compensation coefficient matrix includes adjusting the image data of each pixel of the micro light emitting array based on the compensation coefficient matrix. The method according to claim 18, including the above step.

22. The method according to any one of claims 1 to 21, wherein the image data includes gray values or luminance values.

23. The method according to any one of claims 18 to 22, further comprising displaying an updated virtual image.

24. The method according to any one of claims 1 to 20, wherein the microdisplay projector includes a microdisplay panel and a lens, and the microdisplay panel includes a micro light emitting array.

25. The method according to claim 24, wherein the microdisplay panel is one of a micro inorganic LED (light emitting diode) display panel, a micro OLED (organic light emitting diode) display panel, a DLP (Digital Light Processing) display panel, or a micro LCD (liquid crystal display) display panel.

26. The method according to any one of claims 1 to 25, wherein the near-eye display is one of an augmented reality display, a virtual reality display, a head-up display, or a head-mounted display.

27. A method for evaluating compensation of a virtual image displayed by a near-eye display based on a microdisplay projector, comprising: acquiring a first virtual image displayed by the near-eye display, wherein the virtual image is formed by a source image emitted from the microdisplay projector; preprocessing the image data of the first virtual image to obtain preprocessed image data; acquiring the relationship between the source image and the virtual image; determining an image baseline value of the first virtual image; evaluating the non-uniformity of the first virtual image; acquiring a compensation coefficient matrix including compensation coefficients of each pixel in the source image based on the relationship and the image baseline value; adjusting the image data of each pixel for the first virtual image based on the compensation coefficient matrix and displaying a second virtual image; reevaluating the non-uniformity of the second virtual image; comparing the non-uniformity of the second virtual image with the non-uniformity of the first virtual image; and including.

28. acquiring the relationship between the source image and the virtual image is Calculating a mapping ratio between the source image and the virtual image; Calculating a source image data matrix based on the preprocessed image data and the mapping ratio, wherein the source image data matrix includes the same pixel dimensions as the source image; Determining the source image data matrix so as to represent the relationship between the source image and the virtual image; The method according to claim 27, further comprising.

29. The non-uniformity of the first virtual image is [M] ｎｏｎ = ([M] ｏｒｉｇ - [M] ｂａｓｅｌｉｎｅ ) / [M] ｂａｓｅｌｉｎｅ Calculated by where [M] ｎｏｎ is the non-uniformity of the first virtual image, and [M] ｏｒｉｇ is the source image data, and [M] ｂａｓｅｌｉｎｅ is a baseline matrix that matches the image baseline value The method according to claim 27.

30. Re-evaluating the non-uniformity of the second virtual image is Determining a plurality of regions uniformly distributed within the second virtual image; Summing the luminance values of the plurality of regions; L ａｖ = S / N Calculating an average luminance value of the second virtual image based on, wherein S is the sum of the luminance values of the plurality of regions and N is the number of regions; Calculating an average luminance value; U ｎ = L ｎ / L ａｖ Obtaining a uniformity value for each of the plurality of regions by where U ｎ is the uniformity value in region n, and L ｎ is the luminance value of the region n, and L ａｖ is the average luminance value of the second virtual image, obtaining the uniformity value of each of the plurality of regions The method according to any one of claims 27 to 29, further comprising.

31. NU = 1 - U ｍａｘ evaluating the non-uniformity of the second virtual image by Further comprising where NU is the non-uniformity of the second virtual image, and U ｍａｘ is the maximum value of the uniformity values among the N regions The method according to claim 30.

32. Evaluating the non-uniformity of the first virtual image is Determining the plurality of regions within the first virtual image; Calculating a uniformity value for each of the plurality of regions within the first virtual image Further comprising Comparing the non-uniformity of the second virtual image with the non-uniformity of the first virtual image is Comparing the non-uniformity of each of the plurality of regions of the second virtual image with the non-uniformity of the same region of the plurality of regions of the first virtual image Further comprising The method according to claim 30 or 31.

33. Adjusting the image data of each pixel of the virtual image based on the compensation coefficient matrix is Adjusting the image data of each pixel within the source image based on the compensation coefficient matrix The method according to claim 27, comprising.

34. The microdisplay projector includes a microdisplay panel and a lens, the microdisplay panel includes a micro light-emitting array, and adjusting the image data of each pixel of the virtual image based on the compensation coefficient matrix is Adjusting the image data of each pixel of the micro light-emitting array based on the compensation coefficient matrix The method according to claim 27, comprising

35. An apparatus for compensating a virtual image displayed by a near-eye display based on the microdisplay projector, a memory configured to store instructions, one or more processors configured to execute the instructions to cause the apparatus to perform the method according to any one of claims 1 to 26, the apparatus comprising.

36. An apparatus for evaluating the compensation of a virtual image displayed by a near-eye display based on a microdisplay projector, a memory configured to store instructions, one or more processors configured to execute the instructions to cause the apparatus to perform the method according to any one of claims 27 to 34, the apparatus comprising.

Images