Method and system for correcting non-uniformity of near-eye displays

Abstract

A method for correcting non-uniformity of a near-eye display (NED) having a first display and a second display, the method comprising: generating and displaying a test pattern for the first display and the second display; acquiring a first image at an end of a first optical path coupled to the first display and a second image at an end of a second optical path coupled to the second display in response to the test pattern; fusing the first image and the second image to generate a fused image; and determining a correction scheme for correcting non-uniformity of the NED based on the fused image.

Description

Technical Field

[0001] This disclosure generally relates to near-eye display technology, and more specifically, to a method and system for correcting inhomogeneities in near-eye displays. Background Technology

[0002] Near-eye displays (NEDs) can be provided as augmented reality (AR) displays, virtual reality (VR) displays, head-up displays / head-mounted displays, or other displays. Generally, a near-eye display typically includes an image generator and an optical path, with the optical path comprising an optical combiner. The image generator is typically a projector with a microdisplay (e.g., a micro-LED (light-emitting diode), a micro-OLED (organic light-emitting diode), LCOS (liquid crystal on silicon), or DLP (digital light processing)) and integrated optical lenses. The optical combiner includes reflective and / or diffractive optical elements, such as freeform mirrors / prisms, birdbaths, cascaded mirrors, or grating couplers (waveguides). Virtual images are presented from the NED to the human eye in the presence or absence of ambient light.

[0003] Uniformity is a key performance indicator for evaluating the image quality of a NED (Network ED). Non-uniformity can be caused by defects in the display pixels and the optical path guiding the light emitted from the display. Non-uniformity (also known as mura) exhibits variations in its global distribution as well as in localized areas. Visual artifacts can appear as mottled appearances, bright spots, black spots, or cloud-like patterns. For NEDs such as AR / VR displays, visual artifacts can also be observed in the virtual images presented in the display system. In virtual images presented in AR / VR displays, brightness and chromaticity may exhibit non-uniformity. Furthermore, visual artifacts caused by non-uniformity are more pronounced due to their proximity to the human eye compared to traditional displays.

[0004] Therefore, it is necessary to improve the non-uniformity of NED. Summary of the Invention

[0005] Embodiments of this disclosure provide a method for correcting inhomogeneities in a near-eye display (NED) including a first display and a second display. The method includes: generating and displaying a test pattern for the first display and the second display; acquiring a first image at the end of a first optical path coupled to the first display and a second image at the end of a second optical path coupled to the second display in response to the test pattern; fusing the first image and the second image to generate a fused image; and determining a correction scheme for correcting the inhomogeneities of the NED based on the fused image.

[0006] Embodiments of this disclosure provide a system for correcting inhomogeneities in a near-eye display (NED) including a first display and a second display. The system includes: an optical measurement device (LMD) configured to: acquire a first image at the end of a first optical path coupled to the first display and a second image at the end of a second optical path coupled to the second display in response to a test pattern displayed on the first and second displays; and a processor configured to: fuse the first image and the second image to generate a fused image; and determine a correction scheme for correcting the inhomogeneities of the NED based on the fused image.

[0007] Embodiments of this disclosure provide a non-transient computer-readable storage medium storing a set of instructions executable by one or more processors of a device to cause the device to perform operations for correcting inhomogeneities in a near-eye display (NED), the near-eye display having a first display and a second display, the operations including: generating and displaying a test pattern for the first display and the second display; acquiring a first image at the end of a first optical path coupled to the first display and acquiring a second image at the end of a second optical path coupled to the second display in response to the test pattern; fusing the first image and the second image to generate a fused image; and determining a correction scheme for correcting the inhomogeneities of the NED based on the fused image. Attached Figure Description

[0008] The following detailed description and accompanying drawings illustrate embodiments and aspects of this disclosure. The various features shown in the figures are not drawn to scale.

[0009] Figure 1 A schematic diagram of an exemplary system for correcting non-uniformity of NED according to some embodiments of the present disclosure is shown.

[0010] Figure 2 A schematic diagram of an exemplary VR system according to some embodiments of the present disclosure is shown.

[0011] Figure 3A A schematic diagram of an exemplary AR system according to some embodiments of the present disclosure is shown.

[0012] Figure 3B A schematic diagram illustrates an exemplary process for binocular inhomogeneity correction for NED according to some embodiments of the present disclosure.

[0013] Figure 4 A flowchart is shown of an exemplary method for correcting non-uniformity of NED according to some embodiments of the present disclosure.

[0014] Figure 5Some embodiments according to this disclosure are shown. Figure 4 The flowchart shows a sub-step of an exemplary method for correcting inhomogeneities in NED.

[0015] Figure 6 Some embodiments according to this disclosure are shown. Figure 4 The flowchart shows a sub-step of an exemplary method for correcting inhomogeneities in NED.

[0016] Figure 7 A schematic diagram illustrates an exemplary process for binocular inhomogeneity correction for NED according to some embodiments of the present disclosure.

[0017] Figure 8 Examples of intermediate images and corresponding fused images according to some embodiments of this disclosure are shown.

[0018] Figure 9 Examples of fused images before and after correction according to some embodiments of this disclosure are shown. Detailed Implementation

[0019] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. The following description refers to the accompanying drawings, wherein, unless otherwise stated, the same reference numerals in the different drawings denote the same or similar elements. The embodiments described below are not representative of all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with aspects related to this disclosure. Specific aspects of this disclosure are described below in more detail. In the event of any conflict with terms and / or definitions incorporated by reference, the terms and definitions provided herein shall prevail.

[0020] Figure 1 A schematic diagram of an exemplary system 100 for correcting non-uniformity of NEDs according to some embodiments of the present disclosure is shown. Figure 1 As shown, system 100 is used to correct inhomogeneities in near-eye display (NED) 110. Typically, NED 110 is used to display images to a human eye, and NED 110 can be included in AR or VR devices, such as head-up / head-mounted displays, projectors, or other displays. In this disclosure, system 100 is used to evaluate the imaging quality of NED 110 in place of the human eye and correct potential inhomogeneities accordingly.

[0021] In some embodiments, NED 110 includes an image generator 111. The image generator 111 is provided as one or more microdisplays (e.g., one display per eye), such as a micro-LED display, a micro-OLED display, an LCOS display, or a DLP display, and each of the microdisplays can be configured as a light engine with an additional projection lens. In some embodiments, the display can be coupled to multiple lenses (also referred to as “lens groups,” “design optics,” etc.) for adjusting the image displayed through the microdisplay in a manner suitable for the human eye. The microdisplay of image generator 111 includes a micro-light-emitting array that can form active emission regions. The image projected from the light engine by the designed optics is transmitted to the human eye via an optical path including an optical combiner (not shown). The optical elements of the optical combiner can be reflective and / or diffractive optical elements, such as freeform mirrors / prisms, birdbaths, cascaded mirrors, or grating couplers (waveguides), etc.

[0022] Image generator 111 includes a driver integrated circuit (IC). Figure 1 (Not shown). The driver IC includes the necessary software and hardware for driving the image generator 111. In some embodiments, a control board 112 for image display is also provided in the NED 110. The control board 112 can be coupled to communicate with the NED 110, specifically, to communicate with and control the image generator 111 of the NED 110. The driving system of the NED 110 may include the image generator 111 and the control board 112.

[0023] Figure 2 A schematic diagram of an exemplary VR system 200 according to some embodiments of the present disclosure is shown. Figure 3A A schematic diagram of an exemplary AR system 300 according to some embodiments of the present disclosure is shown. References Figure 2 The VR system 200 includes a first microdisplay 210 (e.g., a right display) and a corresponding lens group 211 for adjusting (e.g., magnifying) the image displayed through the first microdisplay 210 in a manner suitable for the viewer's right eye. Similarly, the VR system 200 also includes a second microdisplay 220 (e.g., a left display) and a corresponding lens group 221 for adjusting the image displayed through the second microdisplay 220 in a manner suitable for the viewer's left eye.

[0024] It should be noted that the terms "left" and "right" used in this disclosure refer to... Figure 2This is as seen from the field of vision of a person (i.e., the user or viewer of system 200). In this disclosure, the light path from the microdisplay to the human eye is also referred to as the optical path, which may include multiple optical elements. That is, lens group 211 and lens group 221 are located within their respective optical paths and are regarded as optical components of their respective optical paths.

[0025] Understandably, the first microdisplay 210 can be used to display the right image, while the second microdisplay 220 can be used to display the left image, which is captured or presented at a different angle than the right image. When the left and right images are viewed simultaneously, the viewer's brain combines the two images into a three-dimensional scene. However, if the uniformity within the left or right image, or the uniformity between the left and right images, is not ideal, the "presence" of the three-dimensional scene created by the two images may be affected. This non-uniformity may be caused by one or more of the second microdisplay 220, the first microdisplay 210, the lens group 211, or the lens group 221. For example, when driven by a signal indicating the same brightness intensity, some pixels in either or both of the first microdisplay 210 and the second microdisplay 220 may be brighter than the others.

[0026] refer to Figure 3A The AR system 300 includes a first microdisplay 310 (e.g., a right display) and a corresponding optical path 320 for transmitting an image displayed through the first microdisplay 310 to a viewer's right eye. As shown, the optical path 320 includes a lens group 321, a waveguide 322, and an optical combiner 323. The lens group 321 is configured to adjust the image displayed through the first microdisplay 310. The waveguide 322 can be used to guide light 330 emitted from the first microdisplay 310 through multiple total internal reflections. The optical combiner 323 guides the light 330 emitted from the first microdisplay 310 and allows ambient light 340 to pass through. Thus, both light 330 and ambient light 340 can reach the viewer's right eye, and the viewer sees an image superimposed on the ambient scene. Similarly, the AR system 300 also includes a second microdisplay 350 (e.g., a left display) and a corresponding optical path 360 for transmitting an image displayed through the second microdisplay 350 to the left eye. Typically, the second microdisplay 350 has the same resolution as the first microdisplay 310. The optical path 360 consists of a lens group 361, a waveguide 362, and an optical combiner 363. It is understood that the viewed image can be affected by any object between the display and the viewer's eye. That is, the viewed image can be affected by one or more of the first microdisplay 310, the second microdisplay 350, the optical path 320, or the optical path 360. When non-uniformity exists in the viewed image, the imaging effect of the AR system 300 deteriorates.

[0027] Return to reference Figure 1 System 100 includes an imaging module (e.g., an imager) 101 and a processing module (e.g., a processor) 104. Imaging module 101 is configured to simulate the human eye to measure display optical properties and observe display performance. In some embodiments, imaging module 101 may include an optical measurement device (LMD) 103 and a lens 102. For example, LMD 103 may be a colorimeter or an imaging camera, such as a CCD (charge-coupled device) or CMOS (complementary metal-oxide-semiconductor) image sensor. Lens 102 may be an NED lens or a conventional lens, measured in absolute or relative values. Lens 102 of imaging module 101 has a front aperture with a small diameter, such as 1 mm to 6 mm. Lens 102 can provide a wide field of view (FOV) in front and is configured to simulate the human eye to observe NED 110. The optical properties of the virtual image displayed through NED 110 are captured by imaging module 101 and measured by processing module 104.

[0028] Processing module 104 is configured to evaluate and improve the uniformity of the virtual image presented by NED 110. In some embodiments, processing module 104 may be included in a computer or server. In some embodiments, processing module 104 may be deployed in the cloud, which is not limited herein. In some embodiments, processing module 104 may include one or more processors.

[0029] Figure 3B A schematic diagram illustrates an exemplary process for binocular inhomogeneity correction for NED according to some embodiments of the present disclosure. Figure 3B As shown, this process involves measuring the distribution of the left and right images (in...) Figure 3B The left and right images of the NED are represented by "left distribution" and "right distribution" respectively to characterize their imaging quality. The left and right images are generated by, for example, two 3-channel displays (e.g., red, green, and blue) included in the NED. The image quality features of the left and right images can then be processed together. Figure 3B This is represented as "uni-processing": First, uni-processing fuses the two images, resulting in bino-fusion nonuniformity in the fused image. Second, the uniform object for each pixel in the fused image can be determined (in... Figure 3B In this context, it is represented as a "Uni-Object," such as the target pixel value. Finally, the unification process determines a coefficient for each pixel (in...). Figure 3BThe coefficients, denoted as "Uni-Comp Coeff" (e.g., a coefficient matrix), are used to correct for detected non-uniformities in the fused image. The determined coefficients can be assigned to correct the left and right images. Specifically, the coefficients can be sent to the left and right displays of the NED for correction. Alternatively, in some embodiments, coefficients for the left and right images can be generated based on left and right distributions, respectively. In this alternative, the left and right images are corrected without considering the uniformity between the two images.

[0030] Figure 4 A flowchart illustrating an exemplary method 400 for correcting non-uniformity of a NED including a first display and a second display, according to some embodiments of this disclosure, is shown. The NED can be a VR system or an AR system, as referred to above respectively. Figure 2 and Figure 3A The first display and the second display can be a first micro-display and a second micro-display, respectively. Method 400 includes steps S402 to S408, which can be performed by a measurement system (such as...) Figure 1 The system 100 is implemented.

[0031] In step S402, one or more test patterns are generated for display on the first and second displays. For example, further reference... Figure 3A In the AR system 300, to measure the imaging characteristics of the left imaging branch including the first microdisplay 310 and the optical path 320, and the right imaging branch including the second microdisplay 350 and the optical path 360, a common test pattern for the first microdisplay 310 and the second microdisplay 350 can be generated. In some embodiments, only a subset of pixels in the first microdisplay 310 and the second microdisplay 350 needs to be corrected. Therefore, a test pattern for the subset of pixels can be generated. For example, a red image for red pixels can be generated to correct for non-uniformity caused by red pixels.

[0032] In some embodiments, all pixels in the first microdisplay 310 and the second microdisplay 350 require calibration. In this case, multiple test patterns can be generated for the displays to measure their performance under different imaging conditions. For example, since the first microdisplay 310 and the second microdisplay 350 include three basic pixels: red, green, and blue, complete non-uniformity correction of the AR system 300 may require testing all of these pixels. That is, multiple test patterns can be generated to illuminate all red, green, and blue pixels to observe their characteristics. Since the first microdisplay 310 and the second microdisplay 350 are typically driven by signals in the RGB color space, in some embodiments, the multiple test patterns may include a standard red pattern, a standard green pattern, and a standard blue pattern in the RGB color space. Thus, all red, green, and blue pixels of the first microdisplay 310 and the second microdisplay 350 can be illuminated sequentially by the standard red pattern, standard green pattern, and standard blue pattern.

[0033] In some embodiments, the test pattern may be a white pattern in the RGB color space. It is understood that brightness non-uniformity is more easily observed by a viewer than chromaticity non-uniformity. Typically, the white points in the test pattern are represented by the red, green, and blue pixels in the display. Therefore, the white pattern will trigger at least a portion of the red, green, and blue pixels, and correction of these pixels will mitigate the non-uniformity of the first microdisplay 310 and the second microdisplay 350, and / or the non-uniformity between both.

[0034] As described above, waveguide 322 and optical combiner 323 are disposed in optical path 320, and waveguide 362 and optical combiner 363 are disposed in optical path 360. The imaging quality of AR system 300 may also be affected by optical path 320 or optical path 360. Specifically, at least a portion of the non-uniformity in AR system 300 may be caused by optical combiner 323 and optical combiner 363. The correction method provided in this disclosure eliminates or at least reduces the source of such non-uniformity.

[0035] Return to reference Figure 4 In step S404, in response to the test pattern displayed on the first and second displays, a first image is obtained at the end of the first optical path coupled to the first display, and a second image is obtained at the end of the second optical path coupled to the second display. For example, further refer to... Figure 3A Imaging module (e.g., Figure 1The imaging module 101 in the AR system 300 can be positioned at the end of the optical path 320 to capture a right image corresponding to the display test pattern shown on the first microdisplay 310. In this disclosure, the imaging module can be positioned at a distance from the optical combiner 323 of the AR system 300 similar to that of the eye. "At the end" means that the imaging module is positioned after the end of the optical path and can obtain a complete image, for example, that is visible to the eye. In other words, the imaging module may not be in contact with the end of the optical path. Similarly, another imaging module (e.g., Figure 1 The imaging module 101 can be positioned at the end of the optical path 360 to capture a left image corresponding to the display test pattern shown on the second microdisplay 350. In some other embodiments, an imaging module can be used to capture both the left and right images. The display time of the left and right images can be long enough for the imaging module to move from one end of the optical path to the other.

[0036] return Figure 4 In step S406, the first image and the second image are merged to generate a merged image. Figure 7 A schematic diagram illustrates an exemplary process for binocular inhomogeneity correction for NED according to some embodiments of the present disclosure. Figure 7 As shown, the left and right images are fused together to form a fused image. A fused image is a combined image in the human brain of the left image seen by the left eye and the right image seen by the right eye. Besides the examples given below, the left and right images can be fused in various ways. This disclosure is not limited to the way the fused image is generated.

[0037] Figure 5 A flowchart illustrating the sub-steps of a method 400 for correcting non-uniformity of NED according to some embodiments of the present disclosure is shown. Figure 5 As shown, step S406 includes sub-steps S502 and S504.

[0038] In sub-step S502, the first image and the second image are downsampled to obtain a first intermediate image and a second intermediate image with a target resolution. Images captured by the imaging module can have a fine resolution (e.g., 9000×6000), which may be too large for image processing. In some embodiments, the resolution of these images can be reduced by pixel decimation. As used herein, pixel decimation refers to the process of reducing the number of pixels in an image, such as downsampling. For example, further reference... Figure 3AThe target resolution can be set to be equal to the display resolution of the first microdisplay 310 (or the second microdisplay 350), for example, 640×480. In one example, 640×480 pixels are selected from the 9000×6000 pixels of the first and second images to represent the first and second images, referred to as the first intermediate image and the second intermediate image. For example, the 9000×6000 pixels are extracted in a uniform manner in both the horizontal and vertical directions, i.e., the number is reduced. In some other examples, the first and second images are scaled in such a way that the representative pixel is the average of its nearest pixels.

[0039] In sub-step S504, the first intermediate image and the second intermediate image are merged to form a merged image. Figure 8 Examples of intermediate images and corresponding fused images according to some embodiments of this disclosure are shown. Continuing with the above embodiments, three test patterns in the RGB color space can be displayed sequentially, including a standard red pattern, a standard green pattern, and a standard blue pattern. When on a first display and a second display (e.g., Figure 3A When a standard red pattern is displayed on the first microdisplay 310 and the second microdisplay 350, the imaging module (e.g., Figure 1 The imaging module 101 can acquire a left image and a right image. In some embodiments, the first image and the second image are represented in the CIE (International Commission on Illumination) XYZ chromaticity space. Therefore, the first intermediate image, the second intermediate image, and the fused image are also represented in the XYZ chromaticity space. Therefore, the first intermediate image, the second intermediate image, and the fused image can be decomposed into CIE X components, CIE Y components, and CIE Z components. Figure 8 As shown, image 811 represents the left-center image of the standard red pattern represented in the CIE X component, image 812 represents the left-center image of the standard red pattern represented in the CIE Y (luminance) component, and image 813 represents the left-center image of the standard red pattern represented in the CIE Z component. Similarly, image 821 represents the right-center image of the standard red pattern represented in the CIE X component, image 822 represents the right-center image of the standard red pattern represented in the CIE Y (luminance) component, and image 823 represents the right-center image of the standard red pattern represented in the CIE Z component. Furthermore, image 831 represents a fused image of the first and second intermediate images represented in the CIE X component, image 832 represents a fused image of the first and second intermediate images represented in the CIE Y (luminance) component, and image 833 represents a fused image of the first and second intermediate images represented in the CIE Z component.

[0040] The fusion method described above for forming a fused image can also be applied to standard green and standard blue patterns, and will not be repeated here. In this disclosure, although images denoted as "CIE X-RED-LEFT", "CIE Z-RED-LEFT", "CIE X-RED-RIGHT", "CIE Z-RED-RIGHT", "CIE X-RED", and "CIE Z-RED" are used to represent the chromaticity components of an image, they can be represented in grayscale to show the intensity of the X, Y, and Z components of each pixel.

[0041] In some embodiments, the first and second images are represented in grayscale, and the grayscale value of the pixel at the target location in the fused image (e.g., coordinates (x, y) in a 640×480 image) is equal to the grayscale value of the pixel at the target location in the first intermediate image plus the grayscale value of the pixel at the target location in the second intermediate image. This process can be represented by the following formula: Gray_fusion (x, y) = Gray_left (x, y) + Gray_right (x, y) (1) in, Gray_fusion(x, y) This represents the grayscale value of the pixel at the target location in the merged image. Gray_left (x, y) This represents the grayscale value of the pixel at the target location in the left image, while Gray_right(x, y) This represents the grayscale value of the pixel at the target location in the right image.

[0042] In some other embodiments, the grayscale values ​​of pixels at the target location in the fused image can be calculated in a more complex manner that is more consistent with the working mechanism of the human visual system. This disclosure is not limited to the method of calculating the grayscale values ​​of pixels at the target location.

[0043] In some embodiments, the correction method is applied to both luminance and chromaticity nonuniformity. Therefore, in step S504, the luminance and chromaticity components of the first intermediate image and the second intermediate image are fused to form a fused image. In some embodiments, the correction method may focus on luminance nonuniformity. Therefore, in step S504, only the luminance components of the first intermediate image and the second intermediate image are fused to form a fused image. That is, further referencing... Figure 8 Only the images represented as "CIE Y-RED" (i.e., images 812 and 822) are retained for fusion to form a fused image for further processing, while the images represented as "CIE X-RED" (i.e., images 811 and 821) and "CIE Z-RED" (i.e., images 813 and 823) are discarded.

[0044] Return to reference Figure 4In step S408, a correction scheme for correcting inhomogeneities in the NED is determined based on the fused image. In this way, the correction scheme can be determined based on binocular fusion, and the determined scheme will reflect the true working mechanism of the human visual system. Furthermore, inhomogeneities in the two optical paths and inhomogeneities between the two optical paths can be corrected in a single process.

[0045] Figure 6 A flowchart illustrating the sub-steps of a method 400 for correcting non-uniformity of NED according to some embodiments of the present disclosure is shown. Figure 6 As shown, step S408 further includes the following sub-steps S602 and S604.

[0046] In sub-step S602, a target grayscale value for the fused image is set based on the distribution of grayscale values ​​of pixels in the fused image. In some embodiments, the target grayscale value is the average grayscale value of each pixel in the fused image. In some other embodiments, the target grayscale value is the grayscale value with the highest probability in the distribution. The target value typically reflects the statistical state of the fused image and can be used to represent the fused image.

[0047] refer to Figure 8 For the first and second intermediate images, the distribution and uniformity of luminance (CIE Y component) and chromaticity (CIE X and CIE Z components) for each channel (i.e., the standard red pattern, standard green pattern, and standard blue pattern) can be obtained. Then, the target luminance and target chromaticity of the display can be determined based on the fused image of the first and second intermediate images. For example, as described above, the target luminance can be calculated by considering the average value or the maximum value in the probability distribution of all pixels. The target chromaticity can be determined based on the chromaticity distribution (e.g., the distribution of the CIE X and CIE Z components). The target chromaticity can also consider the white point or standard color temperature (e.g., D65 or D55), and the color temperature of the target chromaticity can be shifted towards the standard color temperature. Once the target grayscale values ​​of CIE X, CIE Y, and CIE Z are determined, the correction target for each pixel in the first and second displays is set, which can be represented as the following target grayscale value matrix. : (2).

[0048] In this 3×3 matrix, Represents the chromaticity components under the standard red pattern. The target grayscale value, Represents the luminance component under the standard green pattern. The target grayscale value, Represents the chromaticity components under a standard blue pattern. The target grayscale value, etc. Understandably, this matrix implements both luminance and chrominance correction. If only luminance correction needs to be applied, only the second row needs to be determined. And the matrix becomes a 1×3 matrix.

[0049] Return to reference Figure 6 In sub-step S604, a correction matrix is ​​determined to map the grayscale value of each pixel in the fused image to a target grayscale value. In some embodiments, a standard red pattern, a standard green pattern, and a standard blue pattern contribute to forming a portion of the correction matrix. As described above, the correction matrix... Used to merge CIEXYZ in the image The grayscale value of any pixel represented in the image is mapped to the target grayscale value. Therefore, this process can be represented by the following formula: (3).

[0050] In other words, the correction matrix It can be determined using the following formula: (4) in, Representation matrix The inverse matrix, It contains the actual pixel values ​​that can be determined from the merged image. As understood, it is possible to generate values ​​for each pixel in a 640×480 pixel array. .

[0051] In some embodiments, gamma operations in the display driving system may also be considered when determining the correction scheme. For example, gamma operators can be used. To update the correction matrix used for each pixel in the fused image: = (5) when When applied for correction, the NED's driving system will not perform another gamma correction on the pixel.

[0052] In some embodiments, the determined correction matrix can be saved for further processing. In other embodiments, the determined correction matrix for each pixel in the fused image can be used to update the NED's driving system based on the correction matrix. The driving system can then drive the first and second displays using corresponding driver files with their respective correction matrices. For example, when the display pixels of the first or second display are driving signals in the RGB color space... At that time, the drive system corrects the drive signal to It is actually used to drive the pixels. When considering gamma operation, the driving system corrects this drive signal to... .

[0053] To review the improvement quality of the correction scheme, the uniformity of the first and second displays can be evaluated before and after correction. In some embodiments, the method for correcting non-uniformity further includes the steps (not shown): displaying a test pattern on the first and second displays using an updated driving system; and verifying the updated uniformity of the first and second displays based on an updated fused image of the test pattern.

[0054] Figure 9 Examples of experimental results for fused images before and after correction according to some embodiments of this disclosure are shown. From Figure 9 The experimental results shown demonstrate that by applying the above method, the non-uniformity present before correction can be eliminated or at least reduced.

[0055] Embodiments of this disclosure also provide a non-transient computer-readable storage medium storing a set of instructions executable by one or more processors of a device to cause the device to perform the method described above for correcting non-uniformity of NED.

[0056] It should be noted that relational terms such as “first” and “second” in this document are used only to distinguish one entity or operation from another, and do not require or imply any actual relationship or order between these entities or operations. Furthermore, the words “contains,” “has,” “includes,” and “includes,” as well as other similar forms, are intended to have the same meaning and are open-ended, because one or more items following any of these words are not intended to be an exhaustive list of such one or more items, or to be limited to only the listed one or more items.

[0057] As used herein, unless otherwise expressly stated, the term "or" covers all possible combinations unless impractical. For example, if it is specified that a database may include A or B, then unless otherwise expressly stated or impractical, the database may include A or B, or A and B. As a second example, if it is specified that a database may include A, B, or C, then unless otherwise expressly stated or impractical, the database may include A, or B, or C, or A and B, or A and C, or B and C, or A and B and C.

[0058] In the foregoing description, numerous specific details have been described, which may vary depending on the implementation. Certain adjustments and modifications may be made to some of the described embodiments. Other embodiments will be apparent to those skilled in the art upon consideration of the description and practice of this disclosure herein. This specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims. The sequence of steps shown in the figures is for illustrative purposes only and is not intended to limit the scope to any particular sequence of steps. Therefore, those skilled in the art will understand that these steps may be performed in a different order while implementing the same method.

[0059] Exemplary embodiments have been disclosed in the accompanying drawings and description. However, many variations and modifications can be made to these embodiments. Therefore, although specific terms are used, they are used in a general and descriptive sense only and not for limiting purposes.

Claims

1. A method for correcting inhomogeneities in a near-eye display (NED), the NED having a first display and a second display, characterized in that, The method includes: Generate and display test patterns for the first display and the second display; In response to the test pattern, a first image is acquired at the end of the first optical path coupled to the first display and a second image is acquired at the end of the second optical path coupled to the second display; The first image and the second image are merged to generate a merged image; and A correction scheme for correcting the inhomogeneity of the NED is determined based on the fused image.

2. The method according to claim 1, characterized in that, The first optical path and the second optical path each include an optical combiner, and at least a portion of the non-uniformity is caused by the optical combiner.

3. The method according to claim 1, characterized in that, Fusing the first image and the second image to generate the fused image includes: The first image and the second image are downsampled to obtain a first intermediate image and a second intermediate image with the target resolution, respectively; and The first intermediate image and the second intermediate image are merged to form the merged image.

4. The method according to claim 3, characterized in that, The target resolution is equal to the display resolution of the first display and the second display.

5. The method according to claim 3, characterized in that, Fusing the first intermediate image and the second intermediate image to form the fused image includes: The luminance components of the first intermediate image and the second intermediate image are fused to form the fused image.

6. The method according to claim 5, characterized in that, Fusing the first intermediate image and the second intermediate image to form the fused image includes: The chromaticity components of the first intermediate image and the second intermediate image are fused to form the fused image.

7. The method according to claim 3, characterized in that, The first image and the second image are represented by grayscale values, and the grayscale value of the pixel at the target position of the fused image is equal to the grayscale value of the pixel at the target position of the first intermediate image plus the grayscale value of the pixel at the target position of the second intermediate image.

8. The method according to claim 7, characterized in that, The correction scheme for correcting the inhomogeneity of the NED based on the fused image includes: The target grayscale value for the fused image is set based on the distribution of grayscale values ​​of pixels in the fused image; and A correction matrix is ​​determined to map the grayscale value of each pixel in the fused image to the target grayscale value.

9. The method according to claim 8, characterized in that, The correction scheme for correcting the inhomogeneity of the NED based on the fused image further includes: The correction matrix for each pixel in the fused image is updated using a gamma operator.

10. The method according to claim 8, characterized in that, The target gray value is the average gray value of each pixel in the fused image, or the target gray value is the gray value with the highest probability in the distribution.

11. The method according to claim 8, characterized in that, The test pattern includes multiple patterns, and the target grayscale value is set according to each of the multiple patterns, wherein each of the multiple patterns contributes to forming a part of the correction matrix.

12. The method according to claim 11, characterized in that, The test patterns include standard red, standard green, and standard blue patterns in the RGB color space.

13. The method according to claim 8, characterized in that, The NED includes a driving system for driving the first display and the second display to display images, and the method further includes: The driving system of the NED is updated based on the correction matrix of each pixel in the fused image.

14. The method according to claim 13, characterized in that, Also includes: The test pattern is displayed on the first and second displays using an updated driving system; as well as The uniformity of updates between the first and second displays is verified based on the updated fused image of the test pattern.

15. The method according to claim 1, characterized in that, The first image, the second image, and the fused image are represented in the XYZ color space.

16. A system for correcting inhomogeneities in a near-eye display (NED), the NED comprising a first display and a second display, characterized in that, The system includes: The optical measurement device (LMD) is configured as follows: In response to a test pattern displayed on the first and second displays, a first image is acquired at the end of a first optical path coupled to the first display and a second image is acquired at the end of a second optical path coupled to the second display; and The processor is configured as follows: The first image and the second image are merged to generate a merged image; and A correction scheme for correcting the inhomogeneity of the NED is determined based on the fused image.

17. The system according to claim 16, characterized in that, The processor is also configured to generate the test pattern for the first display and the second display.

18. The system according to claim 16, characterized in that, The first optical path and the second optical path each include an optical combiner, and at least a portion of the non-uniformity is caused by the optical combiner.

19. The system according to claim 16, characterized in that, The processor is configured to: The first image and the second image are downsampled to obtain a first intermediate image and a second intermediate image with the target resolution, respectively; and The first intermediate image and the second intermediate image are merged to form the merged image.

20. The system according to claim 19, characterized in that, The first image and the second image are represented by grayscale values, and the grayscale value of the pixel at the target position of the fused image is equal to the grayscale value of the pixel at the target position of the first intermediate image plus the grayscale value of the pixel at the target position of the second intermediate image.

21. The system according to claim 20, characterized in that, The processor is configured to: The target grayscale value for the fused image is set based on the distribution of grayscale values ​​of pixels in the fused image; and A correction matrix is ​​determined to map the grayscale value of each pixel in the fused image to the target grayscale value.

22. The system according to claim 21, characterized in that, The NED includes a driving system for driving the first display and the second display to display an image, and the processor is further configured to update the driving system of the NED according to the correction matrix of each pixel in the fused image.

23. A non-transient computer-readable storage medium storing a set of instructions, characterized in that, The set of instructions can be executed by one or more processors of the device to cause the device to perform operations for correcting inhomogeneities in a near-eye display (NED), the near-eye display having a first display and a second display, the operations including: Generate and display test patterns for the first display and the second display; In response to the test pattern, a first image is acquired at the end of the first optical path coupled to the first display and a second image is acquired at the end of the second optical path coupled to the second display; The first image and the second image are merged to generate a merged image; and A correction scheme for correcting the inhomogeneity of the NED is determined based on the fused image.

Images