Method and system for Mura detection and Demura

Abstract

A method of detecting a Mura of a virtual image in a near-eye display (NED) and a Demura method are provided, and a method of detecting a Mura of a virtual image in a near-eye display (NED) is provided. The method for detecting Mura comprises: acquiring the virtual image presented in the near-eye display (110, 410, NED); extracting a Mura feature of the virtual image according to a Mura type (320); and evaluating the Mura degree of the virtual image based on the Mura type. The Demura method comprises: acquiring a Mura feature of a first virtual image presented in the near-eye display (110, 410, NED) (320); calculating a compensation factor based on the Mura feature (320); and adjusting a gray value of the near-eye display (110, 410, NED) based on the compensation factor to obtain a second virtual image.

Description

Technical Field

[0001] The present disclosure relates generally to microdisplay technology, and more particularly to a demura system and method for presenting virtual images in a near-eye display. Background Art

[0002] Micro light emitting diode (LED) display panels have the advantages of smaller size, higher refresh rate, and higher brightness. However, micro LED display panels have Mura (such as non-uniformity of micro LED display panels) due to production processes or long operation time, resulting in residual images, mottled spots, bright or black spots, or a cloud-like appearance, which reduces the quality level of micro LED display panels. The non-uniformity of the display panel can usually be compensated by the Demura method. The Demura method is used to remove the non-uniformity of the LED display panel or improve the uniformity. The non-uniformity of the micro LED display panel is usually compensated directly by adjusting the grayscale value of the pixels in the micro LED display panel.

[0003] Near-eye displays can be provided as AR displays, VR displays, head-up displays / head-mounted displays, or other displays. In general, near-eye displays generally include an image generator and an optical combiner that transmits a projected image from the image generator to the human eye. In addition, the projected image is a virtual image in front of the human eye. The image generator can be a micro-LED-based display, an LCOS (liquid crystal on silicon) display, or a DLP (digital light processing) display. The above-mentioned Mura of the micro-LED display panel may affect the quality of the final virtual image delivered to the human eye. The source image presents brightness and color variations between pixels, which is seen as non-uniformity in brightness and / or chromaticity distribution. Non-uniformity originating from the image generator also causes non-uniformity in the final virtual image presented. Mura appears as non-uniformity of the display and can be observed by human vision. Therefore, Mura is an important visual artifact of the display. In addition, compared with traditional displays, the non-uniform artifacts of near-eye displays are more obvious due to their proximity to the human eye. However, there is a need for a way to detect Mura for near-eye displays and apply Demura in the final virtual image presented. Summary of the invention

[0004] An embodiment of the present disclosure provides a method for detecting mura of a virtual image in a near-eye display. The method comprises: acquiring the virtual image presented in the near-eye display; extracting mura features of the virtual image according to mura types; and evaluating the mura degree of the virtual image based on the mura types.

[0005] The embodiments of the present disclosure also provide a method for demodulating a virtual image presented in a near-eye display. The method comprises: obtaining a mura feature of a first virtual image presented in the near-eye display; calculating a compensation factor based on the mura feature; and adjusting the grayscale value of the near-eye display based on the compensation factor to obtain a second virtual image.

[0006] Embodiments of the present disclosure further provide a system for detecting mura in a virtual image presented in a near-eye display. The system includes: an image generator configured to present a virtual image; an imager configured to acquire the virtual image; a positioner coupled to the image generator and the imager and configured to control the relative position of the near-eye display and the imager; and a processor coupled to the imager and configured to evaluate the mura of the virtual image.

[0007] The embodiments of the present disclosure further provide a system for demodulating a virtual image presented in a near-eye display. The system includes: an image generator configured to present a first virtual image; an imager configured to acquire the virtual image; a positioner coupled to the image generator and the imager and configured to control the relative position of the image generator and the imager; a mura extractor coupled to the imager and configured to extract mura features of the first virtual image; a compensation calculator coupled to the extractor and configured to calculate a compensation factor; and a driver coupled to the compensation calculator and the image generator and configured to adjust the grayscale value of the image generator based on the compensation factor to obtain a second virtual image. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Embodiments and various aspects of the present disclosure are presented in the following detailed description and accompanying drawings.The various features shown in the drawings are not drawn to scale.

[0009] Figure 1 is a schematic diagram of an exemplary Demura system according to some embodiments of the present disclosure.

[0010] Figure 2 A flow chart illustrating an exemplary Demura method according to some embodiments of the present disclosure is shown.

[0011] Figure 3 An exemplary Demura process according to some embodiments of the present disclosure is presented.

[0012] Figure 4is a schematic block diagram of an exemplary Demura system according to some embodiments of the present disclosure.

[0013] FIG. 5A to FIG. 5C Different types of Mura features according to some embodiments of the present disclosure are respectively shown.

[0014] Figure 6 A flow chart illustrating an exemplary mura extraction method according to some embodiments of the present disclosure is shown.

[0015] Figure 7 A flow chart illustrating an exemplary Demura method according to some embodiments of the present disclosure is shown.

[0016] Figure 8 An exemplary Demura process according to some embodiments of the present disclosure is presented. DETAILED DESCRIPTION

[0017] Reference will now be made in detail to exemplary embodiments, examples of which are shown in the accompanying drawings. The following description refers to the accompanying drawings, in which the same numbers in different drawings represent the same or similar elements, unless otherwise indicated. The implementations set forth in the following description of the exemplary embodiments do not represent all implementations consistent with the present invention. Instead, they are merely examples of devices and methods consistent with the aspects listed in the appended claims and related to the present invention. Specific aspects of the present disclosure are described in more detail below. In the event of a conflict with terms and / or definitions incorporated by reference, the terms and definitions provided herein shall prevail.

[0018] In order to improve image quality, demura for near-eye displays is needed. Demura refers to the process used to eliminate / suppress visual artifacts and achieve relative uniformity of brightness and / or color in a display.

[0019] In some embodiments, systems and methods are provided for detecting mura and performing demura for near-eye displays.

[0020] Since mura refers to the non-uniformity of brightness and / or chromaticity (perceived as visual artifacts by the human eye), it is necessary to extract mura features for evaluation and further for suppressing visual artifacts. Mura features can be extracted by analyzing profiles such as brightness scale, gradient boundary / frequency domain, or grayscale histogram. Mura features can be identified as three types: corner mura, cloud-like mura, and global mura. For evaluation, mura can be classified into different levels according to human sensitivity to determine the main mura type. For example, the human eye is more sensitive to corner effects than cloud-like effects, and is more sensitive to cloud-like effects than global non-uniformity. Therefore, corner mura can be determined as the main mura for evaluation. In some embodiments, mura can be classified according to the mura degree of each mura type. For example, if the mura degree of cloud-like mura is greater than the mura degree of corner mura, cloud-like mura is determined as the main mura. In some embodiments, the classification is defined by the user. For example, for some images, the user can set global non-uniformity mura as the main mura. In some embodiments, one or more mura types may be determined as primary mura for evaluation. For example, both corner mura and cloud mura may be determined as primary mura, such that both corner mura and cloud mura are evaluated to obtain a final mura degree.

[0021] Based on the extracted and identified Mura features, compensation can be performed to achieve a relatively uniform distribution of the displayed image by adjusting the matrix grayscale values ​​of the image generator (e.g., the image generator of the near-eye display). The compensation factor can be calculated as the inverse of the brightness and / or chromaticity change according to the baseline threshold. The baseline threshold can be determined by a matrix grayscale value histogram. In the histogram, the distribution of the amount ratio in all grayscale values ​​(e.g., 0 to 255) is calculated. The peak value with the maximum amount of grayscale value is extracted. For example, the maximum amount ratio is 0.2, and the corresponding grayscale value is 87. This means that the majority of the grayscale values ​​of the image are 87. Then, the majority of grayscale values ​​are determined as the baseline threshold for compensation. This is a histogram method in determining the compensation baseline. Note that not all Mura features are considered for calculating the compensation factor. For example, the identified Mura features are evaluated as candidates for the calculation of the compensation factor. The calculation of the compensation factor can be based on the Mura features selected from the candidates, that is, the identified Mura features. The display driving capability with sufficient range (e.g., additional 100% grayscale adjustment space) can also be considered within the compensation tolerance used to calculate the compensation factor. The compensation tolerance here means the limitation of compensation during operation. For example, normal operation is 8 bits, which ranges from 0 to 255 grayscale values. With an extra bit, the system can operate in the range of 0 to 511 grayscale values ​​(9 bits), with one-fold (100%) compensation capability. However, if the compensation factor is two times or more (e.g., 200-300%), it exceeds the operating range (e.g., 9 bits), so that the compensation factor is always truncated at the maximum compensation tolerance (100%) and the grayscale cannot be further adjusted to the desired value. That is, the calculation of the compensation coefficient should take into account the display driving capability. After compensation, Mura (i.e., non-uniformity) can be checked / re-evaluated to determine whether the compensated image meets human sensitivity. This compensation process is referred to as the Demura process in this article. It should be noted that in the present disclosure, the compensation term is equivalent to the correction term, and the compensation factor means the correction coefficient. Mura in this article also refers to local non-uniformity and global non-uniformity. Demura can be interpreted as non-uniformity correction or compensation.

[0022] Figure 1 is a schematic diagram of an exemplary Demura system 100 according to some embodiments of the present disclosure. Figure 1As shown in , a system 100 is provided to detect mura / non-uniformity of a virtual image presented in a near-eye display. The system 100 includes a near-eye display (NED) 110 for displaying an image in front of a person's eyes, an imager provided as an imaging module 120, a positioner provided as a positioning device 130, and a processor provided as a processing module 140. Additionally, ambient light may be provided by an ambient light module 150. The near-eye display 110 may be provided as an AR display, a VR display, a head-up display / head-mounted display, or other displays. The positioning device 130 is provided for setting an appropriate spatial relationship between the near-eye display (NED) 110 and the imaging module 120. For example, the positioning device 130 is configured to set the distance between the near-eye display 110 and the imaging module 120 within a range of 10 mm to 25 mm. The positioning device 130 may further adjust the relative position (e.g., distance and spatial position) of the near-eye display 110 and the imaging module 120. The imaging module 120 is configured to simulate the human eye to measure display optical properties and to observe display performance. In some embodiments, the imaging module 120 may include an array light measurement device (LMD) 122 and a near eye display (NED) lens 121. For example, the LMD 122 may be a colorimeter or an imaging camera, such as a CCD (charge coupled device) or a CMOS (complementary metal oxide semiconductor). The near eye display (NED) lens 121 of the imaging module 120 is provided with a front aperture having a small diameter of 1 mm to 6 mm. Therefore, the near eye display (NED) lens 121 can provide a wide field of view (e.g., 60 degrees to 180 degrees) in front, and the near eye display lens 121 is configured to simulate the human eye to observe the near eye display 110. The optical properties of the virtual image are measured by the imaging module 120 based on the positioning device 130.

[0023] In some embodiments, the near-eye display 110 may include an image generator 111 (also referred to herein as an image source) and an optical combiner (also referred to herein as an image optics) ( Figure 1 1 ). The image generator 111 may be a micro display such as a micro LED, micro OLED, LCOS or DLP display, and may be configured to form a light engine with an additional projector lens. The projected image from the light engine is transmitted to the human eye through an optical combiner through designed optics. The optics of the optical combiner may be reflective and / or diffractive optics such as a free-form mirror / prism, Birdbath, or cascaded mirrors, grating couplers (waveguides), etc.

[0024] The processing module 140 is configured to analyze Mura, extract Mura features, and calculate compensation factors, etc. In some embodiments, the processing module 140 may be included in a computer or a server. In some embodiments, the processing module 140 may be deployed in the cloud, which is not limited herein.

[0025] In some embodiments, a driver ( Figure 1 140) may be further provided for compensating the image generator 111 to remove Mura in the virtual image for display. The compensation factor is calculated in the processing module 140 and then transmitted to the driving module. Therefore, using the system 100, the Demura method can be performed. The driving system can be coupled to communicate with the near-eye display 110, specifically with the image generator 111 of the near-eye display 110. For example, the driving module can be configured to adjust the grayscale value of the image generator 111. When the driving system including the display drive and compensation functions (grayscale value adjustment in image processing) is integrated in the near-eye display, the data of the compensation factor from the processing module 140 can be transmitted to the near-eye display system 110.

[0026] In some embodiments, such as for AR applications, ambient light is provided from ambient light module 150. Ambient light module 150 is configured to generate a uniform light source with a corresponding color (such as D65) that can support measurements in ambient light backgrounds and simulations of various scenes such as daylight, outdoors, or indoors.

[0027] Figure 2 A flow chart illustrating an exemplary Demura method 200 is shown, according to some embodiments of the present disclosure. Figure 3 An exemplary Demura process 300 according to some embodiments of the present disclosure is shown. The method 200 may be performed by the Demura system 100. Figure 2 and Figure 3 , the Demura method 200 includes steps 202 to 212 .

[0028] At step 202, an original virtual image is presented in a near-eye display. Presentation refers to the process of generating a two-dimensional or three-dimensional image from a model with the aid of an application, and is not limited here. An original virtual image 310 is presented from a module (e.g., an AR module) in a near-eye display (e.g., near-eye display 110). The original virtual image 310 is then generated by an image generator of the near-eye display and projected through a combiner optical device facing the human eye. The original virtual image 310 can be captured by an imaging module (e.g., image module 120) for analysis. In practice, multiple virtual images can be presented in a display for Mura / non-uniformity analysis under multiple test patterns with various grayscale values. In some embodiments, the image is presented by an ambient light module (e.g., Figure 1 The ambient light module 150 shown in FIG. 1 presents an original virtual image under ambient light conditions such as a D65 daylight scene.

[0029] At step 204, mura features 320 are extracted from the original virtual image 310. The mura features 320 can be extracted by analyzing the profile of the original virtual image 310. The profile shown can include brightness scale, gradient boundary / frequency domain, or grayscale histogram. Mura features can be further identified as corner mura 321, cloud-like mura 322, or global mura (ie, global non-uniformity).

[0030] At step 206, a normal baseline is determined by histogram analysis. The normal baseline (which may also be referred to as a baseline threshold) may be determined by a matrix histogram.

[0031] At step 208, a compensation factor is calculated based on the mura feature. The compensation factor can be further calculated based on a common baseline and a custom (e.g., a mean in a local area or a global area). The custom compensation target / basis can take into account the mean in a local area or a global area. Further considering the compensation tolerance (e.g., one-time compensation capability), the compensation factor can be calculated by comparing the mura feature with the common baseline. In some embodiments, not all mura features are used to calculate the compensation factor. For example, only the mura features identified at step 204 are evaluated as candidates for the calculation of the compensation factor.

[0032] At step 210, the compensation factor 330 is applied to the image generator to adjust the grayscale value. In some embodiments, the compensation factor 330 is applied in the pixel pipeline of the image generator. For example, the compensation factor can be applied in the pixel pipeline of the micro LED display. Different pixels can correspond to different compensation factors. Therefore, the image displayed by the micro LED display can be improved pixel by pixel.

[0033] At step 212, the degree of mura (non-uniformity) of the virtual image improved by the compensation presented in the near-eye display is re-evaluated. The compensated virtual image is presented to obtain an improved virtual image 340. The degree of mura of the improved virtual image 310 is reduced compared to the original virtual image 340 and can be re-evaluated. When evaluating, as described above, mura can be classified into different levels according to human sensitivity. For example, the human eye is more sensitive to corner effects than to cloud effects, and is more sensitive to cloud effects than to global non-uniformity. Therefore, corner mura can be determined as the main type of mura to be evaluated.

[0034] Figure 4 is a schematic block diagram of an exemplary Demura system 400 according to some embodiments of the present disclosure. The Demura system 400 may be configured to perform Figure 2 The Demura method 200 shown in FIG. Figure 4 As shown in the figure, the Demura system 400 includes a near-eye display 410 for displaying an image for a human eye, an imager provided as an imaging module 420, a Mura feature extractor provided as a Mura feature extraction module 430, a compensation calculator provided as a compensation calculation module 440, a driver provided as a display driving module 450, and an evaluator provided as an evaluation module 460.

[0035] The near-eye display 410 may include an image generator (or image source) 411 and an optical combiner (or image optics). The image generator 411 may be a micro display such as a micro LED (µLED) display, a micro OLED, an LCOS, or a DLP display, and may be configured to form a light engine with an additional projector lens. The projected image from the light engine is transmitted to the human eye through the optical combiner through the designed optics. The optics may be reflective and / or diffractive optics such as a free-form mirror / prism, a Birdbath, or a cascaded mirror, a grating coupler (waveguide), etc.

[0036] The imaging module 420 may include an array light measurement device (LMD) and a near-eye display lens. For example, the LMD may be a colorimeter or an imaging camera CCD / CMOS. The near-eye display lens is provided with a front aperture having a small diameter of 1 mm to 6 mm. The near-eye display lens provides a wide field of view (e.g., 60 to 180 degrees) in front and is configured to simulate a human eye to observe the near-eye display 410. Imaging data of a virtual image may be acquired from the virtual image by the imaging module 420. The imaging data may include brightness, chromaticity, grayscale values, etc. It should be noted that ordinary lenses together with cameras may also be used to obtain relative values ​​(e.g., uniformity) in measurements.

[0037] The mura feature extraction module 430 is configured to analyze the rendered virtual image based on the acquired imaging data, the brightness and / or chromaticity or the XYZ intensity of the multi-color virtual image. The mura feature extraction module 430 is coupled to communicate with the imaging module 420 to extract mura features from the virtual image captured by the imaging module 420.

[0038] The compensation calculation module 440 is coupled to communicate with the Mura feature extraction module 430 and is configured to calculate a compensation factor. The compensation calculation module 440 may be configured to calculate a compensation factor based on the Mura features extracted by the Mura feature extraction module 430. The compensation factor may be further calculated based on a common baseline and a custom one (e.g., a mean in a local area or a global area). Further considering the compensation tolerance (e.g., one-time compensation capability), the compensation factor may be calculated by comparing the Mura features with the common baseline. In some embodiments, not all Mura features are used to calculate the compensation factor. For example, only the identified Mura features are evaluated as candidates for the calculation of the compensation factor.

[0039] The display driver module 450 is coupled to communicate with the compensation calculation module 440, and is further coupled to communicate with the near-eye display 410. The display driver module 450 is configured to adjust the grayscale value of the image source (e.g., the image generator 411). In some embodiments, the display driver module 450 is configured to apply the compensation factor calculated by the compensation calculation module 440 in the pixel pipeline of the image generator 411 included in the near-eye display 410. For example, the compensation factor can be applied in the pixel pipeline of the micro-LED display. Different pixels may correspond to different compensation factors.

[0040] The mura evaluation module 460 is coupled to communicate with the imaging module 420 and is configured to evaluate the mura degree (e.g., the degree of non-uniformity) after compensation. After compensation, the virtual image is presented again in the near-eye display 410 to obtain an improved virtual image that can be captured by the imaging module 420. Then, the mura evaluation module 460 can evaluate the mura degree of the improved virtual image captured by the imaging module 420. In some embodiments, the mura evaluation module 460 can be further configured to classify mura. Mura can be classified into different levels according to human sensitivity to determine the main mura type. For example, as described above, the human eye is more sensitive to corner effects than to cloud effects, and is more sensitive to cloud effects than to global non-uniformity. Therefore, corner mura can be determined as the main mura for evaluation. In some embodiments, mura can be classified according to the mura degree of each mura type. For example, if the mura degree of cloud mura is greater than the mura degree of corner mura, the cloud mura is determined as the main mura. In some embodiments, the classification is defined by the user. For example, for certain images, a user may set global mura as primary mura. In some embodiments, one or more mura types may be determined as primary mura for evaluation. For example, both corner mura and cloud mura may be determined as primary mura, and then both corner mura and cloud mura may be evaluated to obtain a final mura degree.

[0041] Therefore, through evaluation, it can be determined that the mura of the improved virtual image 340 is reduced compared to the mura of the original virtual image 310 .

[0042] In some embodiments, the Demura system 400 further includes a preprocessing module (e.g., a preprocessor) coupled to the imaging module 420 and the Mura feature extraction module 430. The preprocessing module is configured to preprocess the original virtual image before the Mura feature extraction. In some embodiments, in the preprocessing, negative effects that deteriorate image quality, such as noise or distortion in the original virtual image, are eliminated. In some embodiments, in the preprocessing, a mapping / registration from a pixel matrix of the virtual image to a source pixel matrix (image generator) is applied. For example, a 10,000 × 10,000 virtual image pixel matrix is ​​converted to a 640 × 480 source pixel matrix of the image generator. Optionally, the mapping / registration can be performed by the compensation factor calculation module 440 to further adjust the grayscale value during the compensation process.

[0043] It is understandable that the connection between the above modules can be wired communication or wireless communication (for example, via the Internet, Bluetooth, etc.), which is not limited here. In some embodiments, the Mura feature extraction module 430, the compensation calculation module 440, the display driver module 450, the Mura evaluation module 460, and the pre-processing module can be integrated in a computer system or server or deployed in the cloud, which is not limited here.

[0044] In some embodiments, a virtual image presented by the near-eye display 410 is obtained from the imaging module 420, in which some noticeable artifacts may be directly shown. FIG. 5A to FIG. 5C Different types of Mura features according to some embodiments of the present disclosure are respectively shown. Figure 5A Noticeable dark areas are shown at each of the three corners of the image, which are referred to herein as corner features. Figure 5B The original image converted into a pseudo-color image showing absolute / relative brightness distribution is shown. As shown, on the local area indicated by the dot-dash line, a cloud region that obviously floats on the global plane is shown, and the cloud region constitutes a cloud-like feature. Figure 5C An original image is shown that is mapped as a 3D surface to obtain a 3D image. As shown, next to the corner artifacts and cloud-like artifact areas, the 3D surface gradually decays in multiple directions, which is referred to as global mura. Therefore, non-uniformities on the global surface are observable. The gradual surface decay of the 3D image can be extracted as a mura feature. In the example of the waveguide, light can decay from one corner to other corners. Since the human eye is sensitive to mura artifacts, these are also non-uniformities that may appear in the virtual image.

[0045] In order to extract the Mura features in the presented virtual image, a method for extracting the Mura features is provided. Figure 6 A flow chart illustrating an exemplary mura extraction method 600 according to some embodiments of the present disclosure is shown. Figure 6 , the mura extraction method 600 includes steps 610 to 640 .

[0046] At step 610, an original virtual image in a near-eye display is presented. The original virtual image with mura is presented from a model (e.g., an AR model). The original virtual image is then displayed by an image generator of the near-eye display and projected in front of a person's eyes through a combiner optical device. The original virtual image can be captured by an imaging module (e.g., imaging module 120 or 420) for analysis. In some embodiments, the original virtual image is presented under ambient light conditions such as a D65 daylight scene.

[0047] At step 620, the raw virtual image is pre-processed to eliminate negative effects, such as noise or distortion. The raw virtual image can be captured by an imaging module (e.g., an array light measurement device (LMD) and a near-eye display lens) under an all-white and / or all-gray test pattern. Then, pre-processing is performed on the captured raw virtual image. In some embodiments, in the pre-processing, negative effects that deteriorate image quality, such as noise or distortion in the raw virtual image, are eliminated. Distortions including camera lenses or NED optical modules are taken into account in the process.

[0048] At step 630, mura features are extracted based on the mura type. Mura types include corner mura, cloud mura, and global mura (or global non-uniformity). For different mura types, different profiles can be used to extract mura features. It is worth mentioning that the appearance of cloud mura refers to the non-uniformity area. The non-uniformity area can also be spots or other descriptions. Here, cloud mura is used to describe the general non-uniformity area, and cloud mura also refers to similar area forms, such as spots.

[0049] More specifically, step 630 may further include steps 631 to 633 .

[0050] At step 631, for corner mura, a mura feature is extracted using a brightness threshold profile. For example, the brightness profile of the image is compared with the brightness threshold profile. The threshold may be an upper threshold and / or a lower threshold. If the brightness in the corner exceeds (or is lower than) the brightness threshold corresponding to the corner, for example, the brightness in the corner is greater than (or less than) the brightness threshold corresponding to the corner, then the corner mura feature is extracted.

[0051] At step 632, for cloud-like mura, the spatial gradient profile or frequency domain is used to extract mura features. For example, the spatial gradient profile can be applied to extract mura features. In some embodiments, the zone change scale can be obtained according to the spatial gradient profile. By comparing the zone change scale with a preset threshold, the mura features can be extracted. In some embodiments, the captured image can be transformed into the frequency domain for further filtering in human contrast sensitivity perception and extracting visual non-uniformity information.

[0052] At step 633, for global mura, a global profile (e.g., histogram) (e.g., grayscale histogram) is used to extract mura features. The histogram can be applied as a profile for analyzing non-uniformity as a baseline. For example, through the image histogram, the maximum amount ratio of grayscale values ​​(i.e., peak value) is extracted as the majority representation of the image and as a baseline analysis of image uniformity. It is also noted that the gradient (including scale and direction) in the whole field can also be considered in the global non-uniformity analysis.

[0053] At step 640, the degree of mura is evaluated considering the type of mura, the characteristic amount of mura, and human perception. The type of mura may refer to corner mura, cloud-like / spotted mura, or global mura. For mura, the evaluation is performed not only in terms of type but also in terms of quantity. The quantity means the scale of non-uniformity, for example, the percentage of the difference scale. For example, an 8% difference scale is more serious than 5%. The human eye has a relative sensitivity to non-uniformity. For example, when the difference / non-uniformity is less than 1%, the difference / non-uniformity is not obvious / invisible to the human eye. After obtaining the mura characteristics, an evaluation can be performed to determine the degree of mura in the near-eye display. Mura can be classified into different levels according to human sensitivity to determine the main type of mura. For example, as described above, the human eye is more sensitive to corner effects than to cloud-like effects, and is more sensitive to cloud-like effects than to global non-uniformity. Therefore, corner mura can be determined as the main mura for evaluation. In some embodiments, mura can be classified according to the degree of mura of each mura type. For example, if the degree of mura of cloud-like mura is greater than the degree of mura of corner mura, the cloud-like mura is determined to be the main mura. In some embodiments, the classification is defined by the user. For example, for certain images, the user can set the global mura as the main mura. In some embodiments, one or more types of mura may be determined as the main mura for evaluation. For example, if both corner mura and cloud-like mura are determined to be the main mura, both corner mura and cloud-like mura are evaluated to obtain the final degree of mura. The brightness scale or area size (e.g., 30% area of ​​the image) can be set as a threshold for evaluating the degree of mura. For example, for corner mura, the brightness scale is set to a threshold. For cloud-like mura, the area size is set to a threshold. In some embodiments, quantitative evaluation can be performed depending on human perception. For example, when the brightness difference is less than 1%, the brightness difference is not obvious / invisible to the human eye.

[0054] Mura of multi-color virtual images presented in near-eye displays is also considered. For each individual primary color channel, Mura appears on the image surface after the light propagates in the path. The image light is generated in the microdisplay, transmitted through the lens of the light engine / projector, and further transmitted through an optical combiner (such as a waveguide) until it is finally received by the human eye, which refers to the propagation of light in the path. The Mura characteristics are the same as those described above and are not repeated here. In addition, for three primary color channels (e.g., RGB), each color channel has its own Mura type and ratio. For a channel, the characteristics may include one or more Mura types. The ratio of each channel may vary. For example, for a global attenuation from one corner of the virtual image to the other three corners, the variation may be from 30% to 80%. Therefore, in addition to brightness non-uniformity, serious issues such as color shift and white balance are also raised.

[0055] To solve these problems, a demura method is provided for color correction that depends on the doping ratio on the three primary color channels. The doping is performed in driving the display.

[0056] In some embodiments, a method for Demura is provided. Figure 7 A flow chart illustrating an exemplary Demura method 700 is shown, according to some embodiments of the present disclosure. Figure 8 An exemplary Demura process 800 is shown according to some embodiments of the present disclosure. Figure 7 and Figure 8 , the Demura method 700 includes steps 702 to 712 .

[0057] At step 702, an original virtual image 810 in a near-eye display is presented and acquired under a test pattern. The test pattern may be a full test pattern with various grayscale values ​​(e.g., 63, 127, 255, etc.). In measurements with various grayscale values ​​and colors, multiple test patterns may be applied. Each test pattern may be one or more colors, such as an R / G / B pattern and / or a white image. It is worth mentioning that a partial on / off pixel test pattern may be used in the measurement instead of directly using a full pattern. This means that multiple partial on / off patterns may be used in the measurement and ultimately integrated into a full image. The original virtual image is displayed by an image generator of the near-eye display and projected in front of the human eye through a combiner optical device. The original virtual image may be captured by an imaging module for analysis. In some embodiments, the original virtual image is presented under ambient light conditions such as a D65 daylight scene.

[0058] In some embodiments, a multi-color virtual image is acquired under a white test pattern.

[0059] At step 704, three single-primary color (R, G, B) and multi-color virtual images 820 are obtained. The image may include RGB three-primary color images, which are red, green and blue images respectively. Each single-primary color virtual image corresponds to one channel. Therefore, three monochrome virtual images are captured under the three-color test pattern, and imaging data of the monochrome virtual images can be obtained. A multi-color (white) virtual image is also obtained in the process.

[0060] At step 706, mura features 830 are extracted for each color channel. For each monochrome virtual image, mura features are extracted. Mura features can be extracted based on mura types such as corner mura, cloud-like mura, and global mura. More specifically, for corner mura, mura features are extracted based on a brightness threshold profile. For cloud-like mura, mura features are extracted based on a spatial gradient profile or a frequency domain. For global mura, mura features are extracted based on a global profile (e.g., a histogram) (e.g., a grayscale histogram). More details about mura feature extraction are described above with reference to the above-mentioned method 600 and will not be repeated here.

[0061] At step 708, the compensation factor of each color channel is calculated by considering the Mura characteristics of both brightness and chromaticity non-uniformity. The compensation factor of each channel can be calculated according to each virtual image presented. In some embodiments, the compensation factor is calculated by considering the individual differences between the three channels. For example, a color shift based on the difference between a monochrome virtual image to a full white virtual image in the whole field is obtained. The entire white virtual image is formed by superimposing all monochrome virtual images together. In some embodiments, the compensation factor is calculated according to the doping ratio between the three channels. The doping ratio refers to the ratio between the three primary color channels of red, green and blue. Therefore, the compensation factor can be further used to correct the color and / or white balance by adjusting the doping ratio and global hue of each pixel in the matrix.

[0062] At step 710, the compensation factors of the image generator are applied to the three primary color channels 840. For example, the grayscale values ​​of the virtual image in the image generator (eg, micro-LED display, micro-display) are adjusted according to the compensation factors of each channel.

[0063] At step 712, after compensation, the mura of the multi-color virtual image is re-evaluated 850. After applying compensation to the image data of the original virtual image, the rendered virtual image is re-evaluated to determine whether the non-uniformity / mura has been successfully eliminated. In some embodiments, mura evaluation can be performed in both brightness and color. Since a large amount of mura has been suppressed, the human eye should not be sensitive to the remaining mura.

[0064] In some embodiments, a non-transient computer-readable storage medium including instructions is also provided, and the instructions can be executed by the device to perform the above method. Common forms of non-transient media include: for example, a floppy disk, a flexible disk, a hard disk, a solid-state drive, a tape or any other magnetic data storage medium, a CD-ROM, any other optical data storage medium, any physical medium with a hole pattern, a RAM, a PROM, and an EPROM, a FLASH-EPROM or any other flash memory, an NVRAM, a cache, a register, any other storage chip or storage box, and their networked versions. The device may include one or more processors (CPU), an input / output interface, a network interface, and / or a memory.

[0065] It should be noted that relational terms, such as "first" and "second", herein are used only to distinguish an entity or operation from another entity or operation, without requiring or implying any actual relationship or order between these entities or operations. In addition, the words "comprising", "having", "containing", and "including" and other similar forms are intended to be equivalent in meaning and are open-ended, and one or more items following any of these words are not meant to be an exhaustive list of such one or more items, or to be limited to the listed one or more items.

[0066] As used herein, unless expressly stated otherwise, the term "or" encompasses all possible combinations unless not feasible. For example, if it is stated that a database may include A or B, then unless expressly stated otherwise or not feasible, the database may include A, or B, or A and B. As a second example, if it is stated that a database may include A, B, or C, then unless expressly stated otherwise or not feasible, the database may include A, or B, or C, or A and B, or A and C, or B and C, or A, B, and C.

[0067] It is understood that the above embodiments can be implemented by hardware, or software (program code), or a combination of hardware and software. If implemented by software, it can be stored in the above computer-readable medium. The software can perform the disclosed method when executed by a processor. The computing unit and other functional units described in this disclosure can be implemented by hardware, or software, or a combination of hardware and software. It will also be understood by those of ordinary skill in the art that a plurality of the above modules / units can be combined into one module / unit, and each of the above modules / units can be further divided into a plurality of submodules / subunits.

[0068] In the foregoing description, embodiments have been described with reference to many specific details, which may vary depending on the implementation. Certain changes and modifications may be made to the described embodiments. In view of the description and practice of the invention disclosed herein, other embodiments will be clear to those skilled in the art. The description and examples are intended to be considered as merely exemplary, and the true scope and spirit of the invention are indicated by the following claims. The order of steps shown in the drawings is also intended to be used for illustrative purposes only and is not intended to be limited to any particular order of steps. Therefore, it will be appreciated by those skilled in the art that these steps may be performed in different orders while implementing the same method.

[0069] In the drawings and the specification, exemplary embodiments have been disclosed. However, many variations and modifications may be made to these embodiments. Therefore, although specific terms are employed, they are used only in a general and descriptive sense and not for the purpose of limitation.

Claims

1. A method for detecting mura of a virtual image in a near-eye display, the method comprising: Acquiring the virtual image presented in the near-eye display; Extracting Mura features of the virtual image according to the Mura type; as well as A mura degree of the virtual image is evaluated based on the mura type.

2. The method according to claim 1, wherein: The mura types include corner mura, cloud-like mura, or global mura.

3. The method according to claim 2, wherein: Extracting a Mura feature of the virtual image according to the Mura type includes: When the mura type is the corner mura, the mura feature is extracted based on a brightness threshold profile.

4. The method according to claim 2, wherein: Extracting a Mura feature of the virtual image according to the Mura type includes: When the mura type is the cloud-like mura, the mura features are extracted based on a spatial gradient profile or a frequency domain.

5. The method according to claim 2, wherein: Extracting a Mura feature of the virtual image according to the Mura type includes: When the mura type is the global mura, the mura features are extracted based on a global overview.

6. The method according to claim 2, wherein: Before extracting the mura features of the virtual image according to the mura type, the method further includes: The virtual image is converted into a pseudo-color image, wherein the pseudo-color image presents an absolute brightness distribution or a relative brightness distribution.

7. The method according to claim 2, wherein: Before extracting the mura features of the virtual image based on the mura type, the method further includes: The virtual image is rendered as a 3D surface to obtain a 3D image.

8. The method according to any one of claims 2 to 7, wherein: Evaluating the mura degree of the virtual image based on the mura type further includes: determining one or more dominant mura types of the virtual image; and The mura degree of the virtual image is evaluated based on one or more preset thresholds corresponding to the one or more dominant mura types.

9. The method according to claim 8, wherein: The corner mura is determined to be the dominant mura type.

10. The method according to claim 9, wherein: The one or more preset thresholds include: The brightness scale threshold corresponding to the angular mura, or The area size threshold corresponding to the cloud-like Mura.

11. The method according to claim 10, wherein: The preset threshold value of the brightness scale threshold value is 30%, and the preset threshold value of the area size threshold value is 30%.

12. The method according to any one of claims 1 to 11, wherein: Acquiring the virtual image presented in the near-eye display further comprises: The virtual image is acquired under a complete test pattern or a plurality of partial test patterns, wherein the partial test patterns have various gray values ​​and colors.

13. The method according to claim 12, wherein: The test pattern is a full white test pattern.

14. The method according to claim 12, wherein: The test pattern is a full gray test pattern.

15. The method according to claim 12, wherein: The virtual image is presented under ambient light conditions.

16. The method according to any one of claims 1 to 15, wherein: Before extracting the Mura feature of the virtual image according to the Mura type, the method further includes: The virtual image is preprocessed by removing one or more of noise or distortion.

17. A method for demodulating a virtual image presented by a near-eye display, the method comprising: Acquire a mura characteristic of a first virtual image presented in the near-eye display; Calculating a compensation factor based on the Mura characteristics; as well as The grayscale value of the near-eye display is adjusted based on the compensation factor to obtain a second virtual image.

18. The method according to claim 17, further comprising: The mura degree of the second virtual image is evaluated.

19. The method according to claim 17 or 18, wherein: Acquiring the Mura feature of the first virtual image further includes: Acquiring monochrome virtual images of different colors; and The Mura feature of each of the monochrome virtual images is obtained.

20. The method according to claim 19, wherein: Calculating the compensation factor based on the Mura feature further includes: The compensation factor is calculated based on the Mura characteristics of each monochrome virtual image.

21. The method according to claim 20, wherein: Calculating the compensation factor based on the Mura feature further includes: A mapping from the pixel matrix of the first virtual image to the source pixel matrix is ​​applied.

22. The method according to claim 19, wherein: Calculating the compensation factor based on the Mura feature further includes: obtaining a color shift based on a difference between the monochrome virtual images over the entire virtual image; and The color shift is determined as the Mura feature.

23. The method according to claim 22, wherein: The difference is the doping ratio between the monochrome virtual images.

24. The method according to any one of claims 18 to 23, wherein: Evaluating the Mura degree of the second virtual image further includes: The mura degree of the second virtual image in terms of brightness and color is evaluated.

25. The method according to any one of claims 17 to 24, wherein: Acquiring the Mura feature of the first virtual image further includes: acquiring a first multi-color virtual image presented in the near-eye display; and The Mura features of the multi-color virtual image are obtained.

26. The method according to claim 25, wherein: The first multi-color virtual image is acquired under a white test pattern.

27. The method according to any one of claims 17 to 24, wherein: Acquiring the Mura feature of the first virtual image further includes: Acquire three single primary color (red, green and blue) virtual images presented in the near-eye display; and Get the mura characteristics of each color monochrome image.

28. The method according to claim 27, wherein: The three single-primary-color (red, green and blue) virtual images are acquired under red, green and blue test patterns, respectively.

29. A system for detecting mura in a virtual image presented in a near-eye display, the system comprising: an image generator configured to render a virtual image; an imager configured to acquire the virtual image; a positioner coupled to the image generator and the imager and configured to control a relative position of the near-eye display and the imager; as well as A processor is coupled to the imager and configured to evaluate a mura level of the virtual image.

30. The system of claim 29, wherein: The processor is further configured to: extracting Mura features of the virtual image according to the Mura type; and A mura degree of the virtual image is evaluated based on the mura type.

31. The system of claim 29, wherein: The mura types include corner mura, cloud-like mura, or global mura.

32. The system of claim 31, wherein: The processor is further configured to extract the mura feature based on a brightness threshold profile in response to the mura feature being the corner mura.

33. The system of claim 31, wherein: The processor is further configured to extract the mura feature based on a spatial gradient profile or a frequency domain in response to the mura feature being the cloud-like mura.

34. The system of claim 31, wherein: The processor is further configured to extract the mura signature based on a global profile in response to the mura signature being the global mura.

35. The system of claim 31, wherein: The processor is further configured to: The virtual image is converted into a pseudo-color image, wherein the pseudo-color image presents an absolute brightness distribution or a relative brightness distribution.

36. The system of claim 31, wherein: The processor is further configured to: The virtual image is rendered as a 3D surface to obtain a 3D image.

37. A system according to any one of claims 31 to 36, wherein: The processor is further configured to: determining one or more dominant mura types of the virtual image; and The mura degree of the virtual image is evaluated based on one or more preset thresholds corresponding to the one or more dominant mura types.

38. The system of claim 37, wherein: The corner mura is determined to be the dominant mura type.

39. The system of claim 38, wherein: The one or more preset thresholds include: The brightness scale threshold corresponding to the angular mura, or The area size threshold corresponding to the cloud-like Mura.

40. The system of claim 39, wherein: The preset threshold value of the brightness scale threshold value is 30%, and the preset threshold value of the area size threshold value is 30%.

41. A system according to any one of claims 29 to 40, wherein: The processor is further configured to perform pre-processing on the virtual image.

42. The system of claim 41, wherein: The pre-processing is to eliminate one or more of noise and distortion.

43. A system according to any one of claims 29 to 42, wherein: The imager is further configured to acquire the virtual image under an all-white test pattern.

44. A system according to any one of claims 29 to 42, wherein: The imager is further configured to acquire the virtual image under an all-gray test pattern.

45. A system according to any one of claims 29 to 44, wherein: The imager comprises: a near-eye display lens configured to simulate a human eye to acquire the virtual image; and A light measuring device is configured to measure the virtual image.

46. ​​The system of claim 45, wherein: The light measuring device includes a colorimeter or an imaging camera.

47. The system of claim 45, wherein: The positioner is further configured to determine a spatial relationship between the image generator and the light measuring device.

48. A system according to any one of claims 29 to 47, wherein: The image generator is further configured to present the virtual image under ambient light conditions.

49. A system according to any one of claims 29 to 48, wherein: The image generator is one of a micro-LED based display, an LCOS (Liquid Crystal on Silicon) display, or a DLP (Digital Light Processing) display.

50. A system according to any one of claims 29 to 49, wherein: The near-eye display is one of an augmented reality display, a virtual reality display, a head-up display or a head-mounted display.

51. The system of claim 50, wherein: The near-eye display includes the image generator and an optical combiner configured to project the virtual image from the image generator to a human eye.

52. A system for demodulating a virtual image presented in a near-eye display, the system comprising: an image generator configured to render a first virtual image; an imager configured to acquire the first virtual image; a positioner coupled to the image generator and the imager and configured to control the relative position of the image generator and the imager; a mura feature extractor coupled to the imager and configured to extract mura features of the first virtual image; a compensation calculator coupled to the mura feature extractor and configured to calculate a compensation factor; as well as A driver is coupled to the compensation calculator and the image generator and is configured to adjust a grayscale value of the image generator based on the compensation factor to obtain a second virtual image.

53. The system of claim 52, further comprising a pre-processor configured to perform pre-processing on the first virtual image.

54. The system of claim 53, wherein: The pre-processing is to eliminate one or more of noise and distortion.

55. The system of claim 52, wherein: The pre-processor is further configured to map the pixel matrix of the first virtual image to a source pixel matrix of the image generator.

56. A system according to any one of claims 52 to 55, wherein: The imager is configured to acquire a monochrome virtual image of a different color for each individual channel of the first virtual image; and The Mura feature extractor is further configured to obtain a Mura feature of each monochrome virtual image.

57. The system of claim 56, wherein: The compensation calculator is configured to calculate the compensation factor based on a mura feature of each monochrome virtual image.

58. The system of claim 57, wherein: The compensation calculator is further configured to: obtaining a color shift based on a difference between the monochrome virtual images over the entire virtual image; and The color shift is determined to be the Mura feature.

59. The system of claim 58, wherein: The difference is the doping ratio between the monochrome virtual images.

60. The system of claim 52, wherein: The compensation calculator is further configured to map the pixel matrix of the first virtual image to a source pixel matrix of the image generator.

61. A system according to any one of claims 47 to 60, wherein: The imager is further configured to acquire a first multi-color virtual image presented by the image generator; and The Mura feature extractor is further configured to obtain a Mura feature of the virtual image.

62. The system of claim 61, wherein: The first multi-color virtual image is acquired under a white test pattern.

63. The system of claim 61, wherein: The imager is further configured to acquire three single primary color (red, green and blue) virtual images presented by the image generator.

64. The system of claim 63, wherein: The three single-primary-color (red, green and blue) virtual images are acquired under red, green and blue test patterns, respectively.

65. A system according to any one of claims 52 to 64, wherein: The image generator is further configured to present the first virtual image under ambient light conditions.

66. A system according to any one of claims 52 to 65, wherein: The mura feature extractor is further configured to extract mura features of the first virtual image according to mura types, and the mura types include corner mura, cloud-like mura, or global mura.

67. The system of claim 66, wherein: The mura feature extractor is further configured to: In response to the mura feature being the corner mura, the mura feature is extracted based on a brightness threshold profile.

68. The system of claim 66, wherein: The mura feature extractor is further configured to: In response to the mura feature being the cloud-like mura, the mura feature is extracted based on a spatial gradient profile or a frequency domain.

69. The system of claim 66, wherein: The mura feature extractor is further configured to: In response to the Mura feature being the global Mur, the Mura feature is extracted based on a global profile.

70. The system of claim 66, wherein: The mura feature extractor is further configured to: The virtual image is converted into a pseudo-color image, wherein the pseudo-color image presents an absolute brightness distribution or a relative brightness distribution.

71. The system of claim 66, wherein: The mura feature extractor is further configured to: The virtual image is rendered as a 3D surface to obtain a 3D image.

72. The system of any one of claims 66 to 71, further comprising an evaluator configured to evaluate the first virtual image and the second virtual image based on the mura type.

73. The system of claim 72, wherein: The evaluator is further configured to: determining one or more dominant mura types of the first virtual image; and The mura degree of the second virtual image is evaluated based on one or more preset thresholds corresponding to the one or more dominant mura types.

74. The system of claim 73, wherein: The corner mura is determined to be the dominant mura type.

75. The system of claim 73, wherein: The one or more preset thresholds include: a brightness scale threshold corresponding to the angular mura, and The area size threshold corresponding to the cloud-like Mura.

76. The system of claim 75, wherein: The preset threshold value of the brightness scale threshold value is 30%, and the preset threshold value of the area size threshold value is 30%.

77. A system according to any one of claims 52 to 77, wherein: The imager comprises: a near-eye display lens configured to simulate a human eye to acquire the virtual image; and A light measuring device is configured to measure the virtual image.

78. The system of claim 77, wherein: The light measuring device includes a colorimeter or an imaging camera.

79. The system of claim 78, wherein: The positioner is further configured to determine a spatial relationship between the image generator and the light measuring device.

80. The system of any one of claims 52 to 79, wherein: The image generator is one of a micro-LED based display, an LCOS (Liquid Crystal on Silicon) display, or a DLP (Digital Light Processing) display.

81. A system according to any one of claims 52 to 80, wherein: The near-eye display is one of an augmented reality display, a virtual reality display, a head-up display or a head-mounted display.

82. The system of claim 81, wherein: The near-eye display includes the image generator and an optical combiner configured to project the virtual image from the image generator to a human eye.

Images