An ar glasses and a method and system for displaying a perceived brightness representation thereof

Abstract

The application discloses an AR glasses and a display perceived brightness representation method and system thereof. The application uses mainstream AR glasses in the market as a display device, and proposes a display brightness perceived representation method of AR display under different ambient light based on the visual structure, visual characteristics and perceived process of brightness of human eyes. Objective measurement and evaluation are used to quantify the perceived brightness displayed by the display device, the physical brightness of the display device, the brightness distribution of the ambient light where the display device is located, the pupil diameter of the human eye and the spatial position relationship between the display device and the observer. First, objective parameters in the space are collected through instruments and equipment such as a luminance meter and an illuminance meter; second, the observers observe various test images, and the eye tracker is used to collect the pupil diameter of the observers under different ambient light and the observers are asked to evaluate the perceived brightness level of the display device; the application can be applied to the AR display technology to provide relatively comfortable display brightness for users.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of virtual-real fusion display, and relates to a display brightness measurement and characterization of an AR glasses display device, and also relates to visual perception of a human eye to a display. BACKGROUND

[0002] With the rapid development of information display technology, AR display technology improves our perception and interaction with the environment by seamlessly integrating projected virtual content with real-world scenes, which helps to meta-universe, digital twin and spatial computing. The rapid development of micro-display technology and ultra-compact imaging optics, combined with the progress of high-speed digital processors, has made augmented reality develop from a future concept to a practical and widely used technology. These technologies have been widely used in many fields, including smart education and training, smart medical care, navigation and route finding, game entertainment, intelligent manufacturing and assembly, etc. Since the original concept of AR was proposed in the 1990s, augmented reality has made great progress, especially the emergence and development of AR displays based on holographic optical waveguides, which make wearable systems have the advantages of large field of view, large exit pupil range, lightness and thinness, and similar shape to ordinary glasses.

[0003] AR display mainly superimposes virtual content on the real-world view of the user in real time through wearable devices (similar to glasses or headsets), which are also called optical see-through head-mounted displays. AR display brightness perception is the ability of AR devices to perceive and adapt to ambient light when displaying virtual images. Through automatic brightness adjustment, contrast optimization and brightness uniformity perception, AR devices can present more comfortable and natural virtual images in different lighting conditions. Since the AR display device is usually used in strong light or weak light environments, the display is greatly affected by ambient lighting. The existing AR display device display perception brightness characterization technology is rough and single, and does not fully consider the influence of ambient light on perception brightness, and lacks comprehensive characterization combining psychological factors and other subjective and objective factors. Therefore, it is necessary to establish a perception brightness characterization method for AR display devices. SUMMARY

[0004] Invention purpose: The purpose of the present application is to overcome the shortcomings of the prior art, and a method and system for representing the display brightness perception of AR glasses are proposed. In order to represent the perceived brightness of AR display in different real scenes, the present application carries out environmental light simulation experiments of multiple lighting scenes, and the observer scores the perceived brightness level of the display brightness of AR glasses for different lighting scenes. The main influencing factors of perceived brightness are analyzed through subjective and objective experiments, and are personified into a mathematical model. Further, the perceived brightness is quantified to optimize the display design for human eye perceived brightness. Under the premise of different application scenarios and display content, the visual perception effect of human is improved, which can provide effective help for the development and upgrading of AR display devices. It is verified through experiments that the spatial environmental illuminance, the brightness distribution, the physical brightness of the display and the pupil diameter have a direct significant influence on the perceived brightness of the subjects.

[0005] Technical scheme:

[0006] The present application adopts the following technical scheme to solve the above technical problems:

[0007] A perceived brightness representation method of AR glasses display, comprising the following steps:

[0008] Step 1: Perform environmental light simulation experiments of multiple lighting scenes. The experimental platform includes an observer, AR glasses, an eye tracker and lamps capable of achieving multiple lighting effects. The observer grades the display brightness of AR glasses for different lighting scenes, which is recorded as perceived brightness. The background brightness, AR display brightness and pupil diameter data corresponding to different perceived brightness are recorded.

[0009] Step 2: According to the perceived brightness, AR display brightness and pupil diameter data, a function relationship formula 1 of the three is fitted.

[0010] Step 3: The AR display brightness corresponding to the middle value of the perceived brightness is the best AR display brightness. According to the best AR display brightness and the background brightness data, a function relationship formula 2 of the two is fitted.

[0011] Step 4: In actual use of AR glasses, the measured background brightness data is substituted into the function relationship formula 2 to obtain the best AR display brightness. Then the best AR display brightness and the measured pupil diameter are substituted into the function relationship formula 1 to obtain the perceived brightness. According to the perceived brightness, it is judged whether the best AR display brightness is reasonable. If it is reasonable, the AR glasses adjust the AR display brightness according to the best AR display brightness.

[0012] Preferably, the function relationship formula 1 is:

[0013]

[0014] Wherein, a and b are fitting coefficient values, L rL, D are the perceived brightness, AR display brightness, pupil diameter respectively.

[0015] Preferably, a=7.0756, b=7.4335.

[0016] Preferably, the function relationship 2 is:

[0017]

[0018] Wherein, c and d are fitting coefficient values, L 最佳 , L 背景 are the optimal AR display brightness, background brightness respectively.

[0019] Preferably, c=8.883, d=0.557.

[0020] The application also provides a perceived brightness characterization system for AR glasses display, comprising:

[0021] An eye tracker for detecting the pupil diameter;

[0022] A photosensor for detecting the background brightness;

[0023] An optimal AR display brightness module for calculating the optimal AR display brightness according to the background brightness and the function relationship 2;

[0024] A perceived brightness calculation module for calculating the perceived brightness according to the optimal AR display brightness, the pupil diameter and the function relationship 1, and judging whether the perceived brightness meets the preset perceived brightness range;

[0025] An AR display brightness adjustment module for adjusting the AR display brightness according to the optimal AR display brightness.

[0026] The application also provides an AR glasses adopting the above method for adjusting the AR display brightness.

[0027] Advantages: the above characterization scheme has the following advantages compared with the prior art:

[0028] The application provides a perceived brightness representation method for AR glasses display, compared with the current rough single representation of display brightness perception, insufficient consideration of the influence of ambient light on perceived brightness and lack of subjective and objective comprehensive representation combined with psychological factors, the application provides a more perfect and suitable perceived brightness representation method for AR display. Based on the basic structure of the human eye visual system and the principle of brightness perception, the application constructs an experimental platform, conducts ambient light simulation experiments of various lighting scenes, innovatively grades the human eye perceived brightness through the subjective visual comfort of the observer, combines the AR display brightness and pupil diameter data measured in the experiment, and establishes a functional relationship between the perceived brightness, AR display brightness and pupil diameter. According to the most reasonable perceived brightness, the best AR display brightness is determined, and a functional relationship between the best AR display brightness and the background brightness is established according to the background brightness data measured in the experiment. According to the method of the application, when using AR glasses, only the pupil diameter and background brightness need to be detected to adjust the AR glasses display brightness. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 The overall schematic diagram of the method of the application.

[0030] Figure 2 The holographic waveguide AR glasses effect diagram described in the application.

[0031] Figure 3 The schematic diagram of the main factors affecting the perceived AR display brightness.

[0032] Figure 4 The schematic diagram of the imaging process of the display light emitting surface into the human eye.

[0033] Figure 5 The image schematic diagram for AR brightness evaluation display of the application.

[0034] Figure 6 The trend graph of the perceived brightness of AR display, background brightness and display brightness.

[0035] Figure 7 The relationship graph of the best brightness of AR display and background brightness. DETAILED DESCRIPTION

[0036] The application will be further illustrated below with reference to the accompanying drawings.

[0037] The perceived brightness representation method for AR glasses display described in the application, as shown in Figure 1 , in which ① is an observer, grading the brightness of the AR glasses display image; and ② is a fixed AR glasses, and its actual effect diagram is as shown in Figure 2The display screen shown, that is, the display screen seen, is fused and superimposed with the real backlight; 3 is an eye tracker, mainly used to capture the size of the pupil of the human eye; 4 and 5 are different types of lighting lamps, each type of lamp can be controlled individually, and can be dimmed and colored, so that a light environment with multiple lighting effects can be realized in one scene, and experiments of the display device in different light environments can be realized, so that the perception of the human eye to the perceived brightness of the display device in different light environments can be more comprehensively analyzed.

[0038] The present application is directed to a perceived brightness representation method for AR display, finds out the influence law of spatial environment illuminance and brightness distribution on the perceived brightness of the display device, and materializes the objective parameters affecting the perceived brightness; further designs the objective experimental scene and link of perceived brightness, invites multiple observers to participate in the perceived brightness evaluation experiment, and establishes the relationship between each influencing factor and perceived brightness, the main factors affecting the perceived brightness of AR display such as Figure 3 as shown.

[0039] Firstly, the present application uses environment light simulation of multiple scenes to obtain the spatial brightness distribution, guides the layout and design of lighting and display devices, and sets up specific experimental links to complete the construction of the experimental scene; secondly, the perceived brightness evaluation and pupil diameter data recording are carried out, and the influence mechanism of display perceived brightness is further improved through the law of experiment; finally, the relationship between background brightness, AR display brightness, pupil diameter and perceived brightness is established through the above experiment, the perceived brightness is further quantified, and is materialized into mathematical formula, so that the perceived brightness can be accurately represented, that is, through the mathematical relationship between the above parameters and the values of the related parameters obtained through the experiment, the mathematical relationship between the perceived brightness of the display device and the above parameters is constructed.

[0040] According to the imaging field angle size of the observed object and the observer, the observation target can be divided into two categories: one category is a light-emitting point target with retinal imaging less than a single visual nerve diameter, such as stars in the night sky; the other category is an observation target with a large visual angle, occupying multiple visual nerve cells, and forming a brightness perception surface. Figure 4 As shown, when the display visual angle needs to be large, the perceived brightness of the human eye depends on the received luminous flux per unit area of the visual nerve cell, when the observed target forms a large visual angle in the human visual system, assuming that the AR display brightness is L, the pupil diameter of the eyeball is D, the focal length of the eye is f', and the eye transmittance is τ. At this time, the light flux obtained by the human visual system conforms to the illuminance formula. According to the imaging illuminance formula, the retinal imaging illuminance can be obtained as shown in formula (1):

[0041]

[0042] For an object in air, n = 1, n' is the refractive index of the vitreous body, about 1.336. Since the object distance is much larger than the image distance and the image-side focal length, the image distance is approximately equal to the image-side focal length, so the following relationship holds:

[0043]

[0044] Substituting this relationship and the constant into equation (1), we get:

[0045]

[0046] The retinal illuminance is proportional to the square of the luminance of the emitting surface and the diameter of the pupil:

[0047] E' = tLD 2 (4)

[0048] where is a constant.

[0049] The perceived brightness level score matches the retinal image illuminance, because the object brightness stimulus acts on the retina, and through a complex visual system processing, the human eye finally perceives the brightness.

[0050] As Figure 5 shown, the present application sets up 10 different image brightnesses in the experiment, measures the brightness of different gray-scale white images through a near-eye display measurement system, records the required gray scale, and randomly sorts the images to generate 7 groups of playing sequences. In the present application, the perceived brightness level score is divided into 13 levels from extremely dark to extremely bright; the experimental space is a room with uniformly distributed ceiling lights, the brightness of which can be adjusted to control the change of ambient light, the wall surface is white to ensure the uniformity of ambient light and reduce the influence of glare effect on the experiment. The observer faces the white wall, the head is fixed by a forehead support, the distance between the observer's eye position and the light waveguide display module is 15 mm, and the distance from the wall surface is 2.3 m (virtual image surface). The field of view of the AR glasses: 40° (diagonal). The present application uses holographic light waveguide AR glasses as an example to conduct a brightness perception evaluation experiment.

[0051] As Figure 6As shown, perceived brightness, AR display brightness, and background brightness exhibit a good monotonic relationship. Specifically, perceived brightness and AR display image brightness are monotonically increasing; perceived brightness and background brightness are monotonically decreasing; and background brightness is affected by the indoor lighting environment, i.e., illuminance. Furthermore, it can be seen that as ambient illuminance increases, the observer's perceived brightness of the target screen decreases. The figure also shows that as AR display brightness increases, the range of observers' screen brightness ratings under different ambient light conditions expands, indicating that changes in ambient illuminance have a greater impact on participants' ratings when the screen brightness is higher than when it is low. Therefore, when AR display brightness is constant, higher ambient illuminance (i.e., higher background brightness) results in smaller pupil size, also a monotonically decreasing relationship; and when AR display brightness is constant, higher ambient illuminance (i.e., higher background brightness) results in a smaller perceived brightness score, also a monotonically decreasing relationship.

[0052] The above analysis of the laws governing eye imaging shows that object brightness and pupil diameter play crucial roles in image formation. Therefore, this invention defines the relationship between object brightness and pupil diameter as "perceived brightness"; this invention defines (LD) in equation (4) as... 2 The influence of ) on perceived brightness score is analyzed as a whole variable factor, and the specific functional form is shown in Equation (5):

[0053]

[0054] Where a and b are the fitting coefficients, a = 7.0756 and b = 7.4335, respectively. It should be noted that these constant coefficients may differ slightly depending on the type of AR glasses.

[0055] Furthermore, such as Figure 7 The relationship between the optimal brightness of the AR display and the background brightness is given. The optimal AR display brightness can be found in the trend graph showing the change in perceived brightness with AR display brightness. Since AR is a display technology that blends virtual and real elements, the optimal AR display brightness can be found through perceived brightness; that is, the AR display brightness corresponding to the intermediate level of perceived brightness is the optimal AR display brightness. This invention provides a basic characterization formula for the optimal AR display brightness based on background brightness, as shown in equation (6):

[0056]

[0057] Where c and d are the fitting coefficients, c = 8.883 and d = 0.557, respectively. It should be noted that these constant coefficients may vary depending on the type of environment and background.

[0058] In fact, the optimal display brightness is related to the perceived brightness, glare condition, visual clarity, observation time and other factors, and the application sets the glare, clarity, observation time and other factors in a specific range and does not change them. The application focuses on the correlation between the AR display brightness under different backgrounds and the perceived brightness of the human eye, which can simplify the evaluation of the comfort of the human eye in visual perception in a certain sense and has a positive significance. In this way, feedback and adjustment of the display device can be realized, and the upgrading and iteration of AR display technology and equipment can be guided.

[0059] In actual use of the AR glasses, the measured background brightness data is substituted into formula (6) to obtain the optimal AR display brightness, then the optimal AR display brightness and the measured pupil diameter are substituted into formula (5) to obtain the perceived brightness, and whether the optimal AR display brightness is reasonable is judged according to the perceived brightness, and the AR glasses adjust the AR display brightness according to the optimal AR display brightness if the optimal AR display brightness is reasonable.

[0060] The application also provides a perceived brightness representation system for AR glasses display, which comprises an eye tracker, a photosensor, an optimal AR display brightness calculation module, a perceived brightness calculation module and an AR display brightness adjustment module (the three modules are three core parts of the perceived brightness representation system for AR glasses display and are hardware and software algorithm function modules). The eye tracker is used for detecting the pupil diameter; the photosensor is used for detecting the background brightness; the optimal AR display brightness calculation module is used for calculating the optimal AR display brightness according to the background brightness and formula (6); the perceived brightness calculation module is used for calculating the perceived brightness according to the optimal AR display brightness, the pupil diameter and formula (5) and judging whether the perceived brightness meets the preset perceived brightness range; and the AR display brightness adjustment module is used for adjusting the AR display brightness according to the optimal AR display brightness.

[0061] The application also provides AR glasses, which are adjusted in AR display brightness by using the perceived brightness representation method for AR glasses display.

[0062] The above description is only the preferred embodiments of the application, and it should be pointed out that those skilled in the art can make some improvements and refinements without departing from the principles of the application, and these improvements and refinements should also be regarded as the protection scope of the application.

Claims

1. A method for representing perceived luminance of AR glasses display, characterized in that, Comprise the following steps: Step 1: carry out the ambient light simulation experiment of multiple lighting scenes, the experimental platform comprises an observer, AR glasses, an eye tracker and lamps and lanterns capable of realizing multiple lighting effects, the observer grades the display brightness of the AR glasses for different lighting scenes, recorded as perceived brightness, and records the background brightness, AR display brightness and pupil diameter data corresponding to different perceived brightness; Step 2: according to the perceived brightness, AR display brightness and pupil diameter data, a function relationship formula 1 of the three is fitted; the function relationship formula 1 is: where a = 7.0756 and b = 7.4335 are the fitting coefficient values, L r L, D are the perceived luminance, the AR display luminance, the pupil diameter, respectively; Step 3: the AR display brightness corresponding to the middle value of the perceived brightness is the optimal AR display brightness; according to the optimal AR display brightness and the background brightness data, a function relationship formula 2 of the two is fitted; the function relationship formula 2 is: where c = 8.883 and d = 0.557 are the fitting coefficient values, L 最佳 , L 背景 are the optimal AR display luminance, background luminance, respectively. Step 4: when actually using the AR glasses, the measured background brightness data is substituted into the function relationship formula 2 to obtain the optimal AR display brightness, and then the optimal AR display brightness and the measured pupil diameter are substituted into the function relationship formula 1 to obtain the perceived brightness.

2. A system for representing the perceived luminance of an AR glasses display based on the method of claim 1, characterized in that, Comprise: An eye tracker for detecting pupil diameter; A photosensor for detecting background brightness; An optimal AR display brightness module for calculating the optimal AR display brightness according to the background brightness and the function relationship formula 2; A perceived brightness calculation module for calculating the perceived brightness according to the optimal AR display brightness, the pupil diameter and the function relationship formula 1, and judging whether the perceived brightness meets the preset perceived brightness range.

Images