AR (Augmented Reality) glasses and method and system for representing perceived brightness displayed by AR glasses

Abstract

The invention discloses AR (Augmented Reality) glasses and a method and a system for representing perceived brightness displayed by the AR glasses. According to the invention, market mainstream AR glasses are used as a display device, and a display brightness perception characterization method of AR display under different ambient lights is provided based on a visual structure, visual characteristics and a brightness perception process of human eyes. Objective measurement and evaluation are utilized to quantify the relation between the perceived brightness displayed by the display device and the physical brightness of the display device, the brightness distribution of ambient light where the display device is located, the pupil diameter of the human eye and the spatial position of the display device and an observer. Firstly, objective parameters in a space are collected through instruments and equipment such as a luminance meter and an illuminometer; secondly, enabling an observer to observe through various test images, collecting the pupil diameter of the observer by using an eye tracker under different ambient lights, and asking the observer to evaluate the perceived brightness level of a display device; the method can be applied to an AR display technology, and provides relatively comfortable display brightness for a user.

Description

Technical Field

[0001] The present invention belongs to the technical field of virtual-reality fusion display, relates to the display brightness measurement characterization of AR glasses display devices, and also relates to the visual perception of the display by the human eye. Background Art

[0002] With the rapid development of information display technology, AR display technology has enhanced our perception and interaction with the environment by seamlessly integrating projected virtual content with real-world scenes, which is helpful for metaverse, digital twins and spatial computing. With the vigorous development of microdisplay technology and ultra-compact imaging optical devices, coupled with the progress of high-speed digital processors, augmented reality has evolved from a future concept to a practical and widely used technology. These technologies have been widely used in many fields, including smart education and training, smart medical care, navigation and pathfinding, game entertainment, smart manufacturing and assembly, etc. Since the original concept of AR was proposed in the 1990s, augmented reality has made significant progress, especially the emergence and development of AR displays based on holographic optical waveguides, which have made wearable systems have the advantages of large field of view, large exit pupil range, lightness and thinness, and similar appearance to ordinary glasses.

[0003] AR display mainly uses wearable devices (similar to glasses or helmets) to overlay virtual content on the user's real-world vision in real time. This device is also called an optical perspective head-mounted display. AR display brightness perception is the ability of AR devices to perceive and adapt to ambient light when displaying virtual images. Through technical means such as automatic brightness adjustment, contrast optimization, and brightness uniformity perception, AR devices can present more comfortable and natural virtual images under different lighting conditions. Since AR display devices are usually used in environments with strong or weak external light, the display is greatly affected by ambient lighting. The existing perceived brightness characterization technology of AR display devices has a rough and single characterization of display brightness perception, does not fully consider the impact of ambient light on perceived brightness, and lacks a comprehensive subjective and objective characterization combined with psychological factors. Therefore, it is very necessary to establish a method for characterizing the perceived brightness of AR display devices. Summary of the invention

[0004] Purpose of the invention: The purpose of the present invention is to address the deficiencies of the prior art and to propose a method and system for characterizing the perceived brightness of AR glasses display. In order to characterize the perceived brightness of AR displays in different real scenarios, the present invention conducts ambient light simulation experiments for a variety of lighting scenarios. Observers rate the perceived brightness levels of AR glasses display brightness for different lighting scenarios. The main influencing factors of perceived brightness are analyzed through subjective and objective experiments, and visualized as a mathematical model to further quantify the perceived brightness and optimize the display design for human eye perceived brightness. Improving human visual perception under the premise of different application scenarios and display content can provide effective assistance to the development and upgrading of AR display devices. Experimental verification shows that the spatial ambient light illumination and brightness distribution, the physical brightness of the display, and the pupil diameter have a direct and significant impact on the subjects' perceived brightness.

[0005] Technical solution:

[0006] The present invention adopts the following technical solutions to solve the above technical problems:

[0007] A method for characterizing perceived brightness of an AR glasses display comprises the following steps:

[0008] Step 1: Conduct ambient light simulation experiments in various lighting scenarios. The experimental platform includes observers, AR glasses, eye trackers, and lamps that can achieve various lighting effects. Observers rate the display brightness of AR glasses in different lighting scenarios, which is recorded as perceived brightness. The background brightness, AR display brightness, and pupil diameter data corresponding to different perceived brightness are recorded.

[0009] Step 2: Fit the functional relationship 1 among the perceived brightness, AR display brightness, and pupil diameter data;

[0010] Step 3: The AR display brightness corresponding to the middle value of the perceived brightness is taken as the optimal AR display brightness; and a functional relationship equation 2 between the optimal AR display brightness and the background brightness data is fitted;

[0011] Step 4: When actually using the AR glasses, substitute the measured background brightness data into Functional Relationship 2 to obtain the optimal AR display brightness, and then substitute the optimal AR display brightness and the measured pupil diameter into Functional Relationship 1 to obtain the perceived brightness. Judge whether the optimal AR display brightness is reasonable based on the perceived brightness. If it is reasonable, the AR glasses adjust the AR display brightness based on the optimal AR display brightness.

[0012] Preferably, the functional relationship 1 is:

[0013]

[0014] Among them, a and b are fitting coefficients, L r, L, and D are perceived brightness, AR display brightness, and pupil diameter, respectively.

[0015] Preferably, a=7.0756, b=7.4335.

[0016] Preferably, the functional relationship 2 is:

[0017]

[0018] Among them, c and d are the fitting coefficient values, L 最佳 , L 背景 They are the optimal AR display brightness and background brightness respectively.

[0019] Preferably, c=8.883, d=0.557.

[0020] The present invention also provides a system for characterizing perceived brightness of an AR glasses display, comprising:

[0021] Eye tracker, used to measure pupil diameter;

[0022] Photoreceptor, used to detect background brightness;

[0023] An optimal AR display brightness module, used to calculate the optimal AR display brightness according to the background brightness and functional equation 2;

[0024] A perceived brightness calculation module, used to calculate the perceived brightness according to the optimal AR display brightness, the pupil diameter and the functional relationship 1, and determine whether the perceived brightness meets the preset perceived brightness range;

[0025] The AR display brightness adjustment module is used to adjust the AR display brightness according to the optimal AR display brightness.

[0026] The present invention also provides an AR glasses, which uses the above method to adjust the AR display brightness.

[0027] Beneficial effects: Compared with the existing solutions, the above characterization scheme adopted in the present invention has the following advantages:

[0028] The present invention provides a method for characterizing the perceived brightness of an AR glasses display. Compared with the current rough single characterization of display brightness perception, the lack of full consideration of the influence of ambient light on the perceived brightness, and the lack of subjective and objective comprehensive characterizations such as psychological factors, a more complete and suitable method for characterizing perceived brightness of AR display is provided. Based on the basic structure of the human visual system and the principle of brightness perception, the present invention constructs an experimental platform, conducts ambient light simulation experiments in various lighting scenes, innovatively rates the perceived brightness of the human eye through the subjective visual comfort of the observer, and establishes a functional relationship between the perceived brightness, AR display brightness, and pupil diameter in combination with the experimentally measured AR display brightness and pupil diameter data. The optimal AR display brightness is also determined based on the most reasonable perceived brightness, and the functional relationship between the optimal AR display brightness and background brightness is established in combination with the experimentally measured background brightness data. According to the method of the present invention, when using AR glasses, the display brightness of the AR glasses can be adjusted by simply detecting the pupil diameter and background brightness. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 An overall schematic diagram of the method of the present invention.

[0030] Figure 2 Effect diagram of the holographic waveguide AR glasses described in the present invention.

[0031] Figure 3 Schematic diagram of the main factors affecting the perceived brightness of AR displays.

[0032] Figure 4 Schematic diagram showing the imaging process of a luminous surface entering the human eye.

[0033] Figure 5 A schematic diagram of an image used for AR brightness evaluation display according to the present invention.

[0034] Figure 6 AR display’s perceived brightness versus background brightness and display brightness trend graph.

[0035] Figure 7 A graph showing the relationship between the optimal brightness of the AR display and the background brightness. DETAILED DESCRIPTION

[0036] The present invention will be further explained below in conjunction with the accompanying drawings.

[0037] The method for characterizing the perceived brightness of an AR glasses display described in the present invention is as follows: Figure 1 As shown in the figure, ① is an observer who rates the brightness of the image displayed by the AR glasses; ② is a fixed AR glasses, and its actual effect is shown in the figure Figure 2As shown, the displayed image you see is fused and superimposed with the real backlight; ③ is an eye tracker, which is mainly used to capture the size of the human pupil; ④ and ⑤ are different types of lighting fixtures, each of which can be controlled individually and can be dimmed and adjusted in color, so that a variety of lighting effects can be achieved in one scene. Experiments with display devices in different light environments can be realized, thereby more comprehensively analyzing the human eye's perceived brightness of the flat-panel display device displayed by the AR display device in different light environments.

[0038] The present invention aims at a method for characterizing the perceived brightness of AR display, finds out the influence of the spatial ambient light illumination and brightness distribution on the perceived brightness of the display device, and concretizes the objective parameters that affect the perceived brightness; further designs objective experimental scenes and links of perceived brightness, invites multiple observers to participate in the perceived brightness evaluation experiment, and establishes the relationship between each influencing factor and perceived brightness. The main factors that affect the perception of AR display brightness are as follows: Figure 3 shown.

[0039] The present invention firstly utilizes ambient light simulation of various scenes to obtain the spatial brightness distribution, guides the layout of lighting and display equipment and designs a real light environment with adjustable illumination and different screen brightnesses, and sets specific experimental links to complete the construction of experimental scenes; secondly, the perceived brightness is evaluated and pupil diameter data is recorded, and the influence mechanism of the displayed perceived brightness is further improved through the regular analysis of the experiment; finally, the relationship between background brightness, AR display brightness, pupil diameter and perceived brightness is established through the above experiment, and the perceived brightness is further quantified and visualized as a mathematical formula, so as to accurately characterize the perceived brightness, that is, through the existing mathematical relationship between the above parameters and the values ​​of the relevant parameters obtained through the experiment, the mathematical relationship between the perceived brightness displayed by the display device and the above parameters is constructed.

[0040] According to the size of the field of view of the observed object and the observer, the observed targets can be divided into two categories: one is the luminous point target whose retinal imaging is smaller than the diameter of a single optic nerve, such as the stars in the night sky; the other is the observed object with a larger viewing angle, occupying multiple optic nerve cells, forming a brightness perception surface. Figure 4 As shown, when the display visual angle needs to be larger, the human eye's perception of brightness depends on the luminous flux received per unit area of ​​the optic nerve cell. When the viewing angle of the observed target in the human eye visual system is larger, assuming that the AR display brightness is L, the pupil diameter is D, the focal length of the eye is f', and the eye transmittance is τ. At this time, the luminous flux obtained by the human eye vision conforms to the illuminance formula. According to the illuminance formula for imaging, the retinal imaging illuminance can be obtained as shown in formula (1):

[0041]

[0042] For an object in the air, n = 1, n' is the refractive index of the glass, which is about 1.336. Since the object distance is much larger than the image distance and the image focal length, and the image distance is approximately equal to the image focal length, the following relationship holds true:

[0043]

[0044] Substituting this relationship and constant into equation (1), we can obtain:

[0045]

[0046] The retinal illuminance is proportional to the brightness of the luminous surface and the square of the pupil diameter:

[0047] E'=tLD 2 (4)

[0048] in is a constant.

[0049] The perceived brightness level score is matched to the retinal imaging illuminance because the brightness stimulation of the object acts on the retina, which is ultimately processed by the complex visual system to obtain the perceived brightness of the human eye.

[0050] like Figure 5 As shown, the present invention sets 10 different image brightnesses in the experiment, measures the brightness of white images with different grayscales through a near-eye display measurement system, records the required grayscale, and randomly sorts the images to generate 7 sets of playback orders. In the perceived brightness level scoring in the present invention, a total of 13 levels are divided from extremely dark to extremely bright; the experimental space is a room with evenly distributed ceiling lights, and the brightness of the ceiling lights is adjustable, thereby controlling the changes in ambient light. The walls are white walls to ensure the uniformity of ambient light and reduce the possible impact of glare effects on the experiment. The observer faces the white wall and fixes his head with a forehead support. The distance between the observer's eye position and the optical waveguide display module is 15mm, and the distance from the wall is 2.3m (virtual image plane). The field of view angle of AR glasses: 40° (diagonal). The method of the present invention uses holographic optical waveguide AR glasses as an example to conduct a brightness perception evaluation experiment.

[0051] like Figure 6As shown in the figure, the perceived brightness and the AR display brightness and background brightness show a good monotonic relationship. Among them, the perceived brightness and the image brightness of the AR display are in a monotonically increasing relationship; the perceived brightness and the background brightness are in a monotonically decreasing relationship; and the background brightness is the size of the background brightness that changes through the indoor lighting environment, that is, the illumination. It can be further known that when the ambient illumination increases, the observer's perceived brightness of the target screen will decrease. It can also be seen from the figure that as the AR display brightness increases, the observer's screen brightness rating range under different ambient light conditions expands, indicating that compared with the low screen brightness, when the screen brightness is high, the change in ambient light illumination has a greater impact on the subject's score. Therefore, when the AR display brightness is constant, the higher the ambient light illumination, that is, the higher the background brightness, the smaller the pupil size, which is also a monotonically decreasing relationship; and when the AR display brightness is constant, the higher the ambient light illumination, that is, the higher the background brightness, the smaller the perceived brightness score, which is a monotonically decreasing relationship.

[0052] Through the above analysis of the imaging law of the eye, it can be known that the brightness of the object and the pupil diameter play a key role in imaging. Therefore, the present invention defines the relationship between the brightness of the object and the pupil diameter as "perceived brightness"; the present invention converts (LD 2 ) is used as a whole variable factor to analyze the impact on the perceived brightness score. The specific function form is shown in formula (5):

[0053]

[0054] Among them, a and b are fitting coefficient values, which are a=7.0756 and b=7.4335 respectively. It should be noted that these constant coefficients may vary slightly for different types of AR glasses.

[0055] Further, if Figure 7 The relationship between the optimal brightness of AR display and the background brightness is given. In the trend diagram of the change of perceived brightness with the change of AR display brightness, the optimal AR display brightness can be found. Because AR is a display technology that integrates virtuality and reality, the optimal AR display brightness can be found by perceived brightness, that is, the AR display brightness corresponding to the middle level of perceived brightness level is the optimal AR display brightness. The present invention provides a basic characterization formula for the optimal AR display brightness based on background brightness as shown in formula (6):

[0056]

[0057] Among them, c and d are the fitting coefficient values, which are c = 8.883 and d = 0.557 respectively. It should be noted that these constant coefficients may vary slightly in different types of environments and backgrounds.

[0058] In fact, the optimal display brightness is related not only to perceived brightness, but also to factors such as glare, visual clarity, and observation time. The present invention sets glare, clarity, and observation time within a specific range and does not change them. The present invention focuses on associating the AR display brightness with the human eye perceived brightness under different backgrounds, which can simplify the evaluation of human eye comfort in visual perception in a certain sense, and has positive significance. In this way, feedback and adjustment of the display device can be achieved, and the upgrade and iteration of AR display technology and equipment can be guided.

[0059] When the AR glasses are actually used, the measured background brightness data is substituted into formula (6) to obtain the optimal AR display brightness. Then, the optimal AR display brightness and the measured pupil diameter are substituted into formula (5) to obtain the perceived brightness. The optimal AR display brightness is judged to be reasonable based on the perceived brightness. If it is reasonable, the AR glasses adjust the AR display brightness based on the optimal AR display brightness.

[0060] The present invention also provides a perceived brightness characterization system for an AR glasses display, including an eye tracker installed in the AR glasses, a photoreceptor (mainly monitoring the brightness of the ambient light), an optimal AR display brightness calculation module, a perceived brightness calculation module, and an AR display brightness adjustment module (these three modules are the three core parts of the perceived brightness characterization system for the AR glasses display, and are functional modules of hardware plus software algorithms). The eye tracker is used to detect the pupil diameter; the photoreceptor is used to detect the background brightness; the optimal AR display brightness calculation module is used to calculate the optimal AR display brightness based on the background brightness and formula (6); the perceived brightness calculation module is used to calculate the perceived brightness based on the optimal AR display brightness, the pupil diameter, and formula (5), and to determine whether the perceived brightness meets the preset perceived brightness range; the AR display brightness adjustment module is used to adjust the AR display brightness based on the optimal AR display brightness.

[0061] The present invention also provides an AR glasses, which uses a method for characterizing the perceived brightness of an AR glasses display to adjust the AR display brightness.

[0062] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for characterizing perceived brightness of an AR glasses display, characterized in that: The steps include: Step 1: Conduct ambient light simulation experiments in various lighting scenarios. The experimental platform includes observers, AR glasses, eye trackers, and lamps that can achieve various lighting effects. Observers rate the display brightness of AR glasses in different lighting scenarios, which is recorded as perceived brightness. The background brightness, AR display brightness, and pupil diameter data corresponding to different perceived brightness are recorded. Step 2: Fit the functional relationship 1 among the perceived brightness, AR display brightness, and pupil diameter data; Step 3: The AR display brightness corresponding to the middle value of the perceived brightness is taken as the optimal AR display brightness; and a functional relationship equation 2 between the optimal AR display brightness and the background brightness data is fitted; Step 4: When actually using the AR glasses, substitute the measured background brightness data into Functional Relationship 2 to obtain the optimal AR display brightness, and then substitute the optimal AR display brightness and the measured pupil diameter into Functional Relationship 1 to obtain the perceived brightness. Judge whether the optimal AR display brightness is reasonable based on the perceived brightness. If it is reasonable, the AR glasses adjust the AR display brightness based on the optimal AR display brightness.

2. The method for characterizing perceived brightness of an AR glasses display according to claim 1, characterized in that: The functional relationship 1 is: Among them, a and b are fitting coefficients, L r , L, and D are perceived brightness, AR display brightness, and pupil diameter, respectively.

3. The method for characterizing perceived brightness of an AR glasses display according to claim 2, characterized in that: a=7.0756,b=7.4335.

4. The method for characterizing perceived brightness of an AR glasses display according to claim 1, characterized in that: The functional relationship 2 is: Among them, c and d are the fitting coefficient values, L 最佳 , L 背景 They are the optimal AR display brightness and background brightness respectively.

5. The method for characterizing perceived brightness of an AR glasses display according to claim 4, characterized in that: c=8.883, d=0.

557.

6. A system for characterizing perceived brightness of an AR glasses display based on the method of any one of claims 1 to 5, characterized in that: include: Eye tracker, used to measure pupil diameter; Photoreceptor, used to detect background brightness; An optimal AR display brightness module, used to calculate the optimal AR display brightness according to the background brightness and functional equation 2; A perceived brightness calculation module, used to calculate the perceived brightness according to the optimal AR display brightness, the pupil diameter and the functional relationship 1, and determine whether the perceived brightness meets the preset perceived brightness range; The AR display brightness adjustment module is used to adjust the AR display brightness according to the optimal AR display brightness.

7. An AR glasses, characterized in that: The AR display brightness is adjusted using the method described in any one of claims 1 to 5.

Images