A method for color characterization of an optical see-through augmented reality system

Abstract

The application provides a kind of optical perspective augmented reality system color characterization method, by the full field of view measurement of different ambient light conditions and OST AR virtual stimulus color, the mathematical relationship between the virtual display color that AR display should output and human eye perceived color and ambient light is established, the mathematical relationship can describe how AR display adjusts its color output under different light conditions to compensate for the influence of ambient light on the final perceived color, so as to improve the accuracy and consistency of color reproduction, that is, the application aims to correct the nonlinear influence of ambient light on AR virtual display content through the nonlinear mixing compensation model of ambient light measurement and OST AR virtual stimulus color, correct the human eye perceived color deviation of OST AR device under complex lighting environment, improve the color consistency of OST AR display under lighting environment and the color restoration ability of OST AR system under different lighting conditions, realize more natural virtual-real fusion effect.

Description

Technical Field

[0001] This invention belongs to the field of augmented reality technology, and in particular relates to a method for color characterization in an optical perspective augmented reality system. Background Technology

[0002] In recent years, augmented reality (AR) technology has seen widespread development. Optical perspective augmented reality (OST AR), as an important branch, can overlay virtual information while users perceive the real world, achieving a fusion of the virtual and real worlds. Existing methods are typically based on the mapping from RGB to CIE XYZ tristimulus values ​​and use a gain-offset-gamma (GOG) model for color calibration, optimizing parameters by measuring the spectrum and XYZ response of the input RGB values. GOG characterization is a widely used modeling method for color calibration of display devices. It achieves high-precision color management by establishing a mapping relationship between the input RGB signals and the output color characteristics. This method is based on three core parameters: gain, offset, and gamma, which are used to adjust the amplification factor of the RGB channels, correct the deviation of the black level, and describe the nonlinear relationship between the input signal and the output brightness, respectively, so that the color reproduction conforms to human visual perception. The GOG characterization method for displays is based on assumptions such as color additivity, channel independence, spatial uniformity, and temporal stability. It has low computational cost and fast optimization convergence, and is therefore widely used in the color calibration of self-emissive display devices such as LCD and OLED. However, the GOG characterization method still has significant limitations in the OST AR environment. First, the GOG method assumes channel independence in the display system, meaning the optical characteristics of the red, green, and blue channels can be modeled separately. However, due to the optical path design of the optical combiner in OST AR, channel coupling effects often occur, making it difficult to accurately describe the actual color output by individually optimizing the gain and gamma parameters of each channel. Furthermore, the GOG method relies on the assumptions of additive color and spatial uniformity, assuming that different areas of the same device have the same color response characteristics. In OST AR, due to the angular sensitivity of the optical structure, the color reproduction capability at different locations may vary significantly, making the GOG method, based on a global model, unsuitable for the fine-grained calibration requirements of OST AR.

[0003] Furthermore, traditional GOG methods typically involve measurements and calibrations performed in complete darkness to avoid ambient light interference and ensure accurate color calibration. While this approach is suitable for traditional display devices, in augmented reality (AR) devices, especially optical perspective (OST) AR devices, the color perceived by the user is the result of the display content being superimposed on the ambient background light. Ambient background light significantly impacts the user's perceived color. OST AR devices are used in complex and varied environments. In environments with strong outdoor light or multi-source indoor lighting, background light can weaken the contrast of virtual images, leading to significant deviations in GOG parameters calibrated in dark environments during actual use. Therefore, ignoring the influence of ambient light will result in the displayed image failing to accurately reproduce the user's desired target color, causing display deviations. Moreover, traditional GOG methods assume constant and negligible ambient light, failing to directly consider the impact of dynamic changes in background light on color, necessitating additional compensation mechanisms. Therefore, in response to the specific needs of OSTAR, it is urgent to construct a background calibration and nonlinear superposition model, comprehensively consider the transmission characteristics of the OSTAR system and the influence of ambient light, and establish a color calibration method adapted to different background lighting conditions, thereby improving the color consistency of the augmented reality system and making the integration of virtual and reality more natural. Summary of the Invention

[0004] To address the aforementioned issues, this invention provides a color characterization method for an optical perspective augmented reality system. It establishes a mathematical relationship between the target color that the AR display should output, the color perceived by the human eye, and ambient light. This relationship can describe how the AR display adjusts its color output under different lighting conditions to compensate for the influence of ambient light on the final perceived color, thereby improving the accuracy and consistency of color reproduction.

[0005] A method for color characterization in an optical perspective augmented reality system includes the following steps:

[0006] S1: The mathematical relationship between the theoretical output tristimulus values ​​of the AR display device, the tristimulus values ​​of color perceived by the target human eye, and the tristimulus values ​​of the ambient light in the field of view is as follows:

[0007] XYZ AR (i,j)=[XYZ target (i,j)-(2-t)XYZ env (i,j)] / t

[0008] Among them, XYZ AR (i,j) represents the theoretical output tristimulus value of the AR display device at any position (i,j) within the field of view, XYZ target (i,j) represents the tristimulus values ​​of the target color perceived by the human eye at any position (i,j) within the field of view, XYZenv (i,j) represents the tristimulus value of the pre-calibrated ambient illumination at any position (i,j) within the field of view, and t represents the nonlinear mixing weight.

[0009] S2: XYZ values ​​at the given target position (i*, j*) target The values ​​of (i*,j*) are used to obtain the XYZ values ​​corresponding to the target position (i*,j*) based on the mathematical relationship. AR (i*,j*);

[0010] S3: For the XYZ AR (i*,j*) are sequentially subjected to inverse Gaussian transform and inverse GOG transform to obtain the RGB driving values ​​of the AR display device.

[0011] Furthermore, the tristimulus values ​​XYZ of the ambient illumination at all locations within the field of view. env The method to obtain it is as follows:

[0012] XYZ env =M·RGB env

[0013] Where M is the camera characterization transformation matrix, RGB env These are the color values ​​of the ambient light image within the camera's field of view, obtained when the camera captures the ambient light within its field of view.

[0014] Furthermore, the method for obtaining the camera characteristic transformation matrix M is as follows:

[0015] Under D65 light source illumination, the color values ​​of the original images of the standard color card were captured multiple times by a camera, and the CIE1931 tristimulus values ​​of each color block of the standard color card were measured multiple times by a spectrometer to obtain multiple sets of color values ​​and tristimulus values.

[0016] The mapping relationship between the color values ​​of the original image used to construct the standard color chart and the CIE1931 tristimulus values ​​is as follows:

[0017]

[0018] Where (R,G,B) are the color values ​​on the R, G, and B channels of the original image of the standard color chart, respectively; (X,Y,Z) are the CIE1931 tristimulus values ​​of each color block of the standard color chart; a1~a8, b1~b8, and c1~c8 are the undetermined regression coefficients in the camera characteristic transformation matrix M; and T represents transpose.

[0019] Substitute each set of color values ​​and tristimulus values ​​into the mapping relationship, and use the least squares method to fit the undetermined regression coefficients in the mapping relationship to obtain the camera characterization transformation matrix M.

[0020] Furthermore, the nonlinear hybrid weight t is calculated based on the Weber contrast ratio, and the nonlinear hybrid weight t exhibits a Gaussian-like decay relationship with the Weber contrast ratio.

[0021] Furthermore, the method for obtaining the nonlinear hybrid weight t is as follows:

[0022]

[0023] Among them, C W To measure the Weber contrast ratio, which is used to measure the relative brightness difference between AR stimuli and ambient lighting in the field of view, α t To control the adaptability of the nonlinear hybrid weighting to ambient illumination in the field of view, β t γ represents the initial amplitude of the ambient illumination in the field of view. t k is the decay rate of ambient light in the field of view. t This serves as the baseline value for ambient lighting in the field of view.

[0024] Furthermore, Weber contrast C W The calculation method is as follows:

[0025]

[0026] Among them, L AR L represents the brightness of the tristimulus values ​​theoretically output by the AR display device. BG The brightness is the tristimulus value of the ambient light in the field of view.

[0027] Furthermore, regarding the XYZ... AR The method for performing the inverse Gaussian transform on (i*,j*) is as follows:

[0028]

[0029] Among them, XYZ AR-norm (i*,j*) represents the XYZ values ​​after the inverse Gaussian transform. AR (i*,j*), (i0,j0) represent XYZ AR (i*,j*) represents the coordinates of the location where the maximum normalized brightness value generated by the AR display device is located, and σ is the set Gaussian distribution variance.

[0030] Furthermore, regarding the XYZ... AR-norm The method for performing the GOG inverse transform on (i*,j*) is as follows:

[0031] [L r L g L b ] T =M AR -1 [X AR-norm YAR-norm Z AR-norm ] T

[0032]

[0033] Among them, X AR-norm Y AR-norm Z AR-norm XYZ respectively AR-norm The three stimulus values ​​M of (i*,j*) AR The AR color transformation matrix describes the conversion relationship between RGB driving values ​​and XYZ stimulus values. r L g L b These are the normalized brightness values ​​for the R, G, and B channels, respectively. R, G, and B are the final RGB driving values ​​for the AR display device, and α is the RGB value for the AR display device. r α g α b These represent the gains of the R, G, and B channels of the AR display device, respectively, and C... r C g C b These are the gamma values ​​of the R, G, and B channels of the AR display device, respectively. max G max B max These are the maximum driving values ​​that the R, G, and B channels of the AR display device can output, respectively.

[0034] AR color transformation matrix M AR The method for obtaining it is as follows:

[0035] [X AR Y AR Z AR ] T =M AR [L r (R)L g (G)L b (B)] T

[0036]

[0037] Where R, G, and B are the RGB driving values ​​of the AR display device, and L is the RGB driving value of the AR display device. r (R) is the normalized brightness value generated on the R channel at the location of the maximum brightness of the AR display device when the driving value of the R channel is R. g (G) is the normalized brightness value generated on the G channel at the location of the maximum brightness of the AR display device when the driving value of the G channel is taken as G, L b(B) is the normalized brightness value generated on the B channel at the location of the maximum brightness of the AR display device when the driving value of the B channel is B. These are the tristimulus values ​​at the location of the maximum brightness of the AR display device when the R channel has the maximum drive value and all other channels are 0. These are the tristimulus values ​​at the location of the maximum brightness of the AR display device when the G channel has the maximum drive value and all other channels are 0. X represents the tristimulus value at the location of the maximum brightness of the AR display device when the B channel has the maximum drive value and all other channels are 0. AR Y AR Z AR These are the tristimulus values ​​perceived by the human eye in the field of view of the AR display device;

[0038] By setting different RGB driving values ​​R, G, and B, we can obtain the L corresponding to different RGB driving values ​​R, G, and B. r (R), L g (G), L b (B); Simultaneously, an imaging colorimeter was used to measure the tristimulus values ​​generated by the AR display device in the field of view under different RGB driving values ​​R, G, and B, and these values ​​were used as X. AR Y AR Z AR ;

[0039] The L corresponding to different RGB driving values ​​R, G, B r (R), L g (G), L b (B) and the tristimulus value X AR Y AR Z AR Substituting the theoretical relationship into the equations and fitting the undetermined tristimulus values ​​in the theoretical relationship using the least squares method, we obtain the AR color transformation matrix M. AR .

[0040] Beneficial effects:

[0041] 1. This invention provides a color characterization method for an optical perspective augmented reality system. By performing full-field measurements of the colors of OST AR virtual stimuli under different ambient lighting conditions, a mathematical relationship is established between the virtual display colors that the AR display should output, the colors perceived by the human eye, and ambient light. This mathematical relationship can describe how the AR display adjusts its color output under different lighting conditions to compensate for the influence of ambient light on the final perceived color, thereby improving the accuracy and consistency of color reproduction. In other words, this invention aims to correct the nonlinear influence of ambient light on the AR virtual display content through a nonlinear hybrid compensation model of ambient light measurement and OST AR virtual stimuli colors, correct the human eye's perceived color deviation of OST AR devices in complex lighting environments, improve the color consistency of OST AR displays under lighting conditions, and enhance the color reproduction capability of OST AR systems under different lighting conditions. This reduces the impact of spatial non-uniformity of optical perspective augmented reality display devices on the accuracy of characterization, thereby characterizing optical perspective augmented reality under lighting environments to achieve a more natural virtual-real fusion effect.

[0042] 2. This invention provides a color characterization method for an optical perspective augmented reality system. Based on a graphical measurement strategy and a global characterization strategy, it uses a high-precision camera and an imaging colorimeter to perform global characterization of the ambient light and the spatial position of the OSTAR optical system across the entire field of view. At the same time, it combines an image-based spatial non-uniformity correction method using a high-precision camera and an imaging colorimeter to reduce spatial color deviation caused by changes in viewing angle and optical characteristics of the OSTAR device, improve the spatial non-uniformity of system color, enhance color rendering consistency, and improve the accuracy of color reproduction of the device in a wide field of view. Attached Figure Description

[0043] Figure 1 This invention provides a process for characterizing OST AR displays.

[0044] Figure 2 The camera characterization process provided by this invention;

[0045] Figure 3 This invention provides a process for characterizing OST AR displays in dark environments.

[0046] Figure 4 For ColorChecker Digital SG standard color chart;

[0047] Figure 5 The OST AR brightness Gaussian fitting model provided by this invention

[0048] Figure 6 The OST AR brightness gamma curve provided for this invention;

[0049] Figure 7 The relationship curve between AR virtual stimulus mixing weights and Weber contrast provided by this invention. Detailed Implementation

[0050] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0051] In recent years, the widespread application of Augmented Reality (AR) technology has driven the development of Optical See-Through Augmented Reality (OSTAR) display systems. To ensure accurate color reproduction of OSTAR devices under different lighting conditions, it is crucial to establish color characterization methods tailored to their optical properties. Since OSTAR systems use optical combiners to overlay virtual images onto the real environment, the final visual signal is not only affected by the self-emissive characteristics of the display device but also closely related to the transmission and reflection characteristics of ambient light. Furthermore, the primary color channels of OSTAR systems may exhibit spatial inhomogeneity, meaning that chromaticity and luminance may differ significantly at different viewpoints. Therefore, this invention employs a high-precision camera as a measurement tool, acquiring environmental color data at different spatial locations through image-based measurement methods. Simultaneously, an imaging colorimeter measures the AR virtual stimulus color data at different spatial locations, reducing the impact of spatial inhomogeneity and improving measurement accuracy. Furthermore, by combining ambient light measurement and compensation methods, comprehensive characterization of the OSTAR system under different lighting conditions is achieved.

[0052] like Figure 1 As shown, a method for color characterization in an optical perspective augmented reality system includes the following steps:

[0053] S1: The mathematical relationship between the theoretical output tristimulus values ​​of the AR display device, the tristimulus values ​​of color perceived by the target human eye, and the tristimulus values ​​of the ambient light in the field of view is as follows:

[0054] XYZ AR (i,j)=[XYZ target (i,j)-(2-t)XYZ env (i,j)] / t

[0055] Among them, XYZ AR (i,j) represents the theoretical output tristimulus value of the AR display device at any position (i,j) within the field of view, XYZ target (i,j) represents the tristimulus values ​​of the target color perceived by the human eye at any position (i,j) within the field of view, XYZ env(i,j) represents the tristimulus value of the pre-calibrated ambient illumination at any position (i,j) within the field of view, and t represents the nonlinear mixing weight.

[0056] S2: XYZ values ​​at the given target position (i*, j*) target The values ​​of (i*,j*) are used to obtain the XYZ values ​​corresponding to the target position (i*,j*) based on the mathematical relationship. AR (i*,j*);

[0057] S3: For the XYZ AR (i*,j*) are sequentially subjected to inverse Gaussian transform and inverse GOG transform to obtain the RGB driving values ​​of the AR display device.

[0058] Thus, this invention has obtained the display driving value of the color to be displayed under ambient light at the target spatial location of OST AR.

[0059] It should be noted that the target color tristimulus value (target) of this invention refers to the tristimulus value that is ultimately perceived by the human eye. For example, if the human eye expects to see a red color with a tristimulus value of (X=100, Y=20, Z=20) at a certain pixel, then XYZ target (i,j) is (X=100,Y=20,Z=20). However, since what is seen in an AR display device is the result of the superposition of the virtual display content color and the real background color, when driving the AR display device, it is not possible to simply look up the tristimulus values ​​corresponding to all driving values ​​of the AR display device in a dark environment. Based on this, the present invention introduces ambient light (env) modeling. That is to say, the characteristic process of the present invention is essentially to establish the RGB driving values ​​of the AR display device and the tristimulus values ​​XYZ that the human eye expects to see. target The relationship between (i,j), where XYZ target (i,j) are known and can be determined by the user to display the desired color. For example, suppose the user wants to see a color with tristimulus values ​​of (X=100, Y=20, Z=20). Based on the above mathematical relationship, the RGB driving value of the AR display device can be derived. This RGB driving value of the AR display device causes the AR display device to generate tristimulus values ​​XYZ. AR (i,j) and the tristimulus values ​​XYZ of the ambient illumination in the background field of view. env The values ​​(i,j) are superimposed to obtain the color that the user expects to see with tristimulus values ​​(X=100,Y=20,Z=20).

[0060] It should be noted that, to accurately obtain the spatial color distribution of ambient light, this invention employs a high-precision camera to acquire ambient light color data. First, the measuring camera is color-characterized to establish a mapping relationship between the RGB values ​​acquired by the camera and the standard CIE1931 XYZ color space. This process uses a standard D65 light source and acquires color data from a standard color chart under 0° / 45° measurement geometry conditions. Specifically, the tristimulus values ​​XYZ of the ambient light at all locations within the field of view are... env The method to obtain it is as follows:

[0061] XYZ env =M·RGB env

[0062] Where M is the camera characterization transformation matrix, RGB env These are the color values ​​of the ambient light image within the camera's field of view, obtained when the camera captures the ambient light within its field of view.

[0063] In other words, to accurately characterize the effect of ambient light on color, this invention uses a camera to capture images of ambient light and obtains the RGB values ​​of the ambient light image within the camera's field of view. env By applying the camera characteristic transformation matrix M to convert the RGB values ​​to the CIEXYZ color space, the tristimulus values ​​XYZ of the ambient illumination in the field of view can be obtained. env .

[0064] Specifically, such as Figure 2 As shown, the method for obtaining the camera characteristic transformation matrix M is as follows:

[0065] Under the illumination of the D65 light source, multiple shots were taken with the camera as follows: Figure 4 The original image of the ColorChecker Digital SG standard color chart is shown. At the same time, the CIE1931 tristimulus values ​​of each color block of the standard color chart are measured multiple times using a high-precision spectrometer to obtain multiple sets of color values ​​and tristimulus values.

[0066] To improve accuracy, this invention introduces higher-order terms based on the standard linear transformation to capture the complex nonlinear relationship between the camera's RGB and CIE XYZ values. Based on this, the mapping relationship between the color values ​​of the original image of the standard color chart and the CIE1931 tristimulus values ​​is constructed as follows:

[0067]

[0068] Where (R,G,B) are the color values ​​of the R, G, and B channels of the original image of the standard color chart, respectively; (X,Y,Z) are the CIE1931 tristimulus values ​​of each color patch of the standard color chart; the spectral data of the color chart are directly measured by a spectrometer under the same lighting and observation conditions, and calculated by the observer according to the CIE1931 standard; a1~a8, b1~b8, c1~c8 are the undetermined regression coefficients in the camera characteristic transformation matrix M; and T represents transpose.

[0069] By substituting each set of color values ​​and tristimulus values ​​into the mapping relationship, and using the least squares method to fit the undetermined regression coefficients in the mapping relationship, the camera characteristic transformation matrix M is obtained. This matrix establishes a mapping relationship from the camera's RGB space to the CIEXYZ color space.

[0070] Therefore, the present invention aims to use the least squares method to fit the parameters to be determined during the solution process, find the optimal coefficient matrix, and make the predicted XYZ values ​​closest to the actual measured values, thus obtaining the characteristic matrix M; through the transformation matrix M, the data captured by all cameras can be converted to the standard CIE XYZ color space.

[0071] Since the color perceived by the human eye in AR displays is not solely determined by the light emitted by the AR device, but rather by a non-linear mixture of AR virtual stimuli and ambient light, it is necessary to model its mixing characteristics. This mixing process is influenced by the contrast between the AR stimuli and the background light. Contrast plays a crucial role in visual perception, affecting the salience and visibility of colors. To quantify the contrast between the AR stimuli and the background light, Weber contrast is used for calculation. Weber contrast measures the degree of change in the brightness of the displayed target relative to the background brightness, and its expression is shown below:

[0072]

[0073] Among them, L AR L represents the brightness of the tristimulus values ​​theoretically output by the AR display device. BG The brightness is the tristimulus value of the ambient light in the field of view.

[0074] Meanwhile, since there is a stable mathematical relationship between the nonlinear mixing weights of AR virtual stimuli and background light and Weber contrast, this relationship can be used to quantify the visual fusion characteristics of AR color appearance. Experimental results show that the perception of AR color is significantly affected by background light in different lighting environments, and the degree of influence can be described by the nonlinear mixing weights t. Based on the fitting analysis of the experimental data, the following results were obtained: Figure 7 The mixing weights between the AR stimulus and the background light shown are compared with Weber contrast C.W The mathematical relationship between them, specifically, the method for obtaining the nonlinear mixed weight t is as follows:

[0075]

[0076] Among them, C W Weber contrast, which measures the relative brightness difference between AR stimuli and ambient light in the field of view, and α, which measures the relative brightness difference between AR stimuli and background light, are used to measure the relative brightness difference between AR stimuli and background light. t To control the adaptability of the nonlinear hybrid weighting to ambient illumination in the field of view, β t γ represents the initial amplitude of the ambient illumination in the field of view. t k is the decay rate of ambient light in the field of view. t The baseline value for ambient lighting in the field of view is denoted as t. The nonlinear mixing weight t reflects the proportion of AR color in the final perceived color, which corrects the OSTAR color characterization model, optimizes the fusion effect of AR color and ambient light, and improves the color consistency and perceived quality of OSTAR under different lighting conditions.

[0077] For the XYZ AR The method for performing the inverse Gaussian transform on (i*,j*) is as follows:

[0078]

[0079] Among them, XYZ AR-norm (i*,j*) represents the XYZ values ​​after the inverse Gaussian transform. AR (i*,j*), (i0,j0) represent XYZ AR (i*,j*) represents the coordinates of the location where the maximum normalized brightness value generated by the AR display device is located, and σ is the set Gaussian distribution variance.

[0080] For the XYZ AR-norm The method for performing the GOG inverse transform on (i*,j*) is as follows:

[0081] [L r L g L b ] T =M AR -1 [X AR-norm Y AR-norm Z AR-norm ] T

[0082]

[0083] Among them, X AR-norm Y AR-norm Z AR-norm XYZ respectivelyAR-norm The three stimulus values ​​M of (i*,j*) AR The AR color transformation matrix describes the conversion relationship between RGB driving values ​​and XYZ stimulus values. r L g L b These are the normalized brightness values ​​for the R, G, and B channels, respectively. R, G, and B are the final RGB driving values ​​for the AR display device, and α is the RGB value for the AR display device. r α g α b These represent the gains of the R, G, and B channels of the AR display device, respectively, and C... r C g C b These are the gamma values ​​of the R, G, and B channels of the AR display device, respectively. max G max B max These are the maximum driving values ​​that the R, G, and B channels of the AR display device can output, respectively.

[0084] Furthermore, to establish the relationship between the colors of the virtual stimuli emitted by the AR display device and the RGB driving values ​​of the AR display device, dark-field characterization of the OST AR display device is required in a dark environment free from ambient light interference. First, a high-precision imaging colorimeter is used to measure the virtual stimuli emitted by the AR display device, collecting luminance and chromaticity data under different RGB driving values ​​within the AR display range, establishing the relationship between the RGB input signal and the CIEXYZ color space output; based on this, such as... Figure 3 As shown, the AR color transformation matrix M AR The method for obtaining it is as follows:

[0085] The theoretical relationship between constructing arbitrary input RGB driving values ​​and CIEXYZ color space output is as follows:

[0086] [X AR Y AR Z AR ] T =M AR [L r (R)L g (G)L b (B)] T

[0087]

[0088] Where R, G, and B are the RGB driving values ​​of the AR display device, and L is the RGB driving value of the AR display device. r(R) is the normalized brightness value generated on the R channel at the location of the maximum brightness of the AR display device when the driving value of the R channel is R. g (G) is the normalized brightness value generated on the G channel at the location of the maximum brightness of the AR display device when the driving value of the G channel is taken as G, L b (B) is the normalized brightness value generated on the B channel at the location of the maximum brightness of the AR display device when the driving value of the B channel is B. These are the tristimulus values ​​at the location of the maximum brightness of the AR display device when the R channel has the maximum drive value and all other channels are 0. These are the tristimulus values ​​at the location of the maximum brightness of the AR display device when the G channel has the maximum drive value and all other channels are 0. X represents the tristimulus value at the location of the maximum brightness of the AR display device when the B channel has the maximum drive value and all other channels are 0. AR Y AR Z AR These are the tristimulus values ​​perceived by the human eye in the field of view of the AR display device; at the same time, since the OST AR display is a transmission device with no display when all channels are 0, there is no dark current interference, so there is no need to consider black spot compensation in the process.

[0089] By setting different RGB driving values ​​R, G, and B, we can obtain the L corresponding to different RGB driving values ​​R, G, and B. r (R), L g (G), L b (B); Simultaneously, an imaging colorimeter was used to measure the tristimulus values ​​generated by the AR display device in the field of view under different RGB driving values ​​R, G, and B, and these values ​​were used as X. AR Y AR Z AR ;

[0090] The L corresponding to different RGB driving values ​​R, G, B r (R), L g (G), L b (B) and the tristimulus value X AR Y AR Z AR Substituting the theoretical relationship into the equations and fitting the undetermined tristimulus values ​​in the theoretical relationship using the least squares method, we obtain the AR color transformation matrix M. AR .

[0091] Furthermore, since the brightness output of AR display devices is typically not linear, the GOG model is used in the data modeling process to describe the brightness response characteristics of AR display devices. A gamma function is used to represent the relationship between normalized brightness and the drive value, such as... Figure 6As shown. Specifically, by setting different RGB driving values ​​R, G, and B, the corresponding L values ​​for different RGB driving values ​​R, G, and B are obtained. r (R), L g (G), L b (B) Specifically as follows:

[0092]

[0093] Where, α r α g α b These represent the gains of the R, G, and B channels of the AR display device, respectively, and C... r C g C b These are the gamma values ​​for the R, G, and B channels of the AR display device, respectively. Both the gain and gamma value can be obtained through measurement data fitting and optimization. max G max B max These are the maximum driving values ​​that the R, G, and B channels of the AR display device can output, respectively.

[0094] Because AR displays exhibit spatial non-uniformity in brightness—meaning there are differences in brightness response at different locations—further optimization is needed to address the brightness distribution at various spatial locations within the display area. Figure 5 As shown, a Gaussian model is used for fitting to accommodate the spatial brightness non-uniformity of the AR display. The expression is as follows:

[0095]

[0096] Among them, L r ′(i,j),L g ′(i,j),L b ′(i,j) represents the corrected normalized luminance value, (i,j) is the spatial pixel coordinate within the display area, L r L g L b For the normalized brightness calculated by the aforementioned GOG model, σ controls the diffusion range of the Gaussian distribution. This can be optimized and adjusted based on the actual measured brightness results by minimizing the error between the measured brightness and the model's predicted brightness. Let L... r ′(i,j),L g ′(i,j),L bSubstituting (i,j) into the theoretical relationship between the RGB driving value and the CIEXYZ color space output, we can obtain the conversion relationship between the tristimulus values ​​of the display color at different spatial locations within the field of view after considering spatial non-uniformity and the single RGB display driving value. This correction process can ensure that the virtual image has a more consistent brightness performance throughout the entire display area, thereby improving the visual quality of AR content and enhancing the consistency of the user's color perception.

[0097] In summary, this invention, after comprehensively considering the influence of ambient light on AR display colors, establishes a mathematical relationship between the virtual display colors that the AR display should output, the colors perceived by the human eye, and ambient light. This relationship can describe how the AR display adjusts its color output under different lighting conditions to compensate for the influence of ambient light on the final perceived color, thereby improving the accuracy and consistency of color reproduction. The target color XYZ corresponds to a given target location. target (i,j), according to XYZ AR (i,j)=[XYZ target (i,j)-(2-t)XYZ env The tristimulus values ​​XYZ for the AR display color at the target location can be obtained by using (i,j)] / t. AR (i,j), then, based on the relationship between the color tristimulus values ​​of OST AR and the display driver established earlier in the dark environment according to the present invention, the characteristic inverse process is performed, that is, the RGB driving value of AR display can be solved by inverse Gaussian transformation and inverse GOG transformation.

[0098] Therefore, this invention achieves high-precision color calibration of OSTAR under different lighting conditions by establishing a camera color characteristic model, an AR display system characteristic model, and an ambient light compensation model. This method fully considers the spatial non-uniformity of OSTAR devices, employs multi-point measurement to acquire color characteristics of different field-of-view areas, and utilizes ambient light measurement and compensation mechanisms to reduce the impact of external lighting on AR displays. Compared to the traditional RGB-CIE XYZ linear mapping method, this invention improves the color consistency of the OSTAR system under complex lighting environments through ambient light weight calculation and a multi-point measurement strategy, providing reliable technical support for high-precision color reproduction in augmented reality systems.

[0099] Of course, the present invention may have other various embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art can make various corresponding changes and modifications according to the present invention, but these corresponding changes and modifications should all fall within the protection scope of the appended claims.

Claims

1. A method for color characterization in an optical perspective augmented reality system, characterized in that, Includes the following steps: S1: The mathematical relationship between the theoretical output tristimulus values ​​of the AR display device, the tristimulus values ​​of color perceived by the target human eye, and the tristimulus values ​​of the ambient light in the field of view is as follows: XYZ AR (i,j)=[XYZ target (i,j)-(2-t)XYZ env (i,j)] / t Among them, XYZ AR (i,j) represents the theoretical output tristimulus value of the AR display device at any position (i,j) within the field of view, XYZ target (i,j) represents the tristimulus values ​​of the target color perceived by the human eye at any position (i,j) within the field of view, XYZ env (i,j) represents the tristimulus value of the pre-calibrated ambient illumination at any position (i,j) within the field of view, and t represents the nonlinear mixing weight. S2: XYZ values ​​at the given target position (i*, j*) target The values ​​of (i*,j*) are used to obtain the XYZ values ​​corresponding to the target position (i*,j*) based on the mathematical relationship. AR (i*,j*); S3: For the XYZ AR (i*,j*) are sequentially subjected to inverse Gaussian transform and inverse GOG transform to obtain the RGB driving values ​​of the AR display device.

2. The color characterization method for an optical perspective augmented reality system as described in claim 1, characterized in that, The tristimulus values ​​XYZ of the ambient illumination at all locations in the field of view env The method to obtain it is as follows: XYZ env =M·RGB env Where M is the camera characterization transformation matrix, RGB env These are the color values ​​of the ambient light image within the camera's field of view, obtained when the camera captures the ambient light within its field of view.

3. The color characterization method for an optical perspective augmented reality system as described in claim 2, characterized in that, The method for obtaining the camera characteristic transformation matrix M is as follows: Under D65 light source illumination, the color values ​​of the original images of the standard color card were captured multiple times by a camera, and the CIE1931 tristimulus values ​​of each color block of the standard color card were measured multiple times by a spectrometer to obtain multiple sets of color values ​​and tristimulus values. The mapping relationship between the color values ​​of the original image used to construct the standard color chart and the CIE1931 tristimulus values ​​is as follows: Where (R,G,B) are the color values ​​on the R, G, and B channels of the original image of the standard color chart, respectively; (X,Y,Z) are the CIE1931 tristimulus values ​​of each color block of the standard color chart; a1~a8, b1~b8, and c1~c8 are the undetermined regression coefficients in the camera characteristic transformation matrix M; and T represents transpose. Substitute each set of color values ​​and tristimulus values ​​into the mapping relationship, and use the least squares method to fit the undetermined regression coefficients in the mapping relationship to obtain the camera characterization transformation matrix M.

4. The color characterization method for an optical perspective augmented reality system as described in claim 1, characterized in that, The nonlinear hybrid weight t is calculated based on the Weber contrast ratio, and the nonlinear hybrid weight t exhibits a Gaussian-like decay relationship with the Weber contrast ratio.

5. A color characterization method for an optical perspective augmented reality system as described in claim 1 or 4, characterized in that, The method for obtaining the nonlinear mixed weight t is as follows: Among them, C W To measure the Weber contrast ratio, which is used to measure the relative brightness difference between AR stimuli and ambient lighting in the field of view, α t To control the adaptability of the nonlinear hybrid weighting to ambient illumination in the field of view, β t γ represents the initial amplitude of the ambient illumination in the field of view. t k is the decay rate of ambient light in the field of view. t This serves as the baseline value for ambient lighting in the field of view.

6. The color characterization method for an optical perspective augmented reality system as described in claim 5, characterized in that, Weber contrast ratio C W The calculation method is as follows: Among them, L AR L represents the brightness of the tristimulus values ​​theoretically output by the AR display device. BG The brightness is the tristimulus value of the ambient light in the field of view.

7. The color characterization method for an optical perspective augmented reality system as described in claim 1, characterized in that, For the XYZ AR The method for performing the inverse Gaussian transform on (i*,j*) is as follows: Among them, XYZ AR-norm (i*,j*) represents the XYZ values ​​after the inverse Gaussian transform. AR (i*,j*), (i0,j0) represent XYZ AR (i*,j*) represents the coordinates of the location where the maximum normalized brightness value generated by the AR display device is located, and σ is the set Gaussian distribution variance.

8. The color characterization method for an optical perspective augmented reality system as described in claim 7, characterized in that, For the XYZ AR-norm The method for performing the GOG inverse transform on (i*,j*) is as follows: [L r L g L b ] T =M AR -1 [X AR-norm Y AR-norm Z AR-norm ] T Among them, X AR-norm Y AR-norm Z AR-norm XYZ respectively AR-norm The three stimulus values ​​M of (i*,j*) AR The AR color transformation matrix describes the conversion relationship between RGB driving values ​​and XYZ stimulus values. r L g L b These are the normalized brightness values ​​for the R, G, and B channels, respectively. R, G, and B are the final RGB driving values ​​for the AR display device, and α is the RGB value for the AR display device. r α g α b These represent the gains of the R, G, and B channels of the AR display device, respectively, and C... r C g C b These are the gamma values ​​of the R, G, and B channels of the AR display device, respectively. max G max B max These are the maximum driving values ​​that the R, G, and B channels of the AR display device can output, respectively. AR color transformation matrix M AR The method for obtaining it is as follows: [X AR Y AR Z AR ] T =M AR [L r (R)L g (G)L b (B)] T Where R, G, and B are the RGB driving values ​​of the AR display device, and L is the RGB driving value of the AR display device. r (R) is the normalized brightness value generated on the R channel at the location of the maximum brightness of the AR display device when the driving value of the R channel is R. g (G) is the normalized brightness value generated on the G channel at the location of the maximum brightness of the AR display device when the driving value of the G channel is taken as G. b (B) is the normalized brightness value generated on the B channel at the location of the maximum brightness of the AR display device when the driving value of the B channel is B. These are the tristimulus values ​​at the location of the maximum brightness of the AR display device when the R channel has the maximum drive value and the other channels are 0. These are the tristimulus values ​​at the location of the maximum brightness of the AR display device when the G channel has the maximum drive value and all other channels are 0. X represents the tristimulus value at the location of the maximum brightness of the AR display device when the B channel has the maximum drive value and all other channels are 0. AR Y AR Z AR These are the tristimulus values ​​perceived by the human eye in the field of view of the AR display device; By setting different RGB driving values ​​R, G, and B, we can obtain the L corresponding to different RGB driving values ​​R, G, and B. r (R), L g (G), L b (B); Simultaneously, an imaging colorimeter was used to measure the tristimulus values ​​generated by the AR display device in the field of view under different RGB driving values ​​R, G, and B, and these values ​​were used as X. AR Y AR Z AR ; The L corresponding to different RGB driving values ​​R, G, B r (R), L g (G), L b (B) and the tristimulus value X AR Y AR Z AR Substituting the theoretical relationship into the equations and fitting the undetermined tristimulus values ​​in the theoretical relationship using the least squares method, we obtain the AR color transformation matrix M. AR .

Images