Optical perspective augmented reality system color characterization method

Abstract

The invention provides a color characterization method for an optical perspective augmented reality system, and the method comprises the steps: carrying out the full-view-field measurement of different environment illumination conditions and OST AR virtual stimulation colors, building a mathematic relation between a virtual display color which should be outputted by AR display, a human eye perception color, and ambient light, and enabling the mathematic relation to describe the color characterization of the virtual display color under different illumination conditions. According to AR display, how to adjust own color output is used for compensating the influence of ambient light on final perception colors, so that the accuracy and consistency of color reproduction are improved; that is to say, the nonlinear influence of the ambient light on the AR virtual display content is corrected through the nonlinear mixed compensation model of the ambient light measurement and the OST AR virtual stimulation color, and the human eye perception color deviation of the OST AR equipment in the complex illumination environment is corrected. The color consistency of OST AR display in an illumination environment and the color rendition capability of an OST AR system under different illumination conditions are improved, and a more natural virtual-real fusion effect is achieved.

Description

Technical Field

[0001] The present invention belongs to the field of augmented reality technology, and in particular relates to a color characterization method for an optical perspective augmented reality system. Background Art

[0002] Augmented reality (AR) technology has seen widespread development in recent years. Optical see-through augmented reality (OSTAR), a key sub-category, enables users to overlay virtual information onto the real world, achieving a fusion of reality and virtuality. Existing methods typically map RGB to CIE XYZ tristimulus values and employ the Gain-Offset-Gamma (GOG) model for color calibration. This method optimizes parameters by measuring the spectrum and XYZ response of the input RGB values. GOG characterization is a modeling approach widely used in display device color calibration. It achieves high-precision color management by mapping the input RGB signals to the output color characteristics. This method relies on three core parameters: gain, offset, and gamma. These parameters are used to adjust the amplification of the RGB channels, correct for black level deviations, and describe the nonlinear relationship between the input signal and output luminance, respectively, ensuring that color reproduction conforms to human visual perception. The GOG characterization method for displays, based on assumptions such as color additivity, channel independence, spatial uniformity, and temporal stability, offers low computational complexity and fast optimization convergence. Consequently, it has been widely adopted in color calibration of self-luminous display devices such as LCDs and OLEDs. However, the GOG characterization method still has obvious limitations in the OST AR environment. First, the GOG method assumes that the display system has channel independence, that is, the optical characteristics of the red, green, and blue channels can be modeled separately. However, due to the optical path design of the optical combiner in OST AR, it often leads to inter-channel coupling effects, making it difficult to accurately describe the actual color output by optimizing the gain and gamma parameters of each channel separately. In addition, the GOG method relies on the assumptions of additive color and spatial uniformity, that is, it assumes that different areas of the same device have the same color response characteristics. In OST AR, due to the angular sensitivity of the optical structure, the color reproduction capabilities of different positions may vary significantly, which makes it difficult for the GOG method based on the global model to meet the fine calibration requirements of OST AR.

[0003] In addition, the measurement and calibration of the traditional GOG method are usually performed in a completely dark environment to avoid interference from ambient light and ensure the accuracy of color calibration. This method is suitable for traditional display devices. However, in augmented reality (AR) display devices, especially optical see-through (OST) AR devices, the color that the user ultimately perceives is the result of the superposition of the device display content and the ambient background light. The ambient background light has a significant impact on the user's final perceived color. The usage scenarios of OST AR devices are complex and changeable. In environments such as strong outdoor light and multiple light sources indoors, the background light may weaken the contrast of the virtual image, resulting in significant deviations in the GOG parameters obtained by calibration in a dark environment in actual use. Therefore, ignoring the influence of ambient light will result in the displayed image color not being able to accurately restore the target color expected by the user, resulting in a deviation in the display effect. In addition, the traditional GOG method assumes that the ambient light is constant and negligible, and cannot directly consider the impact of the dynamic changes in background light on color, requiring an additional compensation mechanism. Therefore, in response to the special needs of OST AR, it is urgent to build a background calibration and nonlinear superposition model, comprehensively consider the transmission characteristics of the OST AR system and the influence of ambient light, and establish a color calibration method that is suitable for different background lighting conditions, so as to improve the color consistency of the augmented reality system and make the fusion of virtual and reality more natural. Summary of the Invention

[0004] To address the above issues, the present invention provides a color characterization method for an optical see-through augmented reality system. This method establishes a mathematical relationship between the target color that the AR display should output and the color perceived by the human eye and ambient light. This relationship can describe how the AR display adjusts its own color output under different lighting conditions to compensate for the impact of ambient light on the final perceived color, thereby improving the accuracy and consistency of color reproduction.

[0005] A color characterization method for an optical see-through augmented reality system comprises the following steps:

[0006] S1: The mathematical relationship between the theoretical output tristimulus values of the AR display device, the target human eye's color tristimulus values, and the tristimulus values of the field of view ambient light is constructed as follows:

[0007] XYZ AR (i,j)=[XYZ target (i,j)-(2-t)XYZ env (i,j)] / t

[0008] Among them, XYZ AR (i, j) is the theoretical output tristimulus value of the AR display device at any position (i, j) in the field of view, XYZ target (i, j) is the tristimulus value of the color perceived by the target human eye at any position (i, j) in the field of view, XYZenv (i, j) is the tristimulus value of the pre-calibrated field of view ambient light at any position (i, j) in the field of view, and t is the nonlinear mixing weight;

[0009] S2: XYZ at a given target position (i*, j*) target The value of (i*, j*) is obtained according to the mathematical relationship to obtain the XYZ corresponding to the target position (i*, j*). AR (i*,j*);

[0010] S3: XYZ AR (i*, j*) performs inverse Gaussian transform and inverse GOG transform in sequence to obtain the RGB driving value of the AR display device.

[0011] Furthermore, the tristimulus values XYZ of the field of view ambient light at all positions env The method to obtain is as follows:

[0012] XYZ env =M·RGB env

[0013] Among them, M is the camera characteristic transformation matrix, RGB env It is the color value of the ambient light image within the camera's field of view obtained when the camera is used to shoot the ambient light within the field of view.

[0014] Furthermore, the camera characterization transformation matrix M is obtained as follows:

[0015] Under D65 light source, a camera is used to capture the color values of the original image of the standard color chart multiple times. At the same time, a spectrometer is used to measure the CIE1931 tristimulus values of each color block of the standard color chart multiple times to obtain multiple sets of color values and tristimulus values.

[0016] The mapping relationship between the color values of the original image and the CIE1931 tristimulus values for constructing the standard color card is as follows:

[0017]

[0018] Where (R, G, B) are the color values of the R, G, and B channels of the original image of the standard color chart, respectively; (X, Y, Z) are the CIE1931 tristimulus values of each color block of the standard color chart; a1-a8, b1-b8, and c1-c8 are the unknown regression coefficients in the camera characterization transformation matrix M; and T represents transpose.

[0019] Each set of color values and tristimulus values is brought into the mapping relationship, and the undetermined regression coefficients in the mapping relationship are fitted using the least squares method to obtain the camera characterization transformation matrix M.

[0020] Furthermore, the nonlinear mixing weight t is calculated according to the Weber contrast, and there is a Gaussian-like attenuation relationship between the nonlinear mixing weight t and the Weber contrast.

[0021] Furthermore, the nonlinear mixing weight t is obtained as follows:

[0022]

[0023] Among them, C W The Weber contrast is used to measure the relative brightness difference between AR stimulus and field of view ambient light. t To control the adaptability of nonlinear mixing weight to the field of view environment lighting, β t is the initial amplitude of the field of view ambient light, γ t is the attenuation rate of the field of view ambient light, k t It is the baseline value of the field of view ambient lighting.

[0024] Furthermore, the Weber contrast C W The calculation method is as follows:

[0025]

[0026] Among them, L AR is the brightness of the tristimulus values theoretically output by the AR display device, L BG is the brightness of the tristimulus values of the field of view ambient light.

[0027] Furthermore, the XYZ AR The method for inverse Gaussian transform of (i*,j*) is:

[0028]

[0029] Among them, XYZ AR-norm (i*,j*) is the XYZ after Gaussian inverse transformation AR (i*,j*), (i0,j0) is XYZ AR The position coordinates of the maximum normalized brightness value generated by the AR display device corresponding to (i*, j*), and σ is the set Gaussian distribution variance.

[0030] Furthermore, the XYZ AR-norm The method for performing inverse GOG transform of (i*,j*) is:

[0031] [L r L g L b ] T =M AR -1 [X AR-norm YAR-norm Z AR-norm ] T

[0032]

[0033] Among them, X AR-norm 、Y AR-norm , Z AR-norm XYZ respectively AR-norm The three stimulus values of (i*,j*), M AR L is the AR color transformation matrix that describes the conversion relationship between RGB driving values and XYZ stimulus values. r , L g , L b They are the normalized brightness values of the R channel, G channel, and B channel respectively. R, G, and B are the final RGB driving values of the AR display device respectively. α r , α g , α b They are the gains of the R channel, G channel, and B channel of the AR display device, respectively. r 、C g 、C b They are the gamma values of the R channel, G channel, and B channel of the AR display device, R max , G max 、B max These are the maximum drive values that can be output by the R channel, G channel, and B channel of the AR display device respectively;

[0034] AR color transformation matrix M AR The method to obtain is:

[0035] [X AR Y AR Z AR ] T =M AR [L r (R)L g (G)L b (B)] T

[0036]

[0037] Among them, R, G, and B are the RGB driving values of the AR display device, L r (R) is the normalized brightness value generated on the R channel at the location of the maximum brightness of the AR display device when the driving value of the R channel is R, L g (G) is the normalized brightness value generated on the G channel at the location where the brightness of the AR display device is at its maximum when the driving value of the G channel is G, L b(B) is the normalized brightness value generated on the B channel at the position where the brightness of the AR display device is at its maximum when the driving value of the B channel is B. The three stimulus values at the location of the maximum brightness of the AR display device when the R channel takes the maximum driving value and the other channels are 0, The three stimulus values at the position where the brightness of the AR display device is at its maximum when the G channel takes the maximum driving value and the other channels are 0, The three stimulus values at the location of the maximum brightness of the AR display device when the B channel takes the maximum driving value and the other channels are 0, X AR 、Y AR , Z AR are the tristimulus values perceived by the human eye in the field of view of the AR display device;

[0038] Set different RGB drive values R, G, B to get the L corresponding to different RGB drive values R, G, B r (R), L g (G), L b (B); At the same time, an imaging colorimeter is used to measure the three stimulus values generated by the AR display device in the field of view under different RGB driving values R, G, B, and use them as X AR 、Y AR , Z AR ;

[0039] The L corresponding to different RGB drive values R, G, B r (R), L g (G), L b (B) and tristimulus value X AR 、Y AR , Z AR Substitute the theoretical relationship and use the least squares method to fit the undetermined three stimulus values in the theoretical relationship to obtain the AR color transformation matrix M AR .

[0040] Beneficial effects:

[0041] 1. The present invention provides a color characterization method for an optical see-through augmented reality system. By performing full-field measurement of different ambient lighting conditions and OST AR virtual stimulus colors, a mathematical relationship is established between the virtual display color that the AR display should output and the color perceived by the human eye and ambient light. This mathematical relationship can describe how the AR display adjusts its own color output under different lighting conditions to compensate for the influence of ambient light on the final perceived color, thereby improving the accuracy and consistency of color reproduction. In other words, the present invention aims to correct the nonlinear influence of ambient light on AR virtual display content through a nonlinear hybrid compensation model of ambient light measurement and OST AR virtual stimulus color, correct the color deviation perceived by the human eye of the OST AR device in complex lighting environments, improve the color consistency of the OST AR display in the lighting environment and the color reproduction capability of the OST AR system under different lighting conditions, and reduce the impact of spatial non-uniformity of the optical see-through augmented reality display device on the characterization accuracy, thereby performing characterization for optical see-through augmented reality under lighting environments and achieving a more natural virtual-reality fusion effect.

[0042] 2. The present invention provides a color characterization method for an optical see-through augmented reality system. Based on a graphical measurement strategy and a global characterization strategy, a high-precision camera and an imaging colorimeter are used to perform global characterization of the ambient light and the spatial position of the OST AR optical system in the entire field of view, respectively. At the same time, an image-based spatial non-uniformity correction method using a high-precision camera and an imaging colorimeter is also combined to reduce the spatial color deviation of the OST AR device caused by changes in viewing angle and optical characteristics, improve the spatial non-uniformity of the system color, and enhance the color rendering consistency and the color reproduction accuracy of the device within a wide field of view. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 The OST AR display characterization process provided by the present invention;

[0044] Figure 2 The camera characterization process provided by the present invention;

[0045] Figure 3 The dark environment OST AR display characterization process provided by the present invention;

[0046] Figure 4 ColorChecker Digital SG standard color card;

[0047] Figure 5 The OST AR brightness Gaussian fitting model provided by the present invention

[0048] Figure 6 The OST AR brightness gamma curve provided by the present invention;

[0049] Figure 7 This is the relationship curve between the AR virtual stimulus mixing weight and Weber contrast provided by the present invention. DETAILED DESCRIPTION

[0050] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0051] In recent years, the widespread application of augmented reality (AR) technology has promoted the development of optical see-through augmented reality (OST AR) display systems. In order to ensure that OSTAR devices can achieve accurate color reproduction under different lighting conditions, it is urgent to establish a color characterization method for their optical characteristics. Since the OST AR system uses an optical combiner to superimpose virtual images on the real environment, the visual signal it finally presents is not only affected by the self-luminous characteristics of the display device, but is also closely related to the transmission and reflection characteristics of the external ambient light. In addition, the primary color channel of the OST AR system may have spatial non-uniformity, that is, the chromaticity and brightness of different field positions may be significantly different. Therefore, the present invention adopts a high-precision camera as a measuring tool, obtains the environmental color data of different spatial positions by an image measurement method, and uses an imaging colorimeter to measure the AR virtual stimulus color data of different spatial positions, thereby reducing the impact of spatial non-uniformity to improve measurement accuracy. At the same time, combined with the ambient light measurement and compensation method, the comprehensive characterization of the OST AR system under the lighting environment is achieved.

[0052] like Figure 1 As shown, a color characterization method for an optical see-through augmented reality system includes the following steps:

[0053] S1: The mathematical relationship between the theoretical output tristimulus values of the AR display device, the target human eye's color tristimulus values, and the tristimulus values of the field of view ambient light is constructed as follows:

[0054] XYZ AR (i,j)=[XYZ target (i,j)-(2-t)XYZ env (i,j)] / t

[0055] Among them, XYZ AR (i, j) is the theoretical output tristimulus value of the AR display device at any position (i, j) in the field of view, XYZ target (i, j) is the tristimulus value of the color perceived by the target human eye at any position (i, j) in the field of view, XYZ env(i, j) is the tristimulus value of the pre-calibrated field of view ambient light at any position (i, j) in the field of view, and t is the nonlinear mixing weight;

[0056] S2: XYZ at a given target position (i*, j*) target The value of (i*, j*) is obtained according to the mathematical relationship to obtain the XYZ corresponding to the target position (i*, j*). AR (i*,j*);

[0057] S3: XYZ AR (i*, j*) performs inverse Gaussian transform and inverse GOG transform in sequence to obtain the RGB driving value of the AR display device.

[0058] So far, the present invention has obtained the display driving value of the intended display color under ambient light at the OST AR target space position.

[0059] It should be noted that the target human eye color tristimulus values (target) of the present invention refer to the tristimulus values that are finally perceived by the human eye. For example, the human eye expects to see a red tristimulus value of (X=100, Y=20, Z=20) at a certain pixel point, then XYZ target (i, j) is (X=100, Y=20, Z=20). However, because what is seen in the AR display device is the result of the superposition of the virtual display content color and the real background color, when driving the AR display device, it is not possible to simply reversely search the tristimulus values corresponding to all the driving values of the AR display device in a dark environment. Based on this, the present invention introduces the modeling of ambient light (env); that is, the characterization process of the present invention is essentially to establish the RGB driving value of the AR display device and the color tristimulus values XYZ that the human eye expects to see. target (i, j) relationship, where XYZ target (i, j) is known, and the user can determine the color they want to display. For example, assuming that the user wants to see a color with a tristimulus value of (X=100, Y=20, Z=20), the RGB driving value of the AR display device can be inferred based on the above mathematical relationship. The RGB driving value of the AR display device makes the tristimulus value XYZ generated by the AR display device AR (i, j) and the tristimulus values XYZ of the field of view ambient light as the background env (i, j) are superimposed, and the color with the tristimulus value (X=100, Y=20, Z=20) that the user expects to see is finally obtained.

[0060] It should be noted that in order to accurately obtain the spatial color distribution of ambient light, the present invention uses a high-precision camera to obtain the color data of the ambient light. First, the measurement camera is color characterized to establish a mapping relationship between the RGB values collected by the camera and the standard CIE1931XYZ color space. This process uses a standard D65 light source and obtains the color data of a standard color card under 0° / 45° measurement geometry. Specifically, the tristimulus values XYZ of the field of view ambient light at all positions are obtained. env The method to obtain is as follows:

[0061] XYZ env =M·RGB env

[0062] Among them, M is the camera characteristic transformation matrix, RGB env It is the color value of the ambient light image within the camera's field of view obtained when the camera is used to shoot the ambient light within the field of view.

[0063] That is to say, in order to accurately characterize the influence of ambient light on color, the present invention uses a camera to shoot the ambient light and obtain the RGB image of the ambient light within the camera field of view. env , the camera characteristic change matrix M is used to convert the RGB value to the CIEXYZ color space to obtain the three stimulus values XYZ of the field of view ambient light env .

[0064] Specifically, such as Figure 2 As shown, the method for obtaining the camera characteristic transformation matrix M is:

[0065] Under the D65 light source, the camera was used to take multiple shots. Figure 4 The color values of the original image of the ColorChecker Digital SG standard color card shown in the figure are measured using a high-precision spectrometer multiple times to measure the CIE1931 tristimulus values of each color block of the standard color card to obtain multiple sets of color values and tristimulus values.

[0066] To improve accuracy, this paper introduces higher-order terms on the basis of standard linear transformation to capture the complex nonlinear relationship between camera RGB and CIE XYZ. Based on this, the present invention constructs the mapping relationship between the color values of the original image of the standard color chart and the CIE1931 tristimulus values as follows:

[0067]

[0068] Where (R, G, B) are the color values of the R, G, and B channels of the original image of the standard color chart, respectively; (X, Y, Z) are the CIE1931 tristimulus values of each color block of the standard color chart. The spectral data of the color chart is directly measured by a spectrometer under the same lighting and observation conditions and calculated according to the CIE1931 standard observer; a1-a8, b1-b8, c1-c8 are the unknown regression coefficients in the camera characterization transformation matrix M, and T represents the transpose.

[0069] Each set of color values and tristimulus values are brought into the mapping relationship, and the undetermined regression coefficients in the mapping relationship are fitted using the least squares method to obtain the camera characterization transformation matrix M, which establishes a mapping relationship from the camera RGB space to the CIEXYZ color space.

[0070] It can be seen that the present invention aims to use the least squares method to fit the unknown parameters in the solution process, find the optimal coefficient matrix, make the predicted XYZ values closest to the actual measured values, and obtain the characterization matrix M; through the transformation matrix M, all camera-captured data can be converted to the standard CIE XYZ color space through this matrix.

[0071] Since the color perceived by the human eye on AR displays is not simply determined by the light emitted by the AR device, but is the result of a nonlinear mixture of AR virtual stimuli and ambient light, its mixing characteristics need to be modeled. This mixing process is affected by the contrast between the AR stimulus and the background light. Contrast plays a particularly critical role in visual perception and can affect the saliency and visibility of colors. In order to quantify the contrast between AR stimuli and background light, Weber contrast is used for calculation. Weber contrast measures the degree of change in the brightness of the displayed target relative to the background brightness. Its expression is as follows:

[0072]

[0073] Among them, L AR is the brightness of the tristimulus values theoretically output by the AR display device, L BG is the brightness of the tristimulus values of the field of view ambient light.

[0074] At the same time, since there is a stable mathematical relationship between the nonlinear mixing weight and Weber contrast between AR virtual stimulus and background light, this relationship can be used to quantify the visual fusion characteristics of AR color appearance. Experimental results show that in different lighting environments, the perception of AR color is significantly affected by the background light, and the degree of its influence can be described by the nonlinear mixing weight t. Based on the fitting analysis of experimental data, the following is obtained: Figure 7 The mixing weight between the AR stimulus and the background light is shown as the Weber contrast CW The mathematical relationship between them, specifically, the method for obtaining the nonlinear mixing weight t is:

[0075]

[0076] Among them, C W Weber contrast is used to measure the relative brightness difference between AR stimulus and ambient light, and α is used to measure the relative brightness difference between AR stimulus and background light. t To control the adaptability of nonlinear mixing weight to the field of view environment lighting, β t is the initial amplitude of the field of view ambient light, γ t is the attenuation rate of the field of view ambient light, k t is the baseline value for field-of-view ambient lighting. The nonlinear blending weight t reflects the contribution of AR color to the final perceived color. This calibrates the OSTAR color characterization model, optimizes the fusion of AR color and ambient light, and improves the color consistency and perceived quality of OSTAR under different lighting conditions.

[0077] About XYZ AR The method for inverse Gaussian transform of (i*,j*) is:

[0078]

[0079] Among them, XYZ AR-norm (i*,j*) is the XYZ after Gaussian inverse transformation AR (i*,j*), (i0,j0) is XYZ AR The position coordinates of the maximum normalized brightness value generated by the AR display device corresponding to (i*, j*), and σ is the set Gaussian distribution variance.

[0080] About XYZ AR-norm The method for performing inverse GOG transform of (i*,j*) is:

[0081] [L r L g L b ] T =M AR -1 [X AR-norm Y AR-norm Z AR-norm ] T

[0082]

[0083] Among them, X AR-norm 、Y AR-norm , Z AR-norm XYZ respectivelyAR-norm The three stimulus values of (i*,j*), M AR L is the AR color transformation matrix that describes the conversion relationship between RGB driving values and XYZ stimulus values. r , L g , L b They are the normalized brightness values of the R channel, G channel, and B channel respectively. R, G, and B are the final RGB driving values of the AR display device respectively. α r , α g , α b They are the gains of the R channel, G channel, and B channel of the AR display device, respectively. r 、C g 、C b They are the gamma values of the R channel, G channel, and B channel of the AR display device, R max , G max 、B max These are the maximum drive values that can be output by the R channel, G channel, and B channel of the AR display device.

[0084] Furthermore, in order to establish the relationship between the virtual stimulus color emitted by the AR display device and the RGB drive value of the AR display device, it is necessary to perform dark field characterization on the OST AR display device in a dark environment without ambient light interference. First, a high-precision imaging colorimeter is used to measure the virtual stimulus emitted by the AR display device, and the brightness and chromaticity data at different RGB drive values within the AR display range are collected to establish the relationship between the RGB input signal and the CIEXYZ color space output; based on this, Figure 3 As shown, the AR color transformation matrix M AR The method to obtain is:

[0085] The theoretical relationship between any input RGB driving value and the CIEXYZ color space output is constructed as follows:

[0086] [X AR Y AR Z AR ] T =M AR [L r (R)L g (G)L b (B)] T

[0087]

[0088] Among them, R, G, and B are the RGB driving values of the AR display device, L r(R) is the normalized brightness value generated on the R channel at the location of the maximum brightness of the AR display device when the driving value of the R channel is R, L g (G) is the normalized brightness value generated on the G channel at the location where the brightness of the AR display device is at its maximum when the driving value of the G channel is G, L b (B) is the normalized brightness value generated on the B channel at the position where the brightness of the AR display device is at its maximum when the driving value of the B channel is B. The three stimulus values at the location of the maximum brightness of the AR display device when the R channel takes the maximum driving value and the other channels are 0, The three stimulus values at the position where the brightness of the AR display device is at its maximum when the G channel takes the maximum driving value and the other channels are 0, The three stimulus values at the location of the maximum brightness of the AR display device when the B channel takes the maximum driving value and the other channels are 0, X AR 、Y AR , Z AR These are the three stimulus values perceived by the human eye in the field of view of the AR display device. At the same time, since the OST AR display is a transmissive device with no display when all channels are 0, there is no dark current interference, so there is no need to consider black point compensation in the process.

[0089] Set different RGB drive values R, G, B to get the L corresponding to different RGB drive values R, G, B r (R), L g (G), L b (B); At the same time, an imaging colorimeter is used to measure the three stimulus values generated by the AR display device in the field of view under different RGB driving values R, G, B, and use them as X AR 、Y AR , Z AR ;

[0090] The L corresponding to different RGB drive values R, G, B r (R), L g (G), L b (B) and tristimulus value X AR 、Y AR , Z AR Substitute the theoretical relationship and use the least squares method to fit the undetermined three stimulus values in the theoretical relationship to obtain the AR color transformation matrix M AR .

[0091] Furthermore, since the brightness output of AR display devices usually does not change linearly, in the data modeling process, the GOG model is used to describe the brightness response characteristics of AR display devices, and the gamma function is used to represent the relationship between normalized brightness and driving value, such as Figure 6Specifically, different RGB driving values R, G, B are set to obtain the L corresponding to different RGB driving values R, G, B. r (R), L g (G), L b (B) are as follows:

[0092]

[0093] Among them, α r , α g , α b They are the gains of the R channel, G channel, and B channel of the AR display device, respectively. r 、C g 、C b They are the gamma values of the R channel, G channel, and B channel of the AR display device. The gain and gamma values can be obtained by fitting and optimizing the measured data. max , G max 、B max These are the maximum drive values that can be output by the R channel, G channel, and B channel of the AR display device.

[0094] Since the brightness of AR display has spatial non-uniformity, that is, the brightness response at different positions is different, in order to further optimize the brightness characteristics of AR display, the brightness distribution of each spatial position within its display range is calculated, such as Figure 5 As shown, a Gaussian model is used for fitting to adapt to the spatial brightness non-uniformity of AR display. The expression is as follows:

[0095]

[0096] Among them, L r ′(i,j), L g ′(i,j), L b ′(i,j) represents the normalized brightness value after correction, (i,j) is the spatial pixel coordinate within the display area, L r , L g , L b is the normalized brightness calculated by the GOG model, and σ controls the diffusion range of the Gaussian distribution. It can be optimized and adjusted based on the actual measured brightness results by minimizing the error between the measured brightness and the model-predicted brightness. r ′(i,j), L g ′(i,j), L bBy substituting ′(i, j) into the theoretical relationship between the RGB drive value and the CIEXYZ color space output, we can obtain the conversion relationship between the display color tristimulus values at different spatial positions within the field of view and the single RGB display drive value after taking spatial non-uniformity into account. This correction process can ensure that the virtual image has a more consistent brightness performance across the entire display area, thereby improving the visual quality of AR content and enhancing the consistency of user color perception.

[0097] In summary, after comprehensively considering the impact of ambient light on AR display color, the present invention establishes a mathematical relationship between the virtual display color that the AR display should output and the color perceived by the human eye and ambient light. This relationship can describe how the AR display adjusts its own color output under different lighting conditions to compensate for the impact of ambient light on the final perceived color, thereby improving the accuracy and consistency of color reproduction. target (i,j), according to XYZ AR (i,j)=[XYZ target (i,j)-(2-t)XYZ env (i, j)] / t can get the display color tristimulus value XYZ of AR at the target position AR (i, j), and then, according to the relationship between the color tristimulus values of OST AR and the display drive in a dark environment established before the present invention, a characterized inverse process is performed, that is, the RGB drive value of the AR display can be solved by inverse Gaussian transform and inverse GOG transform.

[0098] Thus, the present invention achieves high-precision color calibration of OST AR under different lighting conditions by establishing a camera color characterization model, an AR display system characterization model, and an ambient light compensation model. This method fully considers the spatial non-uniformity of the OSTAR device, uses a multi-point measurement method to obtain the color characteristics of different field of view areas, and uses an ambient light measurement and compensation mechanism to reduce the impact of external light on AR display. Compared with the traditional RGB-CIE XYZ linear mapping method, the present invention improves the color consistency of the OST AR system in complex lighting environments through ambient light weight calculation and multi-point measurement strategy, providing reliable technical support for high-precision color reproduction of augmented reality systems.

[0099] Of course, the present invention may have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may of course make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. A color characterization method for an optical see-through augmented reality system, characterized in that: The following steps are involved: S1: The mathematical relationship between the theoretical output tristimulus values of the AR display device, the target human eye's color tristimulus values, and the tristimulus values of the field of view ambient light is constructed as follows: XYZ AR (i,j)=[XYZ target (i,j)-(2-t)XYZ env (i,j)] / t Among them, XYZ AR (i, j) is the theoretical output tristimulus value of the AR display device at any position (i, j) in the field of view, XYZ target (i, j) is the tristimulus value of the color perceived by the target human eye at any position (i, j) in the field of view, XYZ env (i, j) is the tristimulus value of the pre-calibrated field of view ambient light at any position (i, j) in the field of view, and t is the nonlinear mixing weight; S2: XYZ at a given target position (i*, j*) target The value of (i*, j*) is obtained according to the mathematical relationship to obtain the XYZ corresponding to the target position (i*, j*). AR (i*,j*); S3: XYZ AR (i*, j*) performs inverse Gaussian transform and inverse GOG transform in sequence to obtain the RGB driving value of the AR display device.

2. The color characterization method of an optical see-through augmented reality system according to claim 1, wherein: The tristimulus values XYZ of the field of view ambient light at all positions env The method to obtain is as follows: XYZ env =M·RGB env Among them, M is the camera characteristic transformation matrix, RGB env It is the color value of the ambient light image within the camera's field of view obtained when the camera is used to shoot the ambient light within the field of view.

3. The color characterization method of an optical see-through augmented reality system according to claim 2, wherein: The method for obtaining the camera characteristic transformation matrix M is: Under D65 light source, a camera is used to capture the color values of the original image of the standard color chart multiple times. At the same time, a spectrometer is used to measure the CIE1931 tristimulus values of each color block of the standard color chart multiple times to obtain multiple sets of color values and tristimulus values. The mapping relationship between the color values of the original image and the CIE1931 tristimulus values for constructing the standard color card is as follows: Where (R, G, B) are the color values of the R, G, and B channels of the original image of the standard color chart, respectively; (X, Y, Z) are the CIE1931 tristimulus values of each color block of the standard color chart; a1-a8, b1-b8, and c1-c8 are the unknown regression coefficients in the camera characterization transformation matrix M; and T represents transpose. Each set of color values and tristimulus values is brought into the mapping relationship, and the undetermined regression coefficients in the mapping relationship are fitted using the least squares method to obtain the camera characterization transformation matrix M.

4. The color characterization method of an optical see-through augmented reality system according to claim 1, wherein: The nonlinear mixing weight t is calculated according to the Weber contrast, and there is a Gaussian-like attenuation relationship between the nonlinear mixing weight t and the Weber contrast.

5. The color characterization method of an optical see-through augmented reality system according to claim 1 or 4, wherein: The method for obtaining the nonlinear mixing weight t is: Among them, C W The Weber contrast is used to measure the relative brightness difference between AR stimulus and field of view ambient light. t To control the adaptability of nonlinear mixing weight to the field of view environment lighting, β t is the initial amplitude of the field of view ambient light, γ t is the attenuation rate of the field of view ambient light, k t It is the baseline value of the field of view ambient lighting.

6. The color characterization method of an optical see-through augmented reality system according to claim 5, wherein: Weber Contrast C W The calculation method is as follows: Among them, L AR is the brightness of the tristimulus values theoretically output by the AR display device, L BG is the brightness of the tristimulus values of the field of view ambient light.

7. The color characterization method of an optical see-through augmented reality system according to claim 1, wherein: About XYZ AR The method for inverse Gaussian transform of (i*,j*) is: Among them, XYZ AR-norm (i*,j*) is the XYZ after Gaussian inverse transformation AR (i*,j*), (i0,j0) is XYZ AR The position coordinates of the maximum normalized brightness value generated by the AR display device corresponding to (i*, j*), and σ is the set Gaussian distribution variance.

8. The color characterization method of an optical see-through augmented reality system according to claim 7, wherein: About XYZ AR-norm The method for performing inverse GOG transform of (i*,j*) is: [L r L g L b ] T =M AR -1 [X AR-norm Y AR-norm Z AR-norm ] T Among them, X AR-norm 、Y AR-norm , Z AR-norm XYZ respectively AR-norm The three stimulus values of (i*,j*), M AR L is the AR color transformation matrix that describes the conversion relationship between RGB driving values and XYZ stimulus values. r 、L g 、L b They are the normalized brightness values of the R channel, G channel, and B channel respectively. R, G, and B are the final RGB driving values of the AR display device respectively. α r , α g , α b They are the gains of the R channel, G channel, and B channel of the AR display device, respectively. r 、C g 、C b They are the gamma values of the R channel, G channel, and B channel of the AR display device, R max , G max 、B max These are the maximum drive values that can be output by the R channel, G channel, and B channel of the AR display device respectively; AR color transformation matrix M AR The method to obtain is: [X AR Y AR Z AR ] T =M AR [L r (R)L g (G)L b (B)] T Among them, R, G, and B are the RGB driving values of the AR display device, L r (R) is the normalized brightness value generated on the R channel at the location of the maximum brightness of the AR display device when the driving value of the R channel is R, L g (G) is the normalized brightness value generated on the G channel at the location where the brightness of the AR display device is at its maximum when the driving value of the G channel is G, L b (B) is the normalized brightness value generated on the B channel at the position where the brightness of the AR display device is at its maximum when the driving value of the B channel is B. The three stimulus values at the location of the maximum brightness of the AR display device when the R channel takes the maximum driving value and the other channels are 0, The three stimulus values at the position where the brightness of the AR display device is at its maximum when the G channel takes the maximum driving value and the other channels are 0, The three stimulus values at the location of the maximum brightness of the AR display device when the B channel takes the maximum driving value and the other channels are 0, X AR 、Y AR , Z AR are the tristimulus values perceived by the human eye in the field of view of the AR display device; Set different RGB drive values R, G, B to get the L corresponding to different RGB drive values R, G, B r (R), L g (G), L b (B); At the same time, an imaging colorimeter is used to measure the three stimulus values generated by the AR display device in the field of view under different RGB driving values R, G, B, and use them as X AR 、Y AR , Z AR ; The L corresponding to different RGB drive values R, G, B r (R), L g (G), L b (B) and tristimulus value X AR 、Y AR , Z AR Substitute the theoretical relationship and use the least squares method to fit the undetermined three stimulus values in the theoretical relationship to obtain the AR color transformation matrix M AR .

Images