Augmented reality image synthesis method based on optical properties of pearl pigments

Abstract

This invention relates to the field of augmented reality technology and discloses an augmented reality image synthesis method based on the optical properties of pearlescent pigments. The method includes: acquiring spectral reflectance data of pearlescent pigments under different illumination and viewing angles; analyzing the spectral reflectance data to calculate color correlation metrics under multiple angle combinations, thereby determining the delay parameter for signal transmission between angles; using the delay parameter and the distribution characteristics of the spectral reflectance data within a preset time interval to derive a gloss fluctuation index of the pearlescent pigment within the time interval; combining the gloss fluctuation index and the attribute differences of the spectral reflectance data after frequency domain transformation to evaluate the synthesis interference probability at each frequency point and extract key interference frequencies; integrating the distribution patterns of all key interference frequencies to generate an image rendering error probability; and finally adjusting the texture rendering parameters of virtual objects in the augmented reality scene according to the image rendering error probability to achieve the generation and output of the synthesized image.

Description

Technical Field

[0001] This invention relates to the field of augmented reality technology, specifically to an augmented reality image synthesis method based on the optical properties of pearlescent pigments. Background Technology

[0002] In the development of augmented reality (AR) technology, the natural integration of virtual objects with the real environment has always been a key focus within the industry. Among these aspects, the texture rendering effect of virtual objects directly impacts the realism of augmented reality scenes, and this challenge becomes even more pronounced when virtual objects involve pearlescent pigment materials. Pearlescent pigments, with their unique optical properties, exhibit significant color and gloss variations under different lighting conditions and viewing angles. This angle-dependent and dynamic characteristic makes it difficult for traditional image compositing methods to accurately reproduce their visual effects.

[0003] In existing augmented reality image synthesis techniques, texture processing is mostly based on static color information or simple lighting models, often ignoring the dynamic changes in the optical properties of pearlescent pigments. For example, some methods only collect spectral data from a single angle as the basis for texture rendering, resulting in color deviations of virtual objects under different viewing angles; other methods, while considering the angle factor, fail to analyze the distribution characteristics of spectral reflectance data over time, thus failing to capture the gloss fluctuations of pearlescent pigments caused by changes in lighting in dynamic scenes, making the virtual objects in the synthesized images appear stiff and unnatural.

[0004] In the process of integrating virtual and real environments, the optical signals of pearlescent pigments are easily affected by factors such as ambient light interference and equipment acquisition errors. Existing technologies lack precise assessment mechanisms for these interferences. Traditional interference processing methods often employ uniform filtering or correction strategies, failing to address the unique frequency domain differences of pearlescent pigments. This results in problems such as color banding and gloss distortion in the synthesized images. Furthermore, existing methods lack quantitative analysis of the fluctuation patterns of the optical properties of pearlescent pigments when determining texture rendering parameters. Parameter adjustments often rely on empirical values ​​or simple threshold judgments, making it difficult to achieve accurate visual matching between virtual objects and the real environment, thus affecting the overall immersive experience of augmented reality scenes.

[0005] With the widespread application of augmented reality technology in fields such as industrial design, virtual makeup try-on, and cultural relic restoration, the demand for virtual representation of special materials such as pearlescent pigments is increasing. The shortcomings of existing technologies in terms of dynamic optical characteristic capture, interference processing accuracy, and parameter adjustment rationality have become important factors restricting the improvement of augmented reality image synthesis quality. Summary of the Invention

[0006] The purpose of this invention is to provide an augmented reality image synthesis method based on the optical properties of pearlescent pigments, so as to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides an augmented reality image synthesis method based on the optical properties of pearlescent pigments, the method comprising:

[0008] The optical property signals of the pearlescent pigment are collected, including spectral reflectance data under different illumination angles and viewing angles;

[0009] By analyzing the spectral reflectance data, the color correlation metric of the pearlescent pigment under multiple angle combinations is calculated, and the delay parameter of signal transmission between angles is determined based on the color correlation metric.

[0010] By utilizing the distribution characteristics of the delay parameter and spectral reflectance data within a preset time interval, the gloss fluctuation index of pearlescent pigments within the time interval is derived.

[0011] By combining the gloss fluctuation index and the attribute differences of spectral reflectance data after frequency domain transformation, the probability of synthetic interference at each frequency point is evaluated, and key interference frequency points are extracted from the synthetic interference probability.

[0012] Integrate the distribution patterns of all key interference frequencies to generate image rendering error probabilities;

[0013] The texture rendering parameters of virtual objects in the augmented reality scene are adjusted according to the image rendering error probability to achieve the generation and output of the synthesized image.

[0014] Preferably, the calculation of the color correlation measure of pearlescent pigment under multiple angle combinations includes defining the initial estimate and iteration update step size of the color correlation measure, constructing an angle sequence covariance model using the signal values ​​of spectral reflectance data at discrete sampling points, calculating the output value of the covariance model through multiple iterations, and identifying the measurement result corresponding to the peak value of the output value as the final color correlation measure.

[0015] Preferably, the derivation of the gloss fluctuation index of the pearlescent pigment within the time interval includes: dividing the spectral reflectance data into multiple continuous time sub-intervals, comparing the similarity between the amplitude distribution pattern of the pearlescent pigment in each time sub-interval and the amplitude distribution pattern of adjacent time sub-intervals after adjustment of the delay parameter, and directly using the similarity quantification result as the gloss fluctuation index of the corresponding time sub-interval.

[0016] Preferably, the spectral reflectance data is divided into multiple continuous time sub-intervals, and a short time window function is used to transform the spectral reflectance data into a time-frequency domain representation to generate a continuous time sub-interval sequence.

[0017] Preferably, the evaluation of the synthetic interference probability at each frequency point includes selecting any pair of angle combinations, applying a delay parameter to each time sub-interval of the optical characteristic signal of the previous angle combination and using it as a time delay interval, counting the sampling time points of all time sub-intervals of the optical characteristic signal of the next angle combination that overlap with the time delay interval, and aggregating the time sub-intervals to which the sampling time points belong to form a combined interval.

[0018] Based on the amplitude change rate and phase shift rate of the optical characteristic signal of the previous angle combination at the same frequency point in each time sub-interval and the combination interval, calculate the gloss restoration coefficient of each frequency point in each time sub-interval;

[0019] The quotient of the gloss fluctuation index and the gloss recovery coefficient is defined as the scaling factor. The difference between the normalization constant and the scaling factor is accumulated over all time sub-intervals and averaged to obtain the synthetic interference probability for each frequency point in each time sub-interval.

[0020] Preferably, the calculation of the gloss restoration coefficient for each frequency point in each time sub-interval includes calculating the amplitude change rate of the optical characteristic signal time sub-intervals of the combined interval and the previous angle combination at the same frequency point as the amplitude change amount, and calculating the phase shift rate of the optical characteristic signal time sub-intervals of the combined interval and the previous angle combination at the same frequency point as the phase change amount. The gloss restoration coefficient is composed of a linear combination of the amplitude change amount and the phase change amount, and the gloss restoration coefficient is directly proportional to both the amplitude change amount and the phase change amount.

[0021] Preferably, the step of extracting key interference frequencies from the synthetic interference probability includes calculating the synthetic interference probability of each frequency point in all time sub-intervals of the optical characteristic signal for each angle combination, and marking the frequency points whose synthetic interference probability exceeds a preset threshold as key interference frequencies.

[0022] Preferably, the generated image rendering error probability includes collecting all key interference frequency points of the optical characteristic signal for each angle combination, and obtaining the common frequency point set of the key interference frequency points of the optical characteristic signal for all angle combinations as a feature frequency point set;

[0023] Calculate the mean variance of the distribution of the optical characteristic signals of each angle combination and the adjacent angle combinations at all characteristic frequency points, and use it as the rendering interference error of the angle combination.

[0024] The rendering interference error of all angle combinations is normalized, and the normalized error value is defined as the image rendering error probability.

[0025] Preferably, adjusting the texture rendering parameters of virtual objects in the augmented reality scene includes multiplying the sum of the image rendering error probability of each angle combination and the normalization constant by the texture gain value of the rendering engine at the historical rendering time, as the texture gain value of the angle combination at the rendering time.

[0026] The actual texture gain value is obtained by averaging the texture gain value of all angle combinations at the rendering time. Based on the actual texture gain value, the rendering engine is driven to generate control instructions, which are then used to adjust the surface gloss properties of the virtual object and output the composite image.

[0027] Preferably, the method further includes using a spectrometer to collect optical characteristic signals of pearlescent pigments, inputting the optical characteristic signals into an image processing unit to calculate the probability of synthetic interference, and simultaneously feeding back the image rendering error probability to the rendering engine to dynamically optimize texture rendering parameters.

[0028] Compared with the prior art, the beneficial effects of the present invention are:

[0029] This augmented reality image synthesis method based on the optical properties of pearlescent pigments provides comprehensive technical support for enhancing the realism of virtual objects in augmented reality scenes by capturing and analyzing the optical properties of pearlescent pigments from multiple dimensions. In the optical property data acquisition stage, the method focuses on spectral reflectance data under different lighting and viewing angles, breaking through the limitations of traditional methods that rely on single-angle data. This allows for a more complete acquisition of the optical performance of pearlescent pigments in complex environments, laying a solid data foundation for subsequent texture rendering.

[0030] By analyzing spectral reflectance data to calculate color correlation metrics and determine the delay parameters of signal transmission between angles, the method can simultaneously capture the dynamic changes of pearlescent pigments from both angular and temporal dimensions. This dual consideration of angle-dependent and temporal distribution characteristics allows the synthesis process to move beyond static color mapping and accurately reflect the subtle gloss changes of pearlescent pigments as the viewing angle and time progress, resulting in a more realistic visual experience of the texture of virtual objects.

[0031] By utilizing the distribution characteristics of delay parameters and spectral reflectance data, a gloss fluctuation index was derived, further refining the description of the dynamic optical properties of pearlescent pigments. As one of the core elements of the visual effect of pearlescent pigments, the precise quantification of its fluctuation patterns allows virtual objects to exhibit more natural gloss transitions in augmented reality scenes, avoiding the harshness and lack of depth in gloss found in traditional methods, and enhancing the visual harmony between virtual objects and the real environment.

[0032] By combining gloss fluctuation indices with attribute differences after frequency domain transformation to assess the probability of synthetic interference and extracting key interference frequencies, the method demonstrates its targeted and precise approach to interference handling. The optical signals of pearlescent pigments are susceptible to interference from various factors during acquisition and synthesis, and the impact of interference at different frequencies on visual effects varies. By locating key interference frequencies, targeted interference suppression can be achieved during texture rendering parameter adjustment, reducing the loss of effective optical information caused by indiscriminate processing and making the color and gloss in the synthesized image closer to the true representation of pearlescent pigments.

[0033] By integrating key interference frequency distribution patterns to generate image rendering error probabilities, a quantitative basis is provided for adjusting texture rendering parameters. This data-driven parameter adjustment method breaks free from the limitations of traditional experience-based reliance and can dynamically optimize rendering parameters according to the specific changes in the optical properties of pearlescent pigments. This allows the textures of virtual objects to more accurately match the real environment in terms of color saturation, gloss intensity, and angular response. The resulting synthetic images are visually more consistent and realistic, enhancing the overall immersive experience of augmented reality scenes. Attached Figure Description

[0034] Figure 1 This is a schematic diagram illustrating the working principle of the augmented reality image synthesis method based on the optical properties of pearlescent pigments described in this invention.

[0035] Figure 2 A flowchart for calculating color correlation metrics;

[0036] Figure 3 A flowchart illustrating the derivation of the gloss fluctuation index;

[0037] Figure 4 A flowchart for generating the probability of image rendering errors. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0039] Please see Figure 1 This invention provides an augmented reality image synthesis method based on the optical properties of pearlescent pigments, the method comprising:

[0040] First, the optical property signals of the pearlescent pigment are acquired, encompassing spectral reflectance data under different illumination and viewing angles. By analyzing this spectral reflectance data, the color correlation metric of the pearlescent pigment under multiple angle combinations is calculated, and based on this metric, the signal transmission delay parameter between angles is determined. Subsequently, using this delay parameter and the distribution characteristics of the spectral reflectance data within a preset time interval, the gloss fluctuation index of the pearlescent pigment within that time interval is derived. Next, combining the gloss fluctuation index with the attribute differences of the spectral reflectance data after frequency domain transformation, the probability of synthetic interference at each frequency point is evaluated, and key interference frequencies are extracted from these probabilities. Then, the distribution patterns of all key interference frequencies are integrated to generate image rendering error probabilities. Finally, the texture rendering parameters of virtual objects in the augmented reality scene are adjusted according to the image rendering error probabilities, thereby achieving the generation and output of the synthetic image.

[0041] Example 1: See Figure 2 When collecting optical property signals of pearlescent pigments, a spectrometer is used. The spectrometer's operation involves illuminating and capturing reflected light from the pearlescent pigment sample. By adjusting the illumination angle of the light source and the receiving angle of the spectrometer, spectral reflectance data under different combinations of illumination and observation angles can be acquired. The illumination angle can be adjusted to cover multiple discrete angles from 0 to 90 degrees, and the observation angle is correspondingly set to multiple positions forming different angles with the illumination angle, ensuring that complete reflectance signals can be collected under various angle combinations. After each angle adjustment, the spectrometer samples a specific area on the pearlescent pigment surface multiple times to reduce errors caused by ambient light interference. The sampling interval is set according to the optical stability of the pearlescent pigment, typically on the order of milliseconds, thus ensuring the temporal continuity of the data. The collected spectral reflectance data contains intensity information of wavelengths from visible to near-infrared light. This data is stored in digital signal form for easy subsequent processing and analysis.

[0042] After the acquired optical characteristic signals are input into the image processing unit, the color correlation metric of the pearlescent pigment under multiple angle combinations is calculated. First, the initial estimate and iterative update step size are defined. The initial estimate is determined based on the basic composition and common optical properties of the pearlescent pigment. For example, for pearlescent pigments containing mica substrates, their reflectance characteristics are relatively stable at specific angles, and the initial color correlation metric can be set accordingly. The iterative update step size is adjusted according to the data resolution and computational requirements. If the sampling points of the spectral reflectance data are dense, the step size can be set to a smaller value to improve computational accuracy; if the data volume is large, the step size can be appropriately increased to avoid excessive computation.

[0043] An angle sequence covariance model is constructed using the signal values ​​of spectral reflectance data at discrete sampling points. Specifically, the spectral reflectance data for each angle combination is arranged into a sequence in chronological order, with each element in the sequence corresponding to the reflectance intensity value at a specific time. For two different angle combinations, the corresponding two sequences are taken, and the covariance between the sequences is calculated. The covariance calculation involves averaging the products of the deviations of corresponding elements in the two sequences to reflect the correlation between the reflected signals under the two angle combinations. By performing this process on all possible angle combinations, a matrix containing multiple covariance values ​​is constructed, forming the angle sequence covariance model.

[0044] After constructing the model, the output value of the covariance model is calculated iteratively multiple times. In each iteration, the parameters in the model are fine-tuned based on the covariance result obtained in the previous calculation. For example, if the covariance value of a certain combination of angles is found to be abnormally low, it may be due to noise interference in the data. In this case, the outliers in these two sequences can be corrected, and the covariance can be recalculated. The iterative process continues until the difference between the output values ​​of the covariance model obtained from two consecutive calculations is less than a set threshold, at which point the model output is considered to have stabilized.

[0045] The metric corresponding to the peak value in the output is used as the final color correlation metric. In the output of the covariance model after iterative stabilization, the peak value represents the strongest correlation of the reflected signals under two angle combinations, and the corresponding metric value is the color correlation metric for these two angle combinations. By performing this peak identification on all angle combinations, a complete color correlation metric result can be obtained.

[0046] After receiving the optical characteristic signal, the image processing unit calculates the synthesis interference probability according to the steps described above. During the calculation, the image processing unit calls its internal algorithm module to automatically complete operations such as data processing, model construction, and iterative calculation, and stores the intermediate results in real time. Simultaneously, the image processing unit feeds back the generated image rendering error probability to the rendering engine through a data interface. Based on the received error probability, the rendering engine dynamically adjusts the texture rendering parameters. For example, when the error probability is high, it corrects the gloss texture of the virtual object's surface to reduce interference in the synthesized image and ensure the realism of the augmented reality scene. This process forms a closed loop, achieving continuous optimization of augmented reality image synthesis through continuous signal acquisition, processing, feedback, and parameter adjustment.

[0047] Example 2: See Figure 3To derive the gloss fluctuation index of pearlescent pigments within a preset time interval, it is first necessary to divide the spectral reflectance data into time sub-intervals. This division process employs a short-time window function transformation, segmenting the continuous spectral reflectance data by selecting an appropriate window function. The type of window function can be selected based on the characteristics of the spectral reflectance data. For example, when the data contains many high-frequency components, a window function with lower sidelobes can be chosen to reduce the impact of spectral leakage on subsequent analysis. The length of the window function needs to be determined by considering the data sampling rate and the time scale of the gloss changes in the pearlescent pigments. If the gloss changes rapidly, the window function length can be set shorter to capture more subtle temporal variations; if the gloss changes relatively gradually, the window function length can be appropriately increased to reduce computational complexity.

[0048] When applying short-time window function transformation, the window function is slid along the time axis. The step size of each slide can be the same as or a portion of the window function length to achieve continuous coverage of the spectral reflectance data. After each slide, the data within the window function's coverage area is processed, and the time-domain data is converted into a frequency-domain representation using a Fourier transform, thus obtaining the spectral characteristics within that time period. As the window function continues to slide, a series of continuous time-frequency domain representations are obtained, each corresponding to a time sub-interval, thereby generating a continuous sequence of time sub-intervals. The start and end times of each time sub-interval are determined by the position of the window function. Adjacent sub-intervals may overlap, and the overlap ratio is set according to the data continuity requirements to ensure that information in the time dimension is not lost.

[0049] After dividing the spectral reflectance data into time sub-intervals, the gloss fluctuation index for each time sub-interval is calculated. For each time sub-interval, the amplitude distribution pattern of the pearlescent pigment is first analyzed. The amplitude distribution pattern analysis includes statistically analyzing the amplitude range of the spectral reflectance data within the sub-interval, the frequency of amplitude occurrence, the location of peak amplitudes, and the trend of amplitude changes over time. This statistical information clearly describes the intensity distribution characteristics of the pearlescent pigment reflectance signal within that time sub-interval.

[0050] Processing adjacent time sub-intervals. Due to the signal transmission delay between angles, it is necessary to adjust adjacent time sub-intervals using previously determined delay parameters. The delay parameters are applied by shifting adjacent time sub-intervals along the time axis, ensuring that the adjusted adjacent sub-intervals correspond to the current time sub-interval in time, thus eliminating the time difference effect caused by signal transmission delay. After adjustment, the amplitude distribution pattern of these adjacent time sub-intervals is analyzed to obtain their amplitude range, frequency, peak value, and other characteristics.

[0051] Compare the similarity of the amplitude distribution patterns of the current time sub-interval with those of adjacent time sub-intervals adjusted for delay parameters. Similarity comparison can be achieved through various methods, such as calculating the distance between the probability density functions of the amplitude distributions of the two sub-intervals; a smaller distance indicates higher similarity. Alternatively, compare the amplitude statistical characteristics of the two sub-intervals, such as mean, variance, and peak value, and measure the similarity by calculating the degree of difference between these characteristics. Furthermore, correlation analysis can be used to calculate the correlation coefficient between the amplitude sequences of the two sub-intervals; the magnitude of the correlation coefficient directly reflects the degree of similarity between the two.

[0052] The quantification results of the aforementioned similarity are directly used as the gloss fluctuation index for the corresponding time sub-interval. The quantification result can be a specific numerical value, the magnitude of which reflects the degree of gloss fluctuation of the pearlescent pigment within that time sub-interval. High similarity indicates that the gloss characteristics of that time sub-interval change less compared to adjacent sub-intervals, resulting in a lower gloss fluctuation index value; conversely, low similarity indicates greater gloss characteristics change, leading to a higher gloss fluctuation index value. In this way, the gloss fluctuation index for each time sub-interval can be obtained, thus comprehensively reflecting the gloss changes of the pearlescent pigment within the preset time interval.

[0053] It is important to strike a balance between the precision of the time sub-interval division and computational efficiency. Overly fine division of the factor intervals can lead to excessive computation, while overly coarse divisions may result in the loss of crucial gloss variation information. Simultaneously, the application of the delay parameter must be accurate to ensure that adjustments to adjacent time sub-intervals truly reflect the impact of signal transmission delay, thereby guaranteeing the accuracy of the gloss fluctuation index calculation.

[0054] Example 3: To evaluate the probability of synthetic interference at each frequency point, paired angle combinations must first be selected. The selection of angle combinations needs to cover various lighting and observation angle combinations that pearlescent pigments may encounter in real-world scenarios. For example, the lighting angle may be fixed at 45 degrees while the observation angle varies from 10 to 80 degrees, or the observation angle may be kept at 30 degrees while the lighting angle is adjusted within the range of 20 to 70 degrees. This also includes cases where the lighting and observation angles change simultaneously with different gradients. Each pair of angle combinations contains two different angle settings, such as one pair with a lighting angle of 30 degrees and an observation angle of 50 degrees, and another pair with a lighting angle of 40 degrees and an observation angle of 60 degrees. These combinations are used to analyze the mutual influence of optical characteristic signals under different angle conditions.

[0055] For a selected pair of angle combinations, the optical characteristic signal of the previous angle combination is processed. Each time sub-interval of this signal is adjusted according to a predetermined delay parameter to obtain a time delay interval. During adjustment, the time sub-intervals are shifted on the time axis according to the specific value of the delay parameter, thereby simulating the time delay caused by signal transmission between different angles. After adjustment, each time sub-interval results in a time delay interval that is offset from the original time sub-interval in terms of time range. This offset accurately reflects the signal transmission delay effect caused by angle changes.

[0056] The process involves statistically analyzing all sampling time points in the optical characteristic signal of the subsequent angle combination that overlap with the aforementioned time delay interval. Specifically, each time sub-interval of the subsequent angle combination is compared one by one with the adjusted time delay interval of the preceding angle combination to determine their overlapping portion on the time axis. Then, all sampling time points contained within this overlapping portion are extracted. Next, the time sub-intervals of the subsequent angle combination to which these sampling time points belong are aggregated to form a combined interval. The combined interval encompasses all signal segments in the subsequent angle combination that overlap temporally with the time delay interval of the preceding angle combination, thus fully reflecting the temporal correlation between the two angle combination signals.

[0057] Based on the optical characteristic signal of the previous angle combination, the gloss restoration coefficient at each frequency point in each time sub-interval and the combined interval is calculated. Before calculation, the same frequency points are determined. These frequency points are obtained by frequency domain transformation of the spectral reflectance data, covering the main frequency components of the pearlescent pigment reflectance signal, and are uniformly distributed from low to high frequencies. For each frequency point, the amplitude change rate and phase shift rate of the optical characteristic signal time sub-interval of the combined interval and the previous angle combination are calculated respectively.

[0058] The amplitude change rate is the ratio of the amplitude of the combined interval at a given frequency to the amplitude of the previous angular combined time sub-interval at the same frequency. It reflects the relative change in amplitude between the two intervals at that frequency and is used as the amplitude change quantity. The phase shift rate is the difference between the phase of the combined interval at a given frequency and the phase of the previous angular combined time sub-interval at the same frequency. The unit can be angle or radians. It reflects the degree of phase shift between the two intervals at that frequency and is used as the phase change quantity. The gloss restoration coefficient is formed by a linear combination of the amplitude change and the phase change, and can be expressed as:

[0059]

[0060] in, Indicates the gloss recovery coefficient. It represents the amplitude change (i.e., the rate of change of amplitude of the optical characteristic signal time sub-interval of the combination interval and the previous angle at the same frequency point). It represents the phase change (i.e., the phase shift rate of the optical characteristic signal time sub-interval of the combined interval and the previous angle combination at the same frequency point). and All are weighting coefficients (positive numbers used to adjust the proportion of amplitude change and phase change in the gloss restoration coefficient).

[0061] After obtaining the gloss recovery coefficient, the quotient of the gloss fluctuation index and the gloss recovery coefficient is defined as the scaling factor. The gloss fluctuation index is the quantified result of gloss fluctuation for each time sub-interval derived previously, reflecting the degree of gloss instability within that time sub-interval. The scaling factor is positively correlated with the gloss fluctuation index, i.e., the greater the gloss fluctuation, the larger the scaling factor; and negatively correlated with the gloss recovery coefficient, i.e., the weaker the gloss recovery ability, the larger the scaling factor.

[0062] A normalization constant is selected, with a value between 0 and 1, to control the subsequent calculation results within a specific range, facilitating the comparison of synthetic interference probabilities across different time sub-intervals and frequency points. The difference between the normalization constant and the scaling factor is calculated, and then this difference is accumulated over all time points within the time sub-interval, divided by the number of time points. The average value obtained is the synthetic interference probability at each frequency point in the time sub-interval.

[0063] For each time sub-interval and frequency point of the previous angle combination, the above steps are repeated to calculate the combined interference probability for each frequency point in each time sub-interval. These probability values ​​clearly reflect the degree of signal interference caused by angle changes at different time segments and frequencies, providing detailed data for subsequent extraction of key interference frequencies. The entire calculation process involves a large number of time sub-intervals and frequency points, requiring automated algorithms to improve processing efficiency and accuracy, ensuring reliable results within a reasonable timeframe. During processing, real-time data verification is crucial to avoid deviations in calculation results due to data anomalies, while ensuring the rigorous calculation logic of each step conforms to the overall method requirements.

[0064] Example 4: See Figure 4To extract key interference frequencies from synthetic interference probabilities, a comprehensive analysis of the optical characteristic signals for each angle combination is required. Angle combinations are composed of both illumination and observation angles; for example, an illumination angle of 15° and an observation angle of 40°, or an illumination angle of 50° and an observation angle of 25°, etc. These angle combinations cover various lighting and observation position combinations that may occur in practical applications. The optical characteristic signals for each angle combination have been divided into several consecutive time sub-intervals, each containing synthetic interference probability data for multiple frequencies. During processing, all time sub-intervals under each angle combination must be examined one by one, and then the corresponding synthetic interference probability value must be extracted for each frequency point within each time sub-interval.

[0065] After obtaining all the synthetic interference probability values, a preset threshold needs to be set. The determination of this threshold depends on the optical performance of the pearlescent pigment in different scenarios and the acceptable level of interference during augmented reality image synthesis. For example, in scenarios where image detail requirements are not high, the threshold can be set slightly higher; while in scenarios where image accuracy is critical, the threshold needs to be set lower. When the synthetic interference probability of a certain frequency point exceeds this preset threshold in any time sub-interval, that frequency point will be marked as a critical interference frequency point. For example, if the preset threshold is 0.55, and the synthetic interference probability of a certain frequency point in a certain time sub-interval is 0.58, then that frequency point will be marked as a critical interference frequency point.

[0066] When generating the probability of image rendering errors, the first step is to collect all the key interference frequencies for each angle combination. For example, the key interference frequencies for angle combination 1 (10° illumination, 20° observation) are f2, f4, and f6; for angle combination 2 (20° illumination, 30° observation) they are f4, f6, and f8; for angle combination 3 (30° illumination, 40° observation) they are f3, f4, and f6; and for angle combination 4 (40° illumination, 50° observation) they are f4, f6, and f9. These frequencies need to be recorded separately.

[0067] From the key interference frequencies of all angle combinations, identify the common frequencies to form a characteristic frequency set. In the example above, f4 and f6 are marked as key interference frequencies in all four angle combinations, so the characteristic frequency set is {f4, f6}.

[0068] Calculate the rendering interference error for each angle combination. For each angle combination, it is necessary to consider the signal amplitude distribution of its own optical characteristic signals and those of its adjacent angle combinations at characteristic frequency points. Adjacent angle combinations refer to combinations where the illumination angle or observation angle differs little from the current angle combination. For example, the adjacent combinations of angle combination 1 (illumination 10°, observation 20°) might be illumination 15°, observation 20°, or illumination 10°, observation 25°, etc. For each characteristic frequency point, calculate the variance of the signal amplitude distribution of this angle combination and its adjacent angle combinations at that frequency point, and then average the variances of all characteristic frequency points. The result is the rendering interference error of this angle combination. For example, the amplitude distribution variance of angle combination 1 is 0.25 at f4, 0.35 at f6, and the average value is 0.3, so its rendering interference error is 0.3.

[0069] The rendering interference error for all angle combinations is normalized. Normalization involves dividing the rendering interference error for each angle combination by the maximum value among all angle combination rendering interference errors, ensuring the processed error value is between 0 and 1. For example, if the maximum rendering interference error among all angle combinations is 0.6, and the rendering interference error for a certain angle combination is 0.3, then the normalized value is 0.5, which represents the image rendering error probability corresponding to that angle combination. Below is an example table of key interference frequencies and feature frequency sets for a certain set of angle combinations.

[0070] Table 1: Example table of key interference frequencies and characteristic frequency sets for a certain angle combination.

[0071]

[0072] This process allows for the systematic extraction of key interference frequencies from the synthesized interference probability, and further generates image rendering error probabilities. During processing, it's crucial to maintain consistency in the treatment of each angle combination, each time sub-interval, and each frequency point to avoid deviations due to omissions or misjudgments. The accuracy of the feature frequency set is paramount; errors in common frequency point selection directly impact the calculation of rendering interference errors, thus affecting the final image synthesis effect. Therefore, when selecting common frequencies, it's essential to carefully verify the key interference frequencies for each angle combination to ensure the feature frequency set accurately reflects the common interference characteristics of all angle combinations. Simultaneously, the calculation of rendering interference errors and normalization processes must strictly adhere to predetermined steps to ensure that the results at each stage accurately reflect the actual interference situation.

[0073] Example 5: Adjusting the texture rendering parameters of virtual objects in an augmented reality scene requires starting with the image rendering error probability for each angle combination. Angle combinations encompass different combinations of lighting and viewing angles, such as a 10° lighting angle and a 30° viewing angle, or a 20° lighting angle and a 45° viewing angle. These combinations cover various viewing conditions under which virtual objects might be observed in an augmented reality scene. For each angle combination, its corresponding image rendering error probability is added to a normalization constant. The value of the normalization constant needs to be determined based on the overall visual style of the augmented reality scene. If the scene leans towards a bright style, a larger value can be chosen for the constant; if the scene leans towards a dark style, a smaller value should be chosen for the constant to ensure that the subsequently calculated texture gain value meets the scene's basic brightness requirements.

[0074] The sum of the above values ​​is multiplied by the texture gain values ​​from the rendering engine's historical rendering times to obtain the texture gain value of that angle combination at the current rendering time. The rendering engine's historical rendering times refer to the most recent one or more rendering times. Historical texture gain values ​​record the parameter values ​​used to adjust the texture of virtual objects at these times. By introducing historical data, the current texture gain value can maintain a certain consistency with previous rendering effects, avoiding abrupt changes in the texture of virtual objects at different times. For example, if the image rendering error probability of a certain angle combination is 0.3, the normalization constant is 0.5, and the historical texture gain value is 1.2, then the current texture gain value of that angle combination is (0.3 + 0.5) × 1.2, and the calculated result is the texture gain value of that angle combination at the current rendering time.

[0075] After obtaining the texture gain values ​​for all angle combinations at the current rendering moment, the average of these values ​​is calculated to obtain the actual texture gain value. When calculating the average, the texture gain values ​​for each angle combination are summed and then divided by the total number of angle combinations. This process balances the impact of different angle combinations on the final texture effect, preventing extreme texture gain values ​​from a single angle combination from distorting the overall rendering effect. For example, if there are 5 angle combinations with current texture gain values ​​of 0.9, 1.1, 1.0, 0.8, and 1.2, the actual texture gain value is (0.9 + 1.1 + 1.0 + 0.8 + 1.2) divided by 5.

[0076] The rendering engine generates control instructions based on actual texture gain values. Internally, the rendering engine contains multiple modules for adjusting the surface properties of virtual objects, and the control instructions specify the operations each module needs to perform. For example, if the actual texture gain value is high, the control instructions might instruct the gloss adjustment module to enhance the reflectivity of the virtual object's surface, making the texture appear brighter; if the actual texture gain value is low, the control instructions might instruct the module to reduce the reflectivity, making the texture appear softer. The control instructions also involve adjusting parameters such as texture detail and color saturation to ensure that the virtual object's texture exhibits optical properties similar to real pearlescent pigments from different angles.

[0077] These control commands are used to adjust the surface gloss properties of virtual objects and output composite images. During the adjustment process, the rendering engine updates the texture data of the virtual object in real time, applying the new gloss attribute parameters to the surface of the 3D model. For example, for areas on the virtual object coated with pearlescent pigment, the control commands change the reflectivity and intensity of these areas under different lighting and viewing angles, creating a visually flowing gloss effect that changes with the viewing angle. After adjustment, the rendering engine merges the processed virtual object with an image of the real scene to generate the final augmented reality composite image, which is then output through a display device.

[0078] Throughout the process, it is crucial to ensure that the calculated texture gain value accurately reflects the probability of image rendering errors for each angle combination, while also maintaining the consistency of historical rendering effects. The generation of control commands must match the hardware performance of the rendering engine to avoid rendering delays caused by overly complex commands. Furthermore, the adjustment of the gloss properties of virtual object surfaces must adapt to the lighting conditions of the real scene. For example, in bright light environments, the gloss of virtual objects should be enhanced to match the ambient brightness; in low light environments, the gloss should be reduced to maintain image harmony.

[0079] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0080] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An augmented reality image synthesis method based on the optical properties of pearlescent pigments, characterized in that, This method performs the following operation: acquiring optical property signals of pearlescent pigments, wherein the optical property signals include: Spectral reflectance data under different illumination angles and viewing angles; By analyzing the spectral reflectance data, the color correlation metric of the pearlescent pigment under multiple angle combinations is calculated, and the delay parameter of signal transmission between angles is determined based on the color correlation metric. By utilizing the distribution characteristics of the delay parameter and spectral reflectance data within a preset time interval, the gloss fluctuation index of pearlescent pigments within the time interval is derived. By combining the gloss fluctuation index and the attribute differences of spectral reflectance data after frequency domain transformation, the probability of synthetic interference at each frequency point is evaluated, and key interference frequency points are extracted from the synthetic interference probability. Integrate the distribution patterns of all key interference frequencies to generate image rendering error probabilities; The texture rendering parameters of virtual objects in the augmented reality scene are adjusted according to the image rendering error probability to achieve the generation and output of the synthesized image; The assessment of the probability of synthetic interference at each frequency point includes: Select any pair of angle combinations, apply a delay parameter to each time sub-interval of the optical characteristic signal of the previous angle combination and use it as a time delay interval, count the sampling time points of all time sub-intervals of the optical characteristic signal of the next angle combination that overlap with the time delay interval, and aggregate the time sub-intervals to which the sampling time points belong to form a combined interval. Based on the amplitude change rate and phase shift rate of the optical characteristic signal of the previous angle combination at the same frequency point in each time sub-interval and the combination interval, calculate the gloss restoration coefficient of each frequency point in each time sub-interval; The quotient of the gloss fluctuation index and the gloss recovery coefficient is defined as the scaling factor. The difference between the normalization constant and the scaling factor is accumulated over all time sub-intervals and averaged to obtain the synthetic interference probability for each frequency point in each time sub-interval. The calculation of the gloss restoration coefficient for each frequency point in each time sub-interval includes: The amplitude change rate of the optical characteristic signal time sub-interval of the combined interval and the previous angle combination at the same frequency point is calculated as the amplitude change amount. The phase shift rate of the optical characteristic signal time sub-interval of the combined interval and the previous angle combination at the same frequency point is calculated as the phase change amount. The gloss restoration coefficient is composed of a linear combination of the amplitude change amount and the phase change amount. The gloss restoration coefficient is directly proportional to both the amplitude change amount and the phase change amount.

2. The augmented reality image synthesis method based on the optical properties of pearlescent pigments as described in claim 1, characterized in that, The calculation of the color correlation measure of pearlescent pigments under multiple angle combinations includes: Define the initial estimate and iteration update step size for the color correlation metric. Construct an angle sequence covariance model using the signal values ​​of spectral reflectance data at discrete sampling points. Calculate the output value of the covariance model through multiple iterations and identify the metric result corresponding to the peak value of the output value as the final color correlation metric.

3. The augmented reality image synthesis method based on the optical properties of pearlescent pigments as described in claim 1, characterized in that, The derived gloss fluctuation index of the pearlescent pigment within the time interval includes: The spectral reflectance data is divided into multiple continuous time sub-intervals. The similarity between the amplitude distribution pattern of the pearlescent pigment in each time sub-interval and the amplitude distribution pattern of the adjacent time sub-interval after adjustment of the delay parameter is compared. The quantification result of the similarity is directly used as the gloss fluctuation index of the corresponding time sub-interval.

4. The augmented reality image synthesis method based on the optical properties of pearlescent pigments as described in claim 3, characterized in that, The spectral reflectance data is divided into multiple continuous time sub-intervals. A short time window function is used to transform the spectral reflectance data into a time-frequency domain representation to generate a continuous time sub-interval sequence.

5. The augmented reality image synthesis method based on the optical properties of pearlescent pigments as described in claim 1, characterized in that, The extraction of key interference frequency points from the synthetic interference probability includes: The synthetic interference probability of each frequency point in all time sub-intervals of the optical characteristic signal for each angle combination is calculated, and the frequency points where the synthetic interference probability exceeds a preset threshold are marked as key interference frequency points.

6. The augmented reality image synthesis method based on the optical properties of pearlescent pigments as described in claim 1, characterized in that, The generated image rendering error probability includes: Collect all key interference frequency points of the optical characteristic signal for each angle combination, and obtain the common frequency point set of the key interference frequency points of the optical characteristic signal for all angle combinations as the feature frequency point set; Calculate the mean variance of the distribution of the optical characteristic signals of each angle combination and the adjacent angle combinations at all characteristic frequency points, and use it as the rendering interference error of the angle combination. The rendering interference error of all angle combinations is normalized, and the normalized error value is defined as the image rendering error probability.

7. The augmented reality image synthesis method based on the optical properties of pearlescent pigments as described in claim 1, characterized in that, The adjustment of texture rendering parameters for virtual objects in augmented reality scenes includes: The sum of the image rendering error probability of each angle combination and the normalization constant is multiplied by the texture gain value of the rendering engine at the historical rendering time, and this is taken as the texture gain value of the angle combination at the rendering time. The actual texture gain value is obtained by averaging the texture gain value of all angle combinations at the rendering time. Based on the actual texture gain value, the rendering engine is driven to generate control instructions, which are then used to adjust the surface gloss properties of the virtual object and output the composite image.

8. The augmented reality image synthesis method based on the optical properties of pearlescent pigments as described in claim 1, characterized in that, The method further includes: The optical property signals of pearlescent pigments are collected using a spectrometer and input into the image processing unit to calculate the probability of synthetic interference. At the same time, the probability of image rendering error is fed back to the rendering engine to dynamically optimize the texture rendering parameters.

Images