Vision assessment method and system based on augmented reality

Abstract

The invention provides a vision assessment method and system based on augmented reality, and relates to the technical field of vision assessment, and the method comprises the steps: obtaining an environment correction parameter based on an environment parameter; generating visual stimulation according to the vision evaluation task, the environment correction parameter and the initial visual stimulation parameter; s103, obtaining response feedback of the user to the current visual stimulation, and performing feature extraction on the response feedback to obtain a feature vector; s104, inputting the feature vector into a pre-trained adjustment model to obtain a visual stimulation parameter adjustment amount, and adjusting the visual stimulation parameter based on the visual stimulation parameter adjustment amount to obtain an updated visual stimulation parameter; s105, generating updated visual stimulation according to the vision evaluation task, the updated visual stimulation parameters and the environment correction parameters; the steps S103 to S105 are iteratively executed until a preset evaluation termination condition is met; and inputting the structured evaluation sequence data into a pre-trained evaluation model to generate a vision evaluation result of the user.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vision assessment, in particular to a vision assessment method and system based on augmented reality. BACKGROUND

[0002] In the field of eye health, vision assessment is a key basis for ophthalmic disease diagnosis and vision correction scheme formulation. Traditional assessment methods have obvious limitations; for example, they are greatly affected by environmental factors, different lighting conditions, contrast, etc. can interfere with the accuracy of vision detection results; they rely on patient subjective feedback and detection personnel experience judgment, lack of precise quantitative analysis; and the detection process is relatively fixed, and it is difficult to flexibly adjust according to individual visual characteristics.

[0003] With the development of science and technology, augmented reality (AR) technology has been gradually applied in the medical field. AR technology can integrate virtual information with real environment, which can effectively reduce environmental interference; at the same time, it can present diversified and dynamic visual stimuli, and realize more comprehensive and accurate vision assessment. SUMMARY

[0004] In order to solve the above technical problems, the present application provides a vision assessment method and system based on augmented reality, which can eliminate environmental interference, accurately match user visual characteristics, and realize objective and quantitative vision assessment, and provide more reliable basis for ophthalmic clinical diagnosis.

[0005] The first aspect of the embodiment of the present application provides a vision assessment method based on augmented reality, comprising: S1: obtaining an environment parameter of an environment where the augmented reality device is located, and calculating an environment correction parameter based on the environment parameter; S2: generating visual stimuli and presenting to the user according to a preset vision assessment task, the environment correction parameter and a preset initial visual stimulus parameter; S3: iteratively performing steps S31-S33 until a preset evaluation termination condition is met, wherein: S31: obtaining a response feedback of the user to the current visual stimuli, extracting features of the response feedback to obtain a feature vector representing the response characteristics of the user; S32: inputting the feature vector into a pre-trained adjustment model to obtain a visual stimulus parameter adjustment amount, adjusting the visual stimulus parameter based on the visual stimulus parameter adjustment amount to obtain updated visual stimulus parameters; S33: generating updated visual stimuli and presenting to the user according to the preset vision assessment task, the updated visual stimulus parameters and the environment correction parameter; S4: inputting the structured assessment sequence data into the pre-trained assessment model to generate the vision assessment result of the user; the structured assessment sequence data comprises visual stimulation parameters presented in each iteration, corresponding response feedback of the user and a corresponding feature vector.

[0006] Preferably, the environmental parameters comprise environmental optical parameters and spatial parameters. The environmental correction parameters are generated based on the environmental parameters, comprising: The environmental optical parameters are processed to obtain a brightness compensation coefficient and a color compensation coefficient. A projection transformation matrix of a display plane of the augmented reality device to an ideal plane of the user's visual axis is calculated based on the spatial parameters. The brightness compensation coefficient, the color compensation coefficient and the projection transformation matrix are taken as the environmental correction parameters.

[0007] In this embodiment, the core physical factors of visual presentation are adjusted using environmental optical parameters and spatial parameters, which is more robust than the traditional method which only relies on fixed lighting conditions. Secondly, the device display plane is mapped to the ideal plane of the user's visual axis through the projection transformation matrix, so that the visual stimulus is correctly presented in the three-dimensional space, avoiding test errors caused by device wearing position deviation. Through brightness compensation, color compensation and spatial position compensation, this embodiment suppresses environmental interference, so that this method can be adapted to complex scenes such as home and outdoor use.

[0008] Preferably, the environmental optical parameters comprise environmental light intensity and environmental color temperature. The environmental optical parameters are processed to obtain a brightness compensation coefficient and a color compensation coefficient, comprising: The brightness compensation coefficient is calculated based on the ratio of the environmental light intensity to a preset reference environmental light intensity. The color compensation coefficient is calculated based on the difference between the environmental color temperature and a preset reference color temperature.

[0009] In this embodiment, the compensation coefficient is calculated based on the ratio of the environmental light intensity to the reference value, which quantifies the influence of environmental brightness on visual stimulation, so that the display brightness of the device remains consistent under different lighting conditions, avoiding test deviation caused by excessive brightness or darkness. Secondly, the compensation coefficient is calculated based on the difference between the environmental color temperature and the reference color temperature, which solves the color perception deviation problem in high or low color temperature environments and improves the accuracy of the color of the presented visual stimulus. This embodiment uses a standardized ratio and difference value calculation method, which is convenient for uniform calibration between different devices and improves the compatibility of this method.

[0010] Preferably, the color compensation coefficient is calculated based on the difference between the environmental color temperature and a preset reference color temperature, comprising: Determine a corresponding basic color compensation factor calculation mode based on a preset interval in which the ambient color temperature is located; Calculate a basic color compensation factor according to the basic color compensation factor calculation mode and a difference between the ambient color temperature and a preset reference color temperature; Correct the basic color compensation factor based on ambient light intensity to obtain a color compensation coefficient.

[0011] In this embodiment, the compensation factor calculation mode is selected according to the interval in which the ambient color temperature is located, different compensation strategies can be formulated based on different color temperature ranges, and the compensation accuracy is improved. The ambient light intensity correction term is introduced into the basic color compensation factor to solve the problem of color perception threshold change in high brightness environment, so as to realize more accurate color restoration. Through segmented calculation and dynamic correction, the limitations of the traditional linear compensation method under extreme color temperature or light intensity are overcome, and the color consistency in complex environment is improved.

[0012] Preferably, the method comprises: According to the relative position and posture of the augmented reality device and the user's eyes, a rigid transformation model of the device display coordinate system to the user's visual axis ideal coordinate system is established; The parameters of the rigid transformation model are solved by the least square method to generate the projection transformation matrix.

[0013] In this embodiment, the rigid transformation model is established based on the relative position and posture of the device and the user's eyes, which describes the position relationship in three-dimensional space and provides a basis for spatial positioning of visual stimulation. The transformation model parameters are solved by the least square method, which can eliminate the device posture measurement error to improve the accuracy of the projection transformation matrix and also improve the spatial accuracy of the test. This embodiment supports real-time updating of device position and posture data during the test process and dynamic adjustment of the projection transformation matrix to solve the measurement error caused by changes in the display plane due to head movement or device displacement.

[0014] Preferably, the response feedback includes: recognition result data input by the user through the interactive interface of the augmented reality device, timestamp sequence data of the user from visual stimulation presentation to completion of the response operation, and interactive behavior data of the user during the operation; The feature extraction of the response feedback obtains a feature vector representing the user response characteristics, which comprises: Calculate a response correctness index based on the recognition result data; Calculate a response time index based on the timestamp sequence data; The interactive behavior data is extracted to obtain user operation features; The response correctness index, the response time index and the user operation are fused to obtain a feature vector representing a user response characteristic.

[0015] In this embodiment, the comprehensive identification result, the reaction time and the interaction behavior data are combined to construct a multi-dimensional feature vector, which can simultaneously evaluate the visual acuity, reaction speed and attention state of the user, and improve the comprehensiveness of the evaluation. The user operation features are extracted from the interaction data such as head posture and eye movement trajectory, which can detect potential visual fatigue or attention distraction, and provide behavioral evidence for the test results. The dynamic generation mechanism of the feature vector can meet the differentiated evaluation needs of different test tasks.

[0016] Preferably, when the interaction behavior data is head posture change data, the extracting the interaction behavior data to obtain user operation features comprises: Based on the head posture change data, a change trajectory of an offset angle of the head relative to a preset reference posture during the user response is calculated; Features of the change trajectory of the offset angle are extracted to obtain head movement features as the user operation features; and wherein The head movement features include an average change rate of the offset angle and a smoothness index of the change trajectory of the offset angle.

[0017] In this embodiment, the average rate and the smoothness index of the change trajectory of the offset angle represent the head stability during the user response, which can identify the head shaking caused by fatigue or lack of attention, and improve the reliability of the test results. The head movement features of the user are related to the visual cognitive process, for example, fast and smooth head movement can be considered as good visual tracking ability of the user, and violent shaking can imply visual blur or cognitive difficulty of the user. Secondly, the head movement features of this embodiment can be fed back to the adjustment model in real time, and the adjustment model adjusts the visual stimulation parameters according to the head movement features, so that the visual stimulation is more accurate.

[0018] Preferably, the smoothness index in the extracting the features of the change trajectory of the offset angle comprises: Based on the angle values of the continuous sampling points in the change trajectory of the offset angle, the angle change amount between adjacent sampling points is calculated; According to the angle change amounts of all adjacent sampling points in a preset time window, a standard deviation of the angle change amounts is calculated; The standard deviation is taken as the smoothness index of the change trajectory of the offset angle.

[0019] In the embodiment, the trajectory smoothness is represented by the standard deviation of the angle change amount, which is used to provide an objective motion stability evaluation index and avoid subjective judgment errors. The standard deviation is calculated in a preset time window, which can capture short-term fluctuations of motion characteristics and effectively identify the influence of sudden interference on the evaluation test. In addition, the abnormal value of the smoothness index of the embodiment can trigger an alarm to prompt the abnormality of the test environment or the user state, so as to facilitate timely intervention or retesting.

[0020] Preferably, when the interaction behavior data is eye tracking data, the extracting the interaction behavior data to obtain the user operation feature further includes: Based on the eye tracking data, calculating the duration from the moment when the visual stimulus is presented to the moment when the user first meets the preset gaze condition in the preset interest region as a gaze latency; Calculating the duration of the first gaze meeting the gaze condition in the preset interest region; Taking the gaze latency and the duration of the first gaze as user features.

[0021] In the embodiment, the gaze latency and the duration of the first gaze are used to represent the reaction speed and attention concentration of the user to the visual stimulus, which provides a basis for evaluating the visual search efficiency and cognitive load. Based on the gaze data in the preset interest region, it can be judged whether the user correctly identifies the stimulus content, and the result deviation caused by head compensation or random guessing can be avoided.

[0022] The second aspect of the embodiment of the present application provides an augmented reality-based vision evaluation system, including a memory, a processor, and a computer program stored in the memory and running on the processor, and the processor executes the computer program to realize the steps of the above-mentioned augmented reality-based vision evaluation method.

[0023] The third aspect of the embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the steps of the above-mentioned augmented reality-based vision evaluation system.

[0024] The vision assessment method and system based on augmented reality provided by the embodiment of the application have the beneficial effects that: the application can eliminate environmental interference such as light and contrast by acquiring environmental parameters and calculating correction parameters, and ensure the stability of visual stimulation presentation, so as to improve the reliability of the evaluation result. Based on the pre-trained adjustment model, the visual stimulation parameters are adjusted in real time, the visual characteristics of the user can be accurately matched, the fine assessment of the vision function is realized through iterative optimization, and the problems of strong subjectivity and insufficient quantization in the traditional method are solved. Secondly, the combination of diversified visual stimulation and structured sequence data analysis can comprehensively capture the visual response law of the user, and deeply mine the data value based on the evaluation model, so that the vision assessment result is more objective. Finally, based on the characteristics of the augmented reality device, the user can conveniently perform vision assessment in different scenes without relying on specific detection environment and professional personnel assistance, and the convenience of vision assessment is improved. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 A flowchart of a vision assessment method based on augmented reality provided by an embodiment of the application is shown. Figure 2 A schematic block diagram of a vision assessment system based on augmented reality provided by an embodiment of the application is shown. DETAILED DESCRIPTION

[0026] In the following description, specific details such as specific system structures, techniques, etc. are presented in order to thoroughly understand the embodiments of the application. However, it should be clear to those skilled in the art that the application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits and methods are omitted to avoid unnecessary details that hinder the description of the application.

[0027] In order to make the purpose, technical scheme and advantages of the application clearer, the following will be combined with the accompanying drawings to make a detailed description. Figures 1-2 The specific embodiments are described.

[0028] Reference Figure 1 , Figure 1 A flowchart of a vision assessment method based on augmented reality provided by an embodiment of the application is shown. The method comprises: S101: Acquire environmental parameters of the environment where the augmented reality device is located, and calculate environmental correction parameters based on the environmental parameters.

[0029] In this embodiment, when acquiring the environmental parameters of the environment in which the augmented reality device is located, multi-dimensional data needs to be collected, including illumination parameters, spatial parameters, and background parameters. The illumination parameters include ambient light intensity, ambient color temperature, and illumination uniformity, which can be monitored in real time by the optical fiber sensor array built into the augmented reality device. The spatial parameters include the distance between the device and the user's eyes, the attitude angle (pitch angle, yaw angle, roll angle) of the augmented reality device in three-dimensional space, which can be obtained by the infrared distance measuring module and the inertial measurement unit. The background parameters include background texture complexity and background color distribution. The background texture complexity can be obtained by image edge detection algorithm, and the background color distribution can be obtained by HSV color space histogram analysis. In this embodiment, when calculating the environmental correction parameters based on the above environmental parameters, a multi-factor coupling model can be established. For example, for device attitude deviation, the display coordinates of visual stimuli are corrected to a virtual plane directly in front of the user's line of sight through a coordinate transformation matrix to improve the spatial consistency of visual stimulus presentation. In this embodiment, the environmental correction parameters can also be obtained by weighting calculation of each environmental parameter through a preset environmental influence coefficient matrix. For example, the weight coefficient of the illumination parameter is set to 0.3, the weight coefficient of the spatial parameter is set to 0.5, and the weight coefficient of the background parameter is set to 0.2. After weighting, the comprehensive environmental correction value is obtained, which is used to correct the subsequent generated visual stimulus parameters.

[0030] S102: generating visual stimuli according to the preset visual acuity evaluation task, the environmental correction parameters, and the preset initial visual stimulus parameters, and presenting the visual stimuli to the user.

[0031] In this embodiment, the preset visual acuity evaluation task includes distance visual acuity evaluation, near visual acuity evaluation, astigmatism detection, and color discrimination, etc. The initial visual stimulus parameters are set according to the type of evaluation task. For distance visual acuity evaluation, the initial visual stimulus parameters include the size of the visual target, the contrast of the visual target, the presentation time of the visual target, and the position of the visual target. For near visual acuity evaluation, the initial visual target size is the size corresponding to the standard near visual acuity chart, and the other parameters are consistent with the initial parameters for distance visual acuity evaluation. For astigmatism detection, the initial visual target is a radial visual target, the contrast, and the presentation time. For color discrimination, the initial visual target is a dot of different colors, including red, green, and blue, the dot size, and the presentation time.

[0032] In this embodiment, when generating the initial visual stimuli, the environmental correction parameters and the initial visual stimulus parameters need to be fused and calculated. For example, in a high color temperature environment, the wavelength of the red visual target is compensated to maintain the consistency of the perceived color of the visual target by adjusting the RGB channel ratio. When the distance between the device and the user deviates from the standard observation distance, the size of the visual target is scaled in proportion, and the calculation formula is: actual display size = standard size × (measured distance / standard distance), to ensure the optical equivalence of the visual stimuli.

[0033] In another embodiment, when generating the visual stimulus, the initial visual stimulus parameter is fused with the environmental correction parameter to offset the influence of the environment on the presentation effect of the visual stimulus, for example, the size of the target = initial target size x (1 + environmental correction parameter x 0.1).

[0034] S103: Obtain the response feedback of the user to the current visual stimulus, and perform feature extraction on the response feedback to obtain a feature vector representing the response characteristics of the user.

[0035] In this embodiment, the user response feedback includes active input data and passive monitoring data. The active input data includes target identification results input through the interaction module of the augmented reality device, and the user points out the opening direction of the target and the color name, etc. through voice input, gesture click or eye tracking. The passive monitoring data is collected by the sensors of the augmented reality device, including timestamp sequence data of the user from the presentation of the visual stimulus to the completion of the response operation and interaction behavior data in the operation process of the user; wherein the timestamp sequence data is the time interval from the presentation of the stimulus to the input of the feedback; the interaction behavior data includes the eye movement trajectory of the pupil center coordinate sequence recorded by the infrared eye tracking module, and the micro-expression changes of the face analyzed by the face key point detection algorithm through the front camera.

[0036] In another embodiment, the feature extraction process adopts a multi-modal fusion strategy to extract the judgment accuracy (number of correct responses / total number) and the judgment deviation degree of consecutive same difficulty stimuli from the active input data; to extract the moving speed of the eyeball in the target area, the pupil change rate (ratio of pupil diameter before and after the presentation of the visual stimulus) and the response delay fluctuation rate, i.e. the standard deviation of the duration of consecutive responses, from the passive monitoring data; and to normalize the above features.

[0037] S104: Input the feature vector into a pre-trained adjustment model to obtain a visual stimulus parameter adjustment amount, and adjust the visual stimulus parameter based on the visual stimulus parameter adjustment amount to obtain updated visual stimulus parameters.

[0038] In this embodiment, the adjustment model is a neural network model based on deep learning, and the training data thereof is the vision assessment process data of historical users, including the feature vector and the corresponding visual stimulus parameter adjustment amount. The adjustment amount includes adjustment values corresponding to the target size, target contrast, target display duration and target position offset. In the training process of the adjustment model, the dual objective reward function is to minimize the evaluation error and maximize the convergence speed. The parameter adjustment follows the dynamic boundary rule, i.e. the target size adjustment step changes adaptively with the evaluation process; the contrast adjustment range is limited to 10%-90% to avoid invalid stimulation caused by extreme values; and the display duration is not less than 0.5 seconds at the shortest.

[0039] S105: generating updated visual stimuli according to the preset visual assessment task, the updated visual stimulus parameters and the environmental correction parameters and presenting to the user.

[0040] In this embodiment, the updated visual stimuli are generated in the same way as in step S2, and the updated visual stimulus parameters and the environmental correction parameters are fused and calculated, so that the user can clearly perceive the visual stimuli in the current environment, and a smooth transition is adopted when presenting to avoid discomfort caused by sudden changes in visual stimuli.

[0041] In this embodiment, the environmental correction parameters can also be re-called for real-time adaptation when generating updated visual stimuli, so that the generated visual stimuli are more accurate. For example, when the environmental light intensity changes during the iteration process, the luminance compensation coefficient is recalculated; if the user's head moves significantly, the spatial coordinates are real-time corrected.

[0042] S106: iteratively performing steps S103-S105 until a preset evaluation termination condition is met.

[0043] In this embodiment, the evaluation termination condition includes at least one of reaching a preset number of stimulus presentations, reaching a preset evaluation time, reaching a threshold value of the evaluation result confidence based on the user's response, the user's response reaching a preset stability standard, or the user actively terminating and the target size being adjusted to a preset minimum threshold or a maximum threshold.

[0044] S107: inputting the structured evaluation sequence data into a pre-trained evaluation model to generate the user's visual assessment result; the structured evaluation sequence data includes the visual stimulus parameters presented in each iteration, the corresponding user's response feedback and the corresponding feature vector.

[0045] In this embodiment, the structured evaluation sequence data is stored in a time series database format, and each record includes an iteration number, a visual stimulus parameter set, user response raw data, a feature vector, an environmental parameter snapshot and a timestamp; the structured evaluation sequence data described above needs to be cleaned of outliers and aligned in time sequence, i.e., the sequence is reconstructed according to the order of visual stimulus presentation.

[0046] The evaluation model of this embodiment is an ensemble learning model constructed by integrating multiple decision tree models, and the training data includes the user's structured evaluation sequence data and the corresponding professional refraction results. The generated visual assessment result includes far vision value, near vision value, astigmatism degree and axis, color vision abnormality type and visual fatigue index, and the confidence of the evaluation result is also output. When the confidence is lower than the preset confidence threshold, it is prompted that professional refraction review is needed. In addition, the visual assessment result is also accompanied by a brief analysis of the evaluation process, such as the user's response accuracy being higher under high-contrast targets and the accuracy being significantly lower under low-contrast targets.

[0047] From the above, it can be concluded that by acquiring environmental parameters and calculating correction parameters, the present application can eliminate environmental disturbances such as illumination, contrast, etc., ensure the stability of visual stimulation presentation, and improve the reliability of the evaluation results. Based on the pre-trained adjustment model, the visual stimulation parameters are adjusted in real time, which can accurately match the visual characteristics of the user, and through iterative optimization, the fine evaluation of visual function is realized, solving the problems of strong subjectivity and insufficient quantification in traditional methods. Secondly, the combination of diversified visual stimulation and structured sequence data analysis not only comprehensively captures the user's visual response rules, but also deeply excavates the data value based on the evaluation model, making the visual evaluation results more objective. Finally, based on the characteristics of the augmented reality device, the user can conveniently perform visual evaluation in different scenarios without relying on specific detection environments and professional assistance, improving the convenience of visual evaluation.

[0048] In an embodiment of the present application, the environmental parameters include environmental optical parameters and spatial parameters. Generating environmental correction parameters based on the environmental parameters includes: Processing the environmental optical parameters to obtain a brightness compensation coefficient and a color compensation coefficient; Calculating a projection transformation matrix from the display plane of the augmented reality device to the ideal plane of the user's visual axis based on the spatial parameters; The brightness compensation coefficient, the color compensation coefficient, and the projection transformation matrix are used as the environmental correction parameters.

[0049] In this embodiment, the environmental optical parameters specifically include environmental illumination intensity, environmental color temperature, environmental light spectrum distribution, and environmental light uniformity; and the spatial parameters specifically include the straight-line distance of the midpoint of the line connecting the device and the pupil centers of the user's eyes, the horizontal offset, the vertical offset of the intersection of the center of the device display plane and the visual axis of the user's eyes, the tilt angle of the device display plane, and the angular velocity of the device motion. The brightness compensation coefficient calculation in this embodiment uses a piecewise function model. When the actual environmental illumination intensity is less than or equal to the preset reference environmental light intensity, the brightness compensation coefficient is the ratio of the preset reference environmental light intensity to the actual illumination intensity. When the actual illumination intensity is greater than the preset reference environmental light intensity, the brightness compensation coefficient is 1 + (the difference between the actual illumination intensity and the preset reference environmental light intensity) x the first weight coefficient, wherein the first weight coefficient is set according to experience. At the same time, an environmental light uniformity correction term is added. If the uniformity standard deviation is greater than the first uniformity threshold, a decay coefficient of 0.95 is multiplied by the basic compensation coefficient to avoid local strong light interference.

[0050] The color compensation coefficient in this embodiment takes the color space coordinates of the standard color temperature as the reference, converts the spectral distribution data under the actual environmental color temperature into a color deviation matrix, and finally uses the least squares method to solve the color difference deviation matrix to obtain the color temperature compensation coefficient. The ideal plane of the user's visual axis in this embodiment refers to a virtual plane perpendicular to the user's visual axis and at a standard observation distance, which is the ideal presentation plane of the visual stimulus in visual acuity assessment. In this embodiment, the plane where the intersection point of the user's visual axis is the ideal plane of the user's visual axis; a three-dimensional coordinate system is established in the ideal plane of the user's visual axis, wherein the origin is the intersection point of the visual axis, the X-axis is horizontal to the right, the Y-axis is vertical upward, and the Z-axis is along the direction of the visual axis. A projection transformation matrix is constructed according to the device space parameters, wherein the projection transformation matrix is the product of a rotation matrix, a translation matrix and a scaling matrix; wherein the rotation matrix is based on the horizontal offset, the vertical offset and the inclination angle to calculate the Euler angle conversion matrix; the translation matrix is calculated according to the distance between the device and the eye along the Z-axis; the scaling matrix is calculated according to the matching relationship between the device display resolution and the ideal field of view angle. In this embodiment, the device motion angular velocity correction is added in the projection transformation matrix calculation process, when the angular velocity is greater than the first angular velocity threshold, the dynamic smoothing algorithm is used to time-weighted average the parameters of the projection transformation matrix. Wherein, the first angular velocity threshold is set according to experience. In this embodiment, dynamic correction is completed before each visual stimulus rendering, ensuring that the visual stimulus perceived by the user is always optically and geometrically consistent with the standard optometry environment regardless of changes in environmental light or device position.

[0051] In an embodiment of the present application, the environmental optical parameters include: environmental light intensity and environmental color temperature; The environmental optical parameters are processed to obtain a brightness compensation coefficient and a color compensation coefficient, comprising: The brightness compensation coefficient is calculated based on the ratio of the environmental light intensity to the preset reference environmental light intensity; The color compensation coefficient is calculated based on the difference between the environmental color temperature and the preset reference color temperature.

[0052] In this embodiment, the environmental optical parameters are explicitly defined as two core indicators, environmental light intensity and environmental color temperature, which together constitute the basic data for visual stimulus optical correction. The environmental light intensity refers to the radiation intensity of the light in the environment where the device is located, which is collected by the high-precision silicon photocell sensor integrated in the front end of the augmented reality device. The environmental light intensity represents the brightness of the environmental light and is a key factor affecting the brightness perception of the visual stimulus. The environmental color temperature represents the color characteristics of the environmental light, which is detected by the dual-channel color temperature sensor built-in the device, and is expressed in units of Kelvin. Its essence is to convert the energy ratio of red light to blue light in the environmental light, for example, when the color temperature is high, the light appears with a blue tone; when the color temperature is low, it tends to be orange-red.

[0053] The preset reference ambient light intensity in the embodiment is set according to an ophthalmic clinical optometry standard environment; the preset reference ambient light intensity is verified through multi-center clinical trials, and under the illumination condition, the brightness perception error of the human eye to visual stimulation is minimum, and excessive contraction or expansion of the pupil caused by excessively strong or weak light can be effectively avoided.

[0054] In the embodiment, the brightness compensation coefficient is calculated based on the ratio of the ambient light intensity to the preset reference ambient light intensity, specifically, the brightness compensation coefficient is the ratio of the ambient light intensity to the preset reference ambient light intensity; when the ratio is less than a first ratio threshold, the brightness compensation coefficient monotonically increases with the decrease of the ratio, the ambient light deficiency is compensated by enhancing the brightness of the visual stimulation, and the compensation amplitude linearly increases with the darkening of the light; when the ratio is greater than or equal to the first ratio threshold and less than a second ratio threshold, the brightness compensation coefficient is linearly adjusted with the change of the ratio, and the brightness stability is maintained; when the ratio is greater than or equal to the second ratio threshold, the brightness compensation coefficient monotonically decreases with the increase of the ratio, and the strong light interference is avoided by reducing the brightness of the visual stimulation.

[0055] In the embodiment, the preset reference color temperature is set according to the standard white light parameters recommended by the International Commission on Illumination, and the color restoration of the visual stimulation under the preset reference color temperature is closest to the human eye perception under the natural light condition.

[0056] The brightness compensation coefficient and the color compensation coefficient obtained by the above calculation method in the embodiment can accurately offset the interference of the ambient light intensity and the color temperature change on the visual stimulation, so as to ensure that the visual stimulation presented by the augmented reality device always maintains the brightness and color characteristics consistent with the standard optometry environment under different optical environments, and provides a basic guarantee for the accuracy of vision assessment.

[0057] In an embodiment of the present application, the color compensation coefficient is calculated based on the difference between the ambient color temperature and the preset reference color temperature, including: determining a corresponding basic color compensation factor calculation method based on the preset interval in which the ambient color temperature is located; calculating the basic color compensation factor according to the basic color compensation factor calculation method and the difference between the ambient color temperature and the preset reference color temperature; correcting the basic color compensation factor based on the ambient light intensity to obtain the color compensation coefficient.

[0058] In the embodiment, the preset interval includes: a first interval, a second interval, a third interval, a fourth interval and a fifth interval; When the ambient color temperature is in the first interval, the basic color compensation factor is calculated using the first basic color compensation factor calculation method; When the ambient color temperature is in the second interval, the basic color compensation factor is calculated using the second basic color compensation factor calculation method; When the ambient color temperature is in the third interval, the base color compensation factor is calculated using a third base color compensation factor calculation method; When the ambient color temperature is in the fourth interval, the base color compensation factor is calculated using a fourth base color compensation factor calculation method; When the ambient color temperature is in the fifth interval, the base color compensation factor is calculated using a fifth base color compensation factor calculation method; In this embodiment, the base color compensation factor is modified based on the ambient light intensity to obtain a color compensation coefficient, including: The base color compensation factor is multiplied by the ambient light intensity normalization value to obtain the color compensation coefficient. The ambient light intensity normalization value is the ratio of the ambient light intensity value to the preset reference ambient light intensity value.

[0059] In this embodiment, the light in the first interval is warm yellow, with a significantly high proportion of red light. The corresponding calculation method uses an exponential compensation model to calculate the base color compensation factor, which neutralizes the excessive warm tone by strengthening the blue channel compensation, and the compensation strength increases exponentially as the color temperature decreases. The light in the second interval is warm white, with a balanced proportion of red and green light. The base color compensation factor is calculated using a linear compensation model to balance the three primary color channels in a gentle linear adjustment manner, avoiding color distortion caused by excessive compensation. The light in the third interval is close to the preset reference color temperature, and the light is natural white. The base color compensation factor is calculated using a fine-tuning compensation model, which only makes slight adjustments to the three primary color channels to maintain the natural restoration of color. The light in the fourth interval is blueish, with a gradually increasing proportion of blue light. The base color compensation factor is calculated using a segmented linear compensation model. The light in the fifth interval is obviously blue-violet, with a significantly excessive proportion of blue light. The base color compensation factor is calculated using a saturation compensation model.

[0060] In this embodiment, the calculation of the base color compensation factor takes the difference between the ambient color temperature and the preset reference color temperature as the core variable and the corresponding calculation method of each interval, and calculates the red, green and blue channels respectively. For example, in the first interval: as the color temperature decreases, the red channel compensation factor decays exponentially, with a maximum decay amplitude of 30%; the green channel adopts a slow exponential decay, with a decay amplitude controlled within 15%; and the blue channel enhances the blue channel in a faster exponential growth manner to neutralize the warm yellow tone, with a maximum increase of 40%. In the second interval: the red channel has a linear decay rate of 10% per 1000K. The green channel has a linear decay rate of 5% per 1000K. The blue channel has a linear growth rate of 15% per 1000K. In the third interval, the red channel fine-tuning amplitude is controlled within ±2%. The green channel remains unchanged. The blue channel is fine-tuned in the opposite direction of the red channel. In the fourth interval, the red channel linearly increases to enhance the red channel compensation. The green channel linearly decays at a constant speed, with a maximum decay of 10%. The blue channel decays synchronously with the red channel. In the fifth interval, the red channel reaches the upper limit of compensation and no longer changes with the increase of the difference value. The green channel slowly decays to 0.85 and remains stable. The blue channel reaches the upper limit of decay.

[0061] In this embodiment, the ambient light intensity affects the sensitivity of the human eye to color perception (such as reduced color saturation perception in strong light), so the base color compensation factor needs to be modified by multiplying it by a light intensity influence coefficient to obtain the final color compensation coefficient. Through this modification method, the color compensation takes into account both the color temperature difference and the influence of ambient light intensity on human color perception, further improving the accuracy of visual stimulus color presentation.

[0062] Through the above multi-step calculation and modification, the color compensation coefficient can simultaneously offset the effects of ambient color temperature and light intensity changes on visual stimulation, ensuring that the colors presented by the augmented reality device remain consistent with the standard optometric environment in complex light environments, providing a stable color reference for vision assessment.

[0063] In an embodiment of the present application, a projection transformation matrix from the display plane of the augmented reality device to the ideal plane of the user's visual axis is calculated based on spatial parameters, including: According to the relative position and pose of the augmented reality device and the user's eyes, a rigid body transformation model from the device display coordinate system to the ideal coordinate system of the user's visual axis is established; The parameters of the rigid body transformation model are solved by the least squares method to generate the projection transformation matrix.

[0064] In the embodiment, the device display coordinate system is established with the physical center of the augmented reality device display plane as the origin, and a three-dimensional rectangular coordinate system is established, wherein the X axis is along the display plane horizontally to the right, the unit is millimeter, and the positive direction is consistent with the device width direction; the Y axis is along the display plane vertically upward, the unit is millimeter, and the positive direction is consistent with the device height direction; and the Z axis is perpendicular to the display plane and outward, and the right-hand screw rule is followed, and the unit is millimeter. The parameters of the coordinate system are pre-solidified by a hardware calibration module built in the device, including the physical size and pixel density of the display plane, and are used to convert virtual pixel coordinates into physical coordinates. In the embodiment, the user visual axis ideal coordinate system is a virtual three-dimensional coordinate system constructed for vision assessment, and the core role thereof is to provide a reference space for the presentation of visual stimuli, eliminate spatial deviations caused by device wearing position, user head posture and other variables, and ensure the consistency of vision assessment conditions. The origin of the user visual axis ideal coordinate system is set as the projection point of the intersection point of the user's two eye visual axes on the user visual axis ideal plane. The user visual axis ideal plane is a virtual reference plane, which is set as a plane perpendicular to the user's two eye visual axes and at a standard observation distance, and the distance meets the standard observation requirement of the traditional visual acuity chart. The intersection point of the two eye visual axes refers to the intersection point of the user's two eye visual lines (visual axes) in the ideal observation state. When the user gazes at the target on the visual axis ideal plane, the two eye visual axes will naturally intersect at a certain point on the plane, and the origin is based on the projection of the intersection point on the plane. The user visual axis ideal coordinate system of the embodiment is established with the projection of the intersection point of the user's two eye visual axes on the user visual axis ideal plane as the origin, and a three-dimensional rectangular coordinate system is established, wherein the X axis is horizontally to the right and perpendicular to the perpendicular line of the line connecting the user's two eyes; the Y axis is vertically upward and perpendicular to the user's visual line horizontal plane; and the Z axis is outward along the intersection direction of the user's visual axes and points to the equivalent visual distance at a preset distance, wherein the preset distance is set according to the experiment. The user visual axis ideal plane is set in accordance with the principle of comfortable observation of the human eye, and the average distance thereof from the user's two eyes is collected in real time by an infrared distance measuring sensor.

[0065] In the embodiment, the spatial parameters for calculating the relative position and attitude include: the relative position parameter is the three-dimensional coordinate of the origin of the device display coordinate system in the user visual axis ideal coordinate system, which is obtained by a multi-view visual positioning algorithm and infrared marker point tracking. The attitude angle parameter is the rotation angle of the device around its X, Y and Z axes, i.e. roll angle, pitch angle and yaw angle, which is collected by an inertial measurement unit built in the device. The two eye visual axis parameters are the left eye visual axis direction vector and the right eye visual axis direction vector, which are calculated by the corneal reflection imaging technology of the eye tracking module.

[0066] The rigid body transformation model of the embodiment is used to describe the geometric mapping relationship from the device display coordinate system to the ideal coordinate system of the user's visual axis. The core is that in the visual assessment process, the relative motion between the device and the user's eye can be regarded as rigid body motion, and the transformation process only includes translation and rotation.

[0067] The projection transformation matrix generated by the construction and parameter optimization of the rigid body transformation model can accurately describe the spatial mapping relationship from the device display plane to the ideal plane of the user's visual axis, effectively eliminate the geometric distortion of the visual stimulus caused by the deviation of the device wearing position and the micro-motion of the user's head, and ensure that the presented visual stimulus maintains the geometric characteristics consistent with physiological perception in the user's field of view, thereby providing spatial reference protection for the accuracy of visual assessment.

[0068] In an embodiment of the present application, the response feedback includes: recognition result data input by the user through the interactive interface of the augmented reality device, timestamp sequence data of the user from the visual stimulus presentation to the completion of the response operation, and interactive behavior data in the user operation process; The response feedback is subjected to feature extraction to obtain a feature vector representing the response characteristics of the user, including: The response correctness index is calculated based on the recognition result data, the response time index is calculated based on the timestamp sequence data, the interactive behavior data is extracted to obtain the user operation features, and the response correctness index, the response time index and the user operation are fused to obtain the feature vector representing the response characteristics of the user.

[0069] In the embodiment, the recognition result data is the direct judgment input of the user on the visual stimulus, such as the selection of the opening direction of the visual target (up, down, left, right), the recognition result of the color (red, green, blue, etc.), the judgment of the shape of the figure (circle, square, triangle, etc.); the timestamp sequence data includes the timestamp of the start of the visual stimulus presentation, the timestamp of the start of the response operation of the user, and the timestamp of the completion of the response operation of the user, all in milliseconds; the interactive behavior data covers the way of the user operating the interactive interface, the number of corrections in the operation process, the operation force parameter and the operation trajectory coordinate sequence.

[0070] In the embodiment, the response feedback is subjected to feature extraction to obtain a feature vector representing the response characteristics of the user, including: The response correctness index is calculated based on the recognition result data. Specifically, the recognition result data input by the user is compared with the standard result of the visual stimulus. If they are completely consistent, a value of 1 is assigned, otherwise a value of 0 is assigned, to obtain the correctness identification of the current response. For consecutive responses in the multi-round evaluation, the cumulative accuracy rate is calculated, and the formula is: cumulative accuracy rate = cumulative number of correct responses / total number of responses x 100%. The embodiment also adds a consecutive correctness feature, that is, the maximum number of consecutive correct responses, for example, 3 consecutive correct responses are recorded as 3; the correctness identification, the cumulative accuracy rate and the consecutive correctness feature are used as the components of the response correctness index. The response time index is calculated based on the timestamp sequence data. Specifically, the total response time is the difference between the timestamp of the user completing the response operation and the timestamp of the visual stimulus starting to present, that is, the total time from the presentation of the visual stimulus to the completion of the user's response operation; the operation delay time is the difference between the timestamp of the user starting to perform the response operation and the timestamp of the visual stimulus starting to present, that is, the interval time from the presentation of the visual stimulus to the start of the user's operation; the operation execution time is the difference between the timestamp of the user completing the response operation and the timestamp of the user starting to perform the response operation, that is, the time from the start of the operation to the completion of the operation. The mean and standard deviation of the above time indexes in the multi-round response are calculated to obtain the response time index. The interaction behavior data is extracted to obtain the user operation features. For the operation mode, a one-hot encoding is used for representation, for example, the operation trajectory coordinate sequence is represented by calculating the trajectory length and the trajectory smoothness.

[0071] In this embodiment, the response correctness index, the response time index and the user operation features are fused to obtain the feature vector representing the user response characteristics. The fusion method can use feature splicing, and the feature vector with fixed dimensions is arranged in the order of the response correctness index, the response time index and the user operation features.

[0072] The feature vector of this embodiment includes the correctness, time efficiency and operation behavior characteristics of the user in the response process, which can accurately represent the response characteristics of the user, and provides a reliable basis for the subsequent adjustment of the visual stimulus parameters and the generation of the visual assessment results.

[0073] The embodiment can also use a weighted fusion mechanism to dynamically adjust the feature weights for fusion according to different evaluation tasks. For example, the basic visual assessment increases the weights of the response correctness index and the eye movement feature to focus on the accuracy of visual identification. The dynamic visual assessment increases the weights of the response time index and the operation correction feature to emphasize time sensitivity. The fatigue state assessment increases the weights of the pupil change rate and the time fluctuation rate to capture the behavior changes caused by fatigue.

[0074] The weighted fused feature vector of the embodiment is attached with a feature importance label for each dimension, which is used for subsequent adjustment of feature selection and weight distribution of the model, ensuring that the model can accurately capture the response characteristics of the user and provide a reliable basis for dynamic adjustment of visual stimulation parameters.

[0075] In an embodiment of the present application, when the interaction behavior data is head posture change data, the interaction behavior data is extracted to obtain user operation features, including: Based on the head posture change data, the offset angle change trajectory of the head relative to the preset reference posture during the user response is calculated. The features of the offset angle change trajectory are extracted to obtain head movement features as user operation features; wherein the head movement features include the average change rate of the offset angle and the smoothness index of the offset angle change trajectory.

[0076] In the embodiment, the preset reference posture is a standard head posture maintained by the user in the initial stage of visual assessment. The initial Euler angles, i.e. the pitch angle, yaw angle and roll angle, in this posture are collected by the head posture sensor built-in the augmented reality device as the reference parameters. During the user response, the head posture sensor collects head posture data in real time to obtain the Euler angles at each sampling time. The offset angles in three angle directions relative to the reference posture, i.e. the pitch offset angle, the yaw offset angle and the roll offset angle, are calculated respectively; the change sequences of the offset angles in three directions with time are integrated to obtain the offset angle change trajectory.

[0077] The offset angle change trajectory of the embodiment can reflect the concentration and cooperation of the user during the visual assessment. During the visual assessment, the user needs to maintain a relatively stable head posture to accurately observe the visual stimulus. If the offset angle change trajectory fluctuates greatly, it means that the user is not concentrated or cannot cooperate well with the assessment, which will affect the accuracy of the perception and response to the visual stimulus. The embodiment can assist in judging the reliability of the response feedback. Second, it provides a reference for analyzing the user's visual accommodation ability. Changes in head posture are often related to the user's attempt to observe the visual stimulus more clearly, such as when the visual stimulus is too blurry, the user may adjust the viewing angle by turning the head. The offset angle change trajectory can reflect the user's adjustment behavior when facing different visual stimuli, and in combination with the visual stimulus parameters, the user's visual condition can be more comprehensively understood. Third, it helps to optimize the adjustment of visual stimulus parameters. The characteristics of the offset angle change trajectory will be input into the adjustment model as user operation characteristics. The model can judge whether the current visual stimulus is suitable for the user based on these characteristics. For example, low trajectory smoothness and fast average change rate may indicate that the user is difficult to adapt to the current stimulus, and the model can adjust the parameters more accurately to make subsequent visual stimuli more suitable for the user's needs. Finally, it provides a check basis for the accuracy of the visual assessment results. If there are abnormal patterns in the offset angle change trajectory of the head posture (such as violent and irregular changes), the assessment model can consider them as influencing factors when generating the visual assessment results, to avoid biases in the assessment results caused by unstable user operations, and to improve the scientificity and reliability of the assessment.

[0078] The average change rate of the offset angle of the embodiment is calculated as follows: first, calculate the angle change between each adjacent sampling time; second, calculate the instantaneous change rate in each direction according to the sampling time interval, then take the average of the instantaneous rate during the response period to obtain the average change rate in three directions; finally, integrate the average change rates in three directions into a vector form, which is the average change rate of the offset angle.

[0079] The smoothness index of the offset angle change trajectory of the embodiment is achieved by calculating the curvature change rate of the trajectory: for a three-dimensional offset angle change trajectory, first calculate the curvature value of each sampling point. Taking the pitch and yaw plane as an example, the curvature of a certain sampling point can be calculated through the vector relationship of the point and its adjacent two points. The curvature sequence is calculated by traversing the entire trajectory, and the smoothness index is defined as the reciprocal of the standard deviation of the curvature sequence. The greater the smoothness index value, the more stable the head movement trajectory; otherwise, it means that the trajectory has more mutations or jitter.

[0080] The embodiment integrates the average change rate vector of the offset angle with the smoothness index to obtain a head movement feature vector. The feature vector can quantitatively represent the speed and stability of the head movement of the user in the response process. For example, when the user frequently turns the head due to the blur of the visual target, the average change rate increases and the smoothness index decreases.

[0081] The head movement feature obtained by the extraction process, the response correctness index and the response time index jointly constitute a complete feature vector, which further improves the accuracy of the description of the response characteristics of the user and makes the vision assessment method based on augmented reality more consistent with the behavior patterns of the user in the actual interactive scene.

[0082] In an embodiment of the present application, the smoothness index in the feature of the offset angle change trajectory is extracted, including: Based on the angle values of the continuous sampling points in the offset angle change trajectory, the angle change amount between adjacent sampling points is calculated. According to the angle change amounts of all adjacent sampling points in the preset time window, the standard deviation of the angle change amount is calculated. The standard deviation is taken as the smoothness index of the offset angle change trajectory.

[0083] In the embodiment, the offset angle change trajectory includes the change sequences of the offset angles in the pitch, yaw and roll directions over time. For each direction, the angle change amount between adjacent sampling points, i.e., the pitch direction adjacent angle change amount, the yaw direction adjacent angle change amount and the roll direction adjacent angle change amount, is calculated. According to the angle change amounts of all adjacent sampling points in the preset time window, the standard deviation of the angle change amount is calculated. Specifically, the length of the preset time window can be set according to actual evaluation requirements. Taking the pitch direction as an example, the pitch angle change amount sequence is cut according to the time window to obtain multiple subsequences, and the standard deviation of each subsequence is calculated. Similarly, the standard deviations of the angle change amounts of the yaw direction and the roll direction in each time window are calculated. If the total length of the response period is short, less than a preset time window, the standard deviation of all angle change amounts in the direction is directly calculated. The standard deviation is taken as the smoothness index of the offset angle change trajectory. Specifically, for the three directions included in the offset angle change trajectory, the mean of the standard deviations in all time windows is taken as the smoothness index of the corresponding direction, and the smoothness indices of the three directions are integrated into a comprehensive smoothness index by using a weighted average method, wherein the weights are set according to the influence of the head movement in each direction on the response to the visual stimulus. The pitch direction is given a higher weight because it is more closely related to the up-down adjustment of the line of sight. The greater the value of the smoothness index in this embodiment, the more unstable the change in the angle between adjacent sampling points, and the poorer the smoothness of the head movement trajectory; on the contrary, the smaller the value, the more stable the head movement. For example, when the user can clearly identify the target, the head is basically stable, the adjacent angle change is small and the fluctuation is small, and the smoothness index value is small; when the target is blurred, the user frequently shakes the head to adjust the line of sight, the angle change fluctuation increases, and the smoothness index value increases. The combination of the smoothness index and the average change rate of the offset angle in this embodiment can more comprehensively reflect the operation characteristics of the user's head movement, provide an objective basis for analyzing the user's operation characteristics and optimizing the vision assessment process, and further improve the accuracy and adaptability of the vision assessment method based on augmented reality.

[0084] In an embodiment of the present application, when the interaction behavior data is eye tracking data, the interaction behavior data is extracted to obtain the user operation characteristics, further comprising: Based on the eye tracking data, the time length from the moment when the visual stimulus is presented to the moment when the user first meets the preset gaze condition in the preset interest region is calculated as the gaze latency; the duration of the first gaze meeting the gaze condition in the preset interest region is calculated; and the gaze latency and the duration of the first gaze are taken as the user characteristics.

[0085] In this embodiment, the preset interest region is the presentation region of the visual stimulus in the augmented reality device display interface, and a square region with a side length of 3 times the size of the target is drawn around the target (if the target is circular, a circular region with a radius of 3 times the size of the target is drawn). The coordinates of the region are pre-calibrated by the device display coordinate system. The eye tracking data is collected by the eye movement sensor built in the device, including the real-time coordinates of the pupil centers of the user's eyes on the display interface. The preset gaze condition is that the pupil center coordinates of the continuous 5 sampling points all fall within the preset interest region, and the coordinate offset of adjacent sampling points is less than the preset offset threshold, i.e. 5 pixels. In this embodiment, the timing starts from the moment T0 when the visual stimulus is presented, and when the first continuous sampling point meeting the above gaze condition appears, the time stamp T1 of the first sampling point in the continuous sequence is recorded, and the calculation formula of the gaze latency is: gaze latency = T1-T0; for example, the visual stimulus 10:00:00.000 is presented, and the first continuous sampling point meeting the gaze condition in the eye movement data starts at 10:00:00.235, then the gaze latency is 235ms. In this embodiment, after determining the sequence of consecutive sampling points that first meet the fixation condition, the eye-tracking data is traversed backward until a sampling point that does not meet the fixation condition appears, i.e., the pupil center moves out of the region of interest or the coordinate offset is greater than a preset offset threshold. The timestamp T2 of the last sampling point in the sequence is recorded. The formula for calculating the duration of the first fixation is: Duration of the first fixation = T2 - T1. Following the above example, if the sequence that first meets the fixation condition terminates at 10:00:00.510, then the duration of the first fixation is 275ms.

[0086] In this embodiment, fixation latency and the duration of the first fixation are used as user features to represent the user's speed of attention capture and initial level of focus on visual stimuli from a temporal perspective. For example, when the target is clearly identifiable, the user can quickly focus their gaze on the area of ​​interest, resulting in a shorter fixation latency and a relatively shorter duration of the first fixation due to the smooth recognition process. When the target is blurry, the user may need more time to search for the target, i.e., the fixation latency is prolonged, and the fixation time is extended to attempt recognition, i.e., the duration of the first fixation is increased. These two features, combined with other operational features such as head movement characteristics, can further enrich the representational dimensions of user response characteristics and improve the accuracy of the adjustment model in adjusting visual stimulus parameters.

[0087] See Figure 2 , Figure 2 This is a schematic block diagram of an augmented reality-based vision assessment system provided in one embodiment of this application. Figure 2 The electronic device 300 in this embodiment may include one or more processors 301, one or more input devices 302, one or more output devices 303, and one or more memories 304. The processors 301, input devices 302, output devices 303, and memories 304 communicate with each other via a communication bus 305. The memories 304 store computer programs, including program instructions. The processors 301 execute the program instructions stored in the memories 304. Specifically, the processors 301 are configured to invoke the program instructions to execute the implementation described in any of the embodiments of the augmented reality-based vision assessment method.

[0088] It should be understood that, in the embodiments of the present application, the processor 301 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or can also be any conventional processor.

[0089] The input device 302 can include a touchpad, a fingerprint collection sensor (for collecting fingerprint information and direction information of a fingerprint of a user), a microphone, etc., and the output device 303 can include a display (LCD, etc.), a speaker, etc.

[0090] The memory 304 can include read-only memory and random access memory, and provide instructions and data to the processor 301. A part of the memory 304 can also include non-volatile random access memory. For example, the memory 304 can also store device type information.

[0091] In specific implementations, the processor 301, the input device 302 and the output device 303 described in the embodiments of the present application can perform the implementation manners described in any embodiment of the vision assessment method based on augmented reality provided by the embodiments of the present application, and details are not repeated here.

[0092] In another embodiment of the present application, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. The computer program includes program instructions, and the program instructions are executed by a processor to implement all or part of the processes of the above-mentioned embodiment methods. The computer program can also be used to instruct related hardware to complete, and the computer program can be stored in a computer readable storage medium. When the computer program is executed by the processor, the steps of each method embodiment can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable file or some intermediate form, etc. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0093] The computer readable storage medium can be an internal storage unit of the electronic device of any of the preceding embodiments, such as a hard disk or a memory of the electronic device. The computer readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, or the like provided on the electronic device. Further, the computer readable storage medium can include both the internal storage unit and the external storage device of the electronic device. The computer readable storage medium is used to store a computer program and other programs and data required by the electronic device. The computer readable storage medium can also be used to temporarily store data that has been output or will be output.

[0094] Those skilled in the art can understand that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in general terms in the foregoing description. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0095] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the electronic device and the units described above can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0096] In several embodiments provided in the present application, it should be understood that the disclosed system and method can be implemented by other ways. For example, the apparatus embodiments described above are merely schematic, for example, the division of units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces or units, and can also be electrical, mechanical or other form of connection.

[0097] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e. can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0098] In addition, each of the functional units in the various embodiments of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0099] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present application, and these modifications or replacements should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for vision assessment based on augmented reality, characterized in that, The method for an augmented reality device, comprising: S101: obtaining an environment parameter of an environment where the augmented reality device is located, and calculating an environment correction parameter based on the environment parameter; S102: generating a visual stimulus according to a preset visual acuity evaluation task, the environment correction parameter, and a preset initial visual stimulus parameter, and presenting the visual stimulus to a user; S103: obtaining a response feedback of the user to the current visual stimulus, performing feature extraction on the response feedback to obtain a feature vector representing a response characteristic of the user; S104: inputting the feature vector into a pre-trained adjustment model to obtain a visual stimulus parameter adjustment amount, adjusting the visual stimulus parameter based on the visual stimulus parameter adjustment amount to obtain an updated visual stimulus parameter; S105: generating an updated visual stimulus according to the preset visual acuity evaluation task, the updated visual stimulus parameter, and the environment correction parameter, and presenting the updated visual stimulus to the user; S106: iteratively performing steps S103-S105 until a preset evaluation termination condition is met; S107: inputting structured evaluation sequence data into a pre-trained evaluation model to generate a visual acuity evaluation result of the user; the structured evaluation sequence data includes the visual stimulus parameter presented in each iteration, the corresponding response feedback of the user, and the corresponding feature vector.

2. The augmented reality-based vision assessment method of claim 1, wherein, The environment parameter includes an environment optical parameter and a spatial parameter; The environment correction parameter is generated based on the environment parameter, comprising: processing the environment optical parameter to obtain a brightness compensation coefficient and a color compensation coefficient; calculating a projection transformation matrix from a display plane of the augmented reality device to an ideal plane of the visual axis of the user based on the spatial parameter; the brightness compensation coefficient, the color compensation coefficient, and the projection transformation matrix are used as the environment correction parameter.

3. The augmented reality-based vision assessment method of claim 2, wherein, The environment optical parameter includes an environment light intensity and an environment color temperature; processing the environment optical parameter to obtain a brightness compensation coefficient and a color compensation coefficient, comprising: calculating the brightness compensation coefficient based on the ratio of the environment light intensity to a preset reference environment light intensity; calculating the color compensation coefficient based on the difference between the environment color temperature and a preset reference color temperature.

4. The augmented reality-based vision assessment method of claim 3, wherein, The color compensation coefficient is calculated based on the difference between the environment color temperature and the preset reference color temperature, comprising: determining a corresponding basic color compensation factor calculation method based on a preset interval of the environment color temperature; calculating a basic color compensation factor according to the basic color compensation factor calculation method and the difference between the environment color temperature and the preset reference color temperature; modifying the basic color compensation factor based on the environment light intensity to obtain the color compensation coefficient.

5. The augmented reality-based vision assessment method of claim 2, wherein, The projection transformation matrix from the display plane of the augmented reality device to the ideal plane of the visual axis of the user is calculated based on the spatial parameter, comprising: establishing a rigid body transformation model from a device display coordinate system to a user visual axis ideal coordinate system according to the relative position and posture of the augmented reality device and the user's eyes; solving the parameters of the rigid body transformation model by least squares method to generate the projection transformation matrix.

6. The augmented reality-based vision assessment method of claim 1, wherein, The response feedback comprises: recognition result data input by a user through an interactive interface of the augmented reality device, timestamp sequence data of a user from a visual stimulus presentation to completion of a response operation, and interactive behavior data in a user operation process; The feature extraction on the response feedback obtains a feature vector representing a user response characteristic, comprising: calculating a response correctness index based on the recognition result data; calculating a response time index based on the timestamp sequence data; extracting the interactive behavior data to obtain a user operation feature; fusing the response correctness index, the response time index, and the user operation to obtain the feature vector representing the user response characteristic.

7. The augmented reality-based vision assessment method of claim 6, wherein, When the interactive behavior data is head posture change data, the extraction of the interactive behavior data to obtain a user operation feature comprises: based on the head posture change data, calculating a change trajectory of an offset angle of a head relative to a preset reference posture during a user response; extracting a feature of the change trajectory of the offset angle to obtain a head movement feature as the user operation feature; wherein, the head movement feature comprises an average change rate of the offset angle and a smoothness index of the change trajectory of the offset angle.

8. The augmented reality-based vision assessment method of claim 7, wherein, The extraction of the smoothness index of the change trajectory of the offset angle comprises: based on angle values of consecutive sampling points in the change trajectory of the offset angle, calculating an angle change amount between adjacent sampling points; according to angle change amounts of all adjacent sampling points within a preset time window, calculating a standard deviation of the angle change amounts; taking the standard deviation as the smoothness index of the change trajectory of the offset angle.

9. The augmented reality-based vision assessment method of claim 6, wherein, When the interactive behavior data is eye movement tracking data, the extraction of the interactive behavior data to obtain a user operation feature further comprises: based on the eye movement tracking data, calculating a duration from a time when a visual stimulus is presented to a time when a preset gaze condition is first met in a preset interest region as a gaze latency; calculating a duration of a first gaze that meets the gaze condition in the preset interest region; taking the gaze latency and the duration of the first gaze as a user feature.

10. An augmented reality based vision assessment system comprising a memory, a processor and a computer program stored in the memory and running on the processor, characterized in that, The processor executes the computer program to implement the steps of the method of any one of claims 1 to 9.

Images